2.2 Nonequivalence in cross-cultural research.

Most researchers do what is possible to conduct comparative research with equivalent samples. However, the very nature of cross-cultural research makes that a difficult goal to achieve. Often investigators are left with what they believe are good approximations of construct and methodological equivalence. It is probably not possible given the influence of culture specific factors to ever create perfect methodological equivalence in the selection of relevant constructs and in data collection in all comparative cultures. Poortinga (1989) recognized this dilemma and suggested several approaches for dealing with nonequivalent data. Cross-cultural studies will in reality be interpreted and published even when comparisons are made with nonequivalent data. However, Poortinga suggest that the researcher can take steps to reduce nonequivalence by identifying and analyzing the research elements that are equivalent separately from those that don’t meet criteria. In research employing attitude scales we have statistical methods to identify nonequivalent items and remove these from analysis (Ommundsen & Larsen, 1997; Ommundsen & Larsen, 1999; Ommundsen, Hak, Morch, Larsen & Van der Veer, 2002; Ommundsen, Van der Veer, Van Le, Krumov, & Larsen, 2007).

Another way of looking at the nonequivalence issue is to examine it as important information about the cultures investigated. What are the reasons for the nonequivalence and what does that tell us about psychology in these societies? Cultural differences may be informed by equivalent data, but also by why some data is not equivalent. Many hypotheses for further research can be developed from these methodological issues and problems.

2.3 Levels of inference.

Berry, Poortinga, Segall and Dasen (1992) suggested the presence of three levels of inferences in cross-cultural psychology. An example of low level inferences occur when the researcher constructs direct samples of content from the domain of interest and from such sample tests make inferences about the domain studied. The use of scales measuring attitudes toward various groups like immigrants or intergroup relations involving contacts between national or ethnic groups are examples of low level inferences (Van der Veer, Ommundsen, Larsen, Van Le, Krumov, Pernice, Pastor Romans, 2004). Through scaling procedures the researcher selects a representative sample of attitude statements that reflect the domain of the attitude in question. If the selection of scale items is representative comparisons to the domain of the attitude are considered valid. Here as elsewhere one must take precautions that the statements selected do not contain information that is culturally unique and relevant only to some respondents and not others. Likewise the methods used must also be clearly understood and equivalent cross-culturally.

Medium level inferences according to Berry et al refer to generalization from domains that are not directly observable including cognitive abilities or personality traits that are nevertheless manifested in predictable behaviors. Here the objective is not to obtain representative samples of the domain of study, but rather to capture the essence of the psychological trait. At this medium level of inference it is difficult to assume without ambiguity whether the domains studied are in fact comparable across cultures. Is the authoritarianism observed in Saudi Arabia equivalent to that observed in the United States? There are probably many differences confounded by other variables like religiosity that makes direct comparisons difficult. Some personality traits may be culturally specific and that makes comparative inferences difficult.

High level inferences refer to domains not easily specified, and that can’t be measured by standard measurement procedures. At this level of inference it is doubtful if measurements can capture the domain of interest. Observed differences are explained post hoc after the exploratory research is completed and without solid empirical evidence. Examples cited by Berry et al include the use of high level concepts like intelligence or adaptation. Since adaptability is required in all cultural environments how it is possible to state with any confidence that one culture is better adapted than another? These types of assertions go beyond what the data will support and tend to be unclear and not easily operationalized with valid comparative methodology.