Research approaches and critical thinking in cross-cultural psychology

The findings of cross-cultural psychology are delimited by the reliability and validity of the research methods used. Since cross-cultural psychology is by far the most complex of the psychological disciplines the student and researcher must develop a critical eye when evaluating research data and results. Furthermore cross-cultural research can be conducted at several levels increasing the possibility that the results contain bias in either the design of the research protocol or in the analysis by the researcher. Therefore, the serious student should consider all possible alternative explanations for the reported outcomes. This chapter will outline the means by which we might evaluate results in both qualitative and quantitative research.

We noted in chapter 1 that there are disagreement as to whether comparative studies can produce valid results given the varying confounding influences and the difficulty of selecting representative sampling. The levels of research refer to both individually based studies based on surveys, interviewing, but also ecological population level variables creating complexity in theoretical analysis. Great care must be employed in evaluating the relevance of any conclusions in cross-cultural psychology that has derived from research at different levels of analysis.

In cross-cultural psychology we must constantly ask whether the results we obtain from different cultures are interpreted correctly and meet the criterion of equivalence. Since the researcher is not immune to cultural bias and ethnocentric interpretations how can we guard against conclusions that in fact are not valid?

2.1 Cultural bias and criterion of equivalence.

The most significant issue in cross-cultural research is the equivalence of comparative research concepts and methodology. Are the concepts employed in comparative research and the research methodology used equivalent in each culture? If the conceptual foundation and research methods are not similar it is not possible to draw comparative conclusions. The lack of equivalence will inevitably produce bias in the results and interpretations. Therefore a significant precursor to valid methodology is creating studies that meet the test of equivalence.

2.1.1 The issue of language equivalence.

By its very nature cross-cultural research is often conducted with respondents who speak different languages. The first issue in comparative research therefore is to evaluate the equivalence of translations of the original research protocol. Whether the researcher employs surveys, attitude scales or interview protocols, linguistic equivalence is the first and most important condition of valid research. The development of research instruments should initially utilize multi-lingual participants (Van de Vijver & Hambleton, 1996). However, what happens in reality most frequently is that the research protocol is developed in the investigators culture and then translated into the language of those cultural groups selected for comparison. The first step in creating linguistic equivalence is to ask a committee of bi-lingual experts to collectively evaluate the research instrument. They will be asked to indicate their opinions as to whether the individual items or statements belong to the research domain selected. The experts will be asked to look for items that do not validly reflect cultural experience and whether a participating cultural group has a unique language context that affect the item responses. Language constructs that are culturally specific, and for which there is no equivalence in other cultures, would introduce bias. The committee of bilingual experts works toward a consensus about the appropriateness of the language used in the research protocol.

The second method used since the 1970’s is called back translation (Brislin, 1970; 1993). The researchers have the research document translated into the other languages of interest, after which independent participants translate it back into the original language. If the retranslation is semantically equivalent to the original language protocol the researcher can have confidence in language equivalence. In other words a satisfactory criterion is met when the independent back translation produces more or less the original language used. The back translation procedure requires the participation of experts that are fluent in the languages employed. If the original form is found not to be translatable, the researcher must return to the drawing boards and use a different alternative language in the protocol. Since many participants in cross-cultural research are bi-lingual and often use English as a second language, researchers have to be cognizant of the possibility that responses are influenced by the participant’s stereotypes of the culture of the particular language (Bond, 1983).