2.6 Bias in psychological assessments.

Bias can be a factor in the construction of psychological assessments. In some cases a few survey items may be poorly worded or inadequately translated introducing error in interpretation by the respondent. However, item error can usually by eliminated by statistical means (Van de Vijver & Leung, 1997b). Item analysis procedures are based on an internal assessment of measurement bias. Eliminating poor or biased statements however, do not provide solutions to the broader problem when the entire method is inadequate. If error affects the entire survey the problems in the methodology require an examination of alternative approaches. However a more frequent bias issue is problems with the concept being measured. When concepts are not well articulated in the methods used there are errors that follow from different interpretation by the respondents. The multiple words for snow in Inuit language makes the conceptual context in that culture different compared to participants living in the tropics. To avoid bias respondents in the cultures examined should have the same probability of responses given similar familiarity (Shepard, Camilli, & Averill, 1981).

Van de Vijver and Poortinga (1997) outlined three types of bias in psychological assessments. The difference in familiarity of concepts and constructs were discussed above and might be caused by a lack of overlap in definitions of the construct measured. Lack of familiarity could also be caused by poor sampling of the relevant construct variables. Another methodological problem is bias introduced by issues affected by social desirability. The pursuit of individual enhancement is prominent in individualistic cultures, whereas group enhancement is more characteristic of collectivistic cultures (Lalwani, Shavitt, & Johnson, 2006). Respondents from individualistic cultures engage in self-deceptive self-enhancement, whereas respondents from collectivistic cultures are more concerned with impression management in society. Both forms of deception bias the results of cultural differences. However, it is possible to statistically control items affected by social desirability. On the other hand it is more likely that response sets play a role when respondents are unfamiliar with the construct measured. Response biases may also vary by culture producing a systematic response tendency to survey items or attitude scales. Consequently it is unclear whether comparative differences are the result of a cultural response bias or if they measure some actual difference in the trait. Bias in interpretation of items can also hamper direct comparisons producing differential meaning of the words measured. Response sets also affects survey results. Some respondents prefer to systematically acquiesce with often conflicting viewpoints or respond with the extreme response categories.

The acquiescence response bias is the general tendency to agree with a statement regardless of the actual content being measured. Other respondents may use primarily extreme ends of a scale independent of content of the statement. Since the responses may be dominated by these systematic tendencies we know less about the response to the content of the variable being measured. Responses to surveys are also affected by people’s tendency to make implicit comparisons to the opinions of the reference groups to which they belong. Responses therefore are less personal and reflect rather the dominant opinions in reference groups. For example Japanese respondents in evaluating themselves often self-rank higher on individualism compared to their Japanese reference group. That inflates their judgment of their actual individualism (Van Herk, Poortinga, & Verhallen, 2004). The major problem with these reference group biases is that participants will respond not with their own assessments, but what they think meet social approval or what they think is the dominant opinion.

Bias may also ensue from the researchers own cultural blinds. Those who organize and carry out cross-cultural studies do so from their own theoretical perspectives confounded by implicit cultural bias. Like all other people researchers see what they need to see and what they are prepared to observe. Preconceptions may influence not only the choice of what to study, but also the methods compatible to the researcher, and the inferences made from the results. Matsumoto and his collaborators noted for example that studies of emotion differences between Japanese and Americans often infer that the Japanese suppress their emotions. However, the results actually show that the difference is not so much Japanese suppression of emotion as the American respondents exaggerating theirs. In other words most researchers are dependent on their own cultural perspective when evaluating the results of research (Matsumoto & Ekman, 1989; Matsomuto, Kasri & Kooken, 1999).