Equivalence of Measurement Instruments for Attitude Variables in Comparative Surveys, Taking Method Effects into Account: The Case of Ethnocentrism

Jaak Billiet, Bart Cambre, and Jerry Welkenhuysen-Gybels

Abstract

This study is focused on the construction of a cross-national comparable measurement instrument for attitude variables in comparative surveys. Multi-group measurement models for latent variables (LISREL), taking method effects into account, are applied. The measurements of the 'out-group' dimension of ethnocentrism (variables q42, q44, q45, q47-q52) in the 1995 ISSP dataset are used. Nearly all the items in the quasi balanced set are written in a Likert format in which respondents are asked how strongly they agree or disagree with each attitude statement. There is considerable evidence that such a response format can be susceptible to an agreeing-response bias called acquiescence (Billiet and McClendon, 2000). It is shown that in all countries, models with a method or style factor (acquiescence) always fit the data better than models without a style factor. It is investigated to what extent the measurement instrument with a content and a method factor is equivalent over the cultural groups. In a first step the factor loadings of the groups are explored by cluster analysis. After the detection of two subsets of groups that are likely to share equivalent measurement instruments, a stepwise procedure was performed starting with the measurement model for one group, and then looking for equivalent groups (countries) only accepting minor changes in the measurement model. The introduction of a style factor allows us to control for a possible source of measurement non-equivalence, namely method bias. Moreover, the inclusion of a method factor gives the opportunity to investigate the differences in method effects between the groups (countries). However, it is found that both, the variance of the style factor and its factor loadings do not differ between the groups in the first subset of countries, but there are differences in the degree of acquiescence in the second one. It seems reasonable to conclude that the agreeing-response bias does not lead to a cross-cultural method bias in the measurement of ethnocentrism in the Western countries of ISSP 1995.