New methods to improve the accuracy of cross-national surveys

Research by Social Statistics has highlighted new methods to improve the accuracy of cross-national surveys. Cross-national surveys run the risk of differential survey errors, where data collected vary in quality from country to country. The study is published in the Journal of Survey Statistics and Methodology.

Responsive and adaptive survey designs (RASDs) have been proposed as a way to reduce survey errors, by leveraging auxiliary variables to inform fieldwork efforts, but have rarely been considered in the context of cross-national surveys.

Using data from the European Social Survey, Dr. Alex Cernat, Dr. Hafsteinn Einarsson and Professor Natalie Shlomo from Social Statistics simulate fieldwork in a repeated cross-national survey using RASD where fieldwork efforts are ended early for selected units in the final stage of data collection.

Demographic variables, paradata (interviewer observations), and contact data are used to inform fieldwork efforts.

Eight combinations of response propensity models and selection mechanisms are evaluated in terms of sample composition (as measured by the coefficient of variation of response propensities), response rates, number of contact attempts saved, and effects on estimates of target variables in the survey.

The researchers find that sample balance can be improved in many country-round combinations. Response rates can be increased marginally and targeting high propensity respondents could lead to significant cost savings associated with making fewer contact attempts.

Estimates of target variables are not changed by the case prioritizations used in the simulations, indicating that they do not impact non-response bias.

They conclude that RASDs should be considered in cross-national surveys, but that more work is needed to identify suitable covariates to inform fieldwork efforts.

Research by Social Statistics has highlighted new methods to improve the accuracy of cross-national surveys. Cross-national surveys run the risk of differential survey errors, where data collected vary in quality from country to country. The study is published in the Journal of Survey Statistics and Methodology.

Responsive and adaptive survey designs (RASDs) have been proposed as a way to reduce survey errors, by leveraging auxiliary variables to inform fieldwork efforts, but have rarely been considered in the context of cross-national surveys.

Using data from the European Social Survey, Dr. Alex Cernat, Dr. Hafsteinn Einarsson and Professor Natalie Shlomo from Social Statistics simulate fieldwork in a repeated cross-national survey using RASD where fieldwork efforts are ended early for selected units in the final stage of data collection.

Demographic variables, paradata (interviewer observations), and contact data are used to inform fieldwork efforts.

Eight combinations of response propensity models and selection mechanisms are evaluated in terms of sample composition (as measured by the coefficient of variation of response propensities), response rates, number of contact attempts saved, and effects on estimates of target variables in the survey.

The researchers find that sample balance can be improved in many country-round combinations. Response rates can be increased marginally and targeting high propensity respondents could lead to significant cost savings associated with making fewer contact attempts.

Estimates of target variables are not changed by the case prioritizations used in the simulations, indicating that they do not impact non-response bias.

They conclude that RASDs should be considered in cross-national surveys, but that more work is needed to identify suitable covariates to inform fieldwork efforts.