Assessing nonresponse bias using call record data with applications to a longitudinal study Seminar

In this talk survey quality indicators currently in use in the literature are reviewed and a method to monitor survey outcomes during fieldwork is proposed. The proposed approach assesses nonresponse bias using call record data by comparing estimated and true distributions of specific survey variables at each call attempt using dissimilarity indices. Key indicators such as response rate, nonresponse bias, R-indicators, coefficients of variation, including partial R-indicators and partial coefficients of variation, and dissimilarity indices are monitored and assessed across time and call outcomes during fieldwork.

Empirical analyses are conducted using data from wave 1 and wave 2 of Understanding Society – the UK Household Longitudinal Study 2009-2012. Results show that survey estimates tend to stabilise after around 5 call attempts and, in some cases, at levels that depart significantly from the corresponding true values even after high response rates are achieved. The study demonstrates that most indicators in current use, although adequate to assess nonresponse bias after data collection are not effective in capturing nonresponse bias during the call process. The study concludes that dissimilarity indices and (partial) coefficients of variation indicate best properties. This research has direct implications to responsive and adaptive survey designs, as focusing on simply achieving high response rates have shown not to be cost-effective and considering alternative indicators is crucial to guide survey practitioners on re-defining data collection strategies in real time.