State space models developed at the University of Southampton have transformed the way that changes to surveys are being made in official statistics. The techniques have increased the availability and accuracy of estimates of the effects of changes in survey designs and procedures in major government surveys.
Official statistics are a crucial component of democracy as they provide evidence against which to judge a government’s performance. Many are derived from surveys with long histories, and a significant aspect of their value comes from their continuity.
However, changes in survey designs and methods are unavoidable, with cost efficiency often being the main driver. Such changes can cause systematic differences in the survey outputs that do not reflect the real changes in the underlying population parameters.
These are known as ‘discontinuities’.
The aims of the research were: to explain which elements of change are attributable to the change in methods, to estimate the underlying true change, and to estimate the effect on the quality of the statistical outputs during the transition.
This way, users are informed about the effects of the change, which increases their confidence in the change and allows them to continue to use statistics during a transition, accounting for any changes in quality.
Ultimately, the research promotes openness about changes and helps to maintain public trust in official statistics outputs.
The International Passenger Survey (IPS) is an important element of government statistics in the UK used for the estimation of both numbers of migrants and tourism expenditure.
During 2017-18, The Office for National Statistics (ONS) changed the data collection method for the IPS from paper to tablet-based interviewing. This improved the data quality and the efficiency of data entry by interviewers.
Professor Paul Smith led a project to give advice on the transition, and to estimate the power of the chosen roll-out strategy to detect discontinuities of a given size.
The work provided the ONS with a full understanding of the impacts of the change in mode of collection for this high-profile survey, meaning confidence in its outputs were maintained throughout this transition.
The National Survey in Wales is a key general-purpose survey that collects information on a series of variables used for monitoring a range of government policies.
In 2014, the Welsh Government implemented a recommendation from the Southampton team to combine data collection from five existing surveys. This integration made the data collection more efficient and the survey management simpler.
The Southampton team used results from a pilot of the integrated survey to estimate discontinuities, including those at local level. The analysis detected some discontinuities in questions from three of the original surveys, and these were particularly important in health-related questions.
By taking steps to understand the discontinuities, the customer organisations have been able to continue using National Survey data for their performance indicators, both at national and local authority levels. These indicators are important for monitoring under the Wellbeing of Future Generations Act 2015.
| Staff Member | Primary Position |
|---|---|
| Paul Smith | Professor of Official Statistics, Director of Southampton Statistical Sciences Research Institute (S3RI) |
| Nikos Tzavidis | Head of Economic, Social and Political Sciences, Professor of Statistical Methodology |