Fast parallel additive smooth modelling of daily pollution monitoring data Seminar

The UK 'black smoke' monitoring network has produced daily particulate air pollution data from a network of up to 2000 monitoring stations over several decades, resulting in >10^7 measurements in total. Spatio-temporal modelling of the data is desirable in order to produce daily exposure estimates for cohort studies, for example. Generalized additive models/Latent Gaussian process models offer one way to do this if we can deal with the data volume and model size. This talk will discuss the development of methods for estimating generalized additive models having of order 10^4 coefficients, from of order 10^8 observations. The strategy combines 4 elements: (i) the use of rank reduced smoothers, (ii) fine scale discretization of covariates, (iii) an efficient approach to marginal likelihood optimization, that avoids computation of numerically awkward log determinant terms and (iv) marginal likelihood optimization algorithms that make good use of numerical linear algebra methods with reasonable scalability on modern multi-core processors. 600 fold speed ups can be achieved relative to the previous state of the art methods. This enables us to estimate spatio-temporal models for UK black smoke data over the last 4 decades at a daily resolution, where previously an annual resolution was challenging.