Assessing Service Needs: Synthetic Estimation

Synthetic estimation provides an extremely cost-effective method of benchmarking the public health needs of populations without recourse to local surveys.

Synthetic estimation has been used since the 1960s to estimate the prevalence of a wide variety of population characteristics at the local level – from indicators of health status and measures of health-risk behaviour through to unemployment rates and average household incomes.

The approach rests on the weighted attribution to local areas of sub-population prevalence rates derived from a national or other large-scale survey. For instance, the Health Survey for England (HSfE) provides a suitable survey upon which to derive age, sex and social class specific prevalence rates for a wide variety of diseases and health-risk behaviours. By applying these sub-population rates to corresponding counts in local areas, and summing the subtotals, it is possible to calculate estimates of local area prevalence rates for those diseases and health-risk behaviours.

The method is straight-forward and extremely cost-effective relative to the implementation of a population survey, but it does assume that the local prevalence of a condition or behaviour is entirely dependent upon the socio-demographic composition of the area.  More recent work has thus turned to multilevel models as a means of capturing area level effects, as, for instance, in work undertaken on behalf of the Health Development Agency to calculate ward-level estimates of smoking prevalence.

Models which combine individual and area level effects represent a significant advance, but it has proved difficult to quantify the precision of small area estimates without simplifying assumptions.  Recent advances in computing have, however, made feasible a radically new approach by which full probability densities of local area prevalence rates can be obtained. It is thus now possible to derive both point estimates and readily interpretable measures of the precision of those estimates.

The Bayesian Approach to Small Area Estimation

The theoretical advantages of the Bayesian approach have long been recognised, but until recently the complex integrations necessary made all but the very simplest modelling frameworks effectively intractable. Fortunately, the recent development of Markov Chain Monte Carlo (MCMC) simulation techniques — along with the remarkable recent improvement in computing power upon which these iterative simulation techniques depend — provides a random sampling approach which overcomes the need for complex numerical integration. Modelling complex situations involving many parameters has become a practical feasibility. 

Today, the Bayesian approach, combined with associated MCMC technology, provides a unified and flexible framework within which multilevel models (involving both individual- and area-level covariates and both fixed and random effects) can be fitted to individual survey data and then used to generate predictive distributions for small area estimates. Adopting a Bayesian approach, and thereby modelling the full predictive distribution of the estimates, it is possible to express the precision of those estimates in terms of, say, 95% ‘credible intervals’ – the range within which we are 95% certain the true value lies.  A good example of how these 'credible intervals' can be used when making data available to practitioners is provided by our recent work estimating local area adult basic skill deficits on behalf of the Department for Education and Skills.

Bayesian Analysis in Practice

Bayesian analyses of complex statistical models using MCMC techniques are computationally extremely intensive and demand specialist skills unlikely to be available to most public sector organisations. Thus, whilst it has huge potential, it has yet to be widely adopted. At RAE Consulting we have the necessary expertise and can provide public sector organisations with a robust and cost-effective means of benchmarking local area service needs.  The approach can never provide the accuracy expected of modelled surveys, but synthetic estimation, because it exploits existing data sources, is far cheaper.  

We would thus normally recommend synthetic estimation as an extremely cost-effective and appropriate method of benchmarking the service needs of local populations, but would advise clients to consider modelled surveys when seeking to monitor change over time.  Please contact us if you would like to discuss your particular needs in detail.