Conducted by Professor Patrick Sturgis the study investigated why self-completion surveys like the Gambling Survey for Great Britain (GSGB) report higher estimates of gambling and PGSI risk levels than traditional face-to-face surveys such as the Health Survey for England (HSE) or Adult Psychiatric Morbidity Survey (APMS).

The aim of the research was to provide causal evidence on how survey design features influence estimates of participation and Problem Gambling Severity Index (PGSI) rates.

The study explored whether mentioning gambling in the survey invitation affects who responds, whether being interviewed by another person suppresses self-reporting of activity and consequences, and the impact of being presented with a longer and updated list of gambling.

Does explicitly mentioning gambling in the survey invitation affect who responds?

The report concludes that mentioning gambling explicitly in the survey invitation did not affect the overall response rate but did lead to a 4 percentage point increase in reported gambling participation, suggesting that individuals with a personal interest in gambling were more likely to take part.

The report found the rate for those who scored 1 or above on the PGSI scale was 1.8 percentage points higher in the gambling-invitation group, though this difference did not reach statistical significance.

Does being asked questions by an interviewer affect participants’ responses to PGSI questions?

The study found an interviewer asking the questions via telephone had a substantial impact on reported PGSI scores – the rate for those who scored 1 or above on the PGSI was 4.4 percentage points higher in the online self-completion condition compared to telephone interviews.

This represents an almost 50 per cent increase in those scoring 1 or above on the PGSI and strongly suggests that respondents under-report undesirable behaviours in the presence of an interviewer, which reflects the general literature in this area.

Does being presented with a longer and updated list of gambling activities impact prevalence estimates?

The report found that updating the list of the gambling activities – as the GSGB did to capture new gambling products – had no significant effect on reported gambling participation or the rate of those who scored 1 or more on the PGSI.

Ben Haden, Director of Research and Policy, said: “We welcome the results of this report which sheds important new light on the impact of different survey methodologies in relation to gambling.

“This research builds our confidence in the outputs of GSGB, helps to understand the differences between surveys published on gambling and will improve our guidance for users.

“We recognise that it is impossible to definitively measure participation and the consequences of gambling through one research vehicle alone. We will continue to work on refining GSGB, accessing different datasets and working with other producers of gambling related surveys to produce a rounded evidence base to inform our work.”

Professor Patrick Sturgis said: “The experimental nature of this research means we can draw strong causal conclusions about the factors that lead to wide variability in gambling estimates across different surveys.

“While no single study will enable us to determine the ’true’ values for key gambling estimates, these findings make an important contribution to our understanding of how different survey design features influence the results obtained.”

The study also recommends the Commission reviews its online guidance on interpreting GSGB estimates of gambling and gambling harm to better reflect the likely causes of differences between them and those of earlier health surveys. This is an action the Commission will be carrying out as part of its ongoing drive to improve research development.