Dodgy data: How can polls on the same topic give us such different results?

There’s an old saying that there are ‘lies, damned lies and statistics’. As the data and insights lead at ASH, I’d like to defend stats when done right, but there are also many ways to mislead.

The Generational Smoking ban contained in the Tobacco and Vapes Bill has been the subject of multiple polls. ASH polling, conducted by YouGov, shows consistent support across the political spectrum but every now and then a poll comes up that shows the opposite. How are we to know which is the most accurate portrayal of public opinion? 

One such poll was published by The Sun in January claiming that a new survey shows that ‘Two-thirds of Brits against new generational smoking ban” in the Tobacco and Vapes Bill. The poll was conducted by Whitestone Insight for the Freedom Association.

You might imagine that the wording of the question was something like

‘Do you agree or disagree with the smokefree generation policy?

In fact, they offered a list of five different policies, plus ‘don’t know’, and asked people to choose their single favourite from those options. The generational smoking ban was the single most popular option, with 30% choosing it as their favourite. The majority of people polled, 59%, most preferred one of the three highlighted policies, some variation of increasing the minimum smoking age:

This table shows how people responded to the Whitestone Poll:

The interpretation ‘if a policy on a list isn’t your single favourite, you must actively oppose all the rest’ is absurd. It’s also muddied because a separate popular issue (raising vaping age) was combined with raising the smoking age to 21 but not offered in combination with the generational smoking ban. The headline that two-thirds were “against” the policy is totally unjustified.

A similar ‘divide-and-conquer’ style of polling was conducted by Forest, although their reporting was phrased more carefully. They offered two different options to raise the age of sale of tobacco (the generational smoking ban and increasing the minimum age to 21), along with one option to keep the status quo. They chose to report it as 58% of people supporting alternatives to the generational smoking ban.

Although this is true, a different way to report the exact same results would be:

• The generational smoking ban was the single most popular option, with 35% of people choosing it as their favourite
• That there was a combined 68% preference for either of the two policies proposing raising the age of sale of tobacco
• Only 25% chose keeping the legal age of sale of cigarettes at 18 as their favourite option, the least popular choice.

A part of our work at ASH is to understand and make policymakers aware of the popularity (or otherwise) of potential policies and interventions around smoking and vaping. ASH commissions annual adult and youth surveys, conducted by YouGov, which include opinion polling on policies as well as questions about behaviour and attitudes.

ASH polling consistently shows strong support for the policy across the political spectrum.

Our nationally representative YouGov survey of over 13,000 adults in GB showed that two thirds of the public (67%) support the Smokefree Generation policy. Even amongst smokers, 51% are supportive of the policy. The lowest level of support was among Reform voters, but in this group support for the policy still far outweighed opposition (49% vs. 29%). Similar results for England (rather than GB) have been previously reported in ASH's report Public Support for A Smokefree Society, 2025.

Seeing various distortions of stats in the media might make it seem like the best approach is to ignore all stats reporting and to conclude that we can’t really know what the public think. This is not the case at all. There are many examples of polling conducted and reported carefully and responsibly which can offer policy makers and politicians vital insights.

Finally, for anyone who missed stats 101 here are some questions you could ask when looking at a poll :

1. Source: Is who conducted the poll reported? Are they known to be responsible?
2. Sample size: Is it reported, and is it reasonably large? What’s reasonable will vary, but a very dishonest person might run a poll lots of times with tiny groups until they find what they hoped for!
3. Question: Was the question phrased neutrally, or did it lead people in a certain direction?
4. Reporting: Does reporting accurately summarise the question used, or does it add an unjustified interpretation?
5. Representativeness: Was the poll conducted on people already likely to have a certain view? A poll of an organisation’s social media followers is likely to strongly support their views, but that doesn’t reflect the views of the whole country.
6. Statistical technique: Is the sample weighted, stratified or otherwise set up to reflect the population as closely as possible? This isn’t essential but helps representativeness and indicates that care has been taken.