Porky Polls: People Lie in Surveys, So How Do You Get to the Facts?
How many times have you embellished your answer when a survey asks how regularly you exercise or how many units of alcohol you drink per week? When it comes to market research, and surveys in particular, how do you know if respondents aren’t doing the same?
There aren’t many studies examining how frequently people lie. It is lunacy to assume that respondents will truthfully answer questions about how truthful they are. Neither is it socially desirable to brand yourself a liar – even in an anonymous questionnaire. Nevertheless, a report published some years ago by ORB International, a data shop, tried to determine how many of us fib in questionnaires.
While researchers found that 80% of people in the UK say they always provide truthful answers to surveys, respondents had a lower opinion of others. Just nine per cent said they were prepared to trust published polls.
This anecdote is familiar to people in the business of gathering sentiment for policy or market research. People generally think that they are truthful, while others are not. And on the latter point, they are correct. When it comes to revealing details about themselves, very rarely do respondents tell the whole truth.
Value-Action Gap
There is a perceivable disconnect between what people do, and what they say they do. This pertains to any action that is connected to ethics, aspiration or self-worth. In recent years, the most pronounced example of this can be found in research on how much people do to care for the environment.
While everyone would agree that they care deeply about the myriad issues facing the planet, this sentiment probably doesn’t match up with how often individuals recycle, engage in environmental activism, or spend more on green products.
Market research giant GfK refers to this sort of thing as a value-action gap. In a global study - called Who Cares, Who Does - in collaboration with Europanel and Kantar, it found that 65% of consumers try to buy products that are packaged more sustainably. However, only 29% regularly manage to avoid plastic packaging.
Separately, Kantar found that people overestimate how much they engage in socially desirable activities, exaggerating how much they vote, recycle or opt for organic produce at the supermarket. Meanwhile, they downplay how often they smoke, binge on fast food or watch reality TV.
Empathic Surveys
There is a way to inspire more truthful answers. Very simply, it is to design questionnaires that inspire less shame from respondents. Someone who enjoys a cigarette only occasionally and in social scenarios might not label themselves a “smoker” if they are asked in a survey.
They are more likely to give a straight answer with options such as “I am currently cutting down” or “I only smoke sometimes”. Giving people the opportunity to make excuses for themselves gets more truthful answers, but more detailed ones too.
Another technique is called deflection priming. This is like a qualitative warm up act, when you ask one question to lay the groundwork for a truthful answer to the next. For instance, many are tempted in surveys to exaggerate the level of education they had, saying that they attended university when they did not, according to Jon Puleston, a researcher at Kantar.
The solution, he writes, is to create a soft intro to the hard question. So instead of asking “did you attend university?” in the first instance, first you ask, “did you enjoy school?”. This helps people give a truthful answer, because it explains – partially – why they did not attend higher education.
As it becomes more crucial for brands to cut to the truth of why people spend, those who design studies will need to get better at spotting the lies in statistics.
At Bulbshare, our researchers are well-versed in creating surveys that encourage truth - and we use in-built functionality that encourages instinctive responses and more accurate data.