Tag Archives: research

Sugar Research Was Anything But Sweet

In Manchester recently I was stopped by a market researcher to ask if I would buy a new product. The whole experience was very poor and left me feeling frustrated. I’m more inclined than most to answer questions since I have an unhealthy interest in research, so if I walked away feeling frustrated what would other people feel like?

The researchers had set themselves up on a side street, next to a busy main shopping street. I think their choice of site wasn’t ideal, since the side street was very quiet. Far better to set yourself up on the edge of the main street, as you have the chance to (pseudo-)randomise then, rather than having to ask everyone who passes. But still…

Without explaining who the researchers were (simply, “we’re researchers…”) they asked if I bought sugar, and what packaging the sugar was in when I bought it. I was showed a cue card with a picture of a sugar packaged in a bag, a cardboard box, and a plastic box. I know I’ve recently bought my wife icing sugar for baking so I know it was in a cardboard box, which I indicated. The researcher, quite rudely, said that it couldn’t have been because that product wasn’t launched yet. I was incredulous. I was giving up my time to answer questions without any compensation and I was being patronised.

I think I should have walked away at this point but I wanted to see what else this researcher had up her sleeve. So, after agreeing that I couldn’t possibly have bought sugar in a cardboard box, she recorded that it was a bag. Whatever.

The next question was, to paraphrase, “If a plastic, resealable box was a penny extra for the same amount of sugar, would you buy it?” Before I get in to my answer and her issues with that, this is not a great way to ask this question. It is asking about a hypothetical situation. The respondent‘s answer could be anything, and there’s no real way to tell if that’s how they would behave. Far better to ask about a similar situation that has recently occurred, and what the respondent did in that case. You are then grounding your question in an actual occurrence, and you can be reasonably confident that the respondent actually behaved in that way. For example, I would ask if the respondent buys other products that are available in plastic, resealable containers.

So, because of this going through my head, I answered that I wasn’t sure if I would buy a plastic container or not. I don’t really care and I don’t really know, so I thought I was doing her a favour by being honest. She was not happy with me. She cajoled me in to answering yes or no, as if it was the simplest question in the world and that I was being stupid or obtuse for not answering her properly.

At that point I actually did walk off because I’d had enough of being patronised, so I’ve no idea what she recorded my answer as. Probably a non-response, or just discarded it. Either way, how can their results even remotely reflect people’s real opinions and buying habits?

If you are reading this and you just happen to work for a large sugar company and have just commissioned some research to see if your consumers would buy a plastic container, discard it. It’s not worth the paper it’s written on. Fire your researchers, and I’ll re-do it for you.


The Worst Survey Ever?

I recently popped into a branch of one of the main high street banks and couldn’t resist picking up the in-branch service questionnaire. These things are relatively common these days, with every organisation from high street shops to the police asking you to rate the quality of the service you received. This one, though, has issues than most.

Paper survey

The first issue is the poor design and quality of the survey. It has been roughly torn along one edge, uses too many colours (including red and green together, which is a disaster for many red-green colour-blind readers), and has a large organisational chart taken straight from Microsoft Word in the middle of the page. Any literature you produce and provide to customers reflects your brand and image, so the poor presentation and finish of this survey is bound to reflect poorly.

The first sentence is one of the most obvious examples of a loaded question I’ve seen, one of the cardinal sins of survey and questionnaire design.

The next four questions are similarly problematic. I don’t think it’s clear that they are questions, as the alignment (centre-aligned) is ambiguous, and there are no places to mark an answer, like a simple empty box. The provided answers are also entirely arbitrary: why can you answer ‘excellent/very good’ for queue experience and not for ‘making you feel valued?’ Is the queue experience actually how long you had to queue, or is it the entertainment provided while you queue that respondents are asked to comment on? And, finally for this section, ‘making you feel valued’ is so vague it’s practically meaningless.

Continue reading

Ethical Requirements of Research

All research should be carried out to the highest ethical standards. It’s important to: protect the respondent or respondents; help ensure good quality research; and maintain the integrity of the research industries who depend on goodwill to attract future respondents.

Complying with the ethical guidelines of the Market Research Society, Social Research Association or British Sociological Association is not arduous for a relatively straightforward research project.

Imagine my dismay, then, at reading that research carried out by the Troubled Families Unit doesn’t seem to have thought about the ethical implications of their research.

Compounding the issue is that this wasn’t a piece of general research, but research where vulnerable members of society were the principal respondent.

Summarising Nick Bailey’s original blog post on the subject, the research seems to have made the following crucial errors:

  • Respondents were not free to decline to participate or to withdraw, a basic tenet of ethical research.
  • Bailey suggests that the identity of the respondents might not be protected.
  • The department’s definition of ‘social research’ and defining the research as a ‘dipstick/informal information gathering’ is dubious.

Neglecting ethical standards has arguably harmed the respondents involved, the social research industry and the government, and I’d certainly take a closer look at the method section and results.

Getting it Right

Getting the ethics right is so crucial for your research; you can’t afford to get it wrong or it will harm your brand. You even need to consider the ethical needs of a straightforward online survey.

The easiest way to make sure you meet your ethical obligations is to employ a market research or social research professional. For a minimal cost they can protect your respondents, the industry (which is important to ensure there are respondents in the future) and your brand.

Contact me if you would like me to look over the ethical requirements of your research >

Don’t Leave Follow-Up Calls too Long

In early July I reported a crime to the local police. It wasn’t an emergency so I just used the number for the local station. Today I received a call from a research company acting on behalf of the police force, asking me to rate my experience of reporting the crime.

My first problem was that I struggled to remember it. I remembered the crime and ringing the police, but couldn’t remember when it was or any of the details of the call because it was so long ago (nearly two months in fact). The only reason I can tell you here that it was early July was because the questioner reminded me. This leads me to rule number one of follow-up research: make the follow-up call within a few days of the completed transaction; certainly don’t leave it two months.

Sometimes you want to wait a while for the respondent to have a chance to use or assess something, for example you might reasonably ask a respondent about the build quality of a product after a couple of months. But in this case I was being asked to rate the call and the quality of the service I received, something which was over and done. I should have received this call a few days after I called.