In Response to ‘Is n=1 Ever Enough?’

Alone (Photo credit: JB London)

I found reading Is n=1 Ever Enough by Nicky Halverson highly thought provoking.

n (lower case) refers to the sample size, that is how many people you have asked your questions. Nicky’s article is therefore asking is it ever ok to just ask one person?

There’s no chance that a sample of one is statistically significant, but research, even quantitative research, isn’t always about producing statistically significant results. Instead, good research is about producing appropriate information to make informed decisions.

Sometimes statistically significant or highly detailed information is necessary, and therefore appropriate. Examples might be high risk decisions involving patients or significant sums of money. It’s probable that n=1 isn’t going to be sufficient.

But what about low risk decisions? Well, I have to agree with Halverson that a sample of one wouldn’t be my first choice, and I would encourage my client to reconsider. I firmly believe that one main criterion of good research is that it is reliable. In the research context, reliable specifically means obtaining consistent results. By achieving reliable – i.e. consistent – results, you can be more confident that your results are going to be meaningful and useful. By having a sample size of one you cannot determine if your results are going to be consistent with each other, simply because you will not have anything to compare your result with.

But in some cases that might not matter. Going back to the purpose of research, it is about producing information to inform a decision. If it involves low risk, your client might be comfortable making their decision on the basis of just one response. My job in this hypothetical situation would be to make my client aware of the risks of basing their decision on one case, but it is up to them if they choose to do it or not.

So, as with so much, it depends. It depends on how comfortable your client is making a decision with very limited information. But ultimately, n=1 is infinitely better than n=0.

Advertisement

Timing and Allowing for Seasonal Variations

Sometimes getting the timing of your research right is just as important as getting the method right.

Typical examples include businesses or organisations with significant seasonal variations in their output or activities. For example, an organisation that wants to measure the effectiveness of a Christmas campaign would do well to carry out their research in the run up to Christmas.

There are, though, less obvious examples.

I recently completed a project for the Friends of Rhyddings Park who were looking to determine the number of people who use the local park. Many of the facilities in the park focus around children – including two play areas – but the main bulk of the research was scheduled to begin after the summer holidays when the children and young people had returned to school so would not be using the park during the day.

To get a picture of the true park use during the school holidays it was important to at least bring some of the research forward, and that was exactly what I suggested and what my client did since they recognised the importance of getting their research right.

When you’re planning your research bear in mind seasonal variation in your activities, and try to plan your research at the best time to answer your research question.

 

Ethical Requirements of Research

All research should be carried out to the highest ethical standards. It’s important to: protect the respondent or respondents; help ensure good quality research; and maintain the integrity of the research industries who depend on goodwill to attract future respondents.

Complying with the ethical guidelines of the Market Research Society, Social Research Association or British Sociological Association is not arduous for a relatively straightforward research project.

Imagine my dismay, then, at reading that research carried out by the Troubled Families Unit doesn’t seem to have thought about the ethical implications of their research.

Compounding the issue is that this wasn’t a piece of general research, but research where vulnerable members of society were the principal respondent.

Summarising Nick Bailey’s original blog post on the subject, the research seems to have made the following crucial errors:

• Respondents were not free to decline to participate or to withdraw, a basic tenet of ethical research.
• Bailey suggests that the identity of the respondents might not be protected.
• The department’s definition of ‘social research’ and defining the research as a ‘dipstick/informal information gathering’ is dubious.

Neglecting ethical standards has arguably harmed the respondents involved, the social research industry and the government, and I’d certainly take a closer look at the method section and results.

Getting it Right

Getting the ethics right is so crucial for your research; you can’t afford to get it wrong or it will harm your brand. You even need to consider the ethical needs of a straightforward online survey.

The easiest way to make sure you meet your ethical obligations is to employ a market research or social research professional. For a minimal cost they can protect your respondents, the industry (which is important to ensure there are respondents in the future) and your brand.

Contact me if you would like me to look over the ethical requirements of your research >

Related articles

• The Listening to Troubled Families report is an ethical failure | Nick Bailey (guardian.co.uk)