Author Archives: Phil

About Phil

Social and market research consultant and practitioner. Co-convenor of Sociologists Outside Academia Group. Prodigious drinker of coffee.

Survey Question Design

I received a link to this survey recently, asking me to rate my experience with a popular brand of sofas and furnishings. While it got a lot of things right, it got a few of the fundamentals wrong.

Survey question

In case you can’t read the screenshot, the two questions are:

  1. Thinking about your overall customer experience, how likely are you to recommend [the company] to friends, family and colleagues? Answers given on a scale of 0 to 10.
  2. Please tell us a little more about why you have given this score.

Using a scale of 0 to 10 is good, and the fact that it’s not part of a matrix is even better. The problem with its implementation here is that the scale reads backwards, i.e. from 10 on the left to 0 on the right. I assume this is either an oversight or, if I was feeling cynical, a suggestive way to improve the score.

The scale should always read naturally, i.e. from 0 on the left to 10 on the right, as someone would read a series of numbers in ascending order.

A progress indicator was good to see, as well as the general lack of matrices, so it wasn’t all bad. In fact this is one of the better laid-out surveys I’ve seen recently.


The Importance of Encrypting Online Survey Data

Yellow Padlock

I co-convene the Sociologists Outside Academia group of the British Sociological Association, and as part of that role I volunteered to help the BSA with their annual membership survey.

One of the recommendations I made to the BSA is that because they intended to collect personal and sensitive information from respondents, it is a good idea to encrypt the online survey to make sure you or your organisation complies with the Data Protection Act 1998 (other countries will invariably have similar legislation).

The encryption is carried out usually by a technology called transport layer security (TLS), and is similar to that used to encrypt your credit card data when you pay for goods online, complete forms requesting personal information, or log in to your online banking service. You know your connection in encrypted because your browser usually displays a padlock logo similar to that in the image below:

Secure browser connection


If you build your survey online, encrypting your survey is usually just a matter of ticking the relevant box in your survey software. In SurveyGizmo, for example, when you have launched your survey, edit your survey link under ‘My Links and Campaigns’ and under ‘Link Protocol’ change this to ‘Secure (https://)’. Simple!

Settings for Survey Gizmo



Other software, such as SurveyMonkey, will have a similar option. If you are using another tool, check the manual or online help.

Why You Should Encrypt

If you are not collecting personal or sensitive data, then you don’t need to encrypt the connection. However, most online survey packages offer encrypted connections with their most basic packages (except their free packages) which cost a minimal amount of money. I’m not a lawyer or solicitor so can’t advise you on whether you should encrypt your connection or not, but if you have the option to enable an encrypted connection anyway, I recommend you do it. It doesn’t cost any more, it doesn’t perceptibly slow down the user’s connection, and it demonstrates to the respondent that you value their data enough to secure it, earning you trust in the process. And it helps you comply with the Data Protection Act if you do decide to collect personal information later.

Travelling by Train with a Physical Disability

In early November I travelled to London to attend the sociologists outside academia (SOA) group which I co-convene with a few colleagues. The trip to London was uneventful, but the return trip turned out to be eye-opening in more ways than one. I experienced first hand what travelling is like for a wheelchair user and was both pleased at how helpful most staff we encountered were, but also surprised and disappointed by how difficult it was to get assistance in the first place and at the attitudes of some station staff.

Continue reading

Negotiating Careers Outside Academia

For the British Sociological Association’s 60th annual conference at LSE in April, I volunteered to present at the Postgraduate Forum on life as an early career sociologist working outside academia. In it, I shared a brief summary of my career history to date, and offered a few hints and tips that I’ve picked up along the way that I hoped would prove useful for postgraduate students thinking about leaving academia. I’ve reproduced my presentation here, along with notes for further detail.

Continue reading

The World Giving Index

CAF's Helping companies helping charities book...

Image by HowardLake via Flickr

Last month (September) the Charities Aid Foundation (CAF) published the first World Giving Index. The report is available online from CAF, and examines how charitable the people of each country are. I’m going to use the next two or three blog posts (not quite decided how many I’ll need yet) to share my reflections on the report, its methodology and conclusions. First blog post (this one) is an introduction to the report.

Continue reading

Only Be As Specific As You Need

I think this question is definitely too specific for anybody’s needs:

When Did You Start Using The Internet?In case you can’t see the screencapture, the question asks, “When did you start using the internet?” Possible responses are:

  • 2010
  • 2009
  • 2008
  • 2007
  • 2006
  • 2005
  • 2004
  • 2003
  • 2002
  • 2001
  • 2000
  • 1999
  • 1998
  • 1997
  • 1996
  • 1995 or before

I’m afraid I just don’t remember. I’ll have to check my diary for ‘started using internet today.’

This question is asking for a very, very specific answer. One that the respondent is going to struggle to remember, and isn’t really going to give much useful insight anyway. Does it really matter if there’s a difference between users who have been using the internet since 1997 and 1998?

To me, this question is like asking how far it is between London and Tokyo, to the nearest centimetre.

Use a level of specificity that is appropriate for its use and that your respondent might be able to remember.

In case you’re wondering, this is a bone fide survey question sent to me to answer from one of the many survey sites out there, but I won’t tell you which one.

Keep Polls Simple

While doing a bit of research on the minimum price of alcohol, I stumbled across this website poll:

Cosmetic Surgery Poll

If you can’t read the screenshot, today’s poll is: “Would you ever consider plastic surgery?” This is a poor question in itself, but lets move on. Possible responses are:

  • Maybe, if it wasn’t so expensive.
  • Yes I would seriously consider it.
  • No way, I’m happy with what I have.
  • Yes but I can’t afford it.

What, exactly, is the difference between ‘maybe, if it wasn’t so expensive’ and ‘yes but I can’t afford it’? While there’s a slight difference in semantics – ‘yes but I can’t afford it’ being more decided – I doubt the average respondent is going to agonise over this. What’s wrong with:

  • Yes
  • No
  • Not sure

And if you really, really want to know why people answer the way they do, you need another question.