Can You Spot the Bias in These Popular Survey Questions?

One of the biggest pitfalls in survey design is the inclusion of biased questions. When your survey is biased, the data you gather will be flawed and nearly useless, meaning that you've wasted your time and money.

Bias can extend beyond the way a question is written. It can also be found in formatting, word choice, scales, question ordering, and question type.

Here are just a few examples of surveys that contain bias. Keep in mind, these are actual real-world survey questions from large, multi-national corporations representing a variety of industries. Let's see if you can spot the bias!
 
1. How satisfied are you with the quality of the final product?
Very Satisfied    Satisfied    Neutral    Dissatisfied    Very Dissatisfied
 
Did you spot the bias? By using a bold font for the "Very Satisfied" response, it immediately creates a bias. The eye automatically focuses on the bold text and makes the respondent more likely to choose that answer.

2. What best describes you? (Please select one only.)
A. I like to cook, but need quick, easy recipes.
B. Cooking is my passion and I love to experiment.
 
This one is easier. Not everyone likes to cook. There need to be more options for the respondent including "I don't like to cook."

3. Credit union hours are conducive to conducting your business?
Excellent        Good       Fair        Poor

This question assumes that the hours of the credit union are conducive. A better question would be: "How would you rate the credit union's hours of operation?"

4. And here are a few more statements describing (Company Name).
Empowers my workforce to be more collaborative         
Offers solutions to improve productivity  
Is a company that delivers on its promises
Enables my business to be more agile

Strongly Agree      Somewhat Agree     Neither Agree Nor Disagree  

Somewhat Disagree     Strongly Disagree

In this case these questions make another assumption, that the person taking the survey has a favorable attitude towards the company and has enough familiarity to rate it.

And my personal favorite (not so much biased, but just plain odd):
 
Have you been on an underwater submarine tour in the past 12 months?                 YES                 NO
 
Survey bias can sneak in when you least expect it. If you design a survey on your own you may be inserting bias without even knowing it. Remaining neutral may be difficult or even impossible. That's where an outside expert comes in. If you want to ensure that your survey isn't biased, contact us today and we'll help make your marketing project a success.

If you'd like your survey to be unbiased, contact us now.

Debra Power

Power Marketing Research, 804 Phoenix Drive, Ann Arbor, MI, 48108