How Valid Is That Survey?

Telephone survey
Click to see

research: validity

LOOKING FOR REALITY

One of the biggest challenges in business is understanding our markets. We often choose to conduct research to get a better handle on market trends, demand, expectations, satisfaction, etc.

There are multiple types of errors in the data we gather. The easiest to quantify and the one most often discussed is sampling error.

Let’s assume we can draw a random sample from your population. But how big a sample do we need to have confidence in our results?

A SIMPLE EXAMPLE

An easy example of how to approach survey sampling is ownership of Ford cars among a population of 100,000 households.

We could pay the state department of transportation for a database, or if we had all the time in the world, we could knock on every door in the county and survey the entire population. Perhaps we would find that 30% of the households own a Ford.

Since we don’t want to spend $1 to $5 a record for 100,000 records, or $5 to $25 each to phone every household, it makes more sense to survey a sample.

But how big a sample do we need, and how small a sample can we get away with?

THE TABLE TELLS A LOT

Population Confidence level Sample size Sampling error
100,000 95% 1,000 3.1%
100,000 95% 500 4.4%
100,000 95% 383 5.0%
100,000 95% 200 6.9%
100,000 95% 100 9.8%
100,000 95% 50 13.9%
100,000 95% 10 31.0%
  • The sweet spot is 95% confidence level with a sampling error of 5%. So the sample size required would be 383. This means that  95% of the time we did a random sample of 383, we would find that between 25% and 35% of respondents own a Ford.
  • But please don’t let someone divide your sample into small sub-groups and try to draw conclusions from within them. For example, a sample size of 100 gives a sampling error of 9.8%. This means that we could get a result that either 21% or 39% own a Ford. This is a very wide range, and we should hesitate to make business critical decisions based on such a small sample.

YOUR FREE STATISTICS CALCULATOR

When we build real-time customer satisfaction system for clients, they get to see the statistical validity of every report they request on the screen.

We’ve reproduced that code for you in a free calculator.

We hope you use it often and tell your friends about it.

Please share this free tool with your friends by clicking Share This below, then Send/Email.

Tags: , ,

Leave a Reply

Free eNewsletter

Stimulating resources for a more fulfilling life & affluent business.


Blog Schedule & Access