Not all respondents are created equal. It’s true. Just because you can convince someone to take your survey doesn’t mean you necessarily want them to take your survey. In fact, ensuring that the right respondents take your survey (in the right proportions) is probably among the most important things you can do to ensure that you finish your project with survey results that you can use.
I’ll give you an example: We once conducted a survey about a concert series we host each spring to try to find out what kind of music our guests wanted to hear. When we go the results back, we were surprised to find that, more than anything else, our visitors wanted to listen to 80s music. By a big margin too.
Fine by me — there is a lot of 80s music that I like. But it seemed a little funny. So we ran a few cross-tabs, looked at some other variables, and found that a disproportionate number of our respondents were between 35-44 years old. Given that research has shown that people tend to be attached to the music they listened to as teenagers, is it so surprising that 80s music was so popular among a bunch of 35-44 year olds? After weighting the data to match attendance at the event, other forms of music, including pop, rock, and a couple of other genres jumped to the front.
Imagine if we had just sent our survey to a bunch of 15-18 year olds. Or if we had sent it to women over 65 years old. Or if we had just sent it to respondents in Mexico.
First rule of survey research: make sure your survey people who represent your customers.
Second role of survey research: make sure you survey them in the right proportion to your customer base.
I’m guessing the the reason for these two rules is fairly straightforward. If you don’t survey the right people in the right proportion, how will you know if your results reflect the behavior and attitudes of all of your customers (or the specific group of customers you are trying to understand)? If the people you survey aren’t representative of the people you’re making decisions about, there is very much the possibility that in conducting your survey you could be doing more harm than good. Its as if, in the example above, we got the results of our survey and then went about featuring 80s bands at all of our concerts. Nobody would come. We’d lose tons of money. And we’d assume that the research must have been flawed.
There are three important steps to having the right survey respondents take your survey. Step #1 is figuring out who the right survey respondents are. Step #2 is finding those people. Step #3 is actually convincing the right people to take your survey. The remainder of this post is about Step #1.
If you want to be technical about it, step #1 could also be called “setting your quotas.” It is the process of making sure in advance that you select people in the right proportions (that is, 50% male, 50% female; or 60% male and 40% female) to best represent the people you want to understand.
If You Want To Survey The World
Maybe you just want to learn more about people living in a particular city, or in a particular state, or even in a particular country.
This information is actually easier to come by than information about your customers, and it may even be possible that the information is available for free. The census bureau offers a great deal of information which can be broken down in a number of different ways, and can be narrowed down to very specific regions. You can find information about age, income, how many cars people in a particular region have, and lot more. Other countries offer similar information.
You’ll find that you have a lot of data to choose from. What you need to do at this point is determine which of it is relevant. For example, if you’ve created a new kind of dental implant, you may not want to spend much time collecting demographic information about people under the age of 40. Conversely, it may turn out that it doesn’t matter how old your respondent is (because you have a product that is equally useful to everyone of every age) but it is really important to survey only women. Make these decisions as you are pulling the data so that you can make sure your “quotas” are in the right proportions.
If You’re Trying To Survey YOUR Customers
If you don’t know the demographic and behavioral makeup of your response base, you may find yourself a little challenged with step #1. After all, if you’ve never surveyed your customers before, how are you going to know who you should survey?
When I used to work in the marketing department of a summer-stock theater, we used to go into the parking lot and write down the states on everyone’s license plates. Pretty crude. But if a substantial number of your customers pay cash and you don’t have a way to know where they are from, it can be a reasonably effective method for understanding where your customers come from.
If you’re an online business, or a business that has invoices that contain content information for your customers, then you can probably get some useful information right from your database. You can examine the breakdown by gender and figure out where they live. You can tell, for example, whether or not all of your customers come from a specific geographic area (indicating that you should only survey people from that area) and even how frequently they visit you (if you’re trying to find out about your repeat customers, it may not be appropriate to survey people who haven’t been visiting you for long).
Theme parks and other out-of-home entertainment destinations periodically conduct something called a “Point of Origin” survey. This is a survey, conducted of randomly selected guests as they enter the facility, is used to establish a baseline understanding of where people are coming from, how old they are, gender, ethnicity, and other information. Because it is a random survey of customers, and theoretically reflects all of your customers, it is a valid way of understanding who your customers are — which means that you can use it to make such statements as “25% of my customers are between 25-35 years old” and then use it to make sure that approximately 25% of your respondents fall within that age range.
A Point of Origin survey doesn’t have to be long. And it isn’t hard to conduct. All you need to do is have an employee stand just inside the front door of your store and ask people to take a short, anonymous survey before they start shopping. Alternatively, you could conduct the survey at the cash register, which would give you the added capability of know who purchased and who didn’t.
Remember — keep your Point of Origin survey short so that it won’t create any hardship for your paying customers. Let them know why you’re doing the survey (you’re trying to learn just a little bit more about the people who are your customers) and perhaps offer an incentive for participating (chance to win a pair of stereo speakers, or something that you actually sell).
Once you’ve conducted your Point of Origin survey, you’ll have a good basis for determining who should participate (and in what proportion) in your other surveys.
What If There Is No Easy Way To Determine Your Quotas
Sometimes you want to survey someone other than you customers. Maybe you’re creating a business plan and don’t yet have any customers. Or maybe there is no easy to way to survey your customers. Or maybe there isn’t time to survey your customers.
When you’re in a situation like this, sometimes the best option is to make an educated guess. That is, watch the customers walking in the front door. If it looks like there are more women than men coming in, then maybe you survey 70% women and 30% men. If it looks like you have a lot of customers with kids, maybe you say that you’re going to survey 60% families with kids and 40% families without kids.
What is important, when you’re doing things this way, is that you make sure that you’re surveying enough people from each group that you want to be able to describe to make your results meaningful. For example, your customers may be mostly women with just a few men and you might decide to survey 95% women and only 5% men. While this will probably give you good overall results, you need to ask yourself up front whether or not you’re going to want to look at the results from the men separately from the women (perhaps you want to know why the men are coming into your store).
To effectively do this, you need to collect at least enough male responses so that you can analyze them properly (do you really want to launch a new men’s product line based on what 10 men had to say?).
In my experience, as a rule of thumb, any time I want to be able to summarize the results from a particular subgroup of respondents, I try to make sure that I have a minimum of 50 responses in that group (although, to be fair, I should say that my average sample size for most projects is about 1,000 responses so collecting 50 from a particular subgroup isn’t much of a challenge for me). If you’re only collecting a few hundred responses you may feel comfortable going as low as 25, but you probably don’t want to go lower than that (and even then you’re going to have to watch your margin of error).
The point of all this is that even if you guess, as long as you have enough of a sample from each of the segments (that is, as long as you have enough male and female respondents) you should be able to compensate if you later find that you’ve collected too many responses from one group or too few responses from another group.
Guessing isn’t optimal, but when it comes down to it, its better than not having any quotas at all, and 90% of the time it is going to get you into the right ball park.
source: http://www.marketresearchtech.com/survey-quotas-establishing-respondents.htm
