How is concept testing like flying an airplane? It’s all about the checklist.
Concept testing is a powerful yet simple market research technique. It has also become quite standardized, with minimal ‘ifs, ands, or buts’ and few potential surprises.
That being said, however, it is possible to derail a concept test. Certain things need to come together perfectly to get the most out of this tried-and-true tool. These include writing a good questionnaire, checking the quality of stimulus (how accurately the product’s features are conveyed), choosing the right target group, and selecting the most pertinent success-defining metrics.
Of all these, getting the questionnaire right is perhaps the most important. As this piece will interface directly with the respondents, and it can have a huge impact on the quality of data collected.
Let’s have a quick talk about the questionnaire basics that every survey designer should consider. Think of them as a pre-flight checkup; any pilot will tell you that, no matter how good they are, they always do a basic checkup before taking off down the runway. Why? To minimize the risk of a crash.
How to Avoid Crashing with Your Concept Test Questionnaire
So, no matter how experienced or how good of a marketer you are, consider these tips when you’re creating your next questionnaire.
1) Start with the end in mind. Before coming up with any questions, ask yourself: What business decisions am I trying to make? Who is my audience? What data will be required to carry out the analysis? Without these objects firmed in mind, your chances of turning out a good questionnaire are slim.
2) Keep it at an optimal length. Surveys can get lengthy quickly. It’s easy to get sucked into the ‘collect-all-available-data’ trap. But take a step back, then dig deeper. You’ll realize that you have filled your questionnaire with a bunch of nice-to-have but nonessential questions.
What’s the problem with that? Too many questions can fatigue or – even worse – irk your respondents and result in poor quality data. Not only that, but the longer your questionnaire is, the higher your cost per interview will be.
One caveat. There’s nothing wrong with having a lot of questions if your test needs lots of questions. As an example, take Sequential Monadic testing, where respondents answer the same questions for more than one concept. Obviously, it’s not wise to leave out anything essential just to save survey space.
3) Choose the right question format. There are a wide variety of question formats at your disposal. These range from unstructured open-ended items to structured multiple-choices to advanced heat map questions. Which do you need?
As a rule of thumb, you should go for the question type that facilitates the collection of the right data without confusing or overwhelming the respondent. A couple of other points to keep in mind:
- Options in single- or multiple-choice questions should account for all potential variables. The option of ‘Other(s), please specify’ should be included when required to give a complete picture.
- Including open-ended questions is a good practice. It doesn’t restrict the respondents and can provide powerful insights. At the same time, don’t rely solely on the open-ended question. Too many can take respondents a long time to fill, which could result in boredom, frustration, and fatigue.
4) Pay attention to your phrasing. First and foremost, a question’s phrasing should be clear and understandable. It should sync up with your audience’s way of thinking and talking. Try to word the question as objectively as possible, thus avoiding any vague or biased answers. Words like “often”, “sometimes”, and “recently” are all open for respondent’s interpretation and rarely result in spot-on data accuracy.
Design your survey so that respondents find it easy and engaging to fill. Keep the overall tone pleasing and provide enough context so that the questionnaire’s flow makes sense to them. Finally, don’t forget to test your questionnaire before sending it into the field – or down the runway, as it were, with its pre-flight checklist properly completed.
