This time we are going to analyze how to create a questionnaire, and more specifically, we will give you some insights into rating scales in survey questions.
It is often one of the most common headaches when designing questionnaires: which rating scale to include in each question. We explain what you need to know before deciding.
The first thing you need to consider is that it is logical to hesitate when choosing the response rating scale because most of the time, several different rating scales are suitable for obtaining an information objective. However, only one will be optimal.
It’s best to see it with an example:
We are conducting research to test a new design for a specific product that will be launched in the market. After showing the new product design, we want to know the purchase intention that consumers have for it.
We can design various types of questions to achieve this single information objective. But the type of information we obtain and its utility for making strategic decisions will differ depending on the question design.
We could choose Option 1:
Would you buy this product?
A dichotomous question that forces the potential consumer to take a position in favor or against.
Advantage: It differentiates us a lot.
Disadvantage: It doesn’t provide nuances (someone who is sure they will buy it will select «Yes.» But someone who is undecided might also choose «Yes,» and we won’t be able to differentiate them during the analysis).
Another thing to consider is that the result of this question will be expressed as a percentage of Yes and No (unlike other options that we will see that yield different types of results).
We could also choose Option 2:
How likely are you to buy this product?
A question with four response categories that differentiates us and also provides nuances. It is common to analyze it in terms of the percentage of mentions for each response category.
Additionally, we can convert it into a dichotomous question during the analysis (by summing the percentage of «Very Likely» and «Somewhat Likely» responses and contrasting it with the sum of percentages for «Not Very Likely» and «Not Likely at All»).
Another option is Option 3:
How likely are you to buy this product? (With 1 being Not likely at all and 5 being Very likely)
1 2 3 4 5
Here, the rating scale is no longer nominal but numeric with 5 positions. It provides even more nuances than the previous options. Keep in mind that during the analysis, we won’t be looking for the percentage of responses in each category but rather the average score obtained.
These rating scales are very useful when we want to compare several product designs with each other and observe the average preference rating achieved by each design.
However, there are other types of rating scales, and this is the last one we will analyze this time; Option 4:
How likely are you to buy this product? (With 0 being Not likely at all and 10 being Very likely)
0 1 2 3 4 5 6 7 8 9 10
Similar to the 5-position scale but with even more nuances. It often works well with projects in Spain because it is a scale used in the educational system, and consumers are very familiar with it..
However, not everything is an advantage. Unlike the dichotomous «Yes» and «No» (Option 1), the 0 to 10 rating scale provides nuances but sometimes discriminates little because scores tend to cluster in the 6 to 8 range. This can lead to similar average scores when comparing several designs, making it less helpful for the business decision we seek.
As you can see, in general, there is no optimal rating scale that works for all cases. We must choose one or the other based on the information objectives for each case, the type of statistical analysis we want to perform, the graphical representation that helps visualize the results, and how we want to communicate them within the organization.
In any case, remember that all rating scales provide information and can be valid. The important thing is to select yours thoughtfully, considering the pros and cons of each one. If you do that, you’re sure to make the right choice.
Update date 15 April, 2024
