Using an open-ended question in a satisfaction survey

Open and closed questions: how do they complement each other?

Theoretically, a satisfaction survey consists of several closed questions and concludes with an open question such as "Please let us know what you think...".

In answering this open-ended question, the respondent can express himself on 2 distinct elements:

• The customer experience itself, often already noted through closed questions. In this case, the customer emphasizes a positive or negative aspect of his or her experience.
• Expectations and suggestions. These are elements that are generally not covered by closed questions.

It should be noted here that the open-ended question must be placed at the end of the questionnaire. Otherwise, respondents will tend to express themselves redundantly on the open question, and then on all the closed questions.

Finally, we recommend having a single open-ended question within the same questionnaire. We have found that respondents find it difficult to distinguish between them, even if the wording is clearly distinct. It's important to remember that respondents quickly scan the questionnaire and its wording... they can easily get lost if the questionnaire becomes too complex.

In this way, the open question complements the closed question, but in no way replaces it.

Frederic Reichheld, in his reference book "The Key Question", suggests the following structure for a satisfaction survey: 3 to 4 closed questions followed by an open question.

How useful is the open-ended question?

This structure (and especially the number of questions to be asked) varies according to the richness of the customer experience itself.

And there's no doubt that the importance and usefulness of the open-ended question depends directly on the number of closed-ended questions asked previously. In the case of a short questionnaire (where only the main attributes of satisfaction are noted in the closed questions), this open question becomes very important, as it enables the respondent to explain his or her customer satisfaction.

In the case of a long questionnaire (more than 30 questions), it is less important and allows us to collect the customer's expectations and suggestions (by nature, not covered by closed questions).

In any case, in a satisfaction survey, the open-ended question is useful for :

Better identify customer complaints

With the open question, your customer expresses his expectations and suggestions, but also his frustrations and, sometimes, his threats (legal risks, threat to express himself on social networks...etc). The open question can therefore be used to feed the flow of customer complaints, often defined in terms of the overall satisfaction rating.

Making the most of customer suggestions

Thanks to the suggestions given by customers on a specific point of your offer, you have access to a wealth of raw information, based on real-life experience, which should not be overlooked. So, even if you have a semantic analysis tool (capable of automatically encoding customer comments), it's absolutely essential to read all the verbatims received wherever possible.

What are the biases in using open-ended questions?

On the other hand, there is one purpose that it would be wrong to attribute to the open-ended question: that of being able to understand customer satisfaction and, consequently, prioritize the corrective actions to be taken to improve it.

On several occasions, we've come across Quality or Marketing managers who, after reading a few dozen verbatims, were certain of the corrective actions they needed to take to improve customer satisfaction (our studies later proved these assertions to be wrong). But how can you blame them? Customer feedback is powerful because it's personal and subjective (even more so than a score from 1 to 10). So it's tempting to react on the spot, especially if several customer comments match up.

But be careful: in an open-ended question, customers don't list, point by point, the things they liked or disliked; they focus on what impressed them (and not on what is most important to them).

Only a detailed statistical analysis, based on the study of closed questions, can boast a precise definition of customer satisfaction. Priority Action Maps (Importance/Satisfaction Matrix) or Bayesian analyses (neural networks) also help to prioritize actions to be taken.

This is because open questions lack the granularity of closed questions. When a customer says "I find it abnormal that the wifi connection has to be paid for", does this correspond to a score of 2, 4 or 6 (out of 10)?

Conclusions

The open question is therefore important on several levels:

- First of all, it's important for the respondent because it allows them to express themselves freely. It is thus seen as a channel of expression in its own right, which the customer uses with varying degrees of conviction (quite often, we read "I don't think you'll answer me, but I'm dissatisfied with ..."). In this way, the customer calls on the company to react and deal with his request. Open-ended questions are also useful, as they allow the customer to talk about aspects of the customer experience not covered by closed-ended questions.

- For the company, at the operational level, it can be used as a complement to closed questions, which help to identify the strengths and weaknesses of your offering. Once you've identified the areas you want to act on, customer feedback can be used to better prioritize and define the corrective actions to be taken. Open-ended questions can also be used to identify customer complaints.

- More strategically, the open question also feeds the company with new ideas (to solve identified problems or on key concepts). That's why it's important to read as many verbatims as possible. They contain the raw information experienced by your customers. No company in the world can do without them.