LE BLOG BY SATISFACTORY
Using an open-ended question in a satisfaction survey
I. Open-ended questions and closed-ended questions: how do they complement each other ?
Typically, a satisfaction survey consists of a number of closed-ended questions followed by an open-ended question of the following type, “What else would you like to draw our attention to ?”
When replying to the open-ended question, the customer can express an opinion in two separate areas :
- The customer experience itself, which has often already been assessed through the closed-ended questions. In this case, a customer will highlight a positive or negative aspect of his/her experience.
- Expectations and suggestions. This information is not generally covered in the closed-ended questions.
Note that the open-ended question should be located at the end of the questionnaire, otherwise customers have a tendency to repeat themselves in the closed-ended questions that follow the open-ended question.
Finally, we recommend the use of only one open-ended question within a given questionnaire because we have observed that customers find it difficult to differentiate between them if there is more than one, even if the headings are clearly different. Remember that customers run through the questionnaire and its headings very quickly and they may well feel confused if the questionnaire becomes too complex.
The open-ended question is more than enough to round off the closed-ended questions but under no circumstances should it replace them.
Frederic Reichheld, in his reference book, “The Ultimate Question”, suggests the following structure for a satisfaction survey: 3 to 4 closed-ended questions followed by one open-ended question.
II. What use is the open-ended question ?
The structure of the survey (and more importantly the number of questions) will vary, depending on the breadth of the actual customer experience.
There is no doubt that the importance and usefulness of the open-ended question depends directly on the number of closed-ended questions asked first. If the questionnaire is short (only the main aspects of satisfaction are graded in the closed-ended questions), the open-ended question becomes very important because it enables the customer to explain his/her level of satisfaction.
If the questionnaire is long (more than 30 questions), it is less important per se but provides a means of obtaining customer expectations and suggestions (by their very nature, these are not covered by the closed-ended questions).
Whatever the circumstances, in a customer satisfaction survey the open-ended question is useful in :
Identifying customer complaints more accurately
In an open-ended question, your customer can express expectations and suggestions but may also vent frustration and, occasionally, make threats (potential court case, threat of publishing his/her views on social media etc.). This means that an open-ended question may also lead to an increased flow of customer complaints, often defined in terms of the mark awarded to the overall level of satisfaction.
Using customer suggestions
Thanks to suggestions made by customers regarding a specific product or service, you can access a mine of raw information based on real experiences and this should not be ignored. Even if you have a semantic analysis tool capable of automatically encoding customer comments, you must read them all yourself as far as this is possible.
III. What are the common mistakes made in the use of an open-ended question ?
There is one purpose that the open-ended question should not be expected to fulfil – the ability to understand customer satisfaction and, as a result, prioritise the necessary corrective actions.
On several occasions we have met Quality or Marketing Managers who, having read a couple of dozen comments, were sure of the corrective actions required to improve customer satisfaction (our studies subsequently proved that they were wrong). Who could blame them? Customer comments are powerful because they are personal and subjective (more than simply a mark between 1 and 10). It is therefore tempting to react immediately, especially if several customer comments agree with each other.
N.B.: In an open-ended question, customers do not list the things that they liked or disliked point by point. They concentrate on the things that left an impression (this is not necessarily what is most important in their opinion).
Only a detailed statistical analysis based on a study of the closed-ended questions can really pinpoint customer satisfaction. Priority Action Matrix (Importance/Satisfaction Matrix) and Bayesian analyses (neuronal networks) can also help to prioritise the actions required.
The open-ended question does not have the granularity of the closed-ended question. When a customer says, “I think it’s unreasonable to make people pay for a Wi-Fi connection”, does that correspond to a mark of 2, 4 or 6 (out of 10) ?
An open-ended question is therefore important from several points of view :
- Firstly it is important for customers because it gives them an opportunity for free expression. It is therefore considered as a real channel for personal expression, although the customer may or may not be convinced of its usefulness when responding to it (often we read, “I don’t think you will answer but I am dissatisfied with …”). In this situation, the customer is seeking a reaction from the company and some indication that his/her request is being followed up. An open-ended question is also useful because it enables customers to talk about aspects of the customer experience not covered by the closed-ended questions.
- For the company, the question can be used on an operational level but only in addition to the closed-ended questions, which are more accurate in pinpointing the strengths and weaknesses in your organisation. Once you have identified the areas on which you want to work, customer comments can be used to more effectively define and prioritise the necessary corrective actions. An open-ended question can also be used to identify customer complaints more accurately.
- On a more strategic level, an open-ended question also gives the company new ideas (to solve the problems identified or regarding key concepts). This is why you should ensure that you read as many comments as possible. They contain raw information about your customers’ experiences: there’s not a company in the world which should ignore these.
ABOUT THE AUTHOR
Do you need more information ? Contact the author : firstname.lastname@example.org