Mobile first and review sites: the impact on rating scales

The choice of a satisfaction scale in a questionnaire should not be made lightly. This is the scale that the indicators will use as a basis for calculation. For example, if you use the "Yes" and "No" response modes for the recommendation question, you won't be able to calculate an NPS.

Here, I look at commonly accepted rules and new satisfaction scales linked to the emergence of Smartphones and review sites. The standard approach to constructing a survey, and moreover, a mobile or Smartphone survey, needs to be rethought.

Focus on consistency

When you choose a satisfaction scale, it should be used for all your questions. You should avoid switching from a numerical to a textual scale during the course of a questionnaire, as this can disorient respondents and therefore impact the reliability of the answers obtained.

Numerical or textual satisfaction scale

We tested the 2 scaling methods: numerical or textual. Personally, I prefer the numerical scale in the case of multilingual surveys. In other cases, I prefer the Likert scale. It ensures that the respondent makes no mistakes when filling in the questionnaire.

For the undecided, there's also the double scale: numerical at question level, with a text legend at the bottom of the page. However, this makes the page considerably heavier, and means that there are fewer questions per page, so that the legend is always visible.

4 modalities or more?

The Likert scale (Very satisfied/Satisfied/Unsatisfied/Not at all satisfied) has 4 modalities. In many of our surveys, and even in most mobile surveys, we use a scale from 1 to 10. A low number of modalities makes it easier for operational staff to use. For analysts, however, a larger scale enables changes in responses to be detected more quickly.

In all cases, we recommend avoiding an odd number of response modes. So-called "refuge values" should be avoided. When it comes to satisfaction, customers are either happy or unhappy. They cannot be neutral. That's because it doesn't call for any action.

Left to right or right to left: it doesn't matter

Several studies have shown that the direction of the satisfaction scale is of little importance. If your scale includes an 'unconcerned', I generally recommend putting it on the right: this way, it will be read last by the respondent. And so, logically, I recommend a descending scale (from left to right), so that the 'not concerned' is close to the lowest satisfaction score.

Smartphones and review sites: the big shake-up on the satisfaction scale

All the advice given above seemed to hold true from the very beginning of paper-based customer satisfaction surveys. Even the arrival of the Internet had little effect on the application of the scales. More recently, however, the arrival of the mobile Internet and review sites seems to have had a lasting impact on the way mobile surveys are managed.

First of all, the mobile Internet: many websites now have a "Mobile First" approach. The rendering is first defined for mobiles, then adapted for desktop computers. This shows just how important rendering on smartphones is. Obviously, the 10-modality scale (11 if you include the one not concerned) seems to pose a problem here, given the limited width of the screen. Even the text scale, in some cases, can become difficult to read.

Certain graphic objects are of course available to allow you to maintain a consistent numerical or textual scale: animation, pop-ups... However, the trend is clearly towards scales more suited to a small display format.

Then there are review sites like TripAdvisor. The latter have democratized 5-star scales. An odd number of ratings is not recommended, however, as it provides a safe haven with 3, allowing the respondent to lean neither positive nor negative. But it does seem to work, and quite frankly, this model is more self-imposed than self-analyzed.

By now, all your customers are used to this satisfaction rating system. It feels natural to them. What's more, it's adapted to your mobile and tablet surveys.

Conclusion

Despite all the recommendations given in the first part of this article, we have to admit that star ratings are gradually taking hold - or rather, our customers are. It's a lightweight, mobile-ready and modern format for your mobile surveys. We therefore recommend this format from now on, even if a case-by-case study is still necessary.