Verbatim Analysis Software – Easily Analyze Open-Ended Questions

Do you want to stay close to your customers?  Of course you do!  Improving the customer experience is central to any business.  More broadly, any organization needs to stay in touch with its constituents, be they customers, employees, clients, or voters.

In today’s world, getting your hands on the voice of the customer is not difficult.  Most companies have more comments from their customers than they know what to do with.  These can come from internal sources, such as

  • E-mails to your support desk
  • Transcriptions from your call center
  • Messages to your customer service staff
  • Notes in your CRM system
  • Comments from web site visitors

For most companies, the problem is not a lack of voice of the customer feedback.  It is the sheer volume of it.  You might well find that you have thousands or even hundreds of thousands of comments from your customers.  How can you make sense of it?

Traditional Survey Research

Traditional survey market research can give us some clues.  Market research firms have been using surveys for years to guide their clients to improve customer experience.  They have developed some very powerful techniques to analyze comments from customers and turn them into actionable insights.  Let’s take a look at how they do it to see whether we can come up with some ways to take our voice of the customer data and find insights to improve our company.

How Survey Market Researchers Deal with Open-Ends

A survey has two fundamental question types: closed-end and open-end.  A closed-end question is one where we know the possible set of answers in advance, such as your gender or the state you live in.  When a respondent provides a free form text response to a question it is an open-end: we don’t know just what the respondent is going to say.  That’s the beauty of open-ends.  They let our customers tell us things we did not know to ask about.

Survey researchers call the open-end responses verbatims.  To analyze the responses the researcher has them coded.  Humans trained in verbatim coding read the responses and invent a set of tags or codes that capture the important concepts from the verbatims.  In a survey about a new juice beverage the codes might be something like

  • Likes
    • Likes flavor
    • Likes color
    • Likes package
  • Dislikes
    • Too sweet
    • Bad color
    • Hard to open

Human coders read each verbatim and tag it with the appropriate codes.  When this is completed, we can pose questions like

  • What percentage of the respondents like the flavor?
  • What is the proportion of overall likes to dislikes?

And if we know other things about the respondents, such as their gender, we can ask

  • Do a greater percentage of men or women like the flavor?

The researcher would say that we have turned the qualitative information (verbatims) into quantitative information (codes).

How Can Verbatim Coding Work for Me?

The technique of verbatim coding can certainly be used to analyze the comments from our customers, but it has its drawbacks.  Verbatim coding is expensive and time consuming.  It typically costs about 15 to 45 cents per comment.

The Ascribe Intelligence platform is used by the leading market research companies around the globe to code verbatims.  While worker productivity using Ascribe is far higher than when using more manual approaches, achieving coding rates above 400 comments per hour is rare.  Verbatim coding is appropriate when we have high quality data and we want the best possible analysis accuracy.

The customer comments we have collected internally normally do not warrant the cost of verbatim coding.  On the other hand, if we have conducted our own surveys, we can certainly use verbatim coding effectively for these open-ends.  We can either

  • Have the verbatims coded by a company specializing in the technique, or
  • License software such as Ascribe to let us code the verbatims ourselves.

Returning attention to our pile of customer comments, we can see that it would be great to get the benefits of verbatim coding without the cost.  It would be great if we could have the computer “just do it”, without human coders involved.  That’s where text analytics comes in.

Using Text Analytics for Verbatim Analysis

Working with customer comments, text analytics can tell us several things

  • What are my customers talking about? These are the topics.
  • What are they saying about each topic? These are expressions about the topics.
  • How do they feel about these things? These are sentiment ratings.

If we are running a gym, we might get this comment:

  • I really like the workout equipment, but the showers are dirty.

Text analytics would yield this result:

Text Analytics to Summarize Sentiment

If we run a thousand or so comments about our gym through text analytics, we might get results like this example from the Ascribe CX Snapshot product:

Now we are getting somewhere!  We see the things our members are talking about most frequently.  Each topic has an associated frequency (the number of times the topic came up), and sentiment rating.  The sentiment rating for this analysis ranges from -2 as strong negative to +2 as strong positive.  The sentiment scores are the averages for each topic.  We can see immediately that our members are generally positive about our club, especially the staff.  But we seem to have a problem with the lockers and with our workout equipment.

Drilling into the Data

A good text analytics tool will present much more information than the simple table above.  It will let us drill into the data, so we can see exactly what our members are saying about the topics.  It should also let us use other information about our members.  For example, if we know the gender of each member, we can look at how the women’s opinion of the showers differs from the men.

A text analytics tool can also guide you to remedial action to improve your sentiment ratings.  For example, the Ascribe CX Inspector product can produce a report like this:

This captures a required action plan very clearly in a single graphic!

Text Analytics Compared to Verbatim Coding

Thinking about verbatim coding as done in survey research, you can see that the computer did two things

  • It invented a set of categories, in this case the topics
  • It classified the responses into the categories

Therefore, the computer did the same two things as humans doing verbatim coding.  It invented a set of “codes” which are the topics, and it classified the responses into these codes.  While the computer did something similar to verbatim coding, it is not really the same.  A big difference is that in verbatim coding the humans invent the categories (codes), while in text analytics the categories (topics) are created by the computer.  Looking at our table of topics above, we may well want to say that “machine” and “equipment” are really the same thing.  That is where a taxonomy comes in.  Taxonomies are used to group things together, or in our example the taxonomy could say that “machine” and “equipment” should all be grouped into “workout equipment” in our table.  Using a taxonomy constructed for your specific business can produce text analytics results that are very similar to verbatim coding.

Conclusion

Verbatim coding is a very well-developed part of traditional survey research.  It gives very high accuracy but can be expensive and time consuming.  Text analytics can provide similar insights at a fraction of the time and cost.

The Ascribe Intelligence platform provides both traditional verbatim coding and advanced text analytics.  Verbatim comments in Ascribe can be coded using Ascribe Coder, or run through text analytics using CX Snapshot or CX Inspector.  This allows you to choose the best approach for your needs on a given project, even allowing you to use a combination of these techniques when a particular project demands it.

 

Leave a Reply

Your email address will not be published. Required fields are marked *