The survey research industry has elevated the art of gleaning actionable insight from customer comments to a science. These techniques are valuable to any company that wants to have an accurate understanding of what their customers are telling them – and what company does not?
To understand how you can use these techniques to keep your finger on the pulse of your customers, let us first introduce some concepts and terminology.
What is Coding?
Open and Closed End Questions
Think about the surveys you have seen. There are two basic types of questions, such as these:
- On a scale of 0 to 10, how likely are you to recommend our business to a friend or colleague?
- Why did you give us that rating?
The first of these is called a closed-end question because the possible answers are known in advance. The second is called an open-end question. There are an infinite number of possible responses to the question.
Perhaps surprisingly, we do not call the text that the respondent types for an open-end question an “answer”. Instead, we call it a verbatim. Yes, I know that “verbatim” is not a noun, but it has been adopted as one in the survey research industry. Why do we not call the text an answer? Well, because we cannot do much with the text alone.
Imagine you have 5,000 responses to the short survey above and your boss asks you: What percentage of our customers rated us below 8 because of a product delivery problem? Perhaps you would read each of the verbatims from the second question and make notes on a piece of paper, then tally up the results. If you are more methodical you might put the verbatims in Excel and make a column for “Delivery Problem” and put a 1 in the column if the verbatim mentions a delivery problem. As you do this, you would probably add other columns for other issues mentioned, perhaps “Product Quality” and “Technical Support”. That way when your boss asks you tomorrow for how many customers gave us a negative rating for technical support, you could have the answer instantly, just by summing the Technical Support column.
In survey research we call what you just did in Excel Coding. The columns such as Delivery Problem and Technical Support are called Codes, but you can see that in a certain sense we could also call them Answers. When you put a 1 in the Technical Support column, you mean that this customer mentioned technical support as an issue.
By coding the verbatims as you have done in Excel you have turned the qualitative data (the verbatims) into quantitative data (the codes).
Sources of Verbatims
In the example above we talked about a simple two question survey. You may have recognized this as a classic Net Promoter Score (NPS®) survey. This is a good place to start when learning about coding, because the technique of verbatim coding was developed and refined in the survey research industry.
But traditional surveys are certainly not the only source of verbatims for coding. Verbatim coding is applicable whenever you want to get actionable insights from comments. This includes such sources as:
- NPS surveys
- Employee satisfaction surveys
- Help desk inquiries
- Call center transcriptions
- CRM systems
Verbatim coding is appropriate whenever you need to quantify comments from constituents.
Reasons for Coding
If we think of coding as tagging comments with codes that attribute meaning to the comment, we see that there are two primary reasons for coding.
Coding turns qualitative data into quantitative data. Once our NPS survey is coded we can easily ask questions such as:
- What percentage of net detractors mentioned product delivery?
- How does the number of net detractors mentioning product delivery in Q1 compare with Q2?
- How does the number of net detractors mentioning product delivery in the Central Region compare with the Eastern Region?
Note that in the second and third example above we assume that we have some additional information. In the second example we assume we know the time the comment was given. This comes along for free; we just need to make a record of the date of receipt of the comment. The third example assumes we know the region the customer is in. We could of course get this information by asking a closed end Region question in the NPS survey. But hopefully we can get this information, and much more, by using our CRM system to augment the data with known information about the customer.
Quantification and analysis of verbatims in this fashion is the bread and butter of the survey research industry. But these techniques are equally applicable for in-house analysis – and far less costly than engaging a research company.
Classification and Indexing
Even after they are coded, verbatims remain a very rich source of insight into your customers thoughts and attitudes. With coded verbatims you can get your hands on the verbatims most applicable for a business question you are considering. For example, a product manager with a delivery problem in the Central Division might, and probably should, read a selection of verbatims from that region that mention delivery. Once coded, finding these verbatims is trivial.
Classification of verbatims by coding allows product managers and others to keep their fingers on the pulse of the customer using the rich texture of verbatim comments, targeted at specific areas of interest by coding.
We started by looking at coding verbatims using Excel. That is not a contrived example. There are small survey research companies that do just that. But there are far more convenient and productive techniques than using Excel.
In the survey research industry verbatim coding is a profession. People trained in coding read and code every verbatim in open end survey questions. The industry uses Ascribe Coder to code over 200 million verbatims each year.
Human coding produces the highest accuracy and allows for nuanced differences in meaning between codes. The construction of a well-designed code frame, which is the set of codes for a question, is part of the art of human coding. Human coders have full control over the form of the code frame and can deliver results tailored to a specific survey research objective.
The accuracy and control afforded by human coding comes at an associated cost of labor and turn-around time.