You hear the term text analytics a lot these days. It seems to be used in so many different ways it is hard to pin down just what it means. I’ll try to clear it up a bit.
The basic idea is that a computer looks at some text to accomplish some goal or provide some service. Just what those goals and services might be we will get to in a moment. First note that text analytics works with text and analyzes it in some way. That’s the analytics part. When I say text, I mean electronic text, like this blog page or some comments from your customers stored in a spreadsheet. Text analytics does not work with voice recordings, videos, or pictures.
Because text analytics examines text written in a natural language (like English or French), it uses techniques from the sciences of Natural Language Processing (NLP) and Linguistic Analysis. The computer techniques used include machine learning, word databases (like a dictionary in a database), taxonomies, part of speech tagging, sentence parsing, and so on. You don’t need to understand these techniques to understand what text analytics does, or how to use it.
Just what can we use text analytics for? Some of the more common uses are:
- Document classification and indexing – Given a bunch of documents, the computer can figure out the key concepts and let you search a library of documents by these. A sophisticated example of this is E-Discovery used in legal practice, which seeks to use the computer to assist in discovery in legal proceedings.
- National security – Governments use computers to monitor web postings for information or discussions of interest to national security.
- Ad serving – We are all experienced with the uncanny ability of some web sites to show us ads that are relevant to our needs and interests. Text analytics applied to the pages we are viewing is a big part of this magic.
- Business intelligence – For most of us this is the big one! We can the computer to give us insights into how to improve our service, retain customers, and give us a competitive advantage.
Text analytics for business intelligence is a rapidly growing market. Using the right tool, you can analyze thousands of customer comments in minutes. If the tool does a good job of presenting the results it is amazing how quickly you can figure out what they care about, their pain points and plaudits, your weaknesses and strengths.
Choosing a Text Analytics Tool
How do you find the right software product for your needs? Well, there are many providers of raw linguistic analysis capabilities. Google, Microsoft, Amazon, SAP, IBM, and many others provide such services via an API. But this takes an engineering effort on your part, and you still need to figure out how to navigate the results of the analysis.
There are several vendors of complete text analytics packages for customer experience management. As you evaluate these consider:
- Does the vendor specialize in customer experience feedback?
- Are the results of the analysis clear and insightful?
- Is text analytics a core competency or a side product?
- Are multiple languages supported?
- Is automatic translation between languages supported?
- How easy is it to tailor the text analytics to your specific business?
Navigating the Linguistic Analysis Results
Suppose you have a database of feedback from your customers. If you run 10,000 customer comments through a linguistic analysis engine it will produce a mountain of data. To gain insight from this data it needs to be organized and navigable. A good text analytics tool will organize the results and display them in a manner that helps you to find the actionable insights in the comments. Optimally the tool will help you pinpoint the strengths and weaknesses of your company from the customer’s viewpoint.
Reducing the Raw Linguistic Analysis Data
Let’s look at a specific example to gain an understanding of what is involved in organizing and presenting the results of linguistic analysis. As I described in Linguistic Analysis Explained, sentence parsing is one tool in the NLP analytics bag of tricks. Google has a very well regarded sentence parser, and it is available via an API. You can try it out at https://cloud.google.com/natural-language/. It’s a fun way to get some insight into the complexities of presenting the results of linguistic analysis. Try running this comment through the API:
The support staff are helpful, but slow to respond.
Now and take a look at the Syntax results. These are the results of the sentence parser. You find:
Wow, that’s a lot of data, but hardly actionable insight! You can see that the sentence has been broken into tokens (words and punctuation). Each token has been tagged with its part of speech (slow is an adjective). Each token has also been assigned a parse label, indicating how it is used in the sentence (slow is used as a conjunction). The green arrows show how the token are interrelated. Imagine how much data would be generated by running 10,000 customer comments through the sentence parser!
The job of text analytics is to distill this pile of data down. In this case the analysis might go something like this:
- “Support staff” is the nominal subject of the root verb. That’s a topic the customer has mentioned.
- “Helpful” is an adjectival complement (acomp) of the subject. The customer has said the staff are helpful.
- “Support staff” is further described by the customer by the coordinating conjunction (cc) “but”, as “slow” (conj).
So we find that the support staff are both helpful and slow. We have extracted a topic of “support staff”, with expressions of “helpful” and “slow” associated with it. This reduction of the raw data from linguistic analysis has resulted in what we are interested in knowing. Our customer thinks the support staff is helpful, but also slow! This is a single finding from linguistic analysis.
Presentation of the Reduced Data
Now that we have extracted the desired nuggets from the raw linguistic analysis data we need to present it in a way that helps you find the insights you seek. An actual analysis of 10,000 customer comments may well produce 50,000 findings. To navigate these the findings need to be organized in a way that emphasises the important insights, and allows you to explore the findings quickly and intuitively. A good text analytics tools will assist you in ways such as:
- Grouping similar topics, preferably automatically or with a custom taxonomy you control.
- Ranking topics by frequency of occurrence or other metrics such as sentiment.
- Allowing you to filter the results in various ways, such as by demographic data.
- Showing trends across time or other variables.
Summary
Text analytics for customer experience management reduces and organizes the results of linguistic analysis. If done well, the results are presented to you, the user, such that you can find actionable insights quickly, and explore the data to help formulate an action plan for improvement.