Sentiment analysis is a data analytics process you can use to improve the customer experience by tapping into the emotional aspects of consumer decision making.
Are your net promoter scores (NPS) stagnating or declining? Are you losing more customers than you are gaining? When you are experiencing these or other customer behaviors that are causing a negative impact on your business, the go-to solution is sentiment analysis.
Sentiment analysis is a cornerstone offering of artificial intelligence and machine learning (AIML) disciplines. Humans are highly emotional beings, and emotions impact everything we do and provide the basis for future actions. Sentiment analysis allows software companies and corporations to put their finger on the pulse of customers’ feelings based on past or current interactions, conversations and observable behaviors. Companies are capitalizing on leveraging sentiment analysis tools to understand customer behavior and emotional states better. For example, sentiments are good indicators for future customer engagement, churn and share of a customer’s wallet.
Sentiment analysis typically leverages textual data, be it emails, chats, social media posts or survey responses. It uses natural language processing (NLP), natural language understanding (NLU), computational linguistics and text analytics to infer positive or negative attitudes: Do consumers say good or bad things about your brand and your products or service?
Companies have successfully used sentiment analysis to manage brand and reputation, recommender systems, content-based filtering, semantic search and understating users’ or consumers’ opinions and needs to inform product design, triaging customer complaints and influencing customer buying behavior.
Let’s take a closer look at sentiment analysis and why you need to use it.
What is Sentiment Analysis?
Typical sentiment analysis tools offer an aggregation of polarity scores across various parameters. For example, you can determine the total number of spoken sentences with positive, negative and neutral reception and descriptive analytics dashboards on top of these indicators based on geographies and product segments. Based on the sign of the polarity score, you can often infer overall sentiment as positive, neutral or negative.
These polarity scores range from -1 to 1, where -1 indicates the sentiments of the speaker toward the subject is highly negative, to +1, which indicates the sentiments of the speaker toward the subject are highly positive. This binary scale used among many popular sentiment analysis tools leaves us with a desire for a more granular scale to capture customer sentiments.
Here is an example graph of how things could look.
Looking broadly at how companies currently perform sentiment analysis, the volume of data available and produced, along with the performance gaps associated with typical sentiment analysis tools, businesses need a more comprehensive approach and scales for effective analysis.
Over the course of our careers, we’ve explored many of the popular sentiment analysis tools and platforms to find the best fit solution. Our endeavors consistently landed us with a negative sentiment polarity – this gave us the insights we need to build a better sentiment analysis process.
Sentiment Analysis in Practice
A normal call center interaction, for instance, is several minutes in length and normally involves a customer experiencing a problem. This means the customer is somewhere between frustrated, angry and tense at the start of the interaction. Let us take an example of a conversation between a customer and a telecom operator regarding a billing issue where the first part of the conversation happened between the customer and the telecom call center operator, and the customer’s problem remained unsolved. The caller then got transferred to a second-level support agent who was also unable to resolve the issue. The customer disconnected the call, dropped the service and moved to another provider.
The call center uses one of the many sentiment analysis tools and platforms available on the market. Sentiment analysis tools generally offer a polarity score for an individual sentence in a long paragraph which tells us that X percent of a total number of positive sentences were positive sentences, Y percent of the sentences were neutral, and Z percent were negative. They do not tell us things like how the conversation escalated and how the customer’s level of sentiment and emotions changed during the call leading to a lost client.
Current out-of-the-box tools were unable to provide the call center with steady changes in the sentiments and customer emotion levels. But, these tools do offer dashboards that do descriptive analytics on the results. One common dashboard analyses arousal and valence. Arousal (or intensity) is the level of autonomic activation that an event creates and ranges from calm (or low) to excited (or high). Valence, on the other hand, is the level of pleasantness that an event generates and is defined along a continuum from negative to positive.
If we look into the arousal versus valence quadrant analysis chart below, we will see that this interaction likely ended in either quadrant II or III – neither of which are preferred outcomes:
Using Sentiment Analysis to Discover Pain Points
Let’s set the above findings aside for the moment and try to answer a few basic questions:
- What are the pain points in the conversation?
- What was the subject of each pain point?
- When did the conversation start to degrade?
- When did you detect the customer’s frustration?
- How many positive (good or happy) and negative (sad or unsatisfactory) moments were there during the interaction?
Although, during an interaction you may not get the answers to all of these questions. What could have helped are potential answers to the following questions:
- What does the sentiment gradation curve look like?
- Can you find the pain point or points of the customer?
- What was the reason for the call?
- What was the focus — product or service?
- How was the sentiment flow within the discourse level (one discourse is one conversation exchange between two parties)?
- How was the sentiment flow for each party?
- How was the sentiment flow between the discourses?
- Were there any sudden bumps in the sentiment during the conversation? If yes, what caused these bumps?
- How should you rate the entire sentiment experience with the customer?
- Does ending the conversation on a high note provide sufficient sentiment feedback to rate the entire experience as positive?
- If you employed a chatbot in the system before transferring an incoming call to a customer care agent, what was the customer’s sentiment before, during and after the transfer?
- Is there a built-in escalation process for when a customer gets frustrated?
- Can the system provide immediate and effective prompts to manage the conversation by prompting the agent?
In order to answer these questions, we need high-performing semantic analysis tools with broad capabilities and applications across various business functions and industries. We need a platform that will allow for a better and customized understanding of each individual business domain and sub-domains based on industry needs, such as in banking, insurance, utilities, healthcare and so on.
You may ask why you need answers to all these questions.
The reason is quite simple and logical. If you want to improve your system’s performance and accuracy by leveraging the sentiment analysis irrespective of the nature of the business. Every business has customers, and a best-in-class customer support environment helps the business stay healthy. Each business has its own lexicon, and a generalized NLU engine is insufficient to understand the jargon to overcome industry-specific word sense disambiguation.
What this means is that in order to provide the highest level of sentiment analysis results during an interaction, we need to provide a best-in-class customer support environment. Unfortunately, after looking far and wide for software with desired characteristics and capabilities, like we described above, we did not find a single one that met at least 50 percent of our stated requirements.
In addition, the majority of these tools only support point solutions and lack a mass application across various industries and business functions. For broad appeal, this software needs to have a wide understanding and vocabulary across business functions and industries. For example:
- Domain-specific understanding (Insurance, Banking, Healthcare, Life Sciences, and so on)
- Domain-specific lexicons
- Domain-specific word models
- Sentiment-polarity analyzer (deep learning or ML model)
- Sentiment-subjectivity analyzer
- Discourse-level sentiment aggregator logic
- Overall sentiment aggregator logic.
It is not uncommon for a typical organization to spend a lot of time and effort to achieve desired results. For example, before joining Centric, we had a client build a sentiment analysis solution from scratch to address business needs. The client then had to invest roughly eight months of time for a beta release where several full-time employees were involved (these included Data Scientists, Data Engineers, Machine Learning Engineers, Software Architects, Business Analysts and others). But after this rigorous eight-month exercise, the client’s net promoter score, customer sentiment charts and customer retention ratios gave strong indications that they are getting better outcomes to support their business needs. The time and financial investments paid off.
Why You Need a Sentiment Analysis Partner
Getting started with sentiment analysis can be a daunting undertaking. While the theory is straightforward, making sure you get the details right requires expert experience. We recommend you partner with a team of professionals who can lead you through a sentiment analysis process best suited to your company – a team that will pave a path for your future success.
When looking for a partner, you want to find someone who focuses on transparency, quality and collaboration. A strong partner will be able to set up the tools and processes needed and then help you read the data to better understand your customer’s pain points and also what makes them happy. They will also help you determine what to do with that data once you have it, so you can move forward with improving your customer’s experience.
Conclusion
As you can see, sentiment analysis is far more complex than it may seem at first, and there are many things to consider to provide effective and high-quality solutions and results. Partnering with an expert in sentiment analysis can help you determine what questions to ask, what to do with the results, and possibly, how to build a custom solution for your company.