How to leverage text analytics for consumer intelligence?

Every day, your company receives structured and unstructured text. It can come from social media, call center transcripts, surveys, phone calls, customer reviews, emails, compliance forms, customer contract agreements or customer location data, etc. It’s raw text that you’ll only be able to interpret accurately, if you use text analytics tools.

Text analysis software collects unstructured data and identifies patterns and trends. It counts and analyzes the relationship between words - categorizes, tags, annotations, visualizes, and predicts. It finds keywords, topics, and semantics. It mines text for sentiment analysis. It gives you consumer intelligence.

Talkwalker AI-powered sentiment analysis of three leading fizzy drink brands.

You could employ a team to do text analysis manually, but it’s a nightmare of a job. There’s just too much text to go through, and there’s always going to be an element of human error.

I’m going to explain text analytics, and share the best text analytics tools on the market. There are easy-to-use ones, and some that are more complicated. I’ve also included paid and free text analytics tools. If you're struggling with any of the technical terms, I've included a glossary of text analytics terms.

Table of contents

• What is text analytics use for?
• Why you need text analytics
• What are the sources for text analytics?
• How do businesses use text analytics?
• Text analytics examples
• Text analytics glossary
• 17 Text analytics tools
• Free consumer intelligence dashboard template

---

What is text analytics used for?

Text analytics definition - text analytics - text mining - is a discipline of computer science. It uses machine learning and natural language processing to extract meaning from unstructured text. It’s how a business can convert 10,000 product reviews into recommendations and user-generated content. How HR can improve employee satisfaction and productivity, and reduce churn.

Text analytics are powered by natural language processing and statistical algorithms. Using sentiment analysis, text classification, word analysis, they’ll help your team analyze customer surveys, vendor correspondence, call center interactions, legal documents, industry reports, social media, and more.

Looking at it from a marketing point of view, text analytics finds meaning in written content. Content about your brand, your products, your customers, your competitors. It will reveal patterns and trending topics that you’ll then be able to take action on.

Perform text-based analytics manually, and you’ll be buried in text. Text analytics software uses text mining and natural language processing algorithms to find meaning in vast amounts of text.

Why you need text analytics

Text analytics - text mining, intelligent content analytics - will help your business make data-driven decisions. Understanding the sentiment, subject, and intent of text will lead to improved product development, strategic decision making, content marketing, competitor intelligence, social media messaging, employee satisfaction, and online brand reputation.

Analyzing customer feedback - emails, social, online reviews, call center notes, survey results - will show you which of your products they’re talking about. If they’re happy with the price, usability, and if they’re experiencing issues. How they feel about your competitors, and what your competitors think about you. Employing sentiment analysis will identify how customers really feel. Essential customer insights.

With this knowledge at hand, you can improve your communication and marketing strategies, drive your customer service to excellence, and become a more customer-centric brand.

Text analytics tools can also warn you of trouble on the horizon, because you’ll discover what consumers are complaining about. Issues they’re experiencing.

The use of text analysis tools has increased, along with the rise of big data. There’s just way too much text to analyze manually. Automating the process means you’ll find patterns and trends that may have been missed in the past.

Your unstructured text will be converted into structured data that will enable you to visualize trends, understand the sentiment behind consumer opinions, and measure the engagement your marketing campaigns receive. Vital insights that will help you make strategic business decisions.

What are the sources for text analysis?

Think you’re collecting all your data? Think again.

Publicly and privately, there are several sources which produce text that can be organized. This is structured data, which isn’t so hard to analyze. But, there’s a ton of unstructured data that you’re probably missing. This is why you need text analytics tools, because this data is valuable to your business.

Structured data

The data that you’re - I seriously hope - collecting, includes call logs to your customer support team, emails, responses to feedback, compliance forms, contracts, customer locations. It also comes from market research and surveys.

While these are structured data sources and easier to collect, they’re held on various databases that may not be able to interact.

Unstructured data

What is unstructured data, a.k.a. dark data?

It’s the information that businesses collect, process, and store during business activities. But - and it’s a biggie - they don’t use it for anything.

Unstructured data can be found in a variety of sources, including unsolicited feedback on review sites, sales data, instant messages, blog posts, email, scanned documents, images, app stores, dates, numbers, contact lists…

Analyzing your unstructured data can be tricky, because some of us - no names - will use a fake name or quirky social media handle, when leaving comments. Also, we may use slang, sarcasm, local dialects, etc. Categorization becomes difficult. And, being unstructured, we could be talking about a tweet or a 2500 word blog post.

Why is unstructured data important?

Unstructured data - dark data - is increasing by 62% per year. By 2022, the International Data Group - IDG - says that 93% of all data will be unstructured.

Wow! That’s some data.

This text-heavy data contains information that will help your business make data-driven decisions. But, because it’s coming from humans rather than data stored in fielded form in databases, or tagged in documents, they’ll be irregularities in content, ambiguities due to language, etc. This means that some text analytics programs will struggle to understand it. But, the fact that it is written by humans, means it’s way more valuable.

Yes, sorting this unstructured data is a slow process, which is why a lot of companies don’t bother, and lose vital insights.

How do businesses use text analytics?

Text analytics is how to achieve data-driven decision making. Capturing text sources and converting to data, reveals insights that will feed product development, business intelligence, competitor analytics, marketing optimization, and more.

Gathering all this valuable data will also support...

• Content management - create rich metadata to make it easier to find and add meaning to key names, places, dates, ideas, etc.
• Semantic search - understanding natural language - the way humans communicate - based on meaning and context
• Regulatory compliance - to find, highlight, and extract key data within regulatory documents
• Content recommendations - increased user engagement via content recommendations

What you have to monitor...

Customer feedback

“The Net Promoter Score® (NPS) is used to quantify your customer loyalty. Its use is on the rise, with two thirds of Fortune 1000 companies using the metric.”
Dan - Why your Net Promoter Score® needs integrated customer data

An NPS survey to measure customer satisfaction usually asks a single question. But it’s a biggie. How likely are you to recommend our product?

Because there’s no ambiguity, NPS surveys are popular with brands looking to understand how customers regard their product.

Your business, of course, will have processes in place for collecting feedback. For instance, your sales team will do follow up calls to consumers, or send an email after your product has been purchased. This will result in large amounts of text containing valuable data to be collected and analyzed.

Feedback data which can’t be gathered with a NPS survey, can be found on review sites, social media, forums, live chats, etc. This feedback is unsolicited, it’s been volunteered. Amazon is a good example of a brand that relies on reviews.

Amazon - convert reviews into recommendations with text analytics tools.

Text analysis is an efficient way to help you understand what consumers really think. Rather than your team trolling through pages and pages of text.

Social media channels

Thanks to social media, we can now interact with brands directly. If we have a complaint, we post on social. If we’re recommending a product to our friends, we share on social. Every day, we’re sharing billions of tweets, images, snaps, videos, blogs, stories, and more.

There’s so much conversational data to be collected…

• What are consumers ranting about?
• What are consumers raving about?
• What are they saying about your competitors?
• How are they responding to your marketing messages?

It must be monitored, but which social media data sources to track, and how? Dan offers advice on the 11 crucial social media data sources for brand monitoring. I’ll also share the best social listening tools for the job.

You’re gonna need tools. Lots of tools.

Brand mentions

Do you know what consumers are saying about your brand and products? Is your reputation good or bad?

Monitoring your brand mentions determines your standing in your industry. What consumers think of you. You’ll be able to find potential issues, and respond quickly. Find the trends people are jumping on, content inspiration, and user-generated content.

90% of consumers say that user-generated content influences their buying behavior. While 81% do online research before spending their money

We trust what other consumers are saying. More than the marketing messages coming from brands. Don’t sell to us. Engage with us.

While a lot of this consumer data comes via social media, there is an abundance to be collected from other sources, such as forums, news sites, TV and radio, review sites, etc.

Knowing what people think of your brand is crucial. Equally important is knowing who and in what context. The best sentiment analysis tool will be able to identify the tone being used by consumers. It will recognize and understand if a comment is sarcastic.

Positive or negative sentiment?

Talkwalker’s sentiment analysis feature monitors in real-time, across multiple channels and languages. It understands the human language, with an average 90% accuracy.

Sentiment analysis tool with 90% accuracy!

Competitor analysis

What do consumers think of your competitors? What do your competitors think of your brand? Why are consumers choosing your competitor's brand, rather than yours? Which channels are your competitors using to engage with consumers?

That’s a lot of questions to answer. How?

Use text analytics in your competitor analysis strategy to improve your communication, customer service, and your marketing strategy.

As a marketer, you’re already tracking consumer behavior. The decision process, influences, and actions that a person takes when buying your product.

You are, aren’t you?

If you don’t understand consumer purchasing habits, you’ll struggle to make long-term decisions or understand the trends that are influencing consumers. You’ll find it tough to win sales.

The data is there, waiting for you to collect and analyze. Fire up those text analytics tools!

Product development

When you launch a new product, you need to know if the launch is working, and how the product is being received. Quickly.

Text analysis allows you to find and analyze feedback by topic and sentiment. Feedback shared in surveys, on social media, forums, or inbound calls to your support team.

Is your product messaging successful? What do consumers think of your product? Are there features you could improve or add?

Answer these questions and you can optimize future product launches and your products.

That’s a biggie.

Customer service

Flooded with inbound messages, text analysis can automatically categorize by topic, time received, priority, etc. With your customer services communication optimized, you’ll be able to escalate urgent messages, and forward to the correct team.

Team analytics

Your HR team is going to love you.

Text analytics can be used to measure the satisfaction levels of your teams. HR can then analyse and identify possible issues that are affecting the performance of your business.

The data can be taken from team surveys, employee reviews, etc. Jump on those problems and address them before they go company-wide.

Text analytics examples

As we are deluged by unstructured data, text analysis has increased. Using text analytics tools, unstructured data can be made accessible and beneficial, delivering TOI from unstructured data management.

As a brand, you'll be able to find consumer insights, patterns and trends. Check out the text analysis examples and learn how companies are using this technology...

Customer care

Used to improve the customer experience, data is analyzed from surveys, customer service tickets, call center transcripts, and calls. This enables your customer care team to improve their operations - quality, time to respond, time to resolve, etc.

Customer experience

An e-commerce business providing a poor customer experience should keep their bags packed. Consumers faced with a bad CX are likely to bounce to your competitor. Text analytics will track and analyze feedback so you can find

Marketing

Use text analytics tools to target your marketing. Analyze audience segments and learn their interests, buying behavior, location, job role, language, etc. With this information you'll be able to target buyer personas and deliver content that's personalized to their wants and needs.

Social media analysis

Social media channels provide an abundance of unstructured data, providing consumer intelligence and market trends. Brands using text analytics can analyze and predict consumer demands, while learning the sentiment towards their products.

Understanding customer reviews & feedback

You've just launched a new product. How do consumers feel about it? You're in the hospitality industry and you need to find areas to improve. You can use text analytics to understand what customers are saying about you in reviews.

You can monitor social media, collect online reviews, send out a survey. Then what? Your team manually goes through all this data, looking for patterns and trends?

Yikes!

AI-powered text analytics tools can analyze this qualitative feedback to find patterns and then allocate quantitative measurements, so you can compare.

Listen to the voice of the customer.

Risk management

In the finance industry, text analytics is employed to evaluate companies and decide whether or not to lend them money.

Information in newspapers, social media, analyst reports, financial records are analyzed to determine the risk factor of a potential recipient of a loan.

Compliance issues can also be detected by using text analysis to search legal documents for keywords such as fraud, risk, finance, etc.

Text analytics glossary

I don't want to teach my grandma to suck eggs, but if you're stuck on any of the terms I've used in my post, this glossary should help...

Text mining

Text mining, also known as text data mining or text analytics, processes unstructured information, identifying patterns and trends. Text mining and analytics turn untapped, unstructured data sources from words to actions.

Structured data

Structured data is data that lives in a formatted field within a record or file, such as a database. The data can be easily processed and analyzed.

Unstructured data

Unstructured data refers to information that's not stored in a database. It includes text and multimedia content such as email, videos, images, documents, audio files, webpages, blog posts, social media messages, etc. These are unstructured because they can't automatically populate a database.

Sentiment analysis

Sentiment analysis - opinion mining - uses AI technology to analyze people's opinions to determine whether a piece of text is positive, negative, or neutral. A sentiment analysis tool combines NLP and machine learning.

Natural language processing - NLP

Natural Language Processing helps machines read text, by copying humans’ ability to understand languages such as English, Russian, Japanese, etc. It includes Natural Language Understanding and Natural Language Generation, giving it the means to create natural language text. For instance, taking part in a conversation or giving an overview of information.

Text analytics software uses NLP algorithms to identify language and process text. To categorize topics and to perform word-relationship analysis.

Machine learning

Machine learning is the idea that a computer program can learn and adapt to data, without needing a human to input information. It's a field of artificial intelligence - AI - that keeps a computer's algorithms up-to-date. The algorithm is able to identify data and predict.

Tokenization

Breaking up text into words, keywords, phrases, symbols, or full sentences. Punctuation symbols may be ignored. The tokens become the input for the processes such as parsing and text mining.

Speech tagging

Marking up a word in text as corresponding to a particular part of speech. Based on its definition and context. For example, reading a sentence and identifying nouns, pronouns, verbs, adverbs, etc.

Text parsing

To breakdown and analyze text into component parts of speech, to reveal the form, function and syntactic relationship of each part, so the deeper meaning becomes clear.

Named entity extraction/recognition

Process where a sentence or block text is parsed through to identify and segment named entities and classify them under different predefined categories - names, organizations, locations, quantities, monetary values, percentages, etc.

Chunking

Pulling out phrases from unstructured text. Analyzing a sentence and identifying - noun groups, verbs, verb groups, etc. It does not specify internal structure, or their role in the sentence. Works with speech tagging.

Text analytics tools

Text analysis tools unlock unstructured text to help you understand its true meaning. Benefits of text analytics tools include...

• Analysis of unstructured and structured text from various sources
• Insights from text enabling actionable data insights
• Identify, understand and meet the needs of consumers and team members
• Early warning signs of potential issues

In no particular order - other than to put my fave first - here are the best text analysis tools on the market...

Talkwalker Analytics | Text analytics platform

There are hundreds and thousands of conversations about your brand. Some good. Some bad. So bad, they’ll harm your brand’s reputation. You must monitor and analyze this text. Doing it manually will eat up a ton of man-hours. It’s inefficient, and inaccurate.

What you need is the best text analytics tool on the market.

Talkwalker Analytics has many awesome features to protect your brand, analyze the data surrounding your marketing campaigns, measure the results, and produce automated reports with comprehensive data visualizations. Mitigate reputational risk with instant and predictive alerting, unique AI-powered sentiment analysis, image and video analytics.

Talkwalker Analytics - data visualizations for comprehensive reporting of social.

I’m going to highlight our social media search engine, Quick Search.

Quick Search’s sentiment analysis technology helps brands identify consumer sentiment with up to an average 90% accuracy.

Sick! I’ll say it again… sentiment analysis with 90% accuracy.

Our AI-powered sentiment analysis is so awesome, it understands the real meaning behind text, recognizing a consumer’s attitude and contextual reactions - in social media posts, blog posts, news sites, forums, articles, review sites, etc.

Quick Search is a social media search engine that analyzes billions of conversations.

Features include:

• Overview of your brand’s KPIs - engagement, volume, sentiment, demographics, geographies
• Track trends in real-time to target your marketing messages
• Create viral content that engages consumers
• Compare multiple brands and benchmark against competitors

"Quick Search provides such an easy and user-friendly opportunity to deep dive into your competitors' social sphere; letting you harness their strengths and weaknesses to improve and cultivate a winning marketing strategy. For a specific breakdown of the importance of this, you should definitely check out Talkwalker's latest article on the necessity and implications of competitor analysis for your business and brand."
Christina Garnett (@ThatChristinaG) | Strategist

Show me your text analytics tool!

Orange | Open source platform

This free text analytics tool covers machine learning, text mining, data analysis, and data visualization. It’s interactive workflow, comes with visual programming support and a rich tool-set.

Open source machine learning and data visualization.

Orange can be used on Windows, macOS, and Linux.

Texminer | Text mining tool

Texminer is a free text mining tool, working with plain text files and PDFs. The tool handles multiple languages, including English, French, German, and Spanish.

Free text mining tool.

It supports co-occurrence analysis, central expressions, and analysis of letter frequency.

Exclusive to Windows.

RapidMiner | Data mining framework

RapidMiner is open source, and provides users with a graphic user interface that includes text processing, web mining, reporting, sentiment analysis, series processing, and more.

Text processing, web mining, reporting, series processing, and more.

It provides statistical text analysis and collects text from different data sources, as well as filtering technology to analyze your text data. Online reviews and social media messages can be tracked and analyzed, plus official publications.

Find trending topics, gather customer feedback from product launches, and identify new areas for business expansion.

GATE | Java NLP tools

This Java suite of tools can be used for all your natural language processing tasks. Since its launch in 1995, it’s grown to offer a desktop client for developers, a Java library, a workflow-based web application, and it’s still improving.

Text analysis and language processing.

You’ll be able to perform diverse language processing tasks, such as morphology and tagging, information extraction for numerous languages, and retrieval tools.

It’s a comprehensive text analytics tool that’s scalable and user-friendly.

Apache OpenNLP | Machine learning toolkit

Build an advanced text processing project.

This tool is a free, open source library for processing natural language text. It provides tokenization, speech tagging, parsing, named entity extraction, chunking, etc. With these features, you’ll be able to build an advanced text processing project.

Microsoft’s Cognitive Services | AI tools

A suite of artificial intelligence tools to help you develop apps with natural and contextual interaction. It includes text analytics features for analyzing speech and language.

Comprehensive family of AI services and cognitive APIs.

Use the Language Understanding intelligent service to teach bots and applications to understand human input, and to talk with people in a natural language.

Nothing worse than a bot that sounds like a bot.

Bismart’s Folksonomy | Analyze key content

Intelligent tags are included, to investigate your unstructured data and find the information you need. This will save you huge amounts of time, because you won’t have to define tags and categories.

Analyze natural language text documents.

It’s flexible, meaning you can set up the program to suit your needs. And, restructure in real-time, for all your tasks. It’s great for collaborative projects, with heaps of different options. It’s quick, and user-friendly.

MonkeyLearn | Machine learning platform

This easy-to-use platform hosts various types of text analysis models. Providing sentiment analysis, keyword extraction, language detection, intent classification, and more.

Plus, it’s fast.

Text analysis and data visualization.

This text analysis and data visualization suite lets you choose your own business templates that you can customize for each task. Upload your data, and the monkey will do the rest for you. After your data has been analyzed, you can visualize it in a dashboard.

In your templates you can build your own text analysis models or use the pre-trained ones to give you granular results.

Build your own text analysis model.

The point and click user interface means that you don’t need to be a coding genius to use this text analysis tool.

MonkeyLearn integrates with many tools, including Excel, Google Sheets, RapidMiner, Zendesk, and Zapier. Meaning you’ll always have access to your data.

Aylien | Business intelligence solution

Aylien uses AI technology, machine learning, and NLP to extract value from text. Looking to boost your marketing strategies, customer support, product development, this tool can pull out insights to help you make data-driven decisions and to create winning content marketing strategies.

Insights to help make data-driven decisions.

Included are models for sentiment analysis, content aggregation, batch processing, topic discovery, entity extraction, automatic hashtagging, and more.

Aylien also provides a news API, making it super-easy to aggregate, search and understand news articles. Set up takes minutes, and the API is available in seven programming languages.

IBM Watson | Artificial intelligence suite

This suite of AI tools extracts and classifies information from structured and unstructured text data.

IBM natural language processing.

IBM Watson Natural Language Understanding pulls out entities, keywords, concepts, categories, and more. For sentiment analysis, it sorts text into positive, negative, and neutral sentiment. It also classifies by particular emotions, such as excited, sad, confused, etc.

IBM Watson Natural Language Classifier allows you to take meaning from text and assign categorize it. You can create a custom machine learning model by uploading your data. Then the model will classify the text, find trending topics, and pull out insights.

IBM Watson Personality Insights focuses on recognizing consumers’ personality traits. It identifies how consumers engage. How they feel about your product - curious, excited, etc. And, reveals the motivating factors that influence consumers buying behavior.

IBM Watson Tone Analyzer uses linguistic analysis to identify emotion - happy, sad - tendencies - extrovert, introvert - speech style - confident, hesitant.

QDA Miner Lite | Qualitative analysis software

This easy to use, free computer software from Provalis Research can be used for analyzing text data, such as news transcripts, interviews, consumer responses, and still images.

Free text analytics tool.

You can import documents from plain text, RTF, PDF, HTML, and data stored in Excel, CSV, MS Access. Also from coding software that includes Altas.ti, HyperResearch, Ethnography.

Features include...

• Code frequency analysis -bar charts, pie charts, tag clouds
• Coding retrieval with Boolean - AND, OR, NOT - and proximity operators - includes, enclosed, near, before, after
• Export tables to Tab Delimited, CSV formats, XLS, Word

QDA Miner Lite is free, which means the features are limited. But, if you’re looking for more, check out the full version of QDA Miner.

Thematic | Analyze customer feedback

Using AI technology - NLP and deep learning - Thematic analyzes customer feedback, using three AI tools. Thematic Intelligence, Thematic Insights, and Thematic Catalyst.

• Thematic Intelligence extracts meaning from text, grouping content into themes
• Thematic Insights brings results related to trends and patterns in themes
• Thematic Catalyst allows for the creation of data visualizations

AI-powered text analytics tool.

Thematic will also integrate tools that include Zendesk, SurveyMonkey, internal databases, or any Net Promoter Score provider.

Google Cloud NLP | Extract consumer insights

This text analytics tool uses sentiment analysis, entity detection, syntax analysis, and content classification to pull out actionable insights that will help with product launches and user experience.

Train your own machine learning models.

You can also train your own machine learning models with training data. Fine-tune your model to your brand - keywords, sentiment, trends, etc.

Voyant Tools | Text analytics for websites

Text analytics on websites.

Voyant tools are for those wanting to perform text analysis on websites. It doesn't go into the depth of some of the other tools I’ve mentioned, but its simple interface means it can analyze a website and create data visualizations in seconds.

Create data visualizations in seconds.

MeaningCloud | SaaS text analytics solution

This solution automates the extraction of insights from unstructured data. It’s easy to use and integrates with tools such as RapidMiner, Zapier, Google Sheets, and Excel.

Cloud-based APIs and graphic interfaces.

Features include…

• Global sentiment - opinion expressed in tweets, blog posts, reviews
• Sentiment at attribute level - analyze sentiment of individual sentences
• Identify opinions and facts - breaks down into objective and subjective
• Irony detection - comments where sentiment is opposite to what’s said
• Graduated polarity - ranks from negative to positive
• Agreement and disagreement - shows opposing opinions - contradictory, ambiguous

MeaningCloud provides cloud-based APIs and graphic interfaces. You can add dictionaries so that your models will focus on a particular product, and proofread technical or jargon-filled texts.

Lexalytics | Analyze sensitive data

Three text analytics tools in one...

Three text analytics tools in one.

Salience - on-premise natural language processing

Text analytics libraries that will integrate with users’ applications. It provides named entity and theme extraction, sentiment analysis, intent analysis, summarization, tokenization, part-of-speech tagging, and language recognition.

Semantria - cloud natural language processing

Cloud-based API for text analytics and NLP, performing the same tasks as Salience.

SSV - storage and visualization

SSV is embedded in Semantria. It stores, manages, and analyzes unstructured text. Then generates dashboards and reports that focus on patterns and trends.

Free consumer intelligence dashboard template

Any clue as to the impact consumer intelligence could have on your business, if you shared the insights you collected with your colleagues in different teams?

Consumer intelligence interprets conversations at scale by analyzing social media, traditional media, consumer and customer data sources in one place. Sharing these insights across different departments in your company will enable data-driven decision making for business success.

Innovative brands rely on consumer intelligence to support decision-making related to meeting customers’ demands. How do they get the full picture of customer behavior across billions of data points? Dashboards.

Download Talkwalker's free - simulated - consumer intelligence dashboard template, and start listening to the voice of the customer.

Takeaway

Text analytics extracts cutting-edge data that can be translated into insights on consumer intent, social media sentiment, employee satisfaction, competitor intelligence… the list goes on.

Consumers will pay more for an awesome buying experience. That’s why you have to find and analyze all your data, so you understand what consumers want. Improve their opinion of your brand. Encourage them to spend their money on your product.

Text analytics tools enable cost-effective collection of consumer & competitor data. It's easy, quick, and accurate. You don’t need to have hundreds of spreadsheets, or a huge team of data scientists… you need the best text analytics tools on the market.

To give Talkwalker's consumer intelligence platform a try, get in touch!