Market Research project managers have been presented with various unique tools to help us better provide insight deliverables to decision makers. One of the toolsets is Natural-Language Processing services that provide automated analytics based on open ends. Combined with video surveys, NLP services have given project managers tools to better decipher the understanding of the meaning and emotions behind survey responses. We took some time to review two of the more popular NLP services; IBM Watson and AWS Comprehend.

IBM Watson

IBM Watson uses machine learning and natural language processing to not only understand a pieces’s overall tone or feeling but to also identify the sentiment within each sentence. In addition to identifying the idea of individual sentences, IBM Watson can also determine the overall polarity of a video. With this information, businesses can better understand customer feedback and gain insight into their target audience.

Beyond sentiment analysis, IBM Watson can detect language nuances and identify key conversation topics, providing valuable insights into customer preferences or areas of concern. By monitoring these conversations across multiple social media platforms, companies can track online trends and adapt their strategies accordingly. IBM Watson also provides powerful visualization tools that allow users to understand the collected data more deeply. With interactive graphs, heatmaps, and other visuals, businesses can quickly analyze complex relationships between variables such as gender, location, and product usage. These insights enable effective marketing campaigns while providing an accurate picture of customer behavior.

AWS Comprehend

Amazon Comprehend uses natural language processing (NLP) to extract insights about the content of documents. It can understand the sentiment of text, identify the main topics in a copy, and track changes in opinion over time. This allows businesses to better understand what their customers are talking about online and how they feel about different products or services. Amazon Comprehend is also able to analyze the sentiment of customer reviews. By studying video responses, businesses can gain insight into customer preferences and make more informed decisions about product development or marketing strategies.

Amazon Comprehend can also detect critical phrases in customer feedback which can be used to refine search engine optimization (SEO) efforts. In addition to sentiment analysis and topic detection, Amazon Comprehend offers advanced capabilities, including entity extraction and named-entity recognition (NER). With these features, businesses can identify people, places, and things mentioned in text documents, allowing for more targeted marketing campaigns. Furthermore, Amazon Comprehend provides language translation services that enable businesses to reach global audiences with localized content. At Clear Seas, we prefer AWS Comprehend to IBM Watson.

Comparing AWS and IBM

First, we have found AWS Comprehend’s privacy protections more robust. IBM Watson's PII detection is limited compared to AWS Comprehend’s privacy controls. Both require coding skills to implement, yet AWS takes the rules to a higher level of control and includes more entities that can be redacted.

Second, IBM Watson's sentiment is limited to Positive, Negative, and Neutral, whereas AWS has added a "Mixed" category.

• Positive – The text expresses an overall positive sentiment.
• Negative – The text expresses an overall negative sentiment.
• Mixed – The text expresses both positive and negative emotions. (*unique to AWS)
• Neutral – The text does not represent either positive or negative feelings.

Third, AWS provides much better accuracy in detecting tone and emotion within transcripts. We have an example here. The model we included here is particularly challenging as the participant changed gears mid-question to showcase a brand he did NOT prefer while explaining why he USED to like it.

For our accuracy comparison, we first look at IBM Watson. IBM Watson would appear to award the comment as 56% positive, and while the respondent does express positive sentiment, he is doing so in the context of a brand he no longer prefers.

And second, below, we have the AWS analysis. Remarkably, AWS Comprehend appears to recognize that these positive comments are presented within a more significant and negative context. AWS Comprehend goes one step further and correctly puts these comments into the “neutral” category.

In conclusion, while both systems save time and effort, and provide compelling analytics, AWS tends to provide more accurate sentiment analysis. Using an NLP service can help project managers sift through lengthy transcripts and provide helpful context to end users in determining the meaning behind respondent statements.