Analyzing Customer Feedback Emotions with IBM Watson and Bluemix OpenWhisk

by Victoria Fedzkovich and Alex KhizhniakAugust 18, 2016

Detecting the tone of customer messages—whether they are text or audio—is also among the many possible scenarios of employing Watson.

Natural language processing and machine learning enable IBM Watson to derive insights from various types of digital communication. For example, its text analysis services can help users to understand the sentiment and concepts of content, get personality characteristics of individuals based on textual information, or develop human-like conversations with virtual agents.

For a hands-on session on a Bluemix day at the IBM Finland office, we prepared a microservice-based app that analyzes text and provides emotion scores for customer feedback. This post shares the details on how the app was implemented using several Bluemix services and reveals some lessons learned.

Table of Contents

Description of the application

The demo application provides functionality for importing customer feedback messages and audio recordings; for audio files, speech is then also converted into text. Its internal services analyze the text and calculate emotion scores. In case of negative feedback, a customer relations manager receives an SMS notification.

To implement such a solution, we used Node.js and the following IBM Bluemix services:

The application architecture

When designing the application architecture, we decided to split it into microservices, which can be developed, deployed, and scaled independently:

The dashboard service visualizes the current state of feedback records. It is also possible to add new feedback records using a form.
The change-listener service provides a change feed for the database. Other services should subscribe to the feed to react to changes.
The analyzer service analyzes feedback content and calculates emotional scores.
The sms-notifier service notifies the customer relations manager if a feedback record is negative.
The text-extractor service extracts text from audio samples. It allows other parts of the system to process the resulting text and to provide an emotional score.

How the microservices work

For executing the application logic, we chose OpenWhisk since its scalable nature makes it suitable for microservice-based architectures. This event-driven compute platform runs code in response to events provided from Bluemix services and from external sources or to direct invocations. In our case, initial events are added to the Cloudant database when users give their feedback.

Once a user submits a feedback form, dashboard inserts a document into Cloudant. The document looks similar to the following:

{
  type: 'event',
  name: 'TextAdded',
  aggregate_type: 'feedback',
  aggregate_id: '14141-12312312-412412414-123123',
  timestamp: 1463629152101,
  attributes: {
    text: 'Sample Text'
  }
}

The change-listener service sets up an OpenWhisk trigger and feed. Basically, it calls all attached OpenWhisk actions when the Cloudant database is updated; in our case—when we insert an event into the database.

The analyzer service is an OpenWhisk action that is subscribed to the change-listener feed. Each time we insert an event, analyzer is executed. The action generates another event—EmotionsAnalyzed.

The sms-notifier and text-extractor services are also implemented as OpenWhisk actions.

The sms-notifier service:

Waits for the EmotionsAnalyzed event
Checks the level of negative emotions (for example, anger)
Sends an SMS message to the customer relations manager phone
Emits the SmsNotificationSent event

You will see the Manager Has Been Notified status if the anger score is above 50% as in the image below.

The text-extractor service:

Waits for the AudioUploaded event
Downloads associated files
Converts speech to text employing Watson APIs
Emits the TextAdded event so that the system can perform emotion analysis using the existing code

After extracting text from an audio file, you should see something similar to this:

BTW, in Bluemix, it is really convenient to view the audio files uploaded to Object Storage:

Lessons learned

While working on the application, we made several observations:

The most common errors are typos in the .env and config.env files. Make sure you have specified your credentials correctly.
If your text does not seem to be analyzed, you can check the logs using either the OpenWhisk dashboard or the OpenWhisk CLI with this command: wsk activation poll.
One of the current limitations of the OpenWhisk and Cloudant integration is that you cannot specify a filter for a trigger.
It is easy to mix up Twilio developer credentials with the ones that actually send an SMS. Make sure you can send messages from the Twilio dashboard.
Bluemix provides a management panel for many services. For example, you can inspect the uploaded audio files by opening the corresponding service on the dashboard.
Do not expect lossless, one-to-one conversion of speech to text. Even for Watson, this is a hard thing to achieve.
Not all WAV files can be processed. Pay attention to the error logs on the OpenWhisk dashboard.

The detailed explanation of the application architecture, source code, and troubleshooting recommendations are available in our GitHub repository. You can follow the steps of the hands-on training described there to create your own microservice-enabled application using IBM Bluemix services, such as IBM Watson and OpenWhisk.

This application was presented on May 31, 2016 during the IBM Bluemix day at the IBM Finland office, which had Cloud Foundry, microservices, and the Internet of Things as main topics on the agenda. In addition to the hands-on demo, the meeting featured a session by Kimmo Kaskikallio (Senior Architect and Cloud Advisor at IBM Finland) about the IoT capabilities of Bluemix. Ari Mutanen, CEO at Altoros Finland, shared his vision of Bluemix and Cloud Foundry as development platforms, presenting a couple of use cases.