Connect to Comprehend

To get the sentiment scores for every message, you need to send them to the DetectSentiment API that Comprehend exposes.

Before writing code to do this programmatically it’s important to take a moment to understand pricing for the service. Access to the real-time analysis APIs, including the DetectSentiment API is paid for in units, where each unit is 100 characters. The first 50,000 units in the real-time analysis APIs are included in the Free Tier. After that, you pay $0.0001 per unit. 50,000 units is plenty for small-scale testing. But you need to be careful to not let the service keep running when you’re done.

The easiest way to call the DetectSentiment API on a batch of messages is to use the batchDetectSentiment() method in the Comprehend client of JavaScript AWS SDK. It takes a list of messages that are no more than 5,000 UTF-8 characters each.

Another thing to note is that the batch DetectSentiment API call takes 25 messages at a time. It’s a good idea to send in the maximum number of messages per call to limit the number of network requests and to avoid getting rate limited. However, the PutMetricData call to CloudWatch takes a maximum of 20 metrics per request. To keep the batching logic simple, put the ceiling at 20 messages for the DetectSentiment call as well. To accomplish this, you need to keep a running list of messages until the length of the array is 20. At that point, you make your request to the Comprehend service and reset the list to be an empty array.

All of this can be handled in a class. Take a look at the src/comprehend.js module to see how this works.

The constructor takes a callback that will be passed into the call to the batchDetectSentiment() method. It will be executed on the response from Comprehend. The messages property starts as an empty list. The comprehend property is the Comprehend client from the AWS SDK. One important thing to point out is that you are required to set the region explicitly. This can be passed in as a parameter to the constructor or as the AWS_REGION environment variable when you run the main script.

Every time a message is added, the addMessage() method truncates it to 5,000 characters and appends it to the messages array. Then if the length of the array is 20, the request is made and the messages property gets reset.

Update index.js

  1. Now, modify the index.js script to send Twitch messages to Comprehend:

    // index.js
    const twitchMessages = require('./src/twitch');
    const DetectSentiment = require('./src/comprehend');
    
    const argv = require('yargs')
        .options({
            'channel': {
                alias: 'c',
                describe: 'Twitch channel',
                demandOption: true,
            },
            'duration': {
              alias: 'd',
              describe: 'Monitoring time (ms)',
              type: 'number',
              default: 300000,
            },
        }).argv;
    
    const detectSentiment = new DetectSentiment(console.log);
    twitchMessages(
      argv.channel,
      argv.duration,
      (_, __, message) => detectSentiment.addMessage(message),
    );
    

    The script now imports the DetectSentiment class and instantiates it with console.log() as a callback – this will simply print the response from Comprehend. The callback to the twitchMessages() function also changes to call the addMessage() method from the DetectSentiment object.

  2. Call this script:

    AWS_REGION=us-east-1 node index.js -c stumptgamers
    

    Now, wait until a Comprehend response is printed to the Terminal. The more active the channel you pass in, the less you will have to wait. The result should look something like this:

    ...
    { ResultList:
      [ { Index: 0, Sentiment: 'NEUTRAL', SentimentScore: [Object] },
        { Index: 1, Sentiment: 'NEUTRAL', SentimentScore: [Object] },
        { Index: 2, Sentiment: 'NEUTRAL', SentimentScore: [Object] },
        { Index: 3, Sentiment: 'POSITIVE', SentimentScore: [Object] },
        ...
        { Index: 24, Sentiment: 'NEUTRAL', SentimentScore: [Object] } ],
      ErrorList: [] }
    ...
    
  3. You can use the Control+C keyboard shortcut to stop the main script.