Twitter Sentiment Analysis part 5: Plotting Live Graph of Sentiment using Matplotlib

Hello and welcome to the 5th and last part of this series, In the previous part we learnt how to load the tweets and save the prediction in a text file, In this part, we will use the same file as a pipeline to get the data at the same time it append and show the graph in real time.

For this, we need two python versions or two instances of the same version because we will run both twitter analysis file and plotting file at the same time.

So we will run one file in a jupyter notebook and another from default IDLE.

If you dont have jupyter, you can install it via pip using this command

pip install jupyter

after successful installation open the command prompt or terminal and type

jupyter notebook 

and it will open a jupyter window in a browser

I am using jupyter to load the tweets and do the analysis and idle to plot the graph, but you are free to choose.

As always, I am adding the full code here, if you want to understand the specific function or specific line then just navigate to the particular line in the explanation

First of all we are importing the liberaries, Here we are using matplotlib to plot the graph, Since the graph is a live graph so we are using animation module so that graph can update in real time.

import matplotlib.pyplot as plt
import matplotlib.animation as animation
from matplotlib import style
import time

style.use("ggplot")

fig = plt.figure()
ax1 = fig.add_subplot(1,1,1)

def animate(i):
    pullData = open("/Users/pushkarsingh/Desktop/twitter/test-yy_twitter.txt","r").read()
    lines = pullData.split('\n')

    xar = []
    yar = []

    x = 0
    y = 0
    for l in lines[-100000:]:
        x += 1
        if "pos" in l:
            y+=1
        elif "neg" in l:
            y-=1

        xar.append(x)
        yar.append(y)
        
    ax1.clear()
    ax1.plot(xar,yar)
ani = animation.FuncAnimation(fig, animate, interval=1000,save_count=1000)
plt.show()

Explanation

First of all we are importing the liberaries, Here we are using matplotlib to plot the graph, Since the graph is a live graph so we are using animation module so that graph can update in real time.

import matplotlib.pyplot as plt
import matplotlib.animation as animation
from matplotlib import style

The basic style of matplotlib is dull so to make the graph more vibrant we use ggplot. this is a style liberary of grsphs for R language but now its also support python.

style.use("ggplot")

Here fig is just a canvas on which the ax1 will plot.

In the next line, we are adding the subplot. this is the plots which are adding on the same canvas, but the position is different. The position is defined by the 3rd number. if its 1 then its first graph and deserve upper left corner, similarly. 2 for upper right corner, 3 for lower left corner and 4th is lower left corner.

Here (2,2,x) refer to the grid size, while in the code we use the grid size of (1,1,x). We can only create a single graph of 1×1 grid size.

fig = plt.figure()
ax1 = fig.add_subplot(1,1,1)

In the first line we are defining the function which we will use to plot the live graph, then we open the file in which data is storing, and split the lines by a new line

Here lines are the number of tweets, so we are showing the last 10000 tweets on the graph. We continuously increasing the counter of x which mimic the number of tweets and we plot it in the x line.

In the next line, if in the lines, “pos” word is found then y increase by 1 or if “neg” word is found then y decrease by 1.

def animate(i):
    pullData = open("/Users/pushkarsingh/Desktop/twitter/test-yy_twitter.txt","r").read()
    lines = pullData.split('\n')

    xar = []
    yar = []

    x = 0
    y = 0
    for l in lines[-10000:]:
        x += 1
        if "pos" in l:
            y+=1
        elif "neg" in l:
            y-=1

In this step, we are appending the array with new values x and y values.

In the next step, we clear the graph and plot the full array again. Every time the array appends, it clears the graph and plot the new array. this process is so fast that we only see the updated part.

In the last 2 steps we are calling the functions and showing the plot.

    xar.append(x)
    yar.append(y)

ax1.clear()
ax1.plot(xar,yar)

ani = animation.FuncAnimation(fig, animate, interval=1000,save_count=1000)
 plt.show()

Now our programs are ready so let’s learn and see how it looks.

I have done Sentiment Analysis on some keywords like India, Pakistan, Modi, Trump, Happy and Terrorists.

Note that this program based on words. The sentences

• This food is bad
• This food is not bad

Both are negative because it contain negative word “bad”. To counter this problem there are other techniquies like Bigrams models and Trigram models.

In Bigrams approach we train the model of two adjacent words like (This food, food is, is not, not bad) which consider the adjacent words and is more reliable.

In Trigrams apporach we train the model of 3 adjacent words like (This food is, food is not, is not bad,) which is even more reliable then Bigrams approach

Our model is quite good too lets see how it performs. I am sharing a video in which I have dome some Sentiment Analysis.

Hope you like this series and if you have any doubt or suggestion then please write in the comment below and I will try to solve it.

Thanks a lot 😀