Language-independent

Bayesian sentiment
Alexander Davies, Zoubin Ghahramani
mining University
of Twitter
of Cambridge

1.
Sentiment mining of Twitter is useful in a variety
Introduction
of
fields and has been shown to have predictive
value for box office revenues, political poll
outcomes and even the stock market.

2.
We assume that each tweet has a hidden
Model
sentiment
and that the words in a tweet are

Most methods use the presence of emoticons in tweets

as noisy labels for tweet sentiment; we present a more
principled way to incorporate this information and a
method to analyze geographical variation.

drawn from a multinomial distribution that

depends only on its sentiment. An example of
the probability of the words in the sentence I
love this great car under two different
multinomial distributions is shown below.

4. Benefits
We also place Bayesian priors on the
multinomials, which allow us to specify which
words we believe will be likely in each sentiment,
before weve seen any tweets. This is how we
incorporate the emoticon information.

3.
Geography
To model
different geographic areas, we could train
models separately for different regions, but some
regions have far fewer tweets than others and we
wont get a good estimate of the word distribution
in those areas.
We can incorporate our knowledge about word
distributions from neighbouring regions and put
them into our prior. To do this, we train the models
separately, create priors based on these models
and then retrain the models with the new priors.

Nave Bayes can be constructed as a special

case of our method. Ours however, has two
main advantages.
1. It allows correct modeling of
uncertainty on labels.
2. It iteratively refines the word
To illustrate
the second advantage, consider
distributions.
the following example dataset:

Tweet 1: Great :) Tweet 2: Holidays!

Great!
Nave Bayes will learn that great is a happy
word, but nothing about holidays. At the first
iteration, our method will learn that great is
probably a happy word. At the second
iteration it will learn that holidays is also
probably a happy word.

5. Results

amour feliz kasih

:-( sad triste :( nooo
:-) happy love :)
booo </3 poor miss
thanks birthday
*cries* gutted
thank follow
aord :'( poorly hate
good welcome
dean ugh fml
luck great nice
urghhh stressed
morning
headache *sigh*
amazing hello
canada prayforkatie
lovely xxx best
richards rip whyyy
hey awesome
horrible revision
Examples of high-likelihood
happy
cheers
*hugs*

and sad words from the UK

6.
We have shown that there are
Conclusion
advantages
to full probabilistic models

over using basic classifiers with noisy

labels. We more correctly model our
assumptions about tweets and see
performance increases because of this.
The framework also makes it easy to
incorporate information into
neighbouring regions, as we show for a
geographic sentiment model.