Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- Sentiment Classification using Machine Learning Techniques
- Pranjal Vachaspati
- pranjal@mit.edu
- Cathy Wu
- cathywu@mit.edu
- Abstract
- We implement a series of classifiers (Naive Bayes, Max-
- imum Entropy, and SVM) to distinguish positive and nega-
- tive sentiment in critic and user reviews. We apply various
- processing methods, including negation tagging, part-of-
- speech tagging, and position tagging to achieve maximum
- accuracy. We test our classifiers on an external dataset to
- see how well they generalize. Finally, we use a majority-
- voting technique to combine classifiers and achieve accu-
- racy of close to 90% in 3-fold cross-validation, far outper-
- forming Pang’s 2002 work [7].
- 1. Introduction
- Sentiment analysis, broadly speaking, is the set of tech-
- niques that allows detection of emotional content in text.
- This has a variety of applications: it is commonly used by
- trading algorithms to process news articles, as well as by
- corporations to better respond to consumer service needs.
- Similar techniques can also be applied to other text analysis
- problems, like spam filtering.
- The source code described in this paper is available at
- https://github.com/cathywu/Sentiment-Analysis.
- 2. Previous Work
- We set out to replicate Pang’s work [7] from 2002 on
- using classical knowledge-free supervised machine learn-
- ing techniques to perform sentiment classification. They
- used the machine learning methods (Naive Bayes, maxi-
- mum entropy classification, and support vector machines),
- methods commonly used for topic classification, to explore
- the difference between and sentiment classification in doc-
- uments. Pang cited a number of related works, but they
- mostly pertain to classifying documents on criteria weakly
- tied to sentiment or using knowledge-based sentiment clas-
- sification methods. We used a similar dataset, as released
- by the authors, and made efforts to use the same libraries
- and pre-processing techniques.
- In addition to replicating Pang’s work as closely as we
- could, we extended the work by exploring an additional
- dataset, additional preprocessing techniques, and combin-
- ing classifiers. We tested how well classifiers trained on
- Pang’s dataset extended to reviews in another domain. Al-
- though Pang limited many of his tests to use only the
- 16165 most common ngrams, advanced processors have
- lifted this computational constraint, and so we addition-
- ally tested on all ngrams. We used a newer parameter es-
- timation algorithm called Limited-Memory Variable Met-
- ric (L-BFGS)[5] for maximum entropy classification. Pang
- used the Improved Iterative Scaling method. We also imple-
- mented and tested the effect of term frequency-inver docu-
- ment frequency (TF-IDF) on classification results.
- 3. The User Review Domain
- For our experiments, we worked with movie re-
- views. Our data source was Pang’s released dataset
- (http://www.cs.cornell.edu/people/pabo/movie-review-
- data/) from their 2004 publication. The dataset contains
- 1000 positive reviews and 1000 negative reviews, each
- labeled with their true sentiment. The original data source
- was the Internet Movie Database (IMDb).
- Pang applied the bag-of-words method to positive and
- negative sentiment classification, but the same method can
- be extended to various other domains, including topic clas-
- sification. We additionally chose to work with a set of 5000
- Yelp reviews, 1000 for each of their five star rating. Yelp
- is a popular online urban city guide that houses reviews
- of restaurants, shopping areas, and businesses. Although
- a movie review and a Yelp review will differ in specialized
- vocabulary, audience, tone, etc., the ways that people con-
- vey sentiment (e.g. I loved it!) may not differ entirely. We
- wished to explore how training classifiers in one domain
- might generalize to neighbor domains.
- The domain of reviews is experimentally convenient be-
- cause there are largely available on-line and because re-
- viewers often summarize their overall sentiment with a
- machine-extractable rating indicator; hence, there was no
- need for hand-labeling of data.
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement