\documentclass[conference]{IEEEtran}
    \usepackage{blindtext, graphicx}
    \usepackage{fixltx2e}
    \usepackage{float}
    \usepackage{amsmath}
    \usepackage{textgreek}





    \ifCLASSINFOpdf
    \else
    \fi




    \hyphenation{op-tical net-works semi-conduc-tor}


    \begin{document}
    \title{Understanding the pulse of students,in order to enhance their learning experience via Social Media}

    \author{\IEEEauthorblockN{Aneesh Subramanya, Kartik Koralla,\\Keerthi Prasad N and Suhas H S}
    \IEEEauthorblockA{Department of Information Technology\\
    National Institute of Technology Karnataka\\
    Surathkal, Mangalore, India\\
    aneeshsubramanya@gmail.com, k.koralla@gmail.com\\keerthy17394@gmail.com,suhas.h.suresh@gmail.com}
    \and

    \IEEEauthorblockN{Ram Mohana Reddy Guddeti}
    \IEEEauthorblockA{Department of Information Technology\\
    National Institute of Technology Karnataka\\
    Surathkal, Mangalore, India\\
    profgrmreddy@nitk.ac.in}}

    \maketitle


    \begin{abstract}
    Students’ informal conversations on social media (e.g., Twitter, Facebook) shed light into their educational experiences— opinions, feelings, and concerns about the learning process. Data from such uninstrumented environments can provide valuable knowledge to inform student learning. Analyzing such data, however, can be challenging. Social media sites such as Twitter, Facebook, and You- Tube provide great venues for students to share their experiences, vent emotion and stress, and seek social support. Students’ digital footprints provide vast amount of implicit knowledge and a whole new perspective for educational researchers and practitioners to understand students’ experiences outside the controlled classroom environment. This understanding can inform institutional decision-making on interventions for at-risk students, improvement of education quality, and thus enhance student recruitment, retention, and success. The abundance of social media data provides opportunities to understand students’ experiences, but also raises methodological difficulties in making sense of social media data for educational purposes. Just imagine the sheer data volumes, the diversity of Internet slang, the unpredictability of location and timing of students posting on the web, as well as the complexity of students’ experiences. Pure manual analysis cannot deal with the ever-growing scale of data, while pure automatic algorithms usually cannot capture in-depth meaning within the data.A dashboard for the institution is developed in order to make the validation and decision making easier and effective using visualization.
    \end{abstract}
    \begin{IEEEkeywords}
    Social Network Analytics, Machine Learning, Naive Bayes' Clas-
    sifier, Maximum Entropy Classifier, SVM Classifier, Decision Tree Classifier
    \end{IEEEkeywords}




    \IEEEpeerreviewmaketitle



    \section{Introduction}

    \subsection{Scope of Work}
    The research goals of this study are to demonstrate a work flow of social media data sense-making for educational purposes, integrating both qualitative analysis and large scale data mining techniques and to explore engineering students’ informal conversations on Twitter, in order to understand the problems students encounter in their learning experiences.
    This project makes two major contributions. First, it proposes a work flow to bridge and integrate a qualitative research methodology and large-scale data mining techniques. We base our data-mining algorithm on qualitative insights resulting from human interpretation, so that we can gain deeper understanding of the data. We apply the algorithm to another large-scale and unexplored data set, so that the manual method is augmented. Second, the paper provides deep insights into engineering students’ educational experiences as reflected in informal, uncontrolled environments. Many issues and problems such as study life balance, lack of sleep, lack of social engagement, and lack of diversity clearly emerge. These could bring awareness to educational pedagogy, policy-making, and educational practice.

    The next section reviews theory of public discourse on line, related work on text classification techniques used for analyzing tweets, and data-driven approaches in education. Section 3.A describes the data collection process. Section 3.B details the inductive content analysis procedures and categories identified . Section 4 details the implementation of the Naive Bayes, Maximum Entropy, SVM and Decision Tree multi-label classifiers and the evaluation results. Following which, we show the comparison results of the Naive Bayes classifier with the popular classifier Support Vector Machine (SVM), Maximum Entropy Classifier and Distributed tree classifier.Section 5 describes the impact of visualization on the students learning experience. Section 6 concludes this study, followed by limitations and possible future work.

    \subsection{Product Scenario}
    It provides a work flow for analyzing social media data for educational purposes that overcomes the major limitations of both manual qualitative analysis and large scale computational analysis of user-generated textual content. Our study can inform educational administrators, practitioners and other relevant decision makers to gain further understanding of engineering students’ college experiences. It can also be used in universities to measure the level of impact the academic schedule has on the students. Based on the result obtained, required modification can be done.

    \section{Related Work}

    \subsection{Mining Twitter Data }
    The abundance of social media data provides opportunities but also presents methodological difficulties for analyzing large-scale informal textual data. Researchers from diverse fields have analyzed Twitter content to generate specific knowledge for their respective subject domains. The data analysis methods include Inductive Content Analysis, Linguistic Analysis and Network Analysis among others. Our classification model is based on inductive content analysis. We use the insights gained from one data set to classify other data sets for detecting student problems so as to augment the human effort involved. This includes Data Mining from Twitter, Machine Learning and Natural Language Processing. We have implemented 4 of the popular classification algorithms-Naïve Bayes, Support Vector Machine (SVM), Decision Tree and Maximum Entropy. Classification can be Binary or Multi-class, Single-label or Multi-label.
    Usually studies on Tweet classification are either on binary classification or multi-class classification on generic classes such as news, entertainment and sports. Another widely used classification is Sentiment Analysis to classify a tweet as positive, negative or neutral.
    Our problem requires a Multi-Label Classification since each data point in our data set can fall into several classes at the same time. Our purpose is to achieve deeper understanding of students’ learning experiences. Thus, a fully unsupervised way of learning is unsuitable for this problem.
    \section{Preliminary Work}
    \begin{figure}[H]
     \caption{Workflow Diagram for Classification of Students' Learning Experiances}
     \centering
       \includegraphics[width=0.5\textwidth]{Flow_Chart.jpg}
    \end{figure}

    \subsection{Data Collection}
    It is challenging to collect social media data related to students’ experiences because of the irregularity and diversity of the language used. The Twitter API, Tweepy was configured to accomplish this task. We used Tweepy to extract tweets using the hashtag #EngineeringProblems. We removed the irrelevant tweets from this collection. A list of all hashtags contained in the previously obtained relevant tweets was made. The list contained hashtags such as #CollegeProblems, #StudentProblems and #Deadlines. We then expanded the tweet collection by using relevant hashtags from this list to further extract tweets.
    \subsection{Inductive Content Analysis}
    Social media content like tweets contain a large amount of informal language, sarcasm, acronyms, and misspellings thus making the meaning ambiguous and subject to human interpretation. In large-scale social media data analysis, qualitative data analysis is necessary to avoid faulty assumptions produced by automatic algorithms. We went through a small fraction of the collected tweets manually and classified them into the following five categories [1] Heavy Study Load [2] Lack of Social Engagement [3] Negative Emotion [4] Sleep Problems [5] Diversity Issues. The classification was multi-label as tweets often belong to more than 1 of the above mentioned categories. These classified tweets were used to extract the top 25 words from each category. We then used the WordNet database to develop synonyms for each of the words. This was done to expand the list of words for each category so as to better the classification process. Thus, we produced around 500 words for each category from the initial 25 words. These words were used to train different classifiers to classify new tweets.
    \subsubsection{Heavy Study Load}
    Upon conducting several manual surveys, we observed that a vast majority of students felt that their lives were overloaded with assignments, tests, exams, labs and so on. Some tweets related to the same were "If my textbook requires 4 authors to write it, how do they expect 1 student to learn it." and "3 hours is not long enough for a 3 question exam". This heavy study load upon students gives rise to various other issues such as lack of sleep, stress, depression and several other health problems.
    \subsubsection{Lack of Social Engagement}
    The analyses show that very often students give up on social gatherings due to classes, homework or stress. For example "No it's cool professor. I wanted to spend my first weekend back from break doing problem sets and pre labs." and "Having a test at 2PM on the friday before spring break is the absolute worst". Social engagements provide an outlet to relieve the stress that every student faces. Thus social, engagements can actually have a positive impact on students performances.
    \subsubsection{Negative Emotion}
    Most of the tweets express negative emotion, given that the hashtag #engineeringproblems which we search for is negative in itself. We classify a tweet into "negative emotion" when it specifically expresses negative emotions such as anger, hate, disgust, despair, etc... For example, "is it bad that before i started studying for my tests today that i considered throwing myself in front of a moving car" and "40 hours in the library in the past 3 days. I hate finals". It is necessary for students to manage such emotions and get support.
    \subsubsection{Sleep Problems}
    Sleep problems were also very abundant in our analyses. We found a large number of students who would miss sleeping at night, and fall asleep at odd times during the day. Some examples of tweets we found pertaining to this category are "That moment when you realize you'll be getting a max of 2.5 hours of sleep" and "This time change doesn't affect me as an engineering student, sleep wise. I'm deprived of sleep regardless".
    \subsubsection{Diversity Issues}
    We found that this category had a relatively lesser number of tweets compared to the others, but it still had a significant amount. Some students had issues with the gender ratio in engineering institutes, whereas others had problems witht he racial distribution. Examples of such tweets are "I’m sorry. We’re not use to having girls around" and "Finally talked to a girl today!!! It was Siri".

    \subsection{Text Preprocessing}
    Twitter users use some special symbols to convey certain meaning. For example, # is used to indicate a hashtag, @ is used to indicate a user account, and RT is used to indicate a re-tweet. Twitter users sometimes repeat letters in words so that to emphasize the words, for example, “huuungryyy”, “sooo muuchh”, and “Monnndayyy”. So, we preprocessed the text before training the classifier to [1] remove Hashtags [2] replace negative words like nothing, never, cannot, none, words ending with n’t with ‘negtoken’ [3] remove http links, RTs, non-letter symbols [4] replace words like ‘huuungryy’, ‘sooo’ with the corrected words [5] remove stop-words. We retained stop-words such as many, more, always since they express extent. The words obtained after this step constitute a document.
    \section{Classifiers, Results and Analysis}
    \subsection{Naïve Bayes' Classifier}
    The Naïve Bayes classifier is a simple probabilistic classifier that assumes strong (naïve) independence between features. We built a multi-label classifier to classify tweets based on the categories developed in the previous content analysis stage. A multi-label classifier was implemented using the Python NLTK package. We assume independence among each word in a document (tweet after text preprocessing). Each word in a document was fed into the classifier to obtain the probability of it belonging to each of the five categories. The probability that a document d\textsubscript{i} containing words w\textsubscript{ik} belongs to a category c is obtained as \[ p(c|d_i)=\frac{p(d_i|c)\cdotp(c)}{p(d_i)} \propto  \prod_{k=1}^{K} p(w_{ik}|c)\cdot p(c) \]

    The probability that d\textsubscript{i} belongs to a category other than c is \[p(c'|d_i)=\frac{p(d_i|c')\cdot p(c')}{p(d_i} \propto \prod_{k=1}^{K}p(w_{ik}|c')\cdot p(c')\]
    A tweet is assigned to a category c if the probability os larger than threshold T. A tweet is assigned to a category ‘others’ if ‘others’ is the only category with a probability larger than T.
    \subsubsection{Evaluation Measures}
    Evaluation measures for the Naïve Bayes classifier include accuracy, precision and recall. For a certain document d, suppose the true set of labels it falls under is Y, and the predicted set of labels by the classifier is Z, then for this specific document, accuracy is the correctly predicted number of labels divided by the number of labels in the union of Y and Z. Precision is the correctly predicted number of labels divided by the total number of labels in Z, while recall is the correctly predicted number of labels divided by the number of true labels. Accuracy, Precision and Recall averaged over a total of M documents are given by \[Accuracy \hspace{2mm} a=\frac{1}{M}\sum_{i=1}^{M}\frac{Y_i\cap Z_i}{Y_i\cup Z_i} \]
    \[Precision\hspace{2mm} p=\frac{1}{M}\sum_{i=1}^{M}\frac{Y_i\cap Z_i}{Z_i}\]
    \[Recall\hspace{2mm} r=\frac{1}{M}\sum_{i=1}^{M}\frac{Y_i\cap Z_i}{Y_i}\]
    The final analysis results is shown below. Our problem is to detect the 5 problems discussed above. Hence we do not discuss the ‘others’ category in our results.

    \subsubsection{Performance Metrics}

    The performance metrics for Naïve Bayes Classifier are shown in the chart below.

    \begin{figure}[H]
     \caption{Performance Metrics for Naïve Bayes Classifier}
     \centering
       \includegraphics[width=0.5\textwidth]{PerformanceMetrics.png}
    \end{figure}

    \subsection{Maximum Entropy Classifier}
    In statistics, multinomial logistic regression is a classification method that generalizes logistic regression to multiclass problems, i.e. with more than two possible discrete outcomes. That is, it is a model that is used to predict the probabilities of the different possible outcomes of a categorically distributed dependent variable, given a set of independent variables. This is appropriate for our problem of tweet classification, where each tweet can fall into one or more categories. The MaxEnt Classifier was implemented using NLTK package for Python.
    As in other forms of linear regression, multinomial logistic regression uses a linear predictor function \textbf{\textit{f(k,i)}} to predict the probability that observation \textit{i} has outcome \textit{k}, of the following form:
    \[f(k,i)=\beta_{0,k}+\beta_{1,k}x_{1,i}+...+\beta_{M,k}x{M,i}\]
    where each B\textsubscript{m,k} is a regression coefficient associated with the mth explanatory variable and the \textit{kth} outcome. As explained in the logistic regression article, the regression coefficients and explanatory variables are normally grouped into vectors of size \textit{M+1}, so that the predictor function can be written more compactly:
    \[f(k,i)=\beta_{k} \cdot x_{i}\]

    \subsection{SVM Classifier}
    Support vector machines (SVMs) are a set of related supervised learning methods used for classification and regression. In simple words, given a set of training examples, each marked as Given a set of pairs of feature data-point vectors \textit{x} and classifier labels \textit{y={0,1}}, the task of the SVM algorithm is to learn to group features x by classifiers. After training on a known data set the SVM machine is intended to correctly predict the class y of an previously unseen feature vector x. Intuitively, an SVM model is a representation of the examples as points in space, mapped so that the examples of the separate categories are divided by a clear gap that is as wide as possible. New examples are then mapped into that same space and predicted to belong to a category based on which side of the gap they fall on. A linear support vector machine is composed of a set of given support vectors \textbf{z} and a set of weights \textbf{w}. The computation for the output of a given SVM with N support vectors \textbf{z\textsubscript{1}, z\textsubscript{2}, … , z\textsubscript{N}} and weights \textbf{w\textsubscript{1}, w\textsubscript{2}, … , w\textsubscript{N}} is then given by:
    \[F(x)=\sum_{i=1}^{N}w_i\langle z_i,x\rangle+b\]

    \subsection{Decision Tree Classifier}
    The decision tree algorithm used is ID3 (Iterative Dichotomiser 3). The ID3 algorithm begins with the original set S as the root node. On each iteration of the algorithm, it iterates through every unused attribute of the set S and calculates the entropy or information gain of that attribute. It then selects the attribute which has the smallest entropy (or largest information gain) value. The set S is then split by the selected attribute to produce subsets of the data. The algorithm continues to recur on each subset, considering only attributes never selected before.

    Tf-idf is used to develop the feature set for the decision tree. Tf-idf denotes the statistical importance of a word in a document. 25 significant words from each category were picked after analyzing a certain number of tweets manually. Totally 125 words were used and a test set using the tf-idf format was used to train the decision tree, i.e. 125 features were used.

    An accuracy of 33\% was obtained on the training set. The less accuracy is due to more tweets belonging to mainly one category, which in our case is heavy work load.

    \subsection{Results}
    Upon training the classifiers with $\sim$1500 tweets and testing with $\sim$500 tweets, we obtained the following accuracies for each classifier:
    \begin{figure}[H]
     \caption{Accuracy of Classifiers}
     \centering
       \includegraphics[width=0.5\textwidth]{AccuracyChart.png}
    \end{figure}

    Upon testing, we observe that Naive Bayes' Classifier gives us the best accuracy, so we proceed to classify further $\sim$2000 tweets using the Naive Bayes' Classifier. The results of $\sim$2000 tweets belonging to the 5 categories is shown below.
    \begin{figure}[H]
     \caption{Number of Tweets in each Category}
     \centering
       \includegraphics[width=0.5\textwidth]{number_of_tweets.png}
    \end{figure}

    \section{Visualization}
    Visualizations help people see things that were not obvious to them before. Even when data volumes are very large, patterns can be spotted quickly and easily. Visualizations convey information in a universal manner and make it simple to share ideas with others. It lets people ask others, “Do you see what I see?” And it can even answer questions like “What would happen if we made an adjustment to that area?”.Data visualization presents the data in a way that the head can easily interpret, saving time and energy.
    In this project Tableau visualization tool was used.A word cloud of most tweeted words were bucketed and was displayed based on various filters.So a keyword extractor was built to extract the most important keyword from a tweet after removal of stop words,stemming,lemmatization and other Natural Language processing.Visualization was done on various filters like timestamp,categories(Heavy Study Load,Lack of Social Engagement,Negative emotion,Sleep Problem and Diversity Issues) and geographical location.So for each filters, word cloud is visually depicted along with its frequency.This helps the management to understand where exactly the problem resides and its degree.
    \section{Conclusion and Future Work}
    Our study is contributes to various fields such as data mining, learning analytics as well as learning technologies. The usage of synonyms of frequently used words increases the accuracy of the classifier.
    The study also benefits educational administrators since it clearly finds the issues students face in educational institutes. Using this information, administrators can rework curriculum, evaluation schemes, or course plans.
    From the work presented above, to the best of our knowledge, we can conclude that the Naïve Bayes’ Classifier gives us the best accuracy for classifying students’ problems. It also has a lesser execution time compared to other classifiers.
    The results shown above state that most students face the problem of heavy workload and sleep issues. These problems may be interrelated. If these problems are taken into consideration by educational institute administrators, it could possibly lead to the betterment of students' performance.
    We can clearly observe from our study that the social media is a large untapped resource for data analytics, and a significant amount of meaningful information can be obtained from it.
    For future work, we can work to analyse the students’ positive sentiments also, and draw conclusions as to what factors into students’ happiness as well. Other social media other than twitter can be used for analysis as well.
    Finally, the workflow we proposed requires human effort for data analysis and interpretation. This is necessary because our purpose is to achieve deeper understanding of the student experiences. To the best of our knowledge, there is currently no unsupervised automatic natural language processing technique that can achieve the depth of understanding that we were able to achieve. There is a trade-off between the amount of human effort and the depth of the understanding.


    \ifCLASSOPTIONcaptionsoff
     \newpage
    \fi



    \begin{thebibliography}{1}

    \bibitem{} Xin Chen, Mihaela Vorvoreanu, Krishna Madhavan"Mining Social Media Data for Understanding Students Learning Experiences"

    \bibitem{} “Using the Twitter Search API | Twitter Developers,” \texttt{ https://
    dev.twitter.com/docs/using-search}
    \bibitem{}
    WordNet Synonym Sets
    \texttt{www.nltk.org/howto/wordnet.html}
    \bibitem{}LibSVM
    \texttt{www.csie.ntu.edu.tw/~cjlin/libsvm/}
    \end{thebibliography}


    \end{document}