{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from nltk.tokenize import RegexpTokenizer\n",
    "from stop_words import get_stop_words\n",
    "from nltk.stem.porter import PorterStemmer\n",
    "from gensim import corpora, models\n",
    "import gensim\n",
    "import re\n",
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Reading data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we'll create a data structure to store all the information about the different ToS documents with which we'll train our LDA model, to get the different topics.\n",
    "\n",
    "The information will be cleaned (removing stop-words, stemmed, etc.) and stores as a list of paragraphs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#utility function to read files (ToS) to an list of paragraphs.\n",
    "def read_file_to_paragraphs(file_path):\n",
    "    file = open(file_path, 'r')\n",
    "    doc = file.read()\n",
    "    file.close()\n",
    "    pars = re.split('\\n\\n+', doc) #some documents have a end of line during what could be considered the same \n",
    "                                    #paragraph, I believe 2 or more \\n's if a better slip for this.\n",
    "    print('reading %s wich have %d paragraphs' % (file_path, len(pars)))\n",
    "    return(pars)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Reading all the files..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "reading ./data/twitter_tos.txt wich have 80 paragraphs\n"
     ]
    }
   ],
   "source": [
    "pars = read_file_to_paragraphs('./data/twitter_tos.txt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "reading ./data/facebook_tos.txt wich have 35 paragraphs\n",
      "reading ./data/github_tos.txt wich have 60 paragraphs\n",
      "reading ./data/google_privacy_tos.txt wich have 71 paragraphs\n",
      "reading ./data/google_tos.txt wich have 52 paragraphs\n",
      "reading ./data/snaptchat_tos.txt wich have 83 paragraphs\n",
      "reading ./data/squarespace_tos.txt wich have 153 paragraphs\n",
      "reading ./data/youtube_tos.txt wich have 13 paragraphs\n"
     ]
    }
   ],
   "source": [
    "pars.extend(read_file_to_paragraphs('./data/facebook_tos.txt'))\n",
    "pars.extend(read_file_to_paragraphs('./data/github_tos.txt'))\n",
    "pars.extend(read_file_to_paragraphs('./data/google_privacy_tos.txt'))\n",
    "pars.extend(read_file_to_paragraphs('./data/google_tos.txt'))\n",
    "pars.extend(read_file_to_paragraphs('./data/snaptchat_tos.txt'))\n",
    "pars.extend(read_file_to_paragraphs('./data/squarespace_tos.txt'))\n",
    "pars.extend(read_file_to_paragraphs('./data/youtube_tos.txt'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "547"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(pars)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll train our model with 547 paragraphs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "tokenizer = RegexpTokenizer(r'\\w+')\n",
    "en_stop = get_stop_words('en')\n",
    "p_stemmer = PorterStemmer()\n",
    "\n",
    "#utility function that will tokenize, remove stop-words and stem the paragraphs.\n",
    "def tokenize_and_stem(text, tokenizer, stemmer, stop_words):\n",
    "    return([stemmer.stem(word) for word in tokenizer.tokenize(text.lower()) if word not in stop_words])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "norm_texts = [tokenize_and_stem(par, tokenizer, p_stemmer, en_stop) for par in pars]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An example of the normalized text:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[['skip',\n",
       "  'main',\n",
       "  'content',\n",
       "  'twitter',\n",
       "  'languag',\n",
       "  'english',\n",
       "  'sign',\n",
       "  'download',\n",
       "  'thetwitteruseragr',\n",
       "  'pdf'],\n",
       " ['live',\n",
       "  'unit',\n",
       "  'state',\n",
       "  'twitter',\n",
       "  'user',\n",
       "  'agreement',\n",
       "  'compris',\n",
       "  'term',\n",
       "  'servic',\n",
       "  'privaci',\n",
       "  'polici',\n",
       "  'twitter',\n",
       "  'rule',\n",
       "  'incorpor',\n",
       "  'polici']]"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "norm_texts[:2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Create Dictionary object"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To train an LDA model, we first need to map words to numeric ids with a _Dictionary_ model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "gensim.corpora.dictionary.Dictionary"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# turn our tokenized documents into a id <-> term dictionary\n",
    "dictionary = corpora.Dictionary(norm_texts)\n",
    "type(dictionary)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#saving the dictionary object to used later (in the web app).\n",
    "dictionary.save('lda_dictionary')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Create BOW object"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also we'll need a bag-of-words representation of our text to train the LDA model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "547"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# convert tokenized documents into a document-term matrix\n",
    "bows = [dictionary.doc2bow(text) for text in norm_texts]\n",
    "len(bows)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An example of how this bow structure looks like (for the first paragraph):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(0, 1),\n",
       " (1, 1),\n",
       " (2, 1),\n",
       " (3, 1),\n",
       " (4, 1),\n",
       " (5, 1),\n",
       " (6, 1),\n",
       " (7, 1),\n",
       " (8, 1),\n",
       " (9, 1)]"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bows[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Train the LDA model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we're ready to train our LDA model. Reading through the documents and trying different parameters we concluded that a good number of topics to set our model with would be 10 (we're classifying our text in only 5 categories, but with this we're saying that the complete documents have (an average) of 10 different topics. When we tried 15 or 20 the topics seem to repeated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(0,\n",
       "  '0.059*\"servic\" + 0.027*\"content\" + 0.024*\"use\" + 0.020*\"term\" + 0.018*\"will\" + 0.015*\"parti\" + 0.014*\"third\" + 0.012*\"damag\" + 0.012*\"includ\" + 0.011*\"access\"'),\n",
       " (1,\n",
       "  '0.037*\"account\" + 0.036*\"twitter\" + 0.021*\"servic\" + 0.018*\"use\" + 0.017*\"term\" + 0.015*\"com\" + 0.012*\"may\" + 0.010*\"inform\" + 0.010*\"provid\" + 0.010*\"agreement\"'),\n",
       " (2,\n",
       "  '0.051*\"privaci\" + 0.049*\"polici\" + 0.034*\"inform\" + 0.033*\"use\" + 0.029*\"servic\" + 0.025*\"term\" + 0.023*\"payment\" + 0.021*\"user\" + 0.019*\"end\" + 0.019*\"site\"'),\n",
       " (3,\n",
       "  '0.072*\"inform\" + 0.046*\"use\" + 0.039*\"googl\" + 0.026*\"servic\" + 0.023*\"share\" + 0.017*\"may\" + 0.017*\"content\" + 0.017*\"facebook\" + 0.016*\"advertis\" + 0.016*\"collect\"'),\n",
       " (4,\n",
       "  '0.037*\"servic\" + 0.019*\"chang\" + 0.017*\"domain\" + 0.017*\"renew\" + 0.017*\"may\" + 0.016*\"13\" + 0.016*\"term\" + 0.016*\"copyright\" + 0.015*\"fee\" + 0.014*\"notic\"'),\n",
       " (5,\n",
       "  '0.023*\"right\" + 0.021*\"account\" + 0.021*\"will\" + 0.017*\"s\" + 0.015*\"may\" + 0.015*\"person\" + 0.014*\"state\" + 0.014*\"user\" + 0.013*\"inform\" + 0.013*\"includ\"'),\n",
       " (6,\n",
       "  '0.045*\"term\" + 0.042*\"arbitr\" + 0.039*\"will\" + 0.026*\"servic\" + 0.026*\"snap\" + 0.025*\"inc\" + 0.019*\"agreement\" + 0.017*\"provis\" + 0.016*\"right\" + 0.014*\"use\"'),\n",
       " (7,\n",
       "  '0.061*\"servic\" + 0.037*\"content\" + 0.028*\"use\" + 0.023*\"may\" + 0.016*\"right\" + 0.014*\"youtub\" + 0.014*\"provid\" + 0.013*\"copyright\" + 0.011*\"user\" + 0.010*\"infring\"'),\n",
       " (8,\n",
       "  '0.030*\"servic\" + 0.021*\"account\" + 0.020*\"may\" + 0.014*\"will\" + 0.013*\"googl\" + 0.013*\"us\" + 0.013*\"parti\" + 0.013*\"facebook\" + 0.011*\"law\" + 0.011*\"respons\"'),\n",
       " (9,\n",
       "  '0.029*\"ecommerc\" + 0.019*\"parti\" + 0.019*\"agreement\" + 0.018*\"payment\" + 0.018*\"may\" + 0.017*\"squarespac\" + 0.017*\"third\" + 0.017*\"privaci\" + 0.016*\"processor\" + 0.015*\"us\"')]"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ldamodel = gensim.models.ldamodel.LdaModel(bows, num_topics=10, id2word = dictionary, passes=20)\n",
    "ldamodel.print_topics()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll save the model for later use (in our web app)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ldamodel.save('lda_model')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Running the model in a new doc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above topics appear to make sense, but they are not labeled nicely for our use. For example `(1,\n",
    "  '0.057*\"servic\" + 0.036*\"inform\" + 0.036*\"use\" + 0.023*\"privaci\" + 0.022*\"polici\" + 0.015*\"provid\" + 0.013*\"access\" + 0.013*\"term\" + 0.012*\"user\" + 0.010*\"collect\"')` seem to be talking about privacy and use of information. We'll need to take these topics and mark them with a more readable label.\n",
    "\n",
    "In the following dictionary we define the topics of our interest and wich words below to those topics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#this topic list can be expanded with more topics and more words related to those topics.\n",
    "topic_dic = {'privacy': ['privacy'], 'copyright': ['copyright'], 'content sharing/use': ['share'], 'cancelation/termination': ['cancelation', 'termination'], 'modification/pricing': ['modification', 'pricing'], 'special': ['law', 'jurisdiction', 'governing']}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we need to use our trained model to analyse a new document, for that we'll need to apply the model to each paragraph to find the most relevant topic and label that paragraph. We'll create a list element will be another list with the original paragraph (not cleaned) and the label, to analyze the paragraph with LDA we do clean the words in the text."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def create_topic_pars(pars, tokenizer, stemmer, stop_words, ldamodel, word_dictionary, topic_dictionary):\n",
    "    norm_pars = [tokenize_and_stem(par, tokenizer, stemmer, stop_words) for par in pars]\n",
    "    print('created normalized paragraphs object of length %d' % len(norm_pars))\n",
    "    bows = [word_dictionary.doc2bow(text) for text in norm_pars]\n",
    "    print('created bag-of-words object of length %d' % len(bows))\n",
    "    topic_pars = []\n",
    "    for idx, val in enumerate(bows):\n",
    "        lda_vector = ldamodel[val]\n",
    "        #original LDA model topic (most relevant) and paragraph:\n",
    "        topic_pars.append([ldamodel.print_topic(max(lda_vector, key=lambda item: item[1])[0]), pars[idx]]) #we attach the original paragraph here, not the cleaned version that we used for LDA.\n",
    "    \n",
    "    #now we'll create a nicely labeled structure.\n",
    "    tagged_pars = []\n",
    "    for topic_name in topic_dictionary:\n",
    "        topic_words = topic_dictionary[topic_name]\n",
    "        for pars in topic_pars:\n",
    "            topic = pars[0]\n",
    "            par = pars[1]\n",
    "            if(len(par) > 50):\n",
    "                for word in topic_words:\n",
    "                    if stemmer.stem(word) in topic:\n",
    "                        tagged_pars.append([par, topic_name])\n",
    "                        break\n",
    "    return(tagged_pars)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll use Medium's ToS to try our model (obviously this was not part of the training data):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "reading ./data/medium_tos.txt wich have 32 paragraphs\n"
     ]
    }
   ],
   "source": [
    "new_pars = read_file_to_paragraphs('./data/medium_tos.txt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "created normalized paragraphs object of length 32\n",
      "created bag-of-words object of length 32\n"
     ]
    }
   ],
   "source": [
    "topic_pars = create_topic_pars(new_pars, tokenizer, p_stemmer, en_stop, ldamodel, dictionary, topic_dic)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[['These Terms of Service (“Terms”) are a contract between you and A Medium Corporation. They govern your use of Medium’s sites, services, mobile apps, products, and content (“Services”). By using Medium, you agree to these Terms. If you don’t agree to any of the Terms, you can’t use Medium. We can change these Terms at any time. We keep a historical record of all changes to our Terms on GitHub. If a change is material, we’ll let you know before they take effect. By using Medium on or after that effective date, you agree to the new Terms. If you don’t agree to them, you should delete your account before they take effect, otherwise your use of the site and content will be subject to the new Terms.',\n",
       "  'copyright'],\n",
       " ['You own the rights to the content you create and post on Medium.',\n",
       "  'copyright'],\n",
       " ['By posting content to Medium, you give us a nonexclusive license to publish it on Medium Services, including anything reasonably related to publishing it (like storing, displaying, reformatting, and distributing it). In consideration for Medium granting you access to and use of the Services, you agree that Medium may enable advertising on the Services, including in connection with the display of your content or other information. We may also use your content to promote Medium, including its products and content. We will never sell your content to third parties without your explicit permission.',\n",
       "  'copyright'],\n",
       " ['You’re responsible for the content you post. This means you assume all risks related to it, including someone else’s reliance on its accuracy, or claims relating to intellectual property or other legal rights.',\n",
       "  'copyright'],\n",
       " ['You’re welcome to post content on Medium that you’ve published elsewhere, as long as you have the rights you need to do so. By posting content to Medium, you represent that doing so doesn’t conflict with any other agreement you’ve made.',\n",
       "  'copyright'],\n",
       " ['By posting content you didn’t create to Medium, you are representing that you have the right to do so. For example, you are posting a work that’s in the public domain, used under license (including a free license, such as Creative Commons), or a fair use.',\n",
       "  'copyright'],\n",
       " ['You can delete any of your posts, or your account, anytime. Processing the deletion may take a little time, but we’ll do it as quickly as possible. We may keep backup copies of your deleted post or account on our servers for up to 14 days after you delete it.',\n",
       "  'copyright'],\n",
       " ['We reserve all rights in Medium’s look and feel. Some parts of Medium are licensed under third-party open source licenses. We also make some of our own code available under open source licenses. As for other parts of Medium, you may not copy or adapt any portion of our code or visual design elements (including logos) without express written permission from Medium unless otherwise permitted by law.',\n",
       "  'copyright'],\n",
       " ['We may change, terminate, or restrict access to any aspect of the service, at any time, without notice.',\n",
       "  'copyright'],\n",
       " ['Medium is only for people 13 years old and over. By using Medium, you affirm that you are over 13. If we learn someone under 13 is using Medium, we’ll terminate their account.',\n",
       "  'copyright'],\n",
       " ['To enable a functioning community, we have Rules. To ensure usernames are distributed and used fairly, we have a Username Policy. Under our DMCA Policy, we’ll remove material after receiving a valid takedown notice. Under our Trademark Policy, we’ll investigate any use of another’s trademark and respond appropriately.',\n",
       "  'copyright'],\n",
       " ['By using Medium, you agree to follow these Rules and Policies. If you don’t, we may remove content, or suspend or delete your account.',\n",
       "  'copyright'],\n",
       " ['Disclaimer of warranty. Medium provides the Services to you as is. You use them at your own risk and discretion. That means they don’t come with any warranty. None express, none implied. No implied warranty of merchantability, fitness for a particular purpose, availability, security, title or non-infringement.',\n",
       "  'copyright'],\n",
       " ['By using the Services, you agree to let Medium collect and use information as detailed in our Privacy Policy. If you’re outside the United States, you consent to letting Medium transfer, store, and process your information (including your personal information and content) in and out of the United States.',\n",
       "  'privacy']]"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "topic_pars"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The result appear to make sense. To test the validity of our model we'll survey the result of several of these ToSs on different people and we'll get their opinions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This part is the same as above but it was used to tried if we could run our model from the different saved parts, to use it in our web app."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Running a saved model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "gensim.corpora.dictionary.Dictionary"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dictionary2 = corpora.Dictionary.load('lda_dictionary')\n",
    "type(dictionary2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "gensim.models.ldamodel.LdaModel"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ldamodel2 = gensim.models.ldamodel.LdaModel.load('lda_model')\n",
    "type(ldamodel2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "created normalized paragraphs object of length 32\n",
      "created bag-of-words object of length 32\n"
     ]
    }
   ],
   "source": [
    "topic_pars2 = create_topic_pars(new_pars, tokenizer, p_stemmer, en_stop, ldamodel2, dictionary2, topic_dic)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[['These Terms of Service (“Terms”) are a contract between you and A Medium Corporation. They govern your use of Medium’s sites, services, mobile apps, products, and content (“Services”). By using Medium, you agree to these Terms. If you don’t agree to any of the Terms, you can’t use Medium. We can change these Terms at any time. We keep a historical record of all changes to our Terms on GitHub. If a change is material, we’ll let you know before they take effect. By using Medium on or after that effective date, you agree to the new Terms. If you don’t agree to them, you should delete your account before they take effect, otherwise your use of the site and content will be subject to the new Terms.',\n",
       "  'copyright'],\n",
       " ['You own the rights to the content you create and post on Medium.',\n",
       "  'copyright'],\n",
       " ['By posting content to Medium, you give us a nonexclusive license to publish it on Medium Services, including anything reasonably related to publishing it (like storing, displaying, reformatting, and distributing it). In consideration for Medium granting you access to and use of the Services, you agree that Medium may enable advertising on the Services, including in connection with the display of your content or other information. We may also use your content to promote Medium, including its products and content. We will never sell your content to third parties without your explicit permission.',\n",
       "  'copyright'],\n",
       " ['You’re responsible for the content you post. This means you assume all risks related to it, including someone else’s reliance on its accuracy, or claims relating to intellectual property or other legal rights.',\n",
       "  'copyright'],\n",
       " ['You’re welcome to post content on Medium that you’ve published elsewhere, as long as you have the rights you need to do so. By posting content to Medium, you represent that doing so doesn’t conflict with any other agreement you’ve made.',\n",
       "  'copyright'],\n",
       " ['By posting content you didn’t create to Medium, you are representing that you have the right to do so. For example, you are posting a work that’s in the public domain, used under license (including a free license, such as Creative Commons), or a fair use.',\n",
       "  'copyright'],\n",
       " ['You can delete any of your posts, or your account, anytime. Processing the deletion may take a little time, but we’ll do it as quickly as possible. We may keep backup copies of your deleted post or account on our servers for up to 14 days after you delete it.',\n",
       "  'copyright'],\n",
       " ['We reserve all rights in Medium’s look and feel. Some parts of Medium are licensed under third-party open source licenses. We also make some of our own code available under open source licenses. As for other parts of Medium, you may not copy or adapt any portion of our code or visual design elements (including logos) without express written permission from Medium unless otherwise permitted by law.',\n",
       "  'copyright'],\n",
       " ['We may change, terminate, or restrict access to any aspect of the service, at any time, without notice.',\n",
       "  'copyright'],\n",
       " ['Medium is only for people 13 years old and over. By using Medium, you affirm that you are over 13. If we learn someone under 13 is using Medium, we’ll terminate their account.',\n",
       "  'copyright'],\n",
       " ['To enable a functioning community, we have Rules. To ensure usernames are distributed and used fairly, we have a Username Policy. Under our DMCA Policy, we’ll remove material after receiving a valid takedown notice. Under our Trademark Policy, we’ll investigate any use of another’s trademark and respond appropriately.',\n",
       "  'copyright'],\n",
       " ['By using Medium, you agree to follow these Rules and Policies. If you don’t, we may remove content, or suspend or delete your account.',\n",
       "  'copyright'],\n",
       " ['Disclaimer of warranty. Medium provides the Services to you as is. You use them at your own risk and discretion. That means they don’t come with any warranty. None express, none implied. No implied warranty of merchantability, fitness for a particular purpose, availability, security, title or non-infringement.',\n",
       "  'copyright'],\n",
       " ['By using the Services, you agree to let Medium collect and use information as detailed in our Privacy Policy. If you’re outside the United States, you consent to letting Medium transfer, store, and process your information (including your personal information and content) in and out of the United States.',\n",
       "  'privacy']]"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "topic_pars2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [Root]",
   "language": "python",
   "name": "Python [Root]"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}