redditTitleInversionProjectIdea

On reddit.com/r/AskReddit sometimes people post a question like "What is usual in Europe but unusual in America?" If that post does fairly well, others will post a new question that's the inversion of the first question; for example, "What is usual in America but unusual in Europe?" I'm wondering if you could create a model that would make predictions related to this event. A few ideas: given a question, predict the chance it will have an inversion posted; given a question, predict its inverse that could be posted; or given all the questions within a time period, determine which posts are tied to each other in this way. Seems like it'd be a cool problem that would involve some Natural Language Processing (NLP) stuff, and gathering data won't be very hard: https://files.pushshift.io/reddit provides archives of Reddit posts, comments, subreddits, etc organized by month; it's just a matter of downloading the (rather large) files and extracting the relevant data.

I suspect that the task of, given a question, predicting a possible 'inverse' question should be the easiest to get something out for. I won't lie; I don't think this'll be a particularly easy problem, but there's a lot to be learned from it. A lot of machine learning is just messing around with the data and seeing what you can get out of it, so I don't think formal training is vital to being able to help out.

Concretely, what we'll be trying to do is to take post titles and use Natural Language Processing libraries to process them into a parse tree that explains how each word in the sentence is related. From there, we will probably look at the parse trees for some of the titles and try to figure out what words need to change to create the 'inverse'. The biggest problem I expect is finding 'training data': examples of the titles of posts, along with the titles of their inverse posts. This may require some manual poking around to find examples; going week by week or day by day and only looking at posts with at least a specific karma score should help here.

Technologies that may/will be used:
- Python: Python is the premier data science/machine learning language.
- SpaCy: Python library that provides Natural Language Processing tools.
- NumPy: Python library that provides efficient numeric data storage; used by almost all Python machine learning libraries under the hood.
- Pandas: Python library that provides tools for manipulating large datasets such as ours.
- SciKit-Learn: Python library used for general purpose machine learning.
- TensorFlow: Python library used for general purpose machine learning.