Digging for Dummies 2.0

DIGGING FOR DUMMIES (Revision 2.0)
~boogiepoprobin (robingale) on 12/02/15.


:Introduction:
The act of digging is a matter of following statements, facts and connections to arrive at deeper conclusions regarding someone, something, or an event. By observing and noting particular pieces of information, we can arrive at a more detailed view of what we are investigating. This allows for a more thorough understanding into why an event took place, why someone acts the way they do, or will provide conclusions regarding specific actions someone has taken.

What will follow is a breakdown of the mindset, application, services and tools I use for digging. We will begin with what I think is the proper mindset and move forward into practical approach, and finally into various services and tools to use. This is meant to act as a guide for those who are interested either getting started in digging or for those who wish to expand their methodology.


:The Appropriate Mindset:
You should be subtle and quiet when it comes to investigation. Avoid revealing any indication that you are investigating as people who may be involved could affect the investigation. If you reveal that you are observing them they may start deleting valuable information before you have a chance to get to it. Remember, they know more about themselves than you do. Don't get cocky. Cultivate humility and grace, be quiet and focused. The less anyone knows about what you are doing, the more you are able to do.

Details. Details are everything. Look for facts, statements of intent, specific opinions. You're looking for "information". Details are your branches. Look for details that convey information. Use that information as bridges for new avenues to search. You aren't just looking at what's stated, but what is unstated. Details are often built on other logically necessary details. Take all the details you've found and build a model, see if you can find any inconsistencies or holes. Look there. Digging is about processing information so that others can use. You are building "knowledge" by sifting through "information".

For that reason, it is important that the information is presented clearly, simply and with context so that the reader may understand "why". You are not dealing with machines here. You are dealing with people. Information is meaningless without reasoning, without meaning. Why? Because people don't think in terms of numbers and calculations. They are emotional. They seek meaning, purpose. They need reasons why. Reasons, meaning, significance provide a foundation for all the rest of the information. They aren't easily forgotten facts now, they "matter".

Expect to spend a large number of hours on digging if you want results. You will be sifting through quite a bit of useless information before you find anything usable. This is normal. You may go days and even weeks without finding anything worthwhile. This is also normal.


:On The Dig Itself:
The key to digging is the anchor. This is your target. Random digging without any purpose can produce results, but a focused approach can be more efficient and productive.

Decide on a target and then map their network. Understand their movements and the various circles they associate with. Observe connections between those they have a financial or personal relationship with, and from those particular connections investigate further.

You should avoid approaching a target or anyone associated with them UNTIL you have exhausted AND archived ALL of the information you can find. Be patient. They will most likely still be there when you're finished gathering the information, but the information MAY NOT be there if you prematurely tip your hand to the target or people in connection to the target. (Given, of course, that the investigation may be negative to them. I doubt you need to take the same precautions for a clean search.)

As you compile statements, facts and other observations a model will begin to develop. The observations will either support or contradict each other. Contradictions in the model are very important and should be noted for further investigation. Take note of where holes in the model are revealed and pursue those as well. Your goal should always be the most complete and accurate model that you can create BASED on what has been found.

When it is time to compile the information, here are some suggestions on an appropriate format. First of all, the simpler the format, the better. An overly complicated design can confuse and distract. Present your summary first to establish the context. This will inform the reader of the purpose behind the information and the conclusions reached. The difference between the reader fully understanding the context or not will determine how well they can incorporate your observations. Furthermore, avoid creating giant walls of text paragraphs. Give your reader some breathing room to process information. You don't eat a steak in one bite, don't expect people to consume a 20 line paragraph without trouble either.


:Services To Dig Through:
There are several platforms which are useful in digging. Twitter, facebook, LinkedIn, patreon, pastebin, message boards, google groups and youtube can all be useful depending on how utilized they are by the individual. Personal websites are another very valuable resource in digging and should be one of the first sought out as they typically contain very detailed information into the individual's activities and personality.

Google is your best friend. You will use much of your time simply googling names. For example, your target has a patreon and one of the patrons is a developer. You take note of how long that developer has been donating and then google both names ("John Smith" AND "Sheryl Lin"). You take note of the cross-references and look for particular facts. Did John Smith write an article on Sheryl Lin at any point? If so, was the article written after donations began?

I also recommend searching pastebin through google rather than the built in search. I have found that results are easier to come by when using google to search through pastebin. If you want to search a particular website through google simply put in what you want to search for and then "site:pastebin.com" for example "digra site:pastebin.com".

Here is a link listing various google search operators. https://support.google.com/websearch/answer/2466433?hl=en
Archive link: https://archive.is/sfaaw

Twitter navigation is tricky as it's format does not lend itself well to digging or archiving in bulk..usually. There are ways to mitigate the shortcomings Twitter has in it's design. Advanced search will be your best friend as it allows for targeted digging with relative ease. When digging for interactions on twitter, you should defer to using mentions rather than direct replies - as mentions will often bring up information related to the referenced individuals even if they are not being directly tweeted at.

For example: "@johnsmith @janesmith" will bring up all tweets that include both twitter tags, not just direct tweets to those two individuals - where as "from:johnsmith to:janesmith" will only bring up direct tweets from @johnsmith to @janesmith".

It is also useful when digging in twitter to isolate your digging efforts. By that I mean to isolate the mentions to certain periods of time. This is very useful as twitter's design means that your searches can scale back several years and be nearly impossible to load as it will eventually strain your browser's memory load. If you want to search two people with an enormous amount of interaction, you should specify by three or four month long "chunks". This is what I have found to be the optimum amount of tweets to dig through at any time. Your advanced search will look something like this if you do this correctly "@johnsmith @janesmith since:2013-12-31 until:2014-03-31". This will bring you every mention including both of those twitter tags from December 31st, 2013 to March 31st, 2014.

NOTE: For twitter advanced search, you will need to perform a regular search using the top right-hand side bar and then select "more options > advanced search" to actually access it from twitter. Here is the direct link to advanced search: https://twitter.com/search-advanced (I suggest bookmarking it)

LinkedIn is useful for finding dates when an individual held certain professional positions. This can help determine the context behind conversations by understanding the backgrounds of those involved.

Another, often overlooked, service is youtube. Whether it's pre-recorded videos or streams, people tend to be chatty in these videos and reveal plenty of useful information. If you wish to be competent then you will take advantage of this and watch those videos. I suggest making a transcript file that includes the time-stamps of various comments which can be useful for reference in the future. A website that you will find useful when it comes to saving youtube videos is called keepvid.com. You can save a stream in progress by pasting the youtube link into vlc, selecting "tools" and finally "codec information". There will be a long link at the bottom that you can paste into your browser. This will give you a file that you can download immediately. The quality will be poor, but you will at least have a copy.


:On Archiving:
Every competent digger needs to archive and archive often. The words "archive everything" should be etched into your mind. If it is even remotely useful, archive it. Not only are several archive services available, but it is best if you archive from several different services. I personally recommend archiving from archive.org, archive.is and also from virtual-notary.org.

Although tweetsave is a useful tool, be aware that if you are bulk archiving using tweetsave - it can alert a target to your activities as their twitter icon and handle will appear on the main page. For that reason I don't recommend using tweetsave when you will be archiving a lot of evidence from one particular source.

As stated above, keepvid.com is great for saving youtube videos. It provides multiple formats for archival including webm.


:Resources:
keepvid.com			Allows you to save youtube videos.
archive.is			Website archiving service
archive.org			Another website archiving service
virtual-notary.org		Allows for encrypted offline archives of webpages, tweets, documents.
copyscape.com			Website that allows you to compare text or webpage for plagiarism.
timeanddate.com			Allows you to add or subtract days from a date.
doesfollow.com			Check if two people follow each other on twitter.
pastebin.com			Website to upload your documents for public viewing.
ghostbin.com			Another website for document uploading.