Working with large datasets
We have our own binary format
Also use parquet files
streaming through them
What is best practice, just one of the examples?

A:
DataFrame with timeseries (microsecond precision)
http://localhost:8889/notebooks/03.11-Working-with-Time-Series.ipynb
Separate data into files e.g. ~100MB, based on start/stop timeseries
pandas_datareader - have a look at how they pull down (web) data based on timeseries
- this uses caches, other versions which dont exist