SHARE
TWEET

Untitled

a guest Jun 18th, 2019 78 Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. import numpy as np
  2. import pandas as pd
  3. import datetime
  4. import os
  5.  
  6. from azure.datalake.store import core, lib, multithread
  7.  
  8. USERNAME = "xxxx"
  9. PASSWORD = "xxxx"
  10. TENANT_ID = "xxxx"
  11. STORE_NAME = "xxxx"
  12.  
  13. token = lib.auth(TENANT_ID, USERNAME, PASSWORD)
  14. adl = core.AzureDLFileSystem(token, store_name=STORE_NAME)
  15.  
  16. #EXAMPLE 1 - OPENING A CSV FILE INTO A DATA FRAME - WORKS FINE
  17. f = adl.open('/path/filename1.txt')
  18. df = pd.read_csv(f,dtype=str, sep='|')
  19. df[:3]
  20.  
  21. #EXAMPLE 2 - OPENING A FLAT FIXED WIDTH FILE INTO A DATA FRAME - DOES NOT WORK
  22. f1 = adl.open('/path/filename2.txt')
  23. df2 = pd.read_fwf(f1)
  24. df2[:3]
RAW Paste Data
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy. OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!
 
Top