Advertisement
Guest User

Untitled

a guest
Jun 25th, 2017
59
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.65 KB | None | 0 0
  1. import boto3
  2. from StringIO import StringIO
  3.  
  4. def write_df_to_s3(df, s3_bucket, s3_key, encoding='utf-8', index=False, **kwargs):
  5. """
  6. Writes a pandas dataframe to a csv on AWS S3
  7.  
  8. Arguments:
  9. df (pd.DataFrame): dataframe to write
  10. s3_bucket (str): s3 bucket name, e.g. "my-bucket"
  11. s3_key (str): s3 bucket key to write file as, e.g. "data/myfile.csv"
  12. kwargs: arguments to pass to pd.DataFrame.to_csv()
  13. """
  14. csv_buffer = StringIO()
  15. df.to_csv(csv_buffer, encoding=encoding, index=index, **kwargs)
  16. s3_resource = boto3.resource('s3')
  17. s3_resource.Object(s3_bucket, s3_key).put(Body=csv_buffer.getvalue())
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement