Advertisement
Guest User

Untitled

a guest
Jun 26th, 2019
74
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.72 KB | None | 0 0
  1. p 2019-06-05 18:53:20 ustctdtus-xx-bbthti-0b1tvlt20190605.1833tvtt20190605.1833tvst20190508
  2. p 2019-06-05 18:53:20 ustctdtus-xx-bbthti-03atvlt20190605.1833tvtt20190605.1833
  3. p 2019-06-05 18:53:20 ustctdtus-xx-bbthti-030
  4.  
  5. log_type time_stamp us d h vl vt vs
  6. p 2019-06-05 18:53:20 c us-xx-bb 0b1 20190605.1833 20190605.1833 20190508
  7. p 2019-06-05 18:53:20 c us-xx-bb 03a 20190605.1833 20190605.1833
  8. p 2019-06-05 18:53:20 c us-xx-bb 030
  9.  
  10. val logSchema = new StructType().add("log_type",StringType).add("time_stamp",StringType).add("kvs",StringType)
  11. val logDF = spark.read.option("delimiter", "t").format("com.databricks.spark.csv").schema(logSchema).load("/tmp/log.tsv")
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement