daily pastebin goal
88%
SHARE
TWEET

Untitled

a guest Jan 19th, 2018 40 Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. val avgfuel = events
  2.     .groupBy(window($"enqueuedTime", "30 seconds"), $"weatherCondition")    
  3.     .agg(avg($"fuelEfficiencyPercentage") as "avg_fuel")    
  4.  
  5. val joined = events.join(avgfuel, Seq("weatherCondition"))
  6.     .filter($"fuelEfficiencyPercentage" > $"avg_fuel")
  7.  
  8. val streamingQuery1 = joined.writeStream
  9.     .outputMode("append").
  10.     .trigger(Trigger.ProcessingTime("10 seconds")).
  11.     .option("checkpointLocation", checkpointLocation).
  12.     .format("json").option("path", containerOutputLocation).start()
RAW Paste Data
Top