Guest User

Untitled

a guest
Jan 19th, 2018
61
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.51 KB | None | 0 0
  1. val avgfuel = events
  2. .groupBy(window($"enqueuedTime", "30 seconds"), $"weatherCondition")
  3. .agg(avg($"fuelEfficiencyPercentage") as "avg_fuel")
  4.  
  5. val joined = events.join(avgfuel, Seq("weatherCondition"))
  6. .filter($"fuelEfficiencyPercentage" > $"avg_fuel")
  7.  
  8. val streamingQuery1 = joined.writeStream
  9. .outputMode("append").
  10. .trigger(Trigger.ProcessingTime("10 seconds")).
  11. .option("checkpointLocation", checkpointLocation).
  12. .format("json").option("path", containerOutputLocation).start()
Add Comment
Please, Sign In to add comment