Advertisement
Guest User

Events

a guest
Aug 28th, 2015
73
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Scala 0.47 KB | None | 0 0
  1. sealed trait Event
  2. case class ClickEvent(blaBla: String) extends Event
  3. case class ViewEvent(blaBla: String) extends Event
  4.  
  5. val input: RDD[String] = sc.textFile("logs/events/*")
  6. val events: RDD[Event] = input.parse
  7.  
  8. val views: RDD[ViewEvent] = events.flatMap {
  9.     case v: ViewEvent => Some(v)
  10.     case _ => None
  11. }
  12.  
  13. val clicks: RDD[ClickEvent] = events.flatMap {
  14.     case c: ClickEvent => Some(c)
  15.     case _ => None
  16. }
  17.  
  18. views.saveAsTextFile("views")
  19. clicks.saveAsTextFile("clicks")
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement