Advertisement
Guest User

sparkshaiba

a guest
Mar 26th, 2015
414
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
SPARK 0.75 KB | None | 0 0
  1.     //take only age groups and movies
  2.                                        //userID  //age    //Set(movie, rating)  //dropped genre, because it is not needed
  3.     val ageGroupsMovies = all.map(x => (x._1, (x._2._1._3, x._2._2)))
  4.    
  5.     //take the first age group and create new RDD containing only movies and ratings (and genres, dammit)
  6.     val justMovies = ageGroupsMovies.filter(x => x._2._1 == 1).flatMap(x => x._2._2)
  7.    
  8.     val plaa2 = justMovies.map(x => (x._1, x._2)).groupByKey
  9.    
  10.     val moviesAndRatings = plaa2.map(x => (x._1, (x._2.reduce(_ + _).toFloat / x._2.size)))
  11.    
  12.     def order(arr: Array[(String, Float)]) : Int = {
  13.       return 0
  14.     }
  15.    
  16.     val plaa = moviesAndRatings.top(10)
  17.    
  18.    
  19.     plaa.foreach {println}
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement