SHARE
TWEET

Untitled

a guest Jun 26th, 2019 61 Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. x=sc.textFile("/home/amel/1MB").filter(!_.contains("NULL"))
  2. .filter(!_.contains("Null"))
  3. val re = x.map(row => {
  4. val cols = row.split(",")
  5. val Cycle = cols(2)
  6. val Duration = Cycle match {
  7. case "Licence" => "3 years"
  8. case "Master" => "2 years"
  9. case "Ingéniorat" => "5 years"
  10. case "Ingeniorat" => "5 years"
  11. case "Doctorat" => "3 years"
  12. case _ => "NULL"
  13. }
  14. (cols(1).split("-")(0) + "," + Cycle + "," + Duration + "," +
  15. cols(3), 1)
  16. }).reduceByKey(_ + _)
  17. re.collect.foreach(println)
  18. }
RAW Paste Data
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy. OK, I Understand
 
Top