Advertisement
Guest User

Untitled

a guest
Jul 7th, 2015
200
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.66 KB | None | 0 0
  1. // create the Spark configuration and context
  2. SparkConf conf = new SparkConf().setAppName("Wordcount").setMaster("local[*]");
  3.  
  4. JavaSparkContext sc = new JavaSparkContext(conf);
  5.  
  6. // load data and create an RDD of string
  7. JavaRDD<String> tweets = sc.textFile("path_To_File")
  8.  
  9. JavaPairRDD<String, Integer> wordcount = tweets.flatMap(line -> Arrays.asList(line.toString().split(" ")))
  10. // mapper step
  11. .mapToPair(word -> new Tuple2<>(word, 1))
  12. // reducer step
  13. .reduceByKey((x, y) -> x + y);
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement