Advertisement
Guest User

Untitled

a guest
Oct 4th, 2017
81
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.62 KB | None | 0 0
  1. `REGISTER hdfs:////user/labuser/mysql_jar/mysql-connector-java-5.1.44-bin.jar;
  2. REGISTER /opt/cloudera/parcels/CDH/lib/pig/piggybank.jar;
  3. dataset = LOAD '/user/labuser/word_count2_eg' USING PigStorage() as (lines:chararray);
  4. words = FOREACH dataset GENERATE FLATTEN(TOKENIZE(lines)) as word;
  5. wgroup = GROUP words by word;
  6. wcount = FOREACH wgroup GENERATE group as dwords, COUNT(words) as dcount;
  7. wsort = ORDER wcount by dcount DESC;
  8. STORE wsort into 'web_rank' USING org.apache.pig.piggybank.storage.DBStorage('com.mysql.jdbc.Driver','jdbc:mysql://x.x.x.x/pig_eg','user','password','insert into web_rank(words,total)values(?,?)' );`
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement