** Pastebin PRO Accounts Easter Special ** Get 40% discount for a limited time only! Click Here to check it out ;-)Want more features on Pastebin? Sign Up, it's FREE!


By: rcompton on Mar 28th, 2013  |  syntax: None  |  size: 0.44 KB  |  views: 42  |  expires: Never
download  |  raw  |  embed  |  report abuse  |  print
Text below is selected. Please press Ctrl+C to copy to your clipboard. (⌘+C on Mac)
  1. define langid_command_line `python2.6 langid_command_line.py` ship('/home/ryan/langid_command_line.py');
  3. --a tab-delineated file, first column is numeric, second is text
  4. lines = LOAD '/data/text/file0.tab.csv' USING PigStorage() as (ids:chararray, text:chararray);
  6. pythonDetect = STREAM lines through langid_command_line AS ( outid:chararray, outlanguage:chararray);
  8. STORE pythonDetect INTO '/user/rfcompton/figurepig-dir4/' USING PigStorage();
clone this paste RAW Paste Data