This week only. Pastebin PRO Accounts Christmas Special! Don't miss out!Want more features on Pastebin? Sign Up, it's FREE!
Guest

Untitled

By: rcompton on Mar 28th, 2013  |  syntax: None  |  size: 0.44 KB  |  views: 41  |  expires: Never
download  |  raw  |  embed  |  report abuse  |  print
Text below is selected. Please press Ctrl+C to copy to your clipboard. (⌘+C on Mac)
  1. define langid_command_line `python2.6 langid_command_line.py` ship('/home/ryan/langid_command_line.py');
  2.  
  3. --a tab-delineated file, first column is numeric, second is text
  4. lines = LOAD '/data/text/file0.tab.csv' USING PigStorage() as (ids:chararray, text:chararray);
  5.  
  6. pythonDetect = STREAM lines through langid_command_line AS ( outid:chararray, outlanguage:chararray);
  7.  
  8. STORE pythonDetect INTO '/user/rfcompton/figurepig-dir4/' USING PigStorage();
clone this paste RAW Paste Data