API tools faq
paste
Advertisement
Guest User

Could not get input splits errors

a guest
Sep 20th, 2013
93
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 69.73 KB | None | 0 0
raw download clone embed print report
  1. Exec 1
  2. ------
  3.  
  4. 10:22:57 [cyril:/tmp] 6 $ ~/tmp/pig_cassandra ~/tmp/example-script.pig
  5. Using /usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar.
  6. 2013-09-20 10:23:07,794 [main] INFO org.apache.pig.Main - Apache Pig version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
  7. 2013-09-20 10:23:07,795 [main] INFO org.apache.pig.Main - Logging error messages to: /private/tmp/pig_1379665387789.log
  8. 2013-09-20 10:23:07.971 java[37700:1703] Unable to load realm info from SCDynamicStore
  9. 2013-09-20 10:23:08,171 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /Users/cyril/.pigbootup not found
  10. 2013-09-20 10:23:08,304 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://localhost:54310/
  11. 2013-09-20 10:23:08,457 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: localhost:8021
  12. data: {ise: chararray,filtre: int,value_1: int}
  13. 2013-09-20 10:23:09,347 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
  14. 2013-09-20 10:23:09,459 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
  15. 2013-09-20 10:23:09,508 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
  16. 2013-09-20 10:23:09,508 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
  17. 2013-09-20 10:23:09,605 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
  18. 2013-09-20 10:23:09,617 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
  19. 2013-09-20 10:23:09,619 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
  20. 2013-09-20 10:23:09,620 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=-1
  21. 2013-09-20 10:23:09,620 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Could not estimate number of reducers and no requested or default parallelism set. Defaulting to 1 reducer.
  22. 2013-09-20 10:23:09,620 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
  23. 2013-09-20 10:23:09,787 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - creating jar file Job6863623309771763045.jar
  24. 2013-09-20 10:23:15,539 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - jar file Job6863623309771763045.jar created
  25. 2013-09-20 10:23:15,558 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
  26. 2013-09-20 10:23:15,605 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
  27. 2013-09-20 10:23:16,107 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
  28. 2013-09-20 10:23:16,114 [JobControl] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
  29. 2013-09-20 10:23:16,855 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_201309192059_0059
  30. 2013-09-20 10:23:16,856 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases data
  31. 2013-09-20 10:23:16,856 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: data[1,7] C: R:
  32. 2013-09-20 10:23:16,856 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - More information at: http://localhost:50030/jobdetails.jsp?jobid=job_201309192059_0059
  33. 2013-09-20 10:23:25,420 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 50% complete
  34. 2013-09-20 10:23:31,486 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
  35. 2013-09-20 10:23:31,488 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
  36.  
  37. HadoopVersion PigVersion UserId StartedAt FinishedAt Features
  38. 1.2.1 0.11.1 cyril 2013-09-20 10:23:09 2013-09-20 10:23:31 UNKNOWN
  39.  
  40. Success!
  41.  
  42. Job Stats (time in seconds):
  43. JobId Maps Reduces MaxMapTime MinMapTIme AvgMapTime MedianMapTime MaxReduceTime MinReduceTime AvgReduceTime MedianReducetime Alias Feature Outputs
  44. job_201309192059_0059 1 0 4 4 4 4 0 0 0 0 data MAP_ONLY hdfs://localhost:54310/tmp/temp-833716857/tmp1325927627,
  45.  
  46. Input(s):
  47. Successfully read 3 records (15143 bytes) from: "cql://ks1/lc"
  48.  
  49. Output(s):
  50. Successfully stored 3 records (139 bytes) in: "hdfs://localhost:54310/tmp/temp-833716857/tmp1325927627"
  51.  
  52. Counters:
  53. Total records written : 3
  54. Total bytes written : 139
  55. Spillable Memory Manager spill count : 0
  56. Total bags proactively spilled: 0
  57. Total records proactively spilled: 0
  58.  
  59. Job DAG:
  60. job_201309192059_0059
  61.  
  62.  
  63. 2013-09-20 10:23:31,497 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Success!
  64. 2013-09-20 10:23:31,537 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
  65. 2013-09-20 10:23:31,538 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
  66. ((ise,cyril3),(filtre,2),(value_1,31415))
  67. ((ise,cyril1),(filtre,2),(value_1,49926))
  68. ((ise,cyril2),(filtre,1),(value_1,18584))
  69. 2013-09-20 10:23:31,633 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
  70. 2013-09-20 10:23:31,640 [main] INFO org.apache.pig.newplan.logical.rules.ColumnPruneVisitor - Columns pruned for data: $0, $2
  71. 2013-09-20 10:23:31,655 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
  72. 2013-09-20 10:23:31,657 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
  73. 2013-09-20 10:23:31,657 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
  74. 2013-09-20 10:23:31,658 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
  75. 2013-09-20 10:23:31,659 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
  76. 2013-09-20 10:23:31,659 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
  77. 2013-09-20 10:23:31,659 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=-1
  78. 2013-09-20 10:23:31,659 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Could not estimate number of reducers and no requested or default parallelism set. Defaulting to 1 reducer.
  79. 2013-09-20 10:23:31,659 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
  80. 2013-09-20 10:23:31,699 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - creating jar file Job2565301253875420543.jar
  81. 2013-09-20 10:23:37,021 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - jar file Job2565301253875420543.jar created
  82. 2013-09-20 10:23:37,030 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
  83. 2013-09-20 10:23:37,039 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Key [pig.schematuple] is false, will not generate code.
  84. 2013-09-20 10:23:37,039 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Starting process to move generated code to distributed cacche
  85. 2013-09-20 10:23:37,039 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Setting key [pig.schematuple.classes] with classes to deserialize []
  86. 2013-09-20 10:23:37,061 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
  87. 2013-09-20 10:23:37,419 [JobControl] INFO org.apache.hadoop.mapred.JobClient - Cleaning up the staging area hdfs://localhost:54310/tmp/hadoop-cyril/mapred/staging/cyril/.staging/job_201309192059_0060
  88. 2013-09-20 10:23:37,419 [JobControl] ERROR org.apache.hadoop.security.UserGroupInformation - PriviledgedActionException as:cyril cause:org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Could not get input splits
  89. 2013-09-20 10:23:37,562 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
  90. 2013-09-20 10:23:37,566 [main] WARN org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure.
  91. 2013-09-20 10:23:37,566 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job null has failed! Stop running all dependent jobs
  92. 2013-09-20 10:23:37,566 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
  93. 2013-09-20 10:23:37,581 [main] ERROR org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to recreate exception from backend error: org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Could not get input splits
  94. at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:288)
  95. at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1054)
  96. at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1071)
  97. at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179)
  98. at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:983)
  99. at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
  100. at java.security.AccessController.doPrivileged(Native Method)
  101. at javax.security.auth.Subject.doAs(Subject.java:415)
  102. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
  103. at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
  104. at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:910)
  105. at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
  106. at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
  107. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  108. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  109. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  110. at java.lang.reflect.Method.invoke(Method.java:601)
  111. at org.apache.pig.backend.hadoop20.PigJobControl.mainLoopAction(PigJobControl.java:157)
  112. at org.apache.pig.backend.hadoop20.PigJobControl.run(PigJobControl.java:134)
  113. at java.lang.Thread.run(Thread.java:722)
  114. at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:257)
  115. Caused by: java.io.IOException: Could not get input splits
  116. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSplits(AbstractColumnFamilyInputFormat.java:189)
  117. at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:274)
  118. ... 20 more
  119. Caused by: java.util.concurrent.ExecutionException: java.io.IOException: failed connecting to all endpoints 0.0.0.0
  120. at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
  121. at java.util.concurrent.FutureTask.get(FutureTask.java:111)
  122. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSplits(AbstractColumnFamilyInputFormat.java:185)
  123. ... 21 more
  124. Caused by: java.io.IOException: failed connecting to all endpoints 0.0.0.0
  125. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSubSplits(AbstractColumnFamilyInputFormat.java:303)
  126. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.access$200(AbstractColumnFamilyInputFormat.java:62)
  127. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat$SplitCallable.call(AbstractColumnFamilyInputFormat.java:222)
  128. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat$SplitCallable.call(AbstractColumnFamilyInputFormat.java:207)
  129. at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
  130. at java.util.concurrent.FutureTask.run(FutureTask.java:166)
  131. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
  132. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
  133. at java.lang.Thread.run(Thread.java:722)
  134.  
  135. 2013-09-20 10:23:37,582 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
  136. 2013-09-20 10:23:37,582 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
  137.  
  138. HadoopVersion PigVersion UserId StartedAt FinishedAt Features
  139. 1.2.1 0.11.1 cyril 2013-09-20 10:23:31 2013-09-20 10:23:37 UNKNOWN
  140.  
  141. Failed!
  142.  
  143. Failed Jobs:
  144. JobId Alias Feature Message Outputs
  145. N/A data,rows MAP_ONLY Message: org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Could not get input splits
  146. at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:288)
  147. at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1054)
  148. at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1071)
  149. at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179)
  150. at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:983)
  151. at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
  152. at java.security.AccessController.doPrivileged(Native Method)
  153. at javax.security.auth.Subject.doAs(Subject.java:415)
  154. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
  155. at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
  156. at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:910)
  157. at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
  158. at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
  159. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  160. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  161. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  162. at java.lang.reflect.Method.invoke(Method.java:601)
  163. at org.apache.pig.backend.hadoop20.PigJobControl.mainLoopAction(PigJobControl.java:157)
  164. at org.apache.pig.backend.hadoop20.PigJobControl.run(PigJobControl.java:134)
  165. at java.lang.Thread.run(Thread.java:722)
  166. at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:257)
  167. Caused by: java.io.IOException: Could not get input splits
  168. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSplits(AbstractColumnFamilyInputFormat.java:189)
  169. at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:274)
  170. ... 20 more
  171. Caused by: java.util.concurrent.ExecutionException: java.io.IOException: failed connecting to all endpoints 0.0.0.0
  172. at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
  173. at java.util.concurrent.FutureTask.get(FutureTask.java:111)
  174. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSplits(AbstractColumnFamilyInputFormat.java:185)
  175. ... 21 more
  176. Caused by: java.io.IOException: failed connecting to all endpoints 0.0.0.0
  177. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSubSplits(AbstractColumnFamilyInputFormat.java:303)
  178. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.access$200(AbstractColumnFamilyInputFormat.java:62)
  179. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat$SplitCallable.call(AbstractColumnFamilyInputFormat.java:222)
  180. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat$SplitCallable.call(AbstractColumnFamilyInputFormat.java:207)
  181. at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
  182. at java.util.concurrent.FutureTask.run(FutureTask.java:166)
  183. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
  184. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
  185. at java.lang.Thread.run(Thread.java:722)
  186. hdfs://localhost:54310/tmp/temp-833716857/tmp-1717877487,
  187.  
  188. Input(s):
  189. Failed to read data from "cql://ks1/lc"
  190.  
  191. Output(s):
  192. Failed to produce result in "hdfs://localhost:54310/tmp/temp-833716857/tmp-1717877487"
  193.  
  194. Counters:
  195. Total records written : 0
  196. Total bytes written : 0
  197. Spillable Memory Manager spill count : 0
  198. Total bags proactively spilled: 0
  199. Total records proactively spilled: 0
  200.  
  201. Job DAG:
  202. null
  203.  
  204.  
  205. 2013-09-20 10:23:37,582 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!
  206. 2013-09-20 10:23:37,585 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias rows
  207. Details at logfile: /private/tmp/pig_1379665387789.log
  208.  
  209.  
  210. Exec 2
  211. ------
  212.  
  213. 10:23:38 [cyril:/tmp] 6 $ ~/tmp/pig_cassandra ~/tmp/example-script.pig
  214. Using /usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar.
  215. 2013-09-20 10:23:58,305 [main] INFO org.apache.pig.Main - Apache Pig version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
  216. 2013-09-20 10:23:58,306 [main] INFO org.apache.pig.Main - Logging error messages to: /private/tmp/pig_1379665438300.log
  217. 2013-09-20 10:23:58.489 java[37937:1703] Unable to load realm info from SCDynamicStore
  218. 2013-09-20 10:23:58,674 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /Users/cyril/.pigbootup not found
  219. 2013-09-20 10:23:58,808 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://localhost:54310/
  220. 2013-09-20 10:23:58,988 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: localhost:8021
  221. data: {ise: chararray,filtre: int,value_1: int}
  222. 2013-09-20 10:23:59,803 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
  223. 2013-09-20 10:23:59,903 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
  224. 2013-09-20 10:23:59,949 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
  225. 2013-09-20 10:23:59,949 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
  226. 2013-09-20 10:24:00,058 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
  227. 2013-09-20 10:24:00,072 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
  228. 2013-09-20 10:24:00,074 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
  229. 2013-09-20 10:24:00,075 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=-1
  230. 2013-09-20 10:24:00,075 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Could not estimate number of reducers and no requested or default parallelism set. Defaulting to 1 reducer.
  231. 2013-09-20 10:24:00,076 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
  232. 2013-09-20 10:24:00,254 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - creating jar file Job1841560987052587159.jar
  233. 2013-09-20 10:24:05,990 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - jar file Job1841560987052587159.jar created
  234. 2013-09-20 10:24:06,008 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
  235. 2013-09-20 10:24:06,055 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
  236. 2013-09-20 10:24:06,558 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
  237. 2013-09-20 10:24:06,602 [JobControl] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
  238. 2013-09-20 10:24:07,333 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_201309192059_0061
  239. 2013-09-20 10:24:07,334 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases data
  240. 2013-09-20 10:24:07,334 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: data[1,7] C: R:
  241. 2013-09-20 10:24:07,334 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - More information at: http://localhost:50030/jobdetails.jsp?jobid=job_201309192059_0061
  242. 2013-09-20 10:24:16,400 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 50% complete
  243. 2013-09-20 10:24:21,963 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
  244. 2013-09-20 10:24:21,965 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
  245.  
  246. HadoopVersion PigVersion UserId StartedAt FinishedAt Features
  247. 1.2.1 0.11.1 cyril 2013-09-20 10:24:00 2013-09-20 10:24:21 UNKNOWN
  248.  
  249. Success!
  250.  
  251. Job Stats (time in seconds):
  252. JobId Maps Reduces MaxMapTime MinMapTIme AvgMapTime MedianMapTime MaxReduceTime MinReduceTime AvgReduceTime MedianReducetime Alias Feature Outputs
  253. job_201309192059_0061 1 0 4 4 4 4 0 0 0 0 data MAP_ONLY hdfs://localhost:54310/tmp/temp991898698/tmp111954958,
  254.  
  255. Input(s):
  256. Successfully read 3 records (15143 bytes) from: "cql://ks1/lc"
  257.  
  258. Output(s):
  259. Successfully stored 3 records (139 bytes) in: "hdfs://localhost:54310/tmp/temp991898698/tmp111954958"
  260.  
  261. Counters:
  262. Total records written : 3
  263. Total bytes written : 139
  264. Spillable Memory Manager spill count : 0
  265. Total bags proactively spilled: 0
  266. Total records proactively spilled: 0
  267.  
  268. Job DAG:
  269. job_201309192059_0061
  270.  
  271.  
  272. 2013-09-20 10:24:21,973 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Success!
  273. 2013-09-20 10:24:22,000 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
  274. 2013-09-20 10:24:22,001 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
  275. ((ise,cyril3),(filtre,2),(value_1,31415))
  276. ((ise,cyril2),(filtre,1),(value_1,18584))
  277. ((ise,cyril1),(filtre,2),(value_1,49926))
  278. 2013-09-20 10:24:22,097 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
  279. 2013-09-20 10:24:22,104 [main] INFO org.apache.pig.newplan.logical.rules.ColumnPruneVisitor - Columns pruned for data: $0, $2
  280. 2013-09-20 10:24:22,119 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
  281. 2013-09-20 10:24:22,121 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
  282. 2013-09-20 10:24:22,121 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
  283. 2013-09-20 10:24:22,122 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
  284. 2013-09-20 10:24:22,123 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
  285. 2013-09-20 10:24:22,123 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
  286. 2013-09-20 10:24:22,123 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=-1
  287. 2013-09-20 10:24:22,123 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Could not estimate number of reducers and no requested or default parallelism set. Defaulting to 1 reducer.
  288. 2013-09-20 10:24:22,123 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
  289. 2013-09-20 10:24:22,171 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - creating jar file Job2262673435089937995.jar
  290. 2013-09-20 10:24:27,732 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - jar file Job2262673435089937995.jar created
  291. 2013-09-20 10:24:27,742 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
  292. 2013-09-20 10:24:27,746 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Key [pig.schematuple] is false, will not generate code.
  293. 2013-09-20 10:24:27,747 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Starting process to move generated code to distributed cacche
  294. 2013-09-20 10:24:27,747 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Setting key [pig.schematuple.classes] with classes to deserialize []
  295. 2013-09-20 10:24:27,762 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
  296. 2013-09-20 10:24:28,119 [JobControl] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
  297. 2013-09-20 10:24:28,263 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_201309192059_0062
  298. 2013-09-20 10:24:28,264 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases data,rows
  299. 2013-09-20 10:24:28,264 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: data[1,7],rows[5,8] C: R:
  300. 2013-09-20 10:24:28,264 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - More information at: http://localhost:50030/jobdetails.jsp?jobid=job_201309192059_0062
  301. 2013-09-20 10:24:28,269 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
  302. 2013-09-20 10:24:37,321 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 50% complete
  303. 2013-09-20 10:24:43,368 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
  304. 2013-09-20 10:24:43,369 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
  305.  
  306. HadoopVersion PigVersion UserId StartedAt FinishedAt Features
  307. 1.2.1 0.11.1 cyril 2013-09-20 10:24:22 2013-09-20 10:24:43 UNKNOWN
  308.  
  309. Success!
  310.  
  311. Job Stats (time in seconds):
  312. JobId Maps Reduces MaxMapTime MinMapTIme AvgMapTime MedianMapTime MaxReduceTime MinReduceTime AvgReduceTime MedianReducetime Alias Feature Outputs
  313. job_201309192059_0062 1 0 4 4 4 4 0 0 0 0 data,rows MAP_ONLY hdfs://localhost:54310/tmp/temp991898698/tmp-204486225,
  314.  
  315. Input(s):
  316. Successfully read 3 records (15143 bytes) from: "cql://ks1/lc"
  317.  
  318. Output(s):
  319. Successfully stored 3 records (47 bytes) in: "hdfs://localhost:54310/tmp/temp991898698/tmp-204486225"
  320.  
  321. Counters:
  322. Total records written : 3
  323. Total bytes written : 47
  324. Spillable Memory Manager spill count : 0
  325. Total bags proactively spilled: 0
  326. Total records proactively spilled: 0
  327.  
  328. Job DAG:
  329. job_201309192059_0062
  330.  
  331.  
  332. 2013-09-20 10:24:43,374 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Success!
  333. 2013-09-20 10:24:43,376 [main] INFO org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
  334. 2013-09-20 10:24:43,386 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
  335. 2013-09-20 10:24:43,386 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
  336. ((filtre,2))
  337. ((filtre,1))
  338. ((filtre,2))
  339. rows: {filtre: int}
  340.  
  341.  
  342. Exec 3
  343. ------
  344.  
  345. 10:25:25 [cyril:/tmp] $ ~/tmp/pig_cassandra ~/tmp/example-script.pig
  346. Using /usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar.
  347. 2013-09-20 10:33:31,962 [main] INFO org.apache.pig.Main - Apache Pig version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
  348. 2013-09-20 10:33:31,963 [main] INFO org.apache.pig.Main - Logging error messages to: /private/tmp/pig_1379666011958.log
  349. 2013-09-20 10:33:32.145 java[39298:1703] Unable to load realm info from SCDynamicStore
  350. 2013-09-20 10:33:32,644 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /Users/cyril/.pigbootup not found
  351. 2013-09-20 10:33:32,781 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://localhost:54310/
  352. 2013-09-20 10:33:32,931 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: localhost:8021
  353. data: {ise: chararray,filtre: int,value_1: int}
  354. 2013-09-20 10:33:33,847 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
  355. 2013-09-20 10:33:33,996 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
  356. 2013-09-20 10:33:34,053 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
  357. 2013-09-20 10:33:34,054 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
  358. 2013-09-20 10:33:34,163 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
  359. 2013-09-20 10:33:34,177 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
  360. 2013-09-20 10:33:34,179 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
  361. 2013-09-20 10:33:34,180 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=-1
  362. 2013-09-20 10:33:34,180 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Could not estimate number of reducers and no requested or default parallelism set. Defaulting to 1 reducer.
  363. 2013-09-20 10:33:34,181 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
  364. 2013-09-20 10:33:34,774 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - creating jar file Job8480144656769849813.jar
  365. 2013-09-20 10:33:40,506 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - jar file Job8480144656769849813.jar created
  366. 2013-09-20 10:33:40,527 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
  367. 2013-09-20 10:33:40,567 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
  368. 2013-09-20 10:33:41,070 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
  369. 2013-09-20 10:33:41,139 [JobControl] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
  370. 2013-09-20 10:33:42,048 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_201309192059_0063
  371. 2013-09-20 10:33:42,048 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases data
  372. 2013-09-20 10:33:42,048 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: data[1,7] C: R:
  373. 2013-09-20 10:33:42,048 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - More information at: http://localhost:50030/jobdetails.jsp?jobid=job_201309192059_0063
  374. 2013-09-20 10:33:50,611 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 50% complete
  375. 2013-09-20 10:33:56,668 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
  376. 2013-09-20 10:33:56,670 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
  377.  
  378. HadoopVersion PigVersion UserId StartedAt FinishedAt Features
  379. 1.2.1 0.11.1 cyril 2013-09-20 10:33:34 2013-09-20 10:33:56 UNKNOWN
  380.  
  381. Success!
  382.  
  383. Job Stats (time in seconds):
  384. JobId Maps Reduces MaxMapTime MinMapTIme AvgMapTime MedianMapTime MaxReduceTime MinReduceTime AvgReduceTime MedianReducetime Alias Feature Outputs
  385. job_201309192059_0063 1 0 4 4 4 4 0 0 0 0 data MAP_ONLY hdfs://localhost:54310/tmp/temp1989598265/tmp92573296,
  386.  
  387. Input(s):
  388. Successfully read 3 records (15143 bytes) from: "cql://ks1/lc"
  389.  
  390. Output(s):
  391. Successfully stored 3 records (139 bytes) in: "hdfs://localhost:54310/tmp/temp1989598265/tmp92573296"
  392.  
  393. Counters:
  394. Total records written : 3
  395. Total bytes written : 139
  396. Spillable Memory Manager spill count : 0
  397. Total bags proactively spilled: 0
  398. Total records proactively spilled: 0
  399.  
  400. Job DAG:
  401. job_201309192059_0063
  402.  
  403.  
  404. 2013-09-20 10:33:56,677 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Success!
  405. 2013-09-20 10:33:56,692 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
  406. 2013-09-20 10:33:56,692 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
  407. ((ise,cyril2),(filtre,1),(value_1,18584))
  408. ((ise,cyril1),(filtre,2),(value_1,49926))
  409. ((ise,cyril3),(filtre,2),(value_1,31415))
  410. 2013-09-20 10:33:56,791 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
  411. 2013-09-20 10:33:56,799 [main] INFO org.apache.pig.newplan.logical.rules.ColumnPruneVisitor - Columns pruned for data: $0, $2
  412. 2013-09-20 10:33:56,813 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
  413. 2013-09-20 10:33:56,815 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
  414. 2013-09-20 10:33:56,815 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
  415. 2013-09-20 10:33:56,817 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
  416. 2013-09-20 10:33:56,817 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
  417. 2013-09-20 10:33:56,818 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
  418. 2013-09-20 10:33:56,818 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=-1
  419. 2013-09-20 10:33:56,818 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Could not estimate number of reducers and no requested or default parallelism set. Defaulting to 1 reducer.
  420. 2013-09-20 10:33:56,818 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
  421. 2013-09-20 10:33:57,282 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - creating jar file Job6442697273402432172.jar
  422. 2013-09-20 10:34:02,884 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - jar file Job6442697273402432172.jar created
  423. 2013-09-20 10:34:02,895 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
  424. 2013-09-20 10:34:02,900 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Key [pig.schematuple] is false, will not generate code.
  425. 2013-09-20 10:34:02,900 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Starting process to move generated code to distributed cacche
  426. 2013-09-20 10:34:02,900 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Setting key [pig.schematuple.classes] with classes to deserialize []
  427. 2013-09-20 10:34:02,923 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
  428. 2013-09-20 10:34:03,258 [JobControl] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
  429. 2013-09-20 10:34:03,424 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_201309192059_0064
  430. 2013-09-20 10:34:03,424 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases data,rows
  431. 2013-09-20 10:34:03,424 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: data[1,7],rows[5,8] C: R:
  432. 2013-09-20 10:34:03,424 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - More information at: http://localhost:50030/jobdetails.jsp?jobid=job_201309192059_0064
  433. 2013-09-20 10:34:03,430 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
  434. 2013-09-20 10:34:12,981 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 50% complete
  435. 2013-09-20 10:34:18,520 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
  436. 2013-09-20 10:34:18,521 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
  437.  
  438. HadoopVersion PigVersion UserId StartedAt FinishedAt Features
  439. 1.2.1 0.11.1 cyril 2013-09-20 10:33:56 2013-09-20 10:34:18 UNKNOWN
  440.  
  441. Success!
  442.  
  443. Job Stats (time in seconds):
  444. JobId Maps Reduces MaxMapTime MinMapTIme AvgMapTime MedianMapTime MaxReduceTime MinReduceTime AvgReduceTime MedianReducetime Alias Feature Outputs
  445. job_201309192059_0064 1 0 4 4 4 4 0 0 0 0 data,rows MAP_ONLY hdfs://localhost:54310/tmp/temp1989598265/tmp2103743115,
  446.  
  447. Input(s):
  448. Successfully read 3 records (15143 bytes) from: "cql://ks1/lc"
  449.  
  450. Output(s):
  451. Successfully stored 3 records (47 bytes) in: "hdfs://localhost:54310/tmp/temp1989598265/tmp2103743115"
  452.  
  453. Counters:
  454. Total records written : 3
  455. Total bytes written : 47
  456. Spillable Memory Manager spill count : 0
  457. Total bags proactively spilled: 0
  458. Total records proactively spilled: 0
  459.  
  460. Job DAG:
  461. job_201309192059_0064
  462.  
  463.  
  464. 2013-09-20 10:34:18,526 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Success!
  465. 2013-09-20 10:34:18,527 [main] INFO org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
  466. 2013-09-20 10:34:18,531 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
  467. 2013-09-20 10:34:18,531 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
  468. ((filtre,1))
  469. ((filtre,2))
  470. ((filtre,2))
  471. rows: {filtre: int}
  472.  
  473.  
  474. Exec 4
  475. ------
  476.  
  477. 10:34:19 ⌂62% [cyril:/tmp] $ ~/tmp/pig_cassandra ~/tmp/example-script.pig
  478. Using /usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar.
  479. 2013-09-20 10:36:20,638 [main] INFO org.apache.pig.Main - Apache Pig version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
  480. 2013-09-20 10:36:20,639 [main] INFO org.apache.pig.Main - Logging error messages to: /private/tmp/pig_1379666180632.log
  481. 2013-09-20 10:36:20.822 java[39840:1703] Unable to load realm info from SCDynamicStore
  482. 2013-09-20 10:36:21,021 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /Users/cyril/.pigbootup not found
  483. 2013-09-20 10:36:21,154 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://localhost:54310/
  484. 2013-09-20 10:36:21,333 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: localhost:8021
  485. data: {ise: chararray,filtre: int,value_1: int}
  486. 2013-09-20 10:36:22,258 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
  487. 2013-09-20 10:36:22,358 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
  488. 2013-09-20 10:36:22,413 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
  489. 2013-09-20 10:36:22,414 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
  490. 2013-09-20 10:36:22,511 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
  491. 2013-09-20 10:36:22,524 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
  492. 2013-09-20 10:36:22,526 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
  493. 2013-09-20 10:36:22,527 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=-1
  494. 2013-09-20 10:36:22,527 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Could not estimate number of reducers and no requested or default parallelism set. Defaulting to 1 reducer.
  495. 2013-09-20 10:36:22,527 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
  496. 2013-09-20 10:36:22,695 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - creating jar file Job4310142123380318924.jar
  497. 2013-09-20 10:36:28,605 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - jar file Job4310142123380318924.jar created
  498. 2013-09-20 10:36:28,657 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
  499. 2013-09-20 10:36:28,753 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
  500. 2013-09-20 10:36:29,255 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
  501. 2013-09-20 10:36:29,296 [JobControl] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
  502. 2013-09-20 10:36:30,059 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_201309192059_0065
  503. 2013-09-20 10:36:30,059 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases data
  504. 2013-09-20 10:36:30,059 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: data[1,7] C: R:
  505. 2013-09-20 10:36:30,059 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - More information at: http://localhost:50030/jobdetails.jsp?jobid=job_201309192059_0065
  506. 2013-09-20 10:36:40,125 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 50% complete
  507. 2013-09-20 10:36:44,681 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
  508. 2013-09-20 10:36:44,685 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
  509.  
  510. HadoopVersion PigVersion UserId StartedAt FinishedAt Features
  511. 1.2.1 0.11.1 cyril 2013-09-20 10:36:22 2013-09-20 10:36:44 UNKNOWN
  512.  
  513. Success!
  514.  
  515. Job Stats (time in seconds):
  516. JobId Maps Reduces MaxMapTime MinMapTIme AvgMapTime MedianMapTime MaxReduceTime MinReduceTime AvgReduceTime MedianReducetime Alias Feature Outputs
  517. job_201309192059_0065 1 0 4 4 4 4 0 0 0 0 data MAP_ONLY hdfs://localhost:54310/tmp/temp1840282157/tmp-1072271629,
  518.  
  519. Input(s):
  520. Successfully read 3 records (15143 bytes) from: "cql://ks1/lc"
  521.  
  522. Output(s):
  523. Successfully stored 3 records (139 bytes) in: "hdfs://localhost:54310/tmp/temp1840282157/tmp-1072271629"
  524.  
  525. Counters:
  526. Total records written : 3
  527. Total bytes written : 139
  528. Spillable Memory Manager spill count : 0
  529. Total bags proactively spilled: 0
  530. Total records proactively spilled: 0
  531.  
  532. Job DAG:
  533. job_201309192059_0065
  534.  
  535.  
  536. 2013-09-20 10:36:44,707 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Success!
  537. 2013-09-20 10:36:44,735 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
  538. 2013-09-20 10:36:44,735 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
  539. ((ise,cyril1),(filtre,2),(value_1,49926))
  540. ((ise,cyril2),(filtre,1),(value_1,18584))
  541. ((ise,cyril3),(filtre,2),(value_1,31415))
  542. 2013-09-20 10:36:44,844 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
  543. 2013-09-20 10:36:44,852 [main] INFO org.apache.pig.newplan.logical.rules.ColumnPruneVisitor - Columns pruned for data: $0, $2
  544. 2013-09-20 10:36:44,876 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
  545. 2013-09-20 10:36:44,881 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
  546. 2013-09-20 10:36:44,881 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
  547. 2013-09-20 10:36:44,889 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
  548. 2013-09-20 10:36:44,890 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
  549. 2013-09-20 10:36:44,891 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
  550. 2013-09-20 10:36:44,891 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=-1
  551. 2013-09-20 10:36:44,891 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Could not estimate number of reducers and no requested or default parallelism set. Defaulting to 1 reducer.
  552. 2013-09-20 10:36:44,891 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
  553. 2013-09-20 10:36:44,947 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - creating jar file Job6948161388483582296.jar
  554. 2013-09-20 10:36:50,318 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - jar file Job6948161388483582296.jar created
  555. 2013-09-20 10:36:50,330 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
  556. 2013-09-20 10:36:50,337 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Key [pig.schematuple] is false, will not generate code.
  557. 2013-09-20 10:36:50,337 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Starting process to move generated code to distributed cacche
  558. 2013-09-20 10:36:50,337 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Setting key [pig.schematuple.classes] with classes to deserialize []
  559. 2013-09-20 10:36:50,360 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
  560. 2013-09-20 10:36:50,690 [JobControl] INFO org.apache.hadoop.mapred.JobClient - Cleaning up the staging area hdfs://localhost:54310/tmp/hadoop-cyril/mapred/staging/cyril/.staging/job_201309192059_0066
  561. 2013-09-20 10:36:50,692 [JobControl] ERROR org.apache.hadoop.security.UserGroupInformation - PriviledgedActionException as:cyril cause:org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Could not get input splits
  562. 2013-09-20 10:36:50,861 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
  563. 2013-09-20 10:36:50,865 [main] WARN org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure.
  564. 2013-09-20 10:36:50,865 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job null has failed! Stop running all dependent jobs
  565. 2013-09-20 10:36:50,865 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
  566. 2013-09-20 10:36:50,877 [main] ERROR org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to recreate exception from backend error: org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Could not get input splits
  567. at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:288)
  568. at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1054)
  569. at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1071)
  570. at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179)
  571. at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:983)
  572. at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
  573. at java.security.AccessController.doPrivileged(Native Method)
  574. at javax.security.auth.Subject.doAs(Subject.java:415)
  575. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
  576. at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
  577. at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:910)
  578. at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
  579. at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
  580. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  581. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  582. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  583. at java.lang.reflect.Method.invoke(Method.java:601)
  584. at org.apache.pig.backend.hadoop20.PigJobControl.mainLoopAction(PigJobControl.java:157)
  585. at org.apache.pig.backend.hadoop20.PigJobControl.run(PigJobControl.java:134)
  586. at java.lang.Thread.run(Thread.java:722)
  587. at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:257)
  588. Caused by: java.io.IOException: Could not get input splits
  589. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSplits(AbstractColumnFamilyInputFormat.java:189)
  590. at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:274)
  591. ... 20 more
  592. Caused by: java.util.concurrent.ExecutionException: java.io.IOException: failed connecting to all endpoints 0.0.0.0
  593. at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
  594. at java.util.concurrent.FutureTask.get(FutureTask.java:111)
  595. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSplits(AbstractColumnFamilyInputFormat.java:185)
  596. ... 21 more
  597. Caused by: java.io.IOException: failed connecting to all endpoints 0.0.0.0
  598. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSubSplits(AbstractColumnFamilyInputFormat.java:303)
  599. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.access$200(AbstractColumnFamilyInputFormat.java:62)
  600. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat$SplitCallable.call(AbstractColumnFamilyInputFormat.java:222)
  601. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat$SplitCallable.call(AbstractColumnFamilyInputFormat.java:207)
  602. at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
  603. at java.util.concurrent.FutureTask.run(FutureTask.java:166)
  604. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
  605. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
  606. at java.lang.Thread.run(Thread.java:722)
  607.  
  608. 2013-09-20 10:36:50,877 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
  609. 2013-09-20 10:36:50,877 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
  610.  
  611. HadoopVersion PigVersion UserId StartedAt FinishedAt Features
  612. 1.2.1 0.11.1 cyril 2013-09-20 10:36:44 2013-09-20 10:36:50 UNKNOWN
  613.  
  614. Failed!
  615.  
  616. Failed Jobs:
  617. JobId Alias Feature Message Outputs
  618. N/A data,rows MAP_ONLY Message: org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Could not get input splits
  619. at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:288)
  620. at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1054)
  621. at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1071)
  622. at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179)
  623. at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:983)
  624. at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
  625. at java.security.AccessController.doPrivileged(Native Method)
  626. at javax.security.auth.Subject.doAs(Subject.java:415)
  627. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
  628. at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
  629. at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:910)
  630. at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
  631. at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
  632. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  633. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  634. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  635. at java.lang.reflect.Method.invoke(Method.java:601)
  636. at org.apache.pig.backend.hadoop20.PigJobControl.mainLoopAction(PigJobControl.java:157)
  637. at org.apache.pig.backend.hadoop20.PigJobControl.run(PigJobControl.java:134)
  638. at java.lang.Thread.run(Thread.java:722)
  639. at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:257)
  640. Caused by: java.io.IOException: Could not get input splits
  641. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSplits(AbstractColumnFamilyInputFormat.java:189)
  642. at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:274)
  643. ... 20 more
  644. Caused by: java.util.concurrent.ExecutionException: java.io.IOException: failed connecting to all endpoints 0.0.0.0
  645. at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
  646. at java.util.concurrent.FutureTask.get(FutureTask.java:111)
  647. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSplits(AbstractColumnFamilyInputFormat.java:185)
  648. ... 21 more
  649. Caused by: java.io.IOException: failed connecting to all endpoints 0.0.0.0
  650. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSubSplits(AbstractColumnFamilyInputFormat.java:303)
  651. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.access$200(AbstractColumnFamilyInputFormat.java:62)
  652. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat$SplitCallable.call(AbstractColumnFamilyInputFormat.java:222)
  653. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat$SplitCallable.call(AbstractColumnFamilyInputFormat.java:207)
  654. at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
  655. at java.util.concurrent.FutureTask.run(FutureTask.java:166)
  656. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
  657. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
  658. at java.lang.Thread.run(Thread.java:722)
  659. hdfs://localhost:54310/tmp/temp1840282157/tmp-2059878217,
  660.  
  661. Input(s):
  662. Failed to read data from "cql://ks1/lc"
  663.  
  664. Output(s):
  665. Failed to produce result in "hdfs://localhost:54310/tmp/temp1840282157/tmp-2059878217"
  666.  
  667. Counters:
  668. Total records written : 0
  669. Total bytes written : 0
  670. Spillable Memory Manager spill count : 0
  671. Total bags proactively spilled: 0
  672. Total records proactively spilled: 0
  673.  
  674. Job DAG:
  675. null
  676.  
  677.  
  678. 2013-09-20 10:36:50,878 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!
  679. 2013-09-20 10:36:50,881 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias rows
  680. Details at logfile: /private/tmp/pig_1379666180632.log
  681.  
  682.  
  683. Exec 5
  684. ------
  685.  
  686. 10:36:51 [cyril:/tmp] 6 $ ~/tmp/pig_cassandra ~/tmp/example-script.pig
  687. Using /usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar.
  688. 2013-09-20 10:39:57,485 [main] INFO org.apache.pig.Main - Apache Pig version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
  689. 2013-09-20 10:39:57,486 [main] INFO org.apache.pig.Main - Logging error messages to: /private/tmp/pig_1379666397481.log
  690. 2013-09-20 10:39:57.682 java[40411:1703] Unable to load realm info from SCDynamicStore
  691. 2013-09-20 10:39:57,879 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /Users/cyril/.pigbootup not found
  692. 2013-09-20 10:39:58,019 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://localhost:54310/
  693. 2013-09-20 10:39:58,183 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: localhost:8021
  694. data: {ise: chararray,filtre: int,value_1: int}
  695. 2013-09-20 10:39:59,099 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
  696. 2013-09-20 10:39:59,233 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
  697. 2013-09-20 10:39:59,290 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
  698. 2013-09-20 10:39:59,290 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
  699. 2013-09-20 10:39:59,386 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
  700. 2013-09-20 10:39:59,399 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
  701. 2013-09-20 10:39:59,400 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
  702. 2013-09-20 10:39:59,402 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=-1
  703. 2013-09-20 10:39:59,402 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Could not estimate number of reducers and no requested or default parallelism set. Defaulting to 1 reducer.
  704. 2013-09-20 10:39:59,402 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
  705. 2013-09-20 10:39:59,588 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - creating jar file Job4430762615936643200.jar
  706. 2013-09-20 10:40:05,251 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - jar file Job4430762615936643200.jar created
  707. 2013-09-20 10:40:05,279 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
  708. 2013-09-20 10:40:05,359 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
  709. 2013-09-20 10:40:05,867 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
  710. 2013-09-20 10:40:06,163 [JobControl] INFO org.apache.hadoop.mapred.JobClient - Cleaning up the staging area hdfs://localhost:54310/tmp/hadoop-cyril/mapred/staging/cyril/.staging/job_201309192059_0067
  711. 2013-09-20 10:40:06,164 [JobControl] ERROR org.apache.hadoop.security.UserGroupInformation - PriviledgedActionException as:cyril cause:org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Could not get input splits
  712. 2013-09-20 10:40:06,165 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job null has failed! Stop running all dependent jobs
  713. 2013-09-20 10:40:06,170 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
  714. 2013-09-20 10:40:06,182 [main] ERROR org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to recreate exception from backend error: org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Could not get input splits
  715. at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:288)
  716. at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1054)
  717. at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1071)
  718. at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179)
  719. at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:983)
  720. at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
  721. at java.security.AccessController.doPrivileged(Native Method)
  722. at javax.security.auth.Subject.doAs(Subject.java:415)
  723. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
  724. at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
  725. at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:910)
  726. at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
  727. at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
  728. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  729. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  730. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  731. at java.lang.reflect.Method.invoke(Method.java:601)
  732. at org.apache.pig.backend.hadoop20.PigJobControl.mainLoopAction(PigJobControl.java:157)
  733. at org.apache.pig.backend.hadoop20.PigJobControl.run(PigJobControl.java:134)
  734. at java.lang.Thread.run(Thread.java:722)
  735. at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:257)
  736. Caused by: java.io.IOException: Could not get input splits
  737. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSplits(AbstractColumnFamilyInputFormat.java:189)
  738. at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:274)
  739. ... 20 more
  740. Caused by: java.util.concurrent.ExecutionException: java.io.IOException: failed connecting to all endpoints 0.0.0.0
  741. at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
  742. at java.util.concurrent.FutureTask.get(FutureTask.java:111)
  743. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSplits(AbstractColumnFamilyInputFormat.java:185)
  744. ... 21 more
  745. Caused by: java.io.IOException: failed connecting to all endpoints 0.0.0.0
  746. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSubSplits(AbstractColumnFamilyInputFormat.java:303)
  747. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.access$200(AbstractColumnFamilyInputFormat.java:62)
  748. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat$SplitCallable.call(AbstractColumnFamilyInputFormat.java:222)
  749. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat$SplitCallable.call(AbstractColumnFamilyInputFormat.java:207)
  750. at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
  751. at java.util.concurrent.FutureTask.run(FutureTask.java:166)
  752. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
  753. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
  754. at java.lang.Thread.run(Thread.java:722)
  755.  
  756. 2013-09-20 10:40:06,183 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
  757. 2013-09-20 10:40:06,184 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
  758.  
  759. HadoopVersion PigVersion UserId StartedAt FinishedAt Features
  760. 1.2.1 0.11.1 cyril 2013-09-20 10:39:59 2013-09-20 10:40:06 UNKNOWN
  761.  
  762. Failed!
  763.  
  764. Failed Jobs:
  765. JobId Alias Feature Message Outputs
  766. N/A data MAP_ONLY Message: org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Could not get input splits
  767. at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:288)
  768. at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1054)
  769. at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1071)
  770. at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179)
  771. at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:983)
  772. at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
  773. at java.security.AccessController.doPrivileged(Native Method)
  774. at javax.security.auth.Subject.doAs(Subject.java:415)
  775. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
  776. at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
  777. at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:910)
  778. at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
  779. at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
  780. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  781. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  782. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  783. at java.lang.reflect.Method.invoke(Method.java:601)
  784. at org.apache.pig.backend.hadoop20.PigJobControl.mainLoopAction(PigJobControl.java:157)
  785. at org.apache.pig.backend.hadoop20.PigJobControl.run(PigJobControl.java:134)
  786. at java.lang.Thread.run(Thread.java:722)
  787. at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:257)
  788. Caused by: java.io.IOException: Could not get input splits
  789. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSplits(AbstractColumnFamilyInputFormat.java:189)
  790. at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:274)
  791. ... 20 more
  792. Caused by: java.util.concurrent.ExecutionException: java.io.IOException: failed connecting to all endpoints 0.0.0.0
  793. at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
  794. at java.util.concurrent.FutureTask.get(FutureTask.java:111)
  795. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSplits(AbstractColumnFamilyInputFormat.java:185)
  796. ... 21 more
  797. Caused by: java.io.IOException: failed connecting to all endpoints 0.0.0.0
  798. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSubSplits(AbstractColumnFamilyInputFormat.java:303)
  799. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.access$200(AbstractColumnFamilyInputFormat.java:62)
  800. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat$SplitCallable.call(AbstractColumnFamilyInputFormat.java:222)
  801. at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat$SplitCallable.call(AbstractColumnFamilyInputFormat.java:207)
  802. at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
  803. at java.util.concurrent.FutureTask.run(FutureTask.java:166)
  804. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
  805. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
  806. at java.lang.Thread.run(Thread.java:722)
  807. hdfs://localhost:54310/tmp/temp944946856/tmp-1975876971,
  808.  
  809. Input(s):
  810. Failed to read data from "cql://ks1/lc"
  811.  
  812. Output(s):
  813. Failed to produce result in "hdfs://localhost:54310/tmp/temp944946856/tmp-1975876971"
  814.  
  815. Counters:
  816. Total records written : 0
  817. Total bytes written : 0
  818. Spillable Memory Manager spill count : 0
  819. Total bags proactively spilled: 0
  820. Total records proactively spilled: 0
  821.  
  822. Job DAG:
  823. null
  824.  
  825.  
  826. 2013-09-20 10:40:06,185 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!
  827. 2013-09-20 10:40:06,192 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias data
  828. Details at logfile: /private/tmp/pig_1379666397481.log
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!