API tools faq
paste
Guest User

Mahout 0.9 release Streaming K-Means log

a guest
Jan 19th, 2014
150
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 96.70 KB | None | 0 0
raw download clone embed print report
  1. Please select a number to choose the corresponding clustering algorithm
  2. 1. kmeans clustering
  3. 2. fuzzykmeans clustering
  4. 3. lda clustering
  5. 4. streamingkmeans clustering
  6. Enter your choice : ok. You chose 4 and we'll use streamingkmeans Clustering
  7. creating work directory at /tmp/mahout-work-frank
  8. MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
  9. hadoop binary is not in PATH,HADOOP_HOME/bin,HADOOP_PREFIX/bin, running locally
  10. SLF4J: Class path contains multiple SLF4J bindings.
  11. SLF4J: Found binding in [jar:file:/home/frank/Desktop/mahout-distribution-0.9/examples/target/mahout-examples-0.9-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]
  12. SLF4J: Found binding in [jar:file:/home/frank/Desktop/mahout-distribution-0.9/examples/target/dependency/slf4j-jcl-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
  13. SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
  14. SLF4J: Actual binding is of type [org.slf4j.impl.JCLLoggerFactory]
  15. Jan 19, 2014 4:14:25 PM org.slf4j.impl.JCLLoggerAdapter info
  16. INFO: Maximum n-gram size is: 1
  17. Jan 19, 2014 4:14:25 PM org.slf4j.impl.JCLLoggerAdapter info
  18. INFO: Deleting /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans
  19. Jan 19, 2014 4:14:25 PM org.slf4j.impl.JCLLoggerAdapter info
  20. INFO: Minimum LLR value: 1.0
  21. Jan 19, 2014 4:14:25 PM org.slf4j.impl.JCLLoggerAdapter info
  22. INFO: Number of reduce tasks: 1
  23. Jan 19, 2014 4:14:25 PM org.slf4j.impl.JCLLoggerAdapter info
  24. INFO: Tokenizing documents in /tmp/mahout-work-frank/reuters-out-seqdir
  25. Jan 19, 2014 4:14:25 PM org.apache.hadoop.util.NativeCodeLoader <clinit>
  26. WARNING: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  27. Jan 19, 2014 4:14:25 PM org.apache.hadoop.mapreduce.lib.input.FileInputFormat listStatus
  28. INFO: Total input paths to process : 1
  29. Jan 19, 2014 4:14:26 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  30. INFO: Running job: job_local2046196399_0001
  31. Jan 19, 2014 4:14:26 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
  32. INFO: Waiting for map tasks
  33. Jan 19, 2014 4:14:26 PM org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable run
  34. INFO: Starting task: attempt_local2046196399_0001_m_000000_0
  35. Jan 19, 2014 4:14:26 PM org.apache.hadoop.util.ProcessTree isSetsidSupported
  36. INFO: setsid exited with exit code 0
  37. Jan 19, 2014 4:14:26 PM org.apache.hadoop.mapred.Task initialize
  38. INFO: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@3379336b
  39. Jan 19, 2014 4:14:26 PM org.apache.hadoop.mapred.MapTask runNewMapper
  40. INFO: Processing split: file:/tmp/mahout-work-frank/reuters-out-seqdir/chunk-0:0+18403906
  41. Jan 19, 2014 4:14:27 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  42. INFO: map 0% reduce 0%
  43. Jan 19, 2014 4:14:27 PM org.apache.hadoop.mapred.Task done
  44. INFO: Task:attempt_local2046196399_0001_m_000000_0 is done. And is in the process of commiting
  45. Jan 19, 2014 4:14:27 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  46. INFO:
  47. Jan 19, 2014 4:14:27 PM org.apache.hadoop.mapred.Task commit
  48. INFO: Task attempt_local2046196399_0001_m_000000_0 is allowed to commit now
  49. Jan 19, 2014 4:14:27 PM org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter commitTask
  50. INFO: Saved output of task 'attempt_local2046196399_0001_m_000000_0' to /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/tokenized-documents
  51. Jan 19, 2014 4:14:27 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  52. INFO:
  53. Jan 19, 2014 4:14:27 PM org.apache.hadoop.mapred.Task sendDone
  54. INFO: Task 'attempt_local2046196399_0001_m_000000_0' done.
  55. Jan 19, 2014 4:14:27 PM org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable run
  56. INFO: Finishing task: attempt_local2046196399_0001_m_000000_0
  57. Jan 19, 2014 4:14:27 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
  58. INFO: Map task executor complete.
  59. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  60. INFO: map 100% reduce 0%
  61. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  62. INFO: Job complete: job_local2046196399_0001
  63. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.Counters log
  64. INFO: Counters: 12
  65. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.Counters log
  66. INFO: File Output Format Counters
  67. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.Counters log
  68. INFO: Bytes Written=15194092
  69. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.Counters log
  70. INFO: File Input Format Counters
  71. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.Counters log
  72. INFO: Bytes Read=18547698
  73. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.Counters log
  74. INFO: FileSystemCounters
  75. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.Counters log
  76. INFO: FILE_BYTES_READ=42605926
  77. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.Counters log
  78. INFO: FILE_BYTES_WRITTEN=39492261
  79. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.Counters log
  80. INFO: Map-Reduce Framework
  81. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.Counters log
  82. INFO: Map input records=21578
  83. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.Counters log
  84. INFO: Physical memory (bytes) snapshot=0
  85. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.Counters log
  86. INFO: Spilled Records=0
  87. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.Counters log
  88. INFO: Total committed heap usage (bytes)=342949888
  89. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.Counters log
  90. INFO: CPU time spent (ms)=0
  91. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.Counters log
  92. INFO: Virtual memory (bytes) snapshot=0
  93. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.Counters log
  94. INFO: SPLIT_RAW_BYTES=119
  95. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.Counters log
  96. INFO: Map output records=21578
  97. Jan 19, 2014 4:14:28 PM org.slf4j.impl.JCLLoggerAdapter info
  98. INFO: Creating Term Frequency Vectors
  99. Jan 19, 2014 4:14:28 PM org.slf4j.impl.JCLLoggerAdapter info
  100. INFO: Creating dictionary from /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/tokenized-documents and saving at /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/wordcount
  101. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapreduce.lib.input.FileInputFormat listStatus
  102. INFO: Total input paths to process : 1
  103. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  104. INFO: Running job: job_local179122454_0002
  105. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
  106. INFO: Waiting for map tasks
  107. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable run
  108. INFO: Starting task: attempt_local179122454_0002_m_000000_0
  109. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.Task initialize
  110. INFO: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@58430cb2
  111. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.MapTask runNewMapper
  112. INFO: Processing split: file:/tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/tokenized-documents/part-m-00000:0+15076300
  113. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  114. INFO: io.sort.mb = 100
  115. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  116. INFO: data buffer = 79691776/99614720
  117. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  118. INFO: record buffer = 262144/327680
  119. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer collect
  120. INFO: Spilling map output: record full = true
  121. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer startSpill
  122. INFO: bufstart = 0; bufend = 3828344; bufvoid = 99614720
  123. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer startSpill
  124. INFO: kvstart = 0; kvend = 262144; length = 327680
  125. Jan 19, 2014 4:14:28 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer sortAndSpill
  126. INFO: Finished spill 0
  127. Jan 19, 2014 4:14:29 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer collect
  128. INFO: Spilling map output: record full = true
  129. Jan 19, 2014 4:14:29 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer startSpill
  130. INFO: bufstart = 3828344; bufend = 7658606; bufvoid = 99614720
  131. Jan 19, 2014 4:14:29 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer startSpill
  132. INFO: kvstart = 262144; kvend = 196607; length = 327680
  133. Jan 19, 2014 4:14:29 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  134. INFO: map 0% reduce 0%
  135. Jan 19, 2014 4:14:29 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer sortAndSpill
  136. INFO: Finished spill 1
  137. Jan 19, 2014 4:14:29 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer collect
  138. INFO: Spilling map output: record full = true
  139. Jan 19, 2014 4:14:29 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer startSpill
  140. INFO: bufstart = 7658606; bufend = 11485868; bufvoid = 99614720
  141. Jan 19, 2014 4:14:29 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer startSpill
  142. INFO: kvstart = 196607; kvend = 131070; length = 327680
  143. Jan 19, 2014 4:14:29 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer sortAndSpill
  144. INFO: Finished spill 2
  145. Jan 19, 2014 4:14:29 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer collect
  146. INFO: Spilling map output: record full = true
  147. Jan 19, 2014 4:14:29 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer startSpill
  148. INFO: bufstart = 11485868; bufend = 15312160; bufvoid = 99614720
  149. Jan 19, 2014 4:14:29 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer startSpill
  150. INFO: kvstart = 131070; kvend = 65533; length = 327680
  151. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer sortAndSpill
  152. INFO: Finished spill 3
  153. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer collect
  154. INFO: Spilling map output: record full = true
  155. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer startSpill
  156. INFO: bufstart = 15312160; bufend = 19138533; bufvoid = 99614720
  157. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer startSpill
  158. INFO: kvstart = 65533; kvend = 327677; length = 327680
  159. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer sortAndSpill
  160. INFO: Finished spill 4
  161. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer flush
  162. INFO: Starting flush of map output
  163. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer sortAndSpill
  164. INFO: Finished spill 5
  165. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
  166. INFO: Merging 6 sorted segments
  167. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
  168. INFO: Down to the last merge-pass, with 6 segments left of total size: 3044575 bytes
  169. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.Task done
  170. INFO: Task:attempt_local179122454_0002_m_000000_0 is done. And is in the process of commiting
  171. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  172. INFO:
  173. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.Task sendDone
  174. INFO: Task 'attempt_local179122454_0002_m_000000_0' done.
  175. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable run
  176. INFO: Finishing task: attempt_local179122454_0002_m_000000_0
  177. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
  178. INFO: Map task executor complete.
  179. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.Task initialize
  180. INFO: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@362f58d7
  181. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  182. INFO:
  183. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
  184. INFO: Merging 1 sorted segments
  185. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
  186. INFO: Down to the last merge-pass, with 1 segments left of total size: 1305776 bytes
  187. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  188. INFO:
  189. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.Task done
  190. INFO: Task:attempt_local179122454_0002_r_000000_0 is done. And is in the process of commiting
  191. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  192. INFO:
  193. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.Task commit
  194. INFO: Task attempt_local179122454_0002_r_000000_0 is allowed to commit now
  195. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter commitTask
  196. INFO: Saved output of task 'attempt_local179122454_0002_r_000000_0' to /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/wordcount
  197. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  198. INFO: reduce > reduce
  199. Jan 19, 2014 4:14:30 PM org.apache.hadoop.mapred.Task sendDone
  200. INFO: Task 'attempt_local179122454_0002_r_000000_0' done.
  201. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  202. INFO: map 100% reduce 100%
  203. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  204. INFO: Job complete: job_local179122454_0002
  205. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  206. INFO: Counters: 20
  207. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  208. INFO: File Output Format Counters
  209. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  210. INFO: Bytes Written=1000743
  211. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  212. INFO: File Input Format Counters
  213. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  214. INFO: Bytes Read=15194092
  215. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  216. INFO: FileSystemCounters
  217. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  218. INFO: FILE_BYTES_READ=171111564
  219. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  220. INFO: FILE_BYTES_WRITTEN=137284163
  221. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  222. INFO: Map-Reduce Framework
  223. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  224. INFO: Reduce input groups=73413
  225. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  226. INFO: Map output materialized bytes=1305780
  227. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  228. INFO: Combine output records=246193
  229. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  230. INFO: Map input records=21578
  231. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  232. INFO: Reduce shuffle bytes=0
  233. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  234. INFO: Physical memory (bytes) snapshot=0
  235. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  236. INFO: Reduce output records=41807
  237. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  238. INFO: Spilled Records=319606
  239. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  240. INFO: Map output bytes=22501851
  241. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  242. INFO: Total committed heap usage (bytes)=1189085184
  243. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  244. INFO: CPU time spent (ms)=0
  245. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  246. INFO: Virtual memory (bytes) snapshot=0
  247. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  248. INFO: SPLIT_RAW_BYTES=167
  249. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  250. INFO: Map output records=1540960
  251. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  252. INFO: Combine input records=1713740
  253. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Counters log
  254. INFO: Reduce input records=73413
  255. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapreduce.lib.input.FileInputFormat listStatus
  256. INFO: Total input paths to process : 1
  257. Jan 19, 2014 4:14:31 PM org.apache.hadoop.filecache.TrackerDistributedCacheManager downloadCacheObject
  258. INFO: Creating dictionary.file-0 in /tmp/hadoop-frank/mapred/local/archive/5264486555619678171_-998685638_722551391/file/tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans-work-8845602536535162168 with rwxr-xr-x
  259. Jan 19, 2014 4:14:31 PM org.apache.hadoop.filecache.TrackerDistributedCacheManager downloadCacheObject
  260. INFO: Cached /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/dictionary.file-0 as /tmp/hadoop-frank/mapred/local/archive/5264486555619678171_-998685638_722551391/file/tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/dictionary.file-0
  261. Jan 19, 2014 4:14:31 PM org.apache.hadoop.filecache.TrackerDistributedCacheManager localizePublicCacheObject
  262. INFO: Cached /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/dictionary.file-0 as /tmp/hadoop-frank/mapred/local/archive/5264486555619678171_-998685638_722551391/file/tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/dictionary.file-0
  263. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  264. INFO: Running job: job_local1434897154_0003
  265. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
  266. INFO: Waiting for map tasks
  267. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable run
  268. INFO: Starting task: attempt_local1434897154_0003_m_000000_0
  269. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.Task initialize
  270. INFO: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@12dabc51
  271. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.MapTask runNewMapper
  272. INFO: Processing split: file:/tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/tokenized-documents/part-m-00000:0+15076300
  273. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  274. INFO: io.sort.mb = 100
  275. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  276. INFO: data buffer = 79691776/99614720
  277. Jan 19, 2014 4:14:31 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  278. INFO: record buffer = 262144/327680
  279. Jan 19, 2014 4:14:32 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer flush
  280. INFO: Starting flush of map output
  281. Jan 19, 2014 4:14:32 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer sortAndSpill
  282. INFO: Finished spill 0
  283. Jan 19, 2014 4:14:32 PM org.apache.hadoop.mapred.Task done
  284. INFO: Task:attempt_local1434897154_0003_m_000000_0 is done. And is in the process of commiting
  285. Jan 19, 2014 4:14:32 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  286. INFO:
  287. Jan 19, 2014 4:14:32 PM org.apache.hadoop.mapred.Task sendDone
  288. INFO: Task 'attempt_local1434897154_0003_m_000000_0' done.
  289. Jan 19, 2014 4:14:32 PM org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable run
  290. INFO: Finishing task: attempt_local1434897154_0003_m_000000_0
  291. Jan 19, 2014 4:14:32 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
  292. INFO: Map task executor complete.
  293. Jan 19, 2014 4:14:32 PM org.apache.hadoop.mapred.Task initialize
  294. INFO: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@68e4bf0e
  295. Jan 19, 2014 4:14:32 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  296. INFO:
  297. Jan 19, 2014 4:14:32 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
  298. INFO: Merging 1 sorted segments
  299. Jan 19, 2014 4:14:32 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
  300. INFO: Down to the last merge-pass, with 1 segments left of total size: 14870362 bytes
  301. Jan 19, 2014 4:14:32 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  302. INFO:
  303. Jan 19, 2014 4:14:32 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  304. INFO: map 100% reduce 0%
  305. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Task done
  306. INFO: Task:attempt_local1434897154_0003_r_000000_0 is done. And is in the process of commiting
  307. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  308. INFO:
  309. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Task commit
  310. INFO: Task attempt_local1434897154_0003_r_000000_0 is allowed to commit now
  311. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter commitTask
  312. INFO: Saved output of task 'attempt_local1434897154_0003_r_000000_0' to /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/partial-vectors-0
  313. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  314. INFO: reduce > reduce
  315. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Task sendDone
  316. INFO: Task 'attempt_local1434897154_0003_r_000000_0' done.
  317. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  318. INFO: map 100% reduce 100%
  319. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  320. INFO: Job complete: job_local1434897154_0003
  321. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  322. INFO: Counters: 20
  323. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  324. INFO: File Output Format Counters
  325. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  326. INFO: Bytes Written=17780621
  327. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  328. INFO: File Input Format Counters
  329. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  330. INFO: Bytes Read=15194092
  331. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  332. INFO: FileSystemCounters
  333. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  334. INFO: FILE_BYTES_READ=270285532
  335. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  336. INFO: FILE_BYTES_WRITTEN=237731253
  337. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  338. INFO: Map-Reduce Framework
  339. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  340. INFO: Reduce input groups=21578
  341. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  342. INFO: Map output materialized bytes=14870366
  343. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  344. INFO: Combine output records=0
  345. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  346. INFO: Map input records=21578
  347. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  348. INFO: Reduce shuffle bytes=0
  349. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  350. INFO: Physical memory (bytes) snapshot=0
  351. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  352. INFO: Reduce output records=21578
  353. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  354. INFO: Spilled Records=43156
  355. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  356. INFO: Map output bytes=14791487
  357. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  358. INFO: Total committed heap usage (bytes)=1870462976
  359. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  360. INFO: CPU time spent (ms)=0
  361. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  362. INFO: Virtual memory (bytes) snapshot=0
  363. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  364. INFO: SPLIT_RAW_BYTES=167
  365. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  366. INFO: Map output records=21578
  367. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  368. INFO: Combine input records=0
  369. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Counters log
  370. INFO: Reduce input records=21578
  371. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapreduce.lib.input.FileInputFormat listStatus
  372. INFO: Total input paths to process : 1
  373. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  374. INFO: Running job: job_local2086654198_0004
  375. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
  376. INFO: Waiting for map tasks
  377. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable run
  378. INFO: Starting task: attempt_local2086654198_0004_m_000000_0
  379. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.Task initialize
  380. INFO: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@21929486
  381. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.MapTask runNewMapper
  382. INFO: Processing split: file:/tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/partial-vectors-0/part-r-00000:0+17642777
  383. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  384. INFO: io.sort.mb = 100
  385. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  386. INFO: data buffer = 79691776/99614720
  387. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  388. INFO: record buffer = 262144/327680
  389. Jan 19, 2014 4:14:33 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer flush
  390. INFO: Starting flush of map output
  391. Jan 19, 2014 4:14:34 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer sortAndSpill
  392. INFO: Finished spill 0
  393. Jan 19, 2014 4:14:34 PM org.apache.hadoop.mapred.Task done
  394. INFO: Task:attempt_local2086654198_0004_m_000000_0 is done. And is in the process of commiting
  395. Jan 19, 2014 4:14:34 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  396. INFO:
  397. Jan 19, 2014 4:14:34 PM org.apache.hadoop.mapred.Task sendDone
  398. INFO: Task 'attempt_local2086654198_0004_m_000000_0' done.
  399. Jan 19, 2014 4:14:34 PM org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable run
  400. INFO: Finishing task: attempt_local2086654198_0004_m_000000_0
  401. Jan 19, 2014 4:14:34 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
  402. INFO: Map task executor complete.
  403. Jan 19, 2014 4:14:34 PM org.apache.hadoop.mapred.Task initialize
  404. INFO: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@33ae7f61
  405. Jan 19, 2014 4:14:34 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  406. INFO:
  407. Jan 19, 2014 4:14:34 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
  408. INFO: Merging 1 sorted segments
  409. Jan 19, 2014 4:14:34 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
  410. INFO: Down to the last merge-pass, with 1 segments left of total size: 17420434 bytes
  411. Jan 19, 2014 4:14:34 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  412. INFO:
  413. Jan 19, 2014 4:14:34 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  414. INFO: map 100% reduce 0%
  415. Jan 19, 2014 4:14:34 PM org.apache.hadoop.mapred.Task done
  416. INFO: Task:attempt_local2086654198_0004_r_000000_0 is done. And is in the process of commiting
  417. Jan 19, 2014 4:14:34 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  418. INFO:
  419. Jan 19, 2014 4:14:34 PM org.apache.hadoop.mapred.Task commit
  420. INFO: Task attempt_local2086654198_0004_r_000000_0 is allowed to commit now
  421. Jan 19, 2014 4:14:34 PM org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter commitTask
  422. INFO: Saved output of task 'attempt_local2086654198_0004_r_000000_0' to /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/tf-vectors-toprune
  423. Jan 19, 2014 4:14:34 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  424. INFO: reduce > reduce
  425. Jan 19, 2014 4:14:34 PM org.apache.hadoop.mapred.Task sendDone
  426. INFO: Task 'attempt_local2086654198_0004_r_000000_0' done.
  427. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  428. INFO: map 100% reduce 100%
  429. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  430. INFO: Job complete: job_local2086654198_0004
  431. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  432. INFO: Counters: 20
  433. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  434. INFO: File Output Format Counters
  435. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  436. INFO: Bytes Written=17780621
  437. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  438. INFO: File Input Format Counters
  439. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  440. INFO: Bytes Read=17780621
  441. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  442. INFO: FileSystemCounters
  443. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  444. INFO: FILE_BYTES_READ=387084660
  445. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  446. INFO: FILE_BYTES_WRITTEN=356732169
  447. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  448. INFO: Map-Reduce Framework
  449. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  450. INFO: Reduce input groups=21578
  451. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  452. INFO: Map output materialized bytes=17420438
  453. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  454. INFO: Combine output records=0
  455. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  456. INFO: Map input records=21578
  457. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  458. INFO: Reduce shuffle bytes=0
  459. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  460. INFO: Physical memory (bytes) snapshot=0
  461. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  462. INFO: Reduce output records=21578
  463. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  464. INFO: Spilled Records=43156
  465. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  466. INFO: Map output bytes=17337483
  467. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  468. INFO: Total committed heap usage (bytes)=2641625088
  469. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  470. INFO: CPU time spent (ms)=0
  471. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  472. INFO: Virtual memory (bytes) snapshot=0
  473. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  474. INFO: SPLIT_RAW_BYTES=165
  475. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  476. INFO: Map output records=21578
  477. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  478. INFO: Combine input records=0
  479. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Counters log
  480. INFO: Reduce input records=21578
  481. Jan 19, 2014 4:14:35 PM org.slf4j.impl.JCLLoggerAdapter info
  482. INFO: Deleting /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/partial-vectors-0
  483. Jan 19, 2014 4:14:35 PM org.slf4j.impl.JCLLoggerAdapter info
  484. INFO: Calculating IDF
  485. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapreduce.lib.input.FileInputFormat listStatus
  486. INFO: Total input paths to process : 1
  487. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  488. INFO: Running job: job_local2036671578_0005
  489. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
  490. INFO: Waiting for map tasks
  491. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable run
  492. INFO: Starting task: attempt_local2036671578_0005_m_000000_0
  493. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.Task initialize
  494. INFO: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@200c09a6
  495. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.MapTask runNewMapper
  496. INFO: Processing split: file:/tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/tf-vectors-toprune/part-r-00000:0+17642777
  497. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  498. INFO: io.sort.mb = 100
  499. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  500. INFO: data buffer = 79691776/99614720
  501. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  502. INFO: record buffer = 262144/327680
  503. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer collect
  504. INFO: Spilling map output: record full = true
  505. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer startSpill
  506. INFO: bufstart = 0; bufend = 3145728; bufvoid = 99614720
  507. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer startSpill
  508. INFO: kvstart = 0; kvend = 262144; length = 327680
  509. Jan 19, 2014 4:14:35 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer sortAndSpill
  510. INFO: Finished spill 0
  511. Jan 19, 2014 4:14:36 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer collect
  512. INFO: Spilling map output: record full = true
  513. Jan 19, 2014 4:14:36 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer startSpill
  514. INFO: bufstart = 3145728; bufend = 6291444; bufvoid = 99614720
  515. Jan 19, 2014 4:14:36 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer startSpill
  516. INFO: kvstart = 262144; kvend = 196607; length = 327680
  517. Jan 19, 2014 4:14:36 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer sortAndSpill
  518. INFO: Finished spill 1
  519. Jan 19, 2014 4:14:36 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer collect
  520. INFO: Spilling map output: record full = true
  521. Jan 19, 2014 4:14:36 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer startSpill
  522. INFO: bufstart = 6291444; bufend = 9437160; bufvoid = 99614720
  523. Jan 19, 2014 4:14:36 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer startSpill
  524. INFO: kvstart = 196607; kvend = 131070; length = 327680
  525. Jan 19, 2014 4:14:36 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer sortAndSpill
  526. INFO: Finished spill 2
  527. Jan 19, 2014 4:14:36 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer collect
  528. INFO: Spilling map output: record full = true
  529. Jan 19, 2014 4:14:36 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer startSpill
  530. INFO: bufstart = 9437160; bufend = 12582876; bufvoid = 99614720
  531. Jan 19, 2014 4:14:36 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer startSpill
  532. INFO: kvstart = 131070; kvend = 65533; length = 327680
  533. Jan 19, 2014 4:14:36 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  534. INFO: map 0% reduce 0%
  535. Jan 19, 2014 4:14:36 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer sortAndSpill
  536. INFO: Finished spill 3
  537. Jan 19, 2014 4:14:36 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer collect
  538. INFO: Spilling map output: record full = true
  539. Jan 19, 2014 4:14:36 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer startSpill
  540. INFO: bufstart = 12582876; bufend = 15728604; bufvoid = 99614720
  541. Jan 19, 2014 4:14:36 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer startSpill
  542. INFO: kvstart = 65533; kvend = 327677; length = 327680
  543. Jan 19, 2014 4:14:36 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer sortAndSpill
  544. INFO: Finished spill 4
  545. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer flush
  546. INFO: Starting flush of map output
  547. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer sortAndSpill
  548. INFO: Finished spill 5
  549. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
  550. INFO: Merging 6 sorted segments
  551. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
  552. INFO: Down to the last merge-pass, with 6 segments left of total size: 1871798 bytes
  553. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Task done
  554. INFO: Task:attempt_local2036671578_0005_m_000000_0 is done. And is in the process of commiting
  555. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  556. INFO:
  557. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Task sendDone
  558. INFO: Task 'attempt_local2036671578_0005_m_000000_0' done.
  559. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable run
  560. INFO: Finishing task: attempt_local2036671578_0005_m_000000_0
  561. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
  562. INFO: Map task executor complete.
  563. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Task initialize
  564. INFO: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@6a102a08
  565. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  566. INFO:
  567. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
  568. INFO: Merging 1 sorted segments
  569. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
  570. INFO: Down to the last merge-pass, with 1 segments left of total size: 585314 bytes
  571. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  572. INFO:
  573. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Task done
  574. INFO: Task:attempt_local2036671578_0005_r_000000_0 is done. And is in the process of commiting
  575. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  576. INFO:
  577. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Task commit
  578. INFO: Task attempt_local2036671578_0005_r_000000_0 is allowed to commit now
  579. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter commitTask
  580. INFO: Saved output of task 'attempt_local2036671578_0005_r_000000_0' to /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/df-count
  581. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  582. INFO: reduce > reduce
  583. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Task sendDone
  584. INFO: Task 'attempt_local2036671578_0005_r_000000_0' done.
  585. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  586. INFO: map 100% reduce 100%
  587. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  588. INFO: Job complete: job_local2036671578_0005
  589. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  590. INFO: Counters: 20
  591. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  592. INFO: File Output Format Counters
  593. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  594. INFO: Bytes Written=851221
  595. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  596. INFO: File Input Format Counters
  597. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  598. INFO: Bytes Read=17780621
  599. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  600. INFO: FileSystemCounters
  601. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  602. INFO: FILE_BYTES_READ=492511852
  603. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  604. INFO: FILE_BYTES_WRITTEN=428876243
  605. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  606. INFO: Map-Reduce Framework
  607. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  608. INFO: Reduce input groups=41808
  609. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  610. INFO: Map output materialized bytes=585318
  611. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  612. INFO: Combine output records=175507
  613. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  614. INFO: Map input records=21578
  615. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  616. INFO: Reduce shuffle bytes=0
  617. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  618. INFO: Physical memory (bytes) snapshot=0
  619. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  620. INFO: Reduce output records=41808
  621. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  622. INFO: Spilled Records=217315
  623. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  624. INFO: Map output bytes=18371184
  625. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  626. INFO: Total committed heap usage (bytes)=2641625088
  627. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  628. INFO: CPU time spent (ms)=0
  629. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  630. INFO: Virtual memory (bytes) snapshot=0
  631. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  632. INFO: SPLIT_RAW_BYTES=166
  633. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  634. INFO: Map output records=1530932
  635. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  636. INFO: Combine input records=1664631
  637. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Counters log
  638. INFO: Reduce input records=41808
  639. Jan 19, 2014 4:14:37 PM org.slf4j.impl.JCLLoggerAdapter info
  640. INFO: Pruning
  641. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapreduce.lib.input.FileInputFormat listStatus
  642. INFO: Total input paths to process : 1
  643. Jan 19, 2014 4:14:37 PM org.apache.hadoop.filecache.TrackerDistributedCacheManager downloadCacheObject
  644. INFO: Creating frequency.file-0 in /tmp/hadoop-frank/mapred/local/archive/-417914292228137401_-1609338674_722557391/file/tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans-work--8948508514380311628 with rwxr-xr-x
  645. Jan 19, 2014 4:14:37 PM org.apache.hadoop.filecache.TrackerDistributedCacheManager downloadCacheObject
  646. INFO: Cached /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/frequency.file-0 as /tmp/hadoop-frank/mapred/local/archive/-417914292228137401_-1609338674_722557391/file/tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/frequency.file-0
  647. Jan 19, 2014 4:14:37 PM org.apache.hadoop.filecache.TrackerDistributedCacheManager localizePublicCacheObject
  648. INFO: Cached /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/frequency.file-0 as /tmp/hadoop-frank/mapred/local/archive/-417914292228137401_-1609338674_722557391/file/tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/frequency.file-0
  649. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  650. INFO: Running job: job_local844366116_0006
  651. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
  652. INFO: Waiting for map tasks
  653. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable run
  654. INFO: Starting task: attempt_local844366116_0006_m_000000_0
  655. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.Task initialize
  656. INFO: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@36ec9d26
  657. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.MapTask runNewMapper
  658. INFO: Processing split: file:/tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/tf-vectors-toprune/part-r-00000:0+17642777
  659. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  660. INFO: io.sort.mb = 100
  661. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  662. INFO: data buffer = 79691776/99614720
  663. Jan 19, 2014 4:14:37 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  664. INFO: record buffer = 262144/327680
  665. Jan 19, 2014 4:14:38 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer flush
  666. INFO: Starting flush of map output
  667. Jan 19, 2014 4:14:38 PM org.apache.hadoop.io.compress.CodecPool getCompressor
  668. INFO: Got brand-new compressor
  669. Jan 19, 2014 4:14:38 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer sortAndSpill
  670. INFO: Finished spill 0
  671. Jan 19, 2014 4:14:38 PM org.apache.hadoop.mapred.Task done
  672. INFO: Task:attempt_local844366116_0006_m_000000_0 is done. And is in the process of commiting
  673. Jan 19, 2014 4:14:38 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  674. INFO:
  675. Jan 19, 2014 4:14:38 PM org.apache.hadoop.mapred.Task sendDone
  676. INFO: Task 'attempt_local844366116_0006_m_000000_0' done.
  677. Jan 19, 2014 4:14:38 PM org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable run
  678. INFO: Finishing task: attempt_local844366116_0006_m_000000_0
  679. Jan 19, 2014 4:14:38 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
  680. INFO: Map task executor complete.
  681. Jan 19, 2014 4:14:38 PM org.apache.hadoop.mapred.Task initialize
  682. INFO: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7df8372b
  683. Jan 19, 2014 4:14:38 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  684. INFO:
  685. Jan 19, 2014 4:14:38 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
  686. INFO: Merging 1 sorted segments
  687. Jan 19, 2014 4:14:38 PM org.apache.hadoop.io.compress.CodecPool getDecompressor
  688. INFO: Got brand-new decompressor
  689. Jan 19, 2014 4:14:38 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
  690. INFO: Down to the last merge-pass, with 1 segments left of total size: 4441445 bytes
  691. Jan 19, 2014 4:14:38 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  692. INFO:
  693. Jan 19, 2014 4:14:38 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  694. INFO: map 100% reduce 0%
  695. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Task done
  696. INFO: Task:attempt_local844366116_0006_r_000000_0 is done. And is in the process of commiting
  697. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  698. INFO:
  699. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Task commit
  700. INFO: Task attempt_local844366116_0006_r_000000_0 is allowed to commit now
  701. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter commitTask
  702. INFO: Saved output of task 'attempt_local844366116_0006_r_000000_0' to /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/tf-vectors-partial/partial-0
  703. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  704. INFO: reduce > reduce
  705. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Task sendDone
  706. INFO: Task 'attempt_local844366116_0006_r_000000_0' done.
  707. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  708. INFO: map 100% reduce 100%
  709. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  710. INFO: Job complete: job_local844366116_0006
  711. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  712. INFO: Counters: 20
  713. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  714. INFO: File Output Format Counters
  715. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  716. INFO: Bytes Written=17156391
  717. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  718. INFO: File Input Format Counters
  719. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  720. INFO: Bytes Read=17780621
  721. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  722. INFO: FileSystemCounters
  723. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  724. INFO: FILE_BYTES_READ=585472456
  725. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  726. INFO: FILE_BYTES_WRITTEN=507772371
  727. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  728. INFO: Map-Reduce Framework
  729. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  730. INFO: Reduce input groups=21578
  731. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  732. INFO: Map output materialized bytes=4441449
  733. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  734. INFO: Combine output records=0
  735. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  736. INFO: Map input records=21578
  737. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  738. INFO: Reduce shuffle bytes=0
  739. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  740. INFO: Physical memory (bytes) snapshot=0
  741. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  742. INFO: Reduce output records=21578
  743. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  744. INFO: Spilled Records=43156
  745. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  746. INFO: Map output bytes=17337483
  747. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  748. INFO: Total committed heap usage (bytes)=2641625088
  749. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  750. INFO: CPU time spent (ms)=0
  751. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  752. INFO: Virtual memory (bytes) snapshot=0
  753. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  754. INFO: SPLIT_RAW_BYTES=166
  755. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  756. INFO: Map output records=21578
  757. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  758. INFO: Combine input records=0
  759. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Counters log
  760. INFO: Reduce input records=21578
  761. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapreduce.lib.input.FileInputFormat listStatus
  762. INFO: Total input paths to process : 1
  763. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  764. INFO: Running job: job_local2006589475_0007
  765. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
  766. INFO: Waiting for map tasks
  767. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable run
  768. INFO: Starting task: attempt_local2006589475_0007_m_000000_0
  769. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.Task initialize
  770. INFO: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@10bcbb94
  771. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.MapTask runNewMapper
  772. INFO: Processing split: file:/tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/tf-vectors-partial/partial-0/part-r-00000:0+17023387
  773. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  774. INFO: io.sort.mb = 100
  775. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  776. INFO: data buffer = 79691776/99614720
  777. Jan 19, 2014 4:14:39 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  778. INFO: record buffer = 262144/327680
  779. Jan 19, 2014 4:14:40 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer flush
  780. INFO: Starting flush of map output
  781. Jan 19, 2014 4:14:40 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer sortAndSpill
  782. INFO: Finished spill 0
  783. Jan 19, 2014 4:14:40 PM org.apache.hadoop.mapred.Task done
  784. INFO: Task:attempt_local2006589475_0007_m_000000_0 is done. And is in the process of commiting
  785. Jan 19, 2014 4:14:40 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  786. INFO:
  787. Jan 19, 2014 4:14:40 PM org.apache.hadoop.mapred.Task sendDone
  788. INFO: Task 'attempt_local2006589475_0007_m_000000_0' done.
  789. Jan 19, 2014 4:14:40 PM org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable run
  790. INFO: Finishing task: attempt_local2006589475_0007_m_000000_0
  791. Jan 19, 2014 4:14:40 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
  792. INFO: Map task executor complete.
  793. Jan 19, 2014 4:14:40 PM org.apache.hadoop.mapred.Task initialize
  794. INFO: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@671373d3
  795. Jan 19, 2014 4:14:40 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  796. INFO:
  797. Jan 19, 2014 4:14:40 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
  798. INFO: Merging 1 sorted segments
  799. Jan 19, 2014 4:14:40 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
  800. INFO: Down to the last merge-pass, with 1 segments left of total size: 16804860 bytes
  801. Jan 19, 2014 4:14:40 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  802. INFO:
  803. Jan 19, 2014 4:14:40 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  804. INFO: map 100% reduce 0%
  805. Jan 19, 2014 4:14:40 PM org.apache.hadoop.mapred.Task done
  806. INFO: Task:attempt_local2006589475_0007_r_000000_0 is done. And is in the process of commiting
  807. Jan 19, 2014 4:14:40 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  808. INFO:
  809. Jan 19, 2014 4:14:40 PM org.apache.hadoop.mapred.Task commit
  810. INFO: Task attempt_local2006589475_0007_r_000000_0 is allowed to commit now
  811. Jan 19, 2014 4:14:40 PM org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter commitTask
  812. INFO: Saved output of task 'attempt_local2006589475_0007_r_000000_0' to /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/tf-vectors
  813. Jan 19, 2014 4:14:40 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  814. INFO: reduce > reduce
  815. Jan 19, 2014 4:14:40 PM org.apache.hadoop.mapred.Task sendDone
  816. INFO: Task 'attempt_local2006589475_0007_r_000000_0' done.
  817. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  818. INFO: map 100% reduce 100%
  819. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  820. INFO: Job complete: job_local2006589475_0007
  821. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  822. INFO: Counters: 20
  823. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  824. INFO: File Output Format Counters
  825. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  826. INFO: Bytes Written=17156391
  827. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  828. INFO: File Input Format Counters
  829. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  830. INFO: Bytes Read=17156391
  831. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  832. INFO: FileSystemCounters
  833. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  834. INFO: FILE_BYTES_READ=689999322
  835. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  836. INFO: FILE_BYTES_WRITTEN=624292581
  837. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  838. INFO: Map-Reduce Framework
  839. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  840. INFO: Reduce input groups=21578
  841. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  842. INFO: Map output materialized bytes=16804864
  843. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  844. INFO: Combine output records=0
  845. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  846. INFO: Map input records=21578
  847. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  848. INFO: Reduce shuffle bytes=0
  849. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  850. INFO: Physical memory (bytes) snapshot=0
  851. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  852. INFO: Reduce output records=21578
  853. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  854. INFO: Spilled Records=43156
  855. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  856. INFO: Map output bytes=16722273
  857. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  858. INFO: Total committed heap usage (bytes)=2642018304
  859. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  860. INFO: CPU time spent (ms)=0
  861. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  862. INFO: Virtual memory (bytes) snapshot=0
  863. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  864. INFO: SPLIT_RAW_BYTES=176
  865. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  866. INFO: Map output records=21578
  867. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  868. INFO: Combine input records=0
  869. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.Counters log
  870. INFO: Reduce input records=21578
  871. Jan 19, 2014 4:14:41 PM org.slf4j.impl.JCLLoggerAdapter info
  872. INFO: Deleting /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/tf-vectors-partial
  873. Jan 19, 2014 4:14:41 PM org.slf4j.impl.JCLLoggerAdapter info
  874. INFO: Deleting /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/tf-vectors-toprune
  875. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapreduce.lib.input.FileInputFormat listStatus
  876. INFO: Total input paths to process : 1
  877. Jan 19, 2014 4:14:41 PM org.apache.hadoop.filecache.TrackerDistributedCacheManager downloadCacheObject
  878. INFO: Creating frequency.file-0 in /tmp/hadoop-frank/mapred/local/archive/-3018431259644783169_-1609338674_722557391/file/tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans-work-7108851975081586509 with rwxr-xr-x
  879. Jan 19, 2014 4:14:41 PM org.apache.hadoop.filecache.TrackerDistributedCacheManager downloadCacheObject
  880. INFO: Cached /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/frequency.file-0 as /tmp/hadoop-frank/mapred/local/archive/-3018431259644783169_-1609338674_722557391/file/tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/frequency.file-0
  881. Jan 19, 2014 4:14:41 PM org.apache.hadoop.filecache.TrackerDistributedCacheManager localizePublicCacheObject
  882. INFO: Cached /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/frequency.file-0 as /tmp/hadoop-frank/mapred/local/archive/-3018431259644783169_-1609338674_722557391/file/tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/frequency.file-0
  883. Jan 19, 2014 4:14:41 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  884. INFO: Running job: job_local692095679_0008
  885. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
  886. INFO: Waiting for map tasks
  887. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable run
  888. INFO: Starting task: attempt_local692095679_0008_m_000000_0
  889. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.Task initialize
  890. INFO: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7e8132ba
  891. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.MapTask runNewMapper
  892. INFO: Processing split: file:/tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/tf-vectors/part-r-00000:0+17023387
  893. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  894. INFO: io.sort.mb = 100
  895. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  896. INFO: data buffer = 79691776/99614720
  897. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  898. INFO: record buffer = 262144/327680
  899. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer flush
  900. INFO: Starting flush of map output
  901. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer sortAndSpill
  902. INFO: Finished spill 0
  903. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.Task done
  904. INFO: Task:attempt_local692095679_0008_m_000000_0 is done. And is in the process of commiting
  905. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  906. INFO:
  907. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.Task sendDone
  908. INFO: Task 'attempt_local692095679_0008_m_000000_0' done.
  909. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable run
  910. INFO: Finishing task: attempt_local692095679_0008_m_000000_0
  911. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
  912. INFO: Map task executor complete.
  913. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.Task initialize
  914. INFO: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@6f51dda8
  915. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  916. INFO:
  917. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
  918. INFO: Merging 1 sorted segments
  919. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
  920. INFO: Down to the last merge-pass, with 1 segments left of total size: 16804860 bytes
  921. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  922. INFO:
  923. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.Task done
  924. INFO: Task:attempt_local692095679_0008_r_000000_0 is done. And is in the process of commiting
  925. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  926. INFO:
  927. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.Task commit
  928. INFO: Task attempt_local692095679_0008_r_000000_0 is allowed to commit now
  929. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter commitTask
  930. INFO: Saved output of task 'attempt_local692095679_0008_r_000000_0' to /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/partial-vectors-0
  931. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  932. INFO: reduce > reduce
  933. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.Task sendDone
  934. INFO: Task 'attempt_local692095679_0008_r_000000_0' done.
  935. Jan 19, 2014 4:14:42 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  936. INFO: map 100% reduce 100%
  937. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  938. INFO: Job complete: job_local692095679_0008
  939. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  940. INFO: Counters: 20
  941. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  942. INFO: File Output Format Counters
  943. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  944. INFO: Bytes Written=17156391
  945. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  946. INFO: File Input Format Counters
  947. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  948. INFO: Bytes Read=17156391
  949. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  950. INFO: FileSystemCounters
  951. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  952. INFO: FILE_BYTES_READ=808591969
  953. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  954. INFO: FILE_BYTES_WRITTEN=742519833
  955. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  956. INFO: Map-Reduce Framework
  957. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  958. INFO: Reduce input groups=21578
  959. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  960. INFO: Map output materialized bytes=16804864
  961. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  962. INFO: Combine output records=0
  963. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  964. INFO: Map input records=21578
  965. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  966. INFO: Reduce shuffle bytes=0
  967. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  968. INFO: Physical memory (bytes) snapshot=0
  969. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  970. INFO: Reduce output records=21578
  971. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  972. INFO: Spilled Records=43156
  973. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  974. INFO: Map output bytes=16722273
  975. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  976. INFO: Total committed heap usage (bytes)=2642018304
  977. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  978. INFO: CPU time spent (ms)=0
  979. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  980. INFO: Virtual memory (bytes) snapshot=0
  981. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  982. INFO: SPLIT_RAW_BYTES=158
  983. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  984. INFO: Map output records=21578
  985. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  986. INFO: Combine input records=0
  987. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Counters log
  988. INFO: Reduce input records=21578
  989. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapreduce.lib.input.FileInputFormat listStatus
  990. INFO: Total input paths to process : 1
  991. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  992. INFO: Running job: job_local1372352436_0009
  993. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
  994. INFO: Waiting for map tasks
  995. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable run
  996. INFO: Starting task: attempt_local1372352436_0009_m_000000_0
  997. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Task initialize
  998. INFO: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@57bfdbc9
  999. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.MapTask runNewMapper
  1000. INFO: Processing split: file:/tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/partial-vectors-0/part-r-00000:0+17023387
  1001. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  1002. INFO: io.sort.mb = 100
  1003. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  1004. INFO: data buffer = 79691776/99614720
  1005. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  1006. INFO: record buffer = 262144/327680
  1007. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer flush
  1008. INFO: Starting flush of map output
  1009. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer sortAndSpill
  1010. INFO: Finished spill 0
  1011. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Task done
  1012. INFO: Task:attempt_local1372352436_0009_m_000000_0 is done. And is in the process of commiting
  1013. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1014. INFO:
  1015. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Task sendDone
  1016. INFO: Task 'attempt_local1372352436_0009_m_000000_0' done.
  1017. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable run
  1018. INFO: Finishing task: attempt_local1372352436_0009_m_000000_0
  1019. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
  1020. INFO: Map task executor complete.
  1021. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Task initialize
  1022. INFO: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@5550040b
  1023. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1024. INFO:
  1025. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
  1026. INFO: Merging 1 sorted segments
  1027. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
  1028. INFO: Down to the last merge-pass, with 1 segments left of total size: 16804860 bytes
  1029. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1030. INFO:
  1031. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Task done
  1032. INFO: Task:attempt_local1372352436_0009_r_000000_0 is done. And is in the process of commiting
  1033. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1034. INFO:
  1035. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Task commit
  1036. INFO: Task attempt_local1372352436_0009_r_000000_0 is allowed to commit now
  1037. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter commitTask
  1038. INFO: Saved output of task 'attempt_local1372352436_0009_r_000000_0' to /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/tfidf-vectors
  1039. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1040. INFO: reduce > reduce
  1041. Jan 19, 2014 4:14:43 PM org.apache.hadoop.mapred.Task sendDone
  1042. INFO: Task 'attempt_local1372352436_0009_r_000000_0' done.
  1043. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1044. INFO: map 100% reduce 100%
  1045. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1046. INFO: Job complete: job_local1372352436_0009
  1047. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1048. INFO: Counters: 20
  1049. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1050. INFO: File Output Format Counters
  1051. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1052. INFO: Bytes Written=17156391
  1053. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1054. INFO: File Input Format Counters
  1055. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1056. INFO: Bytes Read=17156391
  1057. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1058. INFO: FileSystemCounters
  1059. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1060. INFO: FILE_BYTES_READ=925482228
  1061. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1062. INFO: FILE_BYTES_WRITTEN=859041121
  1063. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1064. INFO: Map-Reduce Framework
  1065. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1066. INFO: Reduce input groups=21578
  1067. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1068. INFO: Map output materialized bytes=16804864
  1069. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1070. INFO: Combine output records=0
  1071. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1072. INFO: Map input records=21578
  1073. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1074. INFO: Reduce shuffle bytes=0
  1075. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1076. INFO: Physical memory (bytes) snapshot=0
  1077. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1078. INFO: Reduce output records=21578
  1079. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1080. INFO: Spilled Records=43156
  1081. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1082. INFO: Map output bytes=16722273
  1083. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1084. INFO: Total committed heap usage (bytes)=2668363776
  1085. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1086. INFO: CPU time spent (ms)=0
  1087. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1088. INFO: Virtual memory (bytes) snapshot=0
  1089. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1090. INFO: SPLIT_RAW_BYTES=165
  1091. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1092. INFO: Map output records=21578
  1093. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1094. INFO: Combine input records=0
  1095. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapred.Counters log
  1096. INFO: Reduce input records=21578
  1097. Jan 19, 2014 4:14:44 PM org.slf4j.impl.JCLLoggerAdapter info
  1098. INFO: Deleting /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/partial-vectors-0
  1099. Jan 19, 2014 4:14:44 PM org.slf4j.impl.JCLLoggerAdapter info
  1100. INFO: Program took 18556 ms (Minutes: 0.3092666666666667)
  1101. MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
  1102. hadoop binary is not in PATH,HADOOP_HOME/bin,HADOOP_PREFIX/bin, running locally
  1103. SLF4J: Class path contains multiple SLF4J bindings.
  1104. SLF4J: Found binding in [jar:file:/home/frank/Desktop/mahout-distribution-0.9/examples/target/mahout-examples-0.9-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]
  1105. SLF4J: Found binding in [jar:file:/home/frank/Desktop/mahout-distribution-0.9/examples/target/dependency/slf4j-jcl-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
  1106. SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
  1107. SLF4J: Actual binding is of type [org.slf4j.impl.JCLLoggerFactory]
  1108. Jan 19, 2014 4:14:44 PM org.slf4j.impl.JCLLoggerAdapter warn
  1109. WARNING: No streamingkmeans.props found on classpath, will use command-line arguments only
  1110. Jan 19, 2014 4:14:44 PM org.slf4j.impl.JCLLoggerAdapter info
  1111. INFO: Command line arguments: {--distanceMeasure=[org.apache.mahout.common.distance.SquaredEuclideanDistanceMeasure], --endPhase=[2147483647], --estimatedDistanceCutoff=[-1.0], --estimatedNumMapClusters=[100], --input=[/tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/tfidf-vectors/], --maxNumIterations=[10], --method=[mapreduce], --numBallKMeansRuns=[4], --numClusters=[10], --numProjections=[3], --output=[/tmp/mahout-work-frank/reuters-streamingkmeans], --overwrite=null, --reduceStreamingKMeans=null, --searchSize=[2], --searcherClass=[org.apache.mahout.math.neighborhood.FastProjectionSearch], --startPhase=[0], --tempDir=[/tmp/mahout-work-frank/tmp], --testProbability=[0.1], --trimFraction=[0.9]}
  1112. Jan 19, 2014 4:14:44 PM org.slf4j.impl.JCLLoggerAdapter info
  1113. INFO: Starting to configure options for workers
  1114. Jan 19, 2014 4:14:44 PM org.slf4j.impl.JCLLoggerAdapter info
  1115. INFO: Parameters are: [k] numClusters 10; [SKM] estimatedNumMapClusters 100; estimatedDistanceCutoff -1.0 [BKM] maxNumIterations 10; trimFraction 0.9; randomInit false; ignoreWeights false; testProbability 0.1; numBallKMeansRuns 4; [S] measureClass org.apache.mahout.common.distance.SquaredEuclideanDistanceMeasure; searcherClass org.apache.mahout.math.neighborhood.FastProjectionSearch; searcherSize 2; numProjections 3; method mapreduce; reduceStreamingKMeans true
  1116. Jan 19, 2014 4:14:44 PM org.slf4j.impl.JCLLoggerAdapter info
  1117. INFO: Starting StreamingKMeans clustering for vectors in /tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/tfidf-vectors; results are output to /tmp/mahout-work-frank/reuters-streamingkmeans
  1118. Jan 19, 2014 4:14:44 PM org.apache.hadoop.util.NativeCodeLoader <clinit>
  1119. WARNING: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  1120. Jan 19, 2014 4:14:44 PM org.apache.hadoop.mapreduce.lib.input.FileInputFormat listStatus
  1121. INFO: Total input paths to process : 1
  1122. Jan 19, 2014 4:14:45 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1123. INFO: Running job: job_local1791603598_0001
  1124. Jan 19, 2014 4:14:45 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
  1125. INFO: Waiting for map tasks
  1126. Jan 19, 2014 4:14:45 PM org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable run
  1127. INFO: Starting task: attempt_local1791603598_0001_m_000000_0
  1128. Jan 19, 2014 4:14:45 PM org.apache.hadoop.util.ProcessTree isSetsidSupported
  1129. INFO: setsid exited with exit code 0
  1130. Jan 19, 2014 4:14:45 PM org.apache.hadoop.mapred.Task initialize
  1131. INFO: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@f5db507
  1132. Jan 19, 2014 4:14:45 PM org.apache.hadoop.mapred.MapTask runNewMapper
  1133. INFO: Processing split: file:/tmp/mahout-work-frank/reuters-out-seqdir-sparse-streamingkmeans/tfidf-vectors/part-r-00000:0+17023387
  1134. Jan 19, 2014 4:14:45 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  1135. INFO: io.sort.mb = 100
  1136. Jan 19, 2014 4:14:45 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  1137. INFO: data buffer = 79691776/99614720
  1138. Jan 19, 2014 4:14:45 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
  1139. INFO: record buffer = 262144/327680
  1140. Jan 19, 2014 4:14:46 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1141. INFO: map 0% reduce 0%
  1142. Jan 19, 2014 4:14:51 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1143. INFO:
  1144. Jan 19, 2014 4:14:52 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1145. INFO: map 6% reduce 0%
  1146. Jan 19, 2014 4:14:54 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1147. INFO:
  1148. Jan 19, 2014 4:14:55 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1149. INFO: map 8% reduce 0%
  1150. Jan 19, 2014 4:14:57 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1151. INFO:
  1152. Jan 19, 2014 4:14:58 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1153. INFO: map 10% reduce 0%
  1154. Jan 19, 2014 4:15:00 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1155. INFO:
  1156. Jan 19, 2014 4:15:01 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1157. INFO: map 11% reduce 0%
  1158. Jan 19, 2014 4:15:03 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1159. INFO:
  1160. Jan 19, 2014 4:15:04 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1161. INFO: map 12% reduce 0%
  1162. Jan 19, 2014 4:15:06 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1163. INFO:
  1164. Jan 19, 2014 4:15:07 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1165. INFO: map 13% reduce 0%
  1166. Jan 19, 2014 4:15:09 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1167. INFO:
  1168. Jan 19, 2014 4:15:10 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1169. INFO: map 15% reduce 0%
  1170. Jan 19, 2014 4:15:12 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1171. INFO:
  1172. Jan 19, 2014 4:15:13 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1173. INFO: map 17% reduce 0%
  1174. Jan 19, 2014 4:15:15 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1175. INFO:
  1176. Jan 19, 2014 4:15:16 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1177. INFO: map 18% reduce 0%
  1178. Jan 19, 2014 4:15:18 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1179. INFO:
  1180. Jan 19, 2014 4:15:19 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1181. INFO: map 20% reduce 0%
  1182. Jan 19, 2014 4:15:21 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1183. INFO:
  1184. Jan 19, 2014 4:15:24 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1185. INFO:
  1186. Jan 19, 2014 4:15:25 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1187. INFO: map 22% reduce 0%
  1188. Jan 19, 2014 4:15:27 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1189. INFO:
  1190. Jan 19, 2014 4:15:28 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1191. INFO: map 24% reduce 0%
  1192. Jan 19, 2014 4:15:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1193. INFO:
  1194. Jan 19, 2014 4:15:31 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1195. INFO: map 25% reduce 0%
  1196. Jan 19, 2014 4:15:33 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1197. INFO:
  1198. Jan 19, 2014 4:15:34 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1199. INFO: map 26% reduce 0%
  1200. Jan 19, 2014 4:15:36 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1201. INFO:
  1202. Jan 19, 2014 4:15:37 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1203. INFO: map 27% reduce 0%
  1204. Jan 19, 2014 4:15:39 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1205. INFO:
  1206. Jan 19, 2014 4:15:40 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1207. INFO: map 28% reduce 0%
  1208. Jan 19, 2014 4:15:42 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1209. INFO:
  1210. Jan 19, 2014 4:15:43 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1211. INFO: map 29% reduce 0%
  1212. Jan 19, 2014 4:15:45 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1213. INFO:
  1214. Jan 19, 2014 4:15:46 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1215. INFO: map 30% reduce 0%
  1216. Jan 19, 2014 4:15:48 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1217. INFO:
  1218. Jan 19, 2014 4:15:49 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1219. INFO: map 31% reduce 0%
  1220. Jan 19, 2014 4:15:51 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1221. INFO:
  1222. Jan 19, 2014 4:15:52 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1223. INFO: map 32% reduce 0%
  1224. Jan 19, 2014 4:15:54 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1225. INFO:
  1226. Jan 19, 2014 4:15:55 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1227. INFO: map 33% reduce 0%
  1228. Jan 19, 2014 4:15:57 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1229. INFO:
  1230. Jan 19, 2014 4:16:00 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1231. INFO:
  1232. Jan 19, 2014 4:16:01 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1233. INFO: map 34% reduce 0%
  1234. Jan 19, 2014 4:16:03 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1235. INFO:
  1236. Jan 19, 2014 4:16:04 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1237. INFO: map 35% reduce 0%
  1238. Jan 19, 2014 4:16:06 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1239. INFO:
  1240. Jan 19, 2014 4:16:07 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1241. INFO: map 36% reduce 0%
  1242. Jan 19, 2014 4:16:09 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1243. INFO:
  1244. Jan 19, 2014 4:16:10 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1245. INFO: map 37% reduce 0%
  1246. Jan 19, 2014 4:16:12 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1247. INFO:
  1248. Jan 19, 2014 4:16:15 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1249. INFO:
  1250. Jan 19, 2014 4:16:16 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1251. INFO: map 38% reduce 0%
  1252. Jan 19, 2014 4:16:18 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1253. INFO:
  1254. Jan 19, 2014 4:16:19 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1255. INFO: map 39% reduce 0%
  1256. Jan 19, 2014 4:16:21 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1257. INFO:
  1258. Jan 19, 2014 4:16:24 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1259. INFO:
  1260. Jan 19, 2014 4:16:25 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1261. INFO: map 41% reduce 0%
  1262. Jan 19, 2014 4:16:27 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1263. INFO:
  1264. Jan 19, 2014 4:16:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1265. INFO:
  1266. Jan 19, 2014 4:16:31 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1267. INFO: map 43% reduce 0%
  1268. Jan 19, 2014 4:16:33 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1269. INFO:
  1270. Jan 19, 2014 4:16:36 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1271. INFO:
  1272. Jan 19, 2014 4:16:37 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1273. INFO: map 44% reduce 0%
  1274. Jan 19, 2014 4:16:39 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1275. INFO:
  1276. Jan 19, 2014 4:16:40 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1277. INFO: map 45% reduce 0%
  1278. Jan 19, 2014 4:16:42 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1279. INFO:
  1280. Jan 19, 2014 4:16:45 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1281. INFO:
  1282. Jan 19, 2014 4:16:46 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1283. INFO: map 47% reduce 0%
  1284. Jan 19, 2014 4:16:48 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1285. INFO:
  1286. Jan 19, 2014 4:16:51 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1287. INFO:
  1288. Jan 19, 2014 4:16:52 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1289. INFO: map 48% reduce 0%
  1290. Jan 19, 2014 4:16:54 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1291. INFO:
  1292. Jan 19, 2014 4:16:55 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1293. INFO: map 49% reduce 0%
  1294. Jan 19, 2014 4:16:57 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1295. INFO:
  1296. Jan 19, 2014 4:16:58 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1297. INFO: map 50% reduce 0%
  1298. Jan 19, 2014 4:17:00 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1299. INFO:
  1300. Jan 19, 2014 4:17:01 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1301. INFO: map 51% reduce 0%
  1302. Jan 19, 2014 4:17:03 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1303. INFO:
  1304. Jan 19, 2014 4:17:06 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1305. INFO:
  1306. Jan 19, 2014 4:17:07 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1307. INFO: map 52% reduce 0%
  1308. Jan 19, 2014 4:17:09 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1309. INFO:
  1310. Jan 19, 2014 4:17:10 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1311. INFO: map 53% reduce 0%
  1312. Jan 19, 2014 4:17:12 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1313. INFO:
  1314. Jan 19, 2014 4:17:13 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1315. INFO: map 54% reduce 0%
  1316. Jan 19, 2014 4:17:15 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1317. INFO:
  1318. Jan 19, 2014 4:17:18 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1319. INFO:
  1320. Jan 19, 2014 4:17:19 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1321. INFO: map 55% reduce 0%
  1322. Jan 19, 2014 4:17:21 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1323. INFO:
  1324. Jan 19, 2014 4:17:24 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1325. INFO:
  1326. Jan 19, 2014 4:17:25 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1327. INFO: map 56% reduce 0%
  1328. Jan 19, 2014 4:17:27 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1329. INFO:
  1330. Jan 19, 2014 4:17:28 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1331. INFO: map 57% reduce 0%
  1332. Jan 19, 2014 4:17:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1333. INFO:
  1334. Jan 19, 2014 4:17:33 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1335. INFO:
  1336. Jan 19, 2014 4:17:34 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1337. INFO: map 58% reduce 0%
  1338. Jan 19, 2014 4:17:36 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1339. INFO:
  1340. Jan 19, 2014 4:17:39 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1341. INFO:
  1342. Jan 19, 2014 4:17:42 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1343. INFO:
  1344. Jan 19, 2014 4:17:43 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1345. INFO: map 59% reduce 0%
  1346. Jan 19, 2014 4:17:45 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1347. INFO:
  1348. Jan 19, 2014 4:17:46 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1349. INFO: map 60% reduce 0%
  1350. Jan 19, 2014 4:17:48 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1351. INFO:
  1352. Jan 19, 2014 4:17:49 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1353. INFO: map 61% reduce 0%
  1354. Jan 19, 2014 4:17:51 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1355. INFO:
  1356. Jan 19, 2014 4:17:52 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1357. INFO: map 62% reduce 0%
  1358. Jan 19, 2014 4:17:54 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1359. INFO:
  1360. Jan 19, 2014 4:17:57 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1361. INFO:
  1362. Jan 19, 2014 4:17:58 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1363. INFO: map 63% reduce 0%
  1364. Jan 19, 2014 4:18:00 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1365. INFO:
  1366. Jan 19, 2014 4:18:03 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1367. INFO:
  1368. Jan 19, 2014 4:18:04 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1369. INFO: map 64% reduce 0%
  1370. Jan 19, 2014 4:18:06 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1371. INFO:
  1372. Jan 19, 2014 4:18:09 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1373. INFO:
  1374. Jan 19, 2014 4:18:10 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1375. INFO: map 65% reduce 0%
  1376. Jan 19, 2014 4:18:12 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1377. INFO:
  1378. Jan 19, 2014 4:18:15 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1379. INFO:
  1380. Jan 19, 2014 4:18:18 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1381. INFO:
  1382. Jan 19, 2014 4:18:21 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1383. INFO:
  1384. Jan 19, 2014 4:18:22 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1385. INFO: map 66% reduce 0%
  1386. Jan 19, 2014 4:18:24 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1387. INFO:
  1388. Jan 19, 2014 4:18:25 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1389. INFO: map 67% reduce 0%
  1390. Jan 19, 2014 4:18:27 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1391. INFO:
  1392. Jan 19, 2014 4:18:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1393. INFO:
  1394. Jan 19, 2014 4:18:33 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1395. INFO:
  1396. Jan 19, 2014 4:18:34 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1397. INFO: map 68% reduce 0%
  1398. Jan 19, 2014 4:18:36 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1399. INFO:
  1400. Jan 19, 2014 4:18:39 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1401. INFO:
  1402. Jan 19, 2014 4:18:40 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1403. INFO: map 69% reduce 0%
  1404. Jan 19, 2014 4:18:42 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1405. INFO:
  1406. Jan 19, 2014 4:18:43 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1407. INFO: map 70% reduce 0%
  1408. Jan 19, 2014 4:18:45 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1409. INFO:
  1410. Jan 19, 2014 4:18:48 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1411. INFO:
  1412. Jan 19, 2014 4:18:49 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1413. INFO: map 71% reduce 0%
  1414. Jan 19, 2014 4:18:51 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1415. INFO:
  1416. Jan 19, 2014 4:18:52 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1417. INFO: map 72% reduce 0%
  1418. Jan 19, 2014 4:18:54 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1419. INFO:
  1420. Jan 19, 2014 4:18:57 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1421. INFO:
  1422. Jan 19, 2014 4:18:58 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1423. INFO: map 73% reduce 0%
  1424. Jan 19, 2014 4:19:00 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1425. INFO:
  1426. Jan 19, 2014 4:19:03 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1427. INFO:
  1428. Jan 19, 2014 4:19:04 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1429. INFO: map 74% reduce 0%
  1430. Jan 19, 2014 4:19:06 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1431. INFO:
  1432. Jan 19, 2014 4:19:07 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1433. INFO: map 75% reduce 0%
  1434. Jan 19, 2014 4:19:09 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1435. INFO:
  1436. Jan 19, 2014 4:19:12 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1437. INFO:
  1438. Jan 19, 2014 4:19:13 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1439. INFO: map 76% reduce 0%
  1440. Jan 19, 2014 4:19:15 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1441. INFO:
  1442. Jan 19, 2014 4:19:18 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1443. INFO:
  1444. Jan 19, 2014 4:19:19 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1445. INFO: map 77% reduce 0%
  1446. Jan 19, 2014 4:19:21 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1447. INFO:
  1448. Jan 19, 2014 4:19:24 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1449. INFO:
  1450. Jan 19, 2014 4:19:25 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1451. INFO: map 78% reduce 0%
  1452. Jan 19, 2014 4:19:27 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1453. INFO:
  1454. Jan 19, 2014 4:19:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1455. INFO:
  1456. Jan 19, 2014 4:19:31 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1457. INFO: map 79% reduce 0%
  1458. Jan 19, 2014 4:19:33 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1459. INFO:
  1460. Jan 19, 2014 4:19:34 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1461. INFO: map 80% reduce 0%
  1462. Jan 19, 2014 4:19:36 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1463. INFO:
  1464. Jan 19, 2014 4:19:37 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1465. INFO: map 82% reduce 0%
  1466. Jan 19, 2014 4:19:39 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1467. INFO:
  1468. Jan 19, 2014 4:19:42 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1469. INFO:
  1470. Jan 19, 2014 4:19:43 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1471. INFO: map 83% reduce 0%
  1472. Jan 19, 2014 4:19:45 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1473. INFO:
  1474. Jan 19, 2014 4:19:48 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1475. INFO:
  1476. Jan 19, 2014 4:19:49 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1477. INFO: map 84% reduce 0%
  1478. Jan 19, 2014 4:19:51 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1479. INFO:
  1480. Jan 19, 2014 4:19:54 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1481. INFO:
  1482. Jan 19, 2014 4:19:55 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1483. INFO: map 85% reduce 0%
  1484. Jan 19, 2014 4:19:57 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1485. INFO:
  1486. Jan 19, 2014 4:20:00 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1487. INFO:
  1488. Jan 19, 2014 4:20:01 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1489. INFO: map 86% reduce 0%
  1490. Jan 19, 2014 4:20:03 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1491. INFO:
  1492. Jan 19, 2014 4:20:04 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1493. INFO: map 87% reduce 0%
  1494. Jan 19, 2014 4:20:06 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1495. INFO:
  1496. Jan 19, 2014 4:20:09 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1497. INFO:
  1498. Jan 19, 2014 4:20:12 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1499. INFO:
  1500. Jan 19, 2014 4:20:13 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1501. INFO: map 88% reduce 0%
  1502. Jan 19, 2014 4:20:15 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1503. INFO:
  1504. Jan 19, 2014 4:20:18 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1505. INFO:
  1506. Jan 19, 2014 4:20:19 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1507. INFO: map 89% reduce 0%
  1508. Jan 19, 2014 4:20:21 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1509. INFO:
  1510. Jan 19, 2014 4:20:24 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1511. INFO:
  1512. Jan 19, 2014 4:20:25 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1513. INFO: map 90% reduce 0%
  1514. Jan 19, 2014 4:20:27 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1515. INFO:
  1516. Jan 19, 2014 4:20:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1517. INFO:
  1518. Jan 19, 2014 4:20:31 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1519. INFO: map 91% reduce 0%
  1520. Jan 19, 2014 4:20:33 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1521. INFO:
  1522. Jan 19, 2014 4:20:34 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1523. INFO: map 92% reduce 0%
  1524. Jan 19, 2014 4:20:36 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1525. INFO:
  1526. Jan 19, 2014 4:20:39 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1527. INFO:
  1528. Jan 19, 2014 4:20:40 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1529. INFO: map 93% reduce 0%
  1530. Jan 19, 2014 4:20:42 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1531. INFO:
  1532. Jan 19, 2014 4:20:45 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1533. INFO:
  1534. Jan 19, 2014 4:20:46 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1535. INFO: map 94% reduce 0%
  1536. Jan 19, 2014 4:20:48 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1537. INFO:
  1538. Jan 19, 2014 4:20:51 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1539. INFO:
  1540. Jan 19, 2014 4:20:52 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1541. INFO: map 95% reduce 0%
  1542. Jan 19, 2014 4:20:54 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1543. INFO:
  1544. Jan 19, 2014 4:20:57 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1545. INFO:
  1546. Jan 19, 2014 4:21:00 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1547. INFO:
  1548. Jan 19, 2014 4:21:03 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1549. INFO:
  1550. Jan 19, 2014 4:21:04 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1551. INFO: map 96% reduce 0%
  1552. Jan 19, 2014 4:21:06 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1553. INFO:
  1554. Jan 19, 2014 4:21:09 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1555. INFO:
  1556. Jan 19, 2014 4:21:12 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1557. INFO:
  1558. Jan 19, 2014 4:21:13 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1559. INFO: map 97% reduce 0%
  1560. Jan 19, 2014 4:21:15 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1561. INFO:
  1562. Jan 19, 2014 4:21:18 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1563. INFO:
  1564. Jan 19, 2014 4:21:19 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1565. INFO: map 98% reduce 0%
  1566. Jan 19, 2014 4:21:21 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1567. INFO:
  1568. Jan 19, 2014 4:21:24 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1569. INFO:
  1570. Jan 19, 2014 4:21:25 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1571. INFO: map 99% reduce 0%
  1572. Jan 19, 2014 4:21:27 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1573. INFO:
  1574. Jan 19, 2014 4:21:30 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer flush
  1575. INFO: Starting flush of map output
  1576. Jan 19, 2014 4:21:30 PM org.apache.hadoop.io.compress.CodecPool getCompressor
  1577. INFO: Got brand-new compressor
  1578. Jan 19, 2014 4:21:30 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer sortAndSpill
  1579. INFO: Finished spill 0
  1580. Jan 19, 2014 4:21:30 PM org.apache.hadoop.mapred.Task done
  1581. INFO: Task:attempt_local1791603598_0001_m_000000_0 is done. And is in the process of commiting
  1582. Jan 19, 2014 4:21:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1583. INFO:
  1584. Jan 19, 2014 4:21:30 PM org.apache.hadoop.mapred.Task sendDone
  1585. INFO: Task 'attempt_local1791603598_0001_m_000000_0' done.
  1586. Jan 19, 2014 4:21:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable run
  1587. INFO: Finishing task: attempt_local1791603598_0001_m_000000_0
  1588. Jan 19, 2014 4:21:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
  1589. INFO: Map task executor complete.
  1590. Jan 19, 2014 4:21:30 PM org.apache.hadoop.mapred.Task initialize
  1591. INFO: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@37b74d4e
  1592. Jan 19, 2014 4:21:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1593. INFO:
  1594. Jan 19, 2014 4:21:30 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
  1595. INFO: Merging 1 sorted segments
  1596. Jan 19, 2014 4:21:30 PM org.apache.hadoop.io.compress.CodecPool getDecompressor
  1597. INFO: Got brand-new decompressor
  1598. Jan 19, 2014 4:21:30 PM org.apache.hadoop.mapred.Merger$MergeQueue merge
  1599. INFO: Down to the last merge-pass, with 1 segments left of total size: 1059140 bytes
  1600. Jan 19, 2014 4:21:30 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate
  1601. INFO:
  1602. Jan 19, 2014 4:21:30 PM org.slf4j.impl.JCLLoggerAdapter info
  1603. INFO: Estimated Points: 223
  1604. Jan 19, 2014 4:21:31 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1605. INFO: map 100% reduce 0%
  1606. Jan 19, 2014 4:21:32 PM org.slf4j.impl.JCLLoggerAdapter info
  1607. INFO: Number of Centroids: 0
  1608. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.LocalJobRunner$Job run
  1609. WARNING: job_local1791603598_0001
  1610. java.lang.IllegalArgumentException: Must have nonzero number of training and test vectors. Asked for %.1f %% of %d vectors for test [10.000000149011612, 0]
  1611. at com.google.common.base.Preconditions.checkArgument(Preconditions.java:120)
  1612. at org.apache.mahout.clustering.streaming.cluster.BallKMeans.splitTrainTest(BallKMeans.java:176)
  1613. at org.apache.mahout.clustering.streaming.cluster.BallKMeans.cluster(BallKMeans.java:192)
  1614. at org.apache.mahout.clustering.streaming.mapreduce.StreamingKMeansReducer.getBestCentroids(StreamingKMeansReducer.java:107)
  1615. at org.apache.mahout.clustering.streaming.mapreduce.StreamingKMeansReducer.reduce(StreamingKMeansReducer.java:73)
  1616. at org.apache.mahout.clustering.streaming.mapreduce.StreamingKMeansReducer.reduce(StreamingKMeansReducer.java:37)
  1617. at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:177)
  1618. at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649)
  1619. at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418)
  1620. at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:398)
  1621.  
  1622. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
  1623. INFO: Job complete: job_local1791603598_0001
  1624. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1625. INFO: Counters: 19
  1626. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1627. INFO: File Input Format Counters
  1628. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1629. INFO: Bytes Read=17156391
  1630. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1631. INFO: FileSystemCounters
  1632. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1633. INFO: FILE_BYTES_READ=41214661
  1634. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1635. INFO: FILE_BYTES_WRITTEN=25361029
  1636. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1637. INFO: Map-Reduce Framework
  1638. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1639. INFO: Reduce input groups=0
  1640. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1641. INFO: Map output materialized bytes=1059144
  1642. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1643. INFO: Combine output records=0
  1644. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1645. INFO: Map input records=21578
  1646. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1647. INFO: Reduce shuffle bytes=0
  1648. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1649. INFO: Physical memory (bytes) snapshot=0
  1650. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1651. INFO: Reduce output records=0
  1652. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1653. INFO: Spilled Records=223
  1654. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1655. INFO: Map output bytes=1927748
  1656. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1657. INFO: Total committed heap usage (bytes)=202506240
  1658. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1659. INFO: CPU time spent (ms)=0
  1660. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1661. INFO: Virtual memory (bytes) snapshot=0
  1662. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1663. INFO: SPLIT_RAW_BYTES=161
  1664. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1665. INFO: Map output records=223
  1666. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1667. INFO: Combine input records=0
  1668. Jan 19, 2014 4:21:32 PM org.apache.hadoop.mapred.Counters log
  1669. INFO: Reduce input records=0
  1670. Jan 19, 2014 4:21:32 PM org.slf4j.impl.JCLLoggerAdapter info
  1671. INFO: Program took 407756 ms (Minutes: 6.795933333333333)
  1672. MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
  1673. hadoop binary is not in PATH,HADOOP_HOME/bin,HADOOP_PREFIX/bin, running locally
  1674. SLF4J: Class path contains multiple SLF4J bindings.
  1675. SLF4J: Found binding in [jar:file:/home/frank/Desktop/mahout-distribution-0.9/examples/target/mahout-examples-0.9-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]
  1676. SLF4J: Found binding in [jar:file:/home/frank/Desktop/mahout-distribution-0.9/examples/target/dependency/slf4j-jcl-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
  1677. SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
  1678. SLF4J: Actual binding is of type [org.slf4j.impl.JCLLoggerFactory]
  1679. Jan 19, 2014 4:21:32 PM org.slf4j.impl.JCLLoggerAdapter warn
  1680. WARNING: No qualcluster.props found on classpath, will use command-line arguments only
  1681. Num clusters: 0; maxDistance: 0.000000
  1682. [Dunn Index] First: Infinity
  1683. [Davies-Bouldin Index] First: NaN
  1684. Jan 19, 2014 4:21:32 PM org.slf4j.impl.JCLLoggerAdapter info
  1685. INFO: Program took 289 ms (Minutes: 0.004816666666666667)
  1686. cluster,distance.mean,distance.sd,distance.q0,distance.q1,distance.q2,distance.q3,distance.q4,count,is.train
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!