API tools faq
paste
lamiastella

gensim_32workers

lamiastella
Oct 4th, 2016
281
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 15.28 KB | None | 0 0
raw download clone embed print report
  1. jalal@klein:~/computer_vision/image_retrieval$ python gensim_test.py
  2. 2016-10-04 17:54:14,842 : INFO : collecting all words and their counts
  3. 2016-10-04 17:54:14,844 : INFO : PROGRESS: at sentence #0, processed 0 words, keeping 0 word types
  4. 2016-10-04 17:54:21,369 : INFO : collected 253854 word types from a corpus of 17005207 raw words and 1701 sentences
  5. 2016-10-04 17:54:21,750 : INFO : min_count=5 retains 71290 unique words (drops 182564)
  6. 2016-10-04 17:54:21,750 : INFO : min_count leaves 16718844 word corpus (98% of original 17005207)
  7. 2016-10-04 17:54:21,973 : INFO : deleting the raw counts dictionary of 253854 items
  8. 2016-10-04 17:54:22,002 : INFO : sample=0.001 downsamples 38 most-common words
  9. 2016-10-04 17:54:22,002 : INFO : downsampling leaves estimated 12506280 word corpus (74.8% of prior 16718844)
  10. 2016-10-04 17:54:22,002 : INFO : estimated required memory for 71290 words and 100 dimensions: 92677000 bytes
  11. 2016-10-04 17:54:22,189 : INFO : resetting layer weights
  12. 2016-10-04 17:54:23,061 : INFO : training model with 32 workers on 71290 vocabulary and 100 features, using sg=0 hs=0 sample=0.001 negative=5
  13. 2016-10-04 17:54:23,062 : INFO : expecting 1701 sentences, matching count from corpus used for vocabulary survey
  14. 2016-10-04 17:54:24,109 : INFO : PROGRESS: at 0.76% examples, 475808 words/s, in_qsize 55, out_qsize 6
  15. 2016-10-04 17:54:25,129 : INFO : PROGRESS: at 1.83% examples, 553327 words/s, in_qsize 64, out_qsize 2
  16. 2016-10-04 17:54:26,167 : INFO : PROGRESS: at 2.93% examples, 588192 words/s, in_qsize 61, out_qsize 3
  17. 2016-10-04 17:54:27,188 : INFO : PROGRESS: at 4.02% examples, 607903 words/s, in_qsize 64, out_qsize 0
  18. 2016-10-04 17:54:28,191 : INFO : PROGRESS: at 5.16% examples, 629632 words/s, in_qsize 63, out_qsize 1
  19. 2016-10-04 17:54:29,220 : INFO : PROGRESS: at 6.18% examples, 632255 words/s, in_qsize 59, out_qsize 6
  20. 2016-10-04 17:54:30,229 : INFO : PROGRESS: at 7.29% examples, 640821 words/s, in_qsize 64, out_qsize 2
  21. 2016-10-04 17:54:31,218 : INFO : PROGRESS: at 8.36% examples, 643105 words/s, in_qsize 63, out_qsize 0
  22. 2016-10-04 17:54:32,240 : INFO : PROGRESS: at 9.37% examples, 640836 words/s, in_qsize 63, out_qsize 0
  23. 2016-10-04 17:54:33,276 : INFO : PROGRESS: at 10.39% examples, 639936 words/s, in_qsize 61, out_qsize 5
  24. 2016-10-04 17:54:34,301 : INFO : PROGRESS: at 11.42% examples, 638896 words/s, in_qsize 62, out_qsize 3
  25. 2016-10-04 17:54:35,326 : INFO : PROGRESS: at 12.52% examples, 641399 words/s, in_qsize 59, out_qsize 3
  26. 2016-10-04 17:54:36,338 : INFO : PROGRESS: at 13.52% examples, 640009 words/s, in_qsize 61, out_qsize 4
  27. 2016-10-04 17:54:37,336 : INFO : PROGRESS: at 14.52% examples, 638590 words/s, in_qsize 62, out_qsize 3
  28. 2016-10-04 17:54:38,391 : INFO : PROGRESS: at 15.61% examples, 640367 words/s, in_qsize 62, out_qsize 2
  29. 2016-10-04 17:54:39,351 : INFO : PROGRESS: at 16.61% examples, 638826 words/s, in_qsize 64, out_qsize 0
  30. 2016-10-04 17:54:40,372 : INFO : PROGRESS: at 17.68% examples, 639806 words/s, in_qsize 63, out_qsize 0
  31. 2016-10-04 17:54:41,413 : INFO : PROGRESS: at 18.68% examples, 638431 words/s, in_qsize 61, out_qsize 5
  32. 2016-10-04 17:54:42,388 : INFO : PROGRESS: at 19.76% examples, 640081 words/s, in_qsize 61, out_qsize 5
  33. 2016-10-04 17:54:43,393 : INFO : PROGRESS: at 20.81% examples, 640513 words/s, in_qsize 63, out_qsize 0
  34. 2016-10-04 17:54:44,405 : INFO : PROGRESS: at 21.89% examples, 641336 words/s, in_qsize 63, out_qsize 0
  35. 2016-10-04 17:54:45,424 : INFO : PROGRESS: at 22.94% examples, 641277 words/s, in_qsize 51, out_qsize 0
  36. 2016-10-04 17:54:46,446 : INFO : PROGRESS: at 23.88% examples, 638400 words/s, in_qsize 63, out_qsize 0
  37. 2016-10-04 17:54:47,457 : INFO : PROGRESS: at 24.90% examples, 638269 words/s, in_qsize 63, out_qsize 0
  38. 2016-10-04 17:54:48,469 : INFO : PROGRESS: at 25.83% examples, 636241 words/s, in_qsize 63, out_qsize 3
  39. 2016-10-04 17:54:49,505 : INFO : PROGRESS: at 26.93% examples, 637927 words/s, in_qsize 54, out_qsize 3
  40. 2016-10-04 17:54:50,493 : INFO : PROGRESS: at 27.91% examples, 636921 words/s, in_qsize 60, out_qsize 0
  41. 2016-10-04 17:54:51,517 : INFO : PROGRESS: at 28.88% examples, 635413 words/s, in_qsize 63, out_qsize 4
  42. 2016-10-04 17:54:52,558 : INFO : PROGRESS: at 29.96% examples, 636497 words/s, in_qsize 61, out_qsize 2
  43. 2016-10-04 17:54:53,545 : INFO : PROGRESS: at 30.98% examples, 636498 words/s, in_qsize 63, out_qsize 2
  44. 2016-10-04 17:54:54,545 : INFO : PROGRESS: at 31.98% examples, 636144 words/s, in_qsize 63, out_qsize 1
  45. 2016-10-04 17:54:55,562 : INFO : PROGRESS: at 33.00% examples, 635901 words/s, in_qsize 64, out_qsize 0
  46. 2016-10-04 17:54:56,596 : INFO : PROGRESS: at 34.02% examples, 635229 words/s, in_qsize 63, out_qsize 2
  47. 2016-10-04 17:54:57,631 : INFO : PROGRESS: at 35.11% examples, 636112 words/s, in_qsize 63, out_qsize 2
  48. 2016-10-04 17:54:58,629 : INFO : PROGRESS: at 36.13% examples, 635910 words/s, in_qsize 60, out_qsize 3
  49. 2016-10-04 17:54:59,658 : INFO : PROGRESS: at 37.21% examples, 636335 words/s, in_qsize 64, out_qsize 2
  50. 2016-10-04 17:55:00,682 : INFO : PROGRESS: at 38.32% examples, 637321 words/s, in_qsize 64, out_qsize 0
  51. 2016-10-04 17:55:01,701 : INFO : PROGRESS: at 39.35% examples, 637416 words/s, in_qsize 63, out_qsize 3
  52. 2016-10-04 17:55:02,720 : INFO : PROGRESS: at 40.42% examples, 638056 words/s, in_qsize 60, out_qsize 4
  53. 2016-10-04 17:55:03,710 : INFO : PROGRESS: at 41.45% examples, 637601 words/s, in_qsize 63, out_qsize 0
  54. 2016-10-04 17:55:04,738 : INFO : PROGRESS: at 42.50% examples, 637842 words/s, in_qsize 61, out_qsize 3
  55. 2016-10-04 17:55:05,746 : INFO : PROGRESS: at 43.48% examples, 636879 words/s, in_qsize 63, out_qsize 0
  56. 2016-10-04 17:55:06,771 : INFO : PROGRESS: at 44.54% examples, 637332 words/s, in_qsize 64, out_qsize 3
  57. 2016-10-04 17:55:07,779 : INFO : PROGRESS: at 45.51% examples, 636586 words/s, in_qsize 61, out_qsize 4
  58. 2016-10-04 17:55:08,815 : INFO : PROGRESS: at 46.56% examples, 636825 words/s, in_qsize 63, out_qsize 2
  59. 2016-10-04 17:55:09,824 : INFO : PROGRESS: at 47.57% examples, 636657 words/s, in_qsize 62, out_qsize 1
  60. 2016-10-04 17:55:10,846 : INFO : PROGRESS: at 48.63% examples, 637132 words/s, in_qsize 56, out_qsize 3
  61. 2016-10-04 17:55:11,820 : INFO : PROGRESS: at 49.63% examples, 636934 words/s, in_qsize 63, out_qsize 0
  62. 2016-10-04 17:55:12,839 : INFO : PROGRESS: at 50.64% examples, 636672 words/s, in_qsize 63, out_qsize 1
  63. 2016-10-04 17:55:13,894 : INFO : PROGRESS: at 51.62% examples, 635716 words/s, in_qsize 61, out_qsize 5
  64. 2016-10-04 17:55:14,878 : INFO : PROGRESS: at 52.64% examples, 635830 words/s, in_qsize 60, out_qsize 0
  65. 2016-10-04 17:55:15,887 : INFO : PROGRESS: at 53.62% examples, 635294 words/s, in_qsize 63, out_qsize 0
  66. 2016-10-04 17:55:16,948 : INFO : PROGRESS: at 54.73% examples, 635998 words/s, in_qsize 60, out_qsize 1
  67. 2016-10-04 17:55:17,952 : INFO : PROGRESS: at 55.68% examples, 634818 words/s, in_qsize 55, out_qsize 2
  68. 2016-10-04 17:55:18,954 : INFO : PROGRESS: at 56.71% examples, 634816 words/s, in_qsize 63, out_qsize 0
  69. 2016-10-04 17:55:19,973 : INFO : PROGRESS: at 57.70% examples, 634341 words/s, in_qsize 62, out_qsize 3
  70. 2016-10-04 17:55:20,967 : INFO : PROGRESS: at 58.69% examples, 634065 words/s, in_qsize 64, out_qsize 0
  71. 2016-10-04 17:55:21,987 : INFO : PROGRESS: at 59.73% examples, 634104 words/s, in_qsize 63, out_qsize 1
  72. 2016-10-04 17:55:23,000 : INFO : PROGRESS: at 60.71% examples, 633456 words/s, in_qsize 64, out_qsize 0
  73. 2016-10-04 17:55:24,011 : INFO : PROGRESS: at 61.70% examples, 633057 words/s, in_qsize 64, out_qsize 0
  74. 2016-10-04 17:55:25,045 : INFO : PROGRESS: at 62.66% examples, 632236 words/s, in_qsize 58, out_qsize 9
  75. 2016-10-04 17:55:26,082 : INFO : PROGRESS: at 63.76% examples, 632662 words/s, in_qsize 55, out_qsize 3
  76. 2016-10-04 17:55:27,097 : INFO : PROGRESS: at 64.75% examples, 632308 words/s, in_qsize 63, out_qsize 1
  77. 2016-10-04 17:55:28,135 : INFO : PROGRESS: at 65.66% examples, 631253 words/s, in_qsize 57, out_qsize 7
  78. 2016-10-04 17:55:29,109 : INFO : PROGRESS: at 66.67% examples, 631374 words/s, in_qsize 62, out_qsize 1
  79. 2016-10-04 17:55:30,123 : INFO : PROGRESS: at 67.71% examples, 631685 words/s, in_qsize 63, out_qsize 3
  80. 2016-10-04 17:55:31,126 : INFO : PROGRESS: at 68.71% examples, 631599 words/s, in_qsize 62, out_qsize 1
  81. 2016-10-04 17:55:32,139 : INFO : PROGRESS: at 69.75% examples, 631743 words/s, in_qsize 59, out_qsize 4
  82. 2016-10-04 17:55:33,141 : INFO : PROGRESS: at 70.71% examples, 631347 words/s, in_qsize 61, out_qsize 4
  83. 2016-10-04 17:55:34,159 : INFO : PROGRESS: at 71.76% examples, 631704 words/s, in_qsize 63, out_qsize 1
  84. 2016-10-04 17:55:35,155 : INFO : PROGRESS: at 72.73% examples, 631352 words/s, in_qsize 63, out_qsize 0
  85. 2016-10-04 17:55:36,170 : INFO : PROGRESS: at 73.74% examples, 631335 words/s, in_qsize 64, out_qsize 0
  86. 2016-10-04 17:55:37,156 : INFO : PROGRESS: at 74.80% examples, 631769 words/s, in_qsize 63, out_qsize 0
  87. 2016-10-04 17:55:38,169 : INFO : PROGRESS: at 75.78% examples, 631132 words/s, in_qsize 64, out_qsize 0
  88. 2016-10-04 17:55:39,171 : INFO : PROGRESS: at 76.83% examples, 631409 words/s, in_qsize 63, out_qsize 0
  89. 2016-10-04 17:55:40,187 : INFO : PROGRESS: at 77.88% examples, 631751 words/s, in_qsize 60, out_qsize 1
  90. 2016-10-04 17:55:41,199 : INFO : PROGRESS: at 78.86% examples, 631274 words/s, in_qsize 61, out_qsize 0
  91. 2016-10-04 17:55:42,221 : INFO : PROGRESS: at 79.88% examples, 631295 words/s, in_qsize 63, out_qsize 2
  92. 2016-10-04 17:55:43,224 : INFO : PROGRESS: at 80.88% examples, 631148 words/s, in_qsize 63, out_qsize 2
  93. 2016-10-04 17:55:44,215 : INFO : PROGRESS: at 81.90% examples, 631107 words/s, in_qsize 57, out_qsize 0
  94. 2016-10-04 17:55:45,238 : INFO : PROGRESS: at 82.97% examples, 631366 words/s, in_qsize 61, out_qsize 0
  95. 2016-10-04 17:55:46,250 : INFO : PROGRESS: at 83.95% examples, 631052 words/s, in_qsize 59, out_qsize 1
  96. 2016-10-04 17:55:47,279 : INFO : PROGRESS: at 84.93% examples, 630670 words/s, in_qsize 63, out_qsize 1
  97. 2016-10-04 17:55:48,282 : INFO : PROGRESS: at 85.95% examples, 630802 words/s, in_qsize 61, out_qsize 4
  98. 2016-10-04 17:55:49,295 : INFO : PROGRESS: at 86.98% examples, 630976 words/s, in_qsize 63, out_qsize 1
  99. 2016-10-04 17:55:50,317 : INFO : PROGRESS: at 88.04% examples, 631272 words/s, in_qsize 51, out_qsize 1
  100. 2016-10-04 17:55:51,341 : INFO : PROGRESS: at 88.99% examples, 630854 words/s, in_qsize 57, out_qsize 7
  101. 2016-10-04 17:55:52,333 : INFO : PROGRESS: at 90.03% examples, 630989 words/s, in_qsize 60, out_qsize 3
  102. 2016-10-04 17:55:53,338 : INFO : PROGRESS: at 91.11% examples, 631449 words/s, in_qsize 46, out_qsize 1
  103. 2016-10-04 17:55:54,355 : INFO : PROGRESS: at 92.13% examples, 631537 words/s, in_qsize 62, out_qsize 1
  104. 2016-10-04 17:55:55,397 : INFO : PROGRESS: at 93.06% examples, 630724 words/s, in_qsize 61, out_qsize 7
  105. 2016-10-04 17:55:56,394 : INFO : PROGRESS: at 94.17% examples, 631314 words/s, in_qsize 60, out_qsize 2
  106. 2016-10-04 17:55:57,410 : INFO : PROGRESS: at 95.14% examples, 630927 words/s, in_qsize 61, out_qsize 2
  107. 2016-10-04 17:55:58,419 : INFO : PROGRESS: at 96.18% examples, 630890 words/s, in_qsize 63, out_qsize 0
  108. 2016-10-04 17:55:59,450 : INFO : PROGRESS: at 97.15% examples, 630610 words/s, in_qsize 54, out_qsize 0
  109. 2016-10-04 17:56:00,435 : INFO : PROGRESS: at 98.15% examples, 630534 words/s, in_qsize 54, out_qsize 2
  110. 2016-10-04 17:56:01,457 : INFO : PROGRESS: at 99.12% examples, 630035 words/s, in_qsize 64, out_qsize 2
  111. 2016-10-04 17:56:01,862 : INFO : worker thread finished; awaiting finish of 31 more threads
  112. 2016-10-04 17:56:01,863 : INFO : worker thread finished; awaiting finish of 30 more threads
  113. 2016-10-04 17:56:01,890 : INFO : worker thread finished; awaiting finish of 29 more threads
  114. 2016-10-04 17:56:01,891 : INFO : worker thread finished; awaiting finish of 28 more threads
  115. 2016-10-04 17:56:01,892 : INFO : worker thread finished; awaiting finish of 27 more threads
  116. 2016-10-04 17:56:01,894 : INFO : worker thread finished; awaiting finish of 26 more threads
  117. 2016-10-04 17:56:01,894 : INFO : worker thread finished; awaiting finish of 25 more threads
  118. 2016-10-04 17:56:01,928 : INFO : worker thread finished; awaiting finish of 24 more threads
  119. 2016-10-04 17:56:01,932 : INFO : worker thread finished; awaiting finish of 23 more threads
  120. 2016-10-04 17:56:01,932 : INFO : worker thread finished; awaiting finish of 22 more threads
  121. 2016-10-04 17:56:01,932 : INFO : worker thread finished; awaiting finish of 21 more threads
  122. 2016-10-04 17:56:01,979 : INFO : worker thread finished; awaiting finish of 20 more threads
  123. 2016-10-04 17:56:01,981 : INFO : worker thread finished; awaiting finish of 19 more threads
  124. 2016-10-04 17:56:01,989 : INFO : worker thread finished; awaiting finish of 18 more threads
  125. 2016-10-04 17:56:02,009 : INFO : worker thread finished; awaiting finish of 17 more threads
  126. 2016-10-04 17:56:02,012 : INFO : worker thread finished; awaiting finish of 16 more threads
  127. 2016-10-04 17:56:02,013 : INFO : worker thread finished; awaiting finish of 15 more threads
  128. 2016-10-04 17:56:02,013 : INFO : worker thread finished; awaiting finish of 14 more threads
  129. 2016-10-04 17:56:02,013 : INFO : worker thread finished; awaiting finish of 13 more threads
  130. 2016-10-04 17:56:02,014 : INFO : worker thread finished; awaiting finish of 12 more threads
  131. 2016-10-04 17:56:02,014 : INFO : worker thread finished; awaiting finish of 11 more threads
  132. 2016-10-04 17:56:02,015 : INFO : worker thread finished; awaiting finish of 10 more threads
  133. 2016-10-04 17:56:02,023 : INFO : worker thread finished; awaiting finish of 9 more threads
  134. 2016-10-04 17:56:02,040 : INFO : worker thread finished; awaiting finish of 8 more threads
  135. 2016-10-04 17:56:02,043 : INFO : worker thread finished; awaiting finish of 7 more threads
  136. 2016-10-04 17:56:02,046 : INFO : worker thread finished; awaiting finish of 6 more threads
  137. 2016-10-04 17:56:02,049 : INFO : worker thread finished; awaiting finish of 5 more threads
  138. 2016-10-04 17:56:02,050 : INFO : worker thread finished; awaiting finish of 4 more threads
  139. 2016-10-04 17:56:02,051 : INFO : worker thread finished; awaiting finish of 3 more threads
  140. 2016-10-04 17:56:02,057 : INFO : worker thread finished; awaiting finish of 2 more threads
  141. 2016-10-04 17:56:02,058 : INFO : worker thread finished; awaiting finish of 1 more threads
  142. 2016-10-04 17:56:02,068 : INFO : worker thread finished; awaiting finish of 0 more threads
  143. 2016-10-04 17:56:02,068 : INFO : training on 85026035 raw words (62532846 effective words) took 99.0s, 631672 effective words/s
  144. 2016-10-04 17:56:02,068 : INFO : precomputing L2-norms of word weight vectors
  145. 2016-10-04 17:56:03,006 : INFO : saving Word2Vec object under text8_data, separately None
  146. 2016-10-04 17:56:03,006 : INFO : not storing attribute syn0norm
  147. 2016-10-04 17:56:03,007 : INFO : not storing attribute cum_table
  148. 2016-10-04 17:56:04,303 : INFO : Removed 17 and 25 OOV words from document 1 and 2 (respectively).
  149. 2016-10-04 17:56:04,304 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
  150. 2016-10-04 17:56:04,304 : INFO : built Dictionary(59 unique tokens: [u'heart', u'love', u'departed', u'deck', u'within']...) from 2 documents (total 75 corpus positions)
  151. Distance between the two poems is: 1.07465532301
  152. 2016-10-04 17:56:04,331 : INFO : Removed 0 and 1 OOV words from document 1 and 2 (respectively).
  153. 2016-10-04 17:56:04,332 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
  154. 2016-10-04 17:56:04,332 : INFO : built Dictionary(7 unique tokens: [u'chicago', u'media', u'illinois', u'press', u'president']...) from 2 documents (total 7 corpus positions)
  155. Distance between the two sentences is: 1.10273208025
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!