API tools faq
paste
lamiastella

gensim_1worker

lamiastella
Oct 4th, 2016
315
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 13.27 KB | None | 0 0
raw download clone embed print report
  1. jalal@klein:~/computer_vision/image_retrieval$ python gensim_test.py
  2. 2016-10-04 18:00:04,724 : INFO : collecting all words and their counts
  3. 2016-10-04 18:00:04,727 : INFO : PROGRESS: at sentence #0, processed 0 words, keeping 0 word types
  4. 2016-10-04 18:00:10,596 : INFO : collected 253854 word types from a corpus of 17005207 raw words and 1701 sentences
  5. 2016-10-04 18:00:10,994 : INFO : min_count=5 retains 71290 unique words (drops 182564)
  6. 2016-10-04 18:00:10,994 : INFO : min_count leaves 16718844 word corpus (98% of original 17005207)
  7. 2016-10-04 18:00:11,224 : INFO : deleting the raw counts dictionary of 253854 items
  8. 2016-10-04 18:00:11,254 : INFO : sample=0.001 downsamples 38 most-common words
  9. 2016-10-04 18:00:11,254 : INFO : downsampling leaves estimated 12506280 word corpus (74.8% of prior 16718844)
  10. 2016-10-04 18:00:11,254 : INFO : estimated required memory for 71290 words and 100 dimensions: 92677000 bytes
  11. 2016-10-04 18:00:11,515 : INFO : resetting layer weights
  12. 2016-10-04 18:00:13,210 : INFO : training model with 3 workers on 71290 vocabulary and 100 features, using sg=0 hs=0 sample=0.001 negative=5
  13. 2016-10-04 18:00:13,210 : INFO : expecting 1701 sentences, matching count from corpus used for vocabulary survey
  14. 2016-10-04 18:00:14,213 : INFO : PROGRESS: at 0.82% examples, 515664 words/s, in_qsize 6, out_qsize 0
  15. 2016-10-04 18:00:15,217 : INFO : PROGRESS: at 1.73% examples, 535401 words/s, in_qsize 5, out_qsize 0
  16. 2016-10-04 18:00:16,231 : INFO : PROGRESS: at 2.68% examples, 551869 words/s, in_qsize 5, out_qsize 0
  17. 2016-10-04 18:00:17,261 : INFO : PROGRESS: at 3.60% examples, 553307 words/s, in_qsize 6, out_qsize 1
  18. 2016-10-04 18:00:18,265 : INFO : PROGRESS: at 4.55% examples, 561209 words/s, in_qsize 6, out_qsize 0
  19. 2016-10-04 18:00:19,271 : INFO : PROGRESS: at 5.46% examples, 562400 words/s, in_qsize 5, out_qsize 0
  20. 2016-10-04 18:00:20,278 : INFO : PROGRESS: at 6.29% examples, 557297 words/s, in_qsize 5, out_qsize 0
  21. 2016-10-04 18:00:21,286 : INFO : PROGRESS: at 7.18% examples, 558280 words/s, in_qsize 6, out_qsize 1
  22. 2016-10-04 18:00:22,296 : INFO : PROGRESS: at 8.08% examples, 557361 words/s, in_qsize 5, out_qsize 0
  23. 2016-10-04 18:00:23,304 : INFO : PROGRESS: at 8.99% examples, 558755 words/s, in_qsize 5, out_qsize 0
  24. 2016-10-04 18:00:24,327 : INFO : PROGRESS: at 9.89% examples, 557826 words/s, in_qsize 5, out_qsize 0
  25. 2016-10-04 18:00:25,353 : INFO : PROGRESS: at 10.82% examples, 558780 words/s, in_qsize 5, out_qsize 0
  26. 2016-10-04 18:00:26,363 : INFO : PROGRESS: at 11.71% examples, 558445 words/s, in_qsize 6, out_qsize 0
  27. 2016-10-04 18:00:27,363 : INFO : PROGRESS: at 12.60% examples, 558596 words/s, in_qsize 5, out_qsize 0
  28. 2016-10-04 18:00:28,378 : INFO : PROGRESS: at 13.51% examples, 558560 words/s, in_qsize 5, out_qsize 0
  29. 2016-10-04 18:00:29,403 : INFO : PROGRESS: at 14.40% examples, 557872 words/s, in_qsize 6, out_qsize 0
  30. 2016-10-04 18:00:30,416 : INFO : PROGRESS: at 15.29% examples, 556769 words/s, in_qsize 6, out_qsize 0
  31. 2016-10-04 18:00:31,418 : INFO : PROGRESS: at 16.18% examples, 556274 words/s, in_qsize 6, out_qsize 1
  32. 2016-10-04 18:00:32,443 : INFO : PROGRESS: at 17.07% examples, 555614 words/s, in_qsize 6, out_qsize 2
  33. 2016-10-04 18:00:33,459 : INFO : PROGRESS: at 18.01% examples, 556914 words/s, in_qsize 5, out_qsize 0
  34. 2016-10-04 18:00:34,465 : INFO : PROGRESS: at 18.98% examples, 558710 words/s, in_qsize 6, out_qsize 0
  35. 2016-10-04 18:00:35,482 : INFO : PROGRESS: at 19.89% examples, 558769 words/s, in_qsize 6, out_qsize 0
  36. 2016-10-04 18:00:36,490 : INFO : PROGRESS: at 20.86% examples, 560440 words/s, in_qsize 6, out_qsize 0
  37. 2016-10-04 18:00:37,509 : INFO : PROGRESS: at 21.81% examples, 561051 words/s, in_qsize 5, out_qsize 0
  38. 2016-10-04 18:00:38,528 : INFO : PROGRESS: at 22.76% examples, 561914 words/s, in_qsize 6, out_qsize 1
  39. 2016-10-04 18:00:39,547 : INFO : PROGRESS: at 23.79% examples, 564405 words/s, in_qsize 6, out_qsize 0
  40. 2016-10-04 18:00:40,559 : INFO : PROGRESS: at 24.71% examples, 565078 words/s, in_qsize 5, out_qsize 0
  41. 2016-10-04 18:00:41,560 : INFO : PROGRESS: at 25.71% examples, 567332 words/s, in_qsize 6, out_qsize 0
  42. 2016-10-04 18:00:42,561 : INFO : PROGRESS: at 26.71% examples, 569440 words/s, in_qsize 6, out_qsize 0
  43. 2016-10-04 18:00:43,572 : INFO : PROGRESS: at 27.67% examples, 570204 words/s, in_qsize 6, out_qsize 0
  44. 2016-10-04 18:00:44,589 : INFO : PROGRESS: at 28.61% examples, 570591 words/s, in_qsize 5, out_qsize 0
  45. 2016-10-04 18:00:45,598 : INFO : PROGRESS: at 29.57% examples, 571471 words/s, in_qsize 5, out_qsize 0
  46. 2016-10-04 18:00:46,613 : INFO : PROGRESS: at 30.55% examples, 572411 words/s, in_qsize 6, out_qsize 0
  47. 2016-10-04 18:00:47,618 : INFO : PROGRESS: at 31.50% examples, 573093 words/s, in_qsize 5, out_qsize 0
  48. 2016-10-04 18:00:48,637 : INFO : PROGRESS: at 32.46% examples, 573816 words/s, in_qsize 5, out_qsize 0
  49. 2016-10-04 18:00:49,637 : INFO : PROGRESS: at 33.42% examples, 574296 words/s, in_qsize 5, out_qsize 0
  50. 2016-10-04 18:00:50,656 : INFO : PROGRESS: at 34.36% examples, 574453 words/s, in_qsize 6, out_qsize 0
  51. 2016-10-04 18:00:51,663 : INFO : PROGRESS: at 35.34% examples, 575200 words/s, in_qsize 5, out_qsize 0
  52. 2016-10-04 18:00:52,663 : INFO : PROGRESS: at 36.32% examples, 575934 words/s, in_qsize 6, out_qsize 0
  53. 2016-10-04 18:00:53,664 : INFO : PROGRESS: at 37.28% examples, 576579 words/s, in_qsize 6, out_qsize 0
  54. 2016-10-04 18:00:54,666 : INFO : PROGRESS: at 38.26% examples, 577348 words/s, in_qsize 5, out_qsize 0
  55. 2016-10-04 18:00:55,669 : INFO : PROGRESS: at 39.22% examples, 577856 words/s, in_qsize 5, out_qsize 0
  56. 2016-10-04 18:00:56,688 : INFO : PROGRESS: at 40.15% examples, 577585 words/s, in_qsize 6, out_qsize 0
  57. 2016-10-04 18:00:57,694 : INFO : PROGRESS: at 41.12% examples, 577990 words/s, in_qsize 6, out_qsize 0
  58. 2016-10-04 18:00:58,696 : INFO : PROGRESS: at 42.07% examples, 578187 words/s, in_qsize 6, out_qsize 0
  59. 2016-10-04 18:00:59,707 : INFO : PROGRESS: at 43.06% examples, 578846 words/s, in_qsize 5, out_qsize 0
  60. 2016-10-04 18:01:00,707 : INFO : PROGRESS: at 44.03% examples, 579508 words/s, in_qsize 5, out_qsize 0
  61. 2016-10-04 18:01:01,734 : INFO : PROGRESS: at 45.00% examples, 579873 words/s, in_qsize 5, out_qsize 1
  62. 2016-10-04 18:01:02,731 : INFO : PROGRESS: at 45.98% examples, 580728 words/s, in_qsize 6, out_qsize 0
  63. 2016-10-04 18:01:03,740 : INFO : PROGRESS: at 46.98% examples, 581654 words/s, in_qsize 5, out_qsize 0
  64. 2016-10-04 18:01:04,742 : INFO : PROGRESS: at 47.96% examples, 582220 words/s, in_qsize 6, out_qsize 0
  65. 2016-10-04 18:01:05,752 : INFO : PROGRESS: at 48.94% examples, 582682 words/s, in_qsize 5, out_qsize 0
  66. 2016-10-04 18:01:06,756 : INFO : PROGRESS: at 49.92% examples, 583355 words/s, in_qsize 6, out_qsize 0
  67. 2016-10-04 18:01:07,784 : INFO : PROGRESS: at 50.84% examples, 582939 words/s, in_qsize 5, out_qsize 0
  68. 2016-10-04 18:01:08,802 : INFO : PROGRESS: at 51.84% examples, 583515 words/s, in_qsize 5, out_qsize 0
  69. 2016-10-04 18:01:09,809 : INFO : PROGRESS: at 52.82% examples, 583962 words/s, in_qsize 6, out_qsize 0
  70. 2016-10-04 18:01:10,817 : INFO : PROGRESS: at 53.78% examples, 584235 words/s, in_qsize 6, out_qsize 0
  71. 2016-10-04 18:01:11,839 : INFO : PROGRESS: at 54.76% examples, 584522 words/s, in_qsize 6, out_qsize 0
  72. 2016-10-04 18:01:12,858 : INFO : PROGRESS: at 55.78% examples, 584986 words/s, in_qsize 6, out_qsize 0
  73. 2016-10-04 18:01:13,861 : INFO : PROGRESS: at 56.77% examples, 585466 words/s, in_qsize 5, out_qsize 0
  74. 2016-10-04 18:01:14,870 : INFO : PROGRESS: at 57.74% examples, 585762 words/s, in_qsize 5, out_qsize 0
  75. 2016-10-04 18:01:15,886 : INFO : PROGRESS: at 58.68% examples, 585689 words/s, in_qsize 5, out_qsize 0
  76. 2016-10-04 18:01:16,882 : INFO : PROGRESS: at 59.64% examples, 585732 words/s, in_qsize 5, out_qsize 0
  77. 2016-10-04 18:01:17,894 : INFO : PROGRESS: at 60.61% examples, 586037 words/s, in_qsize 5, out_qsize 0
  78. 2016-10-04 18:01:18,900 : INFO : PROGRESS: at 61.60% examples, 586320 words/s, in_qsize 5, out_qsize 0
  79. 2016-10-04 18:01:19,900 : INFO : PROGRESS: at 62.57% examples, 586623 words/s, in_qsize 6, out_qsize 0
  80. 2016-10-04 18:01:20,907 : INFO : PROGRESS: at 63.54% examples, 586789 words/s, in_qsize 5, out_qsize 0
  81. 2016-10-04 18:01:21,912 : INFO : PROGRESS: at 64.51% examples, 587086 words/s, in_qsize 6, out_qsize 0
  82. 2016-10-04 18:01:22,914 : INFO : PROGRESS: at 65.44% examples, 587061 words/s, in_qsize 6, out_qsize 0
  83. 2016-10-04 18:01:23,918 : INFO : PROGRESS: at 66.42% examples, 587486 words/s, in_qsize 5, out_qsize 0
  84. 2016-10-04 18:01:24,919 : INFO : PROGRESS: at 67.43% examples, 588196 words/s, in_qsize 6, out_qsize 1
  85. 2016-10-04 18:01:25,940 : INFO : PROGRESS: at 68.37% examples, 588047 words/s, in_qsize 6, out_qsize 1
  86. 2016-10-04 18:01:26,951 : INFO : PROGRESS: at 69.32% examples, 588097 words/s, in_qsize 6, out_qsize 0
  87. 2016-10-04 18:01:27,958 : INFO : PROGRESS: at 70.28% examples, 588176 words/s, in_qsize 5, out_qsize 0
  88. 2016-10-04 18:01:28,971 : INFO : PROGRESS: at 71.28% examples, 588609 words/s, in_qsize 5, out_qsize 0
  89. 2016-10-04 18:01:29,991 : INFO : PROGRESS: at 72.26% examples, 588854 words/s, in_qsize 6, out_qsize 0
  90. 2016-10-04 18:01:31,004 : INFO : PROGRESS: at 73.29% examples, 589416 words/s, in_qsize 6, out_qsize 0
  91. 2016-10-04 18:01:32,010 : INFO : PROGRESS: at 74.30% examples, 589985 words/s, in_qsize 4, out_qsize 0
  92. 2016-10-04 18:01:33,018 : INFO : PROGRESS: at 75.24% examples, 589829 words/s, in_qsize 5, out_qsize 0
  93. 2016-10-04 18:01:34,025 : INFO : PROGRESS: at 76.25% examples, 590140 words/s, in_qsize 5, out_qsize 0
  94. 2016-10-04 18:01:35,033 : INFO : PROGRESS: at 77.21% examples, 590227 words/s, in_qsize 5, out_qsize 0
  95. 2016-10-04 18:01:36,051 : INFO : PROGRESS: at 78.19% examples, 590359 words/s, in_qsize 5, out_qsize 0
  96. 2016-10-04 18:01:37,057 : INFO : PROGRESS: at 79.15% examples, 590425 words/s, in_qsize 5, out_qsize 0
  97. 2016-10-04 18:01:38,066 : INFO : PROGRESS: at 80.20% examples, 591073 words/s, in_qsize 6, out_qsize 0
  98. 2016-10-04 18:01:39,072 : INFO : PROGRESS: at 81.13% examples, 590875 words/s, in_qsize 5, out_qsize 1
  99. 2016-10-04 18:01:40,077 : INFO : PROGRESS: at 82.08% examples, 590802 words/s, in_qsize 5, out_qsize 0
  100. 2016-10-04 18:01:41,086 : INFO : PROGRESS: at 83.09% examples, 591190 words/s, in_qsize 6, out_qsize 0
  101. 2016-10-04 18:01:42,101 : INFO : PROGRESS: at 84.07% examples, 591291 words/s, in_qsize 6, out_qsize 0
  102. 2016-10-04 18:01:43,118 : INFO : PROGRESS: at 85.00% examples, 591112 words/s, in_qsize 6, out_qsize 2
  103. 2016-10-04 18:01:44,127 : INFO : PROGRESS: at 86.00% examples, 591534 words/s, in_qsize 5, out_qsize 0
  104. 2016-10-04 18:01:45,129 : INFO : PROGRESS: at 86.97% examples, 591816 words/s, in_qsize 5, out_qsize 0
  105. 2016-10-04 18:01:46,131 : INFO : PROGRESS: at 87.94% examples, 591939 words/s, in_qsize 5, out_qsize 0
  106. 2016-10-04 18:01:47,146 : INFO : PROGRESS: at 88.94% examples, 592216 words/s, in_qsize 6, out_qsize 0
  107. 2016-10-04 18:01:48,178 : INFO : PROGRESS: at 89.91% examples, 592271 words/s, in_qsize 5, out_qsize 1
  108. 2016-10-04 18:01:49,176 : INFO : PROGRESS: at 90.88% examples, 592398 words/s, in_qsize 6, out_qsize 0
  109. 2016-10-04 18:01:50,182 : INFO : PROGRESS: at 91.88% examples, 592710 words/s, in_qsize 5, out_qsize 0
  110. 2016-10-04 18:01:51,188 : INFO : PROGRESS: at 92.89% examples, 593097 words/s, in_qsize 5, out_qsize 0
  111. 2016-10-04 18:01:52,193 : INFO : PROGRESS: at 93.86% examples, 593259 words/s, in_qsize 6, out_qsize 0
  112. 2016-10-04 18:01:53,195 : INFO : PROGRESS: at 94.85% examples, 593538 words/s, in_qsize 6, out_qsize 1
  113. 2016-10-04 18:01:54,203 : INFO : PROGRESS: at 95.87% examples, 593758 words/s, in_qsize 6, out_qsize 0
  114. 2016-10-04 18:01:55,222 : INFO : PROGRESS: at 96.87% examples, 593945 words/s, in_qsize 5, out_qsize 0
  115. 2016-10-04 18:01:56,226 : INFO : PROGRESS: at 97.87% examples, 594229 words/s, in_qsize 5, out_qsize 0
  116. 2016-10-04 18:01:57,235 : INFO : PROGRESS: at 98.87% examples, 594427 words/s, in_qsize 6, out_qsize 0
  117. 2016-10-04 18:01:58,245 : INFO : PROGRESS: at 99.85% examples, 594484 words/s, in_qsize 5, out_qsize 1
  118. 2016-10-04 18:01:58,367 : INFO : worker thread finished; awaiting finish of 2 more threads
  119. 2016-10-04 18:01:58,373 : INFO : worker thread finished; awaiting finish of 1 more threads
  120. 2016-10-04 18:01:58,381 : INFO : worker thread finished; awaiting finish of 0 more threads
  121. 2016-10-04 18:01:58,382 : INFO : training on 85026035 raw words (62534851 effective words) took 105.2s, 594607 effective words/s
  122. 2016-10-04 18:01:58,382 : INFO : precomputing L2-norms of word weight vectors
  123. 2016-10-04 18:01:58,830 : INFO : saving Word2Vec object under text8_data, separately None
  124. 2016-10-04 18:01:58,830 : INFO : not storing attribute syn0norm
  125. 2016-10-04 18:01:58,831 : INFO : not storing attribute cum_table
  126. 2016-10-04 18:01:59,612 : INFO : Removed 17 and 25 OOV words from document 1 and 2 (respectively).
  127. 2016-10-04 18:01:59,612 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
  128. 2016-10-04 18:01:59,612 : INFO : built Dictionary(59 unique tokens: [u'heart', u'love', u'departed', u'deck', u'within']...) from 2 documents (total 75 corpus positions)
  129. Distance between the two poems is: 1.07395798382
  130. 2016-10-04 18:01:59,630 : INFO : Removed 0 and 1 OOV words from document 1 and 2 (respectively).
  131. 2016-10-04 18:01:59,630 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
  132. 2016-10-04 18:01:59,631 : INFO : built Dictionary(7 unique tokens: [u'chicago', u'media', u'illinois', u'press', u'president']...) from 2 documents (total 7 corpus positions)
  133. Distance between the two sentences is: 1.1235094668
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!