API tools faq
paste
lamiastella

gensime_4workers

lamiastella
Oct 4th, 2016
269
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 12.44 KB | None | 0 0
raw download clone embed print report
  1. jalal@klein:~/computer_vision/image_retrieval$ python gensim_test.py
  2. 2016-10-04 17:57:19,953 : INFO : collecting all words and their counts
  3. 2016-10-04 17:57:19,955 : INFO : PROGRESS: at sentence #0, processed 0 words, keeping 0 word types
  4. 2016-10-04 17:57:25,847 : INFO : collected 253854 word types from a corpus of 17005207 raw words and 1701 sentences
  5. 2016-10-04 17:57:26,261 : INFO : min_count=5 retains 71290 unique words (drops 182564)
  6. 2016-10-04 17:57:26,261 : INFO : min_count leaves 16718844 word corpus (98% of original 17005207)
  7. 2016-10-04 17:57:26,501 : INFO : deleting the raw counts dictionary of 253854 items
  8. 2016-10-04 17:57:26,531 : INFO : sample=0.001 downsamples 38 most-common words
  9. 2016-10-04 17:57:26,532 : INFO : downsampling leaves estimated 12506280 word corpus (74.8% of prior 16718844)
  10. 2016-10-04 17:57:26,532 : INFO : estimated required memory for 71290 words and 100 dimensions: 92677000 bytes
  11. 2016-10-04 17:57:26,732 : INFO : resetting layer weights
  12. 2016-10-04 17:57:28,316 : INFO : training model with 4 workers on 71290 vocabulary and 100 features, using sg=0 hs=0 sample=0.001 negative=5
  13. 2016-10-04 17:57:28,316 : INFO : expecting 1701 sentences, matching count from corpus used for vocabulary survey
  14. 2016-10-04 17:57:29,319 : INFO : PROGRESS: at 0.92% examples, 573648 words/s, in_qsize 7, out_qsize 1
  15. 2016-10-04 17:57:30,326 : INFO : PROGRESS: at 1.93% examples, 596075 words/s, in_qsize 7, out_qsize 0
  16. 2016-10-04 17:57:31,330 : INFO : PROGRESS: at 2.96% examples, 610943 words/s, in_qsize 8, out_qsize 0
  17. 2016-10-04 17:57:32,340 : INFO : PROGRESS: at 3.96% examples, 613345 words/s, in_qsize 6, out_qsize 0
  18. 2016-10-04 17:57:33,349 : INFO : PROGRESS: at 4.97% examples, 617366 words/s, in_qsize 7, out_qsize 0
  19. 2016-10-04 17:57:34,346 : INFO : PROGRESS: at 6.04% examples, 627007 words/s, in_qsize 8, out_qsize 0
  20. 2016-10-04 17:57:35,352 : INFO : PROGRESS: at 7.07% examples, 629441 words/s, in_qsize 7, out_qsize 0
  21. 2016-10-04 17:57:36,362 : INFO : PROGRESS: at 8.10% examples, 631056 words/s, in_qsize 7, out_qsize 0
  22. 2016-10-04 17:57:37,368 : INFO : PROGRESS: at 9.12% examples, 631980 words/s, in_qsize 8, out_qsize 0
  23. 2016-10-04 17:57:38,378 : INFO : PROGRESS: at 10.23% examples, 637477 words/s, in_qsize 8, out_qsize 0
  24. 2016-10-04 17:57:39,402 : INFO : PROGRESS: at 11.25% examples, 636784 words/s, in_qsize 8, out_qsize 1
  25. 2016-10-04 17:57:40,415 : INFO : PROGRESS: at 12.33% examples, 639466 words/s, in_qsize 7, out_qsize 0
  26. 2016-10-04 17:57:41,441 : INFO : PROGRESS: at 13.36% examples, 638240 words/s, in_qsize 7, out_qsize 0
  27. 2016-10-04 17:57:42,449 : INFO : PROGRESS: at 14.42% examples, 639743 words/s, in_qsize 7, out_qsize 0
  28. 2016-10-04 17:57:43,460 : INFO : PROGRESS: at 15.49% examples, 640275 words/s, in_qsize 8, out_qsize 0
  29. 2016-10-04 17:57:44,470 : INFO : PROGRESS: at 16.52% examples, 640020 words/s, in_qsize 8, out_qsize 0
  30. 2016-10-04 17:57:45,479 : INFO : PROGRESS: at 17.51% examples, 638451 words/s, in_qsize 7, out_qsize 0
  31. 2016-10-04 17:57:46,500 : INFO : PROGRESS: at 18.55% examples, 638518 words/s, in_qsize 7, out_qsize 0
  32. 2016-10-04 17:57:47,506 : INFO : PROGRESS: at 19.59% examples, 638558 words/s, in_qsize 7, out_qsize 0
  33. 2016-10-04 17:57:48,513 : INFO : PROGRESS: at 20.62% examples, 638764 words/s, in_qsize 8, out_qsize 0
  34. 2016-10-04 17:57:49,521 : INFO : PROGRESS: at 21.66% examples, 638438 words/s, in_qsize 8, out_qsize 0
  35. 2016-10-04 17:57:50,521 : INFO : PROGRESS: at 22.68% examples, 638329 words/s, in_qsize 7, out_qsize 0
  36. 2016-10-04 17:57:51,529 : INFO : PROGRESS: at 23.77% examples, 639999 words/s, in_qsize 8, out_qsize 0
  37. 2016-10-04 17:57:52,549 : INFO : PROGRESS: at 24.84% examples, 641164 words/s, in_qsize 6, out_qsize 0
  38. 2016-10-04 17:57:53,545 : INFO : PROGRESS: at 25.90% examples, 642086 words/s, in_qsize 7, out_qsize 0
  39. 2016-10-04 17:57:54,559 : INFO : PROGRESS: at 26.89% examples, 641203 words/s, in_qsize 8, out_qsize 1
  40. 2016-10-04 17:57:55,563 : INFO : PROGRESS: at 27.97% examples, 642430 words/s, in_qsize 7, out_qsize 0
  41. 2016-10-04 17:57:56,569 : INFO : PROGRESS: at 28.98% examples, 642040 words/s, in_qsize 7, out_qsize 0
  42. 2016-10-04 17:57:57,578 : INFO : PROGRESS: at 30.03% examples, 642361 words/s, in_qsize 8, out_qsize 1
  43. 2016-10-04 17:57:58,617 : INFO : PROGRESS: at 31.05% examples, 641530 words/s, in_qsize 6, out_qsize 2
  44. 2016-10-04 17:57:59,626 : INFO : PROGRESS: at 32.12% examples, 642466 words/s, in_qsize 8, out_qsize 2
  45. 2016-10-04 17:58:00,626 : INFO : PROGRESS: at 33.22% examples, 643609 words/s, in_qsize 7, out_qsize 0
  46. 2016-10-04 17:58:01,634 : INFO : PROGRESS: at 34.33% examples, 645186 words/s, in_qsize 8, out_qsize 0
  47. 2016-10-04 17:58:02,652 : INFO : PROGRESS: at 35.38% examples, 644788 words/s, in_qsize 7, out_qsize 1
  48. 2016-10-04 17:58:03,663 : INFO : PROGRESS: at 36.50% examples, 645896 words/s, in_qsize 7, out_qsize 0
  49. 2016-10-04 17:58:04,665 : INFO : PROGRESS: at 37.55% examples, 646286 words/s, in_qsize 7, out_qsize 0
  50. 2016-10-04 17:58:05,674 : INFO : PROGRESS: at 38.67% examples, 647561 words/s, in_qsize 7, out_qsize 0
  51. 2016-10-04 17:58:06,678 : INFO : PROGRESS: at 39.74% examples, 647898 words/s, in_qsize 8, out_qsize 0
  52. 2016-10-04 17:58:07,683 : INFO : PROGRESS: at 40.80% examples, 648177 words/s, in_qsize 7, out_qsize 0
  53. 2016-10-04 17:58:08,692 : INFO : PROGRESS: at 41.82% examples, 647608 words/s, in_qsize 7, out_qsize 0
  54. 2016-10-04 17:58:09,692 : INFO : PROGRESS: at 42.83% examples, 647148 words/s, in_qsize 7, out_qsize 0
  55. 2016-10-04 17:58:10,710 : INFO : PROGRESS: at 43.87% examples, 646940 words/s, in_qsize 7, out_qsize 0
  56. 2016-10-04 17:58:11,706 : INFO : PROGRESS: at 44.94% examples, 647466 words/s, in_qsize 8, out_qsize 0
  57. 2016-10-04 17:58:12,714 : INFO : PROGRESS: at 46.01% examples, 648042 words/s, in_qsize 7, out_qsize 0
  58. 2016-10-04 17:58:13,723 : INFO : PROGRESS: at 47.03% examples, 647891 words/s, in_qsize 8, out_qsize 0
  59. 2016-10-04 17:58:14,750 : INFO : PROGRESS: at 48.15% examples, 648752 words/s, in_qsize 7, out_qsize 2
  60. 2016-10-04 17:58:15,746 : INFO : PROGRESS: at 49.21% examples, 649018 words/s, in_qsize 8, out_qsize 0
  61. 2016-10-04 17:58:16,747 : INFO : PROGRESS: at 50.22% examples, 648714 words/s, in_qsize 7, out_qsize 0
  62. 2016-10-04 17:58:17,755 : INFO : PROGRESS: at 51.24% examples, 648552 words/s, in_qsize 7, out_qsize 0
  63. 2016-10-04 17:58:18,762 : INFO : PROGRESS: at 52.31% examples, 648912 words/s, in_qsize 7, out_qsize 1
  64. 2016-10-04 17:58:19,764 : INFO : PROGRESS: at 53.37% examples, 649106 words/s, in_qsize 8, out_qsize 0
  65. 2016-10-04 17:58:20,777 : INFO : PROGRESS: at 54.44% examples, 649383 words/s, in_qsize 8, out_qsize 0
  66. 2016-10-04 17:58:21,779 : INFO : PROGRESS: at 55.53% examples, 649755 words/s, in_qsize 7, out_qsize 2
  67. 2016-10-04 17:58:22,784 : INFO : PROGRESS: at 56.59% examples, 649830 words/s, in_qsize 7, out_qsize 0
  68. 2016-10-04 17:58:23,808 : INFO : PROGRESS: at 57.71% examples, 650459 words/s, in_qsize 7, out_qsize 0
  69. 2016-10-04 17:58:24,814 : INFO : PROGRESS: at 58.78% examples, 650646 words/s, in_qsize 7, out_qsize 0
  70. 2016-10-04 17:58:25,824 : INFO : PROGRESS: at 59.84% examples, 650682 words/s, in_qsize 7, out_qsize 0
  71. 2016-10-04 17:58:26,828 : INFO : PROGRESS: at 60.87% examples, 650561 words/s, in_qsize 8, out_qsize 0
  72. 2016-10-04 17:58:27,840 : INFO : PROGRESS: at 61.92% examples, 650359 words/s, in_qsize 7, out_qsize 0
  73. 2016-10-04 17:58:28,839 : INFO : PROGRESS: at 62.93% examples, 650000 words/s, in_qsize 8, out_qsize 0
  74. 2016-10-04 17:58:29,839 : INFO : PROGRESS: at 63.97% examples, 650066 words/s, in_qsize 8, out_qsize 0
  75. 2016-10-04 17:58:30,842 : INFO : PROGRESS: at 65.02% examples, 650168 words/s, in_qsize 8, out_qsize 0
  76. 2016-10-04 17:58:31,861 : INFO : PROGRESS: at 66.06% examples, 650079 words/s, in_qsize 7, out_qsize 0
  77. 2016-10-04 17:58:32,874 : INFO : PROGRESS: at 67.11% examples, 650348 words/s, in_qsize 7, out_qsize 1
  78. 2016-10-04 17:58:33,876 : INFO : PROGRESS: at 68.22% examples, 650962 words/s, in_qsize 7, out_qsize 1
  79. 2016-10-04 17:58:34,880 : INFO : PROGRESS: at 69.28% examples, 651065 words/s, in_qsize 8, out_qsize 0
  80. 2016-10-04 17:58:35,884 : INFO : PROGRESS: at 70.34% examples, 651226 words/s, in_qsize 8, out_qsize 0
  81. 2016-10-04 17:58:36,889 : INFO : PROGRESS: at 71.37% examples, 651174 words/s, in_qsize 7, out_qsize 0
  82. 2016-10-04 17:58:37,904 : INFO : PROGRESS: at 72.42% examples, 651131 words/s, in_qsize 7, out_qsize 0
  83. 2016-10-04 17:58:38,915 : INFO : PROGRESS: at 73.45% examples, 650962 words/s, in_qsize 8, out_qsize 0
  84. 2016-10-04 17:58:39,921 : INFO : PROGRESS: at 74.50% examples, 651004 words/s, in_qsize 7, out_qsize 0
  85. 2016-10-04 17:58:40,922 : INFO : PROGRESS: at 75.63% examples, 651542 words/s, in_qsize 7, out_qsize 0
  86. 2016-10-04 17:58:41,928 : INFO : PROGRESS: at 76.68% examples, 651603 words/s, in_qsize 7, out_qsize 0
  87. 2016-10-04 17:58:42,939 : INFO : PROGRESS: at 77.74% examples, 651640 words/s, in_qsize 8, out_qsize 1
  88. 2016-10-04 17:58:43,947 : INFO : PROGRESS: at 78.75% examples, 651252 words/s, in_qsize 8, out_qsize 0
  89. 2016-10-04 17:58:44,956 : INFO : PROGRESS: at 79.79% examples, 651073 words/s, in_qsize 8, out_qsize 0
  90. 2016-10-04 17:58:45,981 : INFO : PROGRESS: at 80.82% examples, 650839 words/s, in_qsize 7, out_qsize 1
  91. 2016-10-04 17:58:46,991 : INFO : PROGRESS: at 81.81% examples, 650205 words/s, in_qsize 8, out_qsize 0
  92. 2016-10-04 17:58:48,007 : INFO : PROGRESS: at 82.82% examples, 649876 words/s, in_qsize 7, out_qsize 0
  93. 2016-10-04 17:58:49,002 : INFO : PROGRESS: at 83.86% examples, 649793 words/s, in_qsize 8, out_qsize 0
  94. 2016-10-04 17:58:50,005 : INFO : PROGRESS: at 84.86% examples, 649515 words/s, in_qsize 7, out_qsize 0
  95. 2016-10-04 17:58:51,013 : INFO : PROGRESS: at 85.89% examples, 649501 words/s, in_qsize 7, out_qsize 0
  96. 2016-10-04 17:58:52,019 : INFO : PROGRESS: at 86.89% examples, 649256 words/s, in_qsize 7, out_qsize 0
  97. 2016-10-04 17:58:53,026 : INFO : PROGRESS: at 87.89% examples, 648951 words/s, in_qsize 7, out_qsize 0
  98. 2016-10-04 17:58:54,036 : INFO : PROGRESS: at 88.94% examples, 648963 words/s, in_qsize 8, out_qsize 0
  99. 2016-10-04 17:58:55,037 : INFO : PROGRESS: at 89.92% examples, 648631 words/s, in_qsize 7, out_qsize 0
  100. 2016-10-04 17:58:56,056 : INFO : PROGRESS: at 90.98% examples, 648682 words/s, in_qsize 8, out_qsize 0
  101. 2016-10-04 17:58:57,058 : INFO : PROGRESS: at 92.04% examples, 648850 words/s, in_qsize 7, out_qsize 0
  102. 2016-10-04 17:58:58,067 : INFO : PROGRESS: at 93.11% examples, 649008 words/s, in_qsize 7, out_qsize 0
  103. 2016-10-04 17:58:59,071 : INFO : PROGRESS: at 94.20% examples, 649408 words/s, in_qsize 7, out_qsize 0
  104. 2016-10-04 17:59:00,089 : INFO : PROGRESS: at 95.26% examples, 649381 words/s, in_qsize 6, out_qsize 1
  105. 2016-10-04 17:59:01,087 : INFO : PROGRESS: at 96.36% examples, 649609 words/s, in_qsize 7, out_qsize 0
  106. 2016-10-04 17:59:02,089 : INFO : PROGRESS: at 97.40% examples, 649629 words/s, in_qsize 7, out_qsize 0
  107. 2016-10-04 17:59:03,089 : INFO : PROGRESS: at 98.41% examples, 649434 words/s, in_qsize 8, out_qsize 0
  108. 2016-10-04 17:59:04,107 : INFO : PROGRESS: at 99.47% examples, 649476 words/s, in_qsize 7, out_qsize 0
  109. 2016-10-04 17:59:04,575 : INFO : worker thread finished; awaiting finish of 3 more threads
  110. 2016-10-04 17:59:04,593 : INFO : worker thread finished; awaiting finish of 2 more threads
  111. 2016-10-04 17:59:04,600 : INFO : worker thread finished; awaiting finish of 1 more threads
  112. 2016-10-04 17:59:04,601 : INFO : worker thread finished; awaiting finish of 0 more threads
  113. 2016-10-04 17:59:04,601 : INFO : training on 85026035 raw words (62533990 effective words) took 96.3s, 649478 effective words/s
  114. 2016-10-04 17:59:04,602 : INFO : precomputing L2-norms of word weight vectors
  115. 2016-10-04 17:59:05,182 : INFO : saving Word2Vec object under text8_data, separately None
  116. 2016-10-04 17:59:05,182 : INFO : not storing attribute syn0norm
  117. 2016-10-04 17:59:05,182 : INFO : not storing attribute cum_table
  118. 2016-10-04 17:59:05,999 : INFO : Removed 17 and 25 OOV words from document 1 and 2 (respectively).
  119. 2016-10-04 17:59:05,999 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
  120. 2016-10-04 17:59:05,999 : INFO : built Dictionary(59 unique tokens: [u'heart', u'love', u'departed', u'deck', u'within']...) from 2 documents (total 75 corpus positions)
  121. Distance between the two poems is: 1.0767464193
  122. 2016-10-04 17:59:06,014 : INFO : Removed 0 and 1 OOV words from document 1 and 2 (respectively).
  123. 2016-10-04 17:59:06,015 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
  124. 2016-10-04 17:59:06,015 : INFO : built Dictionary(7 unique tokens: [u'chicago', u'media', u'illinois', u'press', u'president']...) from 2 documents (total 7 corpus positions)
  125. Distance between the two sentences is: 1.12212419522
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!