API tools faq
paste
Advertisement
Guest User

ted_en-es_training

a guest
Feb 5th, 2019
257
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 266.96 KB | None | 0 0
raw download clone embed print report
  1.  
  2. The following have been reloaded with a version change:
  3. 1) cuDNN/7.4.1-CUDA-9.0.176 => cuDNN/7.0.5-CUDA-9.0.176
  4.  
  5.  
  6. The following have been reloaded with a version change:
  7. 1) CUDA/9.0.176 => CUDA/9.1.85
  8. 2) cuDNN/7.0.5-CUDA-9.0.176 => cuDNN/7.0.5-CUDA-9.1.85
  9.  
  10. [2019-02-04 17:50:15,386 INFO] * src vocab size = 17138
  11. [2019-02-04 17:50:15,387 INFO] * tgt vocab size = 21763
  12. [2019-02-04 17:50:15,388 INFO] Building model...
  13. [2019-02-04 17:50:30,601 INFO] NMTModel(
  14. (encoder): RNNEncoder(
  15. (embeddings): Embeddings(
  16. (make_embedding): Sequential(
  17. (emb_luts): Elementwise(
  18. (0): Embedding(17138, 512, padding_idx=1)
  19. )
  20. )
  21. )
  22. (rnn): LSTM(512, 256, num_layers=3, dropout=0.25, bidirectional=True)
  23. )
  24. (decoder): InputFeedRNNDecoder(
  25. (embeddings): Embeddings(
  26. (make_embedding): Sequential(
  27. (emb_luts): Elementwise(
  28. (0): Embedding(21763, 512, padding_idx=1)
  29. )
  30. )
  31. )
  32. (dropout): Dropout(p=0.25)
  33. (rnn): StackedLSTM(
  34. (dropout): Dropout(p=0.25)
  35. (layers): ModuleList(
  36. (0): LSTMCell(1024, 512)
  37. (1): LSTMCell(512, 512)
  38. (2): LSTMCell(512, 512)
  39. )
  40. )
  41. (attn): GlobalAttention(
  42. (linear_context): Linear(in_features=512, out_features=512, bias=False)
  43. (linear_query): Linear(in_features=512, out_features=512, bias=True)
  44. (v): Linear(in_features=512, out_features=1, bias=False)
  45. (linear_out): Linear(in_features=1024, out_features=512, bias=True)
  46. (linear_cover): Linear(in_features=1, out_features=512, bias=False)
  47. )
  48. )
  49. (generator): Sequential(
  50. (0): Linear(in_features=512, out_features=21763, bias=True)
  51. (1): LogSoftmax()
  52. )
  53. )
  54. [2019-02-04 17:50:30,603 INFO] encoder: 13505536
  55. [2019-02-04 17:50:30,603 INFO] decoder: 30710019
  56. [2019-02-04 17:50:30,604 INFO] * number of parameters: 44215555
  57. [2019-02-04 17:50:30,610 INFO] Starting training on GPU: [0]
  58. [2019-02-04 17:50:30,611 INFO] Start training...
  59. [2019-02-04 17:50:35,984 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  60. [2019-02-04 17:50:42,208 INFO] Step 50/102583; acc: 22.29; ppl: 2244.55; xent: 7.72; lr: 0.00010; 909/1133 tok/s; 12 sec
  61. [2019-02-04 17:50:47,054 INFO] Step 100/102583; acc: 23.00; ppl: 51.86; xent: 3.95; lr: 0.00010; 2014/2635 tok/s; 16 sec
  62. [2019-02-04 17:50:52,272 INFO] Step 150/102583; acc: 21.82; ppl: 24.18; xent: 3.19; lr: 0.00010; 2002/2624 tok/s; 22 sec
  63. [2019-02-04 17:50:57,151 INFO] Step 200/102583; acc: 26.59; ppl: 17.16; xent: 2.84; lr: 0.00010; 2006/2555 tok/s; 27 sec
  64. [2019-02-04 17:51:02,174 INFO] Step 250/102583; acc: 23.90; ppl: 17.22; xent: 2.85; lr: 0.00010; 2006/2672 tok/s; 32 sec
  65. [2019-02-04 17:51:07,262 INFO] Step 300/102583; acc: 27.05; ppl: 15.16; xent: 2.72; lr: 0.00010; 2033/2526 tok/s; 37 sec
  66. [2019-02-04 17:51:12,237 INFO] Step 350/102583; acc: 26.49; ppl: 16.15; xent: 2.78; lr: 0.00010; 2085/2628 tok/s; 42 sec
  67. [2019-02-04 17:51:17,271 INFO] Step 400/102583; acc: 28.90; ppl: 13.62; xent: 2.61; lr: 0.00010; 2002/2614 tok/s; 47 sec
  68. [2019-02-04 17:51:22,496 INFO] Step 450/102583; acc: 27.29; ppl: 15.19; xent: 2.72; lr: 0.00010; 1957/2524 tok/s; 52 sec
  69. [2019-02-04 17:51:27,372 INFO] Step 500/102583; acc: 33.66; ppl: 12.59; xent: 2.53; lr: 0.00010; 2074/2603 tok/s; 57 sec
  70. [2019-02-04 17:51:32,590 INFO] Step 550/102583; acc: 30.82; ppl: 13.15; xent: 2.58; lr: 0.00010; 1889/2633 tok/s; 62 sec
  71. [2019-02-04 17:51:37,543 INFO] Step 600/102583; acc: 37.45; ppl: 10.44; xent: 2.35; lr: 0.00010; 2151/2526 tok/s; 67 sec
  72. [2019-02-04 17:51:42,645 INFO] Step 650/102583; acc: 37.38; ppl: 10.46; xent: 2.35; lr: 0.00010; 2028/2532 tok/s; 72 sec
  73. [2019-02-04 17:51:47,611 INFO] Step 700/102583; acc: 36.88; ppl: 11.36; xent: 2.43; lr: 0.00010; 1997/2606 tok/s; 77 sec
  74. [2019-02-04 17:51:52,782 INFO] Step 750/102583; acc: 39.15; ppl: 9.41; xent: 2.24; lr: 0.00010; 2129/2609 tok/s; 82 sec
  75. [2019-02-04 17:51:57,713 INFO] Step 800/102583; acc: 42.00; ppl: 8.45; xent: 2.13; lr: 0.00010; 1953/2596 tok/s; 87 sec
  76. [2019-02-04 17:52:02,916 INFO] Step 850/102583; acc: 44.82; ppl: 7.38; xent: 2.00; lr: 0.00010; 1957/2467 tok/s; 92 sec
  77. [2019-02-04 17:52:08,053 INFO] Step 900/102583; acc: 45.57; ppl: 7.43; xent: 2.01; lr: 0.00010; 1964/2591 tok/s; 97 sec
  78. [2019-02-04 17:52:13,079 INFO] Step 950/102583; acc: 52.08; ppl: 5.78; xent: 1.75; lr: 0.00010; 2065/2411 tok/s; 102 sec
  79. [2019-02-04 17:52:18,112 INFO] Step 1000/102583; acc: 45.86; ppl: 6.88; xent: 1.93; lr: 0.00010; 1967/2734 tok/s; 108 sec
  80. [2019-02-04 17:52:23,449 INFO] Step 1050/102583; acc: 47.03; ppl: 6.43; xent: 1.86; lr: 0.00010; 1885/2573 tok/s; 113 sec
  81. [2019-02-04 17:52:28,272 INFO] Step 1100/102583; acc: 55.66; ppl: 5.08; xent: 1.62; lr: 0.00010; 2107/2556 tok/s; 118 sec
  82. [2019-02-04 17:52:33,504 INFO] Step 1150/102583; acc: 50.02; ppl: 5.70; xent: 1.74; lr: 0.00010; 2075/2497 tok/s; 123 sec
  83. [2019-02-04 17:52:38,365 INFO] Step 1200/102583; acc: 54.90; ppl: 5.00; xent: 1.61; lr: 0.00010; 1961/2639 tok/s; 128 sec
  84. [2019-02-04 17:52:43,057 INFO] Step 1250/102583; acc: 55.88; ppl: 4.84; xent: 1.58; lr: 0.00010; 2056/2644 tok/s; 132 sec
  85. [2019-02-04 17:52:48,423 INFO] Step 1300/102583; acc: 53.18; ppl: 5.24; xent: 1.66; lr: 0.00010; 1987/2524 tok/s; 138 sec
  86. [2019-02-04 17:52:53,820 INFO] Step 1350/102583; acc: 51.94; ppl: 5.23; xent: 1.66; lr: 0.00010; 2071/2480 tok/s; 143 sec
  87. [2019-02-04 17:52:58,611 INFO] Step 1400/102583; acc: 57.14; ppl: 4.48; xent: 1.50; lr: 0.00010; 1891/2668 tok/s; 148 sec
  88. [2019-02-04 17:53:03,815 INFO] Step 1450/102583; acc: 54.11; ppl: 4.90; xent: 1.59; lr: 0.00010; 2057/2630 tok/s; 153 sec
  89. [2019-02-04 17:53:08,635 INFO] Step 1500/102583; acc: 62.25; ppl: 3.99; xent: 1.38; lr: 0.00010; 1983/2510 tok/s; 158 sec
  90. [2019-02-04 17:53:13,493 INFO] Step 1550/102583; acc: 58.04; ppl: 4.39; xent: 1.48; lr: 0.00010; 1916/2629 tok/s; 163 sec
  91. [2019-02-04 17:53:18,709 INFO] Step 1600/102583; acc: 58.65; ppl: 4.17; xent: 1.43; lr: 0.00010; 2067/2504 tok/s; 168 sec
  92. [2019-02-04 17:53:23,662 INFO] Step 1650/102583; acc: 55.29; ppl: 4.58; xent: 1.52; lr: 0.00010; 2055/2769 tok/s; 173 sec
  93. [2019-02-04 17:53:28,697 INFO] Step 1700/102583; acc: 63.21; ppl: 3.72; xent: 1.31; lr: 0.00010; 2035/2478 tok/s; 178 sec
  94. [2019-02-04 17:53:33,790 INFO] Step 1750/102583; acc: 59.42; ppl: 4.12; xent: 1.42; lr: 0.00010; 1877/2713 tok/s; 183 sec
  95. [2019-02-04 17:53:38,694 INFO] Step 1800/102583; acc: 61.78; ppl: 3.76; xent: 1.32; lr: 0.00010; 2176/2483 tok/s; 188 sec
  96. [2019-02-04 17:53:43,834 INFO] Step 1850/102583; acc: 57.93; ppl: 4.27; xent: 1.45; lr: 0.00010; 1867/2648 tok/s; 193 sec
  97. [2019-02-04 17:53:48,773 INFO] Step 1900/102583; acc: 63.80; ppl: 3.47; xent: 1.24; lr: 0.00010; 2143/2505 tok/s; 198 sec
  98. [2019-02-04 17:53:53,971 INFO] Step 1950/102583; acc: 61.64; ppl: 3.82; xent: 1.34; lr: 0.00010; 2121/2602 tok/s; 203 sec
  99. [2019-02-04 17:53:58,951 INFO] Step 2000/102583; acc: 61.96; ppl: 3.73; xent: 1.32; lr: 0.00010; 1880/2533 tok/s; 208 sec
  100. [2019-02-04 17:54:03,869 INFO] Step 2050/102583; acc: 63.81; ppl: 3.57; xent: 1.27; lr: 0.00010; 1931/2618 tok/s; 213 sec
  101. [2019-02-04 17:54:08,976 INFO] Step 2100/102583; acc: 63.32; ppl: 3.56; xent: 1.27; lr: 0.00010; 2110/2528 tok/s; 218 sec
  102. [2019-02-04 17:54:13,788 INFO] Step 2150/102583; acc: 65.30; ppl: 3.48; xent: 1.25; lr: 0.00010; 1983/2676 tok/s; 223 sec
  103. [2019-02-04 17:54:18,872 INFO] Step 2200/102583; acc: 63.72; ppl: 3.46; xent: 1.24; lr: 0.00010; 2090/2552 tok/s; 228 sec
  104. [2019-02-04 17:54:23,653 INFO] Step 2250/102583; acc: 67.05; ppl: 3.14; xent: 1.14; lr: 0.00010; 2118/2578 tok/s; 233 sec
  105. [2019-02-04 17:54:28,954 INFO] Step 2300/102583; acc: 62.97; ppl: 3.51; xent: 1.26; lr: 0.00010; 1900/2598 tok/s; 238 sec
  106. [2019-02-04 17:54:34,018 INFO] Step 2350/102583; acc: 65.63; ppl: 3.23; xent: 1.17; lr: 0.00010; 2041/2646 tok/s; 243 sec
  107. [2019-02-04 17:54:38,901 INFO] Step 2400/102583; acc: 64.94; ppl: 3.42; xent: 1.23; lr: 0.00010; 2024/2603 tok/s; 248 sec
  108. [2019-02-04 17:54:43,995 INFO] Step 2450/102583; acc: 67.50; ppl: 3.16; xent: 1.15; lr: 0.00010; 2055/2572 tok/s; 253 sec
  109. [2019-02-04 17:54:48,881 INFO] Step 2500/102583; acc: 65.04; ppl: 3.33; xent: 1.20; lr: 0.00010; 2019/2659 tok/s; 258 sec
  110. [2019-02-04 17:54:53,544 INFO] Step 2550/102583; acc: 69.14; ppl: 2.89; xent: 1.06; lr: 0.00010; 2085/2576 tok/s; 263 sec
  111. [2019-02-04 17:54:58,945 INFO] Step 2600/102583; acc: 65.39; ppl: 3.37; xent: 1.22; lr: 0.00010; 1972/2632 tok/s; 268 sec
  112. [2019-02-04 17:55:03,881 INFO] Step 2650/102583; acc: 69.14; ppl: 2.88; xent: 1.06; lr: 0.00010; 2192/2504 tok/s; 273 sec
  113. [2019-02-04 17:55:08,949 INFO] Step 2700/102583; acc: 66.56; ppl: 3.29; xent: 1.19; lr: 0.00010; 1846/2700 tok/s; 278 sec
  114. [2019-02-04 17:55:14,115 INFO] Step 2750/102583; acc: 64.83; ppl: 3.38; xent: 1.22; lr: 0.00010; 1905/2532 tok/s; 284 sec
  115. [2019-02-04 17:55:19,022 INFO] Step 2800/102583; acc: 71.06; ppl: 2.71; xent: 1.00; lr: 0.00010; 2140/2657 tok/s; 288 sec
  116. [2019-02-04 17:55:23,876 INFO] Step 2850/102583; acc: 71.02; ppl: 2.76; xent: 1.01; lr: 0.00010; 1878/2701 tok/s; 293 sec
  117. [2019-02-04 17:55:29,154 INFO] Step 2900/102583; acc: 67.34; ppl: 3.11; xent: 1.14; lr: 0.00010; 2123/2482 tok/s; 299 sec
  118. [2019-02-04 17:55:34,035 INFO] Step 2950/102583; acc: 70.86; ppl: 2.81; xent: 1.03; lr: 0.00010; 2013/2682 tok/s; 303 sec
  119. [2019-02-04 17:55:39,171 INFO] Step 3000/102583; acc: 67.21; ppl: 3.04; xent: 1.11; lr: 0.00010; 2067/2480 tok/s; 309 sec
  120. [2019-02-04 17:55:44,325 INFO] Step 3050/102583; acc: 68.09; ppl: 3.00; xent: 1.10; lr: 0.00010; 2116/2585 tok/s; 314 sec
  121. [2019-02-04 17:55:49,150 INFO] Step 3100/102583; acc: 71.07; ppl: 2.72; xent: 1.00; lr: 0.00010; 1962/2655 tok/s; 319 sec
  122. [2019-02-04 17:55:53,813 INFO] Step 3150/102583; acc: 72.80; ppl: 2.59; xent: 0.95; lr: 0.00010; 2109/2564 tok/s; 323 sec
  123. [2019-02-04 17:55:59,247 INFO] Step 3200/102583; acc: 68.24; ppl: 2.99; xent: 1.10; lr: 0.00010; 1914/2608 tok/s; 329 sec
  124. [2019-02-04 17:56:08,691 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  125. [2019-02-04 17:56:09,927 INFO] Step 3250/102583; acc: 71.12; ppl: 2.79; xent: 1.03; lr: 0.00010; 922/1178 tok/s; 339 sec
  126. [2019-02-04 17:56:15,038 INFO] Step 3300/102583; acc: 71.52; ppl: 2.69; xent: 0.99; lr: 0.00010; 2126/2531 tok/s; 344 sec
  127. [2019-02-04 17:56:19,809 INFO] Step 3350/102583; acc: 71.30; ppl: 2.68; xent: 0.98; lr: 0.00010; 1952/2681 tok/s; 349 sec
  128. [2019-02-04 17:56:25,104 INFO] Step 3400/102583; acc: 71.99; ppl: 2.62; xent: 0.96; lr: 0.00010; 2033/2637 tok/s; 354 sec
  129. [2019-02-04 17:56:29,953 INFO] Step 3450/102583; acc: 71.08; ppl: 2.69; xent: 0.99; lr: 0.00010; 2073/2600 tok/s; 359 sec
  130. [2019-02-04 17:56:35,001 INFO] Step 3500/102583; acc: 72.46; ppl: 2.62; xent: 0.96; lr: 0.00010; 2074/2637 tok/s; 364 sec
  131. [2019-02-04 17:56:40,046 INFO] Step 3550/102583; acc: 69.78; ppl: 2.77; xent: 1.02; lr: 0.00010; 2007/2597 tok/s; 369 sec
  132. [2019-02-04 17:56:44,837 INFO] Step 3600/102583; acc: 73.71; ppl: 2.47; xent: 0.90; lr: 0.00010; 2113/2745 tok/s; 374 sec
  133. [2019-02-04 17:56:49,852 INFO] Step 3650/102583; acc: 70.85; ppl: 2.71; xent: 1.00; lr: 0.00010; 1907/2675 tok/s; 379 sec
  134. [2019-02-04 17:56:55,015 INFO] Step 3700/102583; acc: 71.95; ppl: 2.67; xent: 0.98; lr: 0.00010; 2076/2465 tok/s; 384 sec
  135. [2019-02-04 17:56:59,726 INFO] Step 3750/102583; acc: 72.81; ppl: 2.52; xent: 0.93; lr: 0.00010; 2070/2666 tok/s; 389 sec
  136. [2019-02-04 17:57:04,979 INFO] Step 3800/102583; acc: 70.89; ppl: 2.72; xent: 1.00; lr: 0.00010; 1972/2642 tok/s; 394 sec
  137. [2019-02-04 17:57:09,688 INFO] Step 3850/102583; acc: 75.06; ppl: 2.37; xent: 0.86; lr: 0.00010; 2170/2522 tok/s; 399 sec
  138. [2019-02-04 17:57:14,764 INFO] Step 3900/102583; acc: 71.22; ppl: 2.64; xent: 0.97; lr: 0.00010; 2026/2585 tok/s; 404 sec
  139. [2019-02-04 17:57:19,725 INFO] Step 3950/102583; acc: 74.64; ppl: 2.45; xent: 0.90; lr: 0.00010; 2038/2636 tok/s; 409 sec
  140. [2019-02-04 17:57:24,985 INFO] Step 4000/102583; acc: 72.62; ppl: 2.53; xent: 0.93; lr: 0.00010; 2134/2582 tok/s; 414 sec
  141. [2019-02-04 17:57:29,779 INFO] Step 4050/102583; acc: 73.47; ppl: 2.50; xent: 0.92; lr: 0.00010; 1990/2623 tok/s; 419 sec
  142. [2019-02-04 17:57:34,869 INFO] Step 4100/102583; acc: 73.40; ppl: 2.48; xent: 0.91; lr: 0.00010; 1989/2556 tok/s; 424 sec
  143. [2019-02-04 17:57:40,017 INFO] Step 4150/102583; acc: 75.85; ppl: 2.33; xent: 0.84; lr: 0.00010; 1972/2624 tok/s; 429 sec
  144. [2019-02-04 17:57:44,991 INFO] Step 4200/102583; acc: 72.46; ppl: 2.57; xent: 0.94; lr: 0.00010; 2049/2456 tok/s; 434 sec
  145. [2019-02-04 17:57:50,066 INFO] Step 4250/102583; acc: 73.46; ppl: 2.48; xent: 0.91; lr: 0.00010; 1986/2714 tok/s; 439 sec
  146. [2019-02-04 17:57:55,363 INFO] Step 4300/102583; acc: 75.00; ppl: 2.39; xent: 0.87; lr: 0.00010; 1916/2528 tok/s; 445 sec
  147. [2019-02-04 17:58:00,420 INFO] Step 4350/102583; acc: 74.95; ppl: 2.37; xent: 0.86; lr: 0.00010; 2160/2599 tok/s; 450 sec
  148. [2019-02-04 17:58:05,244 INFO] Step 4400/102583; acc: 73.52; ppl: 2.48; xent: 0.91; lr: 0.00010; 2071/2570 tok/s; 455 sec
  149. [2019-02-04 17:58:10,107 INFO] Step 4450/102583; acc: 72.24; ppl: 2.53; xent: 0.93; lr: 0.00010; 1932/2692 tok/s; 459 sec
  150. [2019-02-04 17:58:14,885 INFO] Step 4500/102583; acc: 76.38; ppl: 2.30; xent: 0.83; lr: 0.00010; 2093/2611 tok/s; 464 sec
  151. [2019-02-04 17:58:20,251 INFO] Step 4550/102583; acc: 74.49; ppl: 2.40; xent: 0.87; lr: 0.00010; 2050/2464 tok/s; 470 sec
  152. [2019-02-04 17:58:25,401 INFO] Step 4600/102583; acc: 74.40; ppl: 2.39; xent: 0.87; lr: 0.00010; 2049/2574 tok/s; 475 sec
  153. [2019-02-04 17:58:30,362 INFO] Step 4650/102583; acc: 74.07; ppl: 2.42; xent: 0.88; lr: 0.00010; 1811/2706 tok/s; 480 sec
  154. [2019-02-04 17:58:35,372 INFO] Step 4700/102583; acc: 74.73; ppl: 2.37; xent: 0.86; lr: 0.00010; 2152/2652 tok/s; 485 sec
  155. [2019-02-04 17:58:40,258 INFO] Step 4750/102583; acc: 76.38; ppl: 2.29; xent: 0.83; lr: 0.00010; 1961/2492 tok/s; 490 sec
  156. [2019-02-04 17:58:44,977 INFO] Step 4800/102583; acc: 75.65; ppl: 2.38; xent: 0.87; lr: 0.00010; 1961/2607 tok/s; 494 sec
  157. [2019-02-04 17:58:50,278 INFO] Step 4850/102583; acc: 74.95; ppl: 2.30; xent: 0.83; lr: 0.00010; 2094/2544 tok/s; 500 sec
  158. [2019-02-04 17:58:55,205 INFO] Step 4900/102583; acc: 72.58; ppl: 2.49; xent: 0.91; lr: 0.00010; 1976/2755 tok/s; 505 sec
  159. [2019-02-04 17:59:00,258 INFO] Step 4950/102583; acc: 76.83; ppl: 2.21; xent: 0.79; lr: 0.00010; 2084/2570 tok/s; 510 sec
  160. [2019-02-04 17:59:05,386 INFO] Step 5000/102583; acc: 76.24; ppl: 2.26; xent: 0.81; lr: 0.00010; 1995/2649 tok/s; 515 sec
  161. [2019-02-04 17:59:05,388 INFO] Saving checkpoint models/ted1_step_5000.pt
  162. [2019-02-04 17:59:15,964 INFO] Step 5050/102583; acc: 74.93; ppl: 2.33; xent: 0.85; lr: 0.00010; 890/1136 tok/s; 525 sec
  163. [2019-02-04 17:59:21,080 INFO] Step 5100/102583; acc: 75.06; ppl: 2.36; xent: 0.86; lr: 0.00010; 1976/2614 tok/s; 530 sec
  164. [2019-02-04 17:59:26,010 INFO] Step 5150/102583; acc: 76.50; ppl: 2.21; xent: 0.79; lr: 0.00010; 2121/2543 tok/s; 535 sec
  165. [2019-02-04 17:59:31,329 INFO] Step 5200/102583; acc: 76.07; ppl: 2.29; xent: 0.83; lr: 0.00010; 2142/2523 tok/s; 541 sec
  166. [2019-02-04 17:59:36,076 INFO] Step 5250/102583; acc: 76.86; ppl: 2.24; xent: 0.81; lr: 0.00010; 1843/2617 tok/s; 545 sec
  167. [2019-02-04 17:59:41,041 INFO] Step 5300/102583; acc: 74.85; ppl: 2.37; xent: 0.86; lr: 0.00010; 1941/2644 tok/s; 550 sec
  168. [2019-02-04 17:59:46,178 INFO] Step 5350/102583; acc: 75.97; ppl: 2.28; xent: 0.82; lr: 0.00010; 2072/2528 tok/s; 556 sec
  169. [2019-02-04 17:59:51,002 INFO] Step 5400/102583; acc: 77.61; ppl: 2.15; xent: 0.76; lr: 0.00010; 2077/2705 tok/s; 560 sec
  170. [2019-02-04 17:59:56,070 INFO] Step 5450/102583; acc: 75.00; ppl: 2.30; xent: 0.83; lr: 0.00010; 2097/2457 tok/s; 565 sec
  171. [2019-02-04 18:00:00,830 INFO] Step 5500/102583; acc: 77.01; ppl: 2.18; xent: 0.78; lr: 0.00010; 2099/2705 tok/s; 570 sec
  172. [2019-02-04 18:00:06,045 INFO] Step 5550/102583; acc: 76.62; ppl: 2.23; xent: 0.80; lr: 0.00010; 1938/2623 tok/s; 575 sec
  173. [2019-02-04 18:00:11,035 INFO] Step 5600/102583; acc: 76.24; ppl: 2.19; xent: 0.78; lr: 0.00010; 2033/2642 tok/s; 580 sec
  174. [2019-02-04 18:00:15,993 INFO] Step 5650/102583; acc: 76.74; ppl: 2.23; xent: 0.80; lr: 0.00010; 2073/2496 tok/s; 585 sec
  175. [2019-02-04 18:00:20,970 INFO] Step 5700/102583; acc: 77.77; ppl: 2.13; xent: 0.76; lr: 0.00010; 2011/2698 tok/s; 590 sec
  176. [2019-02-04 18:00:25,876 INFO] Step 5750/102583; acc: 74.28; ppl: 2.34; xent: 0.85; lr: 0.00010; 2037/2681 tok/s; 595 sec
  177. [2019-02-04 18:00:30,733 INFO] Step 5800/102583; acc: 77.20; ppl: 2.17; xent: 0.78; lr: 0.00010; 2031/2521 tok/s; 600 sec
  178. [2019-02-04 18:00:35,853 INFO] Step 5850/102583; acc: 76.95; ppl: 2.20; xent: 0.79; lr: 0.00010; 2028/2652 tok/s; 605 sec
  179. [2019-02-04 18:00:40,903 INFO] Step 5900/102583; acc: 77.43; ppl: 2.15; xent: 0.77; lr: 0.00010; 2169/2533 tok/s; 610 sec
  180. [2019-02-04 18:00:46,094 INFO] Step 5950/102583; acc: 75.64; ppl: 2.28; xent: 0.82; lr: 0.00010; 1754/2766 tok/s; 615 sec
  181. [2019-02-04 18:00:50,983 INFO] Step 6000/102583; acc: 75.06; ppl: 2.37; xent: 0.86; lr: 0.00010; 2062/2456 tok/s; 620 sec
  182. [2019-02-04 18:00:55,907 INFO] Step 6050/102583; acc: 78.94; ppl: 2.04; xent: 0.71; lr: 0.00010; 2093/2686 tok/s; 625 sec
  183. [2019-02-04 18:01:00,747 INFO] Step 6100/102583; acc: 77.20; ppl: 2.15; xent: 0.77; lr: 0.00010; 1919/2747 tok/s; 630 sec
  184. [2019-02-04 18:01:06,081 INFO] Step 6150/102583; acc: 75.65; ppl: 2.28; xent: 0.83; lr: 0.00010; 2118/2451 tok/s; 635 sec
  185. [2019-02-04 18:01:10,919 INFO] Step 6200/102583; acc: 78.35; ppl: 2.09; xent: 0.73; lr: 0.00010; 2081/2675 tok/s; 640 sec
  186. [2019-02-04 18:01:16,071 INFO] Step 6250/102583; acc: 76.69; ppl: 2.17; xent: 0.78; lr: 0.00010; 2114/2477 tok/s; 645 sec
  187. [2019-02-04 18:01:21,214 INFO] Step 6300/102583; acc: 75.80; ppl: 2.28; xent: 0.82; lr: 0.00010; 1984/2628 tok/s; 651 sec
  188. [2019-02-04 18:01:25,902 INFO] Step 6350/102583; acc: 78.64; ppl: 2.07; xent: 0.73; lr: 0.00010; 2010/2689 tok/s; 655 sec
  189. [2019-02-04 18:01:30,553 INFO] Step 6400/102583; acc: 78.21; ppl: 2.08; xent: 0.73; lr: 0.00010; 2128/2574 tok/s; 660 sec
  190. [2019-02-04 18:01:35,920 INFO] Step 6450/102583; acc: 76.03; ppl: 2.23; xent: 0.80; lr: 0.00010; 1942/2587 tok/s; 665 sec
  191. [2019-02-04 18:01:45,215 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  192. [2019-02-04 18:01:47,009 INFO] Step 6500/102583; acc: 76.37; ppl: 2.24; xent: 0.81; lr: 0.00010; 889/1160 tok/s; 676 sec
  193. [2019-02-04 18:01:52,132 INFO] Step 6550/102583; acc: 78.30; ppl: 2.11; xent: 0.75; lr: 0.00010; 2149/2532 tok/s; 682 sec
  194. [2019-02-04 18:01:56,882 INFO] Step 6600/102583; acc: 78.17; ppl: 2.10; xent: 0.74; lr: 0.00010; 1909/2708 tok/s; 686 sec
  195. [2019-02-04 18:02:02,061 INFO] Step 6650/102583; acc: 79.22; ppl: 2.02; xent: 0.70; lr: 0.00010; 2061/2631 tok/s; 691 sec
  196. [2019-02-04 18:02:06,992 INFO] Step 6700/102583; acc: 77.14; ppl: 2.17; xent: 0.78; lr: 0.00010; 2143/2653 tok/s; 696 sec
  197. [2019-02-04 18:02:12,062 INFO] Step 6750/102583; acc: 78.50; ppl: 2.07; xent: 0.73; lr: 0.00010; 2077/2654 tok/s; 701 sec
  198. [2019-02-04 18:02:16,948 INFO] Step 6800/102583; acc: 76.57; ppl: 2.18; xent: 0.78; lr: 0.00010; 1994/2527 tok/s; 706 sec
  199. [2019-02-04 18:02:21,890 INFO] Step 6850/102583; acc: 77.54; ppl: 2.11; xent: 0.75; lr: 0.00010; 2069/2768 tok/s; 711 sec
  200. [2019-02-04 18:02:26,753 INFO] Step 6900/102583; acc: 76.60; ppl: 2.15; xent: 0.77; lr: 0.00010; 1985/2689 tok/s; 716 sec
  201. [2019-02-04 18:02:31,985 INFO] Step 6950/102583; acc: 75.94; ppl: 2.27; xent: 0.82; lr: 0.00010; 1987/2507 tok/s; 721 sec
  202. [2019-02-04 18:02:36,690 INFO] Step 7000/102583; acc: 79.45; ppl: 1.98; xent: 0.68; lr: 0.00010; 2208/2641 tok/s; 726 sec
  203. [2019-02-04 18:02:42,041 INFO] Step 7050/102583; acc: 76.48; ppl: 2.20; xent: 0.79; lr: 0.00010; 1956/2590 tok/s; 731 sec
  204. [2019-02-04 18:02:46,540 INFO] Step 7100/102583; acc: 77.86; ppl: 2.13; xent: 0.76; lr: 0.00010; 2105/2599 tok/s; 736 sec
  205. [2019-02-04 18:02:51,777 INFO] Step 7150/102583; acc: 76.74; ppl: 2.19; xent: 0.78; lr: 0.00010; 1976/2661 tok/s; 741 sec
  206. [2019-02-04 18:02:56,592 INFO] Step 7200/102583; acc: 79.80; ppl: 2.01; xent: 0.70; lr: 0.00010; 2126/2585 tok/s; 746 sec
  207. [2019-02-04 18:03:01,817 INFO] Step 7250/102583; acc: 78.44; ppl: 2.05; xent: 0.72; lr: 0.00010; 2111/2605 tok/s; 751 sec
  208. [2019-02-04 18:03:06,526 INFO] Step 7300/102583; acc: 78.02; ppl: 2.11; xent: 0.75; lr: 0.00010; 2054/2577 tok/s; 756 sec
  209. [2019-02-04 18:03:11,642 INFO] Step 7350/102583; acc: 77.96; ppl: 2.07; xent: 0.73; lr: 0.00010; 1966/2592 tok/s; 761 sec
  210. [2019-02-04 18:03:16,800 INFO] Step 7400/102583; acc: 78.92; ppl: 2.04; xent: 0.71; lr: 0.00010; 2005/2631 tok/s; 766 sec
  211. [2019-02-04 18:03:21,826 INFO] Step 7450/102583; acc: 78.28; ppl: 2.10; xent: 0.74; lr: 0.00010; 2068/2421 tok/s; 771 sec
  212. [2019-02-04 18:03:26,847 INFO] Step 7500/102583; acc: 76.10; ppl: 2.18; xent: 0.78; lr: 0.00010; 1930/2767 tok/s; 776 sec
  213. [2019-02-04 18:03:32,101 INFO] Step 7550/102583; acc: 80.03; ppl: 1.98; xent: 0.68; lr: 0.00010; 1956/2500 tok/s; 781 sec
  214. [2019-02-04 18:03:37,146 INFO] Step 7600/102583; acc: 78.67; ppl: 2.06; xent: 0.72; lr: 0.00010; 2166/2635 tok/s; 787 sec
  215. [2019-02-04 18:03:41,948 INFO] Step 7650/102583; acc: 77.05; ppl: 2.16; xent: 0.77; lr: 0.00010; 2094/2558 tok/s; 791 sec
  216. [2019-02-04 18:03:46,910 INFO] Step 7700/102583; acc: 76.89; ppl: 2.16; xent: 0.77; lr: 0.00010; 1879/2678 tok/s; 796 sec
  217. [2019-02-04 18:03:51,847 INFO] Step 7750/102583; acc: 78.73; ppl: 2.06; xent: 0.73; lr: 0.00010; 2152/2686 tok/s; 801 sec
  218. [2019-02-04 18:03:56,905 INFO] Step 7800/102583; acc: 79.56; ppl: 2.01; xent: 0.70; lr: 0.00010; 2092/2400 tok/s; 806 sec
  219. [2019-02-04 18:04:02,176 INFO] Step 7850/102583; acc: 78.29; ppl: 2.06; xent: 0.72; lr: 0.00010; 1957/2648 tok/s; 812 sec
  220. [2019-02-04 18:04:06,971 INFO] Step 7900/102583; acc: 77.98; ppl: 2.08; xent: 0.73; lr: 0.00010; 1834/2663 tok/s; 816 sec
  221. [2019-02-04 18:04:12,100 INFO] Step 7950/102583; acc: 77.95; ppl: 2.09; xent: 0.74; lr: 0.00010; 2126/2644 tok/s; 821 sec
  222. [2019-02-04 18:04:16,824 INFO] Step 8000/102583; acc: 79.33; ppl: 2.04; xent: 0.72; lr: 0.00010; 2029/2504 tok/s; 826 sec
  223. [2019-02-04 18:04:21,607 INFO] Step 8050/102583; acc: 78.64; ppl: 2.08; xent: 0.73; lr: 0.00010; 1975/2649 tok/s; 831 sec
  224. [2019-02-04 18:04:26,962 INFO] Step 8100/102583; acc: 78.51; ppl: 2.01; xent: 0.70; lr: 0.00010; 2073/2593 tok/s; 836 sec
  225. [2019-02-04 18:04:31,910 INFO] Step 8150/102583; acc: 76.97; ppl: 2.14; xent: 0.76; lr: 0.00010; 2049/2640 tok/s; 841 sec
  226. [2019-02-04 18:04:36,860 INFO] Step 8200/102583; acc: 79.74; ppl: 1.99; xent: 0.69; lr: 0.00010; 2007/2659 tok/s; 846 sec
  227. [2019-02-04 18:04:41,880 INFO] Step 8250/102583; acc: 79.16; ppl: 2.02; xent: 0.70; lr: 0.00010; 2063/2613 tok/s; 851 sec
  228. [2019-02-04 18:04:46,674 INFO] Step 8300/102583; acc: 77.07; ppl: 2.14; xent: 0.76; lr: 0.00010; 1936/2536 tok/s; 856 sec
  229. [2019-02-04 18:04:51,823 INFO] Step 8350/102583; acc: 78.96; ppl: 2.01; xent: 0.70; lr: 0.00010; 2044/2674 tok/s; 861 sec
  230. [2019-02-04 18:04:56,772 INFO] Step 8400/102583; acc: 78.37; ppl: 2.06; xent: 0.72; lr: 0.00010; 2090/2555 tok/s; 866 sec
  231. [2019-02-04 18:05:02,098 INFO] Step 8450/102583; acc: 79.00; ppl: 2.02; xent: 0.70; lr: 0.00010; 2129/2458 tok/s; 871 sec
  232. [2019-02-04 18:05:06,683 INFO] Step 8500/102583; acc: 79.68; ppl: 2.00; xent: 0.69; lr: 0.00010; 1924/2724 tok/s; 876 sec
  233. [2019-02-04 18:05:11,658 INFO] Step 8550/102583; acc: 76.34; ppl: 2.21; xent: 0.79; lr: 0.00010; 1937/2624 tok/s; 881 sec
  234. [2019-02-04 18:05:16,763 INFO] Step 8600/102583; acc: 79.71; ppl: 1.99; xent: 0.69; lr: 0.00010; 1997/2574 tok/s; 886 sec
  235. [2019-02-04 18:05:21,524 INFO] Step 8650/102583; acc: 79.90; ppl: 1.95; xent: 0.67; lr: 0.00010; 2105/2677 tok/s; 891 sec
  236. [2019-02-04 18:05:26,570 INFO] Step 8700/102583; acc: 77.60; ppl: 2.11; xent: 0.75; lr: 0.00010; 2118/2462 tok/s; 896 sec
  237. [2019-02-04 18:05:31,693 INFO] Step 8750/102583; acc: 80.29; ppl: 1.91; xent: 0.65; lr: 0.00010; 2138/2590 tok/s; 901 sec
  238. [2019-02-04 18:05:36,495 INFO] Step 8800/102583; acc: 78.02; ppl: 2.08; xent: 0.73; lr: 0.00010; 1853/2780 tok/s; 906 sec
  239. [2019-02-04 18:05:41,602 INFO] Step 8850/102583; acc: 78.33; ppl: 2.04; xent: 0.71; lr: 0.00010; 2051/2655 tok/s; 911 sec
  240. [2019-02-04 18:05:46,583 INFO] Step 8900/102583; acc: 80.58; ppl: 1.93; xent: 0.66; lr: 0.00010; 2138/2522 tok/s; 916 sec
  241. [2019-02-04 18:05:51,505 INFO] Step 8950/102583; acc: 79.49; ppl: 2.00; xent: 0.69; lr: 0.00010; 2006/2618 tok/s; 921 sec
  242. [2019-02-04 18:05:56,544 INFO] Step 9000/102583; acc: 77.81; ppl: 2.05; xent: 0.72; lr: 0.00010; 2068/2652 tok/s; 926 sec
  243. [2019-02-04 18:06:01,241 INFO] Step 9050/102583; acc: 78.59; ppl: 2.06; xent: 0.72; lr: 0.00010; 2011/2618 tok/s; 931 sec
  244. [2019-02-04 18:06:06,298 INFO] Step 9100/102583; acc: 79.34; ppl: 2.00; xent: 0.69; lr: 0.00010; 2091/2615 tok/s; 936 sec
  245. [2019-02-04 18:06:11,430 INFO] Step 9150/102583; acc: 79.83; ppl: 1.95; xent: 0.67; lr: 0.00010; 2085/2544 tok/s; 941 sec
  246. [2019-02-04 18:06:16,509 INFO] Step 9200/102583; acc: 77.93; ppl: 2.09; xent: 0.74; lr: 0.00010; 1743/2830 tok/s; 946 sec
  247. [2019-02-04 18:06:21,397 INFO] Step 9250/102583; acc: 77.01; ppl: 2.19; xent: 0.78; lr: 0.00010; 2010/2444 tok/s; 951 sec
  248. [2019-02-04 18:06:26,346 INFO] Step 9300/102583; acc: 81.44; ppl: 1.86; xent: 0.62; lr: 0.00010; 2173/2724 tok/s; 956 sec
  249. [2019-02-04 18:06:31,126 INFO] Step 9350/102583; acc: 79.46; ppl: 1.99; xent: 0.69; lr: 0.00010; 2010/2670 tok/s; 961 sec
  250. [2019-02-04 18:06:36,498 INFO] Step 9400/102583; acc: 79.04; ppl: 2.03; xent: 0.71; lr: 0.00010; 2043/2523 tok/s; 966 sec
  251. [2019-02-04 18:06:41,299 INFO] Step 9450/102583; acc: 78.68; ppl: 2.04; xent: 0.71; lr: 0.00010; 2057/2678 tok/s; 971 sec
  252. [2019-02-04 18:06:46,454 INFO] Step 9500/102583; acc: 80.20; ppl: 1.94; xent: 0.66; lr: 0.00010; 2152/2466 tok/s; 976 sec
  253. [2019-02-04 18:06:51,531 INFO] Step 9550/102583; acc: 77.93; ppl: 2.11; xent: 0.74; lr: 0.00010; 2035/2593 tok/s; 981 sec
  254. [2019-02-04 18:06:56,188 INFO] Step 9600/102583; acc: 79.44; ppl: 1.99; xent: 0.69; lr: 0.00010; 1941/2698 tok/s; 986 sec
  255. [2019-02-04 18:07:01,021 INFO] Step 9650/102583; acc: 79.67; ppl: 1.96; xent: 0.67; lr: 0.00010; 2101/2565 tok/s; 990 sec
  256. [2019-02-04 18:07:06,308 INFO] Step 9700/102583; acc: 78.58; ppl: 2.01; xent: 0.70; lr: 0.00010; 1999/2607 tok/s; 996 sec
  257. [2019-02-04 18:07:15,279 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  258. [2019-02-04 18:07:17,550 INFO] Step 9750/102583; acc: 77.72; ppl: 2.11; xent: 0.75; lr: 0.00010; 861/1131 tok/s; 1007 sec
  259. [2019-02-04 18:07:22,633 INFO] Step 9800/102583; acc: 80.48; ppl: 1.96; xent: 0.67; lr: 0.00010; 2166/2508 tok/s; 1012 sec
  260. [2019-02-04 18:07:27,427 INFO] Step 9850/102583; acc: 79.88; ppl: 1.97; xent: 0.68; lr: 0.00010; 1878/2747 tok/s; 1017 sec
  261. [2019-02-04 18:07:32,638 INFO] Step 9900/102583; acc: 81.32; ppl: 1.88; xent: 0.63; lr: 0.00010; 2074/2632 tok/s; 1022 sec
  262. [2019-02-04 18:07:37,522 INFO] Step 9950/102583; acc: 78.55; ppl: 2.03; xent: 0.71; lr: 0.00010; 2098/2693 tok/s; 1027 sec
  263. [2019-02-04 18:07:42,740 INFO] Step 10000/102583; acc: 79.83; ppl: 1.94; xent: 0.66; lr: 0.00010; 2047/2677 tok/s; 1032 sec
  264. [2019-02-04 18:07:42,809 INFO] Loading dataset from data/ted.valid.0.pt, number of examples: 873
  265. [2019-02-04 18:07:44,105 INFO] Validation perplexity: 1.81916
  266. [2019-02-04 18:07:44,107 INFO] Validation accuracy: 81.297
  267. [2019-02-04 18:07:44,109 INFO] Saving checkpoint models/ted1_step_10000.pt
  268. [2019-02-04 18:07:54,789 INFO] Step 10050/102583; acc: 78.44; ppl: 2.05; xent: 0.72; lr: 0.00010; 830/1027 tok/s; 1044 sec
  269. [2019-02-04 18:07:59,516 INFO] Step 10100/102583; acc: 80.19; ppl: 1.96; xent: 0.67; lr: 0.00010; 2184/2708 tok/s; 1049 sec
  270. [2019-02-04 18:08:04,625 INFO] Step 10150/102583; acc: 77.24; ppl: 2.08; xent: 0.73; lr: 0.00010; 1972/2586 tok/s; 1054 sec
  271. [2019-02-04 18:08:09,559 INFO] Step 10200/102583; acc: 78.14; ppl: 2.09; xent: 0.74; lr: 0.00010; 1931/2627 tok/s; 1059 sec
  272. [2019-02-04 18:08:14,492 INFO] Step 10250/102583; acc: 81.04; ppl: 1.86; xent: 0.62; lr: 0.00010; 2214/2681 tok/s; 1064 sec
  273. [2019-02-04 18:08:19,598 INFO] Step 10300/102583; acc: 78.10; ppl: 2.09; xent: 0.74; lr: 0.00010; 1920/2546 tok/s; 1069 sec
  274. [2019-02-04 18:08:24,334 INFO] Step 10350/102583; acc: 79.07; ppl: 2.04; xent: 0.71; lr: 0.00010; 2117/2562 tok/s; 1074 sec
  275. [2019-02-04 18:08:29,361 INFO] Step 10400/102583; acc: 78.63; ppl: 2.02; xent: 0.70; lr: 0.00010; 2024/2735 tok/s; 1079 sec
  276. [2019-02-04 18:08:34,355 INFO] Step 10450/102583; acc: 81.81; ppl: 1.87; xent: 0.63; lr: 0.00010; 2083/2617 tok/s; 1084 sec
  277. [2019-02-04 18:08:39,439 INFO] Step 10500/102583; acc: 79.05; ppl: 2.01; xent: 0.70; lr: 0.00010; 2125/2604 tok/s; 1089 sec
  278. [2019-02-04 18:08:44,152 INFO] Step 10550/102583; acc: 80.09; ppl: 1.95; xent: 0.67; lr: 0.00010; 2156/2461 tok/s; 1094 sec
  279. [2019-02-04 18:08:49,264 INFO] Step 10600/102583; acc: 79.61; ppl: 1.96; xent: 0.67; lr: 0.00010; 1859/2722 tok/s; 1099 sec
  280. [2019-02-04 18:08:54,321 INFO] Step 10650/102583; acc: 80.22; ppl: 1.96; xent: 0.67; lr: 0.00010; 2046/2585 tok/s; 1104 sec
  281. [2019-02-04 18:08:59,575 INFO] Step 10700/102583; acc: 78.86; ppl: 2.01; xent: 0.70; lr: 0.00010; 2014/2476 tok/s; 1109 sec
  282. [2019-02-04 18:09:04,614 INFO] Step 10750/102583; acc: 77.63; ppl: 2.07; xent: 0.73; lr: 0.00010; 1897/2688 tok/s; 1114 sec
  283. [2019-02-04 18:09:09,719 INFO] Step 10800/102583; acc: 81.54; ppl: 1.87; xent: 0.62; lr: 0.00010; 1963/2620 tok/s; 1119 sec
  284. [2019-02-04 18:09:14,762 INFO] Step 10850/102583; acc: 79.41; ppl: 2.00; xent: 0.69; lr: 0.00010; 2215/2503 tok/s; 1124 sec
  285. [2019-02-04 18:09:19,394 INFO] Step 10900/102583; acc: 78.49; ppl: 2.05; xent: 0.72; lr: 0.00010; 2090/2617 tok/s; 1129 sec
  286. [2019-02-04 18:09:24,416 INFO] Step 10950/102583; acc: 79.05; ppl: 2.01; xent: 0.70; lr: 0.00010; 1997/2692 tok/s; 1134 sec
  287. [2019-02-04 18:09:29,525 INFO] Step 11000/102583; acc: 80.17; ppl: 1.95; xent: 0.67; lr: 0.00010; 2085/2638 tok/s; 1139 sec
  288. [2019-02-04 18:09:34,525 INFO] Step 11050/102583; acc: 80.87; ppl: 1.91; xent: 0.65; lr: 0.00010; 2158/2369 tok/s; 1144 sec
  289. [2019-02-04 18:09:39,601 INFO] Step 11100/102583; acc: 78.55; ppl: 2.02; xent: 0.70; lr: 0.00010; 1892/2719 tok/s; 1149 sec
  290. [2019-02-04 18:09:44,485 INFO] Step 11150/102583; acc: 79.00; ppl: 2.00; xent: 0.69; lr: 0.00010; 1827/2731 tok/s; 1154 sec
  291. [2019-02-04 18:09:49,479 INFO] Step 11200/102583; acc: 80.35; ppl: 1.91; xent: 0.65; lr: 0.00010; 2171/2577 tok/s; 1159 sec
  292. [2019-02-04 18:09:54,306 INFO] Step 11250/102583; acc: 79.64; ppl: 1.98; xent: 0.69; lr: 0.00010; 1973/2583 tok/s; 1164 sec
  293. [2019-02-04 18:09:59,388 INFO] Step 11300/102583; acc: 79.87; ppl: 1.97; xent: 0.68; lr: 0.00010; 2045/2538 tok/s; 1169 sec
  294. [2019-02-04 18:10:04,494 INFO] Step 11350/102583; acc: 78.81; ppl: 1.98; xent: 0.68; lr: 0.00010; 2029/2666 tok/s; 1174 sec
  295. [2019-02-04 18:10:09,306 INFO] Step 11400/102583; acc: 80.04; ppl: 1.94; xent: 0.66; lr: 0.00010; 2130/2660 tok/s; 1179 sec
  296. [2019-02-04 18:10:14,374 INFO] Step 11450/102583; acc: 79.93; ppl: 1.95; xent: 0.67; lr: 0.00010; 1967/2676 tok/s; 1184 sec
  297. [2019-02-04 18:10:19,284 INFO] Step 11500/102583; acc: 79.86; ppl: 1.96; xent: 0.68; lr: 0.00010; 2025/2624 tok/s; 1189 sec
  298. [2019-02-04 18:10:24,250 INFO] Step 11550/102583; acc: 78.77; ppl: 2.01; xent: 0.70; lr: 0.00010; 1967/2522 tok/s; 1194 sec
  299. [2019-02-04 18:10:29,207 INFO] Step 11600/102583; acc: 79.75; ppl: 1.94; xent: 0.66; lr: 0.00010; 2064/2695 tok/s; 1199 sec
  300. [2019-02-04 18:10:34,318 INFO] Step 11650/102583; acc: 79.29; ppl: 1.98; xent: 0.68; lr: 0.00010; 2073/2537 tok/s; 1204 sec
  301. [2019-02-04 18:10:39,496 INFO] Step 11700/102583; acc: 80.60; ppl: 1.93; xent: 0.66; lr: 0.00010; 2153/2474 tok/s; 1209 sec
  302. [2019-02-04 18:10:44,082 INFO] Step 11750/102583; acc: 79.23; ppl: 2.02; xent: 0.70; lr: 0.00010; 1826/2761 tok/s; 1213 sec
  303. [2019-02-04 18:10:49,028 INFO] Step 11800/102583; acc: 78.62; ppl: 2.01; xent: 0.70; lr: 0.00010; 2077/2597 tok/s; 1218 sec
  304. [2019-02-04 18:10:54,094 INFO] Step 11850/102583; acc: 79.93; ppl: 1.98; xent: 0.68; lr: 0.00010; 2015/2548 tok/s; 1223 sec
  305. [2019-02-04 18:10:58,906 INFO] Step 11900/102583; acc: 81.31; ppl: 1.86; xent: 0.62; lr: 0.00010; 2020/2676 tok/s; 1228 sec
  306. [2019-02-04 18:11:03,976 INFO] Step 11950/102583; acc: 78.85; ppl: 2.01; xent: 0.70; lr: 0.00010; 2193/2484 tok/s; 1233 sec
  307. [2019-02-04 18:11:08,967 INFO] Step 12000/102583; acc: 80.82; ppl: 1.87; xent: 0.63; lr: 0.00010; 2142/2579 tok/s; 1238 sec
  308. [2019-02-04 18:11:14,072 INFO] Step 12050/102583; acc: 79.48; ppl: 1.95; xent: 0.67; lr: 0.00010; 1797/2799 tok/s; 1243 sec
  309. [2019-02-04 18:11:19,144 INFO] Step 12100/102583; acc: 79.86; ppl: 1.95; xent: 0.67; lr: 0.00010; 2024/2603 tok/s; 1249 sec
  310. [2019-02-04 18:11:23,859 INFO] Step 12150/102583; acc: 80.27; ppl: 1.94; xent: 0.66; lr: 0.00010; 2175/2556 tok/s; 1253 sec
  311. [2019-02-04 18:11:28,770 INFO] Step 12200/102583; acc: 80.86; ppl: 1.91; xent: 0.65; lr: 0.00010; 2010/2633 tok/s; 1258 sec
  312. [2019-02-04 18:11:33,887 INFO] Step 12250/102583; acc: 78.54; ppl: 1.99; xent: 0.69; lr: 0.00010; 2078/2628 tok/s; 1263 sec
  313. [2019-02-04 18:11:38,602 INFO] Step 12300/102583; acc: 78.76; ppl: 2.03; xent: 0.71; lr: 0.00010; 1985/2652 tok/s; 1268 sec
  314. [2019-02-04 18:11:43,666 INFO] Step 12350/102583; acc: 81.71; ppl: 1.87; xent: 0.62; lr: 0.00010; 2089/2634 tok/s; 1273 sec
  315. [2019-02-04 18:11:48,742 INFO] Step 12400/102583; acc: 81.06; ppl: 1.88; xent: 0.63; lr: 0.00010; 2107/2557 tok/s; 1278 sec
  316. [2019-02-04 18:11:53,821 INFO] Step 12450/102583; acc: 78.90; ppl: 2.02; xent: 0.70; lr: 0.00010; 1768/2831 tok/s; 1283 sec
  317. [2019-02-04 18:11:58,701 INFO] Step 12500/102583; acc: 78.60; ppl: 2.08; xent: 0.73; lr: 0.00010; 2000/2456 tok/s; 1288 sec
  318. [2019-02-04 18:12:03,577 INFO] Step 12550/102583; acc: 81.55; ppl: 1.82; xent: 0.60; lr: 0.00010; 2180/2706 tok/s; 1293 sec
  319. [2019-02-04 18:12:08,603 INFO] Step 12600/102583; acc: 79.71; ppl: 1.95; xent: 0.67; lr: 0.00010; 1976/2510 tok/s; 1298 sec
  320. [2019-02-04 18:12:13,788 INFO] Step 12650/102583; acc: 80.65; ppl: 1.92; xent: 0.65; lr: 0.00010; 2103/2701 tok/s; 1303 sec
  321. [2019-02-04 18:12:18,843 INFO] Step 12700/102583; acc: 78.94; ppl: 2.01; xent: 0.70; lr: 0.00010; 2113/2550 tok/s; 1308 sec
  322. [2019-02-04 18:12:23,832 INFO] Step 12750/102583; acc: 80.62; ppl: 1.90; xent: 0.64; lr: 0.00010; 2028/2581 tok/s; 1313 sec
  323. [2019-02-04 18:12:28,677 INFO] Step 12800/102583; acc: 78.69; ppl: 2.04; xent: 0.71; lr: 0.00010; 2114/2550 tok/s; 1318 sec
  324. [2019-02-04 18:12:33,515 INFO] Step 12850/102583; acc: 80.16; ppl: 1.92; xent: 0.65; lr: 0.00010; 1998/2643 tok/s; 1323 sec
  325. [2019-02-04 18:12:38,432 INFO] Step 12900/102583; acc: 80.00; ppl: 1.94; xent: 0.66; lr: 0.00010; 1964/2727 tok/s; 1328 sec
  326. [2019-02-04 18:12:43,712 INFO] Step 12950/102583; acc: 79.73; ppl: 1.93; xent: 0.66; lr: 0.00010; 2003/2572 tok/s; 1333 sec
  327. [2019-02-04 18:12:52,119 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  328. [2019-02-04 18:12:55,158 INFO] Step 13000/102583; acc: 80.04; ppl: 1.95; xent: 0.67; lr: 0.00010; 889/1130 tok/s; 1345 sec
  329. [2019-02-04 18:13:00,091 INFO] Step 13050/102583; acc: 80.08; ppl: 1.99; xent: 0.69; lr: 0.00010; 2168/2422 tok/s; 1349 sec
  330. [2019-02-04 18:13:04,937 INFO] Step 13100/102583; acc: 80.39; ppl: 1.90; xent: 0.64; lr: 0.00010; 1938/2801 tok/s; 1354 sec
  331. [2019-02-04 18:13:10,138 INFO] Step 13150/102583; acc: 81.24; ppl: 1.87; xent: 0.63; lr: 0.00010; 1990/2636 tok/s; 1360 sec
  332. [2019-02-04 18:13:14,973 INFO] Step 13200/102583; acc: 79.78; ppl: 1.98; xent: 0.68; lr: 0.00010; 2153/2741 tok/s; 1364 sec
  333. [2019-02-04 18:13:20,073 INFO] Step 13250/102583; acc: 80.30; ppl: 1.88; xent: 0.63; lr: 0.00010; 2019/2657 tok/s; 1369 sec
  334. [2019-02-04 18:13:24,939 INFO] Step 13300/102583; acc: 79.26; ppl: 1.97; xent: 0.68; lr: 0.00010; 2175/2501 tok/s; 1374 sec
  335. [2019-02-04 18:13:29,652 INFO] Step 13350/102583; acc: 80.61; ppl: 1.92; xent: 0.65; lr: 0.00010; 2096/2729 tok/s; 1379 sec
  336. [2019-02-04 18:13:34,884 INFO] Step 13400/102583; acc: 78.31; ppl: 2.02; xent: 0.70; lr: 0.00010; 1944/2586 tok/s; 1384 sec
  337. [2019-02-04 18:13:39,839 INFO] Step 13450/102583; acc: 79.79; ppl: 1.98; xent: 0.68; lr: 0.00010; 1999/2624 tok/s; 1389 sec
  338. [2019-02-04 18:13:44,677 INFO] Step 13500/102583; acc: 81.27; ppl: 1.81; xent: 0.60; lr: 0.00010; 2160/2744 tok/s; 1394 sec
  339. [2019-02-04 18:13:49,906 INFO] Step 13550/102583; acc: 78.08; ppl: 2.07; xent: 0.73; lr: 0.00010; 1947/2486 tok/s; 1399 sec
  340. [2019-02-04 18:13:54,612 INFO] Step 13600/102583; acc: 80.42; ppl: 1.95; xent: 0.67; lr: 0.00010; 2180/2562 tok/s; 1404 sec
  341. [2019-02-04 18:13:59,597 INFO] Step 13650/102583; acc: 79.25; ppl: 1.95; xent: 0.67; lr: 0.00010; 1939/2746 tok/s; 1409 sec
  342. [2019-02-04 18:14:04,683 INFO] Step 13700/102583; acc: 81.73; ppl: 1.85; xent: 0.62; lr: 0.00010; 2171/2555 tok/s; 1414 sec
  343. [2019-02-04 18:14:09,677 INFO] Step 13750/102583; acc: 79.20; ppl: 1.98; xent: 0.68; lr: 0.00010; 2038/2663 tok/s; 1419 sec
  344. [2019-02-04 18:14:14,381 INFO] Step 13800/102583; acc: 80.81; ppl: 1.92; xent: 0.65; lr: 0.00010; 2208/2481 tok/s; 1424 sec
  345. [2019-02-04 18:14:19,499 INFO] Step 13850/102583; acc: 80.34; ppl: 1.86; xent: 0.62; lr: 0.00010; 1853/2760 tok/s; 1429 sec
  346. [2019-02-04 18:14:24,679 INFO] Step 13900/102583; acc: 79.71; ppl: 1.96; xent: 0.67; lr: 0.00010; 2029/2461 tok/s; 1434 sec
  347. [2019-02-04 18:14:29,733 INFO] Step 13950/102583; acc: 80.76; ppl: 1.90; xent: 0.64; lr: 0.00010; 2099/2578 tok/s; 1439 sec
  348. [2019-02-04 18:14:34,773 INFO] Step 14000/102583; acc: 77.56; ppl: 2.07; xent: 0.73; lr: 0.00010; 1820/2704 tok/s; 1444 sec
  349. [2019-02-04 18:14:39,900 INFO] Step 14050/102583; acc: 82.89; ppl: 1.79; xent: 0.58; lr: 0.00010; 2017/2608 tok/s; 1449 sec
  350. [2019-02-04 18:14:44,831 INFO] Step 14100/102583; acc: 78.72; ppl: 2.01; xent: 0.70; lr: 0.00010; 2123/2495 tok/s; 1454 sec
  351. [2019-02-04 18:14:49,436 INFO] Step 14150/102583; acc: 78.63; ppl: 2.02; xent: 0.70; lr: 0.00010; 2172/2612 tok/s; 1459 sec
  352. [2019-02-04 18:14:54,521 INFO] Step 14200/102583; acc: 79.74; ppl: 1.94; xent: 0.66; lr: 0.00010; 1985/2732 tok/s; 1464 sec
  353. [2019-02-04 18:14:59,662 INFO] Step 14250/102583; acc: 81.50; ppl: 1.89; xent: 0.64; lr: 0.00010; 2060/2659 tok/s; 1469 sec
  354. [2019-02-04 18:15:04,845 INFO] Step 14300/102583; acc: 81.28; ppl: 1.86; xent: 0.62; lr: 0.00010; 2158/2321 tok/s; 1474 sec
  355. [2019-02-04 18:15:09,761 INFO] Step 14350/102583; acc: 78.44; ppl: 1.99; xent: 0.69; lr: 0.00010; 1846/2819 tok/s; 1479 sec
  356. [2019-02-04 18:15:14,618 INFO] Step 14400/102583; acc: 79.09; ppl: 1.97; xent: 0.68; lr: 0.00010; 1850/2706 tok/s; 1484 sec
  357. [2019-02-04 18:15:19,653 INFO] Step 14450/102583; acc: 82.39; ppl: 1.81; xent: 0.59; lr: 0.00010; 2253/2503 tok/s; 1489 sec
  358. [2019-02-04 18:15:24,400 INFO] Step 14500/102583; acc: 79.92; ppl: 1.98; xent: 0.68; lr: 0.00010; 1943/2638 tok/s; 1494 sec
  359. [2019-02-04 18:15:29,517 INFO] Step 14550/102583; acc: 79.75; ppl: 1.93; xent: 0.66; lr: 0.00010; 2074/2503 tok/s; 1499 sec
  360. [2019-02-04 18:15:34,663 INFO] Step 14600/102583; acc: 80.37; ppl: 1.90; xent: 0.64; lr: 0.00010; 2045/2746 tok/s; 1504 sec
  361. [2019-02-04 18:15:39,415 INFO] Step 14650/102583; acc: 79.16; ppl: 1.98; xent: 0.68; lr: 0.00010; 2001/2586 tok/s; 1509 sec
  362. [2019-02-04 18:15:44,449 INFO] Step 14700/102583; acc: 80.43; ppl: 1.90; xent: 0.64; lr: 0.00010; 1988/2734 tok/s; 1514 sec
  363. [2019-02-04 18:15:49,500 INFO] Step 14750/102583; acc: 81.52; ppl: 1.86; xent: 0.62; lr: 0.00010; 2108/2586 tok/s; 1519 sec
  364. [2019-02-04 18:15:54,491 INFO] Step 14800/102583; acc: 78.64; ppl: 2.02; xent: 0.70; lr: 0.00010; 1891/2505 tok/s; 1524 sec
  365. [2019-02-04 18:15:59,275 INFO] Step 14850/102583; acc: 80.66; ppl: 1.88; xent: 0.63; lr: 0.00010; 2167/2744 tok/s; 1529 sec
  366. [2019-02-04 18:16:04,334 INFO] Step 14900/102583; acc: 79.52; ppl: 1.97; xent: 0.68; lr: 0.00010; 1981/2516 tok/s; 1534 sec
  367. [2019-02-04 18:16:09,474 INFO] Step 14950/102583; acc: 81.86; ppl: 1.85; xent: 0.62; lr: 0.00010; 2206/2485 tok/s; 1539 sec
  368. [2019-02-04 18:16:14,203 INFO] Step 15000/102583; acc: 78.91; ppl: 2.03; xent: 0.71; lr: 0.00010; 1885/2730 tok/s; 1544 sec
  369. [2019-02-04 18:16:14,205 INFO] Saving checkpoint models/ted1_step_15000.pt
  370. [2019-02-04 18:16:25,284 INFO] Step 15050/102583; acc: 79.62; ppl: 1.94; xent: 0.66; lr: 0.00010; 924/1203 tok/s; 1555 sec
  371. [2019-02-04 18:16:30,152 INFO] Step 15100/102583; acc: 80.14; ppl: 1.95; xent: 0.67; lr: 0.00010; 2030/2573 tok/s; 1560 sec
  372. [2019-02-04 18:16:35,036 INFO] Step 15150/102583; acc: 81.99; ppl: 1.81; xent: 0.59; lr: 0.00010; 2059/2625 tok/s; 1564 sec
  373. [2019-02-04 18:16:40,025 INFO] Step 15200/102583; acc: 79.30; ppl: 2.00; xent: 0.69; lr: 0.00010; 2156/2504 tok/s; 1569 sec
  374. [2019-02-04 18:16:44,956 INFO] Step 15250/102583; acc: 81.92; ppl: 1.81; xent: 0.59; lr: 0.00010; 2173/2590 tok/s; 1574 sec
  375. [2019-02-04 18:16:50,049 INFO] Step 15300/102583; acc: 79.41; ppl: 1.95; xent: 0.67; lr: 0.00010; 1789/2823 tok/s; 1579 sec
  376. [2019-02-04 18:16:55,179 INFO] Step 15350/102583; acc: 80.66; ppl: 1.90; xent: 0.64; lr: 0.00010; 2064/2559 tok/s; 1585 sec
  377. [2019-02-04 18:16:59,878 INFO] Step 15400/102583; acc: 81.69; ppl: 1.85; xent: 0.61; lr: 0.00010; 2118/2600 tok/s; 1589 sec
  378. [2019-02-04 18:17:05,003 INFO] Step 15450/102583; acc: 80.81; ppl: 1.88; xent: 0.63; lr: 0.00010; 2035/2633 tok/s; 1594 sec
  379. [2019-02-04 18:17:09,834 INFO] Step 15500/102583; acc: 78.04; ppl: 2.02; xent: 0.70; lr: 0.00010; 2070/2633 tok/s; 1599 sec
  380. [2019-02-04 18:17:14,739 INFO] Step 15550/102583; acc: 79.99; ppl: 1.97; xent: 0.68; lr: 0.00010; 1906/2663 tok/s; 1604 sec
  381. [2019-02-04 18:17:19,569 INFO] Step 15600/102583; acc: 82.39; ppl: 1.82; xent: 0.60; lr: 0.00010; 2196/2622 tok/s; 1609 sec
  382. [2019-02-04 18:17:24,867 INFO] Step 15650/102583; acc: 80.32; ppl: 1.87; xent: 0.63; lr: 0.00010; 2036/2567 tok/s; 1614 sec
  383. [2019-02-04 18:17:30,088 INFO] Step 15700/102583; acc: 79.10; ppl: 2.01; xent: 0.70; lr: 0.00010; 1827/2745 tok/s; 1619 sec
  384. [2019-02-04 18:17:34,602 INFO] Step 15750/102583; acc: 78.83; ppl: 2.02; xent: 0.70; lr: 0.00010; 2074/2532 tok/s; 1624 sec
  385. [2019-02-04 18:17:39,701 INFO] Step 15800/102583; acc: 81.83; ppl: 1.80; xent: 0.59; lr: 0.00010; 2073/2751 tok/s; 1629 sec
  386. [2019-02-04 18:17:44,716 INFO] Step 15850/102583; acc: 81.16; ppl: 1.89; xent: 0.64; lr: 0.00010; 1991/2493 tok/s; 1634 sec
  387. [2019-02-04 18:17:49,708 INFO] Step 15900/102583; acc: 79.79; ppl: 1.92; xent: 0.65; lr: 0.00010; 2148/2704 tok/s; 1639 sec
  388. [2019-02-04 18:17:54,750 INFO] Step 15950/102583; acc: 80.60; ppl: 1.91; xent: 0.65; lr: 0.00010; 2131/2558 tok/s; 1644 sec
  389. [2019-02-04 18:18:00,059 INFO] Step 16000/102583; acc: 80.75; ppl: 1.89; xent: 0.64; lr: 0.00010; 2053/2509 tok/s; 1649 sec
  390. [2019-02-04 18:18:04,573 INFO] Step 16050/102583; acc: 79.19; ppl: 2.00; xent: 0.69; lr: 0.00010; 2039/2645 tok/s; 1654 sec
  391. [2019-02-04 18:18:09,374 INFO] Step 16100/102583; acc: 80.73; ppl: 1.86; xent: 0.62; lr: 0.00010; 2093/2603 tok/s; 1659 sec
  392. [2019-02-04 18:18:14,414 INFO] Step 16150/102583; acc: 80.85; ppl: 1.90; xent: 0.64; lr: 0.00010; 1964/2662 tok/s; 1664 sec
  393. [2019-02-04 18:18:19,712 INFO] Step 16200/102583; acc: 79.50; ppl: 1.92; xent: 0.65; lr: 0.00010; 1907/2569 tok/s; 1669 sec
  394. [2019-02-04 18:18:27,016 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  395. [2019-02-04 18:18:30,858 INFO] Step 16250/102583; acc: 81.18; ppl: 1.89; xent: 0.64; lr: 0.00010; 969/1197 tok/s; 1680 sec
  396. [2019-02-04 18:18:35,462 INFO] Step 16300/102583; acc: 79.66; ppl: 2.02; xent: 0.70; lr: 0.00010; 2141/2463 tok/s; 1685 sec
  397. [2019-02-04 18:18:40,295 INFO] Step 16350/102583; acc: 81.39; ppl: 1.85; xent: 0.61; lr: 0.00010; 1965/2797 tok/s; 1690 sec
  398. [2019-02-04 18:18:45,598 INFO] Step 16400/102583; acc: 81.15; ppl: 1.87; xent: 0.63; lr: 0.00010; 2017/2613 tok/s; 1695 sec
  399. [2019-02-04 18:18:50,389 INFO] Step 16450/102583; acc: 81.13; ppl: 1.89; xent: 0.64; lr: 0.00010; 2102/2820 tok/s; 1700 sec
  400. [2019-02-04 18:18:55,377 INFO] Step 16500/102583; acc: 79.87; ppl: 1.91; xent: 0.65; lr: 0.00010; 2001/2597 tok/s; 1705 sec
  401. [2019-02-04 18:19:00,428 INFO] Step 16550/102583; acc: 80.56; ppl: 1.88; xent: 0.63; lr: 0.00010; 2218/2521 tok/s; 1710 sec
  402. [2019-02-04 18:19:05,034 INFO] Step 16600/102583; acc: 79.51; ppl: 1.97; xent: 0.68; lr: 0.00010; 2065/2757 tok/s; 1714 sec
  403. [2019-02-04 18:19:10,281 INFO] Step 16650/102583; acc: 79.70; ppl: 1.94; xent: 0.66; lr: 0.00010; 1902/2602 tok/s; 1720 sec
  404. [2019-02-04 18:19:15,341 INFO] Step 16700/102583; acc: 79.99; ppl: 1.95; xent: 0.67; lr: 0.00010; 2158/2563 tok/s; 1725 sec
  405. [2019-02-04 18:19:20,190 INFO] Step 16750/102583; acc: 81.48; ppl: 1.80; xent: 0.59; lr: 0.00010; 2029/2802 tok/s; 1730 sec
  406. [2019-02-04 18:19:25,237 INFO] Step 16800/102583; acc: 78.12; ppl: 2.08; xent: 0.73; lr: 0.00010; 2039/2430 tok/s; 1735 sec
  407. [2019-02-04 18:19:30,181 INFO] Step 16850/102583; acc: 81.89; ppl: 1.86; xent: 0.62; lr: 0.00010; 2066/2628 tok/s; 1740 sec
  408. [2019-02-04 18:19:35,019 INFO] Step 16900/102583; acc: 79.79; ppl: 1.93; xent: 0.66; lr: 0.00010; 1994/2758 tok/s; 1744 sec
  409. [2019-02-04 18:19:39,989 INFO] Step 16950/102583; acc: 82.10; ppl: 1.83; xent: 0.60; lr: 0.00010; 2208/2509 tok/s; 1749 sec
  410. [2019-02-04 18:19:45,236 INFO] Step 17000/102583; acc: 79.32; ppl: 1.95; xent: 0.67; lr: 0.00010; 1956/2689 tok/s; 1755 sec
  411. [2019-02-04 18:19:49,747 INFO] Step 17050/102583; acc: 80.81; ppl: 1.93; xent: 0.66; lr: 0.00010; 2213/2483 tok/s; 1759 sec
  412. [2019-02-04 18:19:54,753 INFO] Step 17100/102583; acc: 81.62; ppl: 1.82; xent: 0.60; lr: 0.00010; 1882/2728 tok/s; 1764 sec
  413. [2019-02-04 18:20:00,089 INFO] Step 17150/102583; acc: 79.95; ppl: 1.93; xent: 0.66; lr: 0.00010; 1990/2473 tok/s; 1769 sec
  414. [2019-02-04 18:20:05,100 INFO] Step 17200/102583; acc: 80.46; ppl: 1.89; xent: 0.64; lr: 0.00010; 2161/2588 tok/s; 1774 sec
  415. [2019-02-04 18:20:10,444 INFO] Step 17250/102583; acc: 79.57; ppl: 1.94; xent: 0.66; lr: 0.00010; 1853/2689 tok/s; 1780 sec
  416. [2019-02-04 18:20:15,234 INFO] Step 17300/102583; acc: 81.48; ppl: 1.86; xent: 0.62; lr: 0.00010; 2047/2630 tok/s; 1785 sec
  417. [2019-02-04 18:20:20,272 INFO] Step 17350/102583; acc: 78.42; ppl: 2.00; xent: 0.69; lr: 0.00010; 2041/2575 tok/s; 1790 sec
  418. [2019-02-04 18:20:24,920 INFO] Step 17400/102583; acc: 80.70; ppl: 1.92; xent: 0.65; lr: 0.00010; 2242/2531 tok/s; 1794 sec
  419. [2019-02-04 18:20:29,887 INFO] Step 17450/102583; acc: 79.03; ppl: 1.97; xent: 0.68; lr: 0.00010; 1960/2755 tok/s; 1799 sec
  420. [2019-02-04 18:20:35,007 INFO] Step 17500/102583; acc: 82.11; ppl: 1.83; xent: 0.60; lr: 0.00010; 2056/2654 tok/s; 1804 sec
  421. [2019-02-04 18:20:40,300 INFO] Step 17550/102583; acc: 81.15; ppl: 1.87; xent: 0.62; lr: 0.00010; 2080/2311 tok/s; 1810 sec
  422. [2019-02-04 18:20:45,130 INFO] Step 17600/102583; acc: 79.41; ppl: 1.92; xent: 0.65; lr: 0.00010; 1942/2851 tok/s; 1815 sec
  423. [2019-02-04 18:20:49,875 INFO] Step 17650/102583; acc: 79.42; ppl: 1.95; xent: 0.67; lr: 0.00010; 1867/2675 tok/s; 1819 sec
  424. [2019-02-04 18:20:54,929 INFO] Step 17700/102583; acc: 82.26; ppl: 1.79; xent: 0.58; lr: 0.00010; 2243/2522 tok/s; 1824 sec
  425. [2019-02-04 18:20:59,748 INFO] Step 17750/102583; acc: 78.59; ppl: 2.02; xent: 0.71; lr: 0.00010; 1833/2676 tok/s; 1829 sec
  426. [2019-02-04 18:21:04,960 INFO] Step 17800/102583; acc: 81.61; ppl: 1.81; xent: 0.59; lr: 0.00010; 2159/2533 tok/s; 1834 sec
  427. [2019-02-04 18:21:09,967 INFO] Step 17850/102583; acc: 80.07; ppl: 1.93; xent: 0.66; lr: 0.00010; 2026/2706 tok/s; 1839 sec
  428. [2019-02-04 18:21:14,834 INFO] Step 17900/102583; acc: 80.24; ppl: 1.91; xent: 0.64; lr: 0.00010; 2047/2594 tok/s; 1844 sec
  429. [2019-02-04 18:21:19,703 INFO] Step 17950/102583; acc: 80.63; ppl: 1.90; xent: 0.64; lr: 0.00010; 1962/2701 tok/s; 1849 sec
  430. [2019-02-04 18:21:24,771 INFO] Step 18000/102583; acc: 81.97; ppl: 1.83; xent: 0.60; lr: 0.00010; 2202/2566 tok/s; 1854 sec
  431. [2019-02-04 18:21:29,751 INFO] Step 18050/102583; acc: 77.89; ppl: 2.07; xent: 0.73; lr: 0.00010; 1818/2528 tok/s; 1859 sec
  432. [2019-02-04 18:21:34,672 INFO] Step 18100/102583; acc: 81.71; ppl: 1.82; xent: 0.60; lr: 0.00010; 2239/2649 tok/s; 1864 sec
  433. [2019-02-04 18:21:39,587 INFO] Step 18150/102583; acc: 79.36; ppl: 1.97; xent: 0.68; lr: 0.00010; 1892/2609 tok/s; 1869 sec
  434. [2019-02-04 18:21:44,950 INFO] Step 18200/102583; acc: 82.31; ppl: 1.81; xent: 0.59; lr: 0.00010; 2174/2494 tok/s; 1874 sec
  435. [2019-02-04 18:21:49,819 INFO] Step 18250/102583; acc: 79.51; ppl: 1.99; xent: 0.69; lr: 0.00010; 1930/2696 tok/s; 1879 sec
  436. [2019-02-04 18:21:54,959 INFO] Step 18300/102583; acc: 81.24; ppl: 1.84; xent: 0.61; lr: 0.00010; 2003/2660 tok/s; 1884 sec
  437. [2019-02-04 18:21:59,686 INFO] Step 18350/102583; acc: 79.55; ppl: 2.01; xent: 0.70; lr: 0.00010; 2054/2603 tok/s; 1889 sec
  438. [2019-02-04 18:22:04,840 INFO] Step 18400/102583; acc: 82.62; ppl: 1.77; xent: 0.57; lr: 0.00010; 1999/2545 tok/s; 1894 sec
  439. [2019-02-04 18:22:09,539 INFO] Step 18450/102583; acc: 79.15; ppl: 1.99; xent: 0.69; lr: 0.00010; 2209/2579 tok/s; 1899 sec
  440. [2019-02-04 18:22:14,640 INFO] Step 18500/102583; acc: 81.77; ppl: 1.81; xent: 0.59; lr: 0.00010; 2105/2585 tok/s; 1904 sec
  441. [2019-02-04 18:22:19,801 INFO] Step 18550/102583; acc: 80.53; ppl: 1.89; xent: 0.63; lr: 0.00010; 1873/2783 tok/s; 1909 sec
  442. [2019-02-04 18:22:24,742 INFO] Step 18600/102583; acc: 80.60; ppl: 1.91; xent: 0.65; lr: 0.00010; 2085/2590 tok/s; 1914 sec
  443. [2019-02-04 18:22:29,491 INFO] Step 18650/102583; acc: 82.46; ppl: 1.80; xent: 0.59; lr: 0.00010; 2157/2567 tok/s; 1919 sec
  444. [2019-02-04 18:22:34,555 INFO] Step 18700/102583; acc: 80.07; ppl: 1.91; xent: 0.65; lr: 0.00010; 1977/2675 tok/s; 1924 sec
  445. [2019-02-04 18:22:39,314 INFO] Step 18750/102583; acc: 78.72; ppl: 1.97; xent: 0.68; lr: 0.00010; 2087/2633 tok/s; 1929 sec
  446. [2019-02-04 18:22:44,362 INFO] Step 18800/102583; acc: 81.24; ppl: 1.87; xent: 0.63; lr: 0.00010; 1968/2651 tok/s; 1934 sec
  447. [2019-02-04 18:22:49,042 INFO] Step 18850/102583; acc: 81.55; ppl: 1.86; xent: 0.62; lr: 0.00010; 2137/2648 tok/s; 1938 sec
  448. [2019-02-04 18:22:54,363 INFO] Step 18900/102583; acc: 81.08; ppl: 1.86; xent: 0.62; lr: 0.00010; 2028/2574 tok/s; 1944 sec
  449. [2019-02-04 18:22:59,528 INFO] Step 18950/102583; acc: 79.29; ppl: 2.00; xent: 0.69; lr: 0.00010; 1872/2696 tok/s; 1949 sec
  450. [2019-02-04 18:23:04,343 INFO] Step 19000/102583; acc: 80.22; ppl: 1.94; xent: 0.66; lr: 0.00010; 2102/2517 tok/s; 1954 sec
  451. [2019-02-04 18:23:09,123 INFO] Step 19050/102583; acc: 81.98; ppl: 1.79; xent: 0.58; lr: 0.00010; 1986/2788 tok/s; 1959 sec
  452. [2019-02-04 18:23:14,426 INFO] Step 19100/102583; acc: 81.32; ppl: 1.86; xent: 0.62; lr: 0.00010; 2043/2521 tok/s; 1964 sec
  453. [2019-02-04 18:23:19,215 INFO] Step 19150/102583; acc: 80.16; ppl: 1.90; xent: 0.64; lr: 0.00010; 2074/2734 tok/s; 1969 sec
  454. [2019-02-04 18:23:24,202 INFO] Step 19200/102583; acc: 80.41; ppl: 1.93; xent: 0.66; lr: 0.00010; 2167/2530 tok/s; 1974 sec
  455. [2019-02-04 18:23:29,658 INFO] Step 19250/102583; acc: 81.81; ppl: 1.82; xent: 0.60; lr: 0.00010; 2058/2514 tok/s; 1979 sec
  456. [2019-02-04 18:23:34,082 INFO] Step 19300/102583; acc: 79.45; ppl: 1.98; xent: 0.69; lr: 0.00010; 2064/2652 tok/s; 1983 sec
  457. [2019-02-04 18:23:38,859 INFO] Step 19350/102583; acc: 80.79; ppl: 1.86; xent: 0.62; lr: 0.00010; 2010/2624 tok/s; 1988 sec
  458. [2019-02-04 18:23:44,218 INFO] Step 19400/102583; acc: 81.01; ppl: 1.88; xent: 0.63; lr: 0.00010; 1992/2564 tok/s; 1994 sec
  459. [2019-02-04 18:23:49,353 INFO] Step 19450/102583; acc: 79.99; ppl: 1.90; xent: 0.64; lr: 0.00010; 1928/2619 tok/s; 1999 sec
  460. [2019-02-04 18:23:56,209 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  461. [2019-02-04 18:24:00,649 INFO] Step 19500/102583; acc: 81.60; ppl: 1.86; xent: 0.62; lr: 0.00010; 956/1154 tok/s; 2010 sec
  462. [2019-02-04 18:24:05,227 INFO] Step 19550/102583; acc: 79.63; ppl: 2.01; xent: 0.70; lr: 0.00010; 2085/2546 tok/s; 2015 sec
  463. [2019-02-04 18:24:10,236 INFO] Step 19600/102583; acc: 82.19; ppl: 1.78; xent: 0.58; lr: 0.00010; 2031/2796 tok/s; 2020 sec
  464. [2019-02-04 18:24:15,419 INFO] Step 19650/102583; acc: 80.31; ppl: 1.91; xent: 0.65; lr: 0.00010; 1960/2616 tok/s; 2025 sec
  465. [2019-02-04 18:24:20,141 INFO] Step 19700/102583; acc: 81.40; ppl: 1.86; xent: 0.62; lr: 0.00010; 2090/2813 tok/s; 2030 sec
  466. [2019-02-04 18:24:25,276 INFO] Step 19750/102583; acc: 80.39; ppl: 1.87; xent: 0.63; lr: 0.00010; 2112/2496 tok/s; 2035 sec
  467. [2019-02-04 18:24:30,189 INFO] Step 19800/102583; acc: 81.19; ppl: 1.85; xent: 0.61; lr: 0.00010; 2181/2631 tok/s; 2040 sec
  468. [2019-02-04 18:24:34,978 INFO] Step 19850/102583; acc: 79.81; ppl: 1.95; xent: 0.67; lr: 0.00010; 2111/2638 tok/s; 2044 sec
  469. [2019-02-04 18:24:40,130 INFO] Step 19900/102583; acc: 79.18; ppl: 1.97; xent: 0.68; lr: 0.00010; 1775/2654 tok/s; 2050 sec
  470. [2019-02-04 18:24:45,074 INFO] Step 19950/102583; acc: 80.89; ppl: 1.89; xent: 0.63; lr: 0.00010; 2194/2635 tok/s; 2054 sec
  471. [2019-02-04 18:24:50,211 INFO] Step 20000/102583; acc: 81.45; ppl: 1.82; xent: 0.60; lr: 0.00010; 1888/2749 tok/s; 2060 sec
  472. [2019-02-04 18:24:50,238 INFO] Loading dataset from data/ted.valid.0.pt, number of examples: 873
  473. [2019-02-04 18:24:51,538 INFO] Validation perplexity: 1.76312
  474. [2019-02-04 18:24:51,539 INFO] Validation accuracy: 82.8277
  475. [2019-02-04 18:24:51,540 INFO] Saving checkpoint models/ted1_step_20000.pt
  476. [2019-02-04 18:25:02,098 INFO] Step 20050/102583; acc: 79.34; ppl: 2.00; xent: 0.69; lr: 0.00010; 915/979 tok/s; 2071 sec
  477. [2019-02-04 18:25:06,963 INFO] Step 20100/102583; acc: 81.28; ppl: 1.88; xent: 0.63; lr: 0.00010; 2047/2636 tok/s; 2076 sec
  478. [2019-02-04 18:25:11,902 INFO] Step 20150/102583; acc: 79.36; ppl: 1.95; xent: 0.67; lr: 0.00010; 1910/2684 tok/s; 2081 sec
  479. [2019-02-04 18:25:17,007 INFO] Step 20200/102583; acc: 82.93; ppl: 1.77; xent: 0.57; lr: 0.00010; 2277/2567 tok/s; 2086 sec
  480. [2019-02-04 18:25:22,037 INFO] Step 20250/102583; acc: 79.31; ppl: 1.95; xent: 0.67; lr: 0.00010; 1928/2730 tok/s; 2091 sec
  481. [2019-02-04 18:25:26,598 INFO] Step 20300/102583; acc: 81.26; ppl: 1.90; xent: 0.64; lr: 0.00010; 2132/2524 tok/s; 2096 sec
  482. [2019-02-04 18:25:31,740 INFO] Step 20350/102583; acc: 81.49; ppl: 1.80; xent: 0.59; lr: 0.00010; 1943/2683 tok/s; 2101 sec
  483. [2019-02-04 18:25:36,845 INFO] Step 20400/102583; acc: 80.53; ppl: 1.92; xent: 0.65; lr: 0.00010; 2006/2466 tok/s; 2106 sec
  484. [2019-02-04 18:25:41,904 INFO] Step 20450/102583; acc: 80.73; ppl: 1.88; xent: 0.63; lr: 0.00010; 2101/2609 tok/s; 2111 sec
  485. [2019-02-04 18:25:47,247 INFO] Step 20500/102583; acc: 80.08; ppl: 1.90; xent: 0.64; lr: 0.00010; 1889/2699 tok/s; 2117 sec
  486. [2019-02-04 18:25:51,958 INFO] Step 20550/102583; acc: 81.86; ppl: 1.84; xent: 0.61; lr: 0.00010; 2081/2581 tok/s; 2121 sec
  487. [2019-02-04 18:25:56,967 INFO] Step 20600/102583; acc: 79.00; ppl: 1.98; xent: 0.69; lr: 0.00010; 2141/2537 tok/s; 2126 sec
  488. [2019-02-04 18:26:01,824 INFO] Step 20650/102583; acc: 80.36; ppl: 1.91; xent: 0.65; lr: 0.00010; 2027/2715 tok/s; 2131 sec
  489. [2019-02-04 18:26:06,577 INFO] Step 20700/102583; acc: 79.34; ppl: 1.97; xent: 0.68; lr: 0.00010; 2062/2617 tok/s; 2136 sec
  490. [2019-02-04 18:26:11,898 INFO] Step 20750/102583; acc: 81.72; ppl: 1.83; xent: 0.61; lr: 0.00010; 1926/2584 tok/s; 2141 sec
  491. [2019-02-04 18:26:17,038 INFO] Step 20800/102583; acc: 82.16; ppl: 1.81; xent: 0.60; lr: 0.00010; 2208/2380 tok/s; 2146 sec
  492. [2019-02-04 18:26:21,816 INFO] Step 20850/102583; acc: 79.41; ppl: 1.92; xent: 0.65; lr: 0.00010; 1896/2875 tok/s; 2151 sec
  493. [2019-02-04 18:26:26,712 INFO] Step 20900/102583; acc: 79.85; ppl: 1.90; xent: 0.64; lr: 0.00010; 1979/2702 tok/s; 2156 sec
  494. [2019-02-04 18:26:31,787 INFO] Step 20950/102583; acc: 82.37; ppl: 1.81; xent: 0.60; lr: 0.00010; 2181/2444 tok/s; 2161 sec
  495. [2019-02-04 18:26:36,497 INFO] Step 21000/102583; acc: 79.73; ppl: 1.96; xent: 0.67; lr: 0.00010; 1832/2685 tok/s; 2166 sec
  496. [2019-02-04 18:26:41,712 INFO] Step 21050/102583; acc: 81.03; ppl: 1.81; xent: 0.59; lr: 0.00010; 2151/2601 tok/s; 2171 sec
  497. [2019-02-04 18:26:46,647 INFO] Step 21100/102583; acc: 80.19; ppl: 1.93; xent: 0.66; lr: 0.00010; 2056/2673 tok/s; 2176 sec
  498. [2019-02-04 18:26:51,528 INFO] Step 21150/102583; acc: 80.33; ppl: 1.90; xent: 0.64; lr: 0.00010; 2016/2590 tok/s; 2181 sec
  499. [2019-02-04 18:26:56,593 INFO] Step 21200/102583; acc: 81.52; ppl: 1.84; xent: 0.61; lr: 0.00010; 1876/2718 tok/s; 2186 sec
  500. [2019-02-04 18:27:01,503 INFO] Step 21250/102583; acc: 81.41; ppl: 1.87; xent: 0.62; lr: 0.00010; 2311/2507 tok/s; 2191 sec
  501. [2019-02-04 18:27:06,585 INFO] Step 21300/102583; acc: 78.99; ppl: 1.98; xent: 0.68; lr: 0.00010; 1810/2675 tok/s; 2196 sec
  502. [2019-02-04 18:27:11,438 INFO] Step 21350/102583; acc: 81.72; ppl: 1.84; xent: 0.61; lr: 0.00010; 2242/2529 tok/s; 2201 sec
  503. [2019-02-04 18:27:16,452 INFO] Step 21400/102583; acc: 80.01; ppl: 1.91; xent: 0.65; lr: 0.00010; 1975/2639 tok/s; 2206 sec
  504. [2019-02-04 18:27:21,543 INFO] Step 21450/102583; acc: 82.13; ppl: 1.83; xent: 0.61; lr: 0.00010; 2047/2450 tok/s; 2211 sec
  505. [2019-02-04 18:27:26,328 INFO] Step 21500/102583; acc: 79.39; ppl: 1.97; xent: 0.68; lr: 0.00010; 2024/2752 tok/s; 2216 sec
  506. [2019-02-04 18:27:31,340 INFO] Step 21550/102583; acc: 81.72; ppl: 1.83; xent: 0.60; lr: 0.00010; 2002/2614 tok/s; 2221 sec
  507. [2019-02-04 18:27:36,142 INFO] Step 21600/102583; acc: 80.03; ppl: 1.96; xent: 0.67; lr: 0.00010; 2049/2644 tok/s; 2226 sec
  508. [2019-02-04 18:27:41,355 INFO] Step 21650/102583; acc: 82.58; ppl: 1.77; xent: 0.57; lr: 0.00010; 2075/2548 tok/s; 2231 sec
  509. [2019-02-04 18:27:45,956 INFO] Step 21700/102583; acc: 79.34; ppl: 1.97; xent: 0.68; lr: 0.00010; 2117/2565 tok/s; 2235 sec
  510. [2019-02-04 18:27:51,077 INFO] Step 21750/102583; acc: 81.77; ppl: 1.81; xent: 0.59; lr: 0.00010; 2090/2590 tok/s; 2240 sec
  511. [2019-02-04 18:27:56,410 INFO] Step 21800/102583; acc: 81.43; ppl: 1.82; xent: 0.60; lr: 0.00010; 1904/2756 tok/s; 2246 sec
  512. [2019-02-04 18:28:01,206 INFO] Step 21850/102583; acc: 80.27; ppl: 1.92; xent: 0.65; lr: 0.00010; 2047/2651 tok/s; 2251 sec
  513. [2019-02-04 18:28:05,979 INFO] Step 21900/102583; acc: 82.16; ppl: 1.82; xent: 0.60; lr: 0.00010; 2135/2534 tok/s; 2255 sec
  514. [2019-02-04 18:28:11,012 INFO] Step 21950/102583; acc: 80.15; ppl: 1.90; xent: 0.64; lr: 0.00010; 2076/2699 tok/s; 2260 sec
  515. [2019-02-04 18:28:15,806 INFO] Step 22000/102583; acc: 80.06; ppl: 1.92; xent: 0.65; lr: 0.00010; 2005/2653 tok/s; 2265 sec
  516. [2019-02-04 18:28:20,825 INFO] Step 22050/102583; acc: 81.59; ppl: 1.84; xent: 0.61; lr: 0.00010; 1995/2641 tok/s; 2270 sec
  517. [2019-02-04 18:28:25,588 INFO] Step 22100/102583; acc: 81.89; ppl: 1.82; xent: 0.60; lr: 0.00010; 2162/2626 tok/s; 2275 sec
  518. [2019-02-04 18:28:30,789 INFO] Step 22150/102583; acc: 80.23; ppl: 1.89; xent: 0.64; lr: 0.00010; 1971/2589 tok/s; 2280 sec
  519. [2019-02-04 18:28:36,028 INFO] Step 22200/102583; acc: 79.49; ppl: 1.96; xent: 0.68; lr: 0.00010; 1920/2661 tok/s; 2285 sec
  520. [2019-02-04 18:28:40,737 INFO] Step 22250/102583; acc: 80.40; ppl: 1.93; xent: 0.65; lr: 0.00010; 2067/2553 tok/s; 2290 sec
  521. [2019-02-04 18:28:45,551 INFO] Step 22300/102583; acc: 82.42; ppl: 1.76; xent: 0.57; lr: 0.00010; 2012/2825 tok/s; 2295 sec
  522. [2019-02-04 18:28:50,897 INFO] Step 22350/102583; acc: 82.05; ppl: 1.81; xent: 0.60; lr: 0.00010; 2075/2523 tok/s; 2300 sec
  523. [2019-02-04 18:28:55,731 INFO] Step 22400/102583; acc: 80.62; ppl: 1.88; xent: 0.63; lr: 0.00010; 2078/2725 tok/s; 2305 sec
  524. [2019-02-04 18:29:00,653 INFO] Step 22450/102583; acc: 80.59; ppl: 1.91; xent: 0.65; lr: 0.00010; 2242/2435 tok/s; 2310 sec
  525. [2019-02-04 18:29:06,040 INFO] Step 22500/102583; acc: 81.20; ppl: 1.84; xent: 0.61; lr: 0.00010; 1944/2604 tok/s; 2315 sec
  526. [2019-02-04 18:29:10,639 INFO] Step 22550/102583; acc: 78.93; ppl: 2.00; xent: 0.69; lr: 0.00010; 2036/2674 tok/s; 2320 sec
  527. [2019-02-04 18:29:15,433 INFO] Step 22600/102583; acc: 81.13; ppl: 1.85; xent: 0.62; lr: 0.00010; 2125/2545 tok/s; 2325 sec
  528. [2019-02-04 18:29:20,691 INFO] Step 22650/102583; acc: 80.50; ppl: 1.89; xent: 0.64; lr: 0.00010; 1879/2662 tok/s; 2330 sec
  529. [2019-02-04 18:29:25,873 INFO] Step 22700/102583; acc: 81.15; ppl: 1.85; xent: 0.61; lr: 0.00010; 1921/2524 tok/s; 2335 sec
  530. [2019-02-04 18:29:31,337 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  531. [2019-02-04 18:29:36,359 INFO] Step 22750/102583; acc: 81.21; ppl: 1.85; xent: 0.62; lr: 0.00010; 1043/1244 tok/s; 2346 sec
  532. [2019-02-04 18:29:41,043 INFO] Step 22800/102583; acc: 80.30; ppl: 1.93; xent: 0.66; lr: 0.00010; 2112/2626 tok/s; 2350 sec
  533. [2019-02-04 18:29:46,139 INFO] Step 22850/102583; acc: 82.39; ppl: 1.78; xent: 0.58; lr: 0.00010; 2018/2744 tok/s; 2356 sec
  534. [2019-02-04 18:29:51,027 INFO] Step 22900/102583; acc: 79.99; ppl: 1.94; xent: 0.66; lr: 0.00010; 2023/2597 tok/s; 2360 sec
  535. [2019-02-04 18:29:55,918 INFO] Step 22950/102583; acc: 81.23; ppl: 1.85; xent: 0.62; lr: 0.00010; 1960/2808 tok/s; 2365 sec
  536. [2019-02-04 18:30:00,845 INFO] Step 23000/102583; acc: 81.16; ppl: 1.83; xent: 0.61; lr: 0.00010; 2219/2505 tok/s; 2370 sec
  537. [2019-02-04 18:30:05,789 INFO] Step 23050/102583; acc: 81.15; ppl: 1.85; xent: 0.61; lr: 0.00010; 2180/2640 tok/s; 2375 sec
  538. [2019-02-04 18:30:10,615 INFO] Step 23100/102583; acc: 80.29; ppl: 1.94; xent: 0.66; lr: 0.00010; 2060/2609 tok/s; 2380 sec
  539. [2019-02-04 18:30:15,809 INFO] Step 23150/102583; acc: 79.59; ppl: 1.93; xent: 0.66; lr: 0.00010; 1854/2712 tok/s; 2385 sec
  540. [2019-02-04 18:30:20,623 INFO] Step 23200/102583; acc: 80.82; ppl: 1.89; xent: 0.64; lr: 0.00010; 2106/2622 tok/s; 2390 sec
  541. [2019-02-04 18:30:25,858 INFO] Step 23250/102583; acc: 81.50; ppl: 1.81; xent: 0.59; lr: 0.00010; 1908/2732 tok/s; 2395 sec
  542. [2019-02-04 18:30:30,655 INFO] Step 23300/102583; acc: 80.79; ppl: 1.91; xent: 0.65; lr: 0.00010; 2288/2523 tok/s; 2400 sec
  543. [2019-02-04 18:30:35,548 INFO] Step 23350/102583; acc: 79.89; ppl: 1.94; xent: 0.66; lr: 0.00010; 2061/2632 tok/s; 2405 sec
  544. [2019-02-04 18:30:40,395 INFO] Step 23400/102583; acc: 79.73; ppl: 1.93; xent: 0.66; lr: 0.00010; 1907/2637 tok/s; 2410 sec
  545. [2019-02-04 18:30:45,484 INFO] Step 23450/102583; acc: 83.06; ppl: 1.76; xent: 0.56; lr: 0.00010; 2297/2552 tok/s; 2415 sec
  546. [2019-02-04 18:30:50,530 INFO] Step 23500/102583; acc: 80.54; ppl: 1.88; xent: 0.63; lr: 0.00010; 1928/2763 tok/s; 2420 sec
  547. [2019-02-04 18:30:55,395 INFO] Step 23550/102583; acc: 80.81; ppl: 1.88; xent: 0.63; lr: 0.00010; 2058/2421 tok/s; 2425 sec
  548. [2019-02-04 18:31:00,339 INFO] Step 23600/102583; acc: 81.92; ppl: 1.81; xent: 0.59; lr: 0.00010; 1975/2694 tok/s; 2430 sec
  549. [2019-02-04 18:31:05,475 INFO] Step 23650/102583; acc: 80.53; ppl: 1.90; xent: 0.64; lr: 0.00010; 2121/2470 tok/s; 2435 sec
  550. [2019-02-04 18:31:10,371 INFO] Step 23700/102583; acc: 80.43; ppl: 1.89; xent: 0.64; lr: 0.00010; 1948/2716 tok/s; 2440 sec
  551. [2019-02-04 18:31:15,727 INFO] Step 23750/102583; acc: 80.70; ppl: 1.86; xent: 0.62; lr: 0.00010; 1968/2697 tok/s; 2445 sec
  552. [2019-02-04 18:31:20,434 INFO] Step 23800/102583; acc: 81.25; ppl: 1.86; xent: 0.62; lr: 0.00010; 2110/2545 tok/s; 2450 sec
  553. [2019-02-04 18:31:25,426 INFO] Step 23850/102583; acc: 80.06; ppl: 1.92; xent: 0.65; lr: 0.00010; 2122/2561 tok/s; 2455 sec
  554. [2019-02-04 18:31:30,367 INFO] Step 23900/102583; acc: 79.94; ppl: 1.92; xent: 0.65; lr: 0.00010; 1984/2679 tok/s; 2460 sec
  555. [2019-02-04 18:31:35,082 INFO] Step 23950/102583; acc: 79.71; ppl: 1.95; xent: 0.67; lr: 0.00010; 2098/2584 tok/s; 2464 sec
  556. [2019-02-04 18:31:40,434 INFO] Step 24000/102583; acc: 81.92; ppl: 1.81; xent: 0.59; lr: 0.00010; 1918/2647 tok/s; 2470 sec
  557. [2019-02-04 18:31:45,556 INFO] Step 24050/102583; acc: 82.68; ppl: 1.80; xent: 0.59; lr: 0.00010; 2248/2331 tok/s; 2475 sec
  558. [2019-02-04 18:31:50,370 INFO] Step 24100/102583; acc: 78.91; ppl: 1.95; xent: 0.67; lr: 0.00010; 1804/2870 tok/s; 2480 sec
  559. [2019-02-04 18:31:55,379 INFO] Step 24150/102583; acc: 80.89; ppl: 1.84; xent: 0.61; lr: 0.00010; 2048/2685 tok/s; 2485 sec
  560. [2019-02-04 18:32:00,235 INFO] Step 24200/102583; acc: 81.41; ppl: 1.86; xent: 0.62; lr: 0.00010; 2180/2504 tok/s; 2490 sec
  561. [2019-02-04 18:32:04,926 INFO] Step 24250/102583; acc: 80.19; ppl: 1.94; xent: 0.66; lr: 0.00010; 1825/2664 tok/s; 2494 sec
  562. [2019-02-04 18:32:10,241 INFO] Step 24300/102583; acc: 81.59; ppl: 1.78; xent: 0.58; lr: 0.00010; 2156/2573 tok/s; 2500 sec
  563. [2019-02-04 18:32:15,101 INFO] Step 24350/102583; acc: 79.83; ppl: 1.93; xent: 0.66; lr: 0.00010; 2038/2726 tok/s; 2504 sec
  564. [2019-02-04 18:32:20,057 INFO] Step 24400/102583; acc: 81.18; ppl: 1.86; xent: 0.62; lr: 0.00010; 2031/2524 tok/s; 2509 sec
  565. [2019-02-04 18:32:25,150 INFO] Step 24450/102583; acc: 81.32; ppl: 1.85; xent: 0.62; lr: 0.00010; 1915/2728 tok/s; 2515 sec
  566. [2019-02-04 18:32:30,084 INFO] Step 24500/102583; acc: 81.09; ppl: 1.85; xent: 0.62; lr: 0.00010; 2201/2560 tok/s; 2519 sec
  567. [2019-02-04 18:32:35,057 INFO] Step 24550/102583; acc: 79.87; ppl: 1.93; xent: 0.66; lr: 0.00010; 1871/2686 tok/s; 2524 sec
  568. [2019-02-04 18:32:39,978 INFO] Step 24600/102583; acc: 80.80; ppl: 1.85; xent: 0.62; lr: 0.00010; 2187/2597 tok/s; 2529 sec
  569. [2019-02-04 18:32:44,909 INFO] Step 24650/102583; acc: 81.49; ppl: 1.84; xent: 0.61; lr: 0.00010; 2060/2585 tok/s; 2534 sec
  570. [2019-02-04 18:32:50,047 INFO] Step 24700/102583; acc: 80.84; ppl: 1.88; xent: 0.63; lr: 0.00010; 1943/2502 tok/s; 2539 sec
  571. [2019-02-04 18:32:54,845 INFO] Step 24750/102583; acc: 80.80; ppl: 1.91; xent: 0.65; lr: 0.00010; 2023/2699 tok/s; 2544 sec
  572. [2019-02-04 18:32:59,766 INFO] Step 24800/102583; acc: 81.40; ppl: 1.85; xent: 0.61; lr: 0.00010; 2100/2619 tok/s; 2549 sec
  573. [2019-02-04 18:33:04,733 INFO] Step 24850/102583; acc: 81.01; ppl: 1.89; xent: 0.64; lr: 0.00010; 2019/2604 tok/s; 2554 sec
  574. [2019-02-04 18:33:09,738 INFO] Step 24900/102583; acc: 81.77; ppl: 1.81; xent: 0.59; lr: 0.00010; 2145/2561 tok/s; 2559 sec
  575. [2019-02-04 18:33:14,405 INFO] Step 24950/102583; acc: 80.76; ppl: 1.89; xent: 0.64; lr: 0.00010; 2143/2597 tok/s; 2564 sec
  576. [2019-02-04 18:33:19,581 INFO] Step 25000/102583; acc: 81.66; ppl: 1.81; xent: 0.60; lr: 0.00010; 2018/2627 tok/s; 2569 sec
  577. [2019-02-04 18:33:19,583 INFO] Saving checkpoint models/ted1_step_25000.pt
  578. [2019-02-04 18:33:31,561 INFO] Step 25050/102583; acc: 81.00; ppl: 1.83; xent: 0.60; lr: 0.00010; 842/1202 tok/s; 2581 sec
  579. [2019-02-04 18:33:36,351 INFO] Step 25100/102583; acc: 80.26; ppl: 1.94; xent: 0.66; lr: 0.00010; 2012/2624 tok/s; 2586 sec
  580. [2019-02-04 18:33:41,146 INFO] Step 25150/102583; acc: 83.25; ppl: 1.76; xent: 0.57; lr: 0.00010; 2202/2566 tok/s; 2591 sec
  581. [2019-02-04 18:33:46,221 INFO] Step 25200/102583; acc: 79.60; ppl: 1.93; xent: 0.66; lr: 0.00010; 2062/2686 tok/s; 2596 sec
  582. [2019-02-04 18:33:50,860 INFO] Step 25250/102583; acc: 80.02; ppl: 1.91; xent: 0.65; lr: 0.00010; 1999/2638 tok/s; 2600 sec
  583. [2019-02-04 18:33:55,960 INFO] Step 25300/102583; acc: 81.40; ppl: 1.86; xent: 0.62; lr: 0.00010; 1936/2674 tok/s; 2605 sec
  584. [2019-02-04 18:34:00,677 INFO] Step 25350/102583; acc: 82.36; ppl: 1.79; xent: 0.58; lr: 0.00010; 2234/2604 tok/s; 2610 sec
  585. [2019-02-04 18:34:05,921 INFO] Step 25400/102583; acc: 80.53; ppl: 1.89; xent: 0.64; lr: 0.00010; 1931/2614 tok/s; 2615 sec
  586. [2019-02-04 18:34:11,063 INFO] Step 25450/102583; acc: 79.49; ppl: 1.96; xent: 0.67; lr: 0.00010; 1956/2624 tok/s; 2620 sec
  587. [2019-02-04 18:34:16,011 INFO] Step 25500/102583; acc: 81.83; ppl: 1.84; xent: 0.61; lr: 0.00010; 2112/2567 tok/s; 2625 sec
  588. [2019-02-04 18:34:20,893 INFO] Step 25550/102583; acc: 82.23; ppl: 1.77; xent: 0.57; lr: 0.00010; 1909/2804 tok/s; 2630 sec
  589. [2019-02-04 18:34:26,115 INFO] Step 25600/102583; acc: 81.82; ppl: 1.83; xent: 0.61; lr: 0.00010; 2171/2503 tok/s; 2636 sec
  590. [2019-02-04 18:34:30,855 INFO] Step 25650/102583; acc: 81.09; ppl: 1.85; xent: 0.61; lr: 0.00010; 2048/2768 tok/s; 2640 sec
  591. [2019-02-04 18:34:35,792 INFO] Step 25700/102583; acc: 80.04; ppl: 1.96; xent: 0.67; lr: 0.00010; 2145/2370 tok/s; 2645 sec
  592. [2019-02-04 18:34:41,101 INFO] Step 25750/102583; acc: 81.80; ppl: 1.82; xent: 0.60; lr: 0.00010; 2083/2657 tok/s; 2650 sec
  593. [2019-02-04 18:34:45,685 INFO] Step 25800/102583; acc: 79.45; ppl: 1.96; xent: 0.67; lr: 0.00010; 1948/2691 tok/s; 2655 sec
  594. [2019-02-04 18:34:50,557 INFO] Step 25850/102583; acc: 81.58; ppl: 1.81; xent: 0.60; lr: 0.00010; 2090/2573 tok/s; 2660 sec
  595. [2019-02-04 18:34:55,785 INFO] Step 25900/102583; acc: 80.55; ppl: 1.88; xent: 0.63; lr: 0.00010; 1908/2666 tok/s; 2665 sec
  596. [2019-02-04 18:35:01,027 INFO] Step 25950/102583; acc: 81.22; ppl: 1.86; xent: 0.62; lr: 0.00010; 1931/2440 tok/s; 2670 sec
  597. [2019-02-04 18:35:06,672 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  598. [2019-02-04 18:35:12,224 INFO] Step 26000/102583; acc: 81.15; ppl: 1.85; xent: 0.62; lr: 0.00010; 950/1159 tok/s; 2682 sec
  599. [2019-02-04 18:35:17,023 INFO] Step 26050/102583; acc: 81.47; ppl: 1.86; xent: 0.62; lr: 0.00010; 2101/2684 tok/s; 2686 sec
  600. [2019-02-04 18:35:22,045 INFO] Step 26100/102583; acc: 82.34; ppl: 1.80; xent: 0.59; lr: 0.00010; 2022/2718 tok/s; 2691 sec
  601. [2019-02-04 18:35:26,933 INFO] Step 26150/102583; acc: 80.34; ppl: 1.91; xent: 0.65; lr: 0.00010; 2010/2591 tok/s; 2696 sec
  602. [2019-02-04 18:35:31,765 INFO] Step 26200/102583; acc: 81.81; ppl: 1.83; xent: 0.60; lr: 0.00010; 2050/2766 tok/s; 2701 sec
  603. [2019-02-04 18:35:36,854 INFO] Step 26250/102583; acc: 79.93; ppl: 1.90; xent: 0.64; lr: 0.00010; 2104/2540 tok/s; 2706 sec
  604. [2019-02-04 18:35:41,656 INFO] Step 26300/102583; acc: 81.15; ppl: 1.85; xent: 0.62; lr: 0.00010; 2173/2626 tok/s; 2711 sec
  605. [2019-02-04 18:35:46,655 INFO] Step 26350/102583; acc: 81.21; ppl: 1.86; xent: 0.62; lr: 0.00010; 2003/2710 tok/s; 2716 sec
  606. [2019-02-04 18:35:51,750 INFO] Step 26400/102583; acc: 79.75; ppl: 1.94; xent: 0.67; lr: 0.00010; 1944/2594 tok/s; 2721 sec
  607. [2019-02-04 18:35:56,509 INFO] Step 26450/102583; acc: 80.66; ppl: 1.88; xent: 0.63; lr: 0.00010; 2125/2654 tok/s; 2726 sec
  608. [2019-02-04 18:36:01,675 INFO] Step 26500/102583; acc: 81.83; ppl: 1.79; xent: 0.58; lr: 0.00010; 1945/2711 tok/s; 2731 sec
  609. [2019-02-04 18:36:06,527 INFO] Step 26550/102583; acc: 81.04; ppl: 1.90; xent: 0.64; lr: 0.00010; 2249/2557 tok/s; 2736 sec
  610. [2019-02-04 18:36:11,517 INFO] Step 26600/102583; acc: 79.75; ppl: 1.91; xent: 0.65; lr: 0.00010; 2023/2589 tok/s; 2741 sec
  611. [2019-02-04 18:36:16,252 INFO] Step 26650/102583; acc: 81.02; ppl: 1.88; xent: 0.63; lr: 0.00010; 2031/2654 tok/s; 2746 sec
  612. [2019-02-04 18:36:21,464 INFO] Step 26700/102583; acc: 81.86; ppl: 1.80; xent: 0.59; lr: 0.00010; 2169/2636 tok/s; 2751 sec
  613. [2019-02-04 18:36:26,346 INFO] Step 26750/102583; acc: 80.94; ppl: 1.87; xent: 0.63; lr: 0.00010; 1986/2662 tok/s; 2756 sec
  614. [2019-02-04 18:36:31,403 INFO] Step 26800/102583; acc: 81.80; ppl: 1.81; xent: 0.59; lr: 0.00010; 2051/2458 tok/s; 2761 sec
  615. [2019-02-04 18:36:36,375 INFO] Step 26850/102583; acc: 80.91; ppl: 1.85; xent: 0.62; lr: 0.00010; 1952/2654 tok/s; 2766 sec
  616. [2019-02-04 18:36:41,444 INFO] Step 26900/102583; acc: 80.56; ppl: 1.88; xent: 0.63; lr: 0.00010; 2073/2467 tok/s; 2771 sec
  617. [2019-02-04 18:36:46,391 INFO] Step 26950/102583; acc: 81.54; ppl: 1.83; xent: 0.61; lr: 0.00010; 2027/2755 tok/s; 2776 sec
  618. [2019-02-04 18:36:51,593 INFO] Step 27000/102583; acc: 80.11; ppl: 1.88; xent: 0.63; lr: 0.00010; 1934/2688 tok/s; 2781 sec
  619. [2019-02-04 18:36:56,441 INFO] Step 27050/102583; acc: 80.93; ppl: 1.88; xent: 0.63; lr: 0.00010; 2082/2545 tok/s; 2786 sec
  620. [2019-02-04 18:37:01,491 INFO] Step 27100/102583; acc: 80.74; ppl: 1.88; xent: 0.63; lr: 0.00010; 2091/2546 tok/s; 2791 sec
  621. [2019-02-04 18:37:06,374 INFO] Step 27150/102583; acc: 80.05; ppl: 1.91; xent: 0.64; lr: 0.00010; 2018/2660 tok/s; 2796 sec
  622. [2019-02-04 18:37:10,941 INFO] Step 27200/102583; acc: 80.00; ppl: 1.94; xent: 0.66; lr: 0.00010; 2084/2651 tok/s; 2800 sec
  623. [2019-02-04 18:37:16,211 INFO] Step 27250/102583; acc: 82.31; ppl: 1.79; xent: 0.58; lr: 0.00010; 1972/2604 tok/s; 2806 sec
  624. [2019-02-04 18:37:21,592 INFO] Step 27300/102583; acc: 82.62; ppl: 1.77; xent: 0.57; lr: 0.00010; 2176/2468 tok/s; 2811 sec
  625. [2019-02-04 18:37:26,341 INFO] Step 27350/102583; acc: 79.25; ppl: 1.94; xent: 0.66; lr: 0.00010; 1881/2725 tok/s; 2816 sec
  626. [2019-02-04 18:37:31,437 INFO] Step 27400/102583; acc: 80.37; ppl: 1.85; xent: 0.62; lr: 0.00010; 2076/2671 tok/s; 2821 sec
  627. [2019-02-04 18:37:36,121 INFO] Step 27450/102583; acc: 81.60; ppl: 1.84; xent: 0.61; lr: 0.00010; 2085/2589 tok/s; 2826 sec
  628. [2019-02-04 18:37:40,826 INFO] Step 27500/102583; acc: 80.37; ppl: 1.94; xent: 0.66; lr: 0.00010; 1929/2615 tok/s; 2830 sec
  629. [2019-02-04 18:37:46,131 INFO] Step 27550/102583; acc: 81.43; ppl: 1.79; xent: 0.58; lr: 0.00010; 2049/2546 tok/s; 2836 sec
  630. [2019-02-04 18:37:51,037 INFO] Step 27600/102583; acc: 80.21; ppl: 1.88; xent: 0.63; lr: 0.00010; 2088/2809 tok/s; 2840 sec
  631. [2019-02-04 18:37:55,997 INFO] Step 27650/102583; acc: 81.65; ppl: 1.84; xent: 0.61; lr: 0.00010; 2040/2503 tok/s; 2845 sec
  632. [2019-02-04 18:38:01,027 INFO] Step 27700/102583; acc: 81.91; ppl: 1.83; xent: 0.60; lr: 0.00010; 1925/2737 tok/s; 2850 sec
  633. [2019-02-04 18:38:05,867 INFO] Step 27750/102583; acc: 80.04; ppl: 1.91; xent: 0.65; lr: 0.00010; 2165/2534 tok/s; 2855 sec
  634. [2019-02-04 18:38:10,874 INFO] Step 27800/102583; acc: 80.63; ppl: 1.88; xent: 0.63; lr: 0.00010; 1922/2704 tok/s; 2860 sec
  635. [2019-02-04 18:38:15,857 INFO] Step 27850/102583; acc: 81.34; ppl: 1.84; xent: 0.61; lr: 0.00010; 2157/2514 tok/s; 2865 sec
  636. [2019-02-04 18:38:20,954 INFO] Step 27900/102583; acc: 81.52; ppl: 1.84; xent: 0.61; lr: 0.00010; 2163/2603 tok/s; 2870 sec
  637. [2019-02-04 18:38:25,890 INFO] Step 27950/102583; acc: 81.30; ppl: 1.87; xent: 0.63; lr: 0.00010; 1871/2570 tok/s; 2875 sec
  638. [2019-02-04 18:38:30,675 INFO] Step 28000/102583; acc: 80.64; ppl: 1.90; xent: 0.64; lr: 0.00010; 2029/2682 tok/s; 2880 sec
  639. [2019-02-04 18:38:35,812 INFO] Step 28050/102583; acc: 81.52; ppl: 1.83; xent: 0.61; lr: 0.00010; 2082/2555 tok/s; 2885 sec
  640. [2019-02-04 18:38:40,566 INFO] Step 28100/102583; acc: 80.98; ppl: 1.89; xent: 0.64; lr: 0.00010; 1994/2683 tok/s; 2890 sec
  641. [2019-02-04 18:38:45,593 INFO] Step 28150/102583; acc: 82.18; ppl: 1.80; xent: 0.59; lr: 0.00010; 2113/2580 tok/s; 2895 sec
  642. [2019-02-04 18:38:50,308 INFO] Step 28200/102583; acc: 81.62; ppl: 1.83; xent: 0.61; lr: 0.00010; 2174/2587 tok/s; 2900 sec
  643. [2019-02-04 18:38:55,567 INFO] Step 28250/102583; acc: 80.54; ppl: 1.86; xent: 0.62; lr: 0.00010; 1903/2631 tok/s; 2905 sec
  644. [2019-02-04 18:39:00,609 INFO] Step 28300/102583; acc: 81.47; ppl: 1.83; xent: 0.60; lr: 0.00010; 2063/2684 tok/s; 2910 sec
  645. [2019-02-04 18:39:05,412 INFO] Step 28350/102583; acc: 80.85; ppl: 1.90; xent: 0.64; lr: 0.00010; 2031/2620 tok/s; 2915 sec
  646. [2019-02-04 18:39:10,432 INFO] Step 28400/102583; acc: 82.80; ppl: 1.77; xent: 0.57; lr: 0.00010; 2073/2603 tok/s; 2920 sec
  647. [2019-02-04 18:39:15,314 INFO] Step 28450/102583; acc: 80.20; ppl: 1.89; xent: 0.64; lr: 0.00010; 2086/2694 tok/s; 2925 sec
  648. [2019-02-04 18:39:19,898 INFO] Step 28500/102583; acc: 80.32; ppl: 1.90; xent: 0.64; lr: 0.00010; 2079/2579 tok/s; 2929 sec
  649. [2019-02-04 18:39:25,263 INFO] Step 28550/102583; acc: 81.90; ppl: 1.81; xent: 0.59; lr: 0.00010; 1973/2673 tok/s; 2935 sec
  650. [2019-02-04 18:39:30,013 INFO] Step 28600/102583; acc: 81.43; ppl: 1.84; xent: 0.61; lr: 0.00010; 2218/2551 tok/s; 2939 sec
  651. [2019-02-04 18:39:35,122 INFO] Step 28650/102583; acc: 80.56; ppl: 1.89; xent: 0.63; lr: 0.00010; 1888/2688 tok/s; 2945 sec
  652. [2019-02-04 18:39:40,194 INFO] Step 28700/102583; acc: 80.37; ppl: 1.93; xent: 0.66; lr: 0.00010; 1933/2604 tok/s; 2950 sec
  653. [2019-02-04 18:39:45,113 INFO] Step 28750/102583; acc: 82.41; ppl: 1.79; xent: 0.58; lr: 0.00010; 2154/2651 tok/s; 2955 sec
  654. [2019-02-04 18:39:49,914 INFO] Step 28800/102583; acc: 81.83; ppl: 1.80; xent: 0.59; lr: 0.00010; 1920/2750 tok/s; 2959 sec
  655. [2019-02-04 18:39:55,085 INFO] Step 28850/102583; acc: 81.72; ppl: 1.83; xent: 0.61; lr: 0.00010; 2142/2475 tok/s; 2964 sec
  656. [2019-02-04 18:39:59,877 INFO] Step 28900/102583; acc: 81.17; ppl: 1.86; xent: 0.62; lr: 0.00010; 2027/2761 tok/s; 2969 sec
  657. [2019-02-04 18:40:04,888 INFO] Step 28950/102583; acc: 80.38; ppl: 1.91; xent: 0.65; lr: 0.00010; 2138/2434 tok/s; 2974 sec
  658. [2019-02-04 18:40:10,141 INFO] Step 29000/102583; acc: 81.20; ppl: 1.84; xent: 0.61; lr: 0.00010; 2091/2624 tok/s; 2980 sec
  659. [2019-02-04 18:40:14,822 INFO] Step 29050/102583; acc: 80.51; ppl: 1.88; xent: 0.63; lr: 0.00010; 1907/2761 tok/s; 2984 sec
  660. [2019-02-04 18:40:19,606 INFO] Step 29100/102583; acc: 81.43; ppl: 1.85; xent: 0.61; lr: 0.00010; 2115/2529 tok/s; 2989 sec
  661. [2019-02-04 18:40:25,000 INFO] Step 29150/102583; acc: 80.92; ppl: 1.85; xent: 0.61; lr: 0.00010; 1964/2635 tok/s; 2994 sec
  662. [2019-02-04 18:40:35,122 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  663. [2019-02-04 18:40:36,172 INFO] Step 29200/102583; acc: 80.93; ppl: 1.89; xent: 0.64; lr: 0.00010; 882/1120 tok/s; 3006 sec
  664. [2019-02-04 18:40:41,250 INFO] Step 29250/102583; acc: 81.64; ppl: 1.82; xent: 0.60; lr: 0.00010; 2152/2559 tok/s; 3011 sec
  665. [2019-02-04 18:40:45,982 INFO] Step 29300/102583; acc: 81.16; ppl: 1.87; xent: 0.63; lr: 0.00010; 1954/2702 tok/s; 3015 sec
  666. [2019-02-04 18:40:51,350 INFO] Step 29350/102583; acc: 82.73; ppl: 1.76; xent: 0.57; lr: 0.00010; 2083/2644 tok/s; 3021 sec
  667. [2019-02-04 18:40:56,086 INFO] Step 29400/102583; acc: 80.38; ppl: 1.90; xent: 0.64; lr: 0.00010; 2080/2662 tok/s; 3025 sec
  668. [2019-02-04 18:41:01,084 INFO] Step 29450/102583; acc: 82.18; ppl: 1.80; xent: 0.59; lr: 0.00010; 2086/2680 tok/s; 3030 sec
  669. [2019-02-04 18:41:06,065 INFO] Step 29500/102583; acc: 79.91; ppl: 1.89; xent: 0.64; lr: 0.00010; 2065/2573 tok/s; 3035 sec
  670. [2019-02-04 18:41:10,901 INFO] Step 29550/102583; acc: 81.58; ppl: 1.83; xent: 0.60; lr: 0.00010; 2097/2762 tok/s; 3040 sec
  671. [2019-02-04 18:41:15,884 INFO] Step 29600/102583; acc: 80.45; ppl: 1.90; xent: 0.64; lr: 0.00010; 1945/2696 tok/s; 3045 sec
  672. [2019-02-04 18:41:20,951 INFO] Step 29650/102583; acc: 80.23; ppl: 1.92; xent: 0.65; lr: 0.00010; 2056/2531 tok/s; 3050 sec
  673. [2019-02-04 18:41:25,716 INFO] Step 29700/102583; acc: 81.02; ppl: 1.84; xent: 0.61; lr: 0.00010; 2069/2644 tok/s; 3055 sec
  674. [2019-02-04 18:41:30,934 INFO] Step 29750/102583; acc: 81.53; ppl: 1.82; xent: 0.60; lr: 0.00010; 1985/2672 tok/s; 3060 sec
  675. [2019-02-04 18:41:35,585 INFO] Step 29800/102583; acc: 81.48; ppl: 1.87; xent: 0.62; lr: 0.00010; 2241/2542 tok/s; 3065 sec
  676. [2019-02-04 18:41:40,648 INFO] Step 29850/102583; acc: 79.64; ppl: 1.92; xent: 0.65; lr: 0.00010; 2032/2632 tok/s; 3070 sec
  677. [2019-02-04 18:41:45,563 INFO] Step 29900/102583; acc: 81.95; ppl: 1.81; xent: 0.59; lr: 0.00010; 2044/2648 tok/s; 3075 sec
  678. [2019-02-04 18:41:50,577 INFO] Step 29950/102583; acc: 81.51; ppl: 1.83; xent: 0.60; lr: 0.00010; 2145/2647 tok/s; 3080 sec
  679. [2019-02-04 18:41:55,527 INFO] Step 30000/102583; acc: 80.69; ppl: 1.88; xent: 0.63; lr: 0.00010; 1997/2597 tok/s; 3085 sec
  680. [2019-02-04 18:41:55,559 INFO] Loading dataset from data/ted.valid.0.pt, number of examples: 873
  681. [2019-02-04 18:41:56,858 INFO] Validation perplexity: 1.74967
  682. [2019-02-04 18:41:56,859 INFO] Validation accuracy: 82.5772
  683. [2019-02-04 18:41:56,861 INFO] Saving checkpoint models/ted1_step_30000.pt
  684. [2019-02-04 18:42:07,550 INFO] Step 30050/102583; acc: 81.17; ppl: 1.82; xent: 0.60; lr: 0.00010; 852/1085 tok/s; 3097 sec
  685. [2019-02-04 18:42:12,613 INFO] Step 30100/102583; acc: 82.26; ppl: 1.79; xent: 0.58; lr: 0.00010; 1993/2630 tok/s; 3102 sec
  686. [2019-02-04 18:42:17,584 INFO] Step 30150/102583; acc: 80.34; ppl: 1.92; xent: 0.65; lr: 0.00010; 2062/2484 tok/s; 3107 sec
  687. [2019-02-04 18:42:22,567 INFO] Step 30200/102583; acc: 80.61; ppl: 1.86; xent: 0.62; lr: 0.00010; 2023/2691 tok/s; 3112 sec
  688. [2019-02-04 18:42:27,765 INFO] Step 30250/102583; acc: 81.00; ppl: 1.84; xent: 0.61; lr: 0.00010; 1873/2600 tok/s; 3117 sec
  689. [2019-02-04 18:42:32,738 INFO] Step 30300/102583; acc: 81.79; ppl: 1.82; xent: 0.60; lr: 0.00010; 2210/2599 tok/s; 3122 sec
  690. [2019-02-04 18:42:37,640 INFO] Step 30350/102583; acc: 80.09; ppl: 1.90; xent: 0.64; lr: 0.00010; 2070/2593 tok/s; 3127 sec
  691. [2019-02-04 18:42:42,388 INFO] Step 30400/102583; acc: 79.65; ppl: 1.94; xent: 0.66; lr: 0.00010; 2019/2658 tok/s; 3132 sec
  692. [2019-02-04 18:42:47,148 INFO] Step 30450/102583; acc: 81.12; ppl: 1.86; xent: 0.62; lr: 0.00010; 2059/2698 tok/s; 3137 sec
  693. [2019-02-04 18:42:52,498 INFO] Step 30500/102583; acc: 82.12; ppl: 1.80; xent: 0.59; lr: 0.00010; 2034/2474 tok/s; 3142 sec
  694. [2019-02-04 18:42:57,653 INFO] Step 30550/102583; acc: 81.47; ppl: 1.83; xent: 0.60; lr: 0.00010; 2071/2591 tok/s; 3147 sec
  695. [2019-02-04 18:43:02,504 INFO] Step 30600/102583; acc: 80.23; ppl: 1.88; xent: 0.63; lr: 0.00010; 1893/2696 tok/s; 3152 sec
  696. [2019-02-04 18:43:07,558 INFO] Step 30650/102583; acc: 80.93; ppl: 1.83; xent: 0.61; lr: 0.00010; 2122/2713 tok/s; 3157 sec
  697. [2019-02-04 18:43:12,270 INFO] Step 30700/102583; acc: 80.87; ppl: 1.88; xent: 0.63; lr: 0.00010; 2002/2524 tok/s; 3162 sec
  698. [2019-02-04 18:43:17,030 INFO] Step 30750/102583; acc: 81.17; ppl: 1.88; xent: 0.63; lr: 0.00010; 1929/2657 tok/s; 3166 sec
  699. [2019-02-04 18:43:22,322 INFO] Step 30800/102583; acc: 81.65; ppl: 1.79; xent: 0.58; lr: 0.00010; 2141/2511 tok/s; 3172 sec
  700. [2019-02-04 18:43:27,116 INFO] Step 30850/102583; acc: 79.37; ppl: 1.93; xent: 0.66; lr: 0.00010; 2023/2860 tok/s; 3177 sec
  701. [2019-02-04 18:43:32,142 INFO] Step 30900/102583; acc: 82.10; ppl: 1.81; xent: 0.59; lr: 0.00010; 2053/2546 tok/s; 3182 sec
  702. [2019-02-04 18:43:37,256 INFO] Step 30950/102583; acc: 82.56; ppl: 1.80; xent: 0.59; lr: 0.00010; 2001/2626 tok/s; 3187 sec
  703. [2019-02-04 18:43:41,982 INFO] Step 31000/102583; acc: 79.71; ppl: 1.92; xent: 0.65; lr: 0.00010; 2074/2621 tok/s; 3191 sec
  704. [2019-02-04 18:43:47,059 INFO] Step 31050/102583; acc: 81.20; ppl: 1.85; xent: 0.62; lr: 0.00010; 1953/2656 tok/s; 3196 sec
  705. [2019-02-04 18:43:51,902 INFO] Step 31100/102583; acc: 80.94; ppl: 1.84; xent: 0.61; lr: 0.00010; 2172/2514 tok/s; 3201 sec
  706. [2019-02-04 18:43:57,257 INFO] Step 31150/102583; acc: 82.73; ppl: 1.77; xent: 0.57; lr: 0.00010; 2151/2578 tok/s; 3207 sec
  707. [2019-02-04 18:44:01,937 INFO] Step 31200/102583; acc: 80.94; ppl: 1.88; xent: 0.63; lr: 0.00010; 1854/2615 tok/s; 3211 sec
  708. [2019-02-04 18:44:06,873 INFO] Step 31250/102583; acc: 79.88; ppl: 1.95; xent: 0.67; lr: 0.00010; 1939/2667 tok/s; 3216 sec
  709. [2019-02-04 18:44:11,972 INFO] Step 31300/102583; acc: 81.90; ppl: 1.80; xent: 0.59; lr: 0.00010; 2100/2552 tok/s; 3221 sec
  710. [2019-02-04 18:44:16,744 INFO] Step 31350/102583; acc: 81.69; ppl: 1.81; xent: 0.60; lr: 0.00010; 2099/2702 tok/s; 3226 sec
  711. [2019-02-04 18:44:21,713 INFO] Step 31400/102583; acc: 81.49; ppl: 1.84; xent: 0.61; lr: 0.00010; 2115/2536 tok/s; 3231 sec
  712. [2019-02-04 18:44:26,416 INFO] Step 31450/102583; acc: 80.77; ppl: 1.86; xent: 0.62; lr: 0.00010; 2089/2708 tok/s; 3236 sec
  713. [2019-02-04 18:44:31,685 INFO] Step 31500/102583; acc: 81.51; ppl: 1.81; xent: 0.59; lr: 0.00010; 1948/2615 tok/s; 3241 sec
  714. [2019-02-04 18:44:36,692 INFO] Step 31550/102583; acc: 81.19; ppl: 1.82; xent: 0.60; lr: 0.00010; 2039/2676 tok/s; 3246 sec
  715. [2019-02-04 18:44:41,618 INFO] Step 31600/102583; acc: 81.43; ppl: 1.86; xent: 0.62; lr: 0.00010; 2086/2513 tok/s; 3251 sec
  716. [2019-02-04 18:44:46,504 INFO] Step 31650/102583; acc: 82.52; ppl: 1.78; xent: 0.57; lr: 0.00010; 2062/2694 tok/s; 3256 sec
  717. [2019-02-04 18:44:51,362 INFO] Step 31700/102583; acc: 80.00; ppl: 1.89; xent: 0.64; lr: 0.00010; 2084/2664 tok/s; 3261 sec
  718. [2019-02-04 18:44:56,163 INFO] Step 31750/102583; acc: 80.94; ppl: 1.89; xent: 0.63; lr: 0.00010; 2032/2529 tok/s; 3266 sec
  719. [2019-02-04 18:45:01,300 INFO] Step 31800/102583; acc: 81.97; ppl: 1.80; xent: 0.59; lr: 0.00010; 1996/2734 tok/s; 3271 sec
  720. [2019-02-04 18:45:06,266 INFO] Step 31850/102583; acc: 81.76; ppl: 1.82; xent: 0.60; lr: 0.00010; 2206/2529 tok/s; 3276 sec
  721. [2019-02-04 18:45:11,324 INFO] Step 31900/102583; acc: 81.09; ppl: 1.85; xent: 0.62; lr: 0.00010; 1823/2799 tok/s; 3281 sec
  722. [2019-02-04 18:45:16,312 INFO] Step 31950/102583; acc: 79.38; ppl: 1.97; xent: 0.68; lr: 0.00010; 2011/2477 tok/s; 3286 sec
  723. [2019-02-04 18:45:21,210 INFO] Step 32000/102583; acc: 82.43; ppl: 1.77; xent: 0.57; lr: 0.00010; 2104/2710 tok/s; 3291 sec
  724. [2019-02-04 18:45:26,602 INFO] Step 32050/102583; acc: 81.40; ppl: 1.81; xent: 0.59; lr: 0.00010; 1913/2771 tok/s; 3296 sec
  725. [2019-02-04 18:45:31,888 INFO] Step 32100/102583; acc: 81.13; ppl: 1.87; xent: 0.62; lr: 0.00010; 2126/2462 tok/s; 3301 sec
  726. [2019-02-04 18:45:36,691 INFO] Step 32150/102583; acc: 82.21; ppl: 1.80; xent: 0.59; lr: 0.00010; 2070/2705 tok/s; 3306 sec
  727. [2019-02-04 18:45:41,840 INFO] Step 32200/102583; acc: 80.86; ppl: 1.86; xent: 0.62; lr: 0.00010; 2141/2482 tok/s; 3311 sec
  728. [2019-02-04 18:45:46,952 INFO] Step 32250/102583; acc: 80.75; ppl: 1.87; xent: 0.63; lr: 0.00010; 2034/2651 tok/s; 3316 sec
  729. [2019-02-04 18:45:51,583 INFO] Step 32300/102583; acc: 81.50; ppl: 1.83; xent: 0.61; lr: 0.00010; 2035/2672 tok/s; 3321 sec
  730. [2019-02-04 18:45:56,205 INFO] Step 32350/102583; acc: 81.36; ppl: 1.85; xent: 0.61; lr: 0.00010; 2128/2570 tok/s; 3326 sec
  731. [2019-02-04 18:46:01,579 INFO] Step 32400/102583; acc: 81.17; ppl: 1.84; xent: 0.61; lr: 0.00010; 1947/2631 tok/s; 3331 sec
  732. [2019-02-04 18:46:11,247 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  733. [2019-02-04 18:46:12,842 INFO] Step 32450/102583; acc: 80.16; ppl: 1.92; xent: 0.65; lr: 0.00010; 875/1134 tok/s; 3342 sec
  734. [2019-02-04 18:46:17,935 INFO] Step 32500/102583; acc: 81.92; ppl: 1.82; xent: 0.60; lr: 0.00010; 2136/2545 tok/s; 3347 sec
  735. [2019-02-04 18:46:22,683 INFO] Step 32550/102583; acc: 81.94; ppl: 1.82; xent: 0.60; lr: 0.00010; 1910/2728 tok/s; 3352 sec
  736. [2019-02-04 18:46:27,809 INFO] Step 32600/102583; acc: 83.09; ppl: 1.75; xent: 0.56; lr: 0.00010; 2120/2633 tok/s; 3357 sec
  737. [2019-02-04 18:46:32,725 INFO] Step 32650/102583; acc: 80.77; ppl: 1.87; xent: 0.62; lr: 0.00010; 2098/2688 tok/s; 3362 sec
  738. [2019-02-04 18:46:37,698 INFO] Step 32700/102583; acc: 81.18; ppl: 1.83; xent: 0.60; lr: 0.00010; 2079/2652 tok/s; 3367 sec
  739. [2019-02-04 18:46:42,591 INFO] Step 32750/102583; acc: 80.62; ppl: 1.86; xent: 0.62; lr: 0.00010; 2070/2540 tok/s; 3372 sec
  740. [2019-02-04 18:46:47,388 INFO] Step 32800/102583; acc: 81.70; ppl: 1.82; xent: 0.60; lr: 0.00010; 2110/2821 tok/s; 3377 sec
  741. [2019-02-04 18:46:52,370 INFO] Step 32850/102583; acc: 80.45; ppl: 1.89; xent: 0.64; lr: 0.00010; 1919/2692 tok/s; 3382 sec
  742. [2019-02-04 18:46:57,516 INFO] Step 32900/102583; acc: 80.17; ppl: 1.92; xent: 0.65; lr: 0.00010; 2070/2498 tok/s; 3387 sec
  743. [2019-02-04 18:47:02,239 INFO] Step 32950/102583; acc: 82.02; ppl: 1.78; xent: 0.58; lr: 0.00010; 2118/2694 tok/s; 3392 sec
  744. [2019-02-04 18:47:07,544 INFO] Step 33000/102583; acc: 80.60; ppl: 1.88; xent: 0.63; lr: 0.00010; 2001/2574 tok/s; 3397 sec
  745. [2019-02-04 18:47:12,016 INFO] Step 33050/102583; acc: 81.49; ppl: 1.88; xent: 0.63; lr: 0.00010; 2189/2598 tok/s; 3401 sec
  746. [2019-02-04 18:47:17,263 INFO] Step 33100/102583; acc: 79.99; ppl: 1.88; xent: 0.63; lr: 0.00010; 1948/2694 tok/s; 3407 sec
  747. [2019-02-04 18:47:22,020 INFO] Step 33150/102583; acc: 82.15; ppl: 1.82; xent: 0.60; lr: 0.00010; 2125/2590 tok/s; 3411 sec
  748. [2019-02-04 18:47:27,249 INFO] Step 33200/102583; acc: 81.93; ppl: 1.80; xent: 0.59; lr: 0.00010; 2122/2633 tok/s; 3417 sec
  749. [2019-02-04 18:47:31,939 INFO] Step 33250/102583; acc: 81.43; ppl: 1.85; xent: 0.61; lr: 0.00010; 2062/2591 tok/s; 3421 sec
  750. [2019-02-04 18:47:37,051 INFO] Step 33300/102583; acc: 81.32; ppl: 1.82; xent: 0.60; lr: 0.00010; 1992/2610 tok/s; 3426 sec
  751. [2019-02-04 18:47:42,133 INFO] Step 33350/102583; acc: 81.65; ppl: 1.82; xent: 0.60; lr: 0.00010; 2036/2621 tok/s; 3432 sec
  752. [2019-02-04 18:47:47,135 INFO] Step 33400/102583; acc: 81.11; ppl: 1.87; xent: 0.63; lr: 0.00010; 2039/2449 tok/s; 3437 sec
  753. [2019-02-04 18:47:52,131 INFO] Step 33450/102583; acc: 79.45; ppl: 1.90; xent: 0.64; lr: 0.00010; 1952/2788 tok/s; 3442 sec
  754. [2019-02-04 18:47:57,357 INFO] Step 33500/102583; acc: 82.14; ppl: 1.79; xent: 0.58; lr: 0.00010; 1954/2517 tok/s; 3447 sec
  755. [2019-02-04 18:48:02,405 INFO] Step 33550/102583; acc: 81.58; ppl: 1.82; xent: 0.60; lr: 0.00010; 2190/2655 tok/s; 3452 sec
  756. [2019-02-04 18:48:07,113 INFO] Step 33600/102583; acc: 80.25; ppl: 1.91; xent: 0.65; lr: 0.00010; 2109/2547 tok/s; 3457 sec
  757. [2019-02-04 18:48:12,099 INFO] Step 33650/102583; acc: 80.29; ppl: 1.89; xent: 0.64; lr: 0.00010; 1908/2710 tok/s; 3461 sec
  758. [2019-02-04 18:48:16,881 INFO] Step 33700/102583; acc: 81.56; ppl: 1.85; xent: 0.62; lr: 0.00010; 2139/2667 tok/s; 3466 sec
  759. [2019-02-04 18:48:22,004 INFO] Step 33750/102583; acc: 81.89; ppl: 1.80; xent: 0.59; lr: 0.00010; 2091/2438 tok/s; 3471 sec
  760. [2019-02-04 18:48:27,267 INFO] Step 33800/102583; acc: 81.48; ppl: 1.81; xent: 0.60; lr: 0.00010; 1968/2661 tok/s; 3477 sec
  761. [2019-02-04 18:48:32,062 INFO] Step 33850/102583; acc: 80.54; ppl: 1.86; xent: 0.62; lr: 0.00010; 1874/2690 tok/s; 3481 sec
  762. [2019-02-04 18:48:37,057 INFO] Step 33900/102583; acc: 80.82; ppl: 1.85; xent: 0.62; lr: 0.00010; 2172/2703 tok/s; 3486 sec
  763. [2019-02-04 18:48:41,859 INFO] Step 33950/102583; acc: 81.55; ppl: 1.85; xent: 0.62; lr: 0.00010; 1981/2476 tok/s; 3491 sec
  764. [2019-02-04 18:48:46,540 INFO] Step 34000/102583; acc: 80.38; ppl: 1.89; xent: 0.64; lr: 0.00010; 2005/2626 tok/s; 3496 sec
  765. [2019-02-04 18:48:51,883 INFO] Step 34050/102583; acc: 81.87; ppl: 1.76; xent: 0.57; lr: 0.00010; 2113/2586 tok/s; 3501 sec
  766. [2019-02-04 18:48:56,701 INFO] Step 34100/102583; acc: 79.42; ppl: 1.94; xent: 0.66; lr: 0.00010; 1994/2760 tok/s; 3506 sec
  767. [2019-02-04 18:49:01,742 INFO] Step 34150/102583; acc: 82.14; ppl: 1.79; xent: 0.58; lr: 0.00010; 2051/2603 tok/s; 3511 sec
  768. [2019-02-04 18:49:06,800 INFO] Step 34200/102583; acc: 82.18; ppl: 1.81; xent: 0.60; lr: 0.00010; 2048/2645 tok/s; 3516 sec
  769. [2019-02-04 18:49:11,577 INFO] Step 34250/102583; acc: 79.55; ppl: 1.93; xent: 0.66; lr: 0.00010; 1957/2559 tok/s; 3521 sec
  770. [2019-02-04 18:49:16,646 INFO] Step 34300/102583; acc: 82.08; ppl: 1.80; xent: 0.59; lr: 0.00010; 2076/2660 tok/s; 3526 sec
  771. [2019-02-04 18:49:21,492 INFO] Step 34350/102583; acc: 81.18; ppl: 1.85; xent: 0.61; lr: 0.00010; 2060/2615 tok/s; 3531 sec
  772. [2019-02-04 18:49:26,792 INFO] Step 34400/102583; acc: 82.36; ppl: 1.79; xent: 0.58; lr: 0.00010; 2136/2488 tok/s; 3536 sec
  773. [2019-02-04 18:49:31,414 INFO] Step 34450/102583; acc: 81.52; ppl: 1.85; xent: 0.62; lr: 0.00010; 1934/2666 tok/s; 3541 sec
  774. [2019-02-04 18:49:36,373 INFO] Step 34500/102583; acc: 79.50; ppl: 1.95; xent: 0.67; lr: 0.00010; 1995/2633 tok/s; 3546 sec
  775. [2019-02-04 18:49:41,453 INFO] Step 34550/102583; acc: 81.92; ppl: 1.82; xent: 0.60; lr: 0.00010; 2007/2586 tok/s; 3551 sec
  776. [2019-02-04 18:49:46,186 INFO] Step 34600/102583; acc: 81.71; ppl: 1.81; xent: 0.60; lr: 0.00010; 2090/2691 tok/s; 3556 sec
  777. [2019-02-04 18:49:51,217 INFO] Step 34650/102583; acc: 81.61; ppl: 1.84; xent: 0.61; lr: 0.00010; 2163/2462 tok/s; 3561 sec
  778. [2019-02-04 18:49:56,056 INFO] Step 34700/102583; acc: 82.08; ppl: 1.79; xent: 0.58; lr: 0.00010; 2061/2730 tok/s; 3565 sec
  779. [2019-02-04 18:50:01,127 INFO] Step 34750/102583; acc: 81.05; ppl: 1.84; xent: 0.61; lr: 0.00010; 1946/2668 tok/s; 3571 sec
  780. [2019-02-04 18:50:06,126 INFO] Step 34800/102583; acc: 80.91; ppl: 1.84; xent: 0.61; lr: 0.00010; 2067/2663 tok/s; 3576 sec
  781. [2019-02-04 18:50:11,119 INFO] Step 34850/102583; acc: 82.72; ppl: 1.79; xent: 0.58; lr: 0.00010; 2123/2513 tok/s; 3581 sec
  782. [2019-02-04 18:50:16,112 INFO] Step 34900/102583; acc: 82.15; ppl: 1.79; xent: 0.58; lr: 0.00010; 2029/2670 tok/s; 3586 sec
  783. [2019-02-04 18:50:21,102 INFO] Step 34950/102583; acc: 80.56; ppl: 1.87; xent: 0.63; lr: 0.00010; 2050/2663 tok/s; 3590 sec
  784. [2019-02-04 18:50:25,731 INFO] Step 35000/102583; acc: 81.06; ppl: 1.88; xent: 0.63; lr: 0.00010; 2027/2627 tok/s; 3595 sec
  785. [2019-02-04 18:50:25,732 INFO] Saving checkpoint models/ted1_step_35000.pt
  786. [2019-02-04 18:50:36,997 INFO] Step 35050/102583; acc: 81.64; ppl: 1.82; xent: 0.60; lr: 0.00010; 927/1178 tok/s; 3606 sec
  787. [2019-02-04 18:50:42,090 INFO] Step 35100/102583; acc: 81.69; ppl: 1.81; xent: 0.59; lr: 0.00010; 2117/2511 tok/s; 3611 sec
  788. [2019-02-04 18:50:47,210 INFO] Step 35150/102583; acc: 80.93; ppl: 1.85; xent: 0.62; lr: 0.00010; 1754/2882 tok/s; 3617 sec
  789. [2019-02-04 18:50:52,075 INFO] Step 35200/102583; acc: 79.30; ppl: 1.97; xent: 0.68; lr: 0.00010; 2046/2469 tok/s; 3621 sec
  790. [2019-02-04 18:50:56,951 INFO] Step 35250/102583; acc: 83.06; ppl: 1.74; xent: 0.55; lr: 0.00010; 2193/2700 tok/s; 3626 sec
  791. [2019-02-04 18:51:01,709 INFO] Step 35300/102583; acc: 81.05; ppl: 1.82; xent: 0.60; lr: 0.00010; 1951/2716 tok/s; 3631 sec
  792. [2019-02-04 18:51:07,260 INFO] Step 35350/102583; acc: 81.33; ppl: 1.85; xent: 0.61; lr: 0.00010; 2086/2522 tok/s; 3637 sec
  793. [2019-02-04 18:51:11,979 INFO] Step 35400/102583; acc: 82.04; ppl: 1.82; xent: 0.60; lr: 0.00010; 2106/2651 tok/s; 3641 sec
  794. [2019-02-04 18:51:17,087 INFO] Step 35450/102583; acc: 81.65; ppl: 1.82; xent: 0.60; lr: 0.00010; 2170/2465 tok/s; 3646 sec
  795. [2019-02-04 18:51:22,257 INFO] Step 35500/102583; acc: 80.64; ppl: 1.88; xent: 0.63; lr: 0.00010; 1987/2672 tok/s; 3652 sec
  796. [2019-02-04 18:51:26,863 INFO] Step 35550/102583; acc: 81.57; ppl: 1.82; xent: 0.60; lr: 0.00010; 1949/2686 tok/s; 3656 sec
  797. [2019-02-04 18:51:31,704 INFO] Step 35600/102583; acc: 81.56; ppl: 1.83; xent: 0.61; lr: 0.00010; 2110/2573 tok/s; 3661 sec
  798. [2019-02-04 18:51:36,891 INFO] Step 35650/102583; acc: 80.81; ppl: 1.84; xent: 0.61; lr: 0.00010; 2006/2648 tok/s; 3666 sec
  799. [2019-02-04 18:51:46,029 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  800. [2019-02-04 18:51:48,154 INFO] Step 35700/102583; acc: 80.14; ppl: 1.92; xent: 0.65; lr: 0.00010; 869/1117 tok/s; 3678 sec
  801. [2019-02-04 18:51:53,261 INFO] Step 35750/102583; acc: 82.81; ppl: 1.78; xent: 0.58; lr: 0.00010; 2181/2541 tok/s; 3683 sec
  802. [2019-02-04 18:51:58,035 INFO] Step 35800/102583; acc: 81.54; ppl: 1.83; xent: 0.60; lr: 0.00010; 1886/2751 tok/s; 3687 sec
  803. [2019-02-04 18:52:03,228 INFO] Step 35850/102583; acc: 82.98; ppl: 1.75; xent: 0.56; lr: 0.00010; 2080/2640 tok/s; 3693 sec
  804. [2019-02-04 18:52:08,050 INFO] Step 35900/102583; acc: 81.44; ppl: 1.85; xent: 0.61; lr: 0.00010; 2138/2680 tok/s; 3697 sec
  805. [2019-02-04 18:52:13,217 INFO] Step 35950/102583; acc: 81.59; ppl: 1.80; xent: 0.59; lr: 0.00010; 2018/2701 tok/s; 3703 sec
  806. [2019-02-04 18:52:18,063 INFO] Step 36000/102583; acc: 80.51; ppl: 1.87; xent: 0.63; lr: 0.00010; 2078/2510 tok/s; 3707 sec
  807. [2019-02-04 18:52:22,841 INFO] Step 36050/102583; acc: 82.28; ppl: 1.80; xent: 0.59; lr: 0.00010; 2175/2765 tok/s; 3712 sec
  808. [2019-02-04 18:52:27,963 INFO] Step 36100/102583; acc: 79.25; ppl: 1.93; xent: 0.66; lr: 0.00010; 1966/2600 tok/s; 3717 sec
  809. [2019-02-04 18:52:32,865 INFO] Step 36150/102583; acc: 80.76; ppl: 1.90; xent: 0.64; lr: 0.00010; 1983/2630 tok/s; 3722 sec
  810. [2019-02-04 18:52:37,646 INFO] Step 36200/102583; acc: 82.33; ppl: 1.75; xent: 0.56; lr: 0.00010; 2200/2713 tok/s; 3727 sec
  811. [2019-02-04 18:52:42,857 INFO] Step 36250/102583; acc: 81.04; ppl: 1.88; xent: 0.63; lr: 0.00010; 1959/2556 tok/s; 3732 sec
  812. [2019-02-04 18:52:47,541 INFO] Step 36300/102583; acc: 80.96; ppl: 1.89; xent: 0.64; lr: 0.00010; 2141/2540 tok/s; 3737 sec
  813. [2019-02-04 18:52:52,576 INFO] Step 36350/102583; acc: 80.12; ppl: 1.88; xent: 0.63; lr: 0.00010; 1970/2766 tok/s; 3742 sec
  814. [2019-02-04 18:52:57,493 INFO] Step 36400/102583; acc: 82.82; ppl: 1.78; xent: 0.57; lr: 0.00010; 2090/2597 tok/s; 3747 sec
  815. [2019-02-04 18:53:02,562 INFO] Step 36450/102583; acc: 81.56; ppl: 1.83; xent: 0.60; lr: 0.00010; 2169/2622 tok/s; 3752 sec
  816. [2019-02-04 18:53:07,332 INFO] Step 36500/102583; acc: 81.30; ppl: 1.84; xent: 0.61; lr: 0.00010; 2144/2498 tok/s; 3757 sec
  817. [2019-02-04 18:53:12,356 INFO] Step 36550/102583; acc: 81.15; ppl: 1.83; xent: 0.60; lr: 0.00010; 1866/2696 tok/s; 3762 sec
  818. [2019-02-04 18:53:17,590 INFO] Step 36600/102583; acc: 82.37; ppl: 1.78; xent: 0.58; lr: 0.00010; 2051/2636 tok/s; 3767 sec
  819. [2019-02-04 18:53:22,807 INFO] Step 36650/102583; acc: 80.78; ppl: 1.88; xent: 0.63; lr: 0.00010; 2053/2469 tok/s; 3772 sec
  820. [2019-02-04 18:53:27,756 INFO] Step 36700/102583; acc: 79.36; ppl: 1.91; xent: 0.65; lr: 0.00010; 1895/2752 tok/s; 3777 sec
  821. [2019-02-04 18:53:32,918 INFO] Step 36750/102583; acc: 83.13; ppl: 1.74; xent: 0.55; lr: 0.00010; 1988/2584 tok/s; 3782 sec
  822. [2019-02-04 18:53:37,902 INFO] Step 36800/102583; acc: 81.14; ppl: 1.86; xent: 0.62; lr: 0.00010; 2216/2504 tok/s; 3787 sec
  823. [2019-02-04 18:53:42,541 INFO] Step 36850/102583; acc: 80.54; ppl: 1.89; xent: 0.64; lr: 0.00010; 2100/2644 tok/s; 3792 sec
  824. [2019-02-04 18:53:47,564 INFO] Step 36900/102583; acc: 80.70; ppl: 1.87; xent: 0.63; lr: 0.00010; 1984/2710 tok/s; 3797 sec
  825. [2019-02-04 18:53:52,604 INFO] Step 36950/102583; acc: 81.32; ppl: 1.84; xent: 0.61; lr: 0.00010; 2101/2611 tok/s; 3802 sec
  826. [2019-02-04 18:53:57,460 INFO] Step 37000/102583; acc: 82.49; ppl: 1.78; xent: 0.58; lr: 0.00010; 2132/2474 tok/s; 3807 sec
  827. [2019-02-04 18:54:02,694 INFO] Step 37050/102583; acc: 80.73; ppl: 1.85; xent: 0.61; lr: 0.00010; 1917/2665 tok/s; 3812 sec
  828. [2019-02-04 18:54:07,545 INFO] Step 37100/102583; acc: 80.80; ppl: 1.86; xent: 0.62; lr: 0.00010; 1865/2728 tok/s; 3817 sec
  829. [2019-02-04 18:54:12,569 INFO] Step 37150/102583; acc: 81.59; ppl: 1.80; xent: 0.59; lr: 0.00010; 2138/2601 tok/s; 3822 sec
  830. [2019-02-04 18:54:17,313 INFO] Step 37200/102583; acc: 81.47; ppl: 1.87; xent: 0.63; lr: 0.00010; 2021/2558 tok/s; 3827 sec
  831. [2019-02-04 18:54:22,359 INFO] Step 37250/102583; acc: 81.09; ppl: 1.85; xent: 0.62; lr: 0.00010; 2072/2527 tok/s; 3832 sec
  832. [2019-02-04 18:54:27,461 INFO] Step 37300/102583; acc: 81.14; ppl: 1.83; xent: 0.60; lr: 0.00010; 1993/2687 tok/s; 3837 sec
  833. [2019-02-04 18:54:32,324 INFO] Step 37350/102583; acc: 80.88; ppl: 1.86; xent: 0.62; lr: 0.00010; 2134/2698 tok/s; 3842 sec
  834. [2019-02-04 18:54:37,319 INFO] Step 37400/102583; acc: 81.80; ppl: 1.81; xent: 0.59; lr: 0.00010; 1996/2664 tok/s; 3847 sec
  835. [2019-02-04 18:54:42,250 INFO] Step 37450/102583; acc: 81.93; ppl: 1.83; xent: 0.61; lr: 0.00010; 2016/2651 tok/s; 3852 sec
  836. [2019-02-04 18:54:47,093 INFO] Step 37500/102583; acc: 80.08; ppl: 1.91; xent: 0.65; lr: 0.00010; 2017/2476 tok/s; 3856 sec
  837. [2019-02-04 18:54:52,137 INFO] Step 37550/102583; acc: 81.81; ppl: 1.79; xent: 0.58; lr: 0.00010; 2015/2764 tok/s; 3862 sec
  838. [2019-02-04 18:54:57,175 INFO] Step 37600/102583; acc: 81.32; ppl: 1.85; xent: 0.61; lr: 0.00010; 2103/2501 tok/s; 3867 sec
  839. [2019-02-04 18:55:02,326 INFO] Step 37650/102583; acc: 82.21; ppl: 1.81; xent: 0.59; lr: 0.00010; 2127/2495 tok/s; 3872 sec
  840. [2019-02-04 18:55:06,894 INFO] Step 37700/102583; acc: 81.41; ppl: 1.86; xent: 0.62; lr: 0.00010; 1903/2744 tok/s; 3876 sec
  841. [2019-02-04 18:55:11,876 INFO] Step 37750/102583; acc: 79.83; ppl: 1.91; xent: 0.65; lr: 0.00010; 2037/2646 tok/s; 3881 sec
  842. [2019-02-04 18:55:16,904 INFO] Step 37800/102583; acc: 82.21; ppl: 1.81; xent: 0.59; lr: 0.00010; 2002/2558 tok/s; 3886 sec
  843. [2019-02-04 18:55:21,611 INFO] Step 37850/102583; acc: 82.22; ppl: 1.78; xent: 0.58; lr: 0.00010; 2060/2693 tok/s; 3891 sec
  844. [2019-02-04 18:55:26,725 INFO] Step 37900/102583; acc: 81.33; ppl: 1.86; xent: 0.62; lr: 0.00010; 2178/2485 tok/s; 3896 sec
  845. [2019-02-04 18:55:31,762 INFO] Step 37950/102583; acc: 82.63; ppl: 1.75; xent: 0.56; lr: 0.00010; 2135/2603 tok/s; 3901 sec
  846. [2019-02-04 18:55:36,794 INFO] Step 38000/102583; acc: 80.64; ppl: 1.85; xent: 0.61; lr: 0.00010; 1785/2818 tok/s; 3906 sec
  847. [2019-02-04 18:55:41,785 INFO] Step 38050/102583; acc: 81.09; ppl: 1.85; xent: 0.61; lr: 0.00010; 2096/2640 tok/s; 3911 sec
  848. [2019-02-04 18:55:46,561 INFO] Step 38100/102583; acc: 83.04; ppl: 1.77; xent: 0.57; lr: 0.00010; 2200/2494 tok/s; 3916 sec
  849. [2019-02-04 18:55:51,484 INFO] Step 38150/102583; acc: 81.91; ppl: 1.81; xent: 0.59; lr: 0.00010; 1979/2656 tok/s; 3921 sec
  850. [2019-02-04 18:55:56,496 INFO] Step 38200/102583; acc: 80.06; ppl: 1.88; xent: 0.63; lr: 0.00010; 2105/2667 tok/s; 3926 sec
  851. [2019-02-04 18:56:01,176 INFO] Step 38250/102583; acc: 81.06; ppl: 1.88; xent: 0.63; lr: 0.00010; 2059/2625 tok/s; 3931 sec
  852. [2019-02-04 18:56:06,349 INFO] Step 38300/102583; acc: 82.41; ppl: 1.77; xent: 0.57; lr: 0.00010; 2014/2670 tok/s; 3936 sec
  853. [2019-02-04 18:56:11,380 INFO] Step 38350/102583; acc: 81.83; ppl: 1.79; xent: 0.58; lr: 0.00010; 2113/2561 tok/s; 3941 sec
  854. [2019-02-04 18:56:16,354 INFO] Step 38400/102583; acc: 81.21; ppl: 1.89; xent: 0.63; lr: 0.00010; 1793/2805 tok/s; 3946 sec
  855. [2019-02-04 18:56:21,309 INFO] Step 38450/102583; acc: 80.29; ppl: 1.92; xent: 0.65; lr: 0.00010; 1983/2506 tok/s; 3951 sec
  856. [2019-02-04 18:56:26,162 INFO] Step 38500/102583; acc: 82.57; ppl: 1.75; xent: 0.56; lr: 0.00010; 2190/2704 tok/s; 3956 sec
  857. [2019-02-04 18:56:31,201 INFO] Step 38550/102583; acc: 81.43; ppl: 1.81; xent: 0.60; lr: 0.00010; 1997/2537 tok/s; 3961 sec
  858. [2019-02-04 18:56:36,334 INFO] Step 38600/102583; acc: 81.74; ppl: 1.80; xent: 0.59; lr: 0.00010; 2112/2680 tok/s; 3966 sec
  859. [2019-02-04 18:56:41,228 INFO] Step 38650/102583; acc: 81.00; ppl: 1.88; xent: 0.63; lr: 0.00010; 2068/2666 tok/s; 3971 sec
  860. [2019-02-04 18:56:46,310 INFO] Step 38700/102583; acc: 81.83; ppl: 1.80; xent: 0.59; lr: 0.00010; 2076/2473 tok/s; 3976 sec
  861. [2019-02-04 18:56:51,232 INFO] Step 38750/102583; acc: 81.17; ppl: 1.86; xent: 0.62; lr: 0.00010; 2133/2594 tok/s; 3981 sec
  862. [2019-02-04 18:56:56,065 INFO] Step 38800/102583; acc: 81.25; ppl: 1.83; xent: 0.61; lr: 0.00010; 1960/2658 tok/s; 3985 sec
  863. [2019-02-04 18:57:00,827 INFO] Step 38850/102583; acc: 81.37; ppl: 1.85; xent: 0.61; lr: 0.00010; 1988/2690 tok/s; 3990 sec
  864. [2019-02-04 18:57:06,104 INFO] Step 38900/102583; acc: 81.12; ppl: 1.82; xent: 0.60; lr: 0.00010; 1991/2586 tok/s; 3995 sec
  865. [2019-02-04 18:57:13,787 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  866. [2019-02-04 18:57:16,574 INFO] Step 38950/102583; acc: 80.82; ppl: 1.87; xent: 0.63; lr: 0.00010; 972/1219 tok/s; 4006 sec
  867. [2019-02-04 18:57:21,558 INFO] Step 39000/102583; acc: 82.34; ppl: 1.81; xent: 0.60; lr: 0.00010; 2172/2458 tok/s; 4011 sec
  868. [2019-02-04 18:57:26,317 INFO] Step 39050/102583; acc: 81.88; ppl: 1.80; xent: 0.59; lr: 0.00010; 1968/2764 tok/s; 4016 sec
  869. [2019-02-04 18:57:31,427 INFO] Step 39100/102583; acc: 82.69; ppl: 1.76; xent: 0.57; lr: 0.00010; 2008/2692 tok/s; 4021 sec
  870. [2019-02-04 18:57:36,423 INFO] Step 39150/102583; acc: 80.47; ppl: 1.90; xent: 0.64; lr: 0.00010; 2089/2756 tok/s; 4026 sec
  871. [2019-02-04 18:57:41,512 INFO] Step 39200/102583; acc: 81.91; ppl: 1.77; xent: 0.57; lr: 0.00010; 2036/2651 tok/s; 4031 sec
  872. [2019-02-04 18:57:46,350 INFO] Step 39250/102583; acc: 81.62; ppl: 1.83; xent: 0.60; lr: 0.00010; 2161/2502 tok/s; 4036 sec
  873. [2019-02-04 18:57:51,025 INFO] Step 39300/102583; acc: 81.82; ppl: 1.84; xent: 0.61; lr: 0.00010; 2112/2724 tok/s; 4040 sec
  874. [2019-02-04 18:57:56,222 INFO] Step 39350/102583; acc: 79.84; ppl: 1.89; xent: 0.64; lr: 0.00010; 1982/2567 tok/s; 4046 sec
  875. [2019-02-04 18:58:01,233 INFO] Step 39400/102583; acc: 81.62; ppl: 1.85; xent: 0.61; lr: 0.00010; 1977/2671 tok/s; 4051 sec
  876. [2019-02-04 18:58:06,034 INFO] Step 39450/102583; acc: 82.39; ppl: 1.76; xent: 0.56; lr: 0.00010; 2163/2725 tok/s; 4055 sec
  877. [2019-02-04 18:58:11,195 INFO] Step 39500/102583; acc: 80.21; ppl: 1.92; xent: 0.65; lr: 0.00010; 1973/2482 tok/s; 4061 sec
  878. [2019-02-04 18:58:15,822 INFO] Step 39550/102583; acc: 80.87; ppl: 1.90; xent: 0.64; lr: 0.00010; 2146/2625 tok/s; 4065 sec
  879. [2019-02-04 18:58:20,879 INFO] Step 39600/102583; acc: 80.97; ppl: 1.82; xent: 0.60; lr: 0.00010; 2014/2772 tok/s; 4070 sec
  880. [2019-02-04 18:58:25,992 INFO] Step 39650/102583; acc: 83.05; ppl: 1.76; xent: 0.56; lr: 0.00010; 2116/2529 tok/s; 4075 sec
  881. [2019-02-04 18:58:30,940 INFO] Step 39700/102583; acc: 80.80; ppl: 1.86; xent: 0.62; lr: 0.00010; 2077/2711 tok/s; 4080 sec
  882. [2019-02-04 18:58:35,818 INFO] Step 39750/102583; acc: 82.36; ppl: 1.80; xent: 0.59; lr: 0.00010; 2202/2455 tok/s; 4085 sec
  883. [2019-02-04 18:58:40,895 INFO] Step 39800/102583; acc: 81.70; ppl: 1.79; xent: 0.58; lr: 0.00010; 1857/2762 tok/s; 4090 sec
  884. [2019-02-04 18:58:46,147 INFO] Step 39850/102583; acc: 80.99; ppl: 1.85; xent: 0.61; lr: 0.00010; 2050/2479 tok/s; 4096 sec
  885. [2019-02-04 18:58:51,177 INFO] Step 39900/102583; acc: 81.70; ppl: 1.81; xent: 0.60; lr: 0.00010; 2075/2595 tok/s; 4101 sec
  886. [2019-02-04 18:58:56,222 INFO] Step 39950/102583; acc: 78.99; ppl: 1.95; xent: 0.67; lr: 0.00010; 1854/2716 tok/s; 4106 sec
  887. [2019-02-04 18:59:01,278 INFO] Step 40000/102583; acc: 83.44; ppl: 1.72; xent: 0.54; lr: 0.00010; 2033/2579 tok/s; 4111 sec
  888. [2019-02-04 18:59:01,509 INFO] Loading dataset from data/ted.valid.0.pt, number of examples: 873
  889. [2019-02-04 18:59:02,811 INFO] Validation perplexity: 1.74125
  890. [2019-02-04 18:59:02,830 INFO] Validation accuracy: 83.1339
  891. [2019-02-04 18:59:02,866 INFO] Saving checkpoint models/ted1_step_40000.pt
  892. [2019-02-04 18:59:16,913 INFO] Step 40050/102583; acc: 80.64; ppl: 1.87; xent: 0.63; lr: 0.00010; 682/809 tok/s; 4126 sec
  893. [2019-02-04 18:59:21,487 INFO] Step 40100/102583; acc: 80.60; ppl: 1.89; xent: 0.64; lr: 0.00010; 2144/2615 tok/s; 4131 sec
  894. [2019-02-04 18:59:26,541 INFO] Step 40150/102583; acc: 80.90; ppl: 1.85; xent: 0.62; lr: 0.00010; 2034/2722 tok/s; 4136 sec
  895. [2019-02-04 18:59:31,709 INFO] Step 40200/102583; acc: 82.18; ppl: 1.80; xent: 0.59; lr: 0.00010; 2052/2678 tok/s; 4141 sec
  896. [2019-02-04 18:59:36,783 INFO] Step 40250/102583; acc: 83.01; ppl: 1.76; xent: 0.56; lr: 0.00010; 2223/2327 tok/s; 4146 sec
  897. [2019-02-04 18:59:41,820 INFO] Step 40300/102583; acc: 79.46; ppl: 1.91; xent: 0.65; lr: 0.00010; 1798/2827 tok/s; 4151 sec
  898. [2019-02-04 18:59:46,659 INFO] Step 40350/102583; acc: 80.50; ppl: 1.85; xent: 0.62; lr: 0.00010; 1870/2718 tok/s; 4156 sec
  899. [2019-02-04 18:59:51,740 INFO] Step 40400/102583; acc: 83.59; ppl: 1.71; xent: 0.54; lr: 0.00010; 2241/2526 tok/s; 4161 sec
  900. [2019-02-04 18:59:56,422 INFO] Step 40450/102583; acc: 80.07; ppl: 1.94; xent: 0.66; lr: 0.00010; 1950/2606 tok/s; 4166 sec
  901. [2019-02-04 19:00:01,581 INFO] Step 40500/102583; acc: 81.50; ppl: 1.81; xent: 0.60; lr: 0.00010; 2057/2537 tok/s; 4171 sec
  902. [2019-02-04 19:00:06,701 INFO] Step 40550/102583; acc: 81.28; ppl: 1.82; xent: 0.60; lr: 0.00010; 2054/2741 tok/s; 4176 sec
  903. [2019-02-04 19:00:11,480 INFO] Step 40600/102583; acc: 80.88; ppl: 1.87; xent: 0.63; lr: 0.00010; 2040/2614 tok/s; 4181 sec
  904. [2019-02-04 19:00:16,483 INFO] Step 40650/102583; acc: 82.13; ppl: 1.79; xent: 0.58; lr: 0.00010; 2013/2743 tok/s; 4186 sec
  905. [2019-02-04 19:00:21,386 INFO] Step 40700/102583; acc: 81.89; ppl: 1.82; xent: 0.60; lr: 0.00010; 2067/2615 tok/s; 4191 sec
  906. [2019-02-04 19:00:26,421 INFO] Step 40750/102583; acc: 80.43; ppl: 1.89; xent: 0.64; lr: 0.00010; 1932/2519 tok/s; 4196 sec
  907. [2019-02-04 19:00:31,287 INFO] Step 40800/102583; acc: 81.26; ppl: 1.82; xent: 0.60; lr: 0.00010; 2140/2706 tok/s; 4201 sec
  908. [2019-02-04 19:00:36,395 INFO] Step 40850/102583; acc: 81.35; ppl: 1.83; xent: 0.61; lr: 0.00010; 2016/2549 tok/s; 4206 sec
  909. [2019-02-04 19:00:41,493 INFO] Step 40900/102583; acc: 82.91; ppl: 1.78; xent: 0.57; lr: 0.00010; 2186/2470 tok/s; 4211 sec
  910. [2019-02-04 19:00:46,150 INFO] Step 40950/102583; acc: 80.46; ppl: 1.92; xent: 0.65; lr: 0.00010; 1846/2798 tok/s; 4216 sec
  911. [2019-02-04 19:00:51,273 INFO] Step 41000/102583; acc: 81.09; ppl: 1.84; xent: 0.61; lr: 0.00010; 2102/2573 tok/s; 4221 sec
  912. [2019-02-04 19:00:56,109 INFO] Step 41050/102583; acc: 81.58; ppl: 1.86; xent: 0.62; lr: 0.00010; 1991/2587 tok/s; 4225 sec
  913. [2019-02-04 19:01:01,014 INFO] Step 41100/102583; acc: 83.15; ppl: 1.72; xent: 0.54; lr: 0.00010; 2090/2640 tok/s; 4230 sec
  914. [2019-02-04 19:01:05,993 INFO] Step 41150/102583; acc: 80.95; ppl: 1.89; xent: 0.63; lr: 0.00010; 2173/2499 tok/s; 4235 sec
  915. [2019-02-04 19:01:10,933 INFO] Step 41200/102583; acc: 82.56; ppl: 1.75; xent: 0.56; lr: 0.00010; 2117/2609 tok/s; 4240 sec
  916. [2019-02-04 19:01:16,017 INFO] Step 41250/102583; acc: 80.83; ppl: 1.84; xent: 0.61; lr: 0.00010; 1793/2830 tok/s; 4245 sec
  917. [2019-02-04 19:01:21,172 INFO] Step 41300/102583; acc: 81.94; ppl: 1.80; xent: 0.59; lr: 0.00010; 2104/2574 tok/s; 4251 sec
  918. [2019-02-04 19:01:25,727 INFO] Step 41350/102583; acc: 82.09; ppl: 1.82; xent: 0.60; lr: 0.00010; 2110/2588 tok/s; 4255 sec
  919. [2019-02-04 19:01:30,784 INFO] Step 41400/102583; acc: 81.96; ppl: 1.79; xent: 0.58; lr: 0.00010; 2052/2637 tok/s; 4260 sec
  920. [2019-02-04 19:01:35,749 INFO] Step 41450/102583; acc: 79.84; ppl: 1.89; xent: 0.64; lr: 0.00010; 2055/2658 tok/s; 4265 sec
  921. [2019-02-04 19:01:40,546 INFO] Step 41500/102583; acc: 81.28; ppl: 1.87; xent: 0.62; lr: 0.00010; 1978/2706 tok/s; 4270 sec
  922. [2019-02-04 19:01:45,463 INFO] Step 41550/102583; acc: 83.21; ppl: 1.74; xent: 0.56; lr: 0.00010; 2138/2586 tok/s; 4275 sec
  923. [2019-02-04 19:01:50,762 INFO] Step 41600/102583; acc: 82.27; ppl: 1.76; xent: 0.56; lr: 0.00010; 2060/2589 tok/s; 4280 sec
  924. [2019-02-04 19:01:55,773 INFO] Step 41650/102583; acc: 80.20; ppl: 1.92; xent: 0.65; lr: 0.00010; 1798/2814 tok/s; 4285 sec
  925. [2019-02-04 19:02:00,415 INFO] Step 41700/102583; acc: 80.53; ppl: 1.90; xent: 0.64; lr: 0.00010; 2090/2455 tok/s; 4290 sec
  926. [2019-02-04 19:02:05,470 INFO] Step 41750/102583; acc: 82.66; ppl: 1.74; xent: 0.55; lr: 0.00010; 2129/2733 tok/s; 4295 sec
  927. [2019-02-04 19:02:10,509 INFO] Step 41800/102583; acc: 81.84; ppl: 1.81; xent: 0.59; lr: 0.00010; 1968/2523 tok/s; 4300 sec
  928. [2019-02-04 19:02:15,471 INFO] Step 41850/102583; acc: 82.30; ppl: 1.78; xent: 0.58; lr: 0.00010; 2161/2705 tok/s; 4305 sec
  929. [2019-02-04 19:02:20,558 INFO] Step 41900/102583; acc: 81.17; ppl: 1.85; xent: 0.62; lr: 0.00010; 2112/2578 tok/s; 4310 sec
  930. [2019-02-04 19:02:25,803 INFO] Step 41950/102583; acc: 82.22; ppl: 1.78; xent: 0.58; lr: 0.00010; 2078/2489 tok/s; 4315 sec
  931. [2019-02-04 19:02:30,331 INFO] Step 42000/102583; acc: 80.75; ppl: 1.90; xent: 0.64; lr: 0.00010; 2047/2655 tok/s; 4320 sec
  932. [2019-02-04 19:02:35,182 INFO] Step 42050/102583; acc: 81.55; ppl: 1.81; xent: 0.59; lr: 0.00010; 2058/2641 tok/s; 4325 sec
  933. [2019-02-04 19:02:40,102 INFO] Step 42100/102583; acc: 81.82; ppl: 1.82; xent: 0.60; lr: 0.00010; 1967/2672 tok/s; 4329 sec
  934. [2019-02-04 19:02:45,421 INFO] Step 42150/102583; acc: 80.81; ppl: 1.83; xent: 0.60; lr: 0.00010; 1928/2534 tok/s; 4335 sec
  935. [2019-02-04 19:02:52,420 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  936. [2019-02-04 19:02:55,957 INFO] Step 42200/102583; acc: 81.88; ppl: 1.82; xent: 0.60; lr: 0.00010; 980/1289 tok/s; 4345 sec
  937. [2019-02-04 19:03:00,697 INFO] Step 42250/102583; acc: 81.10; ppl: 1.89; xent: 0.64; lr: 0.00010; 2221/2401 tok/s; 4350 sec
  938. [2019-02-04 19:03:05,500 INFO] Step 42300/102583; acc: 81.99; ppl: 1.80; xent: 0.59; lr: 0.00010; 1937/2801 tok/s; 4355 sec
  939. [2019-02-04 19:03:10,711 INFO] Step 42350/102583; acc: 82.47; ppl: 1.78; xent: 0.58; lr: 0.00010; 1987/2662 tok/s; 4360 sec
  940. [2019-02-04 19:03:15,559 INFO] Step 42400/102583; acc: 81.87; ppl: 1.84; xent: 0.61; lr: 0.00010; 2134/2776 tok/s; 4365 sec
  941. [2019-02-04 19:03:20,565 INFO] Step 42450/102583; acc: 81.59; ppl: 1.80; xent: 0.59; lr: 0.00010; 2019/2608 tok/s; 4370 sec
  942. [2019-02-04 19:03:25,552 INFO] Step 42500/102583; acc: 81.60; ppl: 1.81; xent: 0.60; lr: 0.00010; 2220/2502 tok/s; 4375 sec
  943. [2019-02-04 19:03:30,156 INFO] Step 42550/102583; acc: 81.27; ppl: 1.86; xent: 0.62; lr: 0.00010; 2080/2776 tok/s; 4380 sec
  944. [2019-02-04 19:03:35,416 INFO] Step 42600/102583; acc: 80.56; ppl: 1.87; xent: 0.63; lr: 0.00010; 1922/2622 tok/s; 4385 sec
  945. [2019-02-04 19:03:40,363 INFO] Step 42650/102583; acc: 81.07; ppl: 1.87; xent: 0.63; lr: 0.00010; 2080/2595 tok/s; 4390 sec
  946. [2019-02-04 19:03:45,265 INFO] Step 42700/102583; acc: 83.07; ppl: 1.71; xent: 0.54; lr: 0.00010; 2123/2713 tok/s; 4395 sec
  947. [2019-02-04 19:03:50,333 INFO] Step 42750/102583; acc: 79.72; ppl: 1.97; xent: 0.68; lr: 0.00010; 1993/2496 tok/s; 4400 sec
  948. [2019-02-04 19:03:55,166 INFO] Step 42800/102583; acc: 81.99; ppl: 1.83; xent: 0.60; lr: 0.00010; 2098/2620 tok/s; 4405 sec
  949. [2019-02-04 19:04:00,086 INFO] Step 42850/102583; acc: 80.73; ppl: 1.84; xent: 0.61; lr: 0.00010; 2016/2753 tok/s; 4409 sec
  950. [2019-02-04 19:04:05,081 INFO] Step 42900/102583; acc: 83.22; ppl: 1.75; xent: 0.56; lr: 0.00010; 2146/2537 tok/s; 4414 sec
  951. [2019-02-04 19:04:10,200 INFO] Step 42950/102583; acc: 80.79; ppl: 1.85; xent: 0.61; lr: 0.00010; 2055/2684 tok/s; 4420 sec
  952. [2019-02-04 19:04:14,837 INFO] Step 43000/102583; acc: 81.86; ppl: 1.83; xent: 0.61; lr: 0.00010; 2153/2521 tok/s; 4424 sec
  953. [2019-02-04 19:04:19,850 INFO] Step 43050/102583; acc: 81.90; ppl: 1.76; xent: 0.57; lr: 0.00010; 1881/2729 tok/s; 4429 sec
  954. [2019-02-04 19:04:25,170 INFO] Step 43100/102583; acc: 80.99; ppl: 1.85; xent: 0.61; lr: 0.00010; 1972/2486 tok/s; 4435 sec
  955. [2019-02-04 19:04:30,167 INFO] Step 43150/102583; acc: 82.26; ppl: 1.79; xent: 0.58; lr: 0.00010; 2179/2576 tok/s; 4440 sec
  956. [2019-02-04 19:04:35,441 INFO] Step 43200/102583; acc: 79.87; ppl: 1.88; xent: 0.63; lr: 0.00010; 1906/2670 tok/s; 4445 sec
  957. [2019-02-04 19:04:40,213 INFO] Step 43250/102583; acc: 82.59; ppl: 1.79; xent: 0.58; lr: 0.00010; 1987/2648 tok/s; 4450 sec
  958. [2019-02-04 19:04:45,229 INFO] Step 43300/102583; acc: 79.43; ppl: 1.92; xent: 0.65; lr: 0.00010; 2062/2562 tok/s; 4455 sec
  959. [2019-02-04 19:04:49,845 INFO] Step 43350/102583; acc: 81.51; ppl: 1.85; xent: 0.62; lr: 0.00010; 2257/2529 tok/s; 4459 sec
  960. [2019-02-04 19:04:54,829 INFO] Step 43400/102583; acc: 80.81; ppl: 1.85; xent: 0.61; lr: 0.00010; 1954/2783 tok/s; 4464 sec
  961. [2019-02-04 19:04:59,971 INFO] Step 43450/102583; acc: 82.52; ppl: 1.77; xent: 0.57; lr: 0.00010; 2060/2661 tok/s; 4469 sec
  962. [2019-02-04 19:05:05,252 INFO] Step 43500/102583; acc: 82.24; ppl: 1.79; xent: 0.58; lr: 0.00010; 2096/2292 tok/s; 4475 sec
  963. [2019-02-04 19:05:10,049 INFO] Step 43550/102583; acc: 80.12; ppl: 1.87; xent: 0.62; lr: 0.00010; 1902/2890 tok/s; 4479 sec
  964. [2019-02-04 19:05:14,857 INFO] Step 43600/102583; acc: 80.86; ppl: 1.86; xent: 0.62; lr: 0.00010; 1891/2700 tok/s; 4484 sec
  965. [2019-02-04 19:05:19,902 INFO] Step 43650/102583; acc: 83.30; ppl: 1.73; xent: 0.55; lr: 0.00010; 2260/2514 tok/s; 4489 sec
  966. [2019-02-04 19:05:24,678 INFO] Step 43700/102583; acc: 79.91; ppl: 1.94; xent: 0.66; lr: 0.00010; 1863/2669 tok/s; 4494 sec
  967. [2019-02-04 19:05:29,855 INFO] Step 43750/102583; acc: 82.23; ppl: 1.77; xent: 0.57; lr: 0.00010; 2124/2522 tok/s; 4499 sec
  968. [2019-02-04 19:05:34,859 INFO] Step 43800/102583; acc: 81.18; ppl: 1.84; xent: 0.61; lr: 0.00010; 2091/2696 tok/s; 4504 sec
  969. [2019-02-04 19:05:39,681 INFO] Step 43850/102583; acc: 81.03; ppl: 1.85; xent: 0.61; lr: 0.00010; 1972/2646 tok/s; 4509 sec
  970. [2019-02-04 19:05:44,650 INFO] Step 43900/102583; acc: 81.79; ppl: 1.81; xent: 0.59; lr: 0.00010; 2001/2719 tok/s; 4514 sec
  971. [2019-02-04 19:05:49,707 INFO] Step 43950/102583; acc: 82.86; ppl: 1.76; xent: 0.56; lr: 0.00010; 2206/2572 tok/s; 4519 sec
  972. [2019-02-04 19:05:54,648 INFO] Step 44000/102583; acc: 79.19; ppl: 1.96; xent: 0.67; lr: 0.00010; 1806/2509 tok/s; 4524 sec
  973. [2019-02-04 19:05:59,576 INFO] Step 44050/102583; acc: 82.29; ppl: 1.77; xent: 0.57; lr: 0.00010; 2223/2665 tok/s; 4529 sec
  974. [2019-02-04 19:06:04,473 INFO] Step 44100/102583; acc: 80.66; ppl: 1.89; xent: 0.64; lr: 0.00010; 1938/2598 tok/s; 4534 sec
  975. [2019-02-04 19:06:09,646 INFO] Step 44150/102583; acc: 83.56; ppl: 1.73; xent: 0.55; lr: 0.00010; 2182/2500 tok/s; 4539 sec
  976. [2019-02-04 19:06:14,353 INFO] Step 44200/102583; acc: 79.86; ppl: 1.94; xent: 0.66; lr: 0.00010; 1881/2731 tok/s; 4544 sec
  977. [2019-02-04 19:06:19,569 INFO] Step 44250/102583; acc: 82.06; ppl: 1.78; xent: 0.58; lr: 0.00010; 2045/2601 tok/s; 4549 sec
  978. [2019-02-04 19:06:24,281 INFO] Step 44300/102583; acc: 80.95; ppl: 1.90; xent: 0.64; lr: 0.00010; 2020/2618 tok/s; 4554 sec
  979. [2019-02-04 19:06:29,368 INFO] Step 44350/102583; acc: 83.44; ppl: 1.70; xent: 0.53; lr: 0.00010; 2043/2593 tok/s; 4559 sec
  980. [2019-02-04 19:06:34,172 INFO] Step 44400/102583; acc: 80.33; ppl: 1.90; xent: 0.64; lr: 0.00010; 2182/2554 tok/s; 4564 sec
  981. [2019-02-04 19:06:39,232 INFO] Step 44450/102583; acc: 82.73; ppl: 1.74; xent: 0.55; lr: 0.00010; 2102/2582 tok/s; 4569 sec
  982. [2019-02-04 19:06:44,326 INFO] Step 44500/102583; acc: 80.82; ppl: 1.82; xent: 0.60; lr: 0.00010; 1910/2760 tok/s; 4574 sec
  983. [2019-02-04 19:06:49,278 INFO] Step 44550/102583; acc: 81.46; ppl: 1.85; xent: 0.62; lr: 0.00010; 2030/2627 tok/s; 4579 sec
  984. [2019-02-04 19:06:54,067 INFO] Step 44600/102583; acc: 83.64; ppl: 1.72; xent: 0.54; lr: 0.00010; 2152/2578 tok/s; 4583 sec
  985. [2019-02-04 19:06:59,038 INFO] Step 44650/102583; acc: 81.24; ppl: 1.83; xent: 0.60; lr: 0.00010; 2039/2650 tok/s; 4588 sec
  986. [2019-02-04 19:07:03,870 INFO] Step 44700/102583; acc: 79.74; ppl: 1.90; xent: 0.64; lr: 0.00010; 2069/2657 tok/s; 4593 sec
  987. [2019-02-04 19:07:08,903 INFO] Step 44750/102583; acc: 82.17; ppl: 1.81; xent: 0.59; lr: 0.00010; 1936/2657 tok/s; 4598 sec
  988. [2019-02-04 19:07:13,578 INFO] Step 44800/102583; acc: 82.19; ppl: 1.79; xent: 0.58; lr: 0.00010; 2125/2639 tok/s; 4603 sec
  989. [2019-02-04 19:07:18,894 INFO] Step 44850/102583; acc: 82.07; ppl: 1.77; xent: 0.57; lr: 0.00010; 2053/2582 tok/s; 4608 sec
  990. [2019-02-04 19:07:24,053 INFO] Step 44900/102583; acc: 80.10; ppl: 1.93; xent: 0.65; lr: 0.00010; 1849/2728 tok/s; 4613 sec
  991. [2019-02-04 19:07:28,744 INFO] Step 44950/102583; acc: 81.30; ppl: 1.86; xent: 0.62; lr: 0.00010; 2139/2550 tok/s; 4618 sec
  992. [2019-02-04 19:07:33,648 INFO] Step 45000/102583; acc: 82.40; ppl: 1.75; xent: 0.56; lr: 0.00010; 2008/2755 tok/s; 4623 sec
  993. [2019-02-04 19:07:33,650 INFO] Saving checkpoint models/ted1_step_45000.pt
  994. [2019-02-04 19:07:44,361 INFO] Step 45050/102583; acc: 82.21; ppl: 1.80; xent: 0.59; lr: 0.00010; 931/1226 tok/s; 4634 sec
  995. [2019-02-04 19:07:49,252 INFO] Step 45100/102583; acc: 81.06; ppl: 1.84; xent: 0.61; lr: 0.00010; 2179/2643 tok/s; 4639 sec
  996. [2019-02-04 19:07:54,297 INFO] Step 45150/102583; acc: 81.70; ppl: 1.83; xent: 0.60; lr: 0.00010; 2155/2565 tok/s; 4644 sec
  997. [2019-02-04 19:07:59,707 INFO] Step 45200/102583; acc: 83.20; ppl: 1.74; xent: 0.55; lr: 0.00010; 2028/2510 tok/s; 4649 sec
  998. [2019-02-04 19:08:04,136 INFO] Step 45250/102583; acc: 80.15; ppl: 1.91; xent: 0.65; lr: 0.00010; 2105/2666 tok/s; 4654 sec
  999. [2019-02-04 19:08:08,877 INFO] Step 45300/102583; acc: 81.03; ppl: 1.83; xent: 0.60; lr: 0.00010; 2052/2587 tok/s; 4658 sec
  1000. [2019-02-04 19:08:14,222 INFO] Step 45350/102583; acc: 82.37; ppl: 1.80; xent: 0.59; lr: 0.00010; 1973/2576 tok/s; 4664 sec
  1001. [2019-02-04 19:08:19,402 INFO] Step 45400/102583; acc: 80.78; ppl: 1.81; xent: 0.60; lr: 0.00010; 1923/2647 tok/s; 4669 sec
  1002. [2019-02-04 19:08:26,453 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  1003. [2019-02-04 19:08:30,701 INFO] Step 45450/102583; acc: 81.84; ppl: 1.82; xent: 0.60; lr: 0.00010; 955/1147 tok/s; 4680 sec
  1004. [2019-02-04 19:08:35,245 INFO] Step 45500/102583; acc: 80.33; ppl: 1.93; xent: 0.66; lr: 0.00010; 2100/2545 tok/s; 4685 sec
  1005. [2019-02-04 19:08:40,207 INFO] Step 45550/102583; acc: 82.83; ppl: 1.75; xent: 0.56; lr: 0.00010; 1985/2776 tok/s; 4690 sec
  1006. [2019-02-04 19:08:45,456 INFO] Step 45600/102583; acc: 81.74; ppl: 1.81; xent: 0.59; lr: 0.00010; 1984/2643 tok/s; 4695 sec
  1007. [2019-02-04 19:08:50,165 INFO] Step 45650/102583; acc: 81.75; ppl: 1.82; xent: 0.60; lr: 0.00010; 2109/2820 tok/s; 4700 sec
  1008. [2019-02-04 19:08:55,305 INFO] Step 45700/102583; acc: 81.64; ppl: 1.78; xent: 0.58; lr: 0.00010; 2098/2514 tok/s; 4705 sec
  1009. [2019-02-04 19:09:00,171 INFO] Step 45750/102583; acc: 82.44; ppl: 1.78; xent: 0.58; lr: 0.00010; 2176/2612 tok/s; 4710 sec
  1010. [2019-02-04 19:09:04,986 INFO] Step 45800/102583; acc: 80.77; ppl: 1.88; xent: 0.63; lr: 0.00010; 2138/2646 tok/s; 4714 sec
  1011. [2019-02-04 19:09:10,226 INFO] Step 45850/102583; acc: 80.16; ppl: 1.88; xent: 0.63; lr: 0.00010; 1768/2662 tok/s; 4720 sec
  1012. [2019-02-04 19:09:15,101 INFO] Step 45900/102583; acc: 81.94; ppl: 1.82; xent: 0.60; lr: 0.00010; 2238/2604 tok/s; 4724 sec
  1013. [2019-02-04 19:09:20,218 INFO] Step 45950/102583; acc: 82.56; ppl: 1.72; xent: 0.54; lr: 0.00010; 1908/2738 tok/s; 4730 sec
  1014. [2019-02-04 19:09:25,077 INFO] Step 46000/102583; acc: 80.56; ppl: 1.91; xent: 0.65; lr: 0.00010; 2186/2515 tok/s; 4734 sec
  1015. [2019-02-04 19:09:29,911 INFO] Step 46050/102583; acc: 81.89; ppl: 1.82; xent: 0.60; lr: 0.00010; 2060/2621 tok/s; 4739 sec
  1016. [2019-02-04 19:09:34,732 INFO] Step 46100/102583; acc: 80.00; ppl: 1.89; xent: 0.63; lr: 0.00010; 1987/2755 tok/s; 4744 sec
  1017. [2019-02-04 19:09:39,960 INFO] Step 46150/102583; acc: 83.68; ppl: 1.72; xent: 0.54; lr: 0.00010; 2220/2507 tok/s; 4749 sec
  1018. [2019-02-04 19:09:44,974 INFO] Step 46200/102583; acc: 80.61; ppl: 1.86; xent: 0.62; lr: 0.00010; 1959/2735 tok/s; 4754 sec
  1019. [2019-02-04 19:09:49,510 INFO] Step 46250/102583; acc: 81.72; ppl: 1.84; xent: 0.61; lr: 0.00010; 2116/2517 tok/s; 4759 sec
  1020. [2019-02-04 19:09:54,658 INFO] Step 46300/102583; acc: 82.24; ppl: 1.75; xent: 0.56; lr: 0.00010; 1891/2689 tok/s; 4764 sec
  1021. [2019-02-04 19:09:59,769 INFO] Step 46350/102583; acc: 81.20; ppl: 1.84; xent: 0.61; lr: 0.00010; 2054/2452 tok/s; 4769 sec
  1022. [2019-02-04 19:10:04,832 INFO] Step 46400/102583; acc: 81.51; ppl: 1.82; xent: 0.60; lr: 0.00010; 2112/2618 tok/s; 4774 sec
  1023. [2019-02-04 19:10:10,207 INFO] Step 46450/102583; acc: 81.00; ppl: 1.82; xent: 0.60; lr: 0.00010; 1866/2730 tok/s; 4780 sec
  1024. [2019-02-04 19:10:14,908 INFO] Step 46500/102583; acc: 82.80; ppl: 1.77; xent: 0.57; lr: 0.00010; 2099/2573 tok/s; 4784 sec
  1025. [2019-02-04 19:10:19,944 INFO] Step 46550/102583; acc: 79.70; ppl: 1.91; xent: 0.65; lr: 0.00010; 2113/2559 tok/s; 4789 sec
  1026. [2019-02-04 19:10:24,705 INFO] Step 46600/102583; acc: 81.28; ppl: 1.86; xent: 0.62; lr: 0.00010; 2108/2648 tok/s; 4794 sec
  1027. [2019-02-04 19:10:29,509 INFO] Step 46650/102583; acc: 80.36; ppl: 1.88; xent: 0.63; lr: 0.00010; 2040/2656 tok/s; 4799 sec
  1028. [2019-02-04 19:10:34,837 INFO] Step 46700/102583; acc: 82.70; ppl: 1.77; xent: 0.57; lr: 0.00010; 1911/2592 tok/s; 4804 sec
  1029. [2019-02-04 19:10:39,972 INFO] Step 46750/102583; acc: 82.32; ppl: 1.78; xent: 0.57; lr: 0.00010; 2186/2388 tok/s; 4809 sec
  1030. [2019-02-04 19:10:44,745 INFO] Step 46800/102583; acc: 80.26; ppl: 1.86; xent: 0.62; lr: 0.00010; 1938/2872 tok/s; 4814 sec
  1031. [2019-02-04 19:10:49,587 INFO] Step 46850/102583; acc: 81.01; ppl: 1.84; xent: 0.61; lr: 0.00010; 1948/2701 tok/s; 4819 sec
  1032. [2019-02-04 19:10:54,622 INFO] Step 46900/102583; acc: 83.03; ppl: 1.75; xent: 0.56; lr: 0.00010; 2206/2486 tok/s; 4824 sec
  1033. [2019-02-04 19:10:59,417 INFO] Step 46950/102583; acc: 80.41; ppl: 1.91; xent: 0.65; lr: 0.00010; 1805/2672 tok/s; 4829 sec
  1034. [2019-02-04 19:11:04,541 INFO] Step 47000/102583; acc: 82.78; ppl: 1.73; xent: 0.55; lr: 0.00010; 2176/2574 tok/s; 4834 sec
  1035. [2019-02-04 19:11:09,521 INFO] Step 47050/102583; acc: 80.69; ppl: 1.86; xent: 0.62; lr: 0.00010; 2075/2684 tok/s; 4839 sec
  1036. [2019-02-04 19:11:14,422 INFO] Step 47100/102583; acc: 81.36; ppl: 1.83; xent: 0.61; lr: 0.00010; 2048/2609 tok/s; 4844 sec
  1037. [2019-02-04 19:11:19,367 INFO] Step 47150/102583; acc: 81.23; ppl: 1.84; xent: 0.61; lr: 0.00010; 1856/2710 tok/s; 4849 sec
  1038. [2019-02-04 19:11:24,391 INFO] Step 47200/102583; acc: 82.79; ppl: 1.76; xent: 0.56; lr: 0.00010; 2267/2532 tok/s; 4854 sec
  1039. [2019-02-04 19:11:29,484 INFO] Step 47250/102583; acc: 80.12; ppl: 1.89; xent: 0.64; lr: 0.00010; 1831/2676 tok/s; 4859 sec
  1040. [2019-02-04 19:11:34,306 INFO] Step 47300/102583; acc: 82.16; ppl: 1.79; xent: 0.58; lr: 0.00010; 2255/2510 tok/s; 4864 sec
  1041. [2019-02-04 19:11:39,184 INFO] Step 47350/102583; acc: 80.79; ppl: 1.87; xent: 0.63; lr: 0.00010; 1963/2666 tok/s; 4869 sec
  1042. [2019-02-04 19:11:44,396 INFO] Step 47400/102583; acc: 82.99; ppl: 1.76; xent: 0.57; lr: 0.00010; 2050/2449 tok/s; 4874 sec
  1043. [2019-02-04 19:11:49,177 INFO] Step 47450/102583; acc: 80.80; ppl: 1.87; xent: 0.63; lr: 0.00010; 2040/2755 tok/s; 4879 sec
  1044. [2019-02-04 19:11:54,187 INFO] Step 47500/102583; acc: 82.07; ppl: 1.79; xent: 0.58; lr: 0.00010; 1990/2622 tok/s; 4884 sec
  1045. [2019-02-04 19:11:58,967 INFO] Step 47550/102583; acc: 80.99; ppl: 1.89; xent: 0.64; lr: 0.00010; 2018/2649 tok/s; 4888 sec
  1046. [2019-02-04 19:12:04,172 INFO] Step 47600/102583; acc: 83.83; ppl: 1.68; xent: 0.52; lr: 0.00010; 2090/2539 tok/s; 4894 sec
  1047. [2019-02-04 19:12:08,755 INFO] Step 47650/102583; acc: 79.52; ppl: 1.95; xent: 0.67; lr: 0.00010; 2153/2547 tok/s; 4898 sec
  1048. [2019-02-04 19:12:13,895 INFO] Step 47700/102583; acc: 82.71; ppl: 1.74; xent: 0.55; lr: 0.00010; 2069/2627 tok/s; 4903 sec
  1049. [2019-02-04 19:12:19,065 INFO] Step 47750/102583; acc: 81.34; ppl: 1.79; xent: 0.58; lr: 0.00010; 1884/2765 tok/s; 4908 sec
  1050. [2019-02-04 19:12:23,962 INFO] Step 47800/102583; acc: 81.11; ppl: 1.86; xent: 0.62; lr: 0.00010; 2090/2614 tok/s; 4913 sec
  1051. [2019-02-04 19:12:28,777 INFO] Step 47850/102583; acc: 83.17; ppl: 1.74; xent: 0.55; lr: 0.00010; 2102/2564 tok/s; 4918 sec
  1052. [2019-02-04 19:12:33,781 INFO] Step 47900/102583; acc: 81.29; ppl: 1.82; xent: 0.60; lr: 0.00010; 2087/2689 tok/s; 4923 sec
  1053. [2019-02-04 19:12:38,535 INFO] Step 47950/102583; acc: 80.67; ppl: 1.88; xent: 0.63; lr: 0.00010; 2023/2640 tok/s; 4928 sec
  1054. [2019-02-04 19:12:43,540 INFO] Step 48000/102583; acc: 82.54; ppl: 1.78; xent: 0.58; lr: 0.00010; 1985/2624 tok/s; 4933 sec
  1055. [2019-02-04 19:12:48,317 INFO] Step 48050/102583; acc: 82.00; ppl: 1.79; xent: 0.58; lr: 0.00010; 2151/2632 tok/s; 4938 sec
  1056. [2019-02-04 19:12:53,557 INFO] Step 48100/102583; acc: 81.58; ppl: 1.81; xent: 0.59; lr: 0.00010; 1993/2619 tok/s; 4943 sec
  1057. [2019-02-04 19:12:58,741 INFO] Step 48150/102583; acc: 80.48; ppl: 1.90; xent: 0.64; lr: 0.00010; 1903/2669 tok/s; 4948 sec
  1058. [2019-02-04 19:13:03,486 INFO] Step 48200/102583; acc: 81.29; ppl: 1.85; xent: 0.62; lr: 0.00010; 2092/2555 tok/s; 4953 sec
  1059. [2019-02-04 19:13:08,291 INFO] Step 48250/102583; acc: 82.89; ppl: 1.75; xent: 0.56; lr: 0.00010; 1976/2817 tok/s; 4958 sec
  1060. [2019-02-04 19:13:13,620 INFO] Step 48300/102583; acc: 82.99; ppl: 1.73; xent: 0.55; lr: 0.00010; 2081/2528 tok/s; 4963 sec
  1061. [2019-02-04 19:13:18,415 INFO] Step 48350/102583; acc: 80.83; ppl: 1.84; xent: 0.61; lr: 0.00010; 2102/2688 tok/s; 4968 sec
  1062. [2019-02-04 19:13:23,281 INFO] Step 48400/102583; acc: 81.39; ppl: 1.86; xent: 0.62; lr: 0.00010; 2192/2514 tok/s; 4973 sec
  1063. [2019-02-04 19:13:28,697 INFO] Step 48450/102583; acc: 83.09; ppl: 1.76; xent: 0.56; lr: 0.00010; 1990/2532 tok/s; 4978 sec
  1064. [2019-02-04 19:13:33,265 INFO] Step 48500/102583; acc: 80.12; ppl: 1.91; xent: 0.65; lr: 0.00010; 2080/2659 tok/s; 4983 sec
  1065. [2019-02-04 19:13:38,074 INFO] Step 48550/102583; acc: 81.60; ppl: 1.81; xent: 0.59; lr: 0.00010; 2064/2585 tok/s; 4987 sec
  1066. [2019-02-04 19:13:43,359 INFO] Step 48600/102583; acc: 81.99; ppl: 1.80; xent: 0.59; lr: 0.00010; 1936/2649 tok/s; 4993 sec
  1067. [2019-02-04 19:13:48,566 INFO] Step 48650/102583; acc: 80.81; ppl: 1.83; xent: 0.61; lr: 0.00010; 1875/2547 tok/s; 4998 sec
  1068. [2019-02-04 19:13:54,729 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  1069. [2019-02-04 19:13:59,577 INFO] Step 48700/102583; acc: 82.56; ppl: 1.77; xent: 0.57; lr: 0.00010; 999/1167 tok/s; 5009 sec
  1070. [2019-02-04 19:14:04,184 INFO] Step 48750/102583; acc: 80.99; ppl: 1.90; xent: 0.64; lr: 0.00010; 2147/2565 tok/s; 5014 sec
  1071. [2019-02-04 19:14:09,356 INFO] Step 48800/102583; acc: 82.97; ppl: 1.73; xent: 0.55; lr: 0.00010; 1988/2800 tok/s; 5019 sec
  1072. [2019-02-04 19:14:14,278 INFO] Step 48850/102583; acc: 80.55; ppl: 1.88; xent: 0.63; lr: 0.00010; 1983/2640 tok/s; 5024 sec
  1073. [2019-02-04 19:14:19,095 INFO] Step 48900/102583; acc: 82.78; ppl: 1.78; xent: 0.58; lr: 0.00010; 2038/2803 tok/s; 5028 sec
  1074. [2019-02-04 19:14:24,107 INFO] Step 48950/102583; acc: 81.90; ppl: 1.79; xent: 0.58; lr: 0.00010; 2161/2487 tok/s; 5033 sec
  1075. [2019-02-04 19:14:29,059 INFO] Step 49000/102583; acc: 81.99; ppl: 1.79; xent: 0.58; lr: 0.00010; 2177/2642 tok/s; 5038 sec
  1076. [2019-02-04 19:14:33,877 INFO] Step 49050/102583; acc: 80.84; ppl: 1.88; xent: 0.63; lr: 0.00010; 2063/2607 tok/s; 5043 sec
  1077. [2019-02-04 19:14:39,054 INFO] Step 49100/102583; acc: 80.35; ppl: 1.87; xent: 0.62; lr: 0.00010; 1848/2708 tok/s; 5048 sec
  1078. [2019-02-04 19:14:43,895 INFO] Step 49150/102583; acc: 81.35; ppl: 1.85; xent: 0.61; lr: 0.00010; 2095/2634 tok/s; 5053 sec
  1079. [2019-02-04 19:14:49,019 INFO] Step 49200/102583; acc: 82.38; ppl: 1.75; xent: 0.56; lr: 0.00010; 1943/2742 tok/s; 5058 sec
  1080. [2019-02-04 19:14:53,886 INFO] Step 49250/102583; acc: 81.43; ppl: 1.87; xent: 0.62; lr: 0.00010; 2261/2486 tok/s; 5063 sec
  1081. [2019-02-04 19:14:58,717 INFO] Step 49300/102583; acc: 80.68; ppl: 1.88; xent: 0.63; lr: 0.00010; 2035/2639 tok/s; 5068 sec
  1082. [2019-02-04 19:15:03,602 INFO] Step 49350/102583; acc: 80.53; ppl: 1.87; xent: 0.63; lr: 0.00010; 1931/2656 tok/s; 5073 sec
  1083. [2019-02-04 19:15:08,725 INFO] Step 49400/102583; acc: 84.16; ppl: 1.69; xent: 0.53; lr: 0.00010; 2282/2559 tok/s; 5078 sec
  1084. [2019-02-04 19:15:13,729 INFO] Step 49450/102583; acc: 80.70; ppl: 1.86; xent: 0.62; lr: 0.00010; 1925/2742 tok/s; 5083 sec
  1085. [2019-02-04 19:15:18,600 INFO] Step 49500/102583; acc: 82.70; ppl: 1.78; xent: 0.58; lr: 0.00010; 2095/2414 tok/s; 5088 sec
  1086. [2019-02-04 19:15:23,423 INFO] Step 49550/102583; acc: 82.36; ppl: 1.76; xent: 0.56; lr: 0.00010; 1999/2809 tok/s; 5093 sec
  1087. [2019-02-04 19:15:28,520 INFO] Step 49600/102583; acc: 80.75; ppl: 1.87; xent: 0.63; lr: 0.00010; 2062/2451 tok/s; 5098 sec
  1088. [2019-02-04 19:15:33,556 INFO] Step 49650/102583; acc: 81.36; ppl: 1.83; xent: 0.60; lr: 0.00010; 2022/2620 tok/s; 5103 sec
  1089. [2019-02-04 19:15:38,970 INFO] Step 49700/102583; acc: 81.59; ppl: 1.79; xent: 0.58; lr: 0.00010; 1912/2732 tok/s; 5108 sec
  1090. [2019-02-04 19:15:43,618 INFO] Step 49750/102583; acc: 81.53; ppl: 1.83; xent: 0.60; lr: 0.00010; 2096/2542 tok/s; 5113 sec
  1091. [2019-02-04 19:15:48,661 INFO] Step 49800/102583; acc: 80.80; ppl: 1.87; xent: 0.63; lr: 0.00010; 2133/2563 tok/s; 5118 sec
  1092. [2019-02-04 19:15:53,607 INFO] Step 49850/102583; acc: 80.84; ppl: 1.85; xent: 0.61; lr: 0.00010; 1994/2701 tok/s; 5123 sec
  1093. [2019-02-04 19:15:58,309 INFO] Step 49900/102583; acc: 80.20; ppl: 1.89; xent: 0.64; lr: 0.00010; 2118/2579 tok/s; 5128 sec
  1094. [2019-02-04 19:16:03,568 INFO] Step 49950/102583; acc: 83.23; ppl: 1.75; xent: 0.56; lr: 0.00010; 1951/2611 tok/s; 5133 sec
  1095. [2019-02-04 19:16:08,689 INFO] Step 50000/102583; acc: 82.68; ppl: 1.76; xent: 0.57; lr: 0.00005; 2179/2400 tok/s; 5138 sec
  1096. [2019-02-04 19:16:08,709 INFO] Loading dataset from data/ted.valid.0.pt, number of examples: 873
  1097. [2019-02-04 19:16:10,015 INFO] Validation perplexity: 1.75152
  1098. [2019-02-04 19:16:10,016 INFO] Validation accuracy: 82.9947
  1099. [2019-02-04 19:16:10,017 INFO] Saving checkpoint models/ted1_step_50000.pt
  1100. [2019-02-04 19:16:20,439 INFO] Step 50050/102583; acc: 80.12; ppl: 1.87; xent: 0.62; lr: 0.00005; 771/1174 tok/s; 5150 sec
  1101. [2019-02-04 19:16:25,497 INFO] Step 50100/102583; acc: 81.55; ppl: 1.80; xent: 0.59; lr: 0.00005; 1987/2623 tok/s; 5155 sec
  1102. [2019-02-04 19:16:30,383 INFO] Step 50150/102583; acc: 82.49; ppl: 1.78; xent: 0.58; lr: 0.00005; 2205/2507 tok/s; 5160 sec
  1103. [2019-02-04 19:16:35,072 INFO] Step 50200/102583; acc: 80.73; ppl: 1.89; xent: 0.64; lr: 0.00005; 1812/2673 tok/s; 5164 sec
  1104. [2019-02-04 19:16:40,405 INFO] Step 50250/102583; acc: 82.94; ppl: 1.71; xent: 0.54; lr: 0.00005; 2195/2618 tok/s; 5170 sec
  1105. [2019-02-04 19:16:45,367 INFO] Step 50300/102583; acc: 80.51; ppl: 1.87; xent: 0.63; lr: 0.00005; 1979/2711 tok/s; 5175 sec
  1106. [2019-02-04 19:16:50,311 INFO] Step 50350/102583; acc: 82.60; ppl: 1.79; xent: 0.58; lr: 0.00005; 2062/2517 tok/s; 5180 sec
  1107. [2019-02-04 19:16:55,368 INFO] Step 50400/102583; acc: 82.41; ppl: 1.78; xent: 0.58; lr: 0.00005; 1928/2719 tok/s; 5185 sec
  1108. [2019-02-04 19:17:00,212 INFO] Step 50450/102583; acc: 82.03; ppl: 1.80; xent: 0.59; lr: 0.00005; 2225/2568 tok/s; 5190 sec
  1109. [2019-02-04 19:17:05,328 INFO] Step 50500/102583; acc: 80.99; ppl: 1.87; xent: 0.62; lr: 0.00005; 1823/2688 tok/s; 5195 sec
  1110. [2019-02-04 19:17:10,174 INFO] Step 50550/102583; acc: 82.23; ppl: 1.77; xent: 0.57; lr: 0.00005; 2231/2541 tok/s; 5200 sec
  1111. [2019-02-04 19:17:15,116 INFO] Step 50600/102583; acc: 81.85; ppl: 1.81; xent: 0.59; lr: 0.00005; 2016/2609 tok/s; 5205 sec
  1112. [2019-02-04 19:17:20,280 INFO] Step 50650/102583; acc: 82.39; ppl: 1.79; xent: 0.58; lr: 0.00005; 1957/2509 tok/s; 5210 sec
  1113. [2019-02-04 19:17:25,033 INFO] Step 50700/102583; acc: 81.12; ppl: 1.85; xent: 0.62; lr: 0.00005; 2042/2680 tok/s; 5214 sec
  1114. [2019-02-04 19:17:30,010 INFO] Step 50750/102583; acc: 82.97; ppl: 1.75; xent: 0.56; lr: 0.00005; 2063/2658 tok/s; 5219 sec
  1115. [2019-02-04 19:17:34,938 INFO] Step 50800/102583; acc: 81.25; ppl: 1.87; xent: 0.63; lr: 0.00005; 2022/2585 tok/s; 5224 sec
  1116. [2019-02-04 19:17:39,955 INFO] Step 50850/102583; acc: 83.41; ppl: 1.71; xent: 0.53; lr: 0.00005; 2143/2567 tok/s; 5229 sec
  1117. [2019-02-04 19:17:44,574 INFO] Step 50900/102583; acc: 81.17; ppl: 1.85; xent: 0.62; lr: 0.00005; 2109/2579 tok/s; 5234 sec
  1118. [2019-02-04 19:17:49,772 INFO] Step 50950/102583; acc: 82.48; ppl: 1.75; xent: 0.56; lr: 0.00005; 2047/2638 tok/s; 5239 sec
  1119. [2019-02-04 19:17:54,994 INFO] Step 51000/102583; acc: 81.76; ppl: 1.77; xent: 0.57; lr: 0.00005; 1944/2713 tok/s; 5244 sec
  1120. [2019-02-04 19:17:59,849 INFO] Step 51050/102583; acc: 81.29; ppl: 1.84; xent: 0.61; lr: 0.00005; 1997/2666 tok/s; 5249 sec
  1121. [2019-02-04 19:18:04,560 INFO] Step 51100/102583; acc: 83.54; ppl: 1.73; xent: 0.55; lr: 0.00005; 2203/2517 tok/s; 5254 sec
  1122. [2019-02-04 19:18:09,673 INFO] Step 51150/102583; acc: 81.12; ppl: 1.82; xent: 0.60; lr: 0.00005; 2060/2705 tok/s; 5259 sec
  1123. [2019-02-04 19:18:14,309 INFO] Step 51200/102583; acc: 81.18; ppl: 1.84; xent: 0.61; lr: 0.00005; 1973/2653 tok/s; 5264 sec
  1124. [2019-02-04 19:18:19,344 INFO] Step 51250/102583; acc: 82.67; ppl: 1.78; xent: 0.57; lr: 0.00005; 1986/2649 tok/s; 5269 sec
  1125. [2019-02-04 19:18:24,114 INFO] Step 51300/102583; acc: 83.40; ppl: 1.72; xent: 0.54; lr: 0.00005; 2222/2624 tok/s; 5274 sec
  1126. [2019-02-04 19:18:29,370 INFO] Step 51350/102583; acc: 81.04; ppl: 1.84; xent: 0.61; lr: 0.00005; 1926/2632 tok/s; 5279 sec
  1127. [2019-02-04 19:18:34,510 INFO] Step 51400/102583; acc: 80.99; ppl: 1.86; xent: 0.62; lr: 0.00005; 1956/2636 tok/s; 5284 sec
  1128. [2019-02-04 19:18:39,382 INFO] Step 51450/102583; acc: 82.59; ppl: 1.77; xent: 0.57; lr: 0.00005; 2119/2533 tok/s; 5289 sec
  1129. [2019-02-04 19:18:44,237 INFO] Step 51500/102583; acc: 83.07; ppl: 1.72; xent: 0.54; lr: 0.00005; 1925/2788 tok/s; 5294 sec
  1130. [2019-02-04 19:18:49,486 INFO] Step 51550/102583; acc: 82.88; ppl: 1.76; xent: 0.56; lr: 0.00005; 2135/2529 tok/s; 5299 sec
  1131. [2019-02-04 19:18:54,294 INFO] Step 51600/102583; acc: 81.62; ppl: 1.80; xent: 0.59; lr: 0.00005; 2060/2784 tok/s; 5304 sec
  1132. [2019-02-04 19:18:59,205 INFO] Step 51650/102583; acc: 81.34; ppl: 1.86; xent: 0.62; lr: 0.00005; 2182/2347 tok/s; 5309 sec
  1133. [2019-02-04 19:19:04,497 INFO] Step 51700/102583; acc: 82.85; ppl: 1.74; xent: 0.55; lr: 0.00005; 2028/2660 tok/s; 5314 sec
  1134. [2019-02-04 19:19:09,081 INFO] Step 51750/102583; acc: 80.40; ppl: 1.89; xent: 0.64; lr: 0.00005; 1989/2690 tok/s; 5318 sec
  1135. [2019-02-04 19:19:13,881 INFO] Step 51800/102583; acc: 82.17; ppl: 1.78; xent: 0.58; lr: 0.00005; 2122/2544 tok/s; 5323 sec
  1136. [2019-02-04 19:19:19,174 INFO] Step 51850/102583; acc: 81.65; ppl: 1.81; xent: 0.59; lr: 0.00005; 1862/2698 tok/s; 5329 sec
  1137. [2019-02-04 19:19:24,337 INFO] Step 51900/102583; acc: 82.27; ppl: 1.80; xent: 0.59; lr: 0.00005; 1953/2469 tok/s; 5334 sec
  1138. [2019-02-04 19:19:29,681 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  1139. [2019-02-04 19:19:35,019 INFO] Step 51950/102583; acc: 82.46; ppl: 1.76; xent: 0.57; lr: 0.00005; 1017/1205 tok/s; 5344 sec
  1140. [2019-02-04 19:19:39,827 INFO] Step 52000/102583; acc: 81.70; ppl: 1.83; xent: 0.60; lr: 0.00005; 2057/2700 tok/s; 5349 sec
  1141. [2019-02-04 19:19:44,855 INFO] Step 52050/102583; acc: 83.18; ppl: 1.73; xent: 0.55; lr: 0.00005; 2032/2721 tok/s; 5354 sec
  1142. [2019-02-04 19:19:49,749 INFO] Step 52100/102583; acc: 81.58; ppl: 1.84; xent: 0.61; lr: 0.00005; 2020/2588 tok/s; 5359 sec
  1143. [2019-02-04 19:19:54,593 INFO] Step 52150/102583; acc: 82.44; ppl: 1.78; xent: 0.58; lr: 0.00005; 2018/2762 tok/s; 5364 sec
  1144. [2019-02-04 19:19:59,659 INFO] Step 52200/102583; acc: 81.82; ppl: 1.78; xent: 0.58; lr: 0.00005; 2139/2526 tok/s; 5369 sec
  1145. [2019-02-04 19:20:04,473 INFO] Step 52250/102583; acc: 81.93; ppl: 1.80; xent: 0.59; lr: 0.00005; 2181/2643 tok/s; 5374 sec
  1146. [2019-02-04 19:20:09,898 INFO] Step 52300/102583; acc: 81.79; ppl: 1.82; xent: 0.60; lr: 0.00005; 2026/2662 tok/s; 5379 sec
  1147. [2019-02-04 19:20:15,122 INFO] Step 52350/102583; acc: 80.31; ppl: 1.88; xent: 0.63; lr: 0.00005; 1884/2673 tok/s; 5385 sec
  1148. [2019-02-04 19:20:19,838 INFO] Step 52400/102583; acc: 82.34; ppl: 1.77; xent: 0.57; lr: 0.00005; 2144/2623 tok/s; 5389 sec
  1149. [2019-02-04 19:20:25,034 INFO] Step 52450/102583; acc: 82.27; ppl: 1.75; xent: 0.56; lr: 0.00005; 1934/2721 tok/s; 5394 sec
  1150. [2019-02-04 19:20:29,889 INFO] Step 52500/102583; acc: 81.81; ppl: 1.84; xent: 0.61; lr: 0.00005; 2261/2558 tok/s; 5399 sec
  1151. [2019-02-04 19:20:34,875 INFO] Step 52550/102583; acc: 81.14; ppl: 1.83; xent: 0.60; lr: 0.00005; 2024/2589 tok/s; 5404 sec
  1152. [2019-02-04 19:20:39,558 INFO] Step 52600/102583; acc: 81.44; ppl: 1.84; xent: 0.61; lr: 0.00005; 2013/2644 tok/s; 5409 sec
  1153. [2019-02-04 19:20:44,748 INFO] Step 52650/102583; acc: 83.33; ppl: 1.71; xent: 0.54; lr: 0.00005; 2215/2627 tok/s; 5414 sec
  1154. [2019-02-04 19:20:49,654 INFO] Step 52700/102583; acc: 81.89; ppl: 1.79; xent: 0.58; lr: 0.00005; 1951/2695 tok/s; 5419 sec
  1155. [2019-02-04 19:20:54,697 INFO] Step 52750/102583; acc: 82.00; ppl: 1.78; xent: 0.58; lr: 0.00005; 2057/2453 tok/s; 5424 sec
  1156. [2019-02-04 19:20:59,523 INFO] Step 52800/102583; acc: 82.35; ppl: 1.78; xent: 0.58; lr: 0.00005; 2014/2698 tok/s; 5429 sec
  1157. [2019-02-04 19:21:04,744 INFO] Step 52850/102583; acc: 81.96; ppl: 1.79; xent: 0.58; lr: 0.00005; 2059/2439 tok/s; 5434 sec
  1158. [2019-02-04 19:21:09,660 INFO] Step 52900/102583; acc: 82.28; ppl: 1.78; xent: 0.58; lr: 0.00005; 2014/2747 tok/s; 5439 sec
  1159. [2019-02-04 19:21:14,893 INFO] Step 52950/102583; acc: 81.18; ppl: 1.81; xent: 0.59; lr: 0.00005; 1896/2721 tok/s; 5444 sec
  1160. [2019-02-04 19:21:19,701 INFO] Step 53000/102583; acc: 82.28; ppl: 1.79; xent: 0.58; lr: 0.00005; 2127/2512 tok/s; 5449 sec
  1161. [2019-02-04 19:21:24,675 INFO] Step 53050/102583; acc: 81.17; ppl: 1.84; xent: 0.61; lr: 0.00005; 2107/2554 tok/s; 5454 sec
  1162. [2019-02-04 19:21:29,525 INFO] Step 53100/102583; acc: 80.84; ppl: 1.84; xent: 0.61; lr: 0.00005; 1984/2729 tok/s; 5459 sec
  1163. [2019-02-04 19:21:34,195 INFO] Step 53150/102583; acc: 81.84; ppl: 1.85; xent: 0.61; lr: 0.00005; 2146/2579 tok/s; 5464 sec
  1164. [2019-02-04 19:21:39,550 INFO] Step 53200/102583; acc: 82.80; ppl: 1.73; xent: 0.55; lr: 0.00005; 1918/2640 tok/s; 5469 sec
  1165. [2019-02-04 19:21:44,914 INFO] Step 53250/102583; acc: 83.37; ppl: 1.69; xent: 0.53; lr: 0.00005; 2180/2462 tok/s; 5474 sec
  1166. [2019-02-04 19:21:49,613 INFO] Step 53300/102583; acc: 80.18; ppl: 1.88; xent: 0.63; lr: 0.00005; 1935/2789 tok/s; 5479 sec
  1167. [2019-02-04 19:21:54,786 INFO] Step 53350/102583; acc: 81.81; ppl: 1.79; xent: 0.58; lr: 0.00005; 2023/2617 tok/s; 5484 sec
  1168. [2019-02-04 19:21:59,508 INFO] Step 53400/102583; acc: 81.87; ppl: 1.81; xent: 0.59; lr: 0.00005; 2115/2600 tok/s; 5489 sec
  1169. [2019-02-04 19:22:04,234 INFO] Step 53450/102583; acc: 81.49; ppl: 1.86; xent: 0.62; lr: 0.00005; 1966/2618 tok/s; 5494 sec
  1170. [2019-02-04 19:22:09,587 INFO] Step 53500/102583; acc: 82.27; ppl: 1.73; xent: 0.55; lr: 0.00005; 2027/2562 tok/s; 5499 sec
  1171. [2019-02-04 19:22:14,460 INFO] Step 53550/102583; acc: 80.84; ppl: 1.84; xent: 0.61; lr: 0.00005; 2076/2802 tok/s; 5504 sec
  1172. [2019-02-04 19:22:19,448 INFO] Step 53600/102583; acc: 82.76; ppl: 1.76; xent: 0.57; lr: 0.00005; 2048/2535 tok/s; 5509 sec
  1173. [2019-02-04 19:22:24,536 INFO] Step 53650/102583; acc: 82.45; ppl: 1.78; xent: 0.58; lr: 0.00005; 1924/2748 tok/s; 5514 sec
  1174. [2019-02-04 19:22:29,369 INFO] Step 53700/102583; acc: 81.30; ppl: 1.82; xent: 0.60; lr: 0.00005; 2181/2525 tok/s; 5519 sec
  1175. [2019-02-04 19:22:34,371 INFO] Step 53750/102583; acc: 80.84; ppl: 1.84; xent: 0.61; lr: 0.00005; 1930/2702 tok/s; 5524 sec
  1176. [2019-02-04 19:22:39,226 INFO] Step 53800/102583; acc: 82.18; ppl: 1.78; xent: 0.58; lr: 0.00005; 2196/2561 tok/s; 5529 sec
  1177. [2019-02-04 19:22:44,454 INFO] Step 53850/102583; acc: 82.56; ppl: 1.77; xent: 0.57; lr: 0.00005; 2075/2583 tok/s; 5534 sec
  1178. [2019-02-04 19:22:49,431 INFO] Step 53900/102583; acc: 81.76; ppl: 1.82; xent: 0.60; lr: 0.00005; 1943/2564 tok/s; 5539 sec
  1179. [2019-02-04 19:22:54,255 INFO] Step 53950/102583; acc: 81.81; ppl: 1.82; xent: 0.60; lr: 0.00005; 2024/2693 tok/s; 5544 sec
  1180. [2019-02-04 19:22:59,389 INFO] Step 54000/102583; acc: 82.97; ppl: 1.75; xent: 0.56; lr: 0.00005; 2108/2544 tok/s; 5549 sec
  1181. [2019-02-04 19:23:04,204 INFO] Step 54050/102583; acc: 80.62; ppl: 1.87; xent: 0.63; lr: 0.00005; 1947/2715 tok/s; 5554 sec
  1182. [2019-02-04 19:23:09,240 INFO] Step 54100/102583; acc: 83.29; ppl: 1.72; xent: 0.54; lr: 0.00005; 2140/2577 tok/s; 5559 sec
  1183. [2019-02-04 19:23:13,984 INFO] Step 54150/102583; acc: 82.15; ppl: 1.78; xent: 0.58; lr: 0.00005; 2182/2590 tok/s; 5563 sec
  1184. [2019-02-04 19:23:19,201 INFO] Step 54200/102583; acc: 81.63; ppl: 1.80; xent: 0.59; lr: 0.00005; 1921/2610 tok/s; 5569 sec
  1185. [2019-02-04 19:23:24,281 INFO] Step 54250/102583; acc: 82.20; ppl: 1.77; xent: 0.57; lr: 0.00005; 2085/2697 tok/s; 5574 sec
  1186. [2019-02-04 19:23:29,095 INFO] Step 54300/102583; acc: 81.41; ppl: 1.84; xent: 0.61; lr: 0.00005; 2000/2641 tok/s; 5578 sec
  1187. [2019-02-04 19:23:34,058 INFO] Step 54350/102583; acc: 83.54; ppl: 1.71; xent: 0.54; lr: 0.00005; 2109/2558 tok/s; 5583 sec
  1188. [2019-02-04 19:23:38,965 INFO] Step 54400/102583; acc: 80.90; ppl: 1.84; xent: 0.61; lr: 0.00005; 2097/2683 tok/s; 5588 sec
  1189. [2019-02-04 19:23:43,739 INFO] Step 54450/102583; acc: 81.66; ppl: 1.82; xent: 0.60; lr: 0.00005; 1997/2653 tok/s; 5593 sec
  1190. [2019-02-04 19:23:49,061 INFO] Step 54500/102583; acc: 82.42; ppl: 1.77; xent: 0.57; lr: 0.00005; 2025/2635 tok/s; 5598 sec
  1191. [2019-02-04 19:23:53,866 INFO] Step 54550/102583; acc: 82.74; ppl: 1.75; xent: 0.56; lr: 0.00005; 2179/2579 tok/s; 5603 sec
  1192. [2019-02-04 19:23:58,833 INFO] Step 54600/102583; acc: 81.71; ppl: 1.83; xent: 0.61; lr: 0.00005; 1908/2688 tok/s; 5608 sec
  1193. [2019-02-04 19:24:03,994 INFO] Step 54650/102583; acc: 80.91; ppl: 1.85; xent: 0.62; lr: 0.00005; 1952/2646 tok/s; 5613 sec
  1194. [2019-02-04 19:24:08,990 INFO] Step 54700/102583; acc: 83.12; ppl: 1.74; xent: 0.55; lr: 0.00005; 2130/2604 tok/s; 5618 sec
  1195. [2019-02-04 19:24:13,806 INFO] Step 54750/102583; acc: 82.75; ppl: 1.73; xent: 0.55; lr: 0.00005; 1941/2753 tok/s; 5623 sec
  1196. [2019-02-04 19:24:19,145 INFO] Step 54800/102583; acc: 82.16; ppl: 1.78; xent: 0.58; lr: 0.00005; 2142/2490 tok/s; 5629 sec
  1197. [2019-02-04 19:24:23,945 INFO] Step 54850/102583; acc: 81.70; ppl: 1.79; xent: 0.58; lr: 0.00005; 1997/2770 tok/s; 5633 sec
  1198. [2019-02-04 19:24:28,860 INFO] Step 54900/102583; acc: 81.11; ppl: 1.87; xent: 0.62; lr: 0.00005; 2174/2392 tok/s; 5638 sec
  1199. [2019-02-04 19:24:34,194 INFO] Step 54950/102583; acc: 82.35; ppl: 1.76; xent: 0.56; lr: 0.00005; 2072/2672 tok/s; 5644 sec
  1200. [2019-02-04 19:24:38,834 INFO] Step 55000/102583; acc: 80.60; ppl: 1.87; xent: 0.63; lr: 0.00005; 1924/2719 tok/s; 5648 sec
  1201. [2019-02-04 19:24:38,927 INFO] Saving checkpoint models/ted1_step_55000.pt
  1202. [2019-02-04 19:24:51,121 INFO] Step 55050/102583; acc: 82.81; ppl: 1.76; xent: 0.57; lr: 0.00005; 830/1012 tok/s; 5661 sec
  1203. [2019-02-04 19:24:56,454 INFO] Step 55100/102583; acc: 81.65; ppl: 1.80; xent: 0.59; lr: 0.00005; 1950/2606 tok/s; 5666 sec
  1204. [2019-02-04 19:25:06,706 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  1205. [2019-02-04 19:25:07,594 INFO] Step 55150/102583; acc: 81.74; ppl: 1.81; xent: 0.59; lr: 0.00005; 907/1136 tok/s; 5677 sec
  1206. [2019-02-04 19:25:12,667 INFO] Step 55200/102583; acc: 82.46; ppl: 1.77; xent: 0.57; lr: 0.00005; 2106/2575 tok/s; 5682 sec
  1207. [2019-02-04 19:25:17,431 INFO] Step 55250/102583; acc: 82.25; ppl: 1.79; xent: 0.58; lr: 0.00005; 2009/2695 tok/s; 5687 sec
  1208. [2019-02-04 19:25:22,684 INFO] Step 55300/102583; acc: 83.06; ppl: 1.73; xent: 0.55; lr: 0.00005; 2051/2661 tok/s; 5692 sec
  1209. [2019-02-04 19:25:27,449 INFO] Step 55350/102583; acc: 81.27; ppl: 1.86; xent: 0.62; lr: 0.00005; 2094/2642 tok/s; 5697 sec
  1210. [2019-02-04 19:25:32,495 INFO] Step 55400/102583; acc: 82.78; ppl: 1.74; xent: 0.56; lr: 0.00005; 2061/2713 tok/s; 5702 sec
  1211. [2019-02-04 19:25:37,365 INFO] Step 55450/102583; acc: 80.70; ppl: 1.85; xent: 0.62; lr: 0.00005; 2052/2591 tok/s; 5707 sec
  1212. [2019-02-04 19:25:42,226 INFO] Step 55500/102583; acc: 82.25; ppl: 1.79; xent: 0.58; lr: 0.00005; 2107/2690 tok/s; 5712 sec
  1213. [2019-02-04 19:25:47,252 INFO] Step 55550/102583; acc: 81.60; ppl: 1.81; xent: 0.59; lr: 0.00005; 2010/2702 tok/s; 5717 sec
  1214. [2019-02-04 19:25:52,379 INFO] Step 55600/102583; acc: 81.49; ppl: 1.82; xent: 0.60; lr: 0.00005; 2030/2570 tok/s; 5722 sec
  1215. [2019-02-04 19:25:57,169 INFO] Step 55650/102583; acc: 81.73; ppl: 1.79; xent: 0.58; lr: 0.00005; 2085/2655 tok/s; 5727 sec
  1216. [2019-02-04 19:26:02,240 INFO] Step 55700/102583; acc: 82.44; ppl: 1.77; xent: 0.57; lr: 0.00005; 1958/2683 tok/s; 5732 sec
  1217. [2019-02-04 19:26:07,056 INFO] Step 55750/102583; acc: 81.78; ppl: 1.83; xent: 0.60; lr: 0.00005; 2232/2544 tok/s; 5736 sec
  1218. [2019-02-04 19:26:12,072 INFO] Step 55800/102583; acc: 80.17; ppl: 1.86; xent: 0.62; lr: 0.00005; 2063/2592 tok/s; 5741 sec
  1219. [2019-02-04 19:26:16,943 INFO] Step 55850/102583; acc: 83.04; ppl: 1.76; xent: 0.57; lr: 0.00005; 2025/2670 tok/s; 5746 sec
  1220. [2019-02-04 19:26:22,005 INFO] Step 55900/102583; acc: 82.80; ppl: 1.74; xent: 0.55; lr: 0.00005; 2175/2653 tok/s; 5751 sec
  1221. [2019-02-04 19:26:26,981 INFO] Step 55950/102583; acc: 81.33; ppl: 1.82; xent: 0.60; lr: 0.00005; 1983/2601 tok/s; 5756 sec
  1222. [2019-02-04 19:26:32,008 INFO] Step 56000/102583; acc: 82.53; ppl: 1.75; xent: 0.56; lr: 0.00005; 2026/2550 tok/s; 5761 sec
  1223. [2019-02-04 19:26:37,074 INFO] Step 56050/102583; acc: 82.22; ppl: 1.77; xent: 0.57; lr: 0.00005; 1953/2652 tok/s; 5766 sec
  1224. [2019-02-04 19:26:42,095 INFO] Step 56100/102583; acc: 81.55; ppl: 1.83; xent: 0.60; lr: 0.00005; 2093/2486 tok/s; 5771 sec
  1225. [2019-02-04 19:26:46,991 INFO] Step 56150/102583; acc: 81.33; ppl: 1.81; xent: 0.59; lr: 0.00005; 2017/2738 tok/s; 5776 sec
  1226. [2019-02-04 19:26:52,210 INFO] Step 56200/102583; acc: 82.07; ppl: 1.78; xent: 0.57; lr: 0.00005; 1923/2614 tok/s; 5782 sec
  1227. [2019-02-04 19:26:57,170 INFO] Step 56250/102583; acc: 82.34; ppl: 1.79; xent: 0.58; lr: 0.00005; 2148/2542 tok/s; 5787 sec
  1228. [2019-02-04 19:27:02,165 INFO] Step 56300/102583; acc: 81.59; ppl: 1.81; xent: 0.59; lr: 0.00005; 2115/2616 tok/s; 5792 sec
  1229. [2019-02-04 19:27:06,959 INFO] Step 56350/102583; acc: 80.66; ppl: 1.85; xent: 0.61; lr: 0.00005; 1992/2659 tok/s; 5796 sec
  1230. [2019-02-04 19:27:11,716 INFO] Step 56400/102583; acc: 81.62; ppl: 1.82; xent: 0.60; lr: 0.00005; 2101/2690 tok/s; 5801 sec
  1231. [2019-02-04 19:27:16,966 INFO] Step 56450/102583; acc: 82.50; ppl: 1.77; xent: 0.57; lr: 0.00005; 2012/2514 tok/s; 5806 sec
  1232. [2019-02-04 19:27:22,203 INFO] Step 56500/102583; acc: 82.80; ppl: 1.74; xent: 0.55; lr: 0.00005; 2087/2559 tok/s; 5812 sec
  1233. [2019-02-04 19:27:26,940 INFO] Step 56550/102583; acc: 80.68; ppl: 1.84; xent: 0.61; lr: 0.00005; 1913/2712 tok/s; 5816 sec
  1234. [2019-02-04 19:27:32,065 INFO] Step 56600/102583; acc: 81.97; ppl: 1.78; xent: 0.58; lr: 0.00005; 2077/2683 tok/s; 5821 sec
  1235. [2019-02-04 19:27:36,813 INFO] Step 56650/102583; acc: 81.62; ppl: 1.83; xent: 0.60; lr: 0.00005; 2013/2535 tok/s; 5826 sec
  1236. [2019-02-04 19:27:41,617 INFO] Step 56700/102583; acc: 81.63; ppl: 1.83; xent: 0.60; lr: 0.00005; 1944/2695 tok/s; 5831 sec
  1237. [2019-02-04 19:27:46,746 INFO] Step 56750/102583; acc: 82.44; ppl: 1.74; xent: 0.56; lr: 0.00005; 2114/2508 tok/s; 5836 sec
  1238. [2019-02-04 19:27:51,670 INFO] Step 56800/102583; acc: 80.76; ppl: 1.84; xent: 0.61; lr: 0.00005; 2029/2843 tok/s; 5841 sec
  1239. [2019-02-04 19:27:56,670 INFO] Step 56850/102583; acc: 83.10; ppl: 1.76; xent: 0.56; lr: 0.00005; 2087/2508 tok/s; 5846 sec
  1240. [2019-02-04 19:28:01,821 INFO] Step 56900/102583; acc: 82.80; ppl: 1.77; xent: 0.57; lr: 0.00005; 1938/2662 tok/s; 5851 sec
  1241. [2019-02-04 19:28:06,473 INFO] Step 56950/102583; acc: 81.12; ppl: 1.85; xent: 0.61; lr: 0.00005; 2189/2565 tok/s; 5856 sec
  1242. [2019-02-04 19:28:11,572 INFO] Step 57000/102583; acc: 81.16; ppl: 1.82; xent: 0.60; lr: 0.00005; 1882/2695 tok/s; 5861 sec
  1243. [2019-02-04 19:28:16,488 INFO] Step 57050/102583; acc: 82.44; ppl: 1.77; xent: 0.57; lr: 0.00005; 2198/2530 tok/s; 5866 sec
  1244. [2019-02-04 19:28:21,731 INFO] Step 57100/102583; acc: 82.91; ppl: 1.75; xent: 0.56; lr: 0.00005; 2136/2593 tok/s; 5871 sec
  1245. [2019-02-04 19:28:26,505 INFO] Step 57150/102583; acc: 81.85; ppl: 1.83; xent: 0.60; lr: 0.00005; 1885/2581 tok/s; 5876 sec
  1246. [2019-02-04 19:28:31,375 INFO] Step 57200/102583; acc: 80.75; ppl: 1.89; xent: 0.63; lr: 0.00005; 1937/2664 tok/s; 5881 sec
  1247. [2019-02-04 19:28:36,399 INFO] Step 57250/102583; acc: 82.64; ppl: 1.75; xent: 0.56; lr: 0.00005; 2145/2550 tok/s; 5886 sec
  1248. [2019-02-04 19:28:41,235 INFO] Step 57300/102583; acc: 82.09; ppl: 1.79; xent: 0.58; lr: 0.00005; 1990/2737 tok/s; 5891 sec
  1249. [2019-02-04 19:28:46,230 INFO] Step 57350/102583; acc: 82.88; ppl: 1.76; xent: 0.57; lr: 0.00005; 2140/2550 tok/s; 5896 sec
  1250. [2019-02-04 19:28:51,001 INFO] Step 57400/102583; acc: 81.76; ppl: 1.80; xent: 0.59; lr: 0.00005; 2157/2622 tok/s; 5900 sec
  1251. [2019-02-04 19:28:56,258 INFO] Step 57450/102583; acc: 82.10; ppl: 1.76; xent: 0.57; lr: 0.00005; 1916/2627 tok/s; 5906 sec
  1252. [2019-02-04 19:29:01,308 INFO] Step 57500/102583; acc: 82.42; ppl: 1.76; xent: 0.56; lr: 0.00005; 2040/2714 tok/s; 5911 sec
  1253. [2019-02-04 19:29:06,285 INFO] Step 57550/102583; acc: 82.11; ppl: 1.80; xent: 0.59; lr: 0.00005; 2100/2527 tok/s; 5916 sec
  1254. [2019-02-04 19:29:11,164 INFO] Step 57600/102583; acc: 82.97; ppl: 1.73; xent: 0.55; lr: 0.00005; 2052/2686 tok/s; 5921 sec
  1255. [2019-02-04 19:29:16,003 INFO] Step 57650/102583; acc: 80.74; ppl: 1.85; xent: 0.62; lr: 0.00005; 2052/2661 tok/s; 5925 sec
  1256. [2019-02-04 19:29:20,607 INFO] Step 57700/102583; acc: 81.67; ppl: 1.83; xent: 0.60; lr: 0.00005; 2087/2600 tok/s; 5930 sec
  1257. [2019-02-04 19:29:25,956 INFO] Step 57750/102583; acc: 83.03; ppl: 1.73; xent: 0.55; lr: 0.00005; 1993/2657 tok/s; 5935 sec
  1258. [2019-02-04 19:29:30,939 INFO] Step 57800/102583; acc: 82.07; ppl: 1.76; xent: 0.57; lr: 0.00005; 2199/2541 tok/s; 5940 sec
  1259. [2019-02-04 19:29:35,929 INFO] Step 57850/102583; acc: 81.50; ppl: 1.83; xent: 0.60; lr: 0.00005; 1848/2752 tok/s; 5945 sec
  1260. [2019-02-04 19:29:40,996 INFO] Step 57900/102583; acc: 80.51; ppl: 1.88; xent: 0.63; lr: 0.00005; 1967/2540 tok/s; 5950 sec
  1261. [2019-02-04 19:29:45,842 INFO] Step 57950/102583; acc: 83.71; ppl: 1.71; xent: 0.54; lr: 0.00005; 2128/2685 tok/s; 5955 sec
  1262. [2019-02-04 19:29:50,656 INFO] Step 58000/102583; acc: 81.91; ppl: 1.77; xent: 0.57; lr: 0.00005; 1916/2747 tok/s; 5960 sec
  1263. [2019-02-04 19:29:55,895 INFO] Step 58050/102583; acc: 82.17; ppl: 1.79; xent: 0.58; lr: 0.00005; 2144/2504 tok/s; 5965 sec
  1264. [2019-02-04 19:30:00,810 INFO] Step 58100/102583; acc: 83.08; ppl: 1.75; xent: 0.56; lr: 0.00005; 2028/2692 tok/s; 5970 sec
  1265. [2019-02-04 19:30:05,911 INFO] Step 58150/102583; acc: 81.18; ppl: 1.83; xent: 0.61; lr: 0.00005; 2124/2459 tok/s; 5975 sec
  1266. [2019-02-04 19:30:10,955 INFO] Step 58200/102583; acc: 81.83; ppl: 1.79; xent: 0.58; lr: 0.00005; 2079/2684 tok/s; 5980 sec
  1267. [2019-02-04 19:30:15,680 INFO] Step 58250/102583; acc: 81.95; ppl: 1.80; xent: 0.59; lr: 0.00005; 2019/2630 tok/s; 5985 sec
  1268. [2019-02-04 19:30:20,351 INFO] Step 58300/102583; acc: 82.12; ppl: 1.80; xent: 0.59; lr: 0.00005; 2106/2597 tok/s; 5990 sec
  1269. [2019-02-04 19:30:25,748 INFO] Step 58350/102583; acc: 81.58; ppl: 1.80; xent: 0.59; lr: 0.00005; 1927/2644 tok/s; 5995 sec
  1270. [2019-02-04 19:30:35,665 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  1271. [2019-02-04 19:30:37,044 INFO] Step 58400/102583; acc: 81.73; ppl: 1.84; xent: 0.61; lr: 0.00005; 899/1127 tok/s; 6006 sec
  1272. [2019-02-04 19:30:42,190 INFO] Step 58450/102583; acc: 82.59; ppl: 1.76; xent: 0.57; lr: 0.00005; 2102/2559 tok/s; 6012 sec
  1273. [2019-02-04 19:30:46,911 INFO] Step 58500/102583; acc: 81.93; ppl: 1.80; xent: 0.59; lr: 0.00005; 1948/2715 tok/s; 6016 sec
  1274. [2019-02-04 19:30:52,061 INFO] Step 58550/102583; acc: 83.67; ppl: 1.70; xent: 0.53; lr: 0.00005; 2110/2649 tok/s; 6021 sec
  1275. [2019-02-04 19:30:56,922 INFO] Step 58600/102583; acc: 81.60; ppl: 1.82; xent: 0.60; lr: 0.00005; 2112/2651 tok/s; 6026 sec
  1276. [2019-02-04 19:31:02,180 INFO] Step 58650/102583; acc: 81.94; ppl: 1.77; xent: 0.57; lr: 0.00005; 2055/2703 tok/s; 6032 sec
  1277. [2019-02-04 19:31:07,092 INFO] Step 58700/102583; acc: 80.98; ppl: 1.83; xent: 0.61; lr: 0.00005; 2062/2556 tok/s; 6036 sec
  1278. [2019-02-04 19:31:11,832 INFO] Step 58750/102583; acc: 82.86; ppl: 1.76; xent: 0.57; lr: 0.00005; 2109/2788 tok/s; 6041 sec
  1279. [2019-02-04 19:31:16,860 INFO] Step 58800/102583; acc: 81.20; ppl: 1.84; xent: 0.61; lr: 0.00005; 1928/2718 tok/s; 6046 sec
  1280. [2019-02-04 19:31:21,995 INFO] Step 58850/102583; acc: 81.26; ppl: 1.85; xent: 0.61; lr: 0.00005; 2087/2479 tok/s; 6051 sec
  1281. [2019-02-04 19:31:26,831 INFO] Step 58900/102583; acc: 82.28; ppl: 1.76; xent: 0.57; lr: 0.00005; 2112/2692 tok/s; 6056 sec
  1282. [2019-02-04 19:31:32,198 INFO] Step 58950/102583; acc: 81.74; ppl: 1.81; xent: 0.59; lr: 0.00005; 1986/2643 tok/s; 6062 sec
  1283. [2019-02-04 19:31:36,850 INFO] Step 59000/102583; acc: 82.56; ppl: 1.79; xent: 0.58; lr: 0.00005; 2218/2543 tok/s; 6066 sec
  1284. [2019-02-04 19:31:42,063 INFO] Step 59050/102583; acc: 80.51; ppl: 1.83; xent: 0.60; lr: 0.00005; 1950/2687 tok/s; 6071 sec
  1285. [2019-02-04 19:31:46,836 INFO] Step 59100/102583; acc: 83.65; ppl: 1.74; xent: 0.56; lr: 0.00005; 2145/2585 tok/s; 6076 sec
  1286. [2019-02-04 19:31:52,114 INFO] Step 59150/102583; acc: 82.36; ppl: 1.74; xent: 0.55; lr: 0.00005; 2102/2653 tok/s; 6082 sec
  1287. [2019-02-04 19:31:56,765 INFO] Step 59200/102583; acc: 81.66; ppl: 1.83; xent: 0.61; lr: 0.00005; 2065/2571 tok/s; 6086 sec
  1288. [2019-02-04 19:32:01,850 INFO] Step 59250/102583; acc: 82.17; ppl: 1.76; xent: 0.56; lr: 0.00005; 1978/2609 tok/s; 6091 sec
  1289. [2019-02-04 19:32:06,952 INFO] Step 59300/102583; acc: 82.46; ppl: 1.76; xent: 0.56; lr: 0.00005; 2015/2636 tok/s; 6096 sec
  1290. [2019-02-04 19:32:11,991 INFO] Step 59350/102583; acc: 81.65; ppl: 1.82; xent: 0.60; lr: 0.00005; 2038/2461 tok/s; 6101 sec
  1291. [2019-02-04 19:32:16,915 INFO] Step 59400/102583; acc: 80.81; ppl: 1.83; xent: 0.60; lr: 0.00005; 2007/2762 tok/s; 6106 sec
  1292. [2019-02-04 19:32:22,214 INFO] Step 59450/102583; acc: 82.48; ppl: 1.76; xent: 0.57; lr: 0.00005; 1927/2544 tok/s; 6112 sec
  1293. [2019-02-04 19:32:27,222 INFO] Step 59500/102583; acc: 83.07; ppl: 1.73; xent: 0.55; lr: 0.00005; 2207/2630 tok/s; 6117 sec
  1294. [2019-02-04 19:32:31,966 INFO] Step 59550/102583; acc: 80.46; ppl: 1.88; xent: 0.63; lr: 0.00005; 2082/2578 tok/s; 6121 sec
  1295. [2019-02-04 19:32:36,908 INFO] Step 59600/102583; acc: 80.32; ppl: 1.86; xent: 0.62; lr: 0.00005; 1926/2676 tok/s; 6126 sec
  1296. [2019-02-04 19:32:41,859 INFO] Step 59650/102583; acc: 82.79; ppl: 1.78; xent: 0.57; lr: 0.00005; 2154/2658 tok/s; 6131 sec
  1297. [2019-02-04 19:32:47,043 INFO] Step 59700/102583; acc: 82.41; ppl: 1.76; xent: 0.57; lr: 0.00005; 2041/2495 tok/s; 6136 sec
  1298. [2019-02-04 19:32:52,240 INFO] Step 59750/102583; acc: 82.36; ppl: 1.76; xent: 0.56; lr: 0.00005; 2042/2624 tok/s; 6142 sec
  1299. [2019-02-04 19:32:57,200 INFO] Step 59800/102583; acc: 80.88; ppl: 1.82; xent: 0.60; lr: 0.00005; 1835/2725 tok/s; 6147 sec
  1300. [2019-02-04 19:33:02,176 INFO] Step 59850/102583; acc: 82.31; ppl: 1.77; xent: 0.57; lr: 0.00005; 2208/2682 tok/s; 6152 sec
  1301. [2019-02-04 19:33:07,017 INFO] Step 59900/102583; acc: 81.64; ppl: 1.83; xent: 0.60; lr: 0.00005; 1939/2502 tok/s; 6156 sec
  1302. [2019-02-04 19:33:11,705 INFO] Step 59950/102583; acc: 81.08; ppl: 1.86; xent: 0.62; lr: 0.00005; 2002/2611 tok/s; 6161 sec
  1303. [2019-02-04 19:33:16,981 INFO] Step 60000/102583; acc: 82.81; ppl: 1.72; xent: 0.54; lr: 0.00003; 2116/2564 tok/s; 6166 sec
  1304. [2019-02-04 19:33:17,171 INFO] Loading dataset from data/ted.valid.0.pt, number of examples: 873
  1305. [2019-02-04 19:33:18,475 INFO] Validation perplexity: 1.74624
  1306. [2019-02-04 19:33:18,476 INFO] Validation accuracy: 82.7164
  1307. [2019-02-04 19:33:18,477 INFO] Saving checkpoint models/ted1_step_60000.pt
  1308. [2019-02-04 19:33:29,593 INFO] Step 60050/102583; acc: 80.24; ppl: 1.87; xent: 0.63; lr: 0.00003; 780/1083 tok/s; 6179 sec
  1309. [2019-02-04 19:33:34,645 INFO] Step 60100/102583; acc: 82.97; ppl: 1.73; xent: 0.55; lr: 0.00003; 2071/2597 tok/s; 6184 sec
  1310. [2019-02-04 19:33:39,703 INFO] Step 60150/102583; acc: 82.82; ppl: 1.77; xent: 0.57; lr: 0.00003; 2010/2657 tok/s; 6189 sec
  1311. [2019-02-04 19:33:44,451 INFO] Step 60200/102583; acc: 80.30; ppl: 1.89; xent: 0.63; lr: 0.00003; 2002/2628 tok/s; 6194 sec
  1312. [2019-02-04 19:33:49,533 INFO] Step 60250/102583; acc: 82.55; ppl: 1.75; xent: 0.56; lr: 0.00003; 2027/2623 tok/s; 6199 sec
  1313. [2019-02-04 19:33:54,381 INFO] Step 60300/102583; acc: 82.57; ppl: 1.78; xent: 0.58; lr: 0.00003; 2131/2538 tok/s; 6204 sec
  1314. [2019-02-04 19:33:59,687 INFO] Step 60350/102583; acc: 83.07; ppl: 1.73; xent: 0.55; lr: 0.00003; 2135/2554 tok/s; 6209 sec
  1315. [2019-02-04 19:34:04,390 INFO] Step 60400/102583; acc: 82.05; ppl: 1.80; xent: 0.59; lr: 0.00003; 1886/2634 tok/s; 6214 sec
  1316. [2019-02-04 19:34:09,361 INFO] Step 60450/102583; acc: 80.87; ppl: 1.87; xent: 0.63; lr: 0.00003; 1977/2639 tok/s; 6219 sec
  1317. [2019-02-04 19:34:14,401 INFO] Step 60500/102583; acc: 82.27; ppl: 1.78; xent: 0.58; lr: 0.00003; 2048/2554 tok/s; 6224 sec
  1318. [2019-02-04 19:34:19,179 INFO] Step 60550/102583; acc: 82.82; ppl: 1.75; xent: 0.56; lr: 0.00003; 2097/2721 tok/s; 6229 sec
  1319. [2019-02-04 19:34:24,222 INFO] Step 60600/102583; acc: 82.46; ppl: 1.78; xent: 0.58; lr: 0.00003; 2145/2469 tok/s; 6234 sec
  1320. [2019-02-04 19:34:28,967 INFO] Step 60650/102583; acc: 82.02; ppl: 1.77; xent: 0.57; lr: 0.00003; 2079/2754 tok/s; 6238 sec
  1321. [2019-02-04 19:34:34,137 INFO] Step 60700/102583; acc: 82.42; ppl: 1.76; xent: 0.57; lr: 0.00003; 1942/2633 tok/s; 6244 sec
  1322. [2019-02-04 19:34:39,102 INFO] Step 60750/102583; acc: 81.72; ppl: 1.79; xent: 0.58; lr: 0.00003; 2056/2655 tok/s; 6248 sec
  1323. [2019-02-04 19:34:44,050 INFO] Step 60800/102583; acc: 82.73; ppl: 1.76; xent: 0.57; lr: 0.00003; 2103/2508 tok/s; 6253 sec
  1324. [2019-02-04 19:34:49,069 INFO] Step 60850/102583; acc: 82.73; ppl: 1.74; xent: 0.55; lr: 0.00003; 2020/2670 tok/s; 6258 sec
  1325. [2019-02-04 19:34:54,468 INFO] Step 60900/102583; acc: 81.03; ppl: 1.82; xent: 0.60; lr: 0.00003; 2089/2671 tok/s; 6264 sec
  1326. [2019-02-04 19:34:59,326 INFO] Step 60950/102583; acc: 81.87; ppl: 1.81; xent: 0.59; lr: 0.00003; 2025/2616 tok/s; 6269 sec
  1327. [2019-02-04 19:35:04,483 INFO] Step 61000/102583; acc: 82.49; ppl: 1.76; xent: 0.57; lr: 0.00003; 2049/2693 tok/s; 6274 sec
  1328. [2019-02-04 19:35:09,590 INFO] Step 61050/102583; acc: 82.81; ppl: 1.73; xent: 0.55; lr: 0.00003; 2145/2500 tok/s; 6279 sec
  1329. [2019-02-04 19:35:14,691 INFO] Step 61100/102583; acc: 81.59; ppl: 1.81; xent: 0.59; lr: 0.00003; 1773/2871 tok/s; 6284 sec
  1330. [2019-02-04 19:35:19,556 INFO] Step 61150/102583; acc: 81.17; ppl: 1.86; xent: 0.62; lr: 0.00003; 2075/2455 tok/s; 6289 sec
  1331. [2019-02-04 19:35:24,461 INFO] Step 61200/102583; acc: 83.34; ppl: 1.70; xent: 0.53; lr: 0.00003; 2141/2710 tok/s; 6294 sec
  1332. [2019-02-04 19:35:29,219 INFO] Step 61250/102583; acc: 81.76; ppl: 1.79; xent: 0.58; lr: 0.00003; 1938/2741 tok/s; 6299 sec
  1333. [2019-02-04 19:35:34,572 INFO] Step 61300/102583; acc: 82.04; ppl: 1.78; xent: 0.58; lr: 0.00003; 2111/2492 tok/s; 6304 sec
  1334. [2019-02-04 19:35:39,328 INFO] Step 61350/102583; acc: 83.24; ppl: 1.74; xent: 0.56; lr: 0.00003; 2090/2668 tok/s; 6309 sec
  1335. [2019-02-04 19:35:44,442 INFO] Step 61400/102583; acc: 82.24; ppl: 1.77; xent: 0.57; lr: 0.00003; 2142/2470 tok/s; 6314 sec
  1336. [2019-02-04 19:35:49,564 INFO] Step 61450/102583; acc: 81.39; ppl: 1.83; xent: 0.60; lr: 0.00003; 1980/2663 tok/s; 6319 sec
  1337. [2019-02-04 19:35:54,215 INFO] Step 61500/102583; acc: 82.75; ppl: 1.75; xent: 0.56; lr: 0.00003; 2026/2689 tok/s; 6324 sec
  1338. [2019-02-04 19:35:58,963 INFO] Step 61550/102583; acc: 81.55; ppl: 1.83; xent: 0.60; lr: 0.00003; 2086/2537 tok/s; 6328 sec
  1339. [2019-02-04 19:36:04,228 INFO] Step 61600/102583; acc: 82.17; ppl: 1.77; xent: 0.57; lr: 0.00003; 2000/2669 tok/s; 6334 sec
  1340. [2019-02-04 19:36:13,052 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  1341. [2019-02-04 19:36:14,991 INFO] Step 61650/102583; acc: 81.10; ppl: 1.86; xent: 0.62; lr: 0.00003; 893/1186 tok/s; 6344 sec
  1342. [2019-02-04 19:36:20,161 INFO] Step 61700/102583; acc: 83.36; ppl: 1.73; xent: 0.55; lr: 0.00003; 2220/2542 tok/s; 6350 sec
  1343. [2019-02-04 19:36:24,857 INFO] Step 61750/102583; acc: 82.50; ppl: 1.77; xent: 0.57; lr: 0.00003; 1917/2712 tok/s; 6354 sec
  1344. [2019-02-04 19:36:30,084 INFO] Step 61800/102583; acc: 83.94; ppl: 1.70; xent: 0.53; lr: 0.00003; 2067/2658 tok/s; 6359 sec
  1345. [2019-02-04 19:36:34,923 INFO] Step 61850/102583; acc: 81.57; ppl: 1.82; xent: 0.60; lr: 0.00003; 2117/2688 tok/s; 6364 sec
  1346. [2019-02-04 19:36:39,943 INFO] Step 61900/102583; acc: 82.60; ppl: 1.74; xent: 0.56; lr: 0.00003; 2097/2654 tok/s; 6369 sec
  1347. [2019-02-04 19:36:44,952 INFO] Step 61950/102583; acc: 81.80; ppl: 1.80; xent: 0.59; lr: 0.00003; 2002/2554 tok/s; 6374 sec
  1348. [2019-02-04 19:36:49,720 INFO] Step 62000/102583; acc: 82.69; ppl: 1.77; xent: 0.57; lr: 0.00003; 2112/2790 tok/s; 6379 sec
  1349. [2019-02-04 19:36:54,838 INFO] Step 62050/102583; acc: 81.27; ppl: 1.82; xent: 0.60; lr: 0.00003; 2043/2577 tok/s; 6384 sec
  1350. [2019-02-04 19:36:59,791 INFO] Step 62100/102583; acc: 80.96; ppl: 1.86; xent: 0.62; lr: 0.00003; 1963/2638 tok/s; 6389 sec
  1351. [2019-02-04 19:37:04,485 INFO] Step 62150/102583; acc: 83.14; ppl: 1.71; xent: 0.54; lr: 0.00003; 2213/2662 tok/s; 6394 sec
  1352. [2019-02-04 19:37:09,768 INFO] Step 62200/102583; acc: 82.04; ppl: 1.80; xent: 0.59; lr: 0.00003; 1957/2587 tok/s; 6399 sec
  1353. [2019-02-04 19:37:14,468 INFO] Step 62250/102583; acc: 81.86; ppl: 1.83; xent: 0.60; lr: 0.00003; 2134/2549 tok/s; 6404 sec
  1354. [2019-02-04 19:37:19,473 INFO] Step 62300/102583; acc: 80.56; ppl: 1.83; xent: 0.60; lr: 0.00003; 1943/2754 tok/s; 6409 sec
  1355. [2019-02-04 19:37:24,394 INFO] Step 62350/102583; acc: 83.91; ppl: 1.72; xent: 0.54; lr: 0.00003; 2101/2595 tok/s; 6414 sec
  1356. [2019-02-04 19:37:29,478 INFO] Step 62400/102583; acc: 82.64; ppl: 1.75; xent: 0.56; lr: 0.00003; 2162/2626 tok/s; 6419 sec
  1357. [2019-02-04 19:37:34,140 INFO] Step 62450/102583; acc: 82.27; ppl: 1.79; xent: 0.58; lr: 0.00003; 2107/2558 tok/s; 6424 sec
  1358. [2019-02-04 19:37:39,270 INFO] Step 62500/102583; acc: 81.88; ppl: 1.77; xent: 0.57; lr: 0.00003; 1941/2626 tok/s; 6429 sec
  1359. [2019-02-04 19:37:44,362 INFO] Step 62550/102583; acc: 83.20; ppl: 1.73; xent: 0.55; lr: 0.00003; 2044/2632 tok/s; 6434 sec
  1360. [2019-02-04 19:37:49,564 INFO] Step 62600/102583; acc: 81.46; ppl: 1.82; xent: 0.60; lr: 0.00003; 1993/2507 tok/s; 6439 sec
  1361. [2019-02-04 19:37:54,517 INFO] Step 62650/102583; acc: 80.38; ppl: 1.85; xent: 0.61; lr: 0.00003; 1971/2731 tok/s; 6444 sec
  1362. [2019-02-04 19:37:59,641 INFO] Step 62700/102583; acc: 84.22; ppl: 1.69; xent: 0.53; lr: 0.00003; 2016/2548 tok/s; 6449 sec
  1363. [2019-02-04 19:38:04,622 INFO] Step 62750/102583; acc: 82.06; ppl: 1.78; xent: 0.58; lr: 0.00003; 2206/2617 tok/s; 6454 sec
  1364. [2019-02-04 19:38:09,374 INFO] Step 62800/102583; acc: 81.30; ppl: 1.84; xent: 0.61; lr: 0.00003; 2062/2572 tok/s; 6459 sec
  1365. [2019-02-04 19:38:14,379 INFO] Step 62850/102583; acc: 81.11; ppl: 1.84; xent: 0.61; lr: 0.00003; 1947/2696 tok/s; 6464 sec
  1366. [2019-02-04 19:38:19,270 INFO] Step 62900/102583; acc: 82.19; ppl: 1.78; xent: 0.58; lr: 0.00003; 2118/2711 tok/s; 6469 sec
  1367. [2019-02-04 19:38:24,315 INFO] Step 62950/102583; acc: 83.41; ppl: 1.72; xent: 0.54; lr: 0.00003; 2109/2412 tok/s; 6474 sec
  1368. [2019-02-04 19:38:29,552 INFO] Step 63000/102583; acc: 82.13; ppl: 1.77; xent: 0.57; lr: 0.00003; 1941/2664 tok/s; 6479 sec
  1369. [2019-02-04 19:38:34,324 INFO] Step 63050/102583; acc: 81.65; ppl: 1.80; xent: 0.59; lr: 0.00003; 1910/2672 tok/s; 6484 sec
  1370. [2019-02-04 19:38:39,422 INFO] Step 63100/102583; acc: 82.25; ppl: 1.77; xent: 0.57; lr: 0.00003; 2102/2661 tok/s; 6489 sec
  1371. [2019-02-04 19:38:44,270 INFO] Step 63150/102583; acc: 81.58; ppl: 1.83; xent: 0.61; lr: 0.00003; 2025/2553 tok/s; 6494 sec
  1372. [2019-02-04 19:38:49,128 INFO] Step 63200/102583; acc: 82.02; ppl: 1.80; xent: 0.59; lr: 0.00003; 2042/2594 tok/s; 6499 sec
  1373. [2019-02-04 19:38:54,313 INFO] Step 63250/102583; acc: 82.09; ppl: 1.75; xent: 0.56; lr: 0.00003; 2038/2642 tok/s; 6504 sec
  1374. [2019-02-04 19:38:59,208 INFO] Step 63300/102583; acc: 80.80; ppl: 1.85; xent: 0.62; lr: 0.00003; 2084/2642 tok/s; 6509 sec
  1375. [2019-02-04 19:39:04,140 INFO] Step 63350/102583; acc: 82.79; ppl: 1.74; xent: 0.56; lr: 0.00003; 2028/2669 tok/s; 6514 sec
  1376. [2019-02-04 19:39:09,169 INFO] Step 63400/102583; acc: 82.91; ppl: 1.77; xent: 0.57; lr: 0.00003; 2047/2635 tok/s; 6519 sec
  1377. [2019-02-04 19:39:13,974 INFO] Step 63450/102583; acc: 80.47; ppl: 1.88; xent: 0.63; lr: 0.00003; 1944/2546 tok/s; 6523 sec
  1378. [2019-02-04 19:39:19,069 INFO] Step 63500/102583; acc: 82.47; ppl: 1.74; xent: 0.55; lr: 0.00003; 2045/2705 tok/s; 6528 sec
  1379. [2019-02-04 19:39:24,097 INFO] Step 63550/102583; acc: 82.54; ppl: 1.77; xent: 0.57; lr: 0.00003; 2095/2513 tok/s; 6533 sec
  1380. [2019-02-04 19:39:29,308 INFO] Step 63600/102583; acc: 82.88; ppl: 1.76; xent: 0.57; lr: 0.00003; 2139/2521 tok/s; 6539 sec
  1381. [2019-02-04 19:39:33,850 INFO] Step 63650/102583; acc: 82.28; ppl: 1.79; xent: 0.58; lr: 0.00003; 1942/2728 tok/s; 6543 sec
  1382. [2019-02-04 19:39:38,869 INFO] Step 63700/102583; acc: 80.24; ppl: 1.88; xent: 0.63; lr: 0.00003; 1981/2645 tok/s; 6548 sec
  1383. [2019-02-04 19:39:43,938 INFO] Step 63750/102583; acc: 83.03; ppl: 1.75; xent: 0.56; lr: 0.00003; 1986/2580 tok/s; 6553 sec
  1384. [2019-02-04 19:39:48,649 INFO] Step 63800/102583; acc: 83.26; ppl: 1.73; xent: 0.55; lr: 0.00003; 2113/2677 tok/s; 6558 sec
  1385. [2019-02-04 19:39:53,691 INFO] Step 63850/102583; acc: 81.34; ppl: 1.83; xent: 0.61; lr: 0.00003; 2121/2477 tok/s; 6563 sec
  1386. [2019-02-04 19:39:58,819 INFO] Step 63900/102583; acc: 83.58; ppl: 1.68; xent: 0.52; lr: 0.00003; 2135/2611 tok/s; 6568 sec
  1387. [2019-02-04 19:40:03,665 INFO] Step 63950/102583; acc: 80.96; ppl: 1.83; xent: 0.60; lr: 0.00003; 1835/2810 tok/s; 6573 sec
  1388. [2019-02-04 19:40:08,686 INFO] Step 64000/102583; acc: 81.90; ppl: 1.79; xent: 0.58; lr: 0.00003; 2112/2632 tok/s; 6578 sec
  1389. [2019-02-04 19:40:13,609 INFO] Step 64050/102583; acc: 84.17; ppl: 1.70; xent: 0.53; lr: 0.00003; 2150/2513 tok/s; 6583 sec
  1390. [2019-02-04 19:40:18,558 INFO] Step 64100/102583; acc: 81.85; ppl: 1.78; xent: 0.58; lr: 0.00003; 1995/2668 tok/s; 6588 sec
  1391. [2019-02-04 19:40:23,543 INFO] Step 64150/102583; acc: 81.57; ppl: 1.80; xent: 0.59; lr: 0.00003; 2078/2656 tok/s; 6593 sec
  1392. [2019-02-04 19:40:28,213 INFO] Step 64200/102583; acc: 81.72; ppl: 1.82; xent: 0.60; lr: 0.00003; 2064/2621 tok/s; 6598 sec
  1393. [2019-02-04 19:40:33,263 INFO] Step 64250/102583; acc: 83.06; ppl: 1.74; xent: 0.55; lr: 0.00003; 2081/2643 tok/s; 6603 sec
  1394. [2019-02-04 19:40:38,404 INFO] Step 64300/102583; acc: 82.65; ppl: 1.73; xent: 0.55; lr: 0.00003; 2081/2582 tok/s; 6608 sec
  1395. [2019-02-04 19:40:43,395 INFO] Step 64350/102583; acc: 81.41; ppl: 1.85; xent: 0.61; lr: 0.00003; 1774/2809 tok/s; 6613 sec
  1396. [2019-02-04 19:40:48,252 INFO] Step 64400/102583; acc: 80.74; ppl: 1.87; xent: 0.63; lr: 0.00003; 1997/2460 tok/s; 6618 sec
  1397. [2019-02-04 19:40:53,206 INFO] Step 64450/102583; acc: 83.52; ppl: 1.69; xent: 0.53; lr: 0.00003; 2197/2743 tok/s; 6623 sec
  1398. [2019-02-04 19:40:58,158 INFO] Step 64500/102583; acc: 82.58; ppl: 1.75; xent: 0.56; lr: 0.00003; 2007/2590 tok/s; 6628 sec
  1399. [2019-02-04 19:41:03,331 INFO] Step 64550/102583; acc: 82.29; ppl: 1.77; xent: 0.57; lr: 0.00003; 2033/2612 tok/s; 6633 sec
  1400. [2019-02-04 19:41:08,205 INFO] Step 64600/102583; acc: 81.54; ppl: 1.83; xent: 0.60; lr: 0.00003; 2103/2646 tok/s; 6638 sec
  1401. [2019-02-04 19:41:13,328 INFO] Step 64650/102583; acc: 83.25; ppl: 1.71; xent: 0.54; lr: 0.00003; 2096/2493 tok/s; 6643 sec
  1402. [2019-02-04 19:41:18,268 INFO] Step 64700/102583; acc: 81.61; ppl: 1.83; xent: 0.60; lr: 0.00003; 2100/2603 tok/s; 6648 sec
  1403. [2019-02-04 19:41:23,105 INFO] Step 64750/102583; acc: 82.19; ppl: 1.78; xent: 0.58; lr: 0.00003; 1984/2659 tok/s; 6652 sec
  1404. [2019-02-04 19:41:27,796 INFO] Step 64800/102583; acc: 81.97; ppl: 1.80; xent: 0.59; lr: 0.00003; 2032/2634 tok/s; 6657 sec
  1405. [2019-02-04 19:41:33,091 INFO] Step 64850/102583; acc: 81.84; ppl: 1.79; xent: 0.58; lr: 0.00003; 1973/2607 tok/s; 6662 sec
  1406. [2019-02-04 19:41:41,598 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  1407. [2019-02-04 19:41:44,136 INFO] Step 64900/102583; acc: 81.32; ppl: 1.84; xent: 0.61; lr: 0.00003; 926/1161 tok/s; 6674 sec
  1408. [2019-02-04 19:41:49,119 INFO] Step 64950/102583; acc: 83.34; ppl: 1.75; xent: 0.56; lr: 0.00003; 2175/2517 tok/s; 6679 sec
  1409. [2019-02-04 19:41:53,985 INFO] Step 65000/102583; acc: 82.47; ppl: 1.76; xent: 0.57; lr: 0.00003; 1909/2743 tok/s; 6683 sec
  1410. [2019-02-04 19:41:54,028 INFO] Saving checkpoint models/ted1_step_65000.pt
  1411. [2019-02-04 19:42:04,946 INFO] Step 65050/102583; acc: 83.91; ppl: 1.71; xent: 0.53; lr: 0.00003; 940/1236 tok/s; 6694 sec
  1412. [2019-02-04 19:42:09,852 INFO] Step 65100/102583; acc: 81.38; ppl: 1.82; xent: 0.60; lr: 0.00003; 2140/2720 tok/s; 6699 sec
  1413. [2019-02-04 19:42:15,050 INFO] Step 65150/102583; acc: 82.45; ppl: 1.73; xent: 0.55; lr: 0.00003; 2006/2705 tok/s; 6704 sec
  1414. [2019-02-04 19:42:19,904 INFO] Step 65200/102583; acc: 81.83; ppl: 1.79; xent: 0.58; lr: 0.00003; 2127/2516 tok/s; 6709 sec
  1415. [2019-02-04 19:42:24,665 INFO] Step 65250/102583; acc: 83.17; ppl: 1.77; xent: 0.57; lr: 0.00003; 2149/2710 tok/s; 6714 sec
  1416. [2019-02-04 19:42:29,773 INFO] Step 65300/102583; acc: 80.23; ppl: 1.85; xent: 0.62; lr: 0.00003; 1997/2587 tok/s; 6719 sec
  1417. [2019-02-04 19:42:34,707 INFO] Step 65350/102583; acc: 81.61; ppl: 1.83; xent: 0.61; lr: 0.00003; 1931/2647 tok/s; 6724 sec
  1418. [2019-02-04 19:42:39,674 INFO] Step 65400/102583; acc: 83.53; ppl: 1.67; xent: 0.51; lr: 0.00003; 2186/2724 tok/s; 6729 sec
  1419. [2019-02-04 19:42:44,701 INFO] Step 65450/102583; acc: 80.81; ppl: 1.88; xent: 0.63; lr: 0.00003; 1950/2531 tok/s; 6734 sec
  1420. [2019-02-04 19:42:49,425 INFO] Step 65500/102583; acc: 81.67; ppl: 1.84; xent: 0.61; lr: 0.00003; 2123/2568 tok/s; 6739 sec
  1421. [2019-02-04 19:42:54,464 INFO] Step 65550/102583; acc: 81.29; ppl: 1.78; xent: 0.58; lr: 0.00003; 2019/2762 tok/s; 6744 sec
  1422. [2019-02-04 19:42:59,501 INFO] Step 65600/102583; acc: 84.05; ppl: 1.70; xent: 0.53; lr: 0.00003; 2111/2566 tok/s; 6749 sec
  1423. [2019-02-04 19:43:04,483 INFO] Step 65650/102583; acc: 81.84; ppl: 1.79; xent: 0.58; lr: 0.00003; 2122/2658 tok/s; 6754 sec
  1424. [2019-02-04 19:43:09,176 INFO] Step 65700/102583; acc: 83.03; ppl: 1.76; xent: 0.56; lr: 0.00003; 2179/2457 tok/s; 6759 sec
  1425. [2019-02-04 19:43:14,263 INFO] Step 65750/102583; acc: 81.73; ppl: 1.77; xent: 0.57; lr: 0.00003; 1843/2744 tok/s; 6764 sec
  1426. [2019-02-04 19:43:19,487 INFO] Step 65800/102583; acc: 82.94; ppl: 1.75; xent: 0.56; lr: 0.00003; 2107/2518 tok/s; 6769 sec
  1427. [2019-02-04 19:43:24,526 INFO] Step 65850/102583; acc: 81.90; ppl: 1.81; xent: 0.59; lr: 0.00003; 2010/2544 tok/s; 6774 sec
  1428. [2019-02-04 19:43:29,627 INFO] Step 65900/102583; acc: 80.45; ppl: 1.86; xent: 0.62; lr: 0.00003; 1875/2744 tok/s; 6779 sec
  1429. [2019-02-04 19:43:34,677 INFO] Step 65950/102583; acc: 84.03; ppl: 1.70; xent: 0.53; lr: 0.00003; 1997/2595 tok/s; 6784 sec
  1430. [2019-02-04 19:43:39,715 INFO] Step 66000/102583; acc: 81.49; ppl: 1.81; xent: 0.59; lr: 0.00003; 2166/2527 tok/s; 6789 sec
  1431. [2019-02-04 19:43:44,302 INFO] Step 66050/102583; acc: 80.99; ppl: 1.85; xent: 0.61; lr: 0.00003; 2124/2622 tok/s; 6794 sec
  1432. [2019-02-04 19:43:49,315 INFO] Step 66100/102583; acc: 82.18; ppl: 1.77; xent: 0.57; lr: 0.00003; 2039/2704 tok/s; 6799 sec
  1433. [2019-02-04 19:43:54,435 INFO] Step 66150/102583; acc: 82.64; ppl: 1.76; xent: 0.56; lr: 0.00003; 2056/2662 tok/s; 6804 sec
  1434. [2019-02-04 19:43:59,393 INFO] Step 66200/102583; acc: 83.91; ppl: 1.70; xent: 0.53; lr: 0.00003; 2188/2358 tok/s; 6809 sec
  1435. [2019-02-04 19:44:04,522 INFO] Step 66250/102583; acc: 80.28; ppl: 1.86; xent: 0.62; lr: 0.00003; 1847/2782 tok/s; 6814 sec
  1436. [2019-02-04 19:44:09,374 INFO] Step 66300/102583; acc: 81.82; ppl: 1.79; xent: 0.58; lr: 0.00003; 1865/2723 tok/s; 6819 sec
  1437. [2019-02-04 19:44:14,276 INFO] Step 66350/102583; acc: 83.54; ppl: 1.71; xent: 0.54; lr: 0.00003; 2199/2531 tok/s; 6824 sec
  1438. [2019-02-04 19:44:19,113 INFO] Step 66400/102583; acc: 81.14; ppl: 1.85; xent: 0.62; lr: 0.00003; 1995/2598 tok/s; 6829 sec
  1439. [2019-02-04 19:44:24,260 INFO] Step 66450/102583; acc: 82.72; ppl: 1.74; xent: 0.55; lr: 0.00003; 2050/2535 tok/s; 6834 sec
  1440. [2019-02-04 19:44:29,297 INFO] Step 66500/102583; acc: 81.50; ppl: 1.80; xent: 0.59; lr: 0.00003; 2090/2685 tok/s; 6839 sec
  1441. [2019-02-04 19:44:34,159 INFO] Step 66550/102583; acc: 81.68; ppl: 1.81; xent: 0.59; lr: 0.00003; 1995/2672 tok/s; 6844 sec
  1442. [2019-02-04 19:44:39,143 INFO] Step 66600/102583; acc: 83.16; ppl: 1.73; xent: 0.55; lr: 0.00003; 2021/2728 tok/s; 6849 sec
  1443. [2019-02-04 19:44:44,043 INFO] Step 66650/102583; acc: 82.19; ppl: 1.80; xent: 0.59; lr: 0.00003; 2055/2604 tok/s; 6853 sec
  1444. [2019-02-04 19:44:49,070 INFO] Step 66700/102583; acc: 80.95; ppl: 1.84; xent: 0.61; lr: 0.00003; 1915/2539 tok/s; 6858 sec
  1445. [2019-02-04 19:44:53,914 INFO] Step 66750/102583; acc: 82.09; ppl: 1.78; xent: 0.58; lr: 0.00003; 2162/2692 tok/s; 6863 sec
  1446. [2019-02-04 19:44:59,054 INFO] Step 66800/102583; acc: 82.07; ppl: 1.79; xent: 0.58; lr: 0.00003; 2012/2565 tok/s; 6868 sec
  1447. [2019-02-04 19:45:04,155 INFO] Step 66850/102583; acc: 83.49; ppl: 1.73; xent: 0.55; lr: 0.00003; 2198/2456 tok/s; 6874 sec
  1448. [2019-02-04 19:45:08,735 INFO] Step 66900/102583; acc: 81.04; ppl: 1.86; xent: 0.62; lr: 0.00003; 1828/2766 tok/s; 6878 sec
  1449. [2019-02-04 19:45:13,774 INFO] Step 66950/102583; acc: 81.14; ppl: 1.83; xent: 0.61; lr: 0.00003; 2058/2606 tok/s; 6883 sec
  1450. [2019-02-04 19:45:18,708 INFO] Step 67000/102583; acc: 83.57; ppl: 1.74; xent: 0.55; lr: 0.00003; 2061/2566 tok/s; 6888 sec
  1451. [2019-02-04 19:45:23,522 INFO] Step 67050/102583; acc: 83.04; ppl: 1.72; xent: 0.54; lr: 0.00003; 2032/2697 tok/s; 6893 sec
  1452. [2019-02-04 19:45:28,554 INFO] Step 67100/102583; acc: 81.44; ppl: 1.82; xent: 0.60; lr: 0.00003; 2184/2482 tok/s; 6898 sec
  1453. [2019-02-04 19:45:33,578 INFO] Step 67150/102583; acc: 83.16; ppl: 1.71; xent: 0.54; lr: 0.00003; 2116/2594 tok/s; 6903 sec
  1454. [2019-02-04 19:45:38,644 INFO] Step 67200/102583; acc: 81.23; ppl: 1.80; xent: 0.59; lr: 0.00003; 1824/2815 tok/s; 6908 sec
  1455. [2019-02-04 19:45:43,766 INFO] Step 67250/102583; acc: 82.87; ppl: 1.75; xent: 0.56; lr: 0.00003; 2043/2581 tok/s; 6913 sec
  1456. [2019-02-04 19:45:48,376 INFO] Step 67300/102583; acc: 82.59; ppl: 1.77; xent: 0.57; lr: 0.00003; 2169/2584 tok/s; 6918 sec
  1457. [2019-02-04 19:45:53,430 INFO] Step 67350/102583; acc: 82.55; ppl: 1.75; xent: 0.56; lr: 0.00003; 2026/2636 tok/s; 6923 sec
  1458. [2019-02-04 19:45:58,362 INFO] Step 67400/102583; acc: 81.09; ppl: 1.83; xent: 0.60; lr: 0.00003; 2095/2633 tok/s; 6928 sec
  1459. [2019-02-04 19:46:03,078 INFO] Step 67450/102583; acc: 81.77; ppl: 1.83; xent: 0.60; lr: 0.00003; 2012/2665 tok/s; 6932 sec
  1460. [2019-02-04 19:46:08,106 INFO] Step 67500/102583; acc: 83.43; ppl: 1.70; xent: 0.53; lr: 0.00003; 2078/2639 tok/s; 6937 sec
  1461. [2019-02-04 19:46:13,275 INFO] Step 67550/102583; acc: 82.66; ppl: 1.73; xent: 0.55; lr: 0.00003; 2075/2567 tok/s; 6943 sec
  1462. [2019-02-04 19:46:18,321 INFO] Step 67600/102583; acc: 81.15; ppl: 1.84; xent: 0.61; lr: 0.00003; 1823/2794 tok/s; 6948 sec
  1463. [2019-02-04 19:46:23,069 INFO] Step 67650/102583; acc: 81.39; ppl: 1.85; xent: 0.61; lr: 0.00003; 2017/2509 tok/s; 6952 sec
  1464. [2019-02-04 19:46:28,012 INFO] Step 67700/102583; acc: 83.19; ppl: 1.70; xent: 0.53; lr: 0.00003; 2150/2761 tok/s; 6957 sec
  1465. [2019-02-04 19:46:33,109 INFO] Step 67750/102583; acc: 82.24; ppl: 1.78; xent: 0.58; lr: 0.00003; 1986/2472 tok/s; 6962 sec
  1466. [2019-02-04 19:46:38,121 INFO] Step 67800/102583; acc: 83.01; ppl: 1.73; xent: 0.55; lr: 0.00003; 2127/2711 tok/s; 6968 sec
  1467. [2019-02-04 19:46:43,185 INFO] Step 67850/102583; acc: 81.39; ppl: 1.82; xent: 0.60; lr: 0.00003; 2134/2571 tok/s; 6973 sec
  1468. [2019-02-04 19:46:48,289 INFO] Step 67900/102583; acc: 82.99; ppl: 1.74; xent: 0.56; lr: 0.00003; 2036/2562 tok/s; 6978 sec
  1469. [2019-02-04 19:46:52,980 INFO] Step 67950/102583; acc: 81.64; ppl: 1.83; xent: 0.60; lr: 0.00003; 2097/2579 tok/s; 6982 sec
  1470. [2019-02-04 19:46:57,863 INFO] Step 68000/102583; acc: 82.26; ppl: 1.76; xent: 0.57; lr: 0.00003; 2071/2663 tok/s; 6987 sec
  1471. [2019-02-04 19:47:02,753 INFO] Step 68050/102583; acc: 81.94; ppl: 1.80; xent: 0.59; lr: 0.00003; 1914/2673 tok/s; 6992 sec
  1472. [2019-02-04 19:47:07,917 INFO] Step 68100/102583; acc: 81.98; ppl: 1.77; xent: 0.57; lr: 0.00003; 1981/2600 tok/s; 6997 sec
  1473. [2019-02-04 19:47:16,082 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  1474. [2019-02-04 19:47:19,365 INFO] Step 68150/102583; acc: 82.85; ppl: 1.75; xent: 0.56; lr: 0.00003; 905/1174 tok/s; 7009 sec
  1475. [2019-02-04 19:47:24,200 INFO] Step 68200/102583; acc: 81.83; ppl: 1.84; xent: 0.61; lr: 0.00003; 2172/2383 tok/s; 7014 sec
  1476. [2019-02-04 19:47:29,005 INFO] Step 68250/102583; acc: 82.20; ppl: 1.77; xent: 0.57; lr: 0.00003; 1949/2798 tok/s; 7018 sec
  1477. [2019-02-04 19:47:34,235 INFO] Step 68300/102583; acc: 83.35; ppl: 1.73; xent: 0.55; lr: 0.00003; 1979/2664 tok/s; 7024 sec
  1478. [2019-02-04 19:47:39,058 INFO] Step 68350/102583; acc: 82.06; ppl: 1.80; xent: 0.59; lr: 0.00003; 2159/2754 tok/s; 7028 sec
  1479. [2019-02-04 19:47:44,142 INFO] Step 68400/102583; acc: 82.67; ppl: 1.74; xent: 0.55; lr: 0.00003; 2034/2630 tok/s; 7034 sec
  1480. [2019-02-04 19:47:49,012 INFO] Step 68450/102583; acc: 82.05; ppl: 1.78; xent: 0.57; lr: 0.00003; 2199/2525 tok/s; 7038 sec
  1481. [2019-02-04 19:47:53,726 INFO] Step 68500/102583; acc: 82.29; ppl: 1.79; xent: 0.58; lr: 0.00003; 2095/2750 tok/s; 7043 sec
  1482. [2019-02-04 19:47:58,918 INFO] Step 68550/102583; acc: 80.60; ppl: 1.85; xent: 0.61; lr: 0.00003; 1934/2587 tok/s; 7048 sec
  1483. [2019-02-04 19:48:04,158 INFO] Step 68600/102583; acc: 82.24; ppl: 1.79; xent: 0.58; lr: 0.00003; 2041/2644 tok/s; 7054 sec
  1484. [2019-02-04 19:48:08,935 INFO] Step 68650/102583; acc: 83.54; ppl: 1.68; xent: 0.52; lr: 0.00003; 2160/2717 tok/s; 7058 sec
  1485. [2019-02-04 19:48:14,146 INFO] Step 68700/102583; acc: 80.55; ppl: 1.89; xent: 0.63; lr: 0.00003; 1967/2496 tok/s; 7064 sec
  1486. [2019-02-04 19:48:18,926 INFO] Step 68750/102583; acc: 82.38; ppl: 1.80; xent: 0.59; lr: 0.00003; 2161/2571 tok/s; 7068 sec
  1487. [2019-02-04 19:48:23,866 INFO] Step 68800/102583; acc: 81.27; ppl: 1.79; xent: 0.58; lr: 0.00003; 1968/2776 tok/s; 7073 sec
  1488. [2019-02-04 19:48:28,887 INFO] Step 68850/102583; acc: 83.63; ppl: 1.73; xent: 0.55; lr: 0.00003; 2148/2549 tok/s; 7078 sec
  1489. [2019-02-04 19:48:33,893 INFO] Step 68900/102583; acc: 81.80; ppl: 1.79; xent: 0.58; lr: 0.00003; 2058/2682 tok/s; 7083 sec
  1490. [2019-02-04 19:48:38,561 INFO] Step 68950/102583; acc: 82.34; ppl: 1.80; xent: 0.59; lr: 0.00003; 2214/2476 tok/s; 7088 sec
  1491. [2019-02-04 19:48:43,682 INFO] Step 69000/102583; acc: 82.41; ppl: 1.72; xent: 0.54; lr: 0.00003; 1828/2784 tok/s; 7093 sec
  1492. [2019-02-04 19:48:48,922 INFO] Step 69050/102583; acc: 82.13; ppl: 1.79; xent: 0.58; lr: 0.00003; 2014/2434 tok/s; 7098 sec
  1493. [2019-02-04 19:48:53,942 INFO] Step 69100/102583; acc: 82.98; ppl: 1.75; xent: 0.56; lr: 0.00003; 2157/2592 tok/s; 7103 sec
  1494. [2019-02-04 19:48:59,107 INFO] Step 69150/102583; acc: 80.03; ppl: 1.88; xent: 0.63; lr: 0.00003; 1851/2671 tok/s; 7108 sec
  1495. [2019-02-04 19:49:04,011 INFO] Step 69200/102583; acc: 84.35; ppl: 1.69; xent: 0.52; lr: 0.00003; 2017/2645 tok/s; 7113 sec
  1496. [2019-02-04 19:49:09,008 INFO] Step 69250/102583; acc: 80.57; ppl: 1.86; xent: 0.62; lr: 0.00003; 2082/2547 tok/s; 7118 sec
  1497. [2019-02-04 19:49:13,570 INFO] Step 69300/102583; acc: 81.31; ppl: 1.83; xent: 0.60; lr: 0.00003; 2206/2606 tok/s; 7123 sec
  1498. [2019-02-04 19:49:18,778 INFO] Step 69350/102583; acc: 81.86; ppl: 1.79; xent: 0.58; lr: 0.00003; 1993/2741 tok/s; 7128 sec
  1499. [2019-02-04 19:49:23,874 INFO] Step 69400/102583; acc: 82.74; ppl: 1.75; xent: 0.56; lr: 0.00003; 2053/2653 tok/s; 7133 sec
  1500. [2019-02-04 19:49:29,069 INFO] Step 69450/102583; acc: 83.66; ppl: 1.71; xent: 0.53; lr: 0.00003; 2152/2337 tok/s; 7138 sec
  1501. [2019-02-04 19:49:34,001 INFO] Step 69500/102583; acc: 80.43; ppl: 1.83; xent: 0.61; lr: 0.00003; 1840/2827 tok/s; 7143 sec
  1502. [2019-02-04 19:49:38,803 INFO] Step 69550/102583; acc: 81.40; ppl: 1.82; xent: 0.60; lr: 0.00003; 1898/2691 tok/s; 7148 sec
  1503. [2019-02-04 19:49:43,865 INFO] Step 69600/102583; acc: 84.42; ppl: 1.67; xent: 0.51; lr: 0.00003; 2253/2531 tok/s; 7153 sec
  1504. [2019-02-04 19:49:48,557 INFO] Step 69650/102583; acc: 81.02; ppl: 1.87; xent: 0.63; lr: 0.00003; 1912/2622 tok/s; 7158 sec
  1505. [2019-02-04 19:49:53,709 INFO] Step 69700/102583; acc: 82.19; ppl: 1.77; xent: 0.57; lr: 0.00003; 2072/2534 tok/s; 7163 sec
  1506. [2019-02-04 19:49:58,789 INFO] Step 69750/102583; acc: 82.43; ppl: 1.76; xent: 0.56; lr: 0.00003; 2085/2728 tok/s; 7168 sec
  1507. [2019-02-04 19:50:03,602 INFO] Step 69800/102583; acc: 81.18; ppl: 1.84; xent: 0.61; lr: 0.00003; 1963/2639 tok/s; 7173 sec
  1508. [2019-02-04 19:50:08,593 INFO] Step 69850/102583; acc: 82.49; ppl: 1.75; xent: 0.56; lr: 0.00003; 2017/2732 tok/s; 7178 sec
  1509. [2019-02-04 19:50:13,606 INFO] Step 69900/102583; acc: 83.57; ppl: 1.73; xent: 0.55; lr: 0.00003; 2150/2558 tok/s; 7183 sec
  1510. [2019-02-04 19:50:18,577 INFO] Step 69950/102583; acc: 79.97; ppl: 1.90; xent: 0.64; lr: 0.00003; 1885/2518 tok/s; 7188 sec
  1511. [2019-02-04 19:50:23,355 INFO] Step 70000/102583; acc: 82.92; ppl: 1.74; xent: 0.55; lr: 0.00001; 2169/2727 tok/s; 7193 sec
  1512. [2019-02-04 19:50:23,384 INFO] Loading dataset from data/ted.valid.0.pt, number of examples: 873
  1513. [2019-02-04 19:50:24,685 INFO] Validation perplexity: 1.74237
  1514. [2019-02-04 19:50:24,686 INFO] Validation accuracy: 83.1895
  1515. [2019-02-04 19:50:24,687 INFO] Saving checkpoint models/ted1_step_70000.pt
  1516. [2019-02-04 19:50:36,050 INFO] Step 70050/102583; acc: 81.49; ppl: 1.83; xent: 0.60; lr: 0.00001; 784/1010 tok/s; 7205 sec
  1517. [2019-02-04 19:50:41,219 INFO] Step 70100/102583; acc: 83.74; ppl: 1.71; xent: 0.54; lr: 0.00001; 2194/2511 tok/s; 7211 sec
  1518. [2019-02-04 19:50:45,906 INFO] Step 70150/102583; acc: 80.53; ppl: 1.89; xent: 0.63; lr: 0.00001; 1861/2729 tok/s; 7215 sec
  1519. [2019-02-04 19:50:51,012 INFO] Step 70200/102583; acc: 82.59; ppl: 1.76; xent: 0.56; lr: 0.00001; 2068/2643 tok/s; 7220 sec
  1520. [2019-02-04 19:50:55,807 INFO] Step 70250/102583; acc: 82.33; ppl: 1.81; xent: 0.59; lr: 0.00001; 2021/2548 tok/s; 7225 sec
  1521. [2019-02-04 19:51:00,825 INFO] Step 70300/102583; acc: 83.84; ppl: 1.68; xent: 0.52; lr: 0.00001; 2032/2636 tok/s; 7230 sec
  1522. [2019-02-04 19:51:05,688 INFO] Step 70350/102583; acc: 81.44; ppl: 1.83; xent: 0.61; lr: 0.00001; 2196/2510 tok/s; 7235 sec
  1523. [2019-02-04 19:51:11,409 INFO] Step 70400/102583; acc: 83.18; ppl: 1.70; xent: 0.53; lr: 0.00001; 2145/2618 tok/s; 7241 sec
  1524. [2019-02-04 19:51:16,487 INFO] Step 70450/102583; acc: 80.81; ppl: 1.82; xent: 0.60; lr: 0.00001; 1840/2804 tok/s; 7246 sec
  1525. [2019-02-04 19:51:21,594 INFO] Step 70500/102583; acc: 82.81; ppl: 1.76; xent: 0.56; lr: 0.00001; 2074/2554 tok/s; 7251 sec
  1526. [2019-02-04 19:51:26,296 INFO] Step 70550/102583; acc: 82.95; ppl: 1.74; xent: 0.55; lr: 0.00001; 2103/2618 tok/s; 7256 sec
  1527. [2019-02-04 19:51:31,390 INFO] Step 70600/102583; acc: 82.56; ppl: 1.74; xent: 0.56; lr: 0.00001; 2072/2637 tok/s; 7261 sec
  1528. [2019-02-04 19:51:36,226 INFO] Step 70650/102583; acc: 80.77; ppl: 1.84; xent: 0.61; lr: 0.00001; 2054/2656 tok/s; 7266 sec
  1529. [2019-02-04 19:51:41,110 INFO] Step 70700/102583; acc: 81.97; ppl: 1.81; xent: 0.59; lr: 0.00001; 1901/2675 tok/s; 7270 sec
  1530. [2019-02-04 19:51:45,941 INFO] Step 70750/102583; acc: 83.35; ppl: 1.72; xent: 0.54; lr: 0.00001; 2178/2617 tok/s; 7275 sec
  1531. [2019-02-04 19:51:51,191 INFO] Step 70800/102583; acc: 82.80; ppl: 1.71; xent: 0.54; lr: 0.00001; 2068/2554 tok/s; 7281 sec
  1532. [2019-02-04 19:51:56,415 INFO] Step 70850/102583; acc: 80.84; ppl: 1.87; xent: 0.63; lr: 0.00001; 1814/2767 tok/s; 7286 sec
  1533. [2019-02-04 19:52:01,040 INFO] Step 70900/102583; acc: 81.97; ppl: 1.81; xent: 0.59; lr: 0.00001; 2128/2521 tok/s; 7290 sec
  1534. [2019-02-04 19:52:06,004 INFO] Step 70950/102583; acc: 82.94; ppl: 1.71; xent: 0.54; lr: 0.00001; 2033/2768 tok/s; 7295 sec
  1535. [2019-02-04 19:52:11,077 INFO] Step 71000/102583; acc: 82.92; ppl: 1.75; xent: 0.56; lr: 0.00001; 1954/2551 tok/s; 7300 sec
  1536. [2019-02-04 19:52:15,999 INFO] Step 71050/102583; acc: 82.01; ppl: 1.78; xent: 0.58; lr: 0.00001; 2181/2668 tok/s; 7305 sec
  1537. [2019-02-04 19:52:21,043 INFO] Step 71100/102583; acc: 82.18; ppl: 1.78; xent: 0.58; lr: 0.00001; 2130/2565 tok/s; 7310 sec
  1538. [2019-02-04 19:52:26,343 INFO] Step 71150/102583; acc: 83.54; ppl: 1.71; xent: 0.54; lr: 0.00001; 2056/2517 tok/s; 7316 sec
  1539. [2019-02-04 19:52:30,866 INFO] Step 71200/102583; acc: 81.24; ppl: 1.85; xent: 0.62; lr: 0.00001; 2078/2640 tok/s; 7320 sec
  1540. [2019-02-04 19:52:35,630 INFO] Step 71250/102583; acc: 82.27; ppl: 1.76; xent: 0.57; lr: 0.00001; 2069/2597 tok/s; 7325 sec
  1541. [2019-02-04 19:52:40,859 INFO] Step 71300/102583; acc: 82.81; ppl: 1.77; xent: 0.57; lr: 0.00001; 1923/2607 tok/s; 7330 sec
  1542. [2019-02-04 19:52:46,119 INFO] Step 71350/102583; acc: 81.93; ppl: 1.77; xent: 0.57; lr: 0.00001; 1975/2603 tok/s; 7336 sec
  1543. [2019-02-04 19:52:52,832 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  1544. [2019-02-04 19:52:56,934 INFO] Step 71400/102583; acc: 82.79; ppl: 1.76; xent: 0.56; lr: 0.00001; 998/1213 tok/s; 7346 sec
  1545. [2019-02-04 19:53:01,458 INFO] Step 71450/102583; acc: 81.07; ppl: 1.89; xent: 0.64; lr: 0.00001; 2095/2528 tok/s; 7351 sec
  1546. [2019-02-04 19:53:06,379 INFO] Step 71500/102583; acc: 83.03; ppl: 1.73; xent: 0.55; lr: 0.00001; 1937/2790 tok/s; 7356 sec
  1547. [2019-02-04 19:53:11,557 INFO] Step 71550/102583; acc: 82.91; ppl: 1.74; xent: 0.56; lr: 0.00001; 2083/2610 tok/s; 7361 sec
  1548. [2019-02-04 19:53:16,370 INFO] Step 71600/102583; acc: 82.35; ppl: 1.78; xent: 0.57; lr: 0.00001; 2052/2839 tok/s; 7366 sec
  1549. [2019-02-04 19:53:21,527 INFO] Step 71650/102583; acc: 82.59; ppl: 1.74; xent: 0.55; lr: 0.00001; 2041/2532 tok/s; 7371 sec
  1550. [2019-02-04 19:53:26,356 INFO] Step 71700/102583; acc: 82.59; ppl: 1.76; xent: 0.56; lr: 0.00001; 2219/2574 tok/s; 7376 sec
  1551. [2019-02-04 19:53:31,142 INFO] Step 71750/102583; acc: 81.86; ppl: 1.83; xent: 0.60; lr: 0.00001; 2128/2697 tok/s; 7381 sec
  1552. [2019-02-04 19:53:36,385 INFO] Step 71800/102583; acc: 80.86; ppl: 1.84; xent: 0.61; lr: 0.00001; 1790/2629 tok/s; 7386 sec
  1553. [2019-02-04 19:53:41,264 INFO] Step 71850/102583; acc: 82.67; ppl: 1.76; xent: 0.57; lr: 0.00001; 2236/2602 tok/s; 7391 sec
  1554. [2019-02-04 19:53:46,183 INFO] Step 71900/102583; acc: 83.12; ppl: 1.70; xent: 0.53; lr: 0.00001; 1958/2764 tok/s; 7396 sec
  1555. [2019-02-04 19:53:51,342 INFO] Step 71950/102583; acc: 81.34; ppl: 1.84; xent: 0.61; lr: 0.00001; 2143/2481 tok/s; 7401 sec
  1556. [2019-02-04 19:53:56,184 INFO] Step 72000/102583; acc: 82.47; ppl: 1.78; xent: 0.58; lr: 0.00001; 2057/2629 tok/s; 7406 sec
  1557. [2019-02-04 19:54:01,020 INFO] Step 72050/102583; acc: 80.76; ppl: 1.82; xent: 0.60; lr: 0.00001; 1994/2759 tok/s; 7410 sec
  1558. [2019-02-04 19:54:06,176 INFO] Step 72100/102583; acc: 84.59; ppl: 1.68; xent: 0.52; lr: 0.00001; 2251/2464 tok/s; 7416 sec
  1559. [2019-02-04 19:54:11,249 INFO] Step 72150/102583; acc: 81.29; ppl: 1.80; xent: 0.59; lr: 0.00001; 1937/2770 tok/s; 7421 sec
  1560. [2019-02-04 19:54:15,750 INFO] Step 72200/102583; acc: 82.74; ppl: 1.78; xent: 0.58; lr: 0.00001; 2161/2506 tok/s; 7425 sec
  1561. [2019-02-04 19:54:20,847 INFO] Step 72250/102583; acc: 82.28; ppl: 1.73; xent: 0.55; lr: 0.00001; 1885/2667 tok/s; 7430 sec
  1562. [2019-02-04 19:54:26,032 INFO] Step 72300/102583; acc: 82.08; ppl: 1.79; xent: 0.58; lr: 0.00001; 2012/2496 tok/s; 7435 sec
  1563. [2019-02-04 19:54:31,106 INFO] Step 72350/102583; acc: 82.57; ppl: 1.76; xent: 0.57; lr: 0.00001; 2129/2635 tok/s; 7440 sec
  1564. [2019-02-04 19:54:36,413 INFO] Step 72400/102583; acc: 81.20; ppl: 1.81; xent: 0.59; lr: 0.00001; 1866/2702 tok/s; 7446 sec
  1565. [2019-02-04 19:54:41,215 INFO] Step 72450/102583; acc: 84.08; ppl: 1.71; xent: 0.54; lr: 0.00001; 2068/2625 tok/s; 7451 sec
  1566. [2019-02-04 19:54:46,188 INFO] Step 72500/102583; acc: 80.26; ppl: 1.86; xent: 0.62; lr: 0.00001; 2092/2539 tok/s; 7456 sec
  1567. [2019-02-04 19:54:50,866 INFO] Step 72550/102583; acc: 82.32; ppl: 1.79; xent: 0.58; lr: 0.00001; 2200/2578 tok/s; 7460 sec
  1568. [2019-02-04 19:54:55,779 INFO] Step 72600/102583; acc: 80.69; ppl: 1.85; xent: 0.61; lr: 0.00001; 1943/2738 tok/s; 7465 sec
  1569. [2019-02-04 19:55:01,095 INFO] Step 72650/102583; acc: 83.77; ppl: 1.70; xent: 0.53; lr: 0.00001; 1990/2569 tok/s; 7470 sec
  1570. [2019-02-04 19:55:06,237 INFO] Step 72700/102583; acc: 83.09; ppl: 1.73; xent: 0.55; lr: 0.00001; 2185/2405 tok/s; 7476 sec
  1571. [2019-02-04 19:55:10,986 INFO] Step 72750/102583; acc: 80.58; ppl: 1.82; xent: 0.60; lr: 0.00001; 1934/2848 tok/s; 7480 sec
  1572. [2019-02-04 19:55:15,796 INFO] Step 72800/102583; acc: 81.73; ppl: 1.81; xent: 0.59; lr: 0.00001; 1855/2729 tok/s; 7485 sec
  1573. [2019-02-04 19:55:20,871 INFO] Step 72850/102583; acc: 84.24; ppl: 1.68; xent: 0.52; lr: 0.00001; 2251/2478 tok/s; 7490 sec
  1574. [2019-02-04 19:55:25,704 INFO] Step 72900/102583; acc: 80.57; ppl: 1.88; xent: 0.63; lr: 0.00001; 1832/2683 tok/s; 7495 sec
  1575. [2019-02-04 19:55:30,777 INFO] Step 72950/102583; acc: 83.23; ppl: 1.70; xent: 0.53; lr: 0.00001; 2173/2550 tok/s; 7500 sec
  1576. [2019-02-04 19:55:35,802 INFO] Step 73000/102583; acc: 81.47; ppl: 1.80; xent: 0.59; lr: 0.00001; 2069/2711 tok/s; 7505 sec
  1577. [2019-02-04 19:55:40,642 INFO] Step 73050/102583; acc: 82.36; ppl: 1.78; xent: 0.58; lr: 0.00001; 2045/2585 tok/s; 7510 sec
  1578. [2019-02-04 19:55:45,464 INFO] Step 73100/102583; acc: 81.81; ppl: 1.80; xent: 0.59; lr: 0.00001; 1915/2720 tok/s; 7515 sec
  1579. [2019-02-04 19:55:50,561 INFO] Step 73150/102583; acc: 83.59; ppl: 1.71; xent: 0.54; lr: 0.00001; 2239/2577 tok/s; 7520 sec
  1580. [2019-02-04 19:55:55,594 INFO] Step 73200/102583; acc: 80.73; ppl: 1.87; xent: 0.63; lr: 0.00001; 1825/2579 tok/s; 7525 sec
  1581. [2019-02-04 19:56:00,451 INFO] Step 73250/102583; acc: 82.53; ppl: 1.74; xent: 0.56; lr: 0.00001; 2216/2591 tok/s; 7530 sec
  1582. [2019-02-04 19:56:05,436 INFO] Step 73300/102583; acc: 81.75; ppl: 1.82; xent: 0.60; lr: 0.00001; 1969/2626 tok/s; 7535 sec
  1583. [2019-02-04 19:56:10,598 INFO] Step 73350/102583; acc: 83.69; ppl: 1.72; xent: 0.54; lr: 0.00001; 2070/2437 tok/s; 7540 sec
  1584. [2019-02-04 19:56:15,373 INFO] Step 73400/102583; acc: 81.40; ppl: 1.84; xent: 0.61; lr: 0.00001; 2030/2743 tok/s; 7545 sec
  1585. [2019-02-04 19:56:20,449 INFO] Step 73450/102583; acc: 82.39; ppl: 1.75; xent: 0.56; lr: 0.00001; 1964/2664 tok/s; 7550 sec
  1586. [2019-02-04 19:56:25,165 INFO] Step 73500/102583; acc: 81.14; ppl: 1.88; xent: 0.63; lr: 0.00001; 2045/2609 tok/s; 7555 sec
  1587. [2019-02-04 19:56:30,370 INFO] Step 73550/102583; acc: 84.55; ppl: 1.64; xent: 0.50; lr: 0.00001; 2054/2547 tok/s; 7560 sec
  1588. [2019-02-04 19:56:35,010 INFO] Step 73600/102583; acc: 81.11; ppl: 1.87; xent: 0.63; lr: 0.00001; 2168/2582 tok/s; 7564 sec
  1589. [2019-02-04 19:56:40,151 INFO] Step 73650/102583; acc: 83.37; ppl: 1.70; xent: 0.53; lr: 0.00001; 2082/2627 tok/s; 7570 sec
  1590. [2019-02-04 19:56:45,323 INFO] Step 73700/102583; acc: 82.14; ppl: 1.75; xent: 0.56; lr: 0.00001; 1885/2762 tok/s; 7575 sec
  1591. [2019-02-04 19:56:50,176 INFO] Step 73750/102583; acc: 81.82; ppl: 1.81; xent: 0.59; lr: 0.00001; 2135/2577 tok/s; 7580 sec
  1592. [2019-02-04 19:56:54,906 INFO] Step 73800/102583; acc: 83.86; ppl: 1.70; xent: 0.53; lr: 0.00001; 2152/2565 tok/s; 7584 sec
  1593. [2019-02-04 19:56:59,986 INFO] Step 73850/102583; acc: 81.52; ppl: 1.78; xent: 0.58; lr: 0.00001; 2021/2681 tok/s; 7589 sec
  1594. [2019-02-04 19:57:04,731 INFO] Step 73900/102583; acc: 80.48; ppl: 1.85; xent: 0.62; lr: 0.00001; 2013/2653 tok/s; 7594 sec
  1595. [2019-02-04 19:57:09,746 INFO] Step 73950/102583; acc: 83.47; ppl: 1.73; xent: 0.55; lr: 0.00001; 1994/2630 tok/s; 7599 sec
  1596. [2019-02-04 19:57:14,600 INFO] Step 74000/102583; acc: 82.70; ppl: 1.75; xent: 0.56; lr: 0.00001; 2147/2655 tok/s; 7604 sec
  1597. [2019-02-04 19:57:19,814 INFO] Step 74050/102583; acc: 82.38; ppl: 1.74; xent: 0.55; lr: 0.00001; 2016/2599 tok/s; 7609 sec
  1598. [2019-02-04 19:57:25,070 INFO] Step 74100/102583; acc: 80.81; ppl: 1.86; xent: 0.62; lr: 0.00001; 1865/2705 tok/s; 7614 sec
  1599. [2019-02-04 19:57:29,802 INFO] Step 74150/102583; acc: 82.15; ppl: 1.80; xent: 0.59; lr: 0.00001; 2084/2536 tok/s; 7619 sec
  1600. [2019-02-04 19:57:34,567 INFO] Step 74200/102583; acc: 83.46; ppl: 1.71; xent: 0.54; lr: 0.00001; 2019/2793 tok/s; 7624 sec
  1601. [2019-02-04 19:57:39,880 INFO] Step 74250/102583; acc: 83.33; ppl: 1.72; xent: 0.54; lr: 0.00001; 2027/2520 tok/s; 7629 sec
  1602. [2019-02-04 19:57:44,628 INFO] Step 74300/102583; acc: 81.70; ppl: 1.80; xent: 0.59; lr: 0.00001; 2111/2732 tok/s; 7634 sec
  1603. [2019-02-04 19:57:49,730 INFO] Step 74350/102583; acc: 81.98; ppl: 1.80; xent: 0.59; lr: 0.00001; 2191/2529 tok/s; 7639 sec
  1604. [2019-02-04 19:57:55,147 INFO] Step 74400/102583; acc: 83.80; ppl: 1.70; xent: 0.53; lr: 0.00001; 2025/2520 tok/s; 7645 sec
  1605. [2019-02-04 19:57:59,718 INFO] Step 74450/102583; acc: 80.78; ppl: 1.87; xent: 0.63; lr: 0.00001; 2061/2627 tok/s; 7649 sec
  1606. [2019-02-04 19:58:04,459 INFO] Step 74500/102583; acc: 82.13; ppl: 1.77; xent: 0.57; lr: 0.00001; 2081/2677 tok/s; 7654 sec
  1607. [2019-02-04 19:58:09,780 INFO] Step 74550/102583; acc: 82.70; ppl: 1.75; xent: 0.56; lr: 0.00001; 1982/2561 tok/s; 7659 sec
  1608. [2019-02-04 19:58:14,880 INFO] Step 74600/102583; acc: 81.72; ppl: 1.79; xent: 0.58; lr: 0.00001; 1890/2616 tok/s; 7664 sec
  1609. [2019-02-04 19:58:21,484 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  1610. [2019-02-04 19:58:26,183 INFO] Step 74650/102583; acc: 82.46; ppl: 1.76; xent: 0.56; lr: 0.00001; 972/1171 tok/s; 7676 sec
  1611. [2019-02-04 19:58:30,643 INFO] Step 74700/102583; acc: 81.15; ppl: 1.89; xent: 0.63; lr: 0.00001; 2110/2569 tok/s; 7680 sec
  1612. [2019-02-04 19:58:35,720 INFO] Step 74750/102583; acc: 83.41; ppl: 1.70; xent: 0.53; lr: 0.00001; 2003/2836 tok/s; 7685 sec
  1613. [2019-02-04 19:58:40,841 INFO] Step 74800/102583; acc: 81.96; ppl: 1.79; xent: 0.58; lr: 0.00001; 1983/2583 tok/s; 7690 sec
  1614. [2019-02-04 19:58:45,651 INFO] Step 74850/102583; acc: 83.00; ppl: 1.74; xent: 0.56; lr: 0.00001; 2067/2806 tok/s; 7695 sec
  1615. [2019-02-04 19:58:50,690 INFO] Step 74900/102583; acc: 82.30; ppl: 1.76; xent: 0.57; lr: 0.00001; 2137/2500 tok/s; 7700 sec
  1616. [2019-02-04 19:58:55,603 INFO] Step 74950/102583; acc: 83.27; ppl: 1.72; xent: 0.54; lr: 0.00001; 2234/2608 tok/s; 7705 sec
  1617. [2019-02-04 19:59:00,478 INFO] Step 75000/102583; acc: 81.25; ppl: 1.84; xent: 0.61; lr: 0.00001; 2039/2641 tok/s; 7710 sec
  1618. [2019-02-04 19:59:00,479 INFO] Saving checkpoint models/ted1_step_75000.pt
  1619. [2019-02-04 19:59:11,515 INFO] Step 75050/102583; acc: 80.48; ppl: 1.86; xent: 0.62; lr: 0.00001; 826/1239 tok/s; 7721 sec
  1620. [2019-02-04 19:59:16,443 INFO] Step 75100/102583; acc: 82.50; ppl: 1.77; xent: 0.57; lr: 0.00001; 2148/2651 tok/s; 7726 sec
  1621. [2019-02-04 19:59:21,547 INFO] Step 75150/102583; acc: 82.88; ppl: 1.71; xent: 0.54; lr: 0.00001; 1913/2741 tok/s; 7731 sec
  1622. [2019-02-04 19:59:26,368 INFO] Step 75200/102583; acc: 82.24; ppl: 1.80; xent: 0.59; lr: 0.00001; 2309/2463 tok/s; 7736 sec
  1623. [2019-02-04 19:59:31,197 INFO] Step 75250/102583; acc: 81.49; ppl: 1.83; xent: 0.60; lr: 0.00001; 2022/2619 tok/s; 7741 sec
  1624. [2019-02-04 19:59:36,121 INFO] Step 75300/102583; acc: 81.52; ppl: 1.81; xent: 0.59; lr: 0.00001; 1954/2673 tok/s; 7746 sec
  1625. [2019-02-04 19:59:41,253 INFO] Step 75350/102583; acc: 84.42; ppl: 1.66; xent: 0.50; lr: 0.00001; 2253/2580 tok/s; 7751 sec
  1626. [2019-02-04 19:59:46,239 INFO] Step 75400/102583; acc: 80.82; ppl: 1.84; xent: 0.61; lr: 0.00001; 1932/2726 tok/s; 7756 sec
  1627. [2019-02-04 19:59:50,903 INFO] Step 75450/102583; acc: 82.61; ppl: 1.77; xent: 0.57; lr: 0.00001; 2086/2452 tok/s; 7760 sec
  1628. [2019-02-04 19:59:55,983 INFO] Step 75500/102583; acc: 83.30; ppl: 1.69; xent: 0.53; lr: 0.00001; 1980/2781 tok/s; 7765 sec
  1629. [2019-02-04 20:00:01,028 INFO] Step 75550/102583; acc: 81.81; ppl: 1.82; xent: 0.60; lr: 0.00001; 2043/2432 tok/s; 7770 sec
  1630. [2019-02-04 20:00:06,101 INFO] Step 75600/102583; acc: 82.22; ppl: 1.77; xent: 0.57; lr: 0.00001; 2070/2627 tok/s; 7775 sec
  1631. [2019-02-04 20:00:11,492 INFO] Step 75650/102583; acc: 81.85; ppl: 1.76; xent: 0.57; lr: 0.00001; 1896/2732 tok/s; 7781 sec
  1632. [2019-02-04 20:00:16,128 INFO] Step 75700/102583; acc: 82.95; ppl: 1.76; xent: 0.57; lr: 0.00001; 2129/2528 tok/s; 7786 sec
  1633. [2019-02-04 20:00:21,182 INFO] Step 75750/102583; acc: 81.03; ppl: 1.84; xent: 0.61; lr: 0.00001; 2110/2569 tok/s; 7791 sec
  1634. [2019-02-04 20:00:26,127 INFO] Step 75800/102583; acc: 81.54; ppl: 1.80; xent: 0.59; lr: 0.00001; 1980/2705 tok/s; 7796 sec
  1635. [2019-02-04 20:00:30,827 INFO] Step 75850/102583; acc: 81.07; ppl: 1.85; xent: 0.61; lr: 0.00001; 2091/2579 tok/s; 7800 sec
  1636. [2019-02-04 20:00:36,107 INFO] Step 75900/102583; acc: 83.57; ppl: 1.70; xent: 0.53; lr: 0.00001; 1968/2620 tok/s; 7805 sec
  1637. [2019-02-04 20:00:41,247 INFO] Step 75950/102583; acc: 83.56; ppl: 1.72; xent: 0.54; lr: 0.00001; 2171/2397 tok/s; 7811 sec
  1638. [2019-02-04 20:00:46,008 INFO] Step 76000/102583; acc: 80.77; ppl: 1.82; xent: 0.60; lr: 0.00001; 1916/2871 tok/s; 7815 sec
  1639. [2019-02-04 20:00:51,081 INFO] Step 76050/102583; acc: 82.17; ppl: 1.77; xent: 0.57; lr: 0.00001; 2006/2622 tok/s; 7820 sec
  1640. [2019-02-04 20:00:55,963 INFO] Step 76100/102583; acc: 83.15; ppl: 1.75; xent: 0.56; lr: 0.00001; 2154/2515 tok/s; 7825 sec
  1641. [2019-02-04 20:01:00,641 INFO] Step 76150/102583; acc: 80.85; ppl: 1.88; xent: 0.63; lr: 0.00001; 1832/2678 tok/s; 7830 sec
  1642. [2019-02-04 20:01:05,891 INFO] Step 76200/102583; acc: 83.24; ppl: 1.68; xent: 0.52; lr: 0.00001; 2184/2621 tok/s; 7835 sec
  1643. [2019-02-04 20:01:10,833 INFO] Step 76250/102583; acc: 81.20; ppl: 1.82; xent: 0.60; lr: 0.00001; 2014/2697 tok/s; 7840 sec
  1644. [2019-02-04 20:01:15,767 INFO] Step 76300/102583; acc: 82.63; ppl: 1.77; xent: 0.57; lr: 0.00001; 2066/2513 tok/s; 7845 sec
  1645. [2019-02-04 20:01:20,821 INFO] Step 76350/102583; acc: 82.41; ppl: 1.76; xent: 0.57; lr: 0.00001; 1891/2720 tok/s; 7850 sec
  1646. [2019-02-04 20:01:25,684 INFO] Step 76400/102583; acc: 82.12; ppl: 1.78; xent: 0.58; lr: 0.00001; 2216/2593 tok/s; 7855 sec
  1647. [2019-02-04 20:01:30,715 INFO] Step 76450/102583; acc: 80.92; ppl: 1.85; xent: 0.62; lr: 0.00001; 1841/2659 tok/s; 7860 sec
  1648. [2019-02-04 20:01:35,582 INFO] Step 76500/102583; acc: 83.08; ppl: 1.72; xent: 0.54; lr: 0.00001; 2248/2544 tok/s; 7865 sec
  1649. [2019-02-04 20:01:40,542 INFO] Step 76550/102583; acc: 81.80; ppl: 1.81; xent: 0.59; lr: 0.00001; 1996/2619 tok/s; 7870 sec
  1650. [2019-02-04 20:01:45,706 INFO] Step 76600/102583; acc: 83.20; ppl: 1.75; xent: 0.56; lr: 0.00001; 2006/2508 tok/s; 7875 sec
  1651. [2019-02-04 20:01:50,500 INFO] Step 76650/102583; acc: 81.18; ppl: 1.84; xent: 0.61; lr: 0.00001; 2012/2690 tok/s; 7880 sec
  1652. [2019-02-04 20:01:55,481 INFO] Step 76700/102583; acc: 82.74; ppl: 1.74; xent: 0.55; lr: 0.00001; 2049/2647 tok/s; 7885 sec
  1653. [2019-02-04 20:02:00,332 INFO] Step 76750/102583; acc: 81.57; ppl: 1.84; xent: 0.61; lr: 0.00001; 2075/2612 tok/s; 7890 sec
  1654. [2019-02-04 20:02:05,419 INFO] Step 76800/102583; acc: 83.78; ppl: 1.68; xent: 0.52; lr: 0.00001; 2095/2546 tok/s; 7895 sec
  1655. [2019-02-04 20:02:10,061 INFO] Step 76850/102583; acc: 81.78; ppl: 1.82; xent: 0.60; lr: 0.00001; 2112/2593 tok/s; 7899 sec
  1656. [2019-02-04 20:02:15,172 INFO] Step 76900/102583; acc: 82.62; ppl: 1.73; xent: 0.55; lr: 0.00001; 2069/2603 tok/s; 7905 sec
  1657. [2019-02-04 20:02:20,506 INFO] Step 76950/102583; acc: 82.38; ppl: 1.73; xent: 0.55; lr: 0.00001; 1909/2752 tok/s; 7910 sec
  1658. [2019-02-04 20:02:25,342 INFO] Step 77000/102583; acc: 81.29; ppl: 1.83; xent: 0.60; lr: 0.00001; 1990/2661 tok/s; 7915 sec
  1659. [2019-02-04 20:02:29,995 INFO] Step 77050/102583; acc: 83.42; ppl: 1.73; xent: 0.55; lr: 0.00001; 2217/2493 tok/s; 7919 sec
  1660. [2019-02-04 20:02:35,022 INFO] Step 77100/102583; acc: 81.58; ppl: 1.79; xent: 0.58; lr: 0.00001; 2078/2715 tok/s; 7924 sec
  1661. [2019-02-04 20:02:39,830 INFO] Step 77150/102583; acc: 81.48; ppl: 1.81; xent: 0.59; lr: 0.00001; 1973/2663 tok/s; 7929 sec
  1662. [2019-02-04 20:02:44,851 INFO] Step 77200/102583; acc: 82.48; ppl: 1.76; xent: 0.57; lr: 0.00001; 1979/2653 tok/s; 7934 sec
  1663. [2019-02-04 20:02:49,577 INFO] Step 77250/102583; acc: 83.28; ppl: 1.72; xent: 0.54; lr: 0.00001; 2222/2602 tok/s; 7939 sec
  1664. [2019-02-04 20:02:54,797 INFO] Step 77300/102583; acc: 81.58; ppl: 1.79; xent: 0.58; lr: 0.00001; 1965/2594 tok/s; 7944 sec
  1665. [2019-02-04 20:03:00,034 INFO] Step 77350/102583; acc: 81.07; ppl: 1.85; xent: 0.61; lr: 0.00001; 1896/2684 tok/s; 7949 sec
  1666. [2019-02-04 20:03:04,779 INFO] Step 77400/102583; acc: 82.74; ppl: 1.77; xent: 0.57; lr: 0.00001; 2146/2539 tok/s; 7954 sec
  1667. [2019-02-04 20:03:09,670 INFO] Step 77450/102583; acc: 83.64; ppl: 1.68; xent: 0.52; lr: 0.00001; 1959/2795 tok/s; 7959 sec
  1668. [2019-02-04 20:03:14,888 INFO] Step 77500/102583; acc: 83.19; ppl: 1.73; xent: 0.55; lr: 0.00001; 2109/2533 tok/s; 7964 sec
  1669. [2019-02-04 20:03:19,784 INFO] Step 77550/102583; acc: 82.08; ppl: 1.76; xent: 0.57; lr: 0.00001; 2071/2741 tok/s; 7969 sec
  1670. [2019-02-04 20:03:24,745 INFO] Step 77600/102583; acc: 81.61; ppl: 1.84; xent: 0.61; lr: 0.00001; 2175/2379 tok/s; 7974 sec
  1671. [2019-02-04 20:03:30,029 INFO] Step 77650/102583; acc: 82.98; ppl: 1.73; xent: 0.55; lr: 0.00001; 2006/2645 tok/s; 7979 sec
  1672. [2019-02-04 20:03:34,632 INFO] Step 77700/102583; acc: 80.69; ppl: 1.87; xent: 0.62; lr: 0.00001; 1981/2696 tok/s; 7984 sec
  1673. [2019-02-04 20:03:39,396 INFO] Step 77750/102583; acc: 82.37; ppl: 1.77; xent: 0.57; lr: 0.00001; 2137/2509 tok/s; 7989 sec
  1674. [2019-02-04 20:03:44,716 INFO] Step 77800/102583; acc: 81.81; ppl: 1.78; xent: 0.58; lr: 0.00001; 1863/2718 tok/s; 7994 sec
  1675. [2019-02-04 20:03:49,870 INFO] Step 77850/102583; acc: 82.65; ppl: 1.76; xent: 0.57; lr: 0.00001; 1931/2489 tok/s; 7999 sec
  1676. [2019-02-04 20:03:55,513 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  1677. [2019-02-04 20:04:00,697 INFO] Step 77900/102583; acc: 82.93; ppl: 1.75; xent: 0.56; lr: 0.00001; 1004/1181 tok/s; 8010 sec
  1678. [2019-02-04 20:04:05,413 INFO] Step 77950/102583; acc: 82.15; ppl: 1.80; xent: 0.59; lr: 0.00001; 2097/2650 tok/s; 8015 sec
  1679. [2019-02-04 20:04:10,475 INFO] Step 78000/102583; acc: 83.40; ppl: 1.71; xent: 0.54; lr: 0.00001; 2032/2737 tok/s; 8020 sec
  1680. [2019-02-04 20:04:15,368 INFO] Step 78050/102583; acc: 81.58; ppl: 1.83; xent: 0.61; lr: 0.00001; 2034/2607 tok/s; 8025 sec
  1681. [2019-02-04 20:04:20,242 INFO] Step 78100/102583; acc: 82.58; ppl: 1.75; xent: 0.56; lr: 0.00001; 1967/2779 tok/s; 8030 sec
  1682. [2019-02-04 20:04:25,196 INFO] Step 78150/102583; acc: 82.60; ppl: 1.75; xent: 0.56; lr: 0.00001; 2203/2528 tok/s; 8035 sec
  1683. [2019-02-04 20:04:30,128 INFO] Step 78200/102583; acc: 82.86; ppl: 1.74; xent: 0.56; lr: 0.00001; 2173/2634 tok/s; 8040 sec
  1684. [2019-02-04 20:04:34,981 INFO] Step 78250/102583; acc: 81.56; ppl: 1.83; xent: 0.60; lr: 0.00001; 2048/2633 tok/s; 8044 sec
  1685. [2019-02-04 20:04:40,266 INFO] Step 78300/102583; acc: 80.88; ppl: 1.83; xent: 0.60; lr: 0.00001; 1840/2698 tok/s; 8050 sec
  1686. [2019-02-04 20:04:45,008 INFO] Step 78350/102583; acc: 82.63; ppl: 1.76; xent: 0.57; lr: 0.00001; 2146/2626 tok/s; 8054 sec
  1687. [2019-02-04 20:04:50,209 INFO] Step 78400/102583; acc: 82.64; ppl: 1.73; xent: 0.55; lr: 0.00001; 1932/2725 tok/s; 8060 sec
  1688. [2019-02-04 20:04:55,056 INFO] Step 78450/102583; acc: 82.03; ppl: 1.82; xent: 0.60; lr: 0.00001; 2237/2561 tok/s; 8064 sec
  1689. [2019-02-04 20:05:00,036 INFO] Step 78500/102583; acc: 81.46; ppl: 1.80; xent: 0.59; lr: 0.00001; 2040/2588 tok/s; 8069 sec
  1690. [2019-02-04 20:05:04,722 INFO] Step 78550/102583; acc: 81.60; ppl: 1.84; xent: 0.61; lr: 0.00001; 1957/2658 tok/s; 8074 sec
  1691. [2019-02-04 20:05:09,877 INFO] Step 78600/102583; acc: 83.70; ppl: 1.68; xent: 0.52; lr: 0.00001; 2305/2572 tok/s; 8079 sec
  1692. [2019-02-04 20:05:14,850 INFO] Step 78650/102583; acc: 82.31; ppl: 1.78; xent: 0.57; lr: 0.00001; 1924/2737 tok/s; 8084 sec
  1693. [2019-02-04 20:05:19,858 INFO] Step 78700/102583; acc: 82.57; ppl: 1.77; xent: 0.57; lr: 0.00001; 2046/2432 tok/s; 8089 sec
  1694. [2019-02-04 20:05:24,717 INFO] Step 78750/102583; acc: 82.47; ppl: 1.75; xent: 0.56; lr: 0.00001; 1989/2714 tok/s; 8094 sec
  1695. [2019-02-04 20:05:29,841 INFO] Step 78800/102583; acc: 82.29; ppl: 1.76; xent: 0.57; lr: 0.00001; 2100/2469 tok/s; 8099 sec
  1696. [2019-02-04 20:05:34,813 INFO] Step 78850/102583; acc: 82.04; ppl: 1.79; xent: 0.58; lr: 0.00001; 1989/2692 tok/s; 8104 sec
  1697. [2019-02-04 20:05:40,101 INFO] Step 78900/102583; acc: 82.00; ppl: 1.77; xent: 0.57; lr: 0.00001; 1927/2728 tok/s; 8109 sec
  1698. [2019-02-04 20:05:44,799 INFO] Step 78950/102583; acc: 82.53; ppl: 1.78; xent: 0.58; lr: 0.00001; 2101/2543 tok/s; 8114 sec
  1699. [2019-02-04 20:05:49,777 INFO] Step 79000/102583; acc: 81.44; ppl: 1.82; xent: 0.60; lr: 0.00001; 2129/2563 tok/s; 8119 sec
  1700. [2019-02-04 20:05:54,740 INFO] Step 79050/102583; acc: 81.17; ppl: 1.82; xent: 0.60; lr: 0.00001; 1974/2695 tok/s; 8124 sec
  1701. [2019-02-04 20:05:59,402 INFO] Step 79100/102583; acc: 81.35; ppl: 1.84; xent: 0.61; lr: 0.00001; 2136/2570 tok/s; 8129 sec
  1702. [2019-02-04 20:06:04,739 INFO] Step 79150/102583; acc: 83.42; ppl: 1.71; xent: 0.54; lr: 0.00001; 1923/2642 tok/s; 8134 sec
  1703. [2019-02-04 20:06:09,962 INFO] Step 79200/102583; acc: 83.86; ppl: 1.69; xent: 0.52; lr: 0.00001; 2192/2391 tok/s; 8139 sec
  1704. [2019-02-04 20:06:14,686 INFO] Step 79250/102583; acc: 80.10; ppl: 1.87; xent: 0.62; lr: 0.00001; 1894/2822 tok/s; 8144 sec
  1705. [2019-02-04 20:06:19,866 INFO] Step 79300/102583; acc: 82.76; ppl: 1.72; xent: 0.55; lr: 0.00001; 2053/2614 tok/s; 8149 sec
  1706. [2019-02-04 20:06:24,599 INFO] Step 79350/102583; acc: 82.33; ppl: 1.78; xent: 0.58; lr: 0.00001; 2090/2614 tok/s; 8154 sec
  1707. [2019-02-04 20:06:29,210 INFO] Step 79400/102583; acc: 80.95; ppl: 1.88; xent: 0.63; lr: 0.00001; 1901/2619 tok/s; 8159 sec
  1708. [2019-02-04 20:06:34,529 INFO] Step 79450/102583; acc: 83.03; ppl: 1.68; xent: 0.52; lr: 0.00001; 2130/2589 tok/s; 8164 sec
  1709. [2019-02-04 20:06:39,410 INFO] Step 79500/102583; acc: 80.84; ppl: 1.84; xent: 0.61; lr: 0.00001; 2068/2740 tok/s; 8169 sec
  1710. [2019-02-04 20:06:44,454 INFO] Step 79550/102583; acc: 83.08; ppl: 1.74; xent: 0.56; lr: 0.00001; 1970/2540 tok/s; 8174 sec
  1711. [2019-02-04 20:06:49,511 INFO] Step 79600/102583; acc: 82.83; ppl: 1.76; xent: 0.57; lr: 0.00001; 1953/2740 tok/s; 8179 sec
  1712. [2019-02-04 20:06:54,364 INFO] Step 79650/102583; acc: 82.00; ppl: 1.78; xent: 0.58; lr: 0.00001; 2185/2532 tok/s; 8184 sec
  1713. [2019-02-04 20:06:59,353 INFO] Step 79700/102583; acc: 81.33; ppl: 1.82; xent: 0.60; lr: 0.00001; 1904/2685 tok/s; 8189 sec
  1714. [2019-02-04 20:07:04,239 INFO] Step 79750/102583; acc: 82.54; ppl: 1.76; xent: 0.56; lr: 0.00001; 2204/2578 tok/s; 8194 sec
  1715. [2019-02-04 20:07:09,188 INFO] Step 79800/102583; acc: 82.64; ppl: 1.76; xent: 0.57; lr: 0.00001; 2053/2586 tok/s; 8199 sec
  1716. [2019-02-04 20:07:14,352 INFO] Step 79850/102583; acc: 82.55; ppl: 1.78; xent: 0.57; lr: 0.00001; 1932/2525 tok/s; 8204 sec
  1717. [2019-02-04 20:07:19,115 INFO] Step 79900/102583; acc: 81.67; ppl: 1.82; xent: 0.60; lr: 0.00001; 2045/2676 tok/s; 8209 sec
  1718. [2019-02-04 20:07:24,231 INFO] Step 79950/102583; acc: 83.14; ppl: 1.74; xent: 0.55; lr: 0.00001; 2102/2550 tok/s; 8214 sec
  1719. [2019-02-04 20:07:29,017 INFO] Step 80000/102583; acc: 81.47; ppl: 1.82; xent: 0.60; lr: 0.00001; 1994/2682 tok/s; 8218 sec
  1720. [2019-02-04 20:07:29,043 INFO] Loading dataset from data/ted.valid.0.pt, number of examples: 873
  1721. [2019-02-04 20:07:30,344 INFO] Validation perplexity: 1.74359
  1722. [2019-02-04 20:07:30,345 INFO] Validation accuracy: 83.2174
  1723. [2019-02-04 20:07:30,346 INFO] Saving checkpoint models/ted1_step_80000.pt
  1724. [2019-02-04 20:07:41,050 INFO] Step 80050/102583; acc: 83.58; ppl: 1.69; xent: 0.53; lr: 0.00001; 888/1086 tok/s; 8230 sec
  1725. [2019-02-04 20:07:45,747 INFO] Step 80100/102583; acc: 81.81; ppl: 1.80; xent: 0.59; lr: 0.00001; 2160/2587 tok/s; 8235 sec
  1726. [2019-02-04 20:07:50,919 INFO] Step 80150/102583; acc: 82.66; ppl: 1.74; xent: 0.55; lr: 0.00001; 1983/2634 tok/s; 8240 sec
  1727. [2019-02-04 20:07:56,085 INFO] Step 80200/102583; acc: 82.12; ppl: 1.76; xent: 0.57; lr: 0.00001; 1940/2705 tok/s; 8245 sec
  1728. [2019-02-04 20:08:00,888 INFO] Step 80250/102583; acc: 81.51; ppl: 1.82; xent: 0.60; lr: 0.00001; 2044/2627 tok/s; 8250 sec
  1729. [2019-02-04 20:08:05,850 INFO] Step 80300/102583; acc: 83.93; ppl: 1.69; xent: 0.52; lr: 0.00001; 2083/2561 tok/s; 8255 sec
  1730. [2019-02-04 20:08:10,733 INFO] Step 80350/102583; acc: 81.50; ppl: 1.81; xent: 0.59; lr: 0.00001; 2146/2680 tok/s; 8260 sec
  1731. [2019-02-04 20:08:15,394 INFO] Step 80400/102583; acc: 81.69; ppl: 1.80; xent: 0.59; lr: 0.00001; 2017/2653 tok/s; 8265 sec
  1732. [2019-02-04 20:08:20,503 INFO] Step 80450/102583; acc: 82.06; ppl: 1.78; xent: 0.58; lr: 0.00001; 1945/2682 tok/s; 8270 sec
  1733. [2019-02-04 20:08:25,300 INFO] Step 80500/102583; acc: 83.05; ppl: 1.72; xent: 0.54; lr: 0.00001; 2218/2559 tok/s; 8275 sec
  1734. [2019-02-04 20:08:30,446 INFO] Step 80550/102583; acc: 82.18; ppl: 1.79; xent: 0.58; lr: 0.00001; 1922/2631 tok/s; 8280 sec
  1735. [2019-02-04 20:08:35,595 INFO] Step 80600/102583; acc: 81.20; ppl: 1.82; xent: 0.60; lr: 0.00001; 1979/2654 tok/s; 8285 sec
  1736. [2019-02-04 20:08:40,569 INFO] Step 80650/102583; acc: 83.11; ppl: 1.73; xent: 0.55; lr: 0.00001; 2114/2580 tok/s; 8290 sec
  1737. [2019-02-04 20:08:45,401 INFO] Step 80700/102583; acc: 83.58; ppl: 1.70; xent: 0.53; lr: 0.00001; 1895/2783 tok/s; 8295 sec
  1738. [2019-02-04 20:08:50,605 INFO] Step 80750/102583; acc: 82.81; ppl: 1.74; xent: 0.55; lr: 0.00001; 2166/2487 tok/s; 8300 sec
  1739. [2019-02-04 20:08:55,411 INFO] Step 80800/102583; acc: 81.67; ppl: 1.79; xent: 0.58; lr: 0.00001; 2008/2792 tok/s; 8305 sec
  1740. [2019-02-04 20:09:00,324 INFO] Step 80850/102583; acc: 81.44; ppl: 1.84; xent: 0.61; lr: 0.00001; 2181/2374 tok/s; 8310 sec
  1741. [2019-02-04 20:09:05,627 INFO] Step 80900/102583; acc: 82.93; ppl: 1.72; xent: 0.54; lr: 0.00001; 2059/2662 tok/s; 8315 sec
  1742. [2019-02-04 20:09:10,254 INFO] Step 80950/102583; acc: 80.81; ppl: 1.86; xent: 0.62; lr: 0.00001; 1916/2713 tok/s; 8320 sec
  1743. [2019-02-04 20:09:15,075 INFO] Step 81000/102583; acc: 82.52; ppl: 1.76; xent: 0.56; lr: 0.00001; 2129/2552 tok/s; 8324 sec
  1744. [2019-02-04 20:09:20,287 INFO] Step 81050/102583; acc: 82.43; ppl: 1.76; xent: 0.57; lr: 0.00001; 1926/2662 tok/s; 8330 sec
  1745. [2019-02-04 20:09:25,627 INFO] Step 81100/102583; acc: 82.52; ppl: 1.76; xent: 0.57; lr: 0.00001; 1961/2403 tok/s; 8335 sec
  1746. [2019-02-04 20:09:31,154 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  1747. [2019-02-04 20:09:36,933 INFO] Step 81150/102583; acc: 82.34; ppl: 1.78; xent: 0.58; lr: 0.00001; 932/1163 tok/s; 8346 sec
  1748. [2019-02-04 20:09:41,674 INFO] Step 81200/102583; acc: 82.86; ppl: 1.76; xent: 0.56; lr: 0.00001; 2059/2693 tok/s; 8351 sec
  1749. [2019-02-04 20:09:46,777 INFO] Step 81250/102583; acc: 83.60; ppl: 1.70; xent: 0.53; lr: 0.00001; 2048/2683 tok/s; 8356 sec
  1750. [2019-02-04 20:09:51,553 INFO] Step 81300/102583; acc: 81.20; ppl: 1.86; xent: 0.62; lr: 0.00001; 2048/2610 tok/s; 8361 sec
  1751. [2019-02-04 20:09:56,471 INFO] Step 81350/102583; acc: 82.97; ppl: 1.73; xent: 0.55; lr: 0.00001; 2049/2729 tok/s; 8366 sec
  1752. [2019-02-04 20:10:01,461 INFO] Step 81400/102583; acc: 81.44; ppl: 1.80; xent: 0.59; lr: 0.00001; 2072/2575 tok/s; 8371 sec
  1753. [2019-02-04 20:10:06,337 INFO] Step 81450/102583; acc: 82.64; ppl: 1.76; xent: 0.56; lr: 0.00001; 2128/2682 tok/s; 8376 sec
  1754. [2019-02-04 20:10:11,270 INFO] Step 81500/102583; acc: 82.16; ppl: 1.78; xent: 0.58; lr: 0.00001; 2043/2668 tok/s; 8381 sec
  1755. [2019-02-04 20:10:16,391 INFO] Step 81550/102583; acc: 81.00; ppl: 1.84; xent: 0.61; lr: 0.00001; 1996/2575 tok/s; 8386 sec
  1756. [2019-02-04 20:10:21,157 INFO] Step 81600/102583; acc: 83.05; ppl: 1.74; xent: 0.55; lr: 0.00001; 2122/2663 tok/s; 8391 sec
  1757. [2019-02-04 20:10:26,273 INFO] Step 81650/102583; acc: 82.47; ppl: 1.75; xent: 0.56; lr: 0.00001; 1928/2686 tok/s; 8396 sec
  1758. [2019-02-04 20:10:31,129 INFO] Step 81700/102583; acc: 82.07; ppl: 1.80; xent: 0.59; lr: 0.00001; 2194/2577 tok/s; 8401 sec
  1759. [2019-02-04 20:10:36,140 INFO] Step 81750/102583; acc: 81.23; ppl: 1.82; xent: 0.60; lr: 0.00001; 2065/2578 tok/s; 8406 sec
  1760. [2019-02-04 20:10:41,005 INFO] Step 81800/102583; acc: 83.32; ppl: 1.74; xent: 0.55; lr: 0.00001; 2038/2660 tok/s; 8410 sec
  1761. [2019-02-04 20:10:46,091 INFO] Step 81850/102583; acc: 82.73; ppl: 1.73; xent: 0.55; lr: 0.00001; 2165/2654 tok/s; 8415 sec
  1762. [2019-02-04 20:10:50,928 INFO] Step 81900/102583; acc: 82.00; ppl: 1.79; xent: 0.58; lr: 0.00001; 1992/2647 tok/s; 8420 sec
  1763. [2019-02-04 20:10:56,054 INFO] Step 81950/102583; acc: 82.83; ppl: 1.73; xent: 0.55; lr: 0.00001; 1986/2503 tok/s; 8425 sec
  1764. [2019-02-04 20:11:01,092 INFO] Step 82000/102583; acc: 82.44; ppl: 1.76; xent: 0.56; lr: 0.00001; 2003/2642 tok/s; 8430 sec
  1765. [2019-02-04 20:11:06,040 INFO] Step 82050/102583; acc: 82.32; ppl: 1.79; xent: 0.58; lr: 0.00001; 2098/2449 tok/s; 8435 sec
  1766. [2019-02-04 20:11:10,997 INFO] Step 82100/102583; acc: 82.42; ppl: 1.76; xent: 0.57; lr: 0.00001; 1998/2777 tok/s; 8440 sec
  1767. [2019-02-04 20:11:16,253 INFO] Step 82150/102583; acc: 81.92; ppl: 1.77; xent: 0.57; lr: 0.00001; 1914/2613 tok/s; 8446 sec
  1768. [2019-02-04 20:11:21,005 INFO] Step 82200/102583; acc: 82.38; ppl: 1.78; xent: 0.58; lr: 0.00001; 2141/2597 tok/s; 8450 sec
  1769. [2019-02-04 20:11:26,152 INFO] Step 82250/102583; acc: 82.14; ppl: 1.77; xent: 0.57; lr: 0.00001; 2104/2532 tok/s; 8456 sec
  1770. [2019-02-04 20:11:30,938 INFO] Step 82300/102583; acc: 81.42; ppl: 1.81; xent: 0.59; lr: 0.00001; 1996/2687 tok/s; 8460 sec
  1771. [2019-02-04 20:11:35,564 INFO] Step 82350/102583; acc: 81.43; ppl: 1.84; xent: 0.61; lr: 0.00001; 2085/2681 tok/s; 8465 sec
  1772. [2019-02-04 20:11:40,863 INFO] Step 82400/102583; acc: 83.05; ppl: 1.72; xent: 0.54; lr: 0.00001; 2012/2556 tok/s; 8470 sec
  1773. [2019-02-04 20:11:46,188 INFO] Step 82450/102583; acc: 83.44; ppl: 1.71; xent: 0.53; lr: 0.00001; 2099/2513 tok/s; 8476 sec
  1774. [2019-02-04 20:11:50,910 INFO] Step 82500/102583; acc: 80.99; ppl: 1.83; xent: 0.61; lr: 0.00001; 1919/2707 tok/s; 8480 sec
  1775. [2019-02-04 20:11:56,036 INFO] Step 82550/102583; acc: 82.55; ppl: 1.75; xent: 0.56; lr: 0.00001; 2089/2670 tok/s; 8485 sec
  1776. [2019-02-04 20:12:00,790 INFO] Step 82600/102583; acc: 82.26; ppl: 1.79; xent: 0.58; lr: 0.00001; 2010/2545 tok/s; 8490 sec
  1777. [2019-02-04 20:12:05,578 INFO] Step 82650/102583; acc: 81.51; ppl: 1.84; xent: 0.61; lr: 0.00001; 1944/2668 tok/s; 8495 sec
  1778. [2019-02-04 20:12:10,719 INFO] Step 82700/102583; acc: 83.05; ppl: 1.70; xent: 0.53; lr: 0.00001; 2097/2540 tok/s; 8500 sec
  1779. [2019-02-04 20:12:15,608 INFO] Step 82750/102583; acc: 81.02; ppl: 1.82; xent: 0.60; lr: 0.00001; 2082/2805 tok/s; 8505 sec
  1780. [2019-02-04 20:12:20,578 INFO] Step 82800/102583; acc: 83.60; ppl: 1.72; xent: 0.54; lr: 0.00001; 2061/2510 tok/s; 8510 sec
  1781. [2019-02-04 20:12:25,622 INFO] Step 82850/102583; acc: 82.56; ppl: 1.78; xent: 0.58; lr: 0.00001; 1895/2739 tok/s; 8515 sec
  1782. [2019-02-04 20:12:30,459 INFO] Step 82900/102583; acc: 81.60; ppl: 1.81; xent: 0.59; lr: 0.00001; 2205/2516 tok/s; 8520 sec
  1783. [2019-02-04 20:12:35,537 INFO] Step 82950/102583; acc: 81.72; ppl: 1.79; xent: 0.58; lr: 0.00001; 1890/2681 tok/s; 8525 sec
  1784. [2019-02-04 20:12:40,418 INFO] Step 83000/102583; acc: 82.80; ppl: 1.75; xent: 0.56; lr: 0.00001; 2169/2535 tok/s; 8530 sec
  1785. [2019-02-04 20:12:45,548 INFO] Step 83050/102583; acc: 83.10; ppl: 1.75; xent: 0.56; lr: 0.00001; 2149/2637 tok/s; 8535 sec
  1786. [2019-02-04 20:12:50,475 INFO] Step 83100/102583; acc: 82.29; ppl: 1.79; xent: 0.58; lr: 0.00001; 1901/2561 tok/s; 8540 sec
  1787. [2019-02-04 20:12:55,329 INFO] Step 83150/102583; acc: 81.33; ppl: 1.84; xent: 0.61; lr: 0.00001; 1956/2652 tok/s; 8545 sec
  1788. [2019-02-04 20:13:00,365 INFO] Step 83200/102583; acc: 82.92; ppl: 1.73; xent: 0.55; lr: 0.00001; 2140/2563 tok/s; 8550 sec
  1789. [2019-02-04 20:13:05,138 INFO] Step 83250/102583; acc: 82.08; ppl: 1.80; xent: 0.59; lr: 0.00001; 2000/2699 tok/s; 8555 sec
  1790. [2019-02-04 20:13:10,160 INFO] Step 83300/102583; acc: 82.77; ppl: 1.74; xent: 0.56; lr: 0.00001; 2116/2584 tok/s; 8560 sec
  1791. [2019-02-04 20:13:14,892 INFO] Step 83350/102583; acc: 82.17; ppl: 1.77; xent: 0.57; lr: 0.00001; 2139/2605 tok/s; 8564 sec
  1792. [2019-02-04 20:13:20,140 INFO] Step 83400/102583; acc: 82.54; ppl: 1.74; xent: 0.55; lr: 0.00001; 1919/2624 tok/s; 8570 sec
  1793. [2019-02-04 20:13:25,162 INFO] Step 83450/102583; acc: 82.39; ppl: 1.77; xent: 0.57; lr: 0.00001; 2058/2668 tok/s; 8575 sec
  1794. [2019-02-04 20:13:29,986 INFO] Step 83500/102583; acc: 82.06; ppl: 1.78; xent: 0.58; lr: 0.00001; 2048/2635 tok/s; 8579 sec
  1795. [2019-02-04 20:13:35,033 INFO] Step 83550/102583; acc: 83.41; ppl: 1.71; xent: 0.53; lr: 0.00001; 2074/2596 tok/s; 8584 sec
  1796. [2019-02-04 20:13:39,872 INFO] Step 83600/102583; acc: 81.43; ppl: 1.81; xent: 0.59; lr: 0.00001; 2039/2685 tok/s; 8589 sec
  1797. [2019-02-04 20:13:44,497 INFO] Step 83650/102583; acc: 81.89; ppl: 1.81; xent: 0.59; lr: 0.00001; 2102/2597 tok/s; 8594 sec
  1798. [2019-02-04 20:13:49,844 INFO] Step 83700/102583; acc: 82.86; ppl: 1.73; xent: 0.55; lr: 0.00001; 1992/2658 tok/s; 8599 sec
  1799. [2019-02-04 20:13:54,742 INFO] Step 83750/102583; acc: 82.22; ppl: 1.76; xent: 0.56; lr: 0.00001; 2209/2523 tok/s; 8604 sec
  1800. [2019-02-04 20:13:59,761 INFO] Step 83800/102583; acc: 82.52; ppl: 1.77; xent: 0.57; lr: 0.00001; 1864/2727 tok/s; 8609 sec
  1801. [2019-02-04 20:14:04,867 INFO] Step 83850/102583; acc: 81.02; ppl: 1.85; xent: 0.61; lr: 0.00001; 1926/2561 tok/s; 8614 sec
  1802. [2019-02-04 20:14:09,720 INFO] Step 83900/102583; acc: 83.45; ppl: 1.71; xent: 0.53; lr: 0.00001; 2164/2687 tok/s; 8619 sec
  1803. [2019-02-04 20:14:14,501 INFO] Step 83950/102583; acc: 82.43; ppl: 1.75; xent: 0.56; lr: 0.00001; 1902/2735 tok/s; 8624 sec
  1804. [2019-02-04 20:14:19,735 INFO] Step 84000/102583; acc: 82.67; ppl: 1.76; xent: 0.56; lr: 0.00001; 2141/2503 tok/s; 8629 sec
  1805. [2019-02-04 20:14:24,574 INFO] Step 84050/102583; acc: 82.68; ppl: 1.75; xent: 0.56; lr: 0.00001; 2030/2705 tok/s; 8634 sec
  1806. [2019-02-04 20:14:29,667 INFO] Step 84100/102583; acc: 81.62; ppl: 1.80; xent: 0.59; lr: 0.00001; 2084/2501 tok/s; 8639 sec
  1807. [2019-02-04 20:14:34,775 INFO] Step 84150/102583; acc: 82.20; ppl: 1.76; xent: 0.57; lr: 0.00001; 2135/2609 tok/s; 8644 sec
  1808. [2019-02-04 20:14:39,553 INFO] Step 84200/102583; acc: 82.13; ppl: 1.78; xent: 0.58; lr: 0.00001; 1981/2680 tok/s; 8649 sec
  1809. [2019-02-04 20:14:44,202 INFO] Step 84250/102583; acc: 82.24; ppl: 1.79; xent: 0.58; lr: 0.00001; 2116/2572 tok/s; 8654 sec
  1810. [2019-02-04 20:14:49,572 INFO] Step 84300/102583; acc: 82.17; ppl: 1.76; xent: 0.57; lr: 0.00001; 1936/2639 tok/s; 8659 sec
  1811. [2019-02-04 20:14:59,656 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  1812. [2019-02-04 20:15:01,209 INFO] Step 84350/102583; acc: 81.80; ppl: 1.82; xent: 0.60; lr: 0.00001; 847/1081 tok/s; 8671 sec
  1813. [2019-02-04 20:15:06,292 INFO] Step 84400/102583; acc: 83.11; ppl: 1.73; xent: 0.55; lr: 0.00001; 2153/2562 tok/s; 8676 sec
  1814. [2019-02-04 20:15:11,022 INFO] Step 84450/102583; acc: 82.14; ppl: 1.78; xent: 0.58; lr: 0.00001; 1969/2704 tok/s; 8680 sec
  1815. [2019-02-04 20:15:16,266 INFO] Step 84500/102583; acc: 83.77; ppl: 1.69; xent: 0.52; lr: 0.00001; 2053/2663 tok/s; 8686 sec
  1816. [2019-02-04 20:15:21,064 INFO] Step 84550/102583; acc: 81.72; ppl: 1.82; xent: 0.60; lr: 0.00001; 2095/2627 tok/s; 8690 sec
  1817. [2019-02-04 20:15:26,061 INFO] Step 84600/102583; acc: 82.57; ppl: 1.75; xent: 0.56; lr: 0.00001; 2094/2664 tok/s; 8695 sec
  1818. [2019-02-04 20:15:31,067 INFO] Step 84650/102583; acc: 81.64; ppl: 1.78; xent: 0.58; lr: 0.00001; 2023/2617 tok/s; 8700 sec
  1819. [2019-02-04 20:15:35,807 INFO] Step 84700/102583; acc: 83.18; ppl: 1.74; xent: 0.55; lr: 0.00001; 2136/2775 tok/s; 8705 sec
  1820. [2019-02-04 20:15:40,788 INFO] Step 84750/102583; acc: 81.10; ppl: 1.84; xent: 0.61; lr: 0.00001; 1920/2692 tok/s; 8710 sec
  1821. [2019-02-04 20:15:45,914 INFO] Step 84800/102583; acc: 81.87; ppl: 1.81; xent: 0.59; lr: 0.00001; 2090/2483 tok/s; 8715 sec
  1822. [2019-02-04 20:15:50,586 INFO] Step 84850/102583; acc: 82.64; ppl: 1.75; xent: 0.56; lr: 0.00001; 2087/2688 tok/s; 8720 sec
  1823. [2019-02-04 20:15:55,799 INFO] Step 84900/102583; acc: 82.28; ppl: 1.76; xent: 0.57; lr: 0.00001; 1987/2662 tok/s; 8725 sec
  1824. [2019-02-04 20:16:00,465 INFO] Step 84950/102583; acc: 82.24; ppl: 1.79; xent: 0.58; lr: 0.00001; 2190/2546 tok/s; 8730 sec
  1825. [2019-02-04 20:16:05,489 INFO] Step 85000/102583; acc: 80.99; ppl: 1.81; xent: 0.59; lr: 0.00001; 2048/2612 tok/s; 8735 sec
  1826. [2019-02-04 20:16:05,490 INFO] Saving checkpoint models/ted1_step_85000.pt
  1827. [2019-02-04 20:16:16,245 INFO] Step 85050/102583; acc: 83.45; ppl: 1.73; xent: 0.55; lr: 0.00001; 940/1216 tok/s; 8746 sec
  1828. [2019-02-04 20:16:21,463 INFO] Step 85100/102583; acc: 82.33; ppl: 1.74; xent: 0.56; lr: 0.00001; 2151/2603 tok/s; 8751 sec
  1829. [2019-02-04 20:16:26,217 INFO] Step 85150/102583; acc: 81.55; ppl: 1.80; xent: 0.59; lr: 0.00001; 2007/2646 tok/s; 8756 sec
  1830. [2019-02-04 20:16:31,263 INFO] Step 85200/102583; acc: 82.40; ppl: 1.74; xent: 0.55; lr: 0.00001; 2006/2578 tok/s; 8761 sec
  1831. [2019-02-04 20:16:36,372 INFO] Step 85250/102583; acc: 83.27; ppl: 1.72; xent: 0.54; lr: 0.00001; 1987/2644 tok/s; 8766 sec
  1832. [2019-02-04 20:16:41,309 INFO] Step 85300/102583; acc: 81.10; ppl: 1.84; xent: 0.61; lr: 0.00001; 2064/2475 tok/s; 8771 sec
  1833. [2019-02-04 20:16:46,349 INFO] Step 85350/102583; acc: 81.93; ppl: 1.77; xent: 0.57; lr: 0.00001; 2000/2732 tok/s; 8776 sec
  1834. [2019-02-04 20:16:51,613 INFO] Step 85400/102583; acc: 82.88; ppl: 1.74; xent: 0.55; lr: 0.00001; 1928/2544 tok/s; 8781 sec
  1835. [2019-02-04 20:16:56,618 INFO] Step 85450/102583; acc: 83.66; ppl: 1.71; xent: 0.54; lr: 0.00001; 2183/2626 tok/s; 8786 sec
  1836. [2019-02-04 20:17:01,395 INFO] Step 85500/102583; acc: 80.68; ppl: 1.85; xent: 0.61; lr: 0.00001; 2092/2595 tok/s; 8791 sec
  1837. [2019-02-04 20:17:06,221 INFO] Step 85550/102583; acc: 80.48; ppl: 1.86; xent: 0.62; lr: 0.00001; 1946/2712 tok/s; 8796 sec
  1838. [2019-02-04 20:17:10,958 INFO] Step 85600/102583; acc: 82.88; ppl: 1.75; xent: 0.56; lr: 0.00001; 2111/2634 tok/s; 8800 sec
  1839. [2019-02-04 20:17:16,293 INFO] Step 85650/102583; acc: 83.32; ppl: 1.71; xent: 0.54; lr: 0.00001; 2063/2479 tok/s; 8806 sec
  1840. [2019-02-04 20:17:21,394 INFO] Step 85700/102583; acc: 82.42; ppl: 1.75; xent: 0.56; lr: 0.00001; 2068/2598 tok/s; 8811 sec
  1841. [2019-02-04 20:17:26,312 INFO] Step 85750/102583; acc: 81.21; ppl: 1.80; xent: 0.59; lr: 0.00001; 1827/2729 tok/s; 8816 sec
  1842. [2019-02-04 20:17:31,277 INFO] Step 85800/102583; acc: 82.86; ppl: 1.75; xent: 0.56; lr: 0.00001; 2172/2675 tok/s; 8821 sec
  1843. [2019-02-04 20:17:36,126 INFO] Step 85850/102583; acc: 82.17; ppl: 1.79; xent: 0.58; lr: 0.00001; 1976/2512 tok/s; 8826 sec
  1844. [2019-02-04 20:17:40,815 INFO] Step 85900/102583; acc: 81.72; ppl: 1.83; xent: 0.61; lr: 0.00001; 1974/2624 tok/s; 8830 sec
  1845. [2019-02-04 20:17:46,074 INFO] Step 85950/102583; acc: 82.54; ppl: 1.71; xent: 0.54; lr: 0.00001; 2111/2564 tok/s; 8835 sec
  1846. [2019-02-04 20:17:50,965 INFO] Step 86000/102583; acc: 80.29; ppl: 1.87; xent: 0.62; lr: 0.00001; 1990/2775 tok/s; 8840 sec
  1847. [2019-02-04 20:17:55,983 INFO] Step 86050/102583; acc: 83.66; ppl: 1.71; xent: 0.54; lr: 0.00001; 2098/2588 tok/s; 8845 sec
  1848. [2019-02-04 20:18:01,075 INFO] Step 86100/102583; acc: 83.27; ppl: 1.74; xent: 0.55; lr: 0.00001; 2009/2668 tok/s; 8850 sec
  1849. [2019-02-04 20:18:05,700 INFO] Step 86150/102583; acc: 80.84; ppl: 1.85; xent: 0.61; lr: 0.00001; 2036/2599 tok/s; 8855 sec
  1850. [2019-02-04 20:18:10,783 INFO] Step 86200/102583; acc: 82.24; ppl: 1.77; xent: 0.57; lr: 0.00001; 1989/2631 tok/s; 8860 sec
  1851. [2019-02-04 20:18:15,682 INFO] Step 86250/102583; acc: 82.85; ppl: 1.75; xent: 0.56; lr: 0.00001; 2135/2560 tok/s; 8865 sec
  1852. [2019-02-04 20:18:20,971 INFO] Step 86300/102583; acc: 83.36; ppl: 1.72; xent: 0.54; lr: 0.00001; 2154/2538 tok/s; 8870 sec
  1853. [2019-02-04 20:18:25,681 INFO] Step 86350/102583; acc: 81.84; ppl: 1.80; xent: 0.59; lr: 0.00001; 1856/2635 tok/s; 8875 sec
  1854. [2019-02-04 20:18:30,609 INFO] Step 86400/102583; acc: 81.03; ppl: 1.86; xent: 0.62; lr: 0.00001; 1955/2664 tok/s; 8880 sec
  1855. [2019-02-04 20:18:35,705 INFO] Step 86450/102583; acc: 82.56; ppl: 1.75; xent: 0.56; lr: 0.00001; 2088/2548 tok/s; 8885 sec
  1856. [2019-02-04 20:18:40,494 INFO] Step 86500/102583; acc: 83.12; ppl: 1.73; xent: 0.55; lr: 0.00001; 2092/2724 tok/s; 8890 sec
  1857. [2019-02-04 20:18:45,537 INFO] Step 86550/102583; acc: 82.22; ppl: 1.78; xent: 0.58; lr: 0.00001; 2107/2469 tok/s; 8895 sec
  1858. [2019-02-04 20:18:50,255 INFO] Step 86600/102583; acc: 82.08; ppl: 1.76; xent: 0.57; lr: 0.00001; 2118/2729 tok/s; 8900 sec
  1859. [2019-02-04 20:18:55,440 INFO] Step 86650/102583; acc: 82.63; ppl: 1.74; xent: 0.55; lr: 0.00001; 1949/2638 tok/s; 8905 sec
  1860. [2019-02-04 20:19:00,402 INFO] Step 86700/102583; acc: 82.10; ppl: 1.77; xent: 0.57; lr: 0.00001; 2045/2657 tok/s; 8910 sec
  1861. [2019-02-04 20:19:05,321 INFO] Step 86750/102583; acc: 82.57; ppl: 1.76; xent: 0.56; lr: 0.00001; 2089/2516 tok/s; 8915 sec
  1862. [2019-02-04 20:19:10,263 INFO] Step 86800/102583; acc: 82.98; ppl: 1.73; xent: 0.55; lr: 0.00001; 2026/2717 tok/s; 8920 sec
  1863. [2019-02-04 20:19:15,142 INFO] Step 86850/102583; acc: 80.92; ppl: 1.83; xent: 0.60; lr: 0.00001; 2048/2696 tok/s; 8925 sec
  1864. [2019-02-04 20:19:19,968 INFO] Step 86900/102583; acc: 82.45; ppl: 1.77; xent: 0.57; lr: 0.00001; 2044/2538 tok/s; 8929 sec
  1865. [2019-02-04 20:19:25,038 INFO] Step 86950/102583; acc: 82.84; ppl: 1.74; xent: 0.56; lr: 0.00001; 2047/2678 tok/s; 8934 sec
  1866. [2019-02-04 20:19:30,055 INFO] Step 87000/102583; acc: 82.62; ppl: 1.73; xent: 0.55; lr: 0.00001; 2184/2551 tok/s; 8939 sec
  1867. [2019-02-04 20:19:35,209 INFO] Step 87050/102583; acc: 81.59; ppl: 1.80; xent: 0.59; lr: 0.00001; 1767/2786 tok/s; 8945 sec
  1868. [2019-02-04 20:19:40,056 INFO] Step 87100/102583; acc: 81.15; ppl: 1.86; xent: 0.62; lr: 0.00001; 2080/2477 tok/s; 8949 sec
  1869. [2019-02-04 20:19:44,942 INFO] Step 87150/102583; acc: 83.73; ppl: 1.69; xent: 0.53; lr: 0.00001; 2110/2708 tok/s; 8954 sec
  1870. [2019-02-04 20:19:49,753 INFO] Step 87200/102583; acc: 82.15; ppl: 1.76; xent: 0.57; lr: 0.00001; 1930/2764 tok/s; 8959 sec
  1871. [2019-02-04 20:19:55,061 INFO] Step 87250/102583; acc: 81.97; ppl: 1.79; xent: 0.58; lr: 0.00001; 2129/2463 tok/s; 8964 sec
  1872. [2019-02-04 20:19:59,867 INFO] Step 87300/102583; acc: 83.35; ppl: 1.72; xent: 0.54; lr: 0.00001; 2095/2693 tok/s; 8969 sec
  1873. [2019-02-04 20:20:04,997 INFO] Step 87350/102583; acc: 82.28; ppl: 1.77; xent: 0.57; lr: 0.00001; 2124/2489 tok/s; 8974 sec
  1874. [2019-02-04 20:20:10,110 INFO] Step 87400/102583; acc: 81.91; ppl: 1.79; xent: 0.58; lr: 0.00001; 1996/2644 tok/s; 8979 sec
  1875. [2019-02-04 20:20:14,769 INFO] Step 87450/102583; acc: 82.57; ppl: 1.76; xent: 0.57; lr: 0.00001; 2023/2706 tok/s; 8984 sec
  1876. [2019-02-04 20:20:19,409 INFO] Step 87500/102583; acc: 81.82; ppl: 1.79; xent: 0.58; lr: 0.00001; 2134/2580 tok/s; 8989 sec
  1877. [2019-02-04 20:20:24,724 INFO] Step 87550/102583; acc: 81.95; ppl: 1.78; xent: 0.58; lr: 0.00001; 1957/2607 tok/s; 8994 sec
  1878. [2019-02-04 20:20:33,457 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  1879. [2019-02-04 20:20:35,233 INFO] Step 87600/102583; acc: 81.56; ppl: 1.84; xent: 0.61; lr: 0.00001; 937/1223 tok/s; 9005 sec
  1880. [2019-02-04 20:20:40,341 INFO] Step 87650/102583; acc: 83.39; ppl: 1.72; xent: 0.54; lr: 0.00001; 2155/2539 tok/s; 9010 sec
  1881. [2019-02-04 20:20:45,062 INFO] Step 87700/102583; acc: 82.46; ppl: 1.77; xent: 0.57; lr: 0.00001; 1921/2725 tok/s; 9014 sec
  1882. [2019-02-04 20:20:50,196 INFO] Step 87750/102583; acc: 83.82; ppl: 1.70; xent: 0.53; lr: 0.00001; 2080/2655 tok/s; 9020 sec
  1883. [2019-02-04 20:20:55,101 INFO] Step 87800/102583; acc: 82.03; ppl: 1.79; xent: 0.58; lr: 0.00001; 2154/2667 tok/s; 9024 sec
  1884. [2019-02-04 20:21:00,134 INFO] Step 87850/102583; acc: 82.87; ppl: 1.74; xent: 0.55; lr: 0.00001; 2092/2673 tok/s; 9030 sec
  1885. [2019-02-04 20:21:04,990 INFO] Step 87900/102583; acc: 81.60; ppl: 1.81; xent: 0.59; lr: 0.00001; 2007/2543 tok/s; 9034 sec
  1886. [2019-02-04 20:21:09,888 INFO] Step 87950/102583; acc: 82.81; ppl: 1.75; xent: 0.56; lr: 0.00001; 2088/2794 tok/s; 9039 sec
  1887. [2019-02-04 20:21:14,734 INFO] Step 88000/102583; acc: 81.30; ppl: 1.82; xent: 0.60; lr: 0.00001; 1992/2698 tok/s; 9044 sec
  1888. [2019-02-04 20:21:19,932 INFO] Step 88050/102583; acc: 81.28; ppl: 1.83; xent: 0.61; lr: 0.00001; 2000/2523 tok/s; 9049 sec
  1889. [2019-02-04 20:21:24,616 INFO] Step 88100/102583; acc: 83.80; ppl: 1.68; xent: 0.52; lr: 0.00001; 2219/2653 tok/s; 9054 sec
  1890. [2019-02-04 20:21:29,933 INFO] Step 88150/102583; acc: 82.24; ppl: 1.78; xent: 0.58; lr: 0.00001; 1968/2607 tok/s; 9059 sec
  1891. [2019-02-04 20:21:34,412 INFO] Step 88200/102583; acc: 81.55; ppl: 1.86; xent: 0.62; lr: 0.00001; 2114/2610 tok/s; 9064 sec
  1892. [2019-02-04 20:21:39,617 INFO] Step 88250/102583; acc: 81.13; ppl: 1.79; xent: 0.58; lr: 0.00001; 1988/2677 tok/s; 9069 sec
  1893. [2019-02-04 20:21:44,414 INFO] Step 88300/102583; acc: 83.80; ppl: 1.73; xent: 0.55; lr: 0.00001; 2134/2595 tok/s; 9074 sec
  1894. [2019-02-04 20:21:49,611 INFO] Step 88350/102583; acc: 82.89; ppl: 1.72; xent: 0.54; lr: 0.00001; 2124/2620 tok/s; 9079 sec
  1895. [2019-02-04 20:21:54,311 INFO] Step 88400/102583; acc: 82.27; ppl: 1.79; xent: 0.58; lr: 0.00001; 2058/2582 tok/s; 9084 sec
  1896. [2019-02-04 20:21:59,406 INFO] Step 88450/102583; acc: 82.30; ppl: 1.74; xent: 0.55; lr: 0.00001; 1974/2602 tok/s; 9089 sec
  1897. [2019-02-04 20:22:04,535 INFO] Step 88500/102583; acc: 82.92; ppl: 1.74; xent: 0.55; lr: 0.00001; 2017/2646 tok/s; 9094 sec
  1898. [2019-02-04 20:22:09,537 INFO] Step 88550/102583; acc: 82.33; ppl: 1.79; xent: 0.58; lr: 0.00001; 2077/2431 tok/s; 9099 sec
  1899. [2019-02-04 20:22:14,520 INFO] Step 88600/102583; acc: 80.52; ppl: 1.83; xent: 0.60; lr: 0.00001; 1945/2788 tok/s; 9104 sec
  1900. [2019-02-04 20:22:19,745 INFO] Step 88650/102583; acc: 83.58; ppl: 1.71; xent: 0.54; lr: 0.00001; 1967/2513 tok/s; 9109 sec
  1901. [2019-02-04 20:22:24,771 INFO] Step 88700/102583; acc: 82.76; ppl: 1.74; xent: 0.55; lr: 0.00001; 2174/2645 tok/s; 9114 sec
  1902. [2019-02-04 20:22:29,537 INFO] Step 88750/102583; acc: 81.18; ppl: 1.83; xent: 0.60; lr: 0.00001; 2110/2577 tok/s; 9119 sec
  1903. [2019-02-04 20:22:34,473 INFO] Step 88800/102583; acc: 80.97; ppl: 1.84; xent: 0.61; lr: 0.00001; 1889/2692 tok/s; 9124 sec
  1904. [2019-02-04 20:22:39,379 INFO] Step 88850/102583; acc: 82.59; ppl: 1.75; xent: 0.56; lr: 0.00001; 2164/2700 tok/s; 9129 sec
  1905. [2019-02-04 20:22:44,425 INFO] Step 88900/102583; acc: 83.44; ppl: 1.72; xent: 0.54; lr: 0.00001; 2097/2405 tok/s; 9134 sec
  1906. [2019-02-04 20:22:49,658 INFO] Step 88950/102583; acc: 82.33; ppl: 1.76; xent: 0.56; lr: 0.00001; 1967/2661 tok/s; 9139 sec
  1907. [2019-02-04 20:22:54,433 INFO] Step 89000/102583; acc: 81.52; ppl: 1.80; xent: 0.59; lr: 0.00001; 1841/2675 tok/s; 9144 sec
  1908. [2019-02-04 20:22:59,538 INFO] Step 89050/102583; acc: 82.17; ppl: 1.77; xent: 0.57; lr: 0.00001; 2136/2657 tok/s; 9149 sec
  1909. [2019-02-04 20:23:04,246 INFO] Step 89100/102583; acc: 82.41; ppl: 1.80; xent: 0.59; lr: 0.00001; 2036/2513 tok/s; 9154 sec
  1910. [2019-02-04 20:23:09,006 INFO] Step 89150/102583; acc: 81.68; ppl: 1.80; xent: 0.59; lr: 0.00001; 1985/2662 tok/s; 9158 sec
  1911. [2019-02-04 20:23:14,334 INFO] Step 89200/102583; acc: 82.79; ppl: 1.71; xent: 0.54; lr: 0.00001; 2083/2607 tok/s; 9164 sec
  1912. [2019-02-04 20:23:19,250 INFO] Step 89250/102583; acc: 80.84; ppl: 1.85; xent: 0.62; lr: 0.00001; 2062/2657 tok/s; 9169 sec
  1913. [2019-02-04 20:23:24,179 INFO] Step 89300/102583; acc: 83.02; ppl: 1.73; xent: 0.55; lr: 0.00001; 2016/2670 tok/s; 9174 sec
  1914. [2019-02-04 20:23:29,192 INFO] Step 89350/102583; acc: 83.21; ppl: 1.75; xent: 0.56; lr: 0.00001; 2066/2617 tok/s; 9179 sec
  1915. [2019-02-04 20:23:33,957 INFO] Step 89400/102583; acc: 80.84; ppl: 1.87; xent: 0.63; lr: 0.00001; 1947/2551 tok/s; 9183 sec
  1916. [2019-02-04 20:23:39,085 INFO] Step 89450/102583; acc: 82.96; ppl: 1.71; xent: 0.54; lr: 0.00001; 2052/2685 tok/s; 9188 sec
  1917. [2019-02-04 20:23:44,009 INFO] Step 89500/102583; acc: 82.34; ppl: 1.78; xent: 0.57; lr: 0.00001; 2101/2568 tok/s; 9193 sec
  1918. [2019-02-04 20:23:49,314 INFO] Step 89550/102583; acc: 83.22; ppl: 1.73; xent: 0.55; lr: 0.00001; 2136/2468 tok/s; 9199 sec
  1919. [2019-02-04 20:23:53,876 INFO] Step 89600/102583; acc: 82.31; ppl: 1.78; xent: 0.58; lr: 0.00001; 1934/2738 tok/s; 9203 sec
  1920. [2019-02-04 20:23:58,824 INFO] Step 89650/102583; acc: 80.02; ppl: 1.91; xent: 0.65; lr: 0.00001; 1947/2638 tok/s; 9208 sec
  1921. [2019-02-04 20:24:03,906 INFO] Step 89700/102583; acc: 83.41; ppl: 1.73; xent: 0.55; lr: 0.00001; 2006/2586 tok/s; 9213 sec
  1922. [2019-02-04 20:24:08,638 INFO] Step 89750/102583; acc: 83.04; ppl: 1.73; xent: 0.55; lr: 0.00001; 2117/2693 tok/s; 9218 sec
  1923. [2019-02-04 20:24:13,661 INFO] Step 89800/102583; acc: 81.93; ppl: 1.81; xent: 0.59; lr: 0.00001; 2128/2473 tok/s; 9223 sec
  1924. [2019-02-04 20:24:18,767 INFO] Step 89850/102583; acc: 83.61; ppl: 1.68; xent: 0.52; lr: 0.00001; 2144/2597 tok/s; 9228 sec
  1925. [2019-02-04 20:24:23,552 INFO] Step 89900/102583; acc: 80.93; ppl: 1.83; xent: 0.60; lr: 0.00001; 1858/2788 tok/s; 9233 sec
  1926. [2019-02-04 20:24:28,642 INFO] Step 89950/102583; acc: 82.14; ppl: 1.77; xent: 0.57; lr: 0.00001; 2058/2664 tok/s; 9238 sec
  1927. [2019-02-04 20:24:33,603 INFO] Step 90000/102583; acc: 84.02; ppl: 1.69; xent: 0.52; lr: 0.00000; 2147/2532 tok/s; 9243 sec
  1928. [2019-02-04 20:24:33,663 INFO] Loading dataset from data/ted.valid.0.pt, number of examples: 873
  1929. [2019-02-04 20:24:34,966 INFO] Validation perplexity: 1.74294
  1930. [2019-02-04 20:24:34,967 INFO] Validation accuracy: 83.2452
  1931. [2019-02-04 20:24:34,968 INFO] Saving checkpoint models/ted1_step_90000.pt
  1932. [2019-02-04 20:24:45,524 INFO] Step 90050/102583; acc: 82.18; ppl: 1.77; xent: 0.57; lr: 0.00000; 828/1081 tok/s; 9255 sec
  1933. [2019-02-04 20:24:50,533 INFO] Step 90100/102583; acc: 81.90; ppl: 1.78; xent: 0.57; lr: 0.00000; 2080/2668 tok/s; 9260 sec
  1934. [2019-02-04 20:24:55,201 INFO] Step 90150/102583; acc: 81.86; ppl: 1.80; xent: 0.59; lr: 0.00000; 2023/2635 tok/s; 9265 sec
  1935. [2019-02-04 20:25:00,233 INFO] Step 90200/102583; acc: 82.79; ppl: 1.74; xent: 0.56; lr: 0.00000; 2101/2628 tok/s; 9270 sec
  1936. [2019-02-04 20:25:05,349 INFO] Step 90250/102583; acc: 82.75; ppl: 1.73; xent: 0.55; lr: 0.00000; 2091/2552 tok/s; 9275 sec
  1937. [2019-02-04 20:25:10,397 INFO] Step 90300/102583; acc: 81.39; ppl: 1.82; xent: 0.60; lr: 0.00000; 1754/2847 tok/s; 9280 sec
  1938. [2019-02-04 20:25:15,261 INFO] Step 90350/102583; acc: 80.79; ppl: 1.87; xent: 0.62; lr: 0.00000; 2021/2458 tok/s; 9285 sec
  1939. [2019-02-04 20:25:20,189 INFO] Step 90400/102583; acc: 83.96; ppl: 1.68; xent: 0.52; lr: 0.00000; 2182/2736 tok/s; 9290 sec
  1940. [2019-02-04 20:25:24,958 INFO] Step 90450/102583; acc: 82.40; ppl: 1.75; xent: 0.56; lr: 0.00000; 2014/2676 tok/s; 9294 sec
  1941. [2019-02-04 20:25:30,321 INFO] Step 90500/102583; acc: 82.64; ppl: 1.76; xent: 0.57; lr: 0.00000; 2047/2527 tok/s; 9300 sec
  1942. [2019-02-04 20:25:35,105 INFO] Step 90550/102583; acc: 81.71; ppl: 1.81; xent: 0.59; lr: 0.00000; 2064/2688 tok/s; 9304 sec
  1943. [2019-02-04 20:25:40,243 INFO] Step 90600/102583; acc: 83.50; ppl: 1.70; xent: 0.53; lr: 0.00000; 2158/2473 tok/s; 9310 sec
  1944. [2019-02-04 20:25:45,293 INFO] Step 90650/102583; acc: 81.53; ppl: 1.81; xent: 0.60; lr: 0.00000; 2046/2606 tok/s; 9315 sec
  1945. [2019-02-04 20:25:49,940 INFO] Step 90700/102583; acc: 81.87; ppl: 1.79; xent: 0.58; lr: 0.00000; 1945/2704 tok/s; 9319 sec
  1946. [2019-02-04 20:25:54,768 INFO] Step 90750/102583; acc: 82.12; ppl: 1.78; xent: 0.57; lr: 0.00000; 2103/2568 tok/s; 9324 sec
  1947. [2019-02-04 20:26:00,037 INFO] Step 90800/102583; acc: 82.65; ppl: 1.74; xent: 0.56; lr: 0.00000; 2007/2617 tok/s; 9329 sec
  1948. [2019-02-04 20:26:08,943 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  1949. [2019-02-04 20:26:11,211 INFO] Step 90850/102583; acc: 80.66; ppl: 1.88; xent: 0.63; lr: 0.00000; 866/1138 tok/s; 9341 sec
  1950. [2019-02-04 20:26:16,286 INFO] Step 90900/102583; acc: 83.46; ppl: 1.73; xent: 0.55; lr: 0.00000; 2169/2512 tok/s; 9346 sec
  1951. [2019-02-04 20:26:21,052 INFO] Step 90950/102583; acc: 82.41; ppl: 1.77; xent: 0.57; lr: 0.00000; 1889/2763 tok/s; 9350 sec
  1952. [2019-02-04 20:26:26,252 INFO] Step 91000/102583; acc: 84.30; ppl: 1.68; xent: 0.52; lr: 0.00000; 2078/2637 tok/s; 9356 sec
  1953. [2019-02-04 20:26:31,114 INFO] Step 91050/102583; acc: 81.67; ppl: 1.80; xent: 0.59; lr: 0.00000; 2107/2705 tok/s; 9361 sec
  1954. [2019-02-04 20:26:36,307 INFO] Step 91100/102583; acc: 82.97; ppl: 1.71; xent: 0.54; lr: 0.00000; 2057/2690 tok/s; 9366 sec
  1955. [2019-02-04 20:26:41,187 INFO] Step 91150/102583; acc: 81.43; ppl: 1.82; xent: 0.60; lr: 0.00000; 2050/2536 tok/s; 9371 sec
  1956. [2019-02-04 20:26:45,891 INFO] Step 91200/102583; acc: 83.31; ppl: 1.75; xent: 0.56; lr: 0.00000; 2195/2721 tok/s; 9375 sec
  1957. [2019-02-04 20:26:50,993 INFO] Step 91250/102583; acc: 80.76; ppl: 1.84; xent: 0.61; lr: 0.00000; 1974/2589 tok/s; 9380 sec
  1958. [2019-02-04 20:26:55,905 INFO] Step 91300/102583; acc: 81.42; ppl: 1.84; xent: 0.61; lr: 0.00000; 1940/2638 tok/s; 9385 sec
  1959. [2019-02-04 20:27:00,823 INFO] Step 91350/102583; acc: 83.75; ppl: 1.66; xent: 0.51; lr: 0.00000; 2221/2690 tok/s; 9390 sec
  1960. [2019-02-04 20:27:05,898 INFO] Step 91400/102583; acc: 81.23; ppl: 1.85; xent: 0.61; lr: 0.00000; 1932/2562 tok/s; 9395 sec
  1961. [2019-02-04 20:27:10,621 INFO] Step 91450/102583; acc: 81.78; ppl: 1.82; xent: 0.60; lr: 0.00000; 2123/2569 tok/s; 9400 sec
  1962. [2019-02-04 20:27:15,624 INFO] Step 91500/102583; acc: 81.21; ppl: 1.79; xent: 0.58; lr: 0.00000; 2033/2748 tok/s; 9405 sec
  1963. [2019-02-04 20:27:20,601 INFO] Step 91550/102583; acc: 84.50; ppl: 1.68; xent: 0.52; lr: 0.00000; 2091/2627 tok/s; 9410 sec
  1964. [2019-02-04 20:27:25,669 INFO] Step 91600/102583; acc: 82.00; ppl: 1.78; xent: 0.58; lr: 0.00000; 2131/2612 tok/s; 9415 sec
  1965. [2019-02-04 20:27:30,368 INFO] Step 91650/102583; acc: 82.91; ppl: 1.76; xent: 0.56; lr: 0.00000; 2163/2469 tok/s; 9420 sec
  1966. [2019-02-04 20:27:35,468 INFO] Step 91700/102583; acc: 81.84; ppl: 1.75; xent: 0.56; lr: 0.00000; 1864/2728 tok/s; 9425 sec
  1967. [2019-02-04 20:27:40,502 INFO] Step 91750/102583; acc: 82.73; ppl: 1.76; xent: 0.56; lr: 0.00000; 2055/2597 tok/s; 9430 sec
  1968. [2019-02-04 20:27:45,743 INFO] Step 91800/102583; acc: 82.01; ppl: 1.79; xent: 0.58; lr: 0.00000; 2019/2483 tok/s; 9435 sec
  1969. [2019-02-04 20:27:50,777 INFO] Step 91850/102583; acc: 80.80; ppl: 1.83; xent: 0.60; lr: 0.00000; 1899/2690 tok/s; 9440 sec
  1970. [2019-02-04 20:27:55,865 INFO] Step 91900/102583; acc: 83.80; ppl: 1.70; xent: 0.53; lr: 0.00000; 1970/2629 tok/s; 9445 sec
  1971. [2019-02-04 20:28:00,884 INFO] Step 91950/102583; acc: 82.36; ppl: 1.77; xent: 0.57; lr: 0.00000; 2226/2515 tok/s; 9450 sec
  1972. [2019-02-04 20:28:05,494 INFO] Step 92000/102583; acc: 81.15; ppl: 1.84; xent: 0.61; lr: 0.00000; 2100/2630 tok/s; 9455 sec
  1973. [2019-02-04 20:28:10,498 INFO] Step 92050/102583; acc: 81.91; ppl: 1.79; xent: 0.58; lr: 0.00000; 2004/2701 tok/s; 9460 sec
  1974. [2019-02-04 20:28:15,590 INFO] Step 92100/102583; acc: 82.58; ppl: 1.74; xent: 0.55; lr: 0.00000; 2092/2647 tok/s; 9465 sec
  1975. [2019-02-04 20:28:20,571 INFO] Step 92150/102583; acc: 83.54; ppl: 1.71; xent: 0.54; lr: 0.00000; 2165/2377 tok/s; 9470 sec
  1976. [2019-02-04 20:28:25,621 INFO] Step 92200/102583; acc: 80.97; ppl: 1.81; xent: 0.59; lr: 0.00000; 1901/2733 tok/s; 9475 sec
  1977. [2019-02-04 20:28:30,491 INFO] Step 92250/102583; acc: 81.84; ppl: 1.80; xent: 0.59; lr: 0.00000; 1832/2738 tok/s; 9480 sec
  1978. [2019-02-04 20:28:35,478 INFO] Step 92300/102583; acc: 83.41; ppl: 1.71; xent: 0.53; lr: 0.00000; 2174/2581 tok/s; 9485 sec
  1979. [2019-02-04 20:28:40,287 INFO] Step 92350/102583; acc: 81.75; ppl: 1.81; xent: 0.60; lr: 0.00000; 1980/2592 tok/s; 9490 sec
  1980. [2019-02-04 20:28:45,355 INFO] Step 92400/102583; acc: 82.17; ppl: 1.77; xent: 0.57; lr: 0.00000; 2051/2545 tok/s; 9495 sec
  1981. [2019-02-04 20:28:50,446 INFO] Step 92450/102583; acc: 81.76; ppl: 1.78; xent: 0.58; lr: 0.00000; 2035/2673 tok/s; 9500 sec
  1982. [2019-02-04 20:28:55,248 INFO] Step 92500/102583; acc: 82.50; ppl: 1.77; xent: 0.57; lr: 0.00000; 2135/2666 tok/s; 9505 sec
  1983. [2019-02-04 20:29:00,292 INFO] Step 92550/102583; acc: 82.70; ppl: 1.75; xent: 0.56; lr: 0.00000; 1976/2689 tok/s; 9510 sec
  1984. [2019-02-04 20:29:05,190 INFO] Step 92600/102583; acc: 82.47; ppl: 1.79; xent: 0.58; lr: 0.00000; 2030/2630 tok/s; 9515 sec
  1985. [2019-02-04 20:29:10,143 INFO] Step 92650/102583; acc: 81.73; ppl: 1.81; xent: 0.59; lr: 0.00000; 1972/2528 tok/s; 9520 sec
  1986. [2019-02-04 20:29:15,089 INFO] Step 92700/102583; acc: 82.20; ppl: 1.75; xent: 0.56; lr: 0.00000; 2068/2701 tok/s; 9524 sec
  1987. [2019-02-04 20:29:20,182 INFO] Step 92750/102583; acc: 82.50; ppl: 1.76; xent: 0.57; lr: 0.00000; 2080/2545 tok/s; 9530 sec
  1988. [2019-02-04 20:29:25,336 INFO] Step 92800/102583; acc: 83.42; ppl: 1.73; xent: 0.55; lr: 0.00000; 2163/2486 tok/s; 9535 sec
  1989. [2019-02-04 20:29:29,907 INFO] Step 92850/102583; acc: 81.21; ppl: 1.84; xent: 0.61; lr: 0.00000; 1832/2770 tok/s; 9539 sec
  1990. [2019-02-04 20:29:34,838 INFO] Step 92900/102583; acc: 80.98; ppl: 1.83; xent: 0.60; lr: 0.00000; 2084/2605 tok/s; 9544 sec
  1991. [2019-02-04 20:29:39,887 INFO] Step 92950/102583; acc: 83.22; ppl: 1.76; xent: 0.56; lr: 0.00000; 2019/2552 tok/s; 9549 sec
  1992. [2019-02-04 20:29:44,684 INFO] Step 93000/102583; acc: 83.33; ppl: 1.71; xent: 0.54; lr: 0.00000; 2026/2685 tok/s; 9554 sec
  1993. [2019-02-04 20:29:49,739 INFO] Step 93050/102583; acc: 82.23; ppl: 1.78; xent: 0.58; lr: 0.00000; 2199/2491 tok/s; 9559 sec
  1994. [2019-02-04 20:29:54,719 INFO] Step 93100/102583; acc: 83.04; ppl: 1.71; xent: 0.53; lr: 0.00000; 2147/2585 tok/s; 9564 sec
  1995. [2019-02-04 20:29:59,810 INFO] Step 93150/102583; acc: 81.99; ppl: 1.77; xent: 0.57; lr: 0.00000; 1802/2806 tok/s; 9569 sec
  1996. [2019-02-04 20:30:04,864 INFO] Step 93200/102583; acc: 82.26; ppl: 1.77; xent: 0.57; lr: 0.00000; 2032/2613 tok/s; 9574 sec
  1997. [2019-02-04 20:30:09,562 INFO] Step 93250/102583; acc: 82.89; ppl: 1.75; xent: 0.56; lr: 0.00000; 2183/2565 tok/s; 9579 sec
  1998. [2019-02-04 20:30:14,456 INFO] Step 93300/102583; acc: 82.55; ppl: 1.75; xent: 0.56; lr: 0.00000; 2017/2642 tok/s; 9584 sec
  1999. [2019-02-04 20:30:19,548 INFO] Step 93350/102583; acc: 81.91; ppl: 1.78; xent: 0.58; lr: 0.00000; 2089/2641 tok/s; 9589 sec
  2000. [2019-02-04 20:30:24,240 INFO] Step 93400/102583; acc: 81.05; ppl: 1.84; xent: 0.61; lr: 0.00000; 1995/2664 tok/s; 9594 sec
  2001. [2019-02-04 20:30:29,299 INFO] Step 93450/102583; acc: 83.83; ppl: 1.69; xent: 0.52; lr: 0.00000; 2091/2637 tok/s; 9599 sec
  2002. [2019-02-04 20:30:34,358 INFO] Step 93500/102583; acc: 82.93; ppl: 1.72; xent: 0.54; lr: 0.00000; 2114/2565 tok/s; 9604 sec
  2003. [2019-02-04 20:30:39,407 INFO] Step 93550/102583; acc: 81.38; ppl: 1.83; xent: 0.61; lr: 0.00000; 1779/2848 tok/s; 9609 sec
  2004. [2019-02-04 20:30:44,276 INFO] Step 93600/102583; acc: 81.50; ppl: 1.84; xent: 0.61; lr: 0.00000; 2005/2462 tok/s; 9614 sec
  2005. [2019-02-04 20:30:49,135 INFO] Step 93650/102583; acc: 83.51; ppl: 1.69; xent: 0.53; lr: 0.00000; 2187/2715 tok/s; 9619 sec
  2006. [2019-02-04 20:30:54,144 INFO] Step 93700/102583; acc: 82.48; ppl: 1.76; xent: 0.56; lr: 0.00000; 1983/2520 tok/s; 9624 sec
  2007. [2019-02-04 20:30:59,319 INFO] Step 93750/102583; acc: 83.03; ppl: 1.72; xent: 0.54; lr: 0.00000; 2107/2706 tok/s; 9629 sec
  2008. [2019-02-04 20:31:04,359 INFO] Step 93800/102583; acc: 81.65; ppl: 1.81; xent: 0.60; lr: 0.00000; 2119/2558 tok/s; 9634 sec
  2009. [2019-02-04 20:31:09,336 INFO] Step 93850/102583; acc: 82.89; ppl: 1.73; xent: 0.55; lr: 0.00000; 2032/2587 tok/s; 9639 sec
  2010. [2019-02-04 20:31:14,177 INFO] Step 93900/102583; acc: 81.79; ppl: 1.82; xent: 0.60; lr: 0.00000; 2116/2553 tok/s; 9644 sec
  2011. [2019-02-04 20:31:19,005 INFO] Step 93950/102583; acc: 82.30; ppl: 1.77; xent: 0.57; lr: 0.00000; 2002/2648 tok/s; 9648 sec
  2012. [2019-02-04 20:31:23,863 INFO] Step 94000/102583; acc: 82.14; ppl: 1.79; xent: 0.58; lr: 0.00000; 1962/2724 tok/s; 9653 sec
  2013. [2019-02-04 20:31:29,133 INFO] Step 94050/102583; acc: 82.41; ppl: 1.74; xent: 0.55; lr: 0.00000; 2006/2576 tok/s; 9659 sec
  2014. [2019-02-04 20:31:37,584 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  2015. [2019-02-04 20:31:40,615 INFO] Step 94100/102583; acc: 82.37; ppl: 1.78; xent: 0.58; lr: 0.00000; 886/1126 tok/s; 9670 sec
  2016. [2019-02-04 20:31:45,530 INFO] Step 94150/102583; acc: 82.67; ppl: 1.79; xent: 0.58; lr: 0.00000; 2176/2431 tok/s; 9675 sec
  2017. [2019-02-04 20:31:50,327 INFO] Step 94200/102583; acc: 82.58; ppl: 1.75; xent: 0.56; lr: 0.00000; 1940/2803 tok/s; 9680 sec
  2018. [2019-02-04 20:31:55,525 INFO] Step 94250/102583; acc: 83.22; ppl: 1.71; xent: 0.54; lr: 0.00000; 1991/2638 tok/s; 9685 sec
  2019. [2019-02-04 20:32:00,352 INFO] Step 94300/102583; acc: 81.65; ppl: 1.82; xent: 0.60; lr: 0.00000; 2157/2745 tok/s; 9690 sec
  2020. [2019-02-04 20:32:05,450 INFO] Step 94350/102583; acc: 82.92; ppl: 1.71; xent: 0.54; lr: 0.00000; 2020/2658 tok/s; 9695 sec
  2021. [2019-02-04 20:32:10,304 INFO] Step 94400/102583; acc: 81.96; ppl: 1.78; xent: 0.58; lr: 0.00000; 2180/2507 tok/s; 9700 sec
  2022. [2019-02-04 20:32:15,005 INFO] Step 94450/102583; acc: 82.70; ppl: 1.77; xent: 0.57; lr: 0.00000; 2101/2736 tok/s; 9704 sec
  2023. [2019-02-04 20:32:20,228 INFO] Step 94500/102583; acc: 80.77; ppl: 1.84; xent: 0.61; lr: 0.00000; 1947/2591 tok/s; 9710 sec
  2024. [2019-02-04 20:32:25,176 INFO] Step 94550/102583; acc: 82.28; ppl: 1.79; xent: 0.58; lr: 0.00000; 2002/2628 tok/s; 9715 sec
  2025. [2019-02-04 20:32:30,011 INFO] Step 94600/102583; acc: 83.37; ppl: 1.67; xent: 0.51; lr: 0.00000; 2161/2745 tok/s; 9719 sec
  2026. [2019-02-04 20:32:35,220 INFO] Step 94650/102583; acc: 80.79; ppl: 1.87; xent: 0.62; lr: 0.00000; 1955/2497 tok/s; 9725 sec
  2027. [2019-02-04 20:32:39,908 INFO] Step 94700/102583; acc: 82.39; ppl: 1.80; xent: 0.59; lr: 0.00000; 2188/2572 tok/s; 9729 sec
  2028. [2019-02-04 20:32:44,882 INFO] Step 94750/102583; acc: 81.14; ppl: 1.79; xent: 0.58; lr: 0.00000; 1943/2753 tok/s; 9734 sec
  2029. [2019-02-04 20:32:49,958 INFO] Step 94800/102583; acc: 84.27; ppl: 1.69; xent: 0.53; lr: 0.00000; 2175/2559 tok/s; 9739 sec
  2030. [2019-02-04 20:32:54,934 INFO] Step 94850/102583; acc: 81.62; ppl: 1.79; xent: 0.58; lr: 0.00000; 2046/2673 tok/s; 9744 sec
  2031. [2019-02-04 20:32:59,625 INFO] Step 94900/102583; acc: 82.89; ppl: 1.76; xent: 0.56; lr: 0.00000; 2214/2488 tok/s; 9749 sec
  2032. [2019-02-04 20:33:04,730 INFO] Step 94950/102583; acc: 82.54; ppl: 1.72; xent: 0.54; lr: 0.00000; 1858/2767 tok/s; 9754 sec
  2033. [2019-02-04 20:33:09,898 INFO] Step 95000/102583; acc: 81.96; ppl: 1.79; xent: 0.58; lr: 0.00000; 2034/2467 tok/s; 9759 sec
  2034. [2019-02-04 20:33:09,900 INFO] Saving checkpoint models/ted1_step_95000.pt
  2035. [2019-02-04 20:33:20,514 INFO] Step 95050/102583; acc: 83.00; ppl: 1.74; xent: 0.55; lr: 0.00000; 999/1227 tok/s; 9770 sec
  2036. [2019-02-04 20:33:25,528 INFO] Step 95100/102583; acc: 79.95; ppl: 1.88; xent: 0.63; lr: 0.00000; 1830/2718 tok/s; 9775 sec
  2037. [2019-02-04 20:33:30,658 INFO] Step 95150/102583; acc: 84.58; ppl: 1.66; xent: 0.50; lr: 0.00000; 2016/2607 tok/s; 9780 sec
  2038. [2019-02-04 20:33:35,570 INFO] Step 95200/102583; acc: 81.26; ppl: 1.83; xent: 0.60; lr: 0.00000; 2131/2504 tok/s; 9785 sec
  2039. [2019-02-04 20:33:40,163 INFO] Step 95250/102583; acc: 81.11; ppl: 1.84; xent: 0.61; lr: 0.00000; 2177/2619 tok/s; 9790 sec
  2040. [2019-02-04 20:33:45,231 INFO] Step 95300/102583; acc: 81.96; ppl: 1.77; xent: 0.57; lr: 0.00000; 1992/2742 tok/s; 9795 sec
  2041. [2019-02-04 20:33:50,362 INFO] Step 95350/102583; acc: 83.31; ppl: 1.73; xent: 0.55; lr: 0.00000; 2064/2663 tok/s; 9800 sec
  2042. [2019-02-04 20:33:55,536 INFO] Step 95400/102583; acc: 83.95; ppl: 1.69; xent: 0.53; lr: 0.00000; 2161/2325 tok/s; 9805 sec
  2043. [2019-02-04 20:34:00,440 INFO] Step 95450/102583; acc: 80.60; ppl: 1.84; xent: 0.61; lr: 0.00000; 1851/2826 tok/s; 9810 sec
  2044. [2019-02-04 20:34:05,277 INFO] Step 95500/102583; acc: 81.69; ppl: 1.81; xent: 0.59; lr: 0.00000; 1858/2718 tok/s; 9815 sec
  2045. [2019-02-04 20:34:10,313 INFO] Step 95550/102583; acc: 84.63; ppl: 1.66; xent: 0.51; lr: 0.00000; 2251/2502 tok/s; 9820 sec
  2046. [2019-02-04 20:34:15,053 INFO] Step 95600/102583; acc: 81.19; ppl: 1.85; xent: 0.61; lr: 0.00000; 1945/2640 tok/s; 9824 sec
  2047. [2019-02-04 20:34:20,162 INFO] Step 95650/102583; acc: 82.19; ppl: 1.77; xent: 0.57; lr: 0.00000; 2077/2507 tok/s; 9830 sec
  2048. [2019-02-04 20:34:25,296 INFO] Step 95700/102583; acc: 82.50; ppl: 1.74; xent: 0.55; lr: 0.00000; 2050/2752 tok/s; 9835 sec
  2049. [2019-02-04 20:34:30,039 INFO] Step 95750/102583; acc: 81.35; ppl: 1.83; xent: 0.61; lr: 0.00000; 2005/2591 tok/s; 9839 sec
  2050. [2019-02-04 20:34:35,052 INFO] Step 95800/102583; acc: 82.69; ppl: 1.75; xent: 0.56; lr: 0.00000; 1996/2745 tok/s; 9844 sec
  2051. [2019-02-04 20:34:40,106 INFO] Step 95850/102583; acc: 83.42; ppl: 1.72; xent: 0.54; lr: 0.00000; 2107/2585 tok/s; 9849 sec
  2052. [2019-02-04 20:34:45,083 INFO] Step 95900/102583; acc: 80.57; ppl: 1.86; xent: 0.62; lr: 0.00000; 1896/2512 tok/s; 9854 sec
  2053. [2019-02-04 20:34:49,897 INFO] Step 95950/102583; acc: 82.66; ppl: 1.74; xent: 0.55; lr: 0.00000; 2172/2751 tok/s; 9859 sec
  2054. [2019-02-04 20:34:54,949 INFO] Step 96000/102583; acc: 81.84; ppl: 1.80; xent: 0.59; lr: 0.00000; 1991/2529 tok/s; 9864 sec
  2055. [2019-02-04 20:35:00,078 INFO] Step 96050/102583; acc: 83.87; ppl: 1.70; xent: 0.53; lr: 0.00000; 2211/2490 tok/s; 9869 sec
  2056. [2019-02-04 20:35:04,795 INFO] Step 96100/102583; acc: 80.83; ppl: 1.87; xent: 0.62; lr: 0.00000; 1890/2737 tok/s; 9874 sec
  2057. [2019-02-04 20:35:09,871 INFO] Step 96150/102583; acc: 81.67; ppl: 1.80; xent: 0.59; lr: 0.00000; 2017/2627 tok/s; 9879 sec
  2058. [2019-02-04 20:35:14,730 INFO] Step 96200/102583; acc: 82.83; ppl: 1.77; xent: 0.57; lr: 0.00000; 2034/2578 tok/s; 9884 sec
  2059. [2019-02-04 20:35:19,607 INFO] Step 96250/102583; acc: 83.72; ppl: 1.69; xent: 0.52; lr: 0.00000; 2062/2628 tok/s; 9889 sec
  2060. [2019-02-04 20:35:24,597 INFO] Step 96300/102583; acc: 81.78; ppl: 1.81; xent: 0.60; lr: 0.00000; 2155/2503 tok/s; 9894 sec
  2061. [2019-02-04 20:35:29,538 INFO] Step 96350/102583; acc: 83.52; ppl: 1.69; xent: 0.52; lr: 0.00000; 2178/2597 tok/s; 9899 sec
  2062. [2019-02-04 20:35:34,608 INFO] Step 96400/102583; acc: 81.39; ppl: 1.80; xent: 0.59; lr: 0.00000; 1797/2836 tok/s; 9904 sec
  2063. [2019-02-04 20:35:39,728 INFO] Step 96450/102583; acc: 82.96; ppl: 1.75; xent: 0.56; lr: 0.00000; 2068/2564 tok/s; 9909 sec
  2064. [2019-02-04 20:35:44,416 INFO] Step 96500/102583; acc: 83.29; ppl: 1.72; xent: 0.54; lr: 0.00000; 2122/2606 tok/s; 9914 sec
  2065. [2019-02-04 20:35:49,520 INFO] Step 96550/102583; acc: 82.64; ppl: 1.74; xent: 0.56; lr: 0.00000; 2043/2644 tok/s; 9919 sec
  2066. [2019-02-04 20:35:54,339 INFO] Step 96600/102583; acc: 80.64; ppl: 1.85; xent: 0.62; lr: 0.00000; 2075/2640 tok/s; 9924 sec
  2067. [2019-02-04 20:35:59,224 INFO] Step 96650/102583; acc: 81.88; ppl: 1.80; xent: 0.59; lr: 0.00000; 1914/2675 tok/s; 9929 sec
  2068. [2019-02-04 20:36:04,048 INFO] Step 96700/102583; acc: 84.15; ppl: 1.68; xent: 0.52; lr: 0.00000; 2195/2621 tok/s; 9933 sec
  2069. [2019-02-04 20:36:09,337 INFO] Step 96750/102583; acc: 82.19; ppl: 1.73; xent: 0.55; lr: 0.00000; 2040/2571 tok/s; 9939 sec
  2070. [2019-02-04 20:36:14,541 INFO] Step 96800/102583; acc: 81.39; ppl: 1.85; xent: 0.61; lr: 0.00000; 1833/2754 tok/s; 9944 sec
  2071. [2019-02-04 20:36:19,053 INFO] Step 96850/102583; acc: 81.31; ppl: 1.85; xent: 0.61; lr: 0.00000; 2075/2533 tok/s; 9948 sec
  2072. [2019-02-04 20:36:24,141 INFO] Step 96900/102583; acc: 83.48; ppl: 1.69; xent: 0.52; lr: 0.00000; 2078/2758 tok/s; 9954 sec
  2073. [2019-02-04 20:36:29,147 INFO] Step 96950/102583; acc: 82.95; ppl: 1.74; xent: 0.55; lr: 0.00000; 1993/2496 tok/s; 9959 sec
  2074. [2019-02-04 20:36:34,132 INFO] Step 97000/102583; acc: 82.10; ppl: 1.77; xent: 0.57; lr: 0.00000; 2151/2708 tok/s; 9964 sec
  2075. [2019-02-04 20:36:39,167 INFO] Step 97050/102583; acc: 82.32; ppl: 1.77; xent: 0.57; lr: 0.00000; 2134/2561 tok/s; 9969 sec
  2076. [2019-02-04 20:36:44,464 INFO] Step 97100/102583; acc: 83.25; ppl: 1.72; xent: 0.54; lr: 0.00000; 2057/2515 tok/s; 9974 sec
  2077. [2019-02-04 20:36:48,976 INFO] Step 97150/102583; acc: 81.32; ppl: 1.86; xent: 0.62; lr: 0.00000; 2040/2647 tok/s; 9978 sec
  2078. [2019-02-04 20:36:53,771 INFO] Step 97200/102583; acc: 82.58; ppl: 1.75; xent: 0.56; lr: 0.00000; 2095/2607 tok/s; 9983 sec
  2079. [2019-02-04 20:36:58,795 INFO] Step 97250/102583; acc: 82.66; ppl: 1.76; xent: 0.57; lr: 0.00000; 1969/2670 tok/s; 9988 sec
  2080. [2019-02-04 20:37:04,084 INFO] Step 97300/102583; acc: 81.54; ppl: 1.77; xent: 0.57; lr: 0.00000; 1910/2573 tok/s; 9993 sec
  2081. [2019-02-04 20:37:11,403 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  2082. [2019-02-04 20:37:15,251 INFO] Step 97350/102583; acc: 83.03; ppl: 1.75; xent: 0.56; lr: 0.00000; 967/1195 tok/s; 10005 sec
  2083. [2019-02-04 20:37:19,849 INFO] Step 97400/102583; acc: 81.40; ppl: 1.87; xent: 0.62; lr: 0.00000; 2145/2468 tok/s; 10009 sec
  2084. [2019-02-04 20:37:24,671 INFO] Step 97450/102583; acc: 82.91; ppl: 1.73; xent: 0.55; lr: 0.00000; 1969/2804 tok/s; 10014 sec
  2085. [2019-02-04 20:37:29,971 INFO] Step 97500/102583; acc: 83.15; ppl: 1.73; xent: 0.55; lr: 0.00000; 2018/2614 tok/s; 10019 sec
  2086. [2019-02-04 20:37:34,755 INFO] Step 97550/102583; acc: 82.34; ppl: 1.77; xent: 0.57; lr: 0.00000; 2104/2823 tok/s; 10024 sec
  2087. [2019-02-04 20:37:39,736 INFO] Step 97600/102583; acc: 81.93; ppl: 1.77; xent: 0.57; lr: 0.00000; 2004/2601 tok/s; 10029 sec
  2088. [2019-02-04 20:37:44,777 INFO] Step 97650/102583; acc: 82.92; ppl: 1.73; xent: 0.55; lr: 0.00000; 2222/2525 tok/s; 10034 sec
  2089. [2019-02-04 20:37:49,367 INFO] Step 97700/102583; acc: 81.67; ppl: 1.83; xent: 0.61; lr: 0.00000; 2073/2767 tok/s; 10039 sec
  2090. [2019-02-04 20:37:54,608 INFO] Step 97750/102583; acc: 81.51; ppl: 1.80; xent: 0.59; lr: 0.00000; 1904/2605 tok/s; 10044 sec
  2091. [2019-02-04 20:37:59,672 INFO] Step 97800/102583; acc: 82.70; ppl: 1.77; xent: 0.57; lr: 0.00000; 2156/2561 tok/s; 10049 sec
  2092. [2019-02-04 20:38:04,514 INFO] Step 97850/102583; acc: 83.45; ppl: 1.68; xent: 0.52; lr: 0.00000; 2031/2805 tok/s; 10054 sec
  2093. [2019-02-04 20:38:09,540 INFO] Step 97900/102583; acc: 80.37; ppl: 1.89; xent: 0.64; lr: 0.00000; 2048/2440 tok/s; 10059 sec
  2094. [2019-02-04 20:38:14,475 INFO] Step 97950/102583; acc: 83.31; ppl: 1.74; xent: 0.55; lr: 0.00000; 2070/2633 tok/s; 10064 sec
  2095. [2019-02-04 20:38:19,312 INFO] Step 98000/102583; acc: 81.04; ppl: 1.81; xent: 0.60; lr: 0.00000; 1994/2759 tok/s; 10069 sec
  2096. [2019-02-04 20:38:24,272 INFO] Step 98050/102583; acc: 84.52; ppl: 1.69; xent: 0.53; lr: 0.00000; 2212/2514 tok/s; 10074 sec
  2097. [2019-02-04 20:38:29,500 INFO] Step 98100/102583; acc: 81.58; ppl: 1.78; xent: 0.58; lr: 0.00000; 1963/2699 tok/s; 10079 sec
  2098. [2019-02-04 20:38:34,008 INFO] Step 98150/102583; acc: 82.38; ppl: 1.79; xent: 0.58; lr: 0.00000; 2214/2484 tok/s; 10083 sec
  2099. [2019-02-04 20:38:39,013 INFO] Step 98200/102583; acc: 82.69; ppl: 1.71; xent: 0.54; lr: 0.00000; 1883/2730 tok/s; 10088 sec
  2100. [2019-02-04 20:38:44,339 INFO] Step 98250/102583; acc: 82.44; ppl: 1.77; xent: 0.57; lr: 0.00000; 1994/2478 tok/s; 10094 sec
  2101. [2019-02-04 20:38:49,347 INFO] Step 98300/102583; acc: 82.70; ppl: 1.75; xent: 0.56; lr: 0.00000; 2161/2588 tok/s; 10099 sec
  2102. [2019-02-04 20:38:54,676 INFO] Step 98350/102583; acc: 81.17; ppl: 1.80; xent: 0.59; lr: 0.00000; 1858/2696 tok/s; 10104 sec
  2103. [2019-02-04 20:38:59,464 INFO] Step 98400/102583; acc: 83.59; ppl: 1.73; xent: 0.55; lr: 0.00000; 2048/2632 tok/s; 10109 sec
  2104. [2019-02-04 20:39:04,468 INFO] Step 98450/102583; acc: 80.40; ppl: 1.86; xent: 0.62; lr: 0.00000; 2041/2576 tok/s; 10114 sec
  2105. [2019-02-04 20:39:09,106 INFO] Step 98500/102583; acc: 82.54; ppl: 1.77; xent: 0.57; lr: 0.00000; 2247/2536 tok/s; 10118 sec
  2106. [2019-02-04 20:39:14,055 INFO] Step 98550/102583; acc: 80.76; ppl: 1.83; xent: 0.60; lr: 0.00000; 1967/2764 tok/s; 10123 sec
  2107. [2019-02-04 20:39:19,165 INFO] Step 98600/102583; acc: 83.68; ppl: 1.70; xent: 0.53; lr: 0.00000; 2060/2660 tok/s; 10129 sec
  2108. [2019-02-04 20:39:24,468 INFO] Step 98650/102583; acc: 83.39; ppl: 1.72; xent: 0.54; lr: 0.00000; 2076/2307 tok/s; 10134 sec
  2109. [2019-02-04 20:39:29,282 INFO] Step 98700/102583; acc: 80.95; ppl: 1.80; xent: 0.59; lr: 0.00000; 1948/2861 tok/s; 10139 sec
  2110. [2019-02-04 20:39:34,011 INFO] Step 98750/102583; acc: 81.54; ppl: 1.83; xent: 0.60; lr: 0.00000; 1873/2684 tok/s; 10143 sec
  2111. [2019-02-04 20:39:39,072 INFO] Step 98800/102583; acc: 84.35; ppl: 1.67; xent: 0.51; lr: 0.00000; 2241/2519 tok/s; 10148 sec
  2112. [2019-02-04 20:39:43,880 INFO] Step 98850/102583; acc: 80.09; ppl: 1.90; xent: 0.64; lr: 0.00000; 1837/2682 tok/s; 10153 sec
  2113. [2019-02-04 20:39:49,086 INFO] Step 98900/102583; acc: 83.74; ppl: 1.68; xent: 0.52; lr: 0.00000; 2161/2535 tok/s; 10158 sec
  2114. [2019-02-04 20:39:54,082 INFO] Step 98950/102583; acc: 81.68; ppl: 1.79; xent: 0.58; lr: 0.00000; 2031/2712 tok/s; 10163 sec
  2115. [2019-02-04 20:39:58,945 INFO] Step 99000/102583; acc: 82.10; ppl: 1.78; xent: 0.58; lr: 0.00000; 2049/2595 tok/s; 10168 sec
  2116. [2019-02-04 20:40:03,789 INFO] Step 99050/102583; acc: 82.16; ppl: 1.78; xent: 0.58; lr: 0.00000; 1972/2714 tok/s; 10173 sec
  2117. [2019-02-04 20:40:08,846 INFO] Step 99100/102583; acc: 83.58; ppl: 1.70; xent: 0.53; lr: 0.00000; 2207/2573 tok/s; 10178 sec
  2118. [2019-02-04 20:40:13,820 INFO] Step 99150/102583; acc: 79.50; ppl: 1.93; xent: 0.66; lr: 0.00000; 1820/2531 tok/s; 10183 sec
  2119. [2019-02-04 20:40:18,741 INFO] Step 99200/102583; acc: 83.52; ppl: 1.69; xent: 0.53; lr: 0.00000; 2239/2649 tok/s; 10188 sec
  2120. [2019-02-04 20:40:23,647 INFO] Step 99250/102583; acc: 81.17; ppl: 1.85; xent: 0.61; lr: 0.00000; 1896/2613 tok/s; 10193 sec
  2121. [2019-02-04 20:40:28,778 INFO] Step 99300/102583; acc: 84.35; ppl: 1.68; xent: 0.52; lr: 0.00000; 2173/2493 tok/s; 10198 sec
  2122. [2019-02-04 20:40:33,625 INFO] Step 99350/102583; acc: 81.02; ppl: 1.85; xent: 0.62; lr: 0.00000; 1930/2697 tok/s; 10203 sec
  2123. [2019-02-04 20:40:38,768 INFO] Step 99400/102583; acc: 82.81; ppl: 1.73; xent: 0.55; lr: 0.00000; 2001/2658 tok/s; 10208 sec
  2124. [2019-02-04 20:40:43,490 INFO] Step 99450/102583; acc: 81.23; ppl: 1.87; xent: 0.62; lr: 0.00000; 2056/2606 tok/s; 10213 sec
  2125. [2019-02-04 20:40:48,637 INFO] Step 99500/102583; acc: 84.22; ppl: 1.66; xent: 0.51; lr: 0.00000; 2002/2548 tok/s; 10218 sec
  2126. [2019-02-04 20:40:53,330 INFO] Step 99550/102583; acc: 81.18; ppl: 1.85; xent: 0.61; lr: 0.00000; 2212/2581 tok/s; 10223 sec
  2127. [2019-02-04 20:40:58,400 INFO] Step 99600/102583; acc: 82.97; ppl: 1.70; xent: 0.53; lr: 0.00000; 2099/2577 tok/s; 10228 sec
  2128. [2019-02-04 20:41:03,548 INFO] Step 99650/102583; acc: 81.99; ppl: 1.77; xent: 0.57; lr: 0.00000; 1878/2790 tok/s; 10233 sec
  2129. [2019-02-04 20:41:08,485 INFO] Step 99700/102583; acc: 82.33; ppl: 1.77; xent: 0.57; lr: 0.00000; 2087/2592 tok/s; 10238 sec
  2130. [2019-02-04 20:41:13,229 INFO] Step 99750/102583; acc: 83.91; ppl: 1.70; xent: 0.53; lr: 0.00000; 2159/2569 tok/s; 10243 sec
  2131. [2019-02-04 20:41:18,274 INFO] Step 99800/102583; acc: 81.70; ppl: 1.78; xent: 0.58; lr: 0.00000; 1984/2685 tok/s; 10248 sec
  2132. [2019-02-04 20:41:23,031 INFO] Step 99850/102583; acc: 80.96; ppl: 1.83; xent: 0.61; lr: 0.00000; 2088/2634 tok/s; 10252 sec
  2133. [2019-02-04 20:41:28,079 INFO] Step 99900/102583; acc: 82.86; ppl: 1.74; xent: 0.55; lr: 0.00000; 1968/2651 tok/s; 10257 sec
  2134. [2019-02-04 20:41:32,761 INFO] Step 99950/102583; acc: 82.80; ppl: 1.75; xent: 0.56; lr: 0.00000; 2136/2647 tok/s; 10262 sec
  2135. [2019-02-04 20:41:38,064 INFO] Step 100000/102583; acc: 82.47; ppl: 1.73; xent: 0.55; lr: 0.00000; 2034/2582 tok/s; 10267 sec
  2136. [2019-02-04 20:41:38,083 INFO] Loading dataset from data/ted.valid.0.pt, number of examples: 873
  2137. [2019-02-04 20:41:39,380 INFO] Validation perplexity: 1.7422
  2138. [2019-02-04 20:41:39,381 INFO] Validation accuracy: 83.3844
  2139. [2019-02-04 20:41:39,382 INFO] Saving checkpoint models/ted1_step_100000.pt
  2140. [2019-02-04 20:41:50,081 INFO] Step 100050/102583; acc: 81.32; ppl: 1.85; xent: 0.62; lr: 0.00000; 805/1159 tok/s; 10279 sec
  2141. [2019-02-04 20:41:54,883 INFO] Step 100100/102583; acc: 82.26; ppl: 1.79; xent: 0.58; lr: 0.00000; 2108/2524 tok/s; 10284 sec
  2142. [2019-02-04 20:41:59,662 INFO] Step 100150/102583; acc: 83.14; ppl: 1.71; xent: 0.53; lr: 0.00000; 1987/2789 tok/s; 10289 sec
  2143. [2019-02-04 20:42:04,956 INFO] Step 100200/102583; acc: 83.43; ppl: 1.72; xent: 0.54; lr: 0.00000; 2046/2525 tok/s; 10294 sec
  2144. [2019-02-04 20:42:09,741 INFO] Step 100250/102583; acc: 81.47; ppl: 1.80; xent: 0.59; lr: 0.00000; 2076/2736 tok/s; 10299 sec
  2145. [2019-02-04 20:42:14,731 INFO] Step 100300/102583; acc: 82.13; ppl: 1.79; xent: 0.58; lr: 0.00000; 2166/2528 tok/s; 10304 sec
  2146. [2019-02-04 20:42:20,179 INFO] Step 100350/102583; acc: 83.97; ppl: 1.68; xent: 0.52; lr: 0.00000; 2061/2518 tok/s; 10310 sec
  2147. [2019-02-04 20:42:24,590 INFO] Step 100400/102583; acc: 80.68; ppl: 1.87; xent: 0.63; lr: 0.00000; 2070/2660 tok/s; 10314 sec
  2148. [2019-02-04 20:42:29,363 INFO] Step 100450/102583; acc: 81.87; ppl: 1.77; xent: 0.57; lr: 0.00000; 2011/2626 tok/s; 10319 sec
  2149. [2019-02-04 20:42:34,710 INFO] Step 100500/102583; acc: 82.78; ppl: 1.74; xent: 0.56; lr: 0.00000; 1997/2570 tok/s; 10324 sec
  2150. [2019-02-04 20:42:39,844 INFO] Step 100550/102583; acc: 81.99; ppl: 1.77; xent: 0.57; lr: 0.00000; 1928/2619 tok/s; 10329 sec
  2151. [2019-02-04 20:42:46,710 INFO] Loading dataset from data/ted.train.0.pt, number of examples: 207592
  2152. [2019-02-04 20:42:51,143 INFO] Step 100600/102583; acc: 83.11; ppl: 1.75; xent: 0.56; lr: 0.00000; 955/1154 tok/s; 10341 sec
  2153. [2019-02-04 20:42:55,713 INFO] Step 100650/102583; acc: 81.20; ppl: 1.87; xent: 0.63; lr: 0.00000; 2088/2550 tok/s; 10345 sec
  2154. [2019-02-04 20:43:00,717 INFO] Step 100700/102583; acc: 83.73; ppl: 1.68; xent: 0.52; lr: 0.00000; 2033/2799 tok/s; 10350 sec
  2155. [2019-02-04 20:43:05,890 INFO] Step 100750/102583; acc: 82.19; ppl: 1.78; xent: 0.58; lr: 0.00000; 1963/2620 tok/s; 10355 sec
  2156. [2019-02-04 20:43:10,601 INFO] Step 100800/102583; acc: 82.68; ppl: 1.77; xent: 0.57; lr: 0.00000; 2095/2819 tok/s; 10360 sec
  2157. [2019-02-04 20:43:15,733 INFO] Step 100850/102583; acc: 82.82; ppl: 1.74; xent: 0.55; lr: 0.00000; 2113/2497 tok/s; 10365 sec
  2158. [2019-02-04 20:43:20,639 INFO] Step 100900/102583; acc: 83.20; ppl: 1.72; xent: 0.54; lr: 0.00000; 2185/2635 tok/s; 10370 sec
  2159. [2019-02-04 20:43:25,428 INFO] Step 100950/102583; acc: 81.34; ppl: 1.83; xent: 0.60; lr: 0.00000; 2111/2638 tok/s; 10375 sec
  2160. [2019-02-04 20:43:30,573 INFO] Step 101000/102583; acc: 80.54; ppl: 1.85; xent: 0.62; lr: 0.00000; 1777/2658 tok/s; 10380 sec
  2161. [2019-02-04 20:43:35,508 INFO] Step 101050/102583; acc: 83.26; ppl: 1.74; xent: 0.55; lr: 0.00000; 2198/2640 tok/s; 10385 sec
  2162. [2019-02-04 20:43:40,633 INFO] Step 101100/102583; acc: 82.77; ppl: 1.71; xent: 0.54; lr: 0.00000; 1892/2756 tok/s; 10390 sec
  2163. [2019-02-04 20:43:45,404 INFO] Step 101150/102583; acc: 81.47; ppl: 1.85; xent: 0.61; lr: 0.00000; 2280/2441 tok/s; 10395 sec
  2164. [2019-02-04 20:43:50,270 INFO] Step 101200/102583; acc: 82.61; ppl: 1.77; xent: 0.57; lr: 0.00000; 2046/2635 tok/s; 10400 sec
  2165. [2019-02-04 20:43:55,200 INFO] Step 101250/102583; acc: 80.88; ppl: 1.83; xent: 0.61; lr: 0.00000; 1913/2689 tok/s; 10405 sec
  2166. [2019-02-04 20:44:00,311 INFO] Step 101300/102583; acc: 84.89; ppl: 1.65; xent: 0.50; lr: 0.00000; 2275/2564 tok/s; 10410 sec
  2167. [2019-02-04 20:44:05,340 INFO] Step 101350/102583; acc: 80.91; ppl: 1.81; xent: 0.59; lr: 0.00000; 1928/2730 tok/s; 10415 sec
  2168. [2019-02-04 20:44:09,894 INFO] Step 101400/102583; acc: 82.34; ppl: 1.79; xent: 0.58; lr: 0.00000; 2136/2528 tok/s; 10419 sec
  2169. [2019-02-04 20:44:15,037 INFO] Step 101450/102583; acc: 83.38; ppl: 1.69; xent: 0.53; lr: 0.00000; 1942/2682 tok/s; 10424 sec
  2170. [2019-02-04 20:44:20,139 INFO] Step 101500/102583; acc: 82.27; ppl: 1.78; xent: 0.58; lr: 0.00000; 2007/2468 tok/s; 10430 sec
  2171. [2019-02-04 20:44:25,188 INFO] Step 101550/102583; acc: 82.52; ppl: 1.76; xent: 0.56; lr: 0.00000; 2105/2614 tok/s; 10435 sec
  2172. [2019-02-04 20:44:30,523 INFO] Step 101600/102583; acc: 81.42; ppl: 1.78; xent: 0.58; lr: 0.00000; 1892/2703 tok/s; 10440 sec
  2173. [2019-02-04 20:44:35,232 INFO] Step 101650/102583; acc: 83.59; ppl: 1.72; xent: 0.54; lr: 0.00000; 2082/2582 tok/s; 10445 sec
  2174. [2019-02-04 20:44:40,243 INFO] Step 101700/102583; acc: 80.70; ppl: 1.84; xent: 0.61; lr: 0.00000; 2140/2536 tok/s; 10450 sec
  2175. [2019-02-04 20:44:45,090 INFO] Step 101750/102583; acc: 82.06; ppl: 1.78; xent: 0.58; lr: 0.00000; 2031/2719 tok/s; 10454 sec
  2176. [2019-02-04 20:44:49,846 INFO] Step 101800/102583; acc: 81.09; ppl: 1.84; xent: 0.61; lr: 0.00000; 2061/2616 tok/s; 10459 sec
  2177. [2019-02-04 20:44:55,163 INFO] Step 101850/102583; acc: 83.10; ppl: 1.73; xent: 0.55; lr: 0.00000; 1927/2585 tok/s; 10465 sec
  2178. [2019-02-04 20:45:00,297 INFO] Step 101900/102583; acc: 83.94; ppl: 1.69; xent: 0.52; lr: 0.00000; 2211/2383 tok/s; 10470 sec
  2179. [2019-02-04 20:45:05,074 INFO] Step 101950/102583; acc: 80.87; ppl: 1.82; xent: 0.60; lr: 0.00000; 1896/2875 tok/s; 10474 sec
  2180. [2019-02-04 20:45:09,966 INFO] Step 102000/102583; acc: 81.75; ppl: 1.78; xent: 0.58; lr: 0.00000; 1981/2705 tok/s; 10479 sec
  2181. [2019-02-04 20:45:15,042 INFO] Step 102050/102583; acc: 84.30; ppl: 1.69; xent: 0.53; lr: 0.00000; 2181/2444 tok/s; 10484 sec
  2182. [2019-02-04 20:45:19,752 INFO] Step 102100/102583; acc: 80.81; ppl: 1.87; xent: 0.63; lr: 0.00000; 1832/2685 tok/s; 10489 sec
  2183. [2019-02-04 20:45:24,960 INFO] Step 102150/102583; acc: 83.24; ppl: 1.69; xent: 0.52; lr: 0.00000; 2153/2604 tok/s; 10494 sec
  2184. [2019-02-04 20:45:29,900 INFO] Step 102200/102583; acc: 81.51; ppl: 1.81; xent: 0.59; lr: 0.00000; 2053/2670 tok/s; 10499 sec
  2185. [2019-02-04 20:45:34,773 INFO] Step 102250/102583; acc: 82.02; ppl: 1.79; xent: 0.58; lr: 0.00000; 2018/2592 tok/s; 10504 sec
  2186. [2019-02-04 20:45:39,833 INFO] Step 102300/102583; acc: 83.06; ppl: 1.74; xent: 0.55; lr: 0.00000; 1878/2721 tok/s; 10509 sec
  2187. [2019-02-04 20:45:44,738 INFO] Step 102350/102583; acc: 83.05; ppl: 1.75; xent: 0.56; lr: 0.00000; 2309/2506 tok/s; 10514 sec
  2188. [2019-02-04 20:45:49,821 INFO] Step 102400/102583; acc: 80.38; ppl: 1.86; xent: 0.62; lr: 0.00000; 1809/2674 tok/s; 10519 sec
  2189. [2019-02-04 20:45:54,650 INFO] Step 102450/102583; acc: 83.30; ppl: 1.73; xent: 0.55; lr: 0.00000; 2239/2526 tok/s; 10524 sec
  2190. [2019-02-04 20:45:59,655 INFO] Step 102500/102583; acc: 81.88; ppl: 1.79; xent: 0.58; lr: 0.00000; 1978/2644 tok/s; 10529 sec
  2191. [2019-02-04 20:46:04,736 INFO] Step 102550/102583; acc: 83.49; ppl: 1.73; xent: 0.55; lr: 0.00000; 2052/2456 tok/s; 10534 sec
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!