API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Mar 19th, 2019
422
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 408.09 KB | None | 0 0
raw download clone embed print report
  1. 2019-03-12 02:49:54,679 : set device 0 out of 2 cuda devices
  2. 2019-03-12 02:49:54,680 : Namespace(checkpoint=None, data_path='./data', dataset='CIFAR10', device=0, e_increasing_layer_size=False, e_layer_size=1000, e_number_of_hidden_layers=3, epochs=100000000000000000000, gamma=1.0, initial_weights=None, lamBda=0.0, lr_init=0.0001, model='ResNet32', momentum=0.0, num_workers=4, resume_checkpoint=False, save_checkpoint=False, seed=2092, train_log_freq=40, w_batch_size=10, weight_sharing=False, wg_number_of_hidden_layers=2, with_adam=True, with_lr_plateau_schedule=False, with_norm=2, with_residual=True, x_batch_size=256, z_batch_size=20, z_dim=500, z_std=1.0)
  3. 2019-03-12 02:50:01,291 : {'weight_sharing': False, 'input_noise_size': 500, 'with_bias': True, 'wg_number_of_hidden_layers': 2, 'wg_hidden_layer_size_formula': <function train.<locals>.wg_hidden_layer_size_formula at 0x7f337b6072f0>, 'with_batchnorm': True, 'e_layer_size': 1000, 'with_residual': True, 'with_layernorm': False, 'e_increasing_layer_size': False, 'code_size_formula': <function train.<locals>.code_size_formula at 0x7f337b607378>, 'e_number_of_hidden_layers': 3}
  4. 2019-03-12 02:50:02,224 : targetnet size: 466906
  5. 2019-03-12 02:50:02,227 : hypernet size: 56617607
  6. 2019-03-12 02:50:06,725 : epoch 0 , step 0 loss0= 0.0000 loss1= 74.7655 loss2=-5793.2637 lr= 0.000100
  7. 2019-03-12 02:50:42,405 : epoch 0 , step 40 loss0= 0.0000 loss1= 44.3598 loss2=-3716.9851 lr= 0.000100
  8. 2019-03-12 02:51:17,632 : epoch 0 , step 80 loss0= 0.0000 loss1= 39.9139 loss2=-3009.6038 lr= 0.000100
  9. 2019-03-12 02:51:52,867 : epoch 0 , step 120 loss0= 0.0000 loss1= 39.1175 loss2=-3706.8103 lr= 0.000100
  10. 2019-03-12 02:52:28,353 : epoch 0 , step 160 loss0= 0.0000 loss1= 36.3611 loss2=-3005.1748 lr= 0.000100
  11. 2019-03-12 02:53:03,794 : ## train ##
  12. 2019-03-12 02:57:58,585 : >> epoch 0 : average_loss= 1.7346, average_accuracy=33.349%, mean_weight_std= 0.1522, mean_normalized_std= 2.2162 mean_dist= 445.0
  13. 2019-03-12 02:57:58,596 : >> epoch 0 : median_loss= 1.7309, median_accuracy=33.890%, median_weight_std= 0.1364, median_normalized_std= 0.3626 median_dist= 442.9
  14. 2019-03-12 02:57:58,596 : >> epoch 0 : ensemble accuracy: 34.96%
  15. 2019-03-12 02:57:58,596 : >> epoch 0 : loss histogram: (0.00,0.72, 18),(0.72,1.43, 0),(1.43,2.15, 4),(2.15,2.87, 0),(2.87,3.59, 18),
  16. 2019-03-12 02:57:58,596 : >> epoch 0 : accuracy histogram: (0.00,14.23, 18),(14.23,28.46, 0),(28.46,42.70, 4),(42.70,56.93, 0),(56.93,71.16, 18),
  17. 2019-03-12 02:57:58,602 : >> epoch 0 : weight std histogram: (0.0000,0.2865, 432257),(0.2865,0.5730, 33690),(0.5730,0.8595, 904),(0.8595,1.1460, 51),(1.1460,1.4325, 4),
  18. 2019-03-12 02:57:58,608 : >> epoch 0 : weight normalized std histogram: (0.0000,4939.9969, 466889),(4939.9969,9879.9937, 8),(9879.9937,14819.9906, 4),(14819.9906,19759.9875, 3),(19759.9875,24699.9844, 2),
  19. 2019-03-12 02:57:58,608 : >> epoch 0 : dist histogram: (383.3129,410.3929, 2),(410.3929,437.4729, 6),(437.4729,464.5529, 9),(464.5529,491.6328, 2),(491.6328,518.7128, 1),
  20. 2019-03-12 02:57:58,608 : ## eval ##
  21. 2019-03-12 03:02:50,536 : >> epoch 0 : average_loss= 1.7608, average_accuracy=32.989%, mean_weight_std= 0.1523, mean_normalized_std= 3.7480 mean_dist= 445.0
  22. 2019-03-12 03:02:50,548 : >> epoch 0 : median_loss= 1.7889, median_accuracy=32.475%, median_weight_std= 0.1366, median_normalized_std= 0.3623 median_dist= 441.4
  23. 2019-03-12 03:02:50,548 : >> epoch 0 : ensemble accuracy: 34.70%
  24. 2019-03-12 03:02:50,548 : >> epoch 0 : loss histogram: (0.00,0.74, 18),(0.74,1.49, 0),(1.49,2.23, 4),(2.23,2.97, 0),(2.97,3.71, 18),
  25. 2019-03-12 03:02:50,548 : >> epoch 0 : accuracy histogram: (0.00,13.67, 18),(13.67,27.34, 0),(27.34,41.02, 4),(41.02,54.69, 0),(54.69,68.36, 18),
  26. 2019-03-12 03:02:50,554 : >> epoch 0 : weight std histogram: (0.0000,0.3430, 450700),(0.3430,0.6860, 15898),(0.6860,1.0290, 294),(1.0290,1.3720, 13),(1.3720,1.7151, 1),
  27. 2019-03-12 03:02:50,560 : >> epoch 0 : weight normalized std histogram: (0.0000,127428.9125, 466905),(127428.9125,254857.8250, 0),(254857.8250,382286.7375, 0),(382286.7375,509715.6500, 0),(509715.6500,637144.5625, 1),
  28. 2019-03-12 03:02:50,560 : >> epoch 0 : dist histogram: (378.6437,405.1288, 1),(405.1288,431.6140, 5),(431.6140,458.0992, 8),(458.0992,484.5844, 3),(484.5844,511.0695, 3),
  29. 2019-03-12 03:02:50,607 : >> epoch 0 : dist pair-wise
  30. 2019-03-12 03:02:50,613 :
  31. [[ 0. 160.412 175.943 146.258 161.496 153.246 176.332 187.989
  32. 154.607 158.557 191.902 169.875 217.684 156.298 173.856 190.599
  33. 163.531 181.961 168.51 164.54 ]
  34. [ 160.412 0. 166.886 158.739 158.515 147.873 169.187 173.981
  35. 163.087 160.295 182.02 156.383 188.341 153.391 168.673 180.751
  36. 167.932 166.455 160.655 161.822]
  37. [ 175.943 166.886 0. 167.077 164.816 161.296 168.714 177.013
  38. 163.068 166.991 172.948 171.929 189.907 172.781 164.755 175.02
  39. 168.046 172.84 168.08 167.016]
  40. [ 146.258 158.739 167.077 0. 155.465 148.282 160.904 169.061
  41. 148.348 155.199 180.96 160.351 195.063 156.985 160.851 183.418
  42. 164.041 165.082 157.416 154.755]
  43. [ 161.496 158.515 164.816 155.465 0. 154.639 164.003 170.73
  44. 156.604 152.294 169.252 159.988 197.838 157.912 157.305 178.355
  45. 163.494 173.434 158.26 161.744]
  46. [ 153.246 147.873 161.296 148.282 154.639 0. 161.793 165.443
  47. 151.397 158.839 178.333 158.767 186.079 153.018 163.183 178.698
  48. 157.537 163.989 159.029 155.431]
  49. [ 176.332 169.187 168.714 160.904 164.003 161.793 0. 172.765
  50. 163.67 167.845 178.111 169.176 182.466 167.645 168.541 175.116
  51. 167.959 171.654 166.381 165.681]
  52. [ 187.989 173.981 177.013 169.061 170.73 165.443 172.765 0.
  53. 170.286 167.088 187.02 176.369 191.557 172.805 172.313 180.38
  54. 180.421 171.042 176.654 173.121]
  55. [ 154.607 163.087 163.068 148.348 156.604 151.397 163.67 170.286
  56. 0. 160.348 175.101 168.498 189.155 155.985 160.247 178.55
  57. 166.085 165.312 168.241 157.492]
  58. [ 158.557 160.295 166.991 155.199 152.294 158.839 167.845 167.088
  59. 160.348 0. 182.146 161.396 195.085 151.365 166.636 185.611
  60. 166.88 173.368 161.851 155.537]
  61. [ 191.902 182.02 172.948 180.96 169.252 178.333 178.111 187.02
  62. 175.101 182.146 0. 168.496 191.178 177.467 173.483 181.039
  63. 172.191 176.788 185.603 183.357]
  64. [ 169.875 156.383 171.929 160.351 159.988 158.767 169.176 176.369
  65. 168.498 161.396 168.496 0. 184.536 161.824 171.511 177.894
  66. 159.918 171.931 165.413 164.026]
  67. [ 217.684 188.341 189.907 195.063 197.838 186.079 182.466 191.557
  68. 189.155 195.085 191.178 184.536 0. 185.806 191.329 183.333
  69. 186.019 184.804 189.403 187.686]
  70. [ 156.298 153.391 172.781 156.985 157.912 153.018 167.645 172.805
  71. 155.985 151.365 177.467 161.824 185.806 0. 161.317 188.554
  72. 163.822 165.693 164.074 154.489]
  73. [ 173.856 168.673 164.755 160.851 157.305 163.183 168.541 172.313
  74. 160.247 166.636 173.483 171.511 191.329 161.317 0. 170.372
  75. 166.665 173.681 168.927 172.025]
  76. [ 190.599 180.751 175.02 183.418 178.355 178.698 175.116 180.38
  77. 178.55 185.611 181.039 177.894 183.333 188.554 170.372 0.
  78. 174.773 174.838 180.529 182.602]
  79. [ 163.531 167.932 168.046 164.041 163.494 157.537 167.959 180.421
  80. 166.085 166.88 172.191 159.918 186.019 163.822 166.665 174.773
  81. 0. 175.418 169.261 163.998]
  82. [ 181.961 166.455 172.84 165.082 173.434 163.989 171.654 171.042
  83. 165.312 173.368 176.788 171.931 184.804 165.693 173.681 174.838
  84. 175.418 0. 172.396 169.087]
  85. [ 168.51 160.655 168.08 157.416 158.26 159.029 166.381 176.654
  86. 168.241 161.851 185.603 165.413 189.403 164.074 168.927 180.529
  87. 169.261 172.396 0. 164.294]
  88. [ 164.54 161.822 167.016 154.755 161.744 155.431 165.681 173.121
  89. 157.492 155.537 183.357 164.026 187.686 154.489 172.025 182.602
  90. 163.998 169.087 164.294 0. ]]
  91. 2019-03-12 03:02:53,743 : epoch 1 , step 0 loss0= 0.0000 loss1= 35.2766 loss2=-3461.6863 lr= 0.000100
  92. 2019-03-12 03:03:29,263 : epoch 1 , step 40 loss0= 0.0000 loss1= 34.1024 loss2=-3367.1699 lr= 0.000100
  93. 2019-03-12 03:04:05,376 : epoch 1 , step 80 loss0= 0.0000 loss1= 31.0294 loss2=-3688.9634 lr= 0.000100
  94. 2019-03-12 03:04:41,830 : epoch 1 , step 120 loss0= 0.0000 loss1= 31.4206 loss2=-3600.2966 lr= 0.000100
  95. 2019-03-12 03:05:18,498 : epoch 1 , step 160 loss0= 0.0000 loss1= 28.9128 loss2=-3305.6035 lr= 0.000100
  96. 2019-03-12 03:05:51,347 : epoch 2 , step 0 loss0= 0.0000 loss1= 29.5496 loss2=-3525.5298 lr= 0.000100
  97. 2019-03-12 03:06:27,572 : epoch 2 , step 40 loss0= 0.0000 loss1= 27.5751 loss2=-3361.8860 lr= 0.000100
  98. 2019-03-12 03:07:04,043 : epoch 2 , step 80 loss0= 0.0000 loss1= 25.6569 loss2=-3732.6653 lr= 0.000100
  99. 2019-03-12 03:07:40,562 : epoch 2 , step 120 loss0= 0.0000 loss1= 25.3179 loss2=-3500.6504 lr= 0.000100
  100. 2019-03-12 03:08:16,816 : epoch 2 , step 160 loss0= 0.0000 loss1= 26.3192 loss2=-3579.7615 lr= 0.000100
  101. 2019-03-12 03:08:49,733 : epoch 3 , step 0 loss0= 0.0000 loss1= 25.6026 loss2=-3640.8220 lr= 0.000100
  102. 2019-03-12 03:09:25,724 : epoch 3 , step 40 loss0= 0.0000 loss1= 26.8385 loss2=-3587.3308 lr= 0.000100
  103. 2019-03-12 03:10:01,465 : epoch 3 , step 80 loss0= 0.0000 loss1= 23.8628 loss2=-3557.9556 lr= 0.000100
  104. 2019-03-12 03:10:36,846 : epoch 3 , step 120 loss0= 0.0000 loss1= 23.6692 loss2=-3634.6274 lr= 0.000100
  105. 2019-03-12 03:11:12,822 : epoch 3 , step 160 loss0= 0.0000 loss1= 24.3740 loss2=-3352.5330 lr= 0.000100
  106. 2019-03-12 03:11:44,982 : epoch 4 , step 0 loss0= 0.0000 loss1= 20.7232 loss2=-3282.6567 lr= 0.000100
  107. 2019-03-12 03:12:21,087 : epoch 4 , step 40 loss0= 0.0000 loss1= 23.7721 loss2=-3376.8562 lr= 0.000100
  108. 2019-03-12 03:12:57,481 : epoch 4 , step 80 loss0= 0.0000 loss1= 23.4711 loss2=-3336.9089 lr= 0.000100
  109. 2019-03-12 03:13:33,784 : epoch 4 , step 120 loss0= 0.0000 loss1= 19.4345 loss2=-3191.9412 lr= 0.000100
  110. 2019-03-12 03:14:09,692 : epoch 4 , step 160 loss0= 0.0000 loss1= 20.6737 loss2=-3525.5774 lr= 0.000100
  111. 2019-03-12 03:14:42,082 : epoch 5 , step 0 loss0= 0.0000 loss1= 21.8199 loss2=-3255.7371 lr= 0.000100
  112. 2019-03-12 03:15:17,866 : epoch 5 , step 40 loss0= 0.0000 loss1= 19.6153 loss2=-3399.9304 lr= 0.000100
  113. 2019-03-12 03:15:54,024 : epoch 5 , step 80 loss0= 0.0000 loss1= 21.7115 loss2=-3797.0703 lr= 0.000100
  114. 2019-03-12 03:16:30,361 : epoch 5 , step 120 loss0= 0.0000 loss1= 20.6905 loss2=-3352.3899 lr= 0.000100
  115. 2019-03-12 03:17:06,192 : epoch 5 , step 160 loss0= 0.0000 loss1= 21.2335 loss2=-3218.9568 lr= 0.000100
  116. 2019-03-12 03:17:38,555 : epoch 6 , step 0 loss0= 0.0000 loss1= 19.1109 loss2=-3570.6326 lr= 0.000100
  117. 2019-03-12 03:18:14,089 : epoch 6 , step 40 loss0= 0.0000 loss1= 19.2832 loss2=-3451.6711 lr= 0.000100
  118. 2019-03-12 03:18:49,621 : epoch 6 , step 80 loss0= 0.0000 loss1= 18.6745 loss2=-3508.3171 lr= 0.000100
  119. 2019-03-12 03:19:25,121 : epoch 6 , step 120 loss0= 0.0000 loss1= 18.7970 loss2=-3520.8716 lr= 0.000100
  120. 2019-03-12 03:20:00,293 : epoch 6 , step 160 loss0= 0.0000 loss1= 20.9232 loss2=-3479.8677 lr= 0.000100
  121. 2019-03-12 03:20:32,810 : epoch 7 , step 0 loss0= 0.0000 loss1= 20.4413 loss2=-3481.8855 lr= 0.000100
  122. 2019-03-12 03:21:08,380 : epoch 7 , step 40 loss0= 0.0000 loss1= 20.6962 loss2=-3393.1719 lr= 0.000100
  123. 2019-03-12 03:21:43,954 : epoch 7 , step 80 loss0= 0.0000 loss1= 15.8022 loss2=-3483.8083 lr= 0.000100
  124. 2019-03-12 03:22:19,535 : epoch 7 , step 120 loss0= 0.0000 loss1= 19.2570 loss2=-3419.3103 lr= 0.000100
  125. 2019-03-12 03:22:55,216 : epoch 7 , step 160 loss0= 0.0000 loss1= 15.6673 loss2=-3310.2546 lr= 0.000100
  126. 2019-03-12 03:23:27,538 : epoch 8 , step 0 loss0= 0.0000 loss1= 18.0010 loss2=-3579.3730 lr= 0.000100
  127. 2019-03-12 03:24:03,206 : epoch 8 , step 40 loss0= 0.0000 loss1= 20.3525 loss2=-3431.9148 lr= 0.000100
  128. 2019-03-12 03:24:38,724 : epoch 8 , step 80 loss0= 0.0000 loss1= 18.9371 loss2=-3576.4146 lr= 0.000100
  129. 2019-03-12 03:25:14,087 : epoch 8 , step 120 loss0= 0.0000 loss1= 17.3000 loss2=-3649.2957 lr= 0.000100
  130. 2019-03-12 03:25:49,455 : epoch 8 , step 160 loss0= 0.0000 loss1= 16.8760 loss2=-3475.9204 lr= 0.000100
  131. 2019-03-12 03:26:21,673 : epoch 9 , step 0 loss0= 0.0000 loss1= 17.2965 loss2=-3595.1462 lr= 0.000100
  132. 2019-03-12 03:26:57,075 : epoch 9 , step 40 loss0= 0.0000 loss1= 16.1975 loss2=-3461.0974 lr= 0.000100
  133. 2019-03-12 03:27:32,177 : epoch 9 , step 80 loss0= 0.0000 loss1= 16.5496 loss2=-3509.3508 lr= 0.000100
  134. 2019-03-12 03:28:07,430 : epoch 9 , step 120 loss0= 0.0000 loss1= 17.1400 loss2=-3525.9910 lr= 0.000100
  135. 2019-03-12 03:28:43,039 : epoch 9 , step 160 loss0= 0.0000 loss1= 16.0297 loss2=-3432.5051 lr= 0.000100
  136. 2019-03-12 03:29:15,919 : epoch 10 , step 0 loss0= 0.0000 loss1= 15.2259 loss2=-3414.4980 lr= 0.000100
  137. 2019-03-12 03:29:51,783 : epoch 10 , step 40 loss0= 0.0000 loss1= 19.0649 loss2=-3377.3982 lr= 0.000100
  138. 2019-03-12 03:30:27,680 : epoch 10 , step 80 loss0= 0.0000 loss1= 15.7811 loss2=-3585.4680 lr= 0.000100
  139. 2019-03-12 03:31:04,149 : epoch 10 , step 120 loss0= 0.0000 loss1= 15.7019 loss2=-3539.0593 lr= 0.000100
  140. 2019-03-12 03:31:40,015 : epoch 10 , step 160 loss0= 0.0000 loss1= 16.0438 loss2=-3571.9360 lr= 0.000100
  141. 2019-03-12 03:32:12,562 : epoch 11 , step 0 loss0= 0.0000 loss1= 14.7233 loss2=-3451.3652 lr= 0.000100
  142. 2019-03-12 03:32:48,377 : epoch 11 , step 40 loss0= 0.0000 loss1= 14.7928 loss2=-3408.2664 lr= 0.000100
  143. 2019-03-12 03:33:24,906 : epoch 11 , step 80 loss0= 0.0000 loss1= 15.0818 loss2=-3543.1843 lr= 0.000100
  144. 2019-03-12 03:34:01,473 : epoch 11 , step 120 loss0= 0.0000 loss1= 15.4709 loss2=-3547.7109 lr= 0.000100
  145. 2019-03-12 03:34:38,029 : epoch 11 , step 160 loss0= 0.0000 loss1= 14.7736 loss2=-3591.7107 lr= 0.000100
  146. 2019-03-12 03:35:11,035 : epoch 12 , step 0 loss0= 0.0000 loss1= 14.4366 loss2=-3061.9417 lr= 0.000100
  147. 2019-03-12 03:35:47,038 : epoch 12 , step 40 loss0= 0.0000 loss1= 16.2096 loss2=-3515.0750 lr= 0.000100
  148. 2019-03-12 03:36:22,648 : epoch 12 , step 80 loss0= 0.0000 loss1= 16.1388 loss2=-3810.1582 lr= 0.000100
  149. 2019-03-12 03:36:58,036 : epoch 12 , step 120 loss0= 0.0000 loss1= 15.8703 loss2=-3475.3032 lr= 0.000100
  150. 2019-03-12 03:37:33,814 : epoch 12 , step 160 loss0= 0.0000 loss1= 13.8783 loss2=-3100.5876 lr= 0.000100
  151. 2019-03-12 03:38:06,134 : epoch 13 , step 0 loss0= 0.0000 loss1= 15.7922 loss2=-3501.5632 lr= 0.000100
  152. 2019-03-12 03:38:42,045 : epoch 13 , step 40 loss0= 0.0000 loss1= 15.1507 loss2=-3430.9705 lr= 0.000100
  153. 2019-03-12 03:39:18,022 : epoch 13 , step 80 loss0= 0.0000 loss1= 15.0710 loss2=-3497.3594 lr= 0.000100
  154. 2019-03-12 03:39:54,218 : epoch 13 , step 120 loss0= 0.0000 loss1= 15.8122 loss2=-3560.0376 lr= 0.000100
  155. 2019-03-12 03:40:30,257 : epoch 13 , step 160 loss0= 0.0000 loss1= 12.8172 loss2=-3752.5813 lr= 0.000100
  156. 2019-03-12 03:41:03,037 : epoch 14 , step 0 loss0= 0.0000 loss1= 14.6717 loss2=-3484.2942 lr= 0.000100
  157. 2019-03-12 03:41:38,834 : epoch 14 , step 40 loss0= 0.0000 loss1= 12.0747 loss2=-3371.4607 lr= 0.000100
  158. 2019-03-12 03:42:14,943 : epoch 14 , step 80 loss0= 0.0000 loss1= 14.7608 loss2=-3397.7075 lr= 0.000100
  159. 2019-03-12 03:42:50,501 : epoch 14 , step 120 loss0= 0.0000 loss1= 15.7137 loss2=-3463.1614 lr= 0.000100
  160. 2019-03-12 03:43:26,231 : epoch 14 , step 160 loss0= 0.0000 loss1= 14.8952 loss2=-3547.0403 lr= 0.000100
  161. 2019-03-12 03:43:58,648 : epoch 15 , step 0 loss0= 0.0000 loss1= 12.5994 loss2=-3414.8457 lr= 0.000100
  162. 2019-03-12 03:44:34,972 : epoch 15 , step 40 loss0= 0.0000 loss1= 13.9379 loss2=-3682.9216 lr= 0.000100
  163. 2019-03-12 03:45:10,614 : epoch 15 , step 80 loss0= 0.0000 loss1= 14.9471 loss2=-3612.2068 lr= 0.000100
  164. 2019-03-12 03:45:46,040 : epoch 15 , step 120 loss0= 0.0000 loss1= 11.4569 loss2=-3363.4290 lr= 0.000100
  165. 2019-03-12 03:46:21,390 : epoch 15 , step 160 loss0= 0.0000 loss1= 12.4909 loss2=-3525.2883 lr= 0.000100
  166. 2019-03-12 03:46:53,702 : epoch 16 , step 0 loss0= 0.0000 loss1= 13.1388 loss2=-3439.1931 lr= 0.000100
  167. 2019-03-12 03:47:29,887 : epoch 16 , step 40 loss0= 0.0000 loss1= 12.4985 loss2=-3366.8489 lr= 0.000100
  168. 2019-03-12 03:48:06,190 : epoch 16 , step 80 loss0= 0.0000 loss1= 12.4555 loss2=-3385.4707 lr= 0.000100
  169. 2019-03-12 03:48:42,294 : epoch 16 , step 120 loss0= 0.0000 loss1= 14.1434 loss2=-3414.2129 lr= 0.000100
  170. 2019-03-12 03:49:18,079 : epoch 16 , step 160 loss0= 0.0000 loss1= 15.4634 loss2=-3600.4197 lr= 0.000100
  171. 2019-03-12 03:49:50,371 : epoch 17 , step 0 loss0= 0.0000 loss1= 11.7380 loss2=-3396.6602 lr= 0.000100
  172. 2019-03-12 03:50:26,143 : epoch 17 , step 40 loss0= 0.0000 loss1= 13.8900 loss2=-3665.5005 lr= 0.000100
  173. 2019-03-12 03:51:01,601 : epoch 17 , step 80 loss0= 0.0000 loss1= 12.2193 loss2=-3833.1191 lr= 0.000100
  174. 2019-03-12 03:51:36,971 : epoch 17 , step 120 loss0= 0.0000 loss1= 13.4748 loss2=-3644.5024 lr= 0.000100
  175. 2019-03-12 03:52:12,378 : epoch 17 , step 160 loss0= 0.0000 loss1= 13.9178 loss2=-3408.4265 lr= 0.000100
  176. 2019-03-12 03:52:44,775 : epoch 18 , step 0 loss0= 0.0000 loss1= 11.6418 loss2=-3804.2329 lr= 0.000100
  177. 2019-03-12 03:53:20,394 : epoch 18 , step 40 loss0= 0.0000 loss1= 12.5832 loss2=-3566.5754 lr= 0.000100
  178. 2019-03-12 03:53:56,130 : epoch 18 , step 80 loss0= 0.0000 loss1= 12.8542 loss2=-3487.5017 lr= 0.000100
  179. 2019-03-12 03:54:32,283 : epoch 18 , step 120 loss0= 0.0000 loss1= 14.2028 loss2=-3388.2712 lr= 0.000100
  180. 2019-03-12 03:55:08,351 : epoch 18 , step 160 loss0= 0.0000 loss1= 14.2021 loss2=-3373.0767 lr= 0.000100
  181. 2019-03-12 03:55:41,193 : epoch 19 , step 0 loss0= 0.0000 loss1= 12.7286 loss2=-3460.1228 lr= 0.000100
  182. 2019-03-12 03:56:16,740 : epoch 19 , step 40 loss0= 0.0000 loss1= 14.6309 loss2=-3678.2954 lr= 0.000100
  183. 2019-03-12 03:56:52,551 : epoch 19 , step 80 loss0= 0.0000 loss1= 14.1342 loss2=-3692.0789 lr= 0.000100
  184. 2019-03-12 03:57:28,773 : epoch 19 , step 120 loss0= 0.0000 loss1= 11.7883 loss2=-3533.4500 lr= 0.000100
  185. 2019-03-12 03:58:05,446 : epoch 19 , step 160 loss0= 0.0000 loss1= 10.7413 loss2=-3378.7231 lr= 0.000100
  186. 2019-03-12 03:58:38,389 : epoch 20 , step 0 loss0= 0.0000 loss1= 15.6214 loss2=-3755.0938 lr= 0.000100
  187. 2019-03-12 03:59:14,089 : epoch 20 , step 40 loss0= 0.0000 loss1= 13.0030 loss2=-3349.0161 lr= 0.000100
  188. 2019-03-12 03:59:50,355 : epoch 20 , step 80 loss0= 0.0000 loss1= 11.9717 loss2=-3496.7676 lr= 0.000100
  189. 2019-03-12 04:00:26,015 : epoch 20 , step 120 loss0= 0.0000 loss1= 12.6866 loss2=-3749.0786 lr= 0.000100
  190. 2019-03-12 04:01:02,463 : epoch 20 , step 160 loss0= 0.0000 loss1= 11.2098 loss2=-3545.2390 lr= 0.000100
  191. 2019-03-12 04:01:33,623 : ## train ##
  192. 2019-03-12 04:06:28,972 : >> epoch 20 : average_loss= 0.7242, average_accuracy=75.277%, mean_weight_std= 0.1006, mean_normalized_std= 0.4809 mean_dist=1.267e+03
  193. 2019-03-12 04:06:28,981 : >> epoch 20 : median_loss= 0.7154, median_accuracy=75.640%, median_weight_std=0.07172, median_normalized_std= 0.0743 median_dist=1.27e+03
  194. 2019-03-12 04:06:28,982 : >> epoch 20 : ensemble accuracy: 76.92%
  195. 2019-03-12 04:06:28,982 : >> epoch 20 : loss histogram: (0.00,0.30, 18),(0.30,0.61, 0),(0.61,0.91, 4),(0.91,1.21, 0),(1.21,1.52, 18),
  196. 2019-03-12 04:06:28,982 : >> epoch 20 : accuracy histogram: (0.00,30.62, 18),(30.62,61.23, 0),(61.23,91.85, 4),(91.85,122.46, 0),(122.46,153.08, 18),
  197. 2019-03-12 04:06:28,988 : >> epoch 20 : weight std histogram: (0.0000,0.6006, 463145),(0.6006,1.2011, 3072),(1.2011,1.8017, 572),(1.8017,2.4022, 102),(2.4022,3.0028, 15),
  198. 2019-03-12 04:06:28,994 : >> epoch 20 : weight normalized std histogram: (0.0000,3585.8656, 466901),(3585.8656,7171.7312, 4),(7171.7312,10757.5969, 0),(10757.5969,14343.4625, 0),(14343.4625,17929.3281, 1),
  199. 2019-03-12 04:06:28,994 : >> epoch 20 : dist histogram: (1207.7101,1236.7641, 5),(1236.7641,1265.8181, 4),(1265.8181,1294.8721, 7),(1294.8721,1323.9261, 3),(1323.9261,1352.9801, 1),
  200. 2019-03-12 04:06:28,994 : ## eval ##
  201. 2019-03-12 04:11:22,498 : >> epoch 20 : average_loss= 0.6620, average_accuracy=77.474%, mean_weight_std= 0.1029, mean_normalized_std= 0.4376 mean_dist=1.267e+03
  202. 2019-03-12 04:11:22,510 : >> epoch 20 : median_loss= 0.6482, median_accuracy=78.020%, median_weight_std=0.07326, median_normalized_std= 0.0756 median_dist=1.266e+03
  203. 2019-03-12 04:11:22,510 : >> epoch 20 : ensemble accuracy: 79.45%
  204. 2019-03-12 04:11:22,510 : >> epoch 20 : loss histogram: (0.00,0.27, 18),(0.27,0.55, 0),(0.55,0.82, 4),(0.82,1.09, 0),(1.09,1.37, 18),
  205. 2019-03-12 04:11:22,510 : >> epoch 20 : accuracy histogram: (0.00,31.24, 18),(31.24,62.48, 0),(62.48,93.72, 4),(93.72,124.96, 0),(124.96,156.20, 18),
  206. 2019-03-12 04:11:22,516 : >> epoch 20 : weight std histogram: (0.0000,0.5855, 462799),(0.5855,1.1710, 3389),(1.1710,1.7565, 577),(1.7565,2.3420, 116),(2.3420,2.9275, 25),
  207. 2019-03-12 04:11:22,521 : >> epoch 20 : weight normalized std histogram: (0.0000,1519.7395, 466895),(1519.7395,3039.4789, 9),(3039.4789,4559.2184, 1),(4559.2184,6078.9578, 0),(6078.9578,7598.6973, 1),
  208. 2019-03-12 04:11:22,521 : >> epoch 20 : dist histogram: (1187.0311,1220.9885, 2),(1220.9885,1254.9459, 4),(1254.9459,1288.9033, 9),(1288.9033,1322.8607, 4),(1322.8607,1356.8181, 1),
  209. 2019-03-12 04:11:22,583 : >> epoch 20 : dist pair-wise
  210. 2019-03-12 04:11:22,594 :
  211. [[ 0. 146.664 147.785 160.733 149.426 151.695 136.462 152.007
  212. 142.351 146.265 138.742 149.577 148.817 135.765 143.324 159.087
  213. 187.743 145.413 156.991 156.701]
  214. [ 146.664 0. 154.454 171.063 152.199 147.88 127.103 150.799
  215. 126.14 143.07 153.754 150.12 128.077 143.482 132.792 156.72
  216. 192.69 141.834 145.965 164.474]
  217. [ 147.785 154.454 0. 142.778 129.505 136.985 140.567 140.782
  218. 147.73 135.838 136.917 156.437 169.264 132.852 160.454 148.115
  219. 152.158 145.527 134.291 141.653]
  220. [ 160.733 171.063 142.778 0. 147.756 144.093 158.826 147.715
  221. 164.289 142.326 148.037 153.292 189.872 152.432 174.602 147.023
  222. 146.779 156.804 164.229 141.664]
  223. [ 149.426 152.199 129.505 147.756 0. 140.249 136.285 146.411
  224. 143.188 135.376 143.754 145.229 162.085 136.645 162.073 147.32
  225. 157.63 141.181 142.761 148.669]
  226. [ 151.695 147.88 136.985 144.093 140.249 0. 144.113 132.277
  227. 140.971 143.97 146.909 138.954 168.056 142.84 161.739 138.696
  228. 155.378 156.093 136.963 152.134]
  229. [ 136.462 127.103 140.567 158.826 136.285 144.113 0. 144.947
  230. 121.67 137.744 134.35 143.121 133.865 131.573 132.291 136.413
  231. 169.552 126.716 127.206 152.006]
  232. [ 152.007 150.799 140.782 147.715 146.411 132.277 144.947 0.
  233. 144.025 145.012 151.366 152.392 174.564 144.526 159.698 150.903
  234. 160.398 141.799 143.519 147.702]
  235. [ 142.351 126.14 147.73 164.289 143.188 140.971 121.67 144.025
  236. 0. 146.503 141.457 139.81 132.904 132.784 138.389 151.9
  237. 184.902 139.214 139.415 163.758]
  238. [ 146.265 143.07 135.838 142.326 135.376 143.97 137.744 145.012
  239. 146.503 0. 138.963 137.971 150.689 147.995 143.783 150.574
  240. 164.47 133.922 143.122 139.802]
  241. [ 138.742 153.754 136.917 148.037 143.754 146.909 134.35 151.366
  242. 141.457 138.963 0. 130.584 150.185 135.893 148.137 146.72
  243. 163.673 134.384 146.512 142.636]
  244. [ 149.577 150.12 156.437 153.292 145.229 138.954 143.121 152.392
  245. 139.81 137.971 130.584 0. 155.654 143.208 153.737 136.744
  246. 159.479 139.793 153.007 147.1 ]
  247. [ 148.817 128.077 169.264 189.872 162.085 168.056 133.865 174.564
  248. 132.904 150.689 150.185 155.654 0. 149.726 117.268 174.537
  249. 214.554 144.033 146.436 182.159]
  250. [ 135.765 143.482 132.852 152.432 136.645 142.84 131.573 144.526
  251. 132.784 147.995 135.893 143.208 149.726 0. 147.096 152.46
  252. 162.387 137.425 137.86 167.434]
  253. [ 143.324 132.792 160.454 174.602 162.073 161.739 132.291 159.698
  254. 138.389 143.783 148.137 153.737 117.268 147.096 0. 174.719
  255. 201.012 137.293 146.82 176.432]
  256. [ 159.087 156.72 148.115 147.023 147.32 138.696 136.413 150.903
  257. 151.9 150.574 146.72 136.744 174.537 152.46 174.719 0.
  258. 149.627 146.611 154.291 142.837]
  259. [ 187.743 192.69 152.158 146.779 157.63 155.378 169.552 160.398
  260. 184.902 164.47 163.673 159.479 214.554 162.387 201.012 149.627
  261. 0. 178.146 166.795 147.87 ]
  262. [ 145.413 141.834 145.527 156.804 141.181 156.093 126.716 141.799
  263. 139.214 133.922 134.384 139.793 144.033 137.425 137.293 146.611
  264. 178.146 0. 147.017 154.163]
  265. [ 156.991 145.965 134.291 164.229 142.761 136.963 127.206 143.519
  266. 139.415 143.122 146.512 153.007 146.436 137.86 146.82 154.291
  267. 166.795 147.017 0. 157.911]
  268. [ 156.701 164.474 141.653 141.664 148.669 152.134 152.006 147.702
  269. 163.758 139.802 142.636 147.1 182.159 167.434 176.432 142.837
  270. 147.87 154.163 157.911 0. ]]
  271. 2019-03-12 04:11:24,062 : epoch 21 , step 0 loss0= 0.0000 loss1= 13.8662 loss2=-3254.4177 lr= 0.000100
  272. 2019-03-12 04:12:00,096 : epoch 21 , step 40 loss0= 0.0000 loss1= 11.9335 loss2=-3537.5396 lr= 0.000100
  273. 2019-03-12 04:12:35,871 : epoch 21 , step 80 loss0= 0.0000 loss1= 12.3769 loss2=-3477.5886 lr= 0.000100
  274. 2019-03-12 04:13:11,641 : epoch 21 , step 120 loss0= 0.0000 loss1= 16.2907 loss2=-3526.1531 lr= 0.000100
  275. 2019-03-12 04:13:47,457 : epoch 21 , step 160 loss0= 0.0000 loss1= 13.3292 loss2=-3504.5513 lr= 0.000100
  276. 2019-03-12 04:14:20,151 : epoch 22 , step 0 loss0= 0.0000 loss1= 12.2137 loss2=-3453.1248 lr= 0.000100
  277. 2019-03-12 04:14:55,653 : epoch 22 , step 40 loss0= 0.0000 loss1= 10.4723 loss2=-3485.2051 lr= 0.000100
  278. 2019-03-12 04:15:30,968 : epoch 22 , step 80 loss0= 0.0000 loss1= 11.0777 loss2=-3546.4360 lr= 0.000100
  279. 2019-03-12 04:16:06,366 : epoch 22 , step 120 loss0= 0.0000 loss1= 13.4381 loss2=-3790.7754 lr= 0.000100
  280. 2019-03-12 04:16:41,568 : epoch 22 , step 160 loss0= 0.0000 loss1= 14.2606 loss2=-3755.1294 lr= 0.000100
  281. 2019-03-12 04:17:14,200 : epoch 23 , step 0 loss0= 0.0000 loss1= 11.6692 loss2=-3557.4539 lr= 0.000100
  282. 2019-03-12 04:17:50,436 : epoch 23 , step 40 loss0= 0.0000 loss1= 12.4512 loss2=-3563.9888 lr= 0.000100
  283. 2019-03-12 04:18:26,918 : epoch 23 , step 80 loss0= 0.0000 loss1= 10.7153 loss2=-3273.4199 lr= 0.000100
  284. 2019-03-12 04:19:03,178 : epoch 23 , step 120 loss0= 0.0000 loss1= 11.6272 loss2=-3601.4329 lr= 0.000100
  285. 2019-03-12 04:19:39,054 : epoch 23 , step 160 loss0= 0.0000 loss1= 11.3831 loss2=-3313.5247 lr= 0.000100
  286. 2019-03-12 04:20:11,481 : epoch 24 , step 0 loss0= 0.0000 loss1= 11.8087 loss2=-3293.4961 lr= 0.000100
  287. 2019-03-12 04:20:47,228 : epoch 24 , step 40 loss0= 0.0000 loss1= 10.0740 loss2=-3395.3188 lr= 0.000100
  288. 2019-03-12 04:21:23,279 : epoch 24 , step 80 loss0= 0.0000 loss1= 11.3189 loss2=-3302.7349 lr= 0.000100
  289. 2019-03-12 04:21:59,095 : epoch 24 , step 120 loss0= 0.0000 loss1= 8.8885 loss2=-3350.2734 lr= 0.000100
  290. 2019-03-12 04:22:35,626 : epoch 24 , step 160 loss0= 0.0000 loss1= 10.7638 loss2=-3570.0312 lr= 0.000100
  291. 2019-03-12 04:23:08,523 : epoch 25 , step 0 loss0= 0.0000 loss1= 10.4262 loss2=-3614.6016 lr= 0.000100
  292. 2019-03-12 04:23:44,268 : epoch 25 , step 40 loss0= 0.0000 loss1= 11.4175 loss2=-3431.8457 lr= 0.000100
  293. 2019-03-12 04:24:20,265 : epoch 25 , step 80 loss0= 0.0000 loss1= 10.9789 loss2=-3374.5371 lr= 0.000100
  294. 2019-03-12 04:24:55,918 : epoch 25 , step 120 loss0= 0.0000 loss1= 9.5045 loss2=-3393.4011 lr= 0.000100
  295. 2019-03-12 04:25:31,586 : epoch 25 , step 160 loss0= 0.0000 loss1= 13.0221 loss2=-3488.3455 lr= 0.000100
  296. 2019-03-12 04:26:03,860 : epoch 26 , step 0 loss0= 0.0000 loss1= 9.3226 loss2=-3175.0962 lr= 0.000100
  297. 2019-03-12 04:26:39,351 : epoch 26 , step 40 loss0= 0.0000 loss1= 10.0237 loss2=-3473.0305 lr= 0.000100
  298. 2019-03-12 04:27:14,887 : epoch 26 , step 80 loss0= 0.0000 loss1= 8.7605 loss2=-3340.8833 lr= 0.000100
  299. 2019-03-12 04:27:50,331 : epoch 26 , step 120 loss0= 0.0000 loss1= 11.6076 loss2=-3693.2271 lr= 0.000100
  300. 2019-03-12 04:28:25,703 : epoch 26 , step 160 loss0= 0.0000 loss1= 10.7764 loss2=-3196.2427 lr= 0.000100
  301. 2019-03-12 04:28:58,177 : epoch 27 , step 0 loss0= 0.0000 loss1= 12.7701 loss2=-3356.6287 lr= 0.000100
  302. 2019-03-12 04:29:34,594 : epoch 27 , step 40 loss0= 0.0000 loss1= 11.4227 loss2=-3448.7148 lr= 0.000100
  303. 2019-03-12 04:30:11,031 : epoch 27 , step 80 loss0= 0.0000 loss1= 9.6591 loss2=-3609.6860 lr= 0.000100
  304. 2019-03-12 04:30:47,258 : epoch 27 , step 120 loss0= 0.0000 loss1= 13.6559 loss2=-3370.9031 lr= 0.000100
  305. 2019-03-12 04:31:23,307 : epoch 27 , step 160 loss0= 0.0000 loss1= 13.0359 loss2=-3563.9812 lr= 0.000100
  306. 2019-03-12 04:31:55,944 : epoch 28 , step 0 loss0= 0.0000 loss1= 11.8779 loss2=-3619.3103 lr= 0.000100
  307. 2019-03-12 04:32:32,106 : epoch 28 , step 40 loss0= 0.0000 loss1= 10.4074 loss2=-3451.3201 lr= 0.000100
  308. 2019-03-12 04:33:08,309 : epoch 28 , step 80 loss0= 0.0000 loss1= 11.1015 loss2=-3529.4473 lr= 0.000100
  309. 2019-03-12 04:33:44,478 : epoch 28 , step 120 loss0= 0.0000 loss1= 10.1154 loss2=-3478.9304 lr= 0.000100
  310. 2019-03-12 04:34:21,086 : epoch 28 , step 160 loss0= 0.0000 loss1= 9.8416 loss2=-3635.3442 lr= 0.000100
  311. 2019-03-12 04:34:53,701 : epoch 29 , step 0 loss0= 0.0000 loss1= 10.0989 loss2=-3635.7673 lr= 0.000100
  312. 2019-03-12 04:35:29,785 : epoch 29 , step 40 loss0= 0.0000 loss1= 9.1983 loss2=-3572.7427 lr= 0.000100
  313. 2019-03-12 04:36:05,520 : epoch 29 , step 80 loss0= 0.0000 loss1= 9.7848 loss2=-3297.9539 lr= 0.000100
  314. 2019-03-12 04:36:41,220 : epoch 29 , step 120 loss0= 0.0000 loss1= 13.0809 loss2=-3751.0896 lr= 0.000100
  315. 2019-03-12 04:37:16,785 : epoch 29 , step 160 loss0= 0.0000 loss1= 10.2474 loss2=-3414.6882 lr= 0.000100
  316. 2019-03-12 04:37:49,035 : epoch 30 , step 0 loss0= 0.0000 loss1= 9.7195 loss2=-3502.1541 lr= 0.000100
  317. 2019-03-12 04:38:25,188 : epoch 30 , step 40 loss0= 0.0000 loss1= 8.4735 loss2=-3472.8455 lr= 0.000100
  318. 2019-03-12 04:39:00,624 : epoch 30 , step 80 loss0= 0.0000 loss1= 10.2907 loss2=-3385.9607 lr= 0.000100
  319. 2019-03-12 04:39:35,931 : epoch 30 , step 120 loss0= 0.0000 loss1= 8.9638 loss2=-3248.0496 lr= 0.000100
  320. 2019-03-12 04:40:11,302 : epoch 30 , step 160 loss0= 0.0000 loss1= 9.1408 loss2=-3537.0759 lr= 0.000100
  321. 2019-03-12 04:40:43,946 : epoch 31 , step 0 loss0= 0.0000 loss1= 8.8777 loss2=-3467.2869 lr= 0.000100
  322. 2019-03-12 04:41:19,598 : epoch 31 , step 40 loss0= 0.0000 loss1= 9.8657 loss2=-3474.2156 lr= 0.000100
  323. 2019-03-12 04:41:55,131 : epoch 31 , step 80 loss0= 0.0000 loss1= 12.7388 loss2=-3668.1072 lr= 0.000100
  324. 2019-03-12 04:42:30,759 : epoch 31 , step 120 loss0= 0.0000 loss1= 10.5675 loss2=-3189.3933 lr= 0.000100
  325. 2019-03-12 04:43:06,387 : epoch 31 , step 160 loss0= 0.0000 loss1= 11.5484 loss2=-3637.9446 lr= 0.000100
  326. 2019-03-12 04:43:39,228 : epoch 32 , step 0 loss0= 0.0000 loss1= 7.9407 loss2=-3051.2231 lr= 0.000100
  327. 2019-03-12 04:44:15,106 : epoch 32 , step 40 loss0= 0.0000 loss1= 9.5918 loss2=-3417.9360 lr= 0.000100
  328. 2019-03-12 04:44:51,390 : epoch 32 , step 80 loss0= 0.0000 loss1= 9.4585 loss2=-3432.4453 lr= 0.000100
  329. 2019-03-12 04:45:27,408 : epoch 32 , step 120 loss0= 0.0000 loss1= 9.2126 loss2=-3449.5754 lr= 0.000100
  330. 2019-03-12 04:46:03,011 : epoch 32 , step 160 loss0= 0.0000 loss1= 11.5609 loss2=-3649.0161 lr= 0.000100
  331. 2019-03-12 04:46:35,282 : epoch 33 , step 0 loss0= 0.0000 loss1= 9.1207 loss2=-3455.9438 lr= 0.000100
  332. 2019-03-12 04:47:11,110 : epoch 33 , step 40 loss0= 0.0000 loss1= 10.5480 loss2=-3373.7563 lr= 0.000100
  333. 2019-03-12 04:47:47,335 : epoch 33 , step 80 loss0= 0.0000 loss1= 9.9581 loss2=-3411.6992 lr= 0.000100
  334. 2019-03-12 04:48:23,264 : epoch 33 , step 120 loss0= 0.0000 loss1= 8.0927 loss2=-3541.1926 lr= 0.000100
  335. 2019-03-12 04:48:59,509 : epoch 33 , step 160 loss0= 0.0000 loss1= 8.9608 loss2=-3738.1775 lr= 0.000100
  336. 2019-03-12 04:49:32,732 : epoch 34 , step 0 loss0= 0.0000 loss1= 10.7021 loss2=-3421.5845 lr= 0.000100
  337. 2019-03-12 04:50:08,721 : epoch 34 , step 40 loss0= 0.0000 loss1= 12.1542 loss2=-3575.2312 lr= 0.000100
  338. 2019-03-12 04:50:44,720 : epoch 34 , step 80 loss0= 0.0000 loss1= 10.3198 loss2=-3088.1123 lr= 0.000100
  339. 2019-03-12 04:51:21,213 : epoch 34 , step 120 loss0= 0.0000 loss1= 8.9341 loss2=-3651.1184 lr= 0.000100
  340. 2019-03-12 04:51:57,174 : epoch 34 , step 160 loss0= 0.0000 loss1= 9.5748 loss2=-3232.4333 lr= 0.000100
  341. 2019-03-12 04:52:30,122 : epoch 35 , step 0 loss0= 0.0000 loss1= 9.6417 loss2=-3467.1914 lr= 0.000100
  342. 2019-03-12 04:53:06,109 : epoch 35 , step 40 loss0= 0.0000 loss1= 9.7057 loss2=-3663.1335 lr= 0.000100
  343. 2019-03-12 04:53:41,672 : epoch 35 , step 80 loss0= 0.0000 loss1= 8.4211 loss2=-3740.9563 lr= 0.000100
  344. 2019-03-12 04:54:17,049 : epoch 35 , step 120 loss0= 0.0000 loss1= 9.6560 loss2=-3547.2283 lr= 0.000100
  345. 2019-03-12 04:54:52,817 : epoch 35 , step 160 loss0= 0.0000 loss1= 10.7710 loss2=-3537.2751 lr= 0.000100
  346. 2019-03-12 04:55:25,382 : epoch 36 , step 0 loss0= 0.0000 loss1= 8.6647 loss2=-3351.3005 lr= 0.000100
  347. 2019-03-12 04:56:01,345 : epoch 36 , step 40 loss0= 0.0000 loss1= 10.1832 loss2=-3450.3633 lr= 0.000100
  348. 2019-03-12 04:56:36,627 : epoch 36 , step 80 loss0= 0.0000 loss1= 9.3568 loss2=-3278.3391 lr= 0.000100
  349. 2019-03-12 04:57:12,054 : epoch 36 , step 120 loss0= 0.0000 loss1= 10.7855 loss2=-3171.4756 lr= 0.000100
  350. 2019-03-12 04:57:47,555 : epoch 36 , step 160 loss0= 0.0000 loss1= 8.8292 loss2=-3439.2930 lr= 0.000100
  351. 2019-03-12 04:58:19,941 : epoch 37 , step 0 loss0= 0.0000 loss1= 9.1013 loss2=-3468.9180 lr= 0.000100
  352. 2019-03-12 04:58:56,089 : epoch 37 , step 40 loss0= 0.0000 loss1= 10.0982 loss2=-3249.3481 lr= 0.000100
  353. 2019-03-12 04:59:31,729 : epoch 37 , step 80 loss0= 0.0000 loss1= 10.2570 loss2=-3553.5085 lr= 0.000100
  354. 2019-03-12 05:00:07,500 : epoch 37 , step 120 loss0= 0.0000 loss1= 8.2777 loss2=-3388.9070 lr= 0.000100
  355. 2019-03-12 05:00:43,193 : epoch 37 , step 160 loss0= 0.0000 loss1= 9.6659 loss2=-3517.8171 lr= 0.000100
  356. 2019-03-12 05:01:15,528 : epoch 38 , step 0 loss0= 0.0000 loss1= 9.0779 loss2=-3451.5103 lr= 0.000100
  357. 2019-03-12 05:01:51,349 : epoch 38 , step 40 loss0= 0.0000 loss1= 9.5859 loss2=-3395.7031 lr= 0.000100
  358. 2019-03-12 05:02:26,938 : epoch 38 , step 80 loss0= 0.0000 loss1= 9.9875 loss2=-3402.7278 lr= 0.000100
  359. 2019-03-12 05:03:02,089 : epoch 38 , step 120 loss0= 0.0000 loss1= 8.5353 loss2=-3303.9478 lr= 0.000100
  360. 2019-03-12 05:03:38,321 : epoch 38 , step 160 loss0= 0.0000 loss1= 9.7405 loss2=-3397.0388 lr= 0.000100
  361. 2019-03-12 05:04:10,877 : epoch 39 , step 0 loss0= 0.0000 loss1= 10.4380 loss2=-3452.9517 lr= 0.000100
  362. 2019-03-12 05:04:47,032 : epoch 39 , step 40 loss0= 0.0000 loss1= 8.4043 loss2=-3467.2947 lr= 0.000100
  363. 2019-03-12 05:05:23,000 : epoch 39 , step 80 loss0= 0.0000 loss1= 9.2927 loss2=-3432.0586 lr= 0.000100
  364. 2019-03-12 05:05:59,081 : epoch 39 , step 120 loss0= 0.0000 loss1= 7.5228 loss2=-3264.5986 lr= 0.000100
  365. 2019-03-12 05:06:35,259 : epoch 39 , step 160 loss0= 0.0000 loss1= 9.7886 loss2=-3711.2349 lr= 0.000100
  366. 2019-03-12 05:07:08,321 : epoch 40 , step 0 loss0= 0.0000 loss1= 8.9446 loss2=-3623.5974 lr= 0.000100
  367. 2019-03-12 05:07:44,071 : epoch 40 , step 40 loss0= 0.0000 loss1= 8.6256 loss2=-3421.9204 lr= 0.000100
  368. 2019-03-12 05:08:20,384 : epoch 40 , step 80 loss0= 0.0000 loss1= 8.0585 loss2=-3542.1985 lr= 0.000100
  369. 2019-03-12 05:08:56,761 : epoch 40 , step 120 loss0= 0.0000 loss1= 8.4160 loss2=-3620.6414 lr= 0.000100
  370. 2019-03-12 05:09:33,110 : epoch 40 , step 160 loss0= 0.0000 loss1= 12.2401 loss2=-3834.3992 lr= 0.000100
  371. 2019-03-12 05:10:04,513 : ## train ##
  372. 2019-03-12 05:14:58,893 : >> epoch 40 : average_loss= 0.6406, average_accuracy=78.672%, mean_weight_std=0.09885, mean_normalized_std= 0.7993 mean_dist=1.56e+03
  373. 2019-03-12 05:14:58,902 : >> epoch 40 : median_loss= 0.6504, median_accuracy=78.285%, median_weight_std=0.06912, median_normalized_std= 0.0581 median_dist=1.559e+03
  374. 2019-03-12 05:14:58,903 : >> epoch 40 : ensemble accuracy: 80.62%
  375. 2019-03-12 05:14:58,903 : >> epoch 40 : loss histogram: (0.00,0.27, 18),(0.27,0.54, 0),(0.54,0.82, 4),(0.82,1.09, 0),(1.09,1.36, 18),
  376. 2019-03-12 05:14:58,903 : >> epoch 40 : accuracy histogram: (0.00,31.86, 18),(31.86,63.71, 0),(63.71,95.57, 4),(95.57,127.42, 0),(127.42,159.28, 18),
  377. 2019-03-12 05:14:58,909 : >> epoch 40 : weight std histogram: (0.0000,0.8009, 464472),(0.8009,1.6018, 2051),(1.6018,2.4028, 302),(2.4028,3.2037, 69),(3.2037,4.0046, 12),
  378. 2019-03-12 05:14:58,915 : >> epoch 40 : weight normalized std histogram: (0.0000,22342.3734, 466903),(22342.3734,44684.7469, 0),(44684.7469,67027.1203, 2),(67027.1203,89369.4937, 0),(89369.4937,111711.8672, 1),
  379. 2019-03-12 05:14:58,915 : >> epoch 40 : dist histogram: (1477.2461,1510.7641, 3),(1510.7641,1544.2821, 2),(1544.2821,1577.8001, 10),(1577.8001,1611.3181, 2),(1611.3181,1644.8361, 3),
  380. 2019-03-12 05:14:58,915 : ## eval ##
  381. 2019-03-12 05:19:53,144 : >> epoch 40 : average_loss= 0.5591, average_accuracy=81.793%, mean_weight_std= 0.1025, mean_normalized_std= 0.5096 mean_dist=1.56e+03
  382. 2019-03-12 05:19:53,153 : >> epoch 40 : median_loss= 0.5535, median_accuracy=82.165%, median_weight_std=0.07129, median_normalized_std= 0.0602 median_dist=1.562e+03
  383. 2019-03-12 05:19:53,154 : >> epoch 40 : ensemble accuracy: 83.47%
  384. 2019-03-12 05:19:53,154 : >> epoch 40 : loss histogram: (0.00,0.24, 18),(0.24,0.48, 0),(0.48,0.72, 4),(0.72,0.96, 0),(0.96,1.20, 18),
  385. 2019-03-12 05:19:53,154 : >> epoch 40 : accuracy histogram: (0.00,33.07, 18),(33.07,66.13, 0),(66.13,99.20, 4),(99.20,132.26, 0),(132.26,165.33, 18),
  386. 2019-03-12 05:19:53,160 : >> epoch 40 : weight std histogram: (0.0000,0.7890, 464297),(0.7890,1.5780, 2159),(1.5780,2.3670, 358),(2.3670,3.1559, 74),(3.1559,3.9449, 18),
  387. 2019-03-12 05:19:53,165 : >> epoch 40 : weight normalized std histogram: (0.0000,10477.0891, 466903),(10477.0891,20954.1781, 2),(20954.1781,31431.2672, 0),(31431.2672,41908.3562, 0),(41908.3562,52385.4453, 1),
  388. 2019-03-12 05:19:53,166 : >> epoch 40 : dist histogram: (1474.1405,1521.6489, 4),(1521.6489,1569.1573, 7),(1569.1573,1616.6657, 8),(1616.6657,1664.1741, 0),(1664.1741,1711.6825, 1),
  389. 2019-03-12 05:19:53,228 : >> epoch 40 : dist pair-wise
  390. 2019-03-12 05:19:53,234 :
  391. [[ 0. 131.098 143.976 154.307 154.748 141.453 156.123 167.081
  392. 160.632 142.118 175.491 146.292 199.509 166.411 137.668 148.639
  393. 146.101 165.569 127.273 132.898]
  394. [ 131.098 0. 140.114 141.564 158.306 130.981 156.33 154.103
  395. 159.323 119.529 182.196 123.809 194.257 179.39 147.832 148.199
  396. 151.955 139.096 126.384 139.685]
  397. [ 143.976 140.114 0. 152.9 117.733 138.525 143.208 169.608
  398. 142.461 131.469 149.422 137.733 209.072 133.706 138.392 139.379
  399. 143.489 153.525 134.986 118.015]
  400. [ 154.307 141.564 152.9 0. 152.929 145.232 140.273 144.231
  401. 154.036 137.68 151.777 144.493 226.735 163.069 172.837 165.6
  402. 140.45 155.694 153.224 151.71 ]
  403. [ 154.748 158.306 117.733 152.929 0. 132.887 135.78 151.6
  404. 126.727 127.691 148.005 141.908 229.557 138.671 148.072 125.093
  405. 138.656 162.933 146.586 116.58 ]
  406. [ 141.453 130.981 138.525 145.232 132.887 0. 134.793 150.434
  407. 142.742 122.511 163.127 124.77 209.357 156.639 133.593 139.036
  408. 132.284 155.415 139.771 126.928]
  409. [ 156.123 156.33 143.208 140.273 135.78 134.793 0. 160.287
  410. 131.306 135.573 141.761 155.412 254.707 138.726 154.727 163.312
  411. 125.495 170.099 136.097 130.308]
  412. [ 167.081 154.103 169.608 144.231 151.6 150.434 160.287 0.
  413. 179.393 152.833 192.764 151.045 209.469 193.166 161.869 147.48
  414. 139.128 140.998 165.924 165.826]
  415. [ 160.632 159.323 142.461 154.036 126.727 142.742 131.306 179.393
  416. 0. 135.395 133.824 149.1 265.802 128.248 169.906 166.888
  417. 142.472 179.249 149.762 123.386]
  418. [ 142.118 119.529 131.469 137.68 127.691 122.511 135.573 152.833
  419. 135.395 0. 159.087 133.696 211.014 149.639 137.503 140.572
  420. 134.602 139.349 127.047 120.787]
  421. [ 175.491 182.196 149.422 151.777 148.005 163.127 141.761 192.764
  422. 133.824 159.087 0. 173.469 278.087 125.724 189.287 180.41
  423. 151.013 190.407 158.406 144.149]
  424. [ 146.292 123.809 137.733 144.493 141.908 124.77 155.412 151.045
  425. 149.1 133.696 173.469 0. 203.184 167.705 150.766 144.415
  426. 143.035 158.408 144.734 142.573]
  427. [ 199.509 194.257 209.072 226.735 229.557 209.357 254.707 209.469
  428. 265.802 211.014 278.087 203.184 0. 269.8 197.81 200.289
  429. 235.904 216.98 204.116 230.442]
  430. [ 166.411 179.39 133.706 163.069 138.671 156.639 138.726 193.166
  431. 128.248 149.639 125.724 167.705 269.8 0. 173.257 169.415
  432. 150.854 189.034 162.209 136.047]
  433. [ 137.668 147.832 138.392 172.837 148.072 133.593 154.727 161.869
  434. 169.906 137.503 189.287 150.766 197.81 173.257 0. 135.701
  435. 144.549 175.549 134.571 134.113]
  436. [ 148.639 148.199 139.379 165.6 125.093 139.036 163.312 147.48
  437. 166.888 140.572 180.41 144.415 200.289 169.415 135.701 0.
  438. 136.734 168.054 157.193 146.234]
  439. [ 146.101 151.955 143.489 140.45 138.656 132.284 125.495 139.128
  440. 142.472 134.602 151.013 143.035 235.904 150.854 144.549 136.734
  441. 0. 158.75 147.376 143.935]
  442. [ 165.569 139.096 153.525 155.694 162.933 155.415 170.099 140.998
  443. 179.249 139.349 190.407 158.408 216.98 189.034 175.549 168.054
  444. 158.75 0. 158.461 162.874]
  445. [ 127.273 126.384 134.986 153.224 146.586 139.771 136.097 165.924
  446. 149.762 127.047 158.406 144.734 204.116 162.209 134.571 157.193
  447. 147.376 158.461 0. 130.362]
  448. [ 132.898 139.685 118.015 151.71 116.58 126.928 130.308 165.826
  449. 123.386 120.787 144.149 142.573 230.442 136.047 134.113 146.234
  450. 143.935 162.874 130.362 0. ]]
  451. 2019-03-12 05:19:54,712 : epoch 41 , step 0 loss0= 0.0000 loss1= 8.9498 loss2=-3384.6648 lr= 0.000100
  452. 2019-03-12 05:20:30,014 : epoch 41 , step 40 loss0= 0.0000 loss1= 7.9239 loss2=-3323.5852 lr= 0.000100
  453. 2019-03-12 05:21:05,296 : epoch 41 , step 80 loss0= 0.0000 loss1= 8.3760 loss2=-3576.3904 lr= 0.000100
  454. 2019-03-12 05:21:40,507 : epoch 41 , step 120 loss0= 0.0000 loss1= 8.5735 loss2=-3251.1482 lr= 0.000100
  455. 2019-03-12 05:22:15,854 : epoch 41 , step 160 loss0= 0.0000 loss1= 10.2548 loss2=-3719.3110 lr= 0.000100
  456. 2019-03-12 05:22:48,693 : epoch 42 , step 0 loss0= 0.0000 loss1= 7.1106 loss2=-3205.6565 lr= 0.000100
  457. 2019-03-12 05:23:24,463 : epoch 42 , step 40 loss0= 0.0000 loss1= 8.1546 loss2=-3529.4888 lr= 0.000100
  458. 2019-03-12 05:24:00,217 : epoch 42 , step 80 loss0= 0.0000 loss1= 10.4078 loss2=-3315.3770 lr= 0.000100
  459. 2019-03-12 05:24:35,736 : epoch 42 , step 120 loss0= 0.0000 loss1= 6.7545 loss2=-3503.7231 lr= 0.000100
  460. 2019-03-12 05:25:11,429 : epoch 42 , step 160 loss0= 0.0000 loss1= 9.4219 loss2=-3474.1958 lr= 0.000100
  461. 2019-03-12 05:25:43,860 : epoch 43 , step 0 loss0= 0.0000 loss1= 7.4674 loss2=-3391.5789 lr= 0.000100
  462. 2019-03-12 05:26:20,005 : epoch 43 , step 40 loss0= 0.0000 loss1= 9.9018 loss2=-3503.2993 lr= 0.000100
  463. 2019-03-12 05:26:56,147 : epoch 43 , step 80 loss0= 0.0000 loss1= 9.0592 loss2=-3691.5911 lr= 0.000100
  464. 2019-03-12 05:27:32,434 : epoch 43 , step 120 loss0= 0.0000 loss1= 9.6874 loss2=-3896.9539 lr= 0.000100
  465. 2019-03-12 05:28:08,818 : epoch 43 , step 160 loss0= 0.0000 loss1= 9.7132 loss2=-3289.1633 lr= 0.000100
  466. 2019-03-12 05:28:41,498 : epoch 44 , step 0 loss0= 0.0000 loss1= 7.8739 loss2=-3476.8677 lr= 0.000100
  467. 2019-03-12 05:29:17,139 : epoch 44 , step 40 loss0= 0.0000 loss1= 8.4846 loss2=-3337.8833 lr= 0.000100
  468. 2019-03-12 05:29:52,407 : epoch 44 , step 80 loss0= 0.0000 loss1= 7.5407 loss2=-3419.2949 lr= 0.000100
  469. 2019-03-12 05:30:28,052 : epoch 44 , step 120 loss0= 0.0000 loss1= 9.1570 loss2=-3599.6902 lr= 0.000100
  470. 2019-03-12 05:31:03,803 : epoch 44 , step 160 loss0= 0.0000 loss1= 10.1413 loss2=-3517.5527 lr= 0.000100
  471. 2019-03-12 05:31:36,108 : epoch 45 , step 0 loss0= 0.0000 loss1= 9.0615 loss2=-3636.4109 lr= 0.000100
  472. 2019-03-12 05:32:12,049 : epoch 45 , step 40 loss0= 0.0000 loss1= 7.9890 loss2=-3551.1353 lr= 0.000100
  473. 2019-03-12 05:32:47,817 : epoch 45 , step 80 loss0= 0.0000 loss1= 8.4382 loss2=-3490.3516 lr= 0.000100
  474. 2019-03-12 05:33:23,519 : epoch 45 , step 120 loss0= 0.0000 loss1= 9.1095 loss2=-3715.6333 lr= 0.000100
  475. 2019-03-12 05:33:59,060 : epoch 45 , step 160 loss0= 0.0000 loss1= 9.3266 loss2=-3607.6333 lr= 0.000100
  476. 2019-03-12 05:34:31,393 : epoch 46 , step 0 loss0= 0.0000 loss1= 8.9430 loss2=-3366.3157 lr= 0.000100
  477. 2019-03-12 05:35:07,125 : epoch 46 , step 40 loss0= 0.0000 loss1= 8.1227 loss2=-3411.2791 lr= 0.000100
  478. 2019-03-12 05:35:42,681 : epoch 46 , step 80 loss0= 0.0000 loss1= 10.5573 loss2=-3383.8835 lr= 0.000100
  479. 2019-03-12 05:36:18,834 : epoch 46 , step 120 loss0= 0.0000 loss1= 10.0906 loss2=-3210.4568 lr= 0.000100
  480. 2019-03-12 05:36:55,593 : epoch 46 , step 160 loss0= 0.0000 loss1= 7.7141 loss2=-3632.7039 lr= 0.000100
  481. 2019-03-12 05:37:28,234 : epoch 47 , step 0 loss0= 0.0000 loss1= 7.9466 loss2=-3610.8965 lr= 0.000100
  482. 2019-03-12 05:38:04,650 : epoch 47 , step 40 loss0= 0.0000 loss1= 10.4595 loss2=-3720.2942 lr= 0.000100
  483. 2019-03-12 05:38:40,822 : epoch 47 , step 80 loss0= 0.0000 loss1= 9.0470 loss2=-3606.0063 lr= 0.000100
  484. 2019-03-12 05:39:16,640 : epoch 47 , step 120 loss0= 0.0000 loss1= 8.1837 loss2=-3253.9463 lr= 0.000100
  485. 2019-03-12 05:39:52,234 : epoch 47 , step 160 loss0= 0.0000 loss1= 10.5143 loss2=-3544.1274 lr= 0.000100
  486. 2019-03-12 05:40:25,096 : epoch 48 , step 0 loss0= 0.0000 loss1= 7.9472 loss2=-3308.2830 lr= 0.000100
  487. 2019-03-12 05:41:00,755 : epoch 48 , step 40 loss0= 0.0000 loss1= 7.7421 loss2=-3365.0173 lr= 0.000100
  488. 2019-03-12 05:41:36,537 : epoch 48 , step 80 loss0= 0.0000 loss1= 9.0691 loss2=-3551.6184 lr= 0.000100
  489. 2019-03-12 05:42:13,140 : epoch 48 , step 120 loss0= 0.0000 loss1= 7.6578 loss2=-3415.8943 lr= 0.000100
  490. 2019-03-12 05:42:48,834 : epoch 48 , step 160 loss0= 0.0000 loss1= 9.5540 loss2=-3341.7498 lr= 0.000100
  491. 2019-03-12 05:43:21,094 : epoch 49 , step 0 loss0= 0.0000 loss1= 10.0973 loss2=-3510.8071 lr= 0.000100
  492. 2019-03-12 05:43:56,464 : epoch 49 , step 40 loss0= 0.0000 loss1= 10.3234 loss2=-3506.1724 lr= 0.000100
  493. 2019-03-12 05:44:32,099 : epoch 49 , step 80 loss0= 0.0000 loss1= 8.6388 loss2=-3458.6990 lr= 0.000100
  494. 2019-03-12 05:45:07,826 : epoch 49 , step 120 loss0= 0.0000 loss1= 8.1710 loss2=-3548.7180 lr= 0.000100
  495. 2019-03-12 05:45:43,229 : epoch 49 , step 160 loss0= 0.0000 loss1= 7.8104 loss2=-3284.7915 lr= 0.000100
  496. 2019-03-12 05:46:15,791 : epoch 50 , step 0 loss0= 0.0000 loss1= 8.5292 loss2=-3270.2349 lr= 0.000100
  497. 2019-03-12 05:46:51,561 : epoch 50 , step 40 loss0= 0.0000 loss1= 7.2434 loss2=-3391.8669 lr= 0.000100
  498. 2019-03-12 05:47:27,029 : epoch 50 , step 80 loss0= 0.0000 loss1= 7.3699 loss2=-3428.3445 lr= 0.000100
  499. 2019-03-12 05:48:02,330 : epoch 50 , step 120 loss0= 0.0000 loss1= 7.4986 loss2=-3397.9609 lr= 0.000100
  500. 2019-03-12 05:48:37,700 : epoch 50 , step 160 loss0= 0.0000 loss1= 8.2830 loss2=-3521.1133 lr= 0.000100
  501. 2019-03-12 05:49:09,796 : epoch 51 , step 0 loss0= 0.0000 loss1= 8.3379 loss2=-3515.1086 lr= 0.000100
  502. 2019-03-12 05:49:45,466 : epoch 51 , step 40 loss0= 0.0000 loss1= 9.6010 loss2=-3670.5037 lr= 0.000100
  503. 2019-03-12 05:50:21,186 : epoch 51 , step 80 loss0= 0.0000 loss1= 7.2310 loss2=-3266.7654 lr= 0.000100
  504. 2019-03-12 05:50:56,834 : epoch 51 , step 120 loss0= 0.0000 loss1= 8.0730 loss2=-3516.3794 lr= 0.000100
  505. 2019-03-12 05:51:32,515 : epoch 51 , step 160 loss0= 0.0000 loss1= 8.1795 loss2=-3450.0005 lr= 0.000100
  506. 2019-03-12 05:52:04,813 : epoch 52 , step 0 loss0= 0.0000 loss1= 8.3440 loss2=-3543.3386 lr= 0.000100
  507. 2019-03-12 05:52:40,682 : epoch 52 , step 40 loss0= 0.0000 loss1= 6.7501 loss2=-3407.8494 lr= 0.000100
  508. 2019-03-12 05:53:16,527 : epoch 52 , step 80 loss0= 0.0000 loss1= 8.6805 loss2=-3458.0605 lr= 0.000100
  509. 2019-03-12 05:53:52,254 : epoch 52 , step 120 loss0= 0.0000 loss1= 8.0190 loss2=-3529.3484 lr= 0.000100
  510. 2019-03-12 05:54:27,892 : epoch 52 , step 160 loss0= 0.0000 loss1= 7.7510 loss2=-3473.9343 lr= 0.000100
  511. 2019-03-12 05:55:00,451 : epoch 53 , step 0 loss0= 0.0000 loss1= 8.5586 loss2=-3496.6060 lr= 0.000100
  512. 2019-03-12 05:55:36,103 : epoch 53 , step 40 loss0= 0.0000 loss1= 7.7991 loss2=-3773.3755 lr= 0.000100
  513. 2019-03-12 05:56:11,979 : epoch 53 , step 80 loss0= 0.0000 loss1= 8.3187 loss2=-3672.6431 lr= 0.000100
  514. 2019-03-12 05:56:47,987 : epoch 53 , step 120 loss0= 0.0000 loss1= 9.6167 loss2=-3492.2478 lr= 0.000100
  515. 2019-03-12 05:57:24,071 : epoch 53 , step 160 loss0= 0.0000 loss1= 8.0223 loss2=-3539.9746 lr= 0.000100
  516. 2019-03-12 05:57:56,798 : epoch 54 , step 0 loss0= 0.0000 loss1= 9.9185 loss2=-3576.7605 lr= 0.000100
  517. 2019-03-12 05:58:32,580 : epoch 54 , step 40 loss0= 0.0000 loss1= 10.5780 loss2=-3594.1313 lr= 0.000100
  518. 2019-03-12 05:59:07,820 : epoch 54 , step 80 loss0= 0.0000 loss1= 5.9476 loss2=-3392.3704 lr= 0.000100
  519. 2019-03-12 05:59:43,097 : epoch 54 , step 120 loss0= 0.0000 loss1= 8.4459 loss2=-3651.9255 lr= 0.000100
  520. 2019-03-12 06:00:18,938 : epoch 54 , step 160 loss0= 0.0000 loss1= 6.1656 loss2=-3283.3853 lr= 0.000100
  521. 2019-03-12 06:00:51,647 : epoch 55 , step 0 loss0= 0.0000 loss1= 8.5085 loss2=-3828.2017 lr= 0.000100
  522. 2019-03-12 06:01:27,247 : epoch 55 , step 40 loss0= 0.0000 loss1= 9.3593 loss2=-3664.4470 lr= 0.000100
  523. 2019-03-12 06:02:02,514 : epoch 55 , step 80 loss0= 0.0000 loss1= 7.4830 loss2=-3397.7075 lr= 0.000100
  524. 2019-03-12 06:02:38,005 : epoch 55 , step 120 loss0= 0.0000 loss1= 8.8421 loss2=-3663.3313 lr= 0.000100
  525. 2019-03-12 06:03:13,480 : epoch 55 , step 160 loss0= 0.0000 loss1= 8.8842 loss2=-3619.3469 lr= 0.000100
  526. 2019-03-12 06:03:45,886 : epoch 56 , step 0 loss0= 0.0000 loss1= 7.3876 loss2=-3418.4263 lr= 0.000100
  527. 2019-03-12 06:04:22,004 : epoch 56 , step 40 loss0= 0.0000 loss1= 6.1111 loss2=-3609.2974 lr= 0.000100
  528. 2019-03-12 06:04:58,044 : epoch 56 , step 80 loss0= 0.0000 loss1= 8.7025 loss2=-3280.4265 lr= 0.000100
  529. 2019-03-12 06:05:33,819 : epoch 56 , step 120 loss0= 0.0000 loss1= 7.7541 loss2=-3488.5063 lr= 0.000100
  530. 2019-03-12 06:06:10,579 : epoch 56 , step 160 loss0= 0.0000 loss1= 9.4754 loss2=-3624.0020 lr= 0.000100
  531. 2019-03-12 06:06:43,681 : epoch 57 , step 0 loss0= 0.0000 loss1= 7.5782 loss2=-3605.5974 lr= 0.000100
  532. 2019-03-12 06:07:19,647 : epoch 57 , step 40 loss0= 0.0000 loss1= 7.8273 loss2=-3272.1570 lr= 0.000100
  533. 2019-03-12 06:07:55,999 : epoch 57 , step 80 loss0= 0.0000 loss1= 8.9646 loss2=-3311.7078 lr= 0.000100
  534. 2019-03-12 06:08:31,919 : epoch 57 , step 120 loss0= 0.0000 loss1= 7.9443 loss2=-3287.2546 lr= 0.000100
  535. 2019-03-12 06:09:07,797 : epoch 57 , step 160 loss0= 0.0000 loss1= 6.5501 loss2=-3449.2434 lr= 0.000100
  536. 2019-03-12 06:09:40,597 : epoch 58 , step 0 loss0= 0.0000 loss1= 6.2132 loss2=-3617.2954 lr= 0.000100
  537. 2019-03-12 06:10:16,274 : epoch 58 , step 40 loss0= 0.0000 loss1= 7.5966 loss2=-3534.4688 lr= 0.000100
  538. 2019-03-12 06:10:52,437 : epoch 58 , step 80 loss0= 0.0000 loss1= 7.0840 loss2=-3328.8438 lr= 0.000100
  539. 2019-03-12 06:11:28,457 : epoch 58 , step 120 loss0= 0.0000 loss1= 7.8605 loss2=-3471.6614 lr= 0.000100
  540. 2019-03-12 06:12:04,460 : epoch 58 , step 160 loss0= 0.0000 loss1= 7.6984 loss2=-3456.7720 lr= 0.000100
  541. 2019-03-12 06:12:37,205 : epoch 59 , step 0 loss0= 0.0000 loss1= 9.1434 loss2=-3492.8508 lr= 0.000100
  542. 2019-03-12 06:13:13,369 : epoch 59 , step 40 loss0= 0.0000 loss1= 9.3115 loss2=-3543.6443 lr= 0.000100
  543. 2019-03-12 06:13:49,330 : epoch 59 , step 80 loss0= 0.0000 loss1= 7.4121 loss2=-3628.0532 lr= 0.000100
  544. 2019-03-12 06:14:25,273 : epoch 59 , step 120 loss0= 0.0000 loss1= 7.5533 loss2=-3434.8931 lr= 0.000100
  545. 2019-03-12 06:15:00,764 : epoch 59 , step 160 loss0= 0.0000 loss1= 8.7007 loss2=-3269.5896 lr= 0.000100
  546. 2019-03-12 06:15:33,138 : epoch 60 , step 0 loss0= 0.0000 loss1= 9.3277 loss2=-3569.0281 lr= 0.000100
  547. 2019-03-12 06:16:09,027 : epoch 60 , step 40 loss0= 0.0000 loss1= 7.7260 loss2=-3659.7759 lr= 0.000100
  548. 2019-03-12 06:16:44,944 : epoch 60 , step 80 loss0= 0.0000 loss1= 7.9212 loss2=-3496.3071 lr= 0.000100
  549. 2019-03-12 06:17:20,738 : epoch 60 , step 120 loss0= 0.0000 loss1= 6.9135 loss2=-3452.7732 lr= 0.000100
  550. 2019-03-12 06:17:56,462 : epoch 60 , step 160 loss0= 0.0000 loss1= 6.7070 loss2=-3449.6086 lr= 0.000100
  551. 2019-03-12 06:18:27,537 : ## train ##
  552. 2019-03-12 06:23:22,602 : >> epoch 60 : average_loss= 0.6097, average_accuracy=80.595%, mean_weight_std= 0.1045, mean_normalized_std= 0.2871 mean_dist=1.758e+03
  553. 2019-03-12 06:23:22,613 : >> epoch 60 : median_loss= 0.6067, median_accuracy=81.175%, median_weight_std=0.07164, median_normalized_std= 0.0538 median_dist=1.745e+03
  554. 2019-03-12 06:23:22,613 : >> epoch 60 : ensemble accuracy: 82.70%
  555. 2019-03-12 06:23:22,614 : >> epoch 60 : loss histogram: (0.00,0.26, 18),(0.26,0.51, 0),(0.51,0.77, 4),(0.77,1.03, 0),(1.03,1.29, 18),
  556. 2019-03-12 06:23:22,614 : >> epoch 60 : accuracy histogram: (0.00,32.60, 18),(32.60,65.20, 0),(65.20,97.81, 4),(97.81,130.41, 0),(130.41,163.01, 18),
  557. 2019-03-12 06:23:22,620 : >> epoch 60 : weight std histogram: (0.0000,0.9352, 464640),(0.9352,1.8705, 1853),(1.8705,2.8057, 318),(2.8057,3.7410, 67),(3.7410,4.6762, 28),
  558. 2019-03-12 06:23:22,625 : >> epoch 60 : weight normalized std histogram: (0.0000,1021.1472, 466897),(1021.1472,2042.2943, 8),(2042.2943,3063.4415, 0),(3063.4415,4084.5887, 0),(4084.5887,5105.7358, 1),
  559. 2019-03-12 06:23:22,625 : >> epoch 60 : dist histogram: (1692.7515,1728.8304, 7),(1728.8304,1764.9093, 6),(1764.9093,1800.9883, 3),(1800.9883,1837.0672, 2),(1837.0672,1873.1461, 2),
  560. 2019-03-12 06:23:22,625 : ## eval ##
  561. 2019-03-12 06:27:54,614 : >> epoch 60 : average_loss= 0.5339, average_accuracy=83.416%, mean_weight_std= 0.11, mean_normalized_std= 0.4532 mean_dist=1.757e+03
  562. 2019-03-12 06:27:54,624 : >> epoch 60 : median_loss= 0.5265, median_accuracy=83.355%, median_weight_std=0.07596, median_normalized_std= 0.0573 median_dist=1.764e+03
  563. 2019-03-12 06:27:54,624 : >> epoch 60 : ensemble accuracy: 85.36%
  564. 2019-03-12 06:27:54,625 : >> epoch 60 : loss histogram: (0.00,0.22, 18),(0.22,0.45, 0),(0.45,0.67, 4),(0.67,0.89, 0),(0.89,1.11, 18),
  565. 2019-03-12 06:27:54,625 : >> epoch 60 : accuracy histogram: (0.00,33.63, 18),(33.63,67.27, 0),(67.27,100.90, 4),(100.90,134.54, 0),(134.54,168.17, 18),
  566. 2019-03-12 06:27:54,630 : >> epoch 60 : weight std histogram: (0.0000,0.8889, 464464),(0.8889,1.7778, 2012),(1.7778,2.6668, 341),(2.6668,3.5557, 69),(3.5557,4.4446, 20),
  567. 2019-03-12 06:27:54,635 : >> epoch 60 : weight normalized std histogram: (0.0000,7318.5086, 466903),(7318.5086,14637.0172, 1),(14637.0172,21955.5258, 1),(21955.5258,29274.0344, 0),(29274.0344,36592.5430, 1),
  568. 2019-03-12 06:27:54,635 : >> epoch 60 : dist histogram: (1650.2699,1691.5497, 3),(1691.5497,1732.8295, 4),(1732.8295,1774.1093, 5),(1774.1093,1815.3890, 5),(1815.3890,1856.6688, 3),
  569. 2019-03-12 06:27:54,689 : >> epoch 60 : dist pair-wise
  570. 2019-03-12 06:27:54,694 :
  571. [[ 0. 145.475 149.174 141.884 155.644 157.917 136.09 154.458
  572. 157.428 152.828 189.649 167.93 149.269 177.438 155.207 151.951
  573. 140.082 130.895 197.994 179.955]
  574. [ 145.475 0. 125.326 132.462 135.443 176.416 108.608 131.918
  575. 147.59 162.092 189.497 147.023 149.874 149.511 155.007 126.872
  576. 140.202 133.641 199.082 180.691]
  577. [ 149.174 125.326 0. 143.347 158.315 143.271 139.828 161.397
  578. 135.04 155.15 203.34 160.056 140.384 143.21 143.919 125.731
  579. 146.813 140.123 208.346 164.991]
  580. [ 141.884 132.462 143.347 0. 142.461 164.566 150.854 156.361
  581. 153.601 161.671 212.39 144.047 145.919 163.397 148.299 134.261
  582. 136.323 135.281 211.472 183.435]
  583. [ 155.644 135.443 158.315 142.461 0. 194.708 148.354 166.966
  584. 164.246 173.531 220.36 179.74 157.137 195.725 145.316 145.184
  585. 153.142 139.269 240.774 192.726]
  586. [ 157.917 176.416 143.271 164.566 194.708 0. 181.218 200.976
  587. 140.382 158.003 247.621 206.928 151.118 191.705 155.67 142.479
  588. 181.171 185.042 232.48 131.527]
  589. [ 136.09 108.608 139.828 150.854 148.354 181.218 0. 125.221
  590. 150.079 175.005 175.429 153.329 156.699 153.701 170.466 138.853
  591. 133.857 133.338 180.896 188.015]
  592. [ 154.458 131.918 161.397 156.361 166.966 200.976 125.221 0.
  593. 171.27 200.668 184.344 160.818 190.395 160.134 179.541 175.483
  594. 134.654 137.504 179.522 206.893]
  595. [ 157.428 147.59 135.04 153.601 164.246 140.382 150.079 171.27
  596. 0. 153.293 246.785 204.731 128.638 173.351 139.556 135.181
  597. 158.517 165.503 243.92 136.328]
  598. [ 152.828 162.092 155.15 161.671 173.531 158.003 175.005 200.668
  599. 153.293 0. 226.071 219.154 164.825 207.079 143.088 157.069
  600. 187.477 157.203 248.775 159.823]
  601. [ 189.649 189.497 203.34 212.39 220.36 247.621 175.429 184.344
  602. 246.785 226.071 0. 187.026 251.584 187.399 247.987 224.034
  603. 189.479 182.245 152.554 277.723]
  604. [ 167.93 147.023 160.056 144.047 179.74 206.928 153.329 160.818
  605. 204.731 219.154 187.026 0. 185.578 153.925 198.445 163.388
  606. 146.742 152.834 187.094 238.456]
  607. [ 149.269 149.874 140.384 145.919 157.137 151.118 156.699 190.395
  608. 128.638 164.825 251.584 185.578 0. 179.664 133.747 126.527
  609. 151.031 166.461 237.784 163.692]
  610. [ 177.438 149.511 143.21 163.397 195.725 191.705 153.701 160.134
  611. 173.351 207.079 187.399 153.925 179.664 0. 200.289 164.811
  612. 146.4 180.77 167.101 227.977]
  613. [ 155.207 155.007 143.919 148.299 145.316 155.67 170.466 179.541
  614. 139.556 143.088 247.987 198.445 133.747 200.289 0. 134.339
  615. 157.544 146.472 261.721 150.463]
  616. [ 151.951 126.872 125.731 134.261 145.184 142.479 138.853 175.483
  617. 135.181 157.069 224.034 163.388 126.527 164.811 134.339 0.
  618. 142.608 151.871 225.103 157.319]
  619. [ 140.082 140.202 146.813 136.323 153.142 181.171 133.857 134.654
  620. 158.517 187.477 189.479 146.742 151.031 146.4 157.544 142.608
  621. 0. 150.619 194.899 202.403]
  622. [ 130.895 133.641 140.123 135.281 139.269 185.042 133.338 137.504
  623. 165.503 157.203 182.245 152.834 166.461 180.77 146.472 151.871
  624. 150.619 0. 202.963 191.074]
  625. [ 197.994 199.082 208.346 211.472 240.774 232.48 180.896 179.522
  626. 243.92 248.775 152.554 187.094 237.784 167.101 261.721 225.103
  627. 194.899 202.963 0. 273.726]
  628. [ 179.955 180.691 164.991 183.435 192.726 131.527 188.015 206.893
  629. 136.328 159.823 277.723 238.456 163.692 227.977 150.463 157.319
  630. 202.403 191.074 273.726 0. ]]
  631. 2019-03-12 06:27:56,055 : epoch 61 , step 0 loss0= 0.0000 loss1= 7.7059 loss2=-3700.2297 lr= 0.000100
  632. 2019-03-12 06:28:29,560 : epoch 61 , step 40 loss0= 0.0000 loss1= 7.8863 loss2=-3767.3481 lr= 0.000100
  633. 2019-03-12 06:29:03,148 : epoch 61 , step 80 loss0= 0.0000 loss1= 6.0377 loss2=-3407.9529 lr= 0.000100
  634. 2019-03-12 06:29:36,688 : epoch 61 , step 120 loss0= 0.0000 loss1= 8.0305 loss2=-3459.8032 lr= 0.000100
  635. 2019-03-12 06:30:09,881 : epoch 61 , step 160 loss0= 0.0000 loss1= 7.3523 loss2=-3102.9580 lr= 0.000100
  636. 2019-03-12 06:30:40,067 : epoch 62 , step 0 loss0= 0.0000 loss1= 5.9755 loss2=-3382.8406 lr= 0.000100
  637. 2019-03-12 06:31:14,106 : epoch 62 , step 40 loss0= 0.0000 loss1= 6.9230 loss2=-3357.2114 lr= 0.000100
  638. 2019-03-12 06:31:47,705 : epoch 62 , step 80 loss0= 0.0000 loss1= 7.8880 loss2=-3313.1411 lr= 0.000100
  639. 2019-03-12 06:32:21,561 : epoch 62 , step 120 loss0= 0.0000 loss1= 6.3863 loss2=-3268.3311 lr= 0.000100
  640. 2019-03-12 06:32:55,658 : epoch 62 , step 160 loss0= 0.0000 loss1= 7.3394 loss2=-3444.5410 lr= 0.000100
  641. 2019-03-12 06:33:26,489 : epoch 63 , step 0 loss0= 0.0000 loss1= 6.8401 loss2=-3427.4243 lr= 0.000100
  642. 2019-03-12 06:34:00,034 : epoch 63 , step 40 loss0= 0.0000 loss1= 6.6197 loss2=-3269.9568 lr= 0.000100
  643. 2019-03-12 06:34:33,763 : epoch 63 , step 80 loss0= 0.0000 loss1= 7.1956 loss2=-3245.2324 lr= 0.000100
  644. 2019-03-12 06:35:07,176 : epoch 63 , step 120 loss0= 0.0000 loss1= 7.1947 loss2=-3425.6411 lr= 0.000100
  645. 2019-03-12 06:35:40,350 : epoch 63 , step 160 loss0= 0.0000 loss1= 6.9116 loss2=-3572.2759 lr= 0.000100
  646. 2019-03-12 06:36:11,042 : epoch 64 , step 0 loss0= 0.0000 loss1= 9.0753 loss2=-3381.8669 lr= 0.000100
  647. 2019-03-12 06:36:44,532 : epoch 64 , step 40 loss0= 0.0000 loss1= 6.9434 loss2=-3400.7078 lr= 0.000100
  648. 2019-03-12 06:37:18,226 : epoch 64 , step 80 loss0= 0.0000 loss1= 6.4282 loss2=-3231.2800 lr= 0.000100
  649. 2019-03-12 06:37:51,844 : epoch 64 , step 120 loss0= 0.0000 loss1= 6.4747 loss2=-3438.8079 lr= 0.000100
  650. 2019-03-12 06:38:25,677 : epoch 64 , step 160 loss0= 0.0000 loss1= 6.4643 loss2=-3371.9204 lr= 0.000100
  651. 2019-03-12 06:38:55,973 : epoch 65 , step 0 loss0= 0.0000 loss1= 9.0611 loss2=-3192.5767 lr= 0.000100
  652. 2019-03-12 06:39:29,601 : epoch 65 , step 40 loss0= 0.0000 loss1= 6.1613 loss2=-3334.4680 lr= 0.000100
  653. 2019-03-12 06:40:02,936 : epoch 65 , step 80 loss0= 0.0000 loss1= 6.9440 loss2=-3443.2883 lr= 0.000100
  654. 2019-03-12 06:40:36,280 : epoch 65 , step 120 loss0= 0.0000 loss1= 6.6501 loss2=-3280.8521 lr= 0.000100
  655. 2019-03-12 06:41:09,663 : epoch 65 , step 160 loss0= 0.0000 loss1= 6.8198 loss2=-3614.4041 lr= 0.000100
  656. 2019-03-12 06:41:40,051 : epoch 66 , step 0 loss0= 0.0000 loss1= 5.2217 loss2=-3363.6770 lr= 0.000100
  657. 2019-03-12 06:42:13,335 : epoch 66 , step 40 loss0= 0.0000 loss1= 6.0943 loss2=-3344.3835 lr= 0.000100
  658. 2019-03-12 06:42:46,569 : epoch 66 , step 80 loss0= 0.0000 loss1= 6.6848 loss2=-3440.2056 lr= 0.000100
  659. 2019-03-12 06:43:19,846 : epoch 66 , step 120 loss0= 0.0000 loss1= 6.4084 loss2=-3110.0264 lr= 0.000100
  660. 2019-03-12 06:43:53,108 : epoch 66 , step 160 loss0= 0.0000 loss1= 9.4778 loss2=-3430.6106 lr= 0.000100
  661. 2019-03-12 06:44:23,330 : epoch 67 , step 0 loss0= 0.0000 loss1= 6.1856 loss2=-3502.6282 lr= 0.000100
  662. 2019-03-12 06:44:57,062 : epoch 67 , step 40 loss0= 0.0000 loss1= 6.8087 loss2=-3281.3821 lr= 0.000100
  663. 2019-03-12 06:45:30,860 : epoch 67 , step 80 loss0= 0.0000 loss1= 8.8435 loss2=-3548.4458 lr= 0.000100
  664. 2019-03-12 06:46:04,853 : epoch 67 , step 120 loss0= 0.0000 loss1= 6.0955 loss2=-3317.6658 lr= 0.000100
  665. 2019-03-12 06:46:38,077 : epoch 67 , step 160 loss0= 0.0000 loss1= 7.4396 loss2=-3398.5579 lr= 0.000100
  666. 2019-03-12 06:47:08,230 : epoch 68 , step 0 loss0= 0.0000 loss1= 4.5321 loss2=-3456.9797 lr= 0.000100
  667. 2019-03-12 06:47:41,528 : epoch 68 , step 40 loss0= 0.0000 loss1= 6.1036 loss2=-3254.7683 lr= 0.000100
  668. 2019-03-12 06:48:15,705 : epoch 68 , step 80 loss0= 0.0000 loss1= 7.6592 loss2=-3676.8103 lr= 0.000100
  669. 2019-03-12 06:48:50,008 : epoch 68 , step 120 loss0= 0.0000 loss1= 7.2238 loss2=-3646.7969 lr= 0.000100
  670. 2019-03-12 06:49:24,343 : epoch 68 , step 160 loss0= 0.0000 loss1= 8.9361 loss2=-3708.3279 lr= 0.000100
  671. 2019-03-12 06:49:55,172 : epoch 69 , step 0 loss0= 0.0000 loss1= 6.7360 loss2=-3541.9294 lr= 0.000100
  672. 2019-03-12 06:50:29,331 : epoch 69 , step 40 loss0= 0.0000 loss1= 7.3396 loss2=-3457.2759 lr= 0.000100
  673. 2019-03-12 06:51:03,191 : epoch 69 , step 80 loss0= 0.0000 loss1= 6.0045 loss2=-3394.1829 lr= 0.000100
  674. 2019-03-12 06:51:36,644 : epoch 69 , step 120 loss0= 0.0000 loss1= 7.5896 loss2=-3532.2446 lr= 0.000100
  675. 2019-03-12 06:52:09,841 : epoch 69 , step 160 loss0= 0.0000 loss1= 7.2463 loss2=-3506.4500 lr= 0.000100
  676. 2019-03-12 06:52:40,773 : epoch 70 , step 0 loss0= 0.0000 loss1= 6.6674 loss2=-3530.0891 lr= 0.000100
  677. 2019-03-12 06:53:14,382 : epoch 70 , step 40 loss0= 0.0000 loss1= 7.6173 loss2=-3312.4255 lr= 0.000100
  678. 2019-03-12 06:53:48,064 : epoch 70 , step 80 loss0= 0.0000 loss1= 6.0692 loss2=-3144.5352 lr= 0.000100
  679. 2019-03-12 06:54:21,855 : epoch 70 , step 120 loss0= 0.0000 loss1= 6.8354 loss2=-3467.0000 lr= 0.000100
  680. 2019-03-12 06:54:55,262 : epoch 70 , step 160 loss0= 0.0000 loss1= 8.0722 loss2=-3799.6892 lr= 0.000100
  681. 2019-03-12 06:55:25,507 : epoch 71 , step 0 loss0= 0.0000 loss1= 6.9031 loss2=-3616.5005 lr= 0.000100
  682. 2019-03-12 06:55:59,492 : epoch 71 , step 40 loss0= 0.0000 loss1= 7.1279 loss2=-3679.9070 lr= 0.000100
  683. 2019-03-12 06:56:33,502 : epoch 71 , step 80 loss0= 0.0000 loss1= 6.9118 loss2=-3464.8689 lr= 0.000100
  684. 2019-03-12 06:57:07,383 : epoch 71 , step 120 loss0= 0.0000 loss1= 7.8244 loss2=-3655.5669 lr= 0.000100
  685. 2019-03-12 06:57:41,057 : epoch 71 , step 160 loss0= 0.0000 loss1= 6.2196 loss2=-3446.7375 lr= 0.000100
  686. 2019-03-12 06:58:11,375 : epoch 72 , step 0 loss0= 0.0000 loss1= 7.5343 loss2=-3190.2134 lr= 0.000100
  687. 2019-03-12 06:58:45,007 : epoch 72 , step 40 loss0= 0.0000 loss1= 6.7354 loss2=-3162.8125 lr= 0.000100
  688. 2019-03-12 06:59:18,356 : epoch 72 , step 80 loss0= 0.0000 loss1= 6.5590 loss2=-3493.2012 lr= 0.000100
  689. 2019-03-12 06:59:51,644 : epoch 72 , step 120 loss0= 0.0000 loss1= 6.8781 loss2=-3821.0430 lr= 0.000100
  690. 2019-03-12 07:00:24,943 : epoch 72 , step 160 loss0= 0.0000 loss1= 6.3469 loss2=-3588.9980 lr= 0.000100
  691. 2019-03-12 07:00:55,297 : epoch 73 , step 0 loss0= 0.0000 loss1= 6.5295 loss2=-3689.7888 lr= 0.000100
  692. 2019-03-12 07:01:28,626 : epoch 73 , step 40 loss0= 0.0000 loss1= 7.1205 loss2=-3223.3767 lr= 0.000100
  693. 2019-03-12 07:02:02,115 : epoch 73 , step 80 loss0= 0.0000 loss1= 6.1828 loss2=-3501.9548 lr= 0.000100
  694. 2019-03-12 07:02:35,725 : epoch 73 , step 120 loss0= 0.0000 loss1= 5.5000 loss2=-3364.5740 lr= 0.000100
  695. 2019-03-12 07:03:09,084 : epoch 73 , step 160 loss0= 0.0000 loss1= 7.0202 loss2=-3494.8298 lr= 0.000100
  696. 2019-03-12 07:03:39,724 : epoch 74 , step 0 loss0= 0.0000 loss1= 6.2162 loss2=-3461.2380 lr= 0.000100
  697. 2019-03-12 07:04:13,320 : epoch 74 , step 40 loss0= 0.0000 loss1= 6.0540 loss2=-3621.4915 lr= 0.000100
  698. 2019-03-12 07:04:46,919 : epoch 74 , step 80 loss0= 0.0000 loss1= 6.5094 loss2=-3530.9148 lr= 0.000100
  699. 2019-03-12 07:05:20,468 : epoch 74 , step 120 loss0= 0.0000 loss1= 6.3569 loss2=-3452.0845 lr= 0.000100
  700. 2019-03-12 07:05:54,024 : epoch 74 , step 160 loss0= 0.0000 loss1= 6.1190 loss2=-3607.5623 lr= 0.000100
  701. 2019-03-12 07:06:24,470 : epoch 75 , step 0 loss0= 0.0000 loss1= 6.6509 loss2=-3250.0513 lr= 0.000100
  702. 2019-03-12 07:06:58,714 : epoch 75 , step 40 loss0= 0.0000 loss1= 5.6944 loss2=-3469.3103 lr= 0.000100
  703. 2019-03-12 07:07:32,909 : epoch 75 , step 80 loss0= 0.0000 loss1= 4.7480 loss2=-3325.2227 lr= 0.000100
  704. 2019-03-12 07:08:07,182 : epoch 75 , step 120 loss0= 0.0000 loss1= 7.4375 loss2=-3443.7751 lr= 0.000100
  705. 2019-03-12 07:08:41,583 : epoch 75 , step 160 loss0= 0.0000 loss1= 5.6281 loss2=-3283.4412 lr= 0.000100
  706. 2019-03-12 07:09:12,611 : epoch 76 , step 0 loss0= 0.0000 loss1= 5.6643 loss2=-3239.4966 lr= 0.000100
  707. 2019-03-12 07:09:46,511 : epoch 76 , step 40 loss0= 0.0000 loss1= 6.1321 loss2=-3647.0994 lr= 0.000100
  708. 2019-03-12 07:10:20,852 : epoch 76 , step 80 loss0= 0.0000 loss1= 7.1268 loss2=-3512.0388 lr= 0.000100
  709. 2019-03-12 07:10:55,296 : epoch 76 , step 120 loss0= 0.0000 loss1= 7.4163 loss2=-3324.0173 lr= 0.000100
  710. 2019-03-12 07:11:29,651 : epoch 76 , step 160 loss0= 0.0000 loss1= 7.7101 loss2=-3249.8298 lr= 0.000100
  711. 2019-03-12 07:12:00,258 : epoch 77 , step 0 loss0= 0.0000 loss1= 5.8303 loss2=-3322.6157 lr= 0.000100
  712. 2019-03-12 07:12:33,882 : epoch 77 , step 40 loss0= 0.0000 loss1= 6.7270 loss2=-3572.3132 lr= 0.000100
  713. 2019-03-12 07:13:07,655 : epoch 77 , step 80 loss0= 0.0000 loss1= 6.7788 loss2=-3503.5493 lr= 0.000100
  714. 2019-03-12 07:13:41,762 : epoch 77 , step 120 loss0= 0.0000 loss1= 6.5738 loss2=-3708.0813 lr= 0.000100
  715. 2019-03-12 07:14:15,356 : epoch 77 , step 160 loss0= 0.0000 loss1= 7.6350 loss2=-3424.7942 lr= 0.000100
  716. 2019-03-12 07:14:45,592 : epoch 78 , step 0 loss0= 0.0000 loss1= 5.4506 loss2=-3232.2346 lr= 0.000100
  717. 2019-03-12 07:15:19,716 : epoch 78 , step 40 loss0= 0.0000 loss1= 7.0561 loss2=-3447.8052 lr= 0.000100
  718. 2019-03-12 07:15:53,905 : epoch 78 , step 80 loss0= 0.0000 loss1= 7.6951 loss2=-3653.6704 lr= 0.000100
  719. 2019-03-12 07:16:28,132 : epoch 78 , step 120 loss0= 0.0000 loss1= 7.5066 loss2=-3597.9304 lr= 0.000100
  720. 2019-03-12 07:17:01,614 : epoch 78 , step 160 loss0= 0.0000 loss1= 6.9362 loss2=-3706.2576 lr= 0.000100
  721. 2019-03-12 07:17:32,238 : epoch 79 , step 0 loss0= 0.0000 loss1= 4.7741 loss2=-3443.8813 lr= 0.000100
  722. 2019-03-12 07:18:06,233 : epoch 79 , step 40 loss0= 0.0000 loss1= 4.6645 loss2=-3539.8867 lr= 0.000100
  723. 2019-03-12 07:18:40,674 : epoch 79 , step 80 loss0= 0.0000 loss1= 8.8934 loss2=-3512.5742 lr= 0.000100
  724. 2019-03-12 07:19:14,507 : epoch 79 , step 120 loss0= 0.0000 loss1= 9.3560 loss2=-3909.8403 lr= 0.000100
  725. 2019-03-12 07:19:48,250 : epoch 79 , step 160 loss0= 0.0000 loss1= 6.0890 loss2=-3624.6907 lr= 0.000100
  726. 2019-03-12 07:20:18,826 : epoch 80 , step 0 loss0= 0.0000 loss1= 4.7422 loss2=-3386.4727 lr= 0.000100
  727. 2019-03-12 07:20:52,306 : epoch 80 , step 40 loss0= 0.0000 loss1= 8.7447 loss2=-3603.8757 lr= 0.000100
  728. 2019-03-12 07:21:25,556 : epoch 80 , step 80 loss0= 0.0000 loss1= 7.5240 loss2=-3442.8599 lr= 0.000100
  729. 2019-03-12 07:21:58,880 : epoch 80 , step 120 loss0= 0.0000 loss1= 5.4754 loss2=-3478.5078 lr= 0.000100
  730. 2019-03-12 07:22:32,737 : epoch 80 , step 160 loss0= 0.0000 loss1= 4.8933 loss2=-3402.3425 lr= 0.000100
  731. 2019-03-12 07:23:01,834 : ## train ##
  732. 2019-03-12 07:27:18,097 : >> epoch 80 : average_loss= 0.5750, average_accuracy=82.136%, mean_weight_std= 0.1277, mean_normalized_std= 0.3124 mean_dist=1.925e+03
  733. 2019-03-12 07:27:18,107 : >> epoch 80 : median_loss= 0.5649, median_accuracy=82.145%, median_weight_std=0.08766, median_normalized_std= 0.0594 median_dist=1.935e+03
  734. 2019-03-12 07:27:18,107 : >> epoch 80 : ensemble accuracy: 84.11%
  735. 2019-03-12 07:27:18,107 : >> epoch 80 : loss histogram: (0.00,0.24, 18),(0.24,0.48, 0),(0.48,0.71, 4),(0.71,0.95, 0),(0.95,1.19, 18),
  736. 2019-03-12 07:27:18,107 : >> epoch 80 : accuracy histogram: (0.00,33.15, 18),(33.15,66.30, 0),(66.30,99.44, 4),(99.44,132.59, 0),(132.59,165.74, 18),
  737. 2019-03-12 07:27:18,113 : >> epoch 80 : weight std histogram: (0.0000,1.3958, 465704),(1.3958,2.7916, 1074),(2.7916,4.1873, 116),(4.1873,5.5831, 10),(5.5831,6.9789, 2),
  738. 2019-03-12 07:27:18,117 : >> epoch 80 : weight normalized std histogram: (0.0000,1458.8303, 466902),(1458.8303,2917.6605, 2),(2917.6605,4376.4908, 1),(4376.4908,5835.3211, 0),(5835.3211,7294.1514, 1),
  739. 2019-03-12 07:27:18,118 : >> epoch 80 : dist histogram: (1809.0199,1854.4355, 5),(1854.4355,1899.8511, 2),(1899.8511,1945.2667, 3),(1945.2667,1990.6823, 6),(1990.6823,2036.0979, 4),
  740. 2019-03-12 07:27:18,118 : ## eval ##
  741. 2019-03-12 07:31:32,747 : >> epoch 80 : average_loss= 0.5152, average_accuracy=84.522%, mean_weight_std= 0.1208, mean_normalized_std= 0.2937 mean_dist=1.925e+03
  742. 2019-03-12 07:31:32,756 : >> epoch 80 : median_loss= 0.5120, median_accuracy=84.685%, median_weight_std=0.08199, median_normalized_std= 0.0562 median_dist=1.943e+03
  743. 2019-03-12 07:31:32,756 : >> epoch 80 : ensemble accuracy: 86.22%
  744. 2019-03-12 07:31:32,757 : >> epoch 80 : loss histogram: (0.00,0.21, 18),(0.21,0.42, 0),(0.42,0.64, 4),(0.64,0.85, 0),(0.85,1.06, 18),
  745. 2019-03-12 07:31:32,757 : >> epoch 80 : accuracy histogram: (0.00,34.14, 18),(34.14,68.29, 0),(68.29,102.43, 4),(102.43,136.58, 0),(136.58,170.72, 18),
  746. 2019-03-12 07:31:32,762 : >> epoch 80 : weight std histogram: (0.0000,1.1441, 465014),(1.1441,2.2883, 1608),(2.2883,3.4324, 221),(3.4324,4.5766, 56),(4.5766,5.7207, 7),
  747. 2019-03-12 07:31:32,767 : >> epoch 80 : weight normalized std histogram: (0.0000,594.5559, 466889),(594.5559,1189.1117, 10),(1189.1117,1783.6676, 1),(1783.6676,2378.2234, 4),(2378.2234,2972.7793, 2),
  748. 2019-03-12 07:31:32,767 : >> epoch 80 : dist histogram: (1823.8857,1876.5710, 6),(1876.5710,1929.2563, 1),(1929.2563,1981.9417, 11),(1981.9417,2034.6270, 1),(2034.6270,2087.3123, 1),
  749. 2019-03-12 07:31:32,822 : >> epoch 80 : dist pair-wise
  750. 2019-03-12 07:31:32,828 :
  751. [[ 0. 229.914 220.029 180.175 218.173 156.637 154.739 169.378
  752. 161.888 186.142 212.634 188.165 160.675 308.867 179.256 127.467
  753. 214.703 179.223 211.729 116.699]
  754. [ 229.914 0. 176.084 180.242 152.08 236.699 197.669 182.954
  755. 173.2 149.641 148.68 162.481 179.153 249.989 176.636 223.875
  756. 252.55 199.81 234.994 211.083]
  757. [ 220.029 176.084 0. 194.079 164.559 229.469 185.942 181.584
  758. 175.227 191.665 168.207 141.927 150.858 242.461 188.949 219.092
  759. 215.506 213.821 214.226 202.662]
  760. [ 180.175 180.242 194.079 0. 184.487 228.092 151.091 208.847
  761. 200.441 141.012 207.611 173.86 189.738 323.879 215.572 169.055
  762. 272.12 220.697 268.751 158.768]
  763. [ 218.173 152.08 164.559 184.487 0. 255.302 179.776 188.872
  764. 187.282 164.3 170.198 159.777 166.608 251.699 189.52 216.801
  765. 244.869 204.69 232.526 193.695]
  766. [ 156.637 236.699 229.469 228.092 255.302 0. 183.272 157.223
  767. 170.849 219.89 201.526 191.384 187.322 268.57 163.976 195.396
  768. 183.323 180.651 173.486 179.294]
  769. [ 154.739 197.669 185.942 151.091 179.776 183.272 0. 177.676
  770. 170.238 156.74 178.931 158.959 164.401 294.782 169.047 148.541
  771. 237.71 183.559 232.36 129.639]
  772. [ 169.378 182.954 181.584 208.847 188.872 157.223 177.676 0.
  773. 134.022 194.461 149.859 139.006 145.741 210.177 129.086 188.324
  774. 165.031 124.147 160.788 166.907]
  775. [ 161.888 173.2 175.227 200.441 187.282 170.849 170.238 134.022
  776. 0. 181.073 151.258 148.815 121.296 232.289 118.029 179.749
  777. 189.27 152.763 165.917 157.309]
  778. [ 186.142 149.641 191.665 141.012 164.3 219.89 156.74 194.461
  779. 181.073 0. 182.689 169.05 178.081 313.54 187.83 180.339
  780. 265.3 213.346 256.206 169.129]
  781. [ 212.634 148.68 168.207 207.611 170.198 201.526 178.931 149.859
  782. 151.258 182.689 0. 141.643 155.395 196.766 127.573 203.03
  783. 201.629 161.071 184.902 186.204]
  784. [ 188.165 162.481 141.927 173.86 159.777 191.384 158.959 139.006
  785. 148.815 169.05 141.643 0. 143.423 230.279 155.893 195.794
  786. 202.247 162.567 193.036 174.128]
  787. [ 160.675 179.153 150.858 189.738 166.608 187.322 164.401 145.741
  788. 121.296 178.081 155.395 143.423 0. 239.41 138.074 178.688
  789. 199.374 170.905 181.766 152.789]
  790. [ 308.867 249.989 242.461 323.879 251.699 268.57 294.782 210.177
  791. 232.289 313.54 196.766 230.279 239.41 0. 209.646 309.608
  792. 197.8 211.241 171.654 294.755]
  793. [ 179.256 176.636 188.949 215.572 189.52 163.976 169.047 129.086
  794. 118.029 187.83 127.573 155.893 138.074 209.646 0. 198.682
  795. 188.308 138.435 162.934 173.075]
  796. [ 127.467 223.875 219.092 169.055 216.801 195.396 148.541 188.324
  797. 179.749 180.339 203.03 195.794 178.688 309.608 198.682 0.
  798. 235.838 187.499 234.95 111.163]
  799. [ 214.703 252.55 215.506 272.12 244.869 183.323 237.71 165.031
  800. 189.27 265.3 201.629 202.247 199.374 197.8 188.308 235.838
  801. 0. 176.131 115.28 227.35 ]
  802. [ 179.223 199.81 213.821 220.697 204.69 180.651 183.559 124.147
  803. 152.763 213.346 161.071 162.567 170.905 211.241 138.435 187.499
  804. 176.131 0. 167.214 166.356]
  805. [ 211.729 234.994 214.226 268.751 232.526 173.486 232.36 160.788
  806. 165.917 256.206 184.902 193.036 181.766 171.654 162.934 234.95
  807. 115.28 167.214 0. 220.183]
  808. [ 116.699 211.083 202.662 158.768 193.695 179.294 129.639 166.907
  809. 157.309 169.129 186.204 174.128 152.789 294.755 173.075 111.163
  810. 227.35 166.356 220.183 0. ]]
  811. 2019-03-12 07:31:34,167 : epoch 81 , step 0 loss0= 0.0000 loss1= 6.3105 loss2=-3633.8735 lr= 0.000100
  812. 2019-03-12 07:32:07,438 : epoch 81 , step 40 loss0= 0.0000 loss1= 6.9374 loss2=-3548.0349 lr= 0.000100
  813. 2019-03-12 07:32:40,649 : epoch 81 , step 80 loss0= 0.0000 loss1= 5.7533 loss2=-3218.0369 lr= 0.000100
  814. 2019-03-12 07:33:14,453 : epoch 81 , step 120 loss0= 0.0000 loss1= 6.1038 loss2=-3586.1899 lr= 0.000100
  815. 2019-03-12 07:33:48,400 : epoch 81 , step 160 loss0= 0.0000 loss1= 6.0667 loss2=-3367.1697 lr= 0.000100
  816. 2019-03-12 07:34:18,932 : epoch 82 , step 0 loss0= 0.0000 loss1= 6.1202 loss2=-3483.8149 lr= 0.000100
  817. 2019-03-12 07:34:52,196 : epoch 82 , step 40 loss0= 0.0000 loss1= 7.0504 loss2=-3523.6223 lr= 0.000100
  818. 2019-03-12 07:35:25,518 : epoch 82 , step 80 loss0= 0.0000 loss1= 6.2388 loss2=-3794.2263 lr= 0.000100
  819. 2019-03-12 07:35:58,763 : epoch 82 , step 120 loss0= 0.0000 loss1= 5.6550 loss2=-3463.3262 lr= 0.000100
  820. 2019-03-12 07:36:31,993 : epoch 82 , step 160 loss0= 0.0000 loss1= 7.6972 loss2=-3490.5442 lr= 0.000100
  821. 2019-03-12 07:37:02,172 : epoch 83 , step 0 loss0= 0.0000 loss1= 5.7911 loss2=-3453.7400 lr= 0.000100
  822. 2019-03-12 07:37:35,805 : epoch 83 , step 40 loss0= 0.0000 loss1= 4.9905 loss2=-3423.8325 lr= 0.000100
  823. 2019-03-12 07:38:09,285 : epoch 83 , step 80 loss0= 0.0000 loss1= 6.9980 loss2=-3575.7043 lr= 0.000100
  824. 2019-03-12 07:38:42,863 : epoch 83 , step 120 loss0= 0.0000 loss1= 6.2929 loss2=-3688.6189 lr= 0.000100
  825. 2019-03-12 07:39:16,560 : epoch 83 , step 160 loss0= 0.0000 loss1= 4.6570 loss2=-3198.1748 lr= 0.000100
  826. 2019-03-12 07:39:47,171 : epoch 84 , step 0 loss0= 0.0000 loss1= 5.0898 loss2=-3260.0420 lr= 0.000100
  827. 2019-03-12 07:40:21,071 : epoch 84 , step 40 loss0= 0.0000 loss1= 6.3344 loss2=-3553.6626 lr= 0.000100
  828. 2019-03-12 07:40:54,656 : epoch 84 , step 80 loss0= 0.0000 loss1= 6.3616 loss2=-3526.6177 lr= 0.000100
  829. 2019-03-12 07:41:28,699 : epoch 84 , step 120 loss0= 0.0000 loss1= 7.0399 loss2=-3689.7341 lr= 0.000100
  830. 2019-03-12 07:42:02,514 : epoch 84 , step 160 loss0= 0.0000 loss1= 6.5973 loss2=-3796.2681 lr= 0.000100
  831. 2019-03-12 07:42:33,313 : epoch 85 , step 0 loss0= 0.0000 loss1= 6.7152 loss2=-3495.7915 lr= 0.000100
  832. 2019-03-12 07:43:07,030 : epoch 85 , step 40 loss0= 0.0000 loss1= 6.3833 loss2=-3572.9883 lr= 0.000100
  833. 2019-03-12 07:43:40,420 : epoch 85 , step 80 loss0= 0.0000 loss1= 5.7873 loss2=-3371.6287 lr= 0.000100
  834. 2019-03-12 07:44:13,608 : epoch 85 , step 120 loss0= 0.0000 loss1= 6.8322 loss2=-3570.1521 lr= 0.000100
  835. 2019-03-12 07:44:47,015 : epoch 85 , step 160 loss0= 0.0000 loss1= 6.4603 loss2=-3380.7473 lr= 0.000100
  836. 2019-03-12 07:45:17,313 : epoch 86 , step 0 loss0= 0.0000 loss1= 6.0661 loss2=-3301.8008 lr= 0.000100
  837. 2019-03-12 07:45:51,527 : epoch 86 , step 40 loss0= 0.0000 loss1= 6.9372 loss2=-3518.5486 lr= 0.000100
  838. 2019-03-12 07:46:25,389 : epoch 86 , step 80 loss0= 0.0000 loss1= 5.7371 loss2=-3455.8867 lr= 0.000100
  839. 2019-03-12 07:46:59,386 : epoch 86 , step 120 loss0= 0.0000 loss1= 7.1235 loss2=-3597.8845 lr= 0.000100
  840. 2019-03-12 07:47:33,819 : epoch 86 , step 160 loss0= 0.0000 loss1= 6.3951 loss2=-3597.9907 lr= 0.000100
  841. 2019-03-12 07:48:04,807 : epoch 87 , step 0 loss0= 0.0000 loss1= 4.5620 loss2=-3533.4224 lr= 0.000100
  842. 2019-03-12 07:48:38,418 : epoch 87 , step 40 loss0= 0.0000 loss1= 6.5766 loss2=-3520.5996 lr= 0.000100
  843. 2019-03-12 07:49:11,826 : epoch 87 , step 80 loss0= 0.0000 loss1= 7.3536 loss2=-3581.5020 lr= 0.000100
  844. 2019-03-12 07:49:45,477 : epoch 87 , step 120 loss0= 0.0000 loss1= 7.6984 loss2=-3666.2791 lr= 0.000100
  845. 2019-03-12 07:50:19,004 : epoch 87 , step 160 loss0= 0.0000 loss1= 6.2953 loss2=-3490.8406 lr= 0.000100
  846. 2019-03-12 07:50:49,705 : epoch 88 , step 0 loss0= 0.0000 loss1= 7.1357 loss2=-3337.0220 lr= 0.000100
  847. 2019-03-12 07:51:23,280 : epoch 88 , step 40 loss0= 0.0000 loss1= 5.3716 loss2=-3758.1697 lr= 0.000100
  848. 2019-03-12 07:51:56,807 : epoch 88 , step 80 loss0= 0.0000 loss1= 5.7390 loss2=-3182.7947 lr= 0.000100
  849. 2019-03-12 07:52:30,400 : epoch 88 , step 120 loss0= 0.0000 loss1= 7.0119 loss2=-3658.6086 lr= 0.000100
  850. 2019-03-12 07:53:03,935 : epoch 88 , step 160 loss0= 0.0000 loss1= 6.6250 loss2=-3518.2188 lr= 0.000100
  851. 2019-03-12 07:53:34,187 : epoch 89 , step 0 loss0= 0.0000 loss1= 5.8319 loss2=-3356.7126 lr= 0.000100
  852. 2019-03-12 07:54:08,118 : epoch 89 , step 40 loss0= 0.0000 loss1= 5.4166 loss2=-3390.8982 lr= 0.000100
  853. 2019-03-12 07:54:41,900 : epoch 89 , step 80 loss0= 0.0000 loss1= 3.9539 loss2=-3099.4121 lr= 0.000100
  854. 2019-03-12 07:55:15,551 : epoch 89 , step 120 loss0= 0.0000 loss1= 7.0591 loss2=-3462.4275 lr= 0.000100
  855. 2019-03-12 07:55:50,026 : epoch 89 , step 160 loss0= 0.0000 loss1= 7.7942 loss2=-3637.4199 lr= 0.000100
  856. 2019-03-12 07:56:21,055 : epoch 90 , step 0 loss0= 0.0000 loss1= 5.3236 loss2=-3529.5203 lr= 0.000100
  857. 2019-03-12 07:56:54,987 : epoch 90 , step 40 loss0= 0.0000 loss1= 4.8077 loss2=-3381.2212 lr= 0.000100
  858. 2019-03-12 07:57:28,277 : epoch 90 , step 80 loss0= 0.0000 loss1= 7.7901 loss2=-3442.9673 lr= 0.000100
  859. 2019-03-12 07:58:02,080 : epoch 90 , step 120 loss0= 0.0000 loss1= 6.1893 loss2=-3634.4265 lr= 0.000100
  860. 2019-03-12 07:58:35,735 : epoch 90 , step 160 loss0= 0.0000 loss1= 8.0306 loss2=-3459.2898 lr= 0.000100
  861. 2019-03-12 07:59:06,266 : epoch 91 , step 0 loss0= 0.0000 loss1= 5.2590 loss2=-3449.8064 lr= 0.000100
  862. 2019-03-12 07:59:40,396 : epoch 91 , step 40 loss0= 0.0000 loss1= 5.8052 loss2=-3318.2212 lr= 0.000100
  863. 2019-03-12 08:00:14,515 : epoch 91 , step 80 loss0= 0.0000 loss1= 4.6671 loss2=-3351.5293 lr= 0.000100
  864. 2019-03-12 08:00:48,488 : epoch 91 , step 120 loss0= 0.0000 loss1= 5.3599 loss2=-3360.3162 lr= 0.000100
  865. 2019-03-12 08:01:22,113 : epoch 91 , step 160 loss0= 0.0000 loss1= 5.6239 loss2=-3640.9785 lr= 0.000100
  866. 2019-03-12 08:01:53,104 : epoch 92 , step 0 loss0= 0.0000 loss1= 5.1744 loss2=-3512.8025 lr= 0.000100
  867. 2019-03-12 08:02:26,990 : epoch 92 , step 40 loss0= 0.0000 loss1= 5.1931 loss2=-3150.1208 lr= 0.000100
  868. 2019-03-12 08:03:00,937 : epoch 92 , step 80 loss0= 0.0000 loss1= 5.5523 loss2=-3509.3892 lr= 0.000100
  869. 2019-03-12 08:03:35,045 : epoch 92 , step 120 loss0= 0.0000 loss1= 6.8412 loss2=-3674.7224 lr= 0.000100
  870. 2019-03-12 08:04:08,847 : epoch 92 , step 160 loss0= 0.0000 loss1= 5.8014 loss2=-3295.6348 lr= 0.000100
  871. 2019-03-12 08:04:39,032 : epoch 93 , step 0 loss0= 0.0000 loss1= 5.7095 loss2=-3764.3008 lr= 0.000100
  872. 2019-03-12 08:05:12,938 : epoch 93 , step 40 loss0= 0.0000 loss1= 5.4364 loss2=-3643.9727 lr= 0.000100
  873. 2019-03-12 08:05:46,545 : epoch 93 , step 80 loss0= 0.0000 loss1= 8.2039 loss2=-3917.3833 lr= 0.000100
  874. 2019-03-12 08:06:20,745 : epoch 93 , step 120 loss0= 0.0000 loss1= 5.2925 loss2=-3627.5474 lr= 0.000100
  875. 2019-03-12 08:06:54,597 : epoch 93 , step 160 loss0= 0.0000 loss1= 6.0872 loss2=-3413.1501 lr= 0.000100
  876. 2019-03-12 08:07:25,300 : epoch 94 , step 0 loss0= 0.0000 loss1= 5.3096 loss2=-3464.8513 lr= 0.000100
  877. 2019-03-12 08:07:58,674 : epoch 94 , step 40 loss0= 0.0000 loss1= 7.6335 loss2=-3694.9146 lr= 0.000100
  878. 2019-03-12 08:08:31,916 : epoch 94 , step 80 loss0= 0.0000 loss1= 5.9045 loss2=-3436.8630 lr= 0.000100
  879. 2019-03-12 08:09:05,582 : epoch 94 , step 120 loss0= 0.0000 loss1= 5.5898 loss2=-3488.1062 lr= 0.000100
  880. 2019-03-12 08:09:39,438 : epoch 94 , step 160 loss0= 0.0000 loss1= 5.1801 loss2=-3505.6228 lr= 0.000100
  881. 2019-03-12 08:10:10,128 : epoch 95 , step 0 loss0= 0.0000 loss1= 6.1486 loss2=-3443.9399 lr= 0.000100
  882. 2019-03-12 08:10:43,853 : epoch 95 , step 40 loss0= 0.0000 loss1= 6.1822 loss2=-3427.5403 lr= 0.000100
  883. 2019-03-12 08:11:17,325 : epoch 95 , step 80 loss0= 0.0000 loss1= 4.9731 loss2=-3443.3340 lr= 0.000100
  884. 2019-03-12 08:11:51,395 : epoch 95 , step 120 loss0= 0.0000 loss1= 6.0298 loss2=-3502.8723 lr= 0.000100
  885. 2019-03-12 08:12:25,637 : epoch 95 , step 160 loss0= 0.0000 loss1= 6.2408 loss2=-3413.2849 lr= 0.000100
  886. 2019-03-12 08:12:56,581 : epoch 96 , step 0 loss0= 0.0000 loss1= 4.9634 loss2=-3598.8481 lr= 0.000100
  887. 2019-03-12 08:13:30,183 : epoch 96 , step 40 loss0= 0.0000 loss1= 5.5524 loss2=-3452.3672 lr= 0.000100
  888. 2019-03-12 08:14:03,547 : epoch 96 , step 80 loss0= 0.0000 loss1= 4.6193 loss2=-3513.2844 lr= 0.000100
  889. 2019-03-12 08:14:37,662 : epoch 96 , step 120 loss0= 0.0000 loss1= 4.8606 loss2=-3392.7415 lr= 0.000100
  890. 2019-03-12 08:15:11,058 : epoch 96 , step 160 loss0= 0.0000 loss1= 4.2490 loss2=-3442.5188 lr= 0.000100
  891. 2019-03-12 08:15:41,318 : epoch 97 , step 0 loss0= 0.0000 loss1= 5.3186 loss2=-3449.2329 lr= 0.000100
  892. 2019-03-12 08:16:14,855 : epoch 97 , step 40 loss0= 0.0000 loss1= 5.2595 loss2=-3538.2532 lr= 0.000100
  893. 2019-03-12 08:16:48,087 : epoch 97 , step 80 loss0= 0.0000 loss1= 6.7085 loss2=-3602.0032 lr= 0.000100
  894. 2019-03-12 08:17:21,272 : epoch 97 , step 120 loss0= 0.0000 loss1= 4.7649 loss2=-3455.9177 lr= 0.000100
  895. 2019-03-12 08:17:54,517 : epoch 97 , step 160 loss0= 0.0000 loss1= 5.8134 loss2=-3308.8481 lr= 0.000100
  896. 2019-03-12 08:18:24,686 : epoch 98 , step 0 loss0= 0.0000 loss1= 4.2423 loss2=-3455.1492 lr= 0.000100
  897. 2019-03-12 08:18:58,129 : epoch 98 , step 40 loss0= 0.0000 loss1= 4.9801 loss2=-3469.6016 lr= 0.000100
  898. 2019-03-12 08:19:31,805 : epoch 98 , step 80 loss0= 0.0000 loss1= 6.0633 loss2=-3784.5242 lr= 0.000100
  899. 2019-03-12 08:20:05,135 : epoch 98 , step 120 loss0= 0.0000 loss1= 4.9199 loss2=-3367.7532 lr= 0.000100
  900. 2019-03-12 08:20:38,820 : epoch 98 , step 160 loss0= 0.0000 loss1= 5.6336 loss2=-3418.1528 lr= 0.000100
  901. 2019-03-12 08:21:09,037 : epoch 99 , step 0 loss0= 0.0000 loss1= 5.3125 loss2=-3268.3225 lr= 0.000100
  902. 2019-03-12 08:21:42,577 : epoch 99 , step 40 loss0= 0.0000 loss1= 5.2778 loss2=-3354.3286 lr= 0.000100
  903. 2019-03-12 08:22:16,497 : epoch 99 , step 80 loss0= 0.0000 loss1= 7.2378 loss2=-3694.2664 lr= 0.000100
  904. 2019-03-12 08:22:49,933 : epoch 99 , step 120 loss0= 0.0000 loss1= 5.0947 loss2=-3171.8240 lr= 0.000100
  905. 2019-03-12 08:23:23,450 : epoch 99 , step 160 loss0= 0.0000 loss1= 5.0257 loss2=-3431.2859 lr= 0.000100
  906. 2019-03-12 08:23:54,027 : epoch 100 , step 0 loss0= 0.0000 loss1= 5.6729 loss2=-3576.9607 lr= 0.000100
  907. 2019-03-12 08:24:27,628 : epoch 100 , step 40 loss0= 0.0000 loss1= 5.5704 loss2=-3466.0266 lr= 0.000100
  908. 2019-03-12 08:25:01,715 : epoch 100 , step 80 loss0= 0.0000 loss1= 5.0316 loss2=-3335.0623 lr= 0.000100
  909. 2019-03-12 08:25:36,009 : epoch 100 , step 120 loss0= 0.0000 loss1= 5.4240 loss2=-3363.3630 lr= 0.000100
  910. 2019-03-12 08:26:10,172 : epoch 100 , step 160 loss0= 0.0000 loss1= 6.4235 loss2=-3539.7832 lr= 0.000100
  911. 2019-03-12 08:26:39,515 : ## train ##
  912. 2019-03-12 08:30:54,977 : >> epoch 100 : average_loss= 0.5997, average_accuracy=81.903%, mean_weight_std= 0.132, mean_normalized_std= 0.3659 mean_dist=2.054e+03
  913. 2019-03-12 08:30:54,987 : >> epoch 100 : median_loss= 0.5857, median_accuracy=82.685%, median_weight_std= 0.0872, median_normalized_std= 0.0559 median_dist=2.057e+03
  914. 2019-03-12 08:30:54,987 : >> epoch 100 : ensemble accuracy: 84.08%
  915. 2019-03-12 08:30:54,988 : >> epoch 100 : loss histogram: (0.00,0.26, 18),(0.26,0.51, 0),(0.51,0.77, 4),(0.77,1.03, 0),(1.03,1.28, 18),
  916. 2019-03-12 08:30:54,988 : >> epoch 100 : accuracy histogram: (0.00,33.07, 18),(33.07,66.14, 0),(66.14,99.20, 4),(99.20,132.27, 0),(132.27,165.34, 18),
  917. 2019-03-12 08:30:54,993 : >> epoch 100 : weight std histogram: (0.0000,1.6161, 465580),(1.6161,3.2322, 1189),(3.2322,4.8483, 119),(4.8483,6.4645, 15),(6.4645,8.0806, 3),
  918. 2019-03-12 08:30:54,998 : >> epoch 100 : weight normalized std histogram: (0.0000,2801.4619, 466900),(2801.4619,5602.9238, 3),(5602.9238,8404.3857, 1),(8404.3857,11205.8477, 1),(11205.8477,14007.3096, 1),
  919. 2019-03-12 08:30:54,998 : >> epoch 100 : dist histogram: (1891.5205,1949.9422, 1),(1949.9422,2008.3639, 4),(2008.3639,2066.7855, 6),(2066.7855,2125.2072, 5),(2125.2072,2183.6289, 4),
  920. 2019-03-12 08:30:54,998 : ## eval ##
  921. 2019-03-12 08:35:09,327 : >> epoch 100 : average_loss= 0.5243, average_accuracy=84.686%, mean_weight_std= 0.145, mean_normalized_std= 0.3303 mean_dist=2.054e+03
  922. 2019-03-12 08:35:09,337 : >> epoch 100 : median_loss= 0.5367, median_accuracy=83.935%, median_weight_std=0.09637, median_normalized_std= 0.0624 median_dist=2.048e+03
  923. 2019-03-12 08:35:09,337 : >> epoch 100 : ensemble accuracy: 86.21%
  924. 2019-03-12 08:35:09,337 : >> epoch 100 : loss histogram: (0.00,0.22, 18),(0.22,0.45, 0),(0.45,0.67, 4),(0.67,0.89, 0),(0.89,1.11, 18),
  925. 2019-03-12 08:35:09,337 : >> epoch 100 : accuracy histogram: (0.00,34.20, 18),(34.20,68.40, 0),(68.40,102.59, 4),(102.59,136.79, 0),(136.79,170.99, 18),
  926. 2019-03-12 08:35:09,343 : >> epoch 100 : weight std histogram: (0.0000,1.6712, 465669),(1.6712,3.3424, 1090),(3.3424,5.0135, 127),(5.0135,6.6847, 18),(6.6847,8.3559, 2),
  927. 2019-03-12 08:35:09,348 : >> epoch 100 : weight normalized std histogram: (0.0000,2357.3879, 466903),(2357.3879,4714.7758, 1),(4714.7758,7072.1637, 0),(7072.1637,9429.5516, 1),(9429.5516,11786.9395, 1),
  928. 2019-03-12 08:35:09,348 : >> epoch 100 : dist histogram: (1946.6985,2019.1014, 7),(2019.1014,2091.5042, 8),(2091.5042,2163.9071, 4),(2163.9071,2236.3100, 0),(2236.3100,2308.7129, 1),
  929. 2019-03-12 08:35:09,403 : >> epoch 100 : dist pair-wise
  930. 2019-03-12 08:35:09,408 :
  931. [[ 0. 153.346 264.508 219.217 180.283 194.636 211.765 242.47
  932. 166.945 167.542 318.841 198.415 210.828 176.743 243.627 124.149
  933. 231.143 201.093 259.597 148.121]
  934. [ 153.346 0. 235.798 195.271 195.002 187.035 208.938 228.031
  935. 193.188 187.114 374.113 204.976 164.584 209.467 204.021 139.253
  936. 203.192 175.847 228.056 139.217]
  937. [ 264.508 235.798 0. 147.66 255.378 225.382 182.868 123.74
  938. 273.941 261.25 473.269 209.229 167.397 325.205 133.331 257.366
  939. 135.207 160.292 154.443 219.079]
  940. [ 219.217 195.271 147.66 0. 223.411 214.14 207.84 168.516
  941. 246.174 223.215 446.154 200.813 149.953 278.759 111.272 196.742
  942. 127.146 144.232 157.253 194.935]
  943. [ 180.283 195.002 255.378 223.411 0. 129.686 244.805 261.685
  944. 260.329 225.736 402.597 277.303 206.074 263.076 229.295 189.834
  945. 267.242 214.837 288.96 214.647]
  946. [ 194.636 187.035 225.382 214.14 129.686 0. 201.202 238.666
  947. 262.272 243.464 441.502 276.738 179.005 283.856 211.649 187.916
  948. 251.236 178.472 278.993 195.842]
  949. [ 211.765 208.938 182.868 207.84 244.805 201.202 0. 188.083
  950. 212.824 235.33 406.572 200.44 197.725 279.757 201.843 210.461
  951. 195.624 163.909 222.384 191.747]
  952. [ 242.47 228.031 123.74 168.516 261.685 238.666 188.083 0.
  953. 237.124 236.254 434.472 192.252 199.332 294.827 150.413 236.26
  954. 138.614 185.908 131.319 198.423]
  955. [ 166.945 193.188 273.941 246.174 260.329 262.272 212.824 237.124
  956. 0. 164.607 310.149 171.654 246.473 200.498 267.893 169.048
  957. 230.198 230.173 249.934 167.415]
  958. [ 167.542 187.114 261.25 223.215 225.736 243.464 235.33 236.254
  959. 164.607 0. 295.043 193.407 224.23 168.754 247.22 169.699
  960. 217.205 219.08 229.859 163.789]
  961. [ 318.841 374.113 473.269 446.154 402.597 441.502 406.572 434.472
  962. 310.149 295.043 0. 364.933 447.951 253.585 466.707 354.28
  963. 421.91 437.513 426.582 353.893]
  964. [ 198.415 204.976 209.229 200.813 277.303 276.738 200.44 192.252
  965. 171.654 193.407 364.933 0. 205.884 237.178 223.31 207.324
  966. 166.879 202.554 194.843 190.85 ]
  967. [ 210.828 164.584 167.397 149.953 206.074 179.005 197.725 199.332
  968. 246.473 224.23 447.951 205.884 0. 282.504 147.453 191.781
  969. 168.519 141.58 205.884 171.154]
  970. [ 176.743 209.467 325.205 278.759 263.076 283.856 279.757 294.827
  971. 200.498 168.754 253.585 237.178 282.504 0. 300.23 198.047
  972. 265.05 275.707 278.633 198.578]
  973. [ 243.627 204.021 133.331 111.272 229.295 211.649 201.843 150.413
  974. 267.893 247.22 466.707 223.31 147.453 300.23 0. 220.506
  975. 137.256 148.439 143.487 207.542]
  976. [ 124.149 139.253 257.366 196.742 189.834 187.916 210.461 236.26
  977. 169.048 169.699 354.28 207.324 191.781 198.047 220.506 0.
  978. 219.387 198.114 240.713 141.908]
  979. [ 231.143 203.192 135.207 127.146 267.242 251.236 195.624 138.614
  980. 230.198 217.205 421.91 166.879 168.519 265.05 137.256 219.387
  981. 0. 160.297 118.589 188.04 ]
  982. [ 201.093 175.847 160.292 144.232 214.837 178.472 163.909 185.908
  983. 230.173 219.08 437.513 202.554 141.58 275.707 148.439 198.114
  984. 160.297 0. 195.616 192.476]
  985. [ 259.597 228.056 154.443 157.253 288.96 278.993 222.384 131.319
  986. 249.934 229.859 426.582 194.843 205.884 278.633 143.487 240.713
  987. 118.589 195.616 0. 210.396]
  988. [ 148.121 139.217 219.079 194.935 214.647 195.842 191.747 198.423
  989. 167.415 163.789 353.893 190.85 171.154 198.578 207.542 141.908
  990. 188.04 192.476 210.396 0. ]]
  991. 2019-03-12 08:35:10,766 : epoch 101 , step 0 loss0= 0.0000 loss1= 6.4391 loss2=-3635.7390 lr= 0.000100
  992. 2019-03-12 08:35:44,312 : epoch 101 , step 40 loss0= 0.0000 loss1= 6.3929 loss2=-3529.8665 lr= 0.000100
  993. 2019-03-12 08:36:18,817 : epoch 101 , step 80 loss0= 0.0000 loss1= 3.7482 loss2=-3409.9685 lr= 0.000100
  994. 2019-03-12 08:36:53,313 : epoch 101 , step 120 loss0= 0.0000 loss1= 4.8206 loss2=-3501.5466 lr= 0.000100
  995. 2019-03-12 08:37:27,759 : epoch 101 , step 160 loss0= 0.0000 loss1= 4.9246 loss2=-3442.6863 lr= 0.000100
  996. 2019-03-12 08:37:58,745 : epoch 102 , step 0 loss0= 0.0000 loss1= 5.2222 loss2=-3649.0496 lr= 0.000100
  997. 2019-03-12 08:38:32,095 : epoch 102 , step 40 loss0= 0.0000 loss1= 4.8779 loss2=-3568.2703 lr= 0.000100
  998. 2019-03-12 08:39:05,528 : epoch 102 , step 80 loss0= 0.0000 loss1= 5.1397 loss2=-3514.2888 lr= 0.000100
  999. 2019-03-12 08:39:38,798 : epoch 102 , step 120 loss0= 0.0000 loss1= 5.5254 loss2=-3523.8164 lr= 0.000100
  1000. 2019-03-12 08:40:12,164 : epoch 102 , step 160 loss0= 0.0000 loss1= 5.0269 loss2=-3316.9961 lr= 0.000100
  1001. 2019-03-12 08:40:42,369 : epoch 103 , step 0 loss0= 0.0000 loss1= 4.5853 loss2=-3371.9490 lr= 0.000100
  1002. 2019-03-12 08:41:15,638 : epoch 103 , step 40 loss0= 0.0000 loss1= 7.1521 loss2=-3718.0105 lr= 0.000100
  1003. 2019-03-12 08:41:48,948 : epoch 103 , step 80 loss0= 0.0000 loss1= 5.5033 loss2=-3527.1934 lr= 0.000100
  1004. 2019-03-12 08:42:22,161 : epoch 103 , step 120 loss0= 0.0000 loss1= 5.7944 loss2=-3559.0466 lr= 0.000100
  1005. 2019-03-12 08:42:55,883 : epoch 103 , step 160 loss0= 0.0000 loss1= 5.1474 loss2=-3538.5227 lr= 0.000100
  1006. 2019-03-12 08:43:26,616 : epoch 104 , step 0 loss0= 0.0000 loss1= 4.3858 loss2=-3610.7754 lr= 0.000100
  1007. 2019-03-12 08:44:00,784 : epoch 104 , step 40 loss0= 0.0000 loss1= 5.3915 loss2=-3407.2773 lr= 0.000100
  1008. 2019-03-12 08:44:34,684 : epoch 104 , step 80 loss0= 0.0000 loss1= 6.0283 loss2=-3844.6216 lr= 0.000100
  1009. 2019-03-12 08:45:07,904 : epoch 104 , step 120 loss0= 0.0000 loss1= 5.6004 loss2=-3358.9797 lr= 0.000100
  1010. 2019-03-12 08:45:41,186 : epoch 104 , step 160 loss0= 0.0000 loss1= 4.1754 loss2=-3120.9924 lr= 0.000100
  1011. 2019-03-12 08:46:11,792 : epoch 105 , step 0 loss0= 0.0000 loss1= 6.3804 loss2=-3675.1868 lr= 0.000100
  1012. 2019-03-12 08:46:45,238 : epoch 105 , step 40 loss0= 0.0000 loss1= 4.0559 loss2=-3465.9258 lr= 0.000100
  1013. 2019-03-12 08:47:18,669 : epoch 105 , step 80 loss0= 0.0000 loss1= 3.8679 loss2=-3303.7781 lr= 0.000100
  1014. 2019-03-12 08:47:51,989 : epoch 105 , step 120 loss0= 0.0000 loss1= 6.6974 loss2=-3872.9146 lr= 0.000100
  1015. 2019-03-12 08:48:26,112 : epoch 105 , step 160 loss0= 0.0000 loss1= 7.4834 loss2=-3417.6392 lr= 0.000100
  1016. 2019-03-12 08:48:56,634 : epoch 106 , step 0 loss0= 0.0000 loss1= 4.8453 loss2=-3184.0435 lr= 0.000100
  1017. 2019-03-12 08:49:30,691 : epoch 106 , step 40 loss0= 0.0000 loss1= 5.7646 loss2=-3527.2688 lr= 0.000100
  1018. 2019-03-12 08:50:03,938 : epoch 106 , step 80 loss0= 0.0000 loss1= 6.3855 loss2=-3870.4275 lr= 0.000100
  1019. 2019-03-12 08:50:37,228 : epoch 106 , step 120 loss0= 0.0000 loss1= 5.3285 loss2=-3689.7759 lr= 0.000100
  1020. 2019-03-12 08:51:10,512 : epoch 106 , step 160 loss0= 0.0000 loss1= 6.4309 loss2=-3695.3228 lr= 0.000100
  1021. 2019-03-12 08:51:40,701 : epoch 107 , step 0 loss0= 0.0000 loss1= 6.4554 loss2=-3737.8196 lr= 0.000100
  1022. 2019-03-12 08:52:14,355 : epoch 107 , step 40 loss0= 0.0000 loss1= 7.4774 loss2=-3602.0291 lr= 0.000100
  1023. 2019-03-12 08:52:47,729 : epoch 107 , step 80 loss0= 0.0000 loss1= 6.5238 loss2=-3494.1907 lr= 0.000100
  1024. 2019-03-12 08:53:21,609 : epoch 107 , step 120 loss0= 0.0000 loss1= 5.6074 loss2=-3487.7000 lr= 0.000100
  1025. 2019-03-12 08:53:55,476 : epoch 107 , step 160 loss0= 0.0000 loss1= 5.7439 loss2=-3489.8669 lr= 0.000100
  1026. 2019-03-12 08:54:26,340 : epoch 108 , step 0 loss0= 0.0000 loss1= 6.3975 loss2=-3386.9641 lr= 0.000100
  1027. 2019-03-12 08:55:00,225 : epoch 108 , step 40 loss0= 0.0000 loss1= 5.4234 loss2=-3350.9751 lr= 0.000100
  1028. 2019-03-12 08:55:33,638 : epoch 108 , step 80 loss0= 0.0000 loss1= 3.9964 loss2=-3219.6248 lr= 0.000100
  1029. 2019-03-12 08:56:07,079 : epoch 108 , step 120 loss0= 0.0000 loss1= 4.7880 loss2=-3402.6965 lr= 0.000100
  1030. 2019-03-12 08:56:41,236 : epoch 108 , step 160 loss0= 0.0000 loss1= 6.3939 loss2=-3350.2805 lr= 0.000100
  1031. 2019-03-12 08:57:11,860 : epoch 109 , step 0 loss0= 0.0000 loss1= 5.6768 loss2=-3495.3672 lr= 0.000100
  1032. 2019-03-12 08:57:45,843 : epoch 109 , step 40 loss0= 0.0000 loss1= 4.9800 loss2=-3262.5381 lr= 0.000100
  1033. 2019-03-12 08:58:19,967 : epoch 109 , step 80 loss0= 0.0000 loss1= 6.2329 loss2=-3788.0801 lr= 0.000100
  1034. 2019-03-12 08:58:53,761 : epoch 109 , step 120 loss0= 0.0000 loss1= 7.2079 loss2=-3419.6599 lr= 0.000100
  1035. 2019-03-12 08:59:26,973 : epoch 109 , step 160 loss0= 0.0000 loss1= 4.7197 loss2=-3418.7141 lr= 0.000100
  1036. 2019-03-12 08:59:57,168 : epoch 110 , step 0 loss0= 0.0000 loss1= 5.6062 loss2=-3518.1289 lr= 0.000100
  1037. 2019-03-12 09:00:31,008 : epoch 110 , step 40 loss0= 0.0000 loss1= 7.2340 loss2=-3910.4998 lr= 0.000100
  1038. 2019-03-12 09:01:04,737 : epoch 110 , step 80 loss0= 0.0000 loss1= 4.6869 loss2=-3599.9211 lr= 0.000100
  1039. 2019-03-12 09:01:38,925 : epoch 110 , step 120 loss0= 0.0000 loss1= 4.6412 loss2=-3368.4727 lr= 0.000100
  1040. 2019-03-12 09:02:12,382 : epoch 110 , step 160 loss0= 0.0000 loss1= 5.4345 loss2=-3384.2935 lr= 0.000100
  1041. 2019-03-12 09:02:43,218 : epoch 111 , step 0 loss0= 0.0000 loss1= 4.9800 loss2=-3427.9294 lr= 0.000100
  1042. 2019-03-12 09:03:16,735 : epoch 111 , step 40 loss0= 0.0000 loss1= 5.1705 loss2=-3501.5767 lr= 0.000100
  1043. 2019-03-12 09:03:50,071 : epoch 111 , step 80 loss0= 0.0000 loss1= 5.7656 loss2=-3406.6848 lr= 0.000100
  1044. 2019-03-12 09:04:23,283 : epoch 111 , step 120 loss0= 0.0000 loss1= 4.3457 loss2=-3553.2703 lr= 0.000100
  1045. 2019-03-12 09:04:56,552 : epoch 111 , step 160 loss0= 0.0000 loss1= 5.0987 loss2=-3626.5540 lr= 0.000100
  1046. 2019-03-12 09:05:26,799 : epoch 112 , step 0 loss0= 0.0000 loss1= 5.3121 loss2=-3561.3711 lr= 0.000100
  1047. 2019-03-12 09:06:00,812 : epoch 112 , step 40 loss0= 0.0000 loss1= 4.9266 loss2=-3646.4470 lr= 0.000100
  1048. 2019-03-12 09:06:34,452 : epoch 112 , step 80 loss0= 0.0000 loss1= 5.2865 loss2=-3701.7427 lr= 0.000100
  1049. 2019-03-12 09:07:08,219 : epoch 112 , step 120 loss0= 0.0000 loss1= 5.1487 loss2=-3602.9653 lr= 0.000100
  1050. 2019-03-12 09:07:41,628 : epoch 112 , step 160 loss0= 0.0000 loss1= 4.5528 loss2=-3164.2791 lr= 0.000100
  1051. 2019-03-12 09:08:11,842 : epoch 113 , step 0 loss0= 0.0000 loss1= 5.6317 loss2=-3669.4050 lr= 0.000100
  1052. 2019-03-12 09:08:45,120 : epoch 113 , step 40 loss0= 0.0000 loss1= 4.8398 loss2=-3504.5610 lr= 0.000100
  1053. 2019-03-12 09:09:18,969 : epoch 113 , step 80 loss0= 0.0000 loss1= 3.4197 loss2=-3371.0427 lr= 0.000100
  1054. 2019-03-12 09:09:52,745 : epoch 113 , step 120 loss0= 0.0000 loss1= 5.8364 loss2=-3633.4031 lr= 0.000100
  1055. 2019-03-12 09:10:26,379 : epoch 113 , step 160 loss0= 0.0000 loss1= 5.4662 loss2=-3509.6313 lr= 0.000100
  1056. 2019-03-12 09:10:56,587 : epoch 114 , step 0 loss0= 0.0000 loss1= 4.6886 loss2=-3566.4836 lr= 0.000100
  1057. 2019-03-12 09:11:30,534 : epoch 114 , step 40 loss0= 0.0000 loss1= 5.6290 loss2=-3692.5845 lr= 0.000100
  1058. 2019-03-12 09:12:03,796 : epoch 114 , step 80 loss0= 0.0000 loss1= 4.6158 loss2=-3374.6477 lr= 0.000100
  1059. 2019-03-12 09:12:37,626 : epoch 114 , step 120 loss0= 0.0000 loss1= 6.3773 loss2=-3682.2063 lr= 0.000100
  1060. 2019-03-12 09:13:11,403 : epoch 114 , step 160 loss0= 0.0000 loss1= 3.3127 loss2=-3185.6228 lr= 0.000100
  1061. 2019-03-12 09:13:41,785 : epoch 115 , step 0 loss0= 0.0000 loss1= 5.5603 loss2=-3492.2947 lr= 0.000100
  1062. 2019-03-12 09:14:15,188 : epoch 115 , step 40 loss0= 0.0000 loss1= 5.1460 loss2=-3588.6423 lr= 0.000100
  1063. 2019-03-12 09:14:48,522 : epoch 115 , step 80 loss0= 0.0000 loss1= 4.3429 loss2=-3529.3364 lr= 0.000100
  1064. 2019-03-12 09:15:21,786 : epoch 115 , step 120 loss0= 0.0000 loss1= 3.9175 loss2=-3676.2258 lr= 0.000100
  1065. 2019-03-12 09:15:55,101 : epoch 115 , step 160 loss0= 0.0000 loss1= 5.4212 loss2=-3534.8142 lr= 0.000100
  1066. 2019-03-12 09:16:25,461 : epoch 116 , step 0 loss0= 0.0000 loss1= 5.3739 loss2=-3804.6013 lr= 0.000100
  1067. 2019-03-12 09:16:59,093 : epoch 116 , step 40 loss0= 0.0000 loss1= 5.3162 loss2=-3609.0071 lr= 0.000100
  1068. 2019-03-12 09:17:33,063 : epoch 116 , step 80 loss0= 0.0000 loss1= 4.1890 loss2=-3273.4255 lr= 0.000100
  1069. 2019-03-12 09:18:07,094 : epoch 116 , step 120 loss0= 0.0000 loss1= 5.1664 loss2=-3593.9966 lr= 0.000100
  1070. 2019-03-12 09:18:40,787 : epoch 116 , step 160 loss0= 0.0000 loss1= 6.4153 loss2=-3633.0232 lr= 0.000100
  1071. 2019-03-12 09:19:11,777 : epoch 117 , step 0 loss0= 0.0000 loss1= 5.1309 loss2=-3501.8970 lr= 0.000100
  1072. 2019-03-12 09:19:45,278 : epoch 117 , step 40 loss0= 0.0000 loss1= 6.7158 loss2=-3885.7642 lr= 0.000100
  1073. 2019-03-12 09:20:18,583 : epoch 117 , step 80 loss0= 0.0000 loss1= 5.9646 loss2=-3660.8357 lr= 0.000100
  1074. 2019-03-12 09:20:51,830 : epoch 117 , step 120 loss0= 0.0000 loss1= 3.8315 loss2=-3265.2122 lr= 0.000100
  1075. 2019-03-12 09:21:25,104 : epoch 117 , step 160 loss0= 0.0000 loss1= 6.3806 loss2=-3315.2693 lr= 0.000100
  1076. 2019-03-12 09:21:55,341 : epoch 118 , step 0 loss0= 0.0000 loss1= 5.5714 loss2=-3284.5688 lr= 0.000100
  1077. 2019-03-12 09:22:29,006 : epoch 118 , step 40 loss0= 0.0000 loss1= 4.1459 loss2=-3330.9744 lr= 0.000100
  1078. 2019-03-12 09:23:02,599 : epoch 118 , step 80 loss0= 0.0000 loss1= 4.6110 loss2=-3505.7478 lr= 0.000100
  1079. 2019-03-12 09:23:36,159 : epoch 118 , step 120 loss0= 0.0000 loss1= 4.7783 loss2=-3390.6365 lr= 0.000100
  1080. 2019-03-12 09:24:10,041 : epoch 118 , step 160 loss0= 0.0000 loss1= 5.0990 loss2=-3491.2634 lr= 0.000100
  1081. 2019-03-12 09:24:40,424 : epoch 119 , step 0 loss0= 0.0000 loss1= 5.8923 loss2=-3653.8547 lr= 0.000100
  1082. 2019-03-12 09:25:13,981 : epoch 119 , step 40 loss0= 0.0000 loss1= 5.5670 loss2=-3448.5349 lr= 0.000100
  1083. 2019-03-12 09:25:47,273 : epoch 119 , step 80 loss0= 0.0000 loss1= 4.1053 loss2=-3570.5154 lr= 0.000100
  1084. 2019-03-12 09:26:20,703 : epoch 119 , step 120 loss0= 0.0000 loss1= 5.2136 loss2=-3432.2664 lr= 0.000100
  1085. 2019-03-12 09:26:54,052 : epoch 119 , step 160 loss0= 0.0000 loss1= 5.4459 loss2=-3620.0266 lr= 0.000100
  1086. 2019-03-12 09:27:24,248 : epoch 120 , step 0 loss0= 0.0000 loss1= 5.0124 loss2=-3589.7673 lr= 0.000100
  1087. 2019-03-12 09:27:58,046 : epoch 120 , step 40 loss0= 0.0000 loss1= 4.2717 loss2=-3295.8142 lr= 0.000100
  1088. 2019-03-12 09:28:32,142 : epoch 120 , step 80 loss0= 0.0000 loss1= 5.8025 loss2=-3657.1658 lr= 0.000100
  1089. 2019-03-12 09:29:05,604 : epoch 120 , step 120 loss0= 0.0000 loss1= 5.8679 loss2=-3728.8250 lr= 0.000100
  1090. 2019-03-12 09:29:39,304 : epoch 120 , step 160 loss0= 0.0000 loss1= 3.9793 loss2=-3486.4883 lr= 0.000100
  1091. 2019-03-12 09:30:08,557 : ## train ##
  1092. 2019-03-12 09:34:24,217 : >> epoch 120 : average_loss= 0.6108, average_accuracy=82.262%, mean_weight_std= 0.1635, mean_normalized_std= 0.4578 mean_dist=2.168e+03
  1093. 2019-03-12 09:34:24,228 : >> epoch 120 : median_loss= 0.6355, median_accuracy=82.610%, median_weight_std= 0.1081, median_normalized_std= 0.0655 median_dist=2.114e+03
  1094. 2019-03-12 09:34:24,229 : >> epoch 120 : ensemble accuracy: 84.26%
  1095. 2019-03-12 09:34:24,229 : >> epoch 120 : loss histogram: (0.00,0.25, 18),(0.25,0.51, 0),(0.51,0.76, 4),(0.76,1.01, 0),(1.01,1.27, 18),
  1096. 2019-03-12 09:34:24,229 : >> epoch 120 : accuracy histogram: (0.00,33.28, 18),(33.28,66.57, 0),(66.57,99.85, 4),(99.85,133.14, 0),(133.14,166.42, 18),
  1097. 2019-03-12 09:34:24,234 : >> epoch 120 : weight std histogram: (0.0000,1.6645, 465187),(1.6645,3.3290, 1553),(3.3290,4.9935, 150),(4.9935,6.6579, 14),(6.6579,8.3224, 2),
  1098. 2019-03-12 09:34:24,239 : >> epoch 120 : weight normalized std histogram: (0.0000,6399.5047, 466903),(6399.5047,12799.0094, 1),(12799.0094,19198.5141, 0),(19198.5141,25598.0187, 0),(25598.0187,31997.5234, 2),
  1099. 2019-03-12 09:34:24,240 : >> epoch 120 : dist histogram: (2011.7216,2076.9124, 4),(2076.9124,2142.1031, 7),(2142.1031,2207.2939, 1),(2207.2939,2272.4847, 4),(2272.4847,2337.6755, 4),
  1100. 2019-03-12 09:34:24,240 : ## eval ##
  1101. 2019-03-12 09:38:38,827 : >> epoch 120 : average_loss= 0.5251, average_accuracy=85.371%, mean_weight_std= 0.1615, mean_normalized_std= 0.5123 mean_dist=2.168e+03
  1102. 2019-03-12 09:38:38,838 : >> epoch 120 : median_loss= 0.5042, median_accuracy=86.020%, median_weight_std= 0.1074, median_normalized_std= 0.0654 median_dist=2.174e+03
  1103. 2019-03-12 09:38:38,838 : >> epoch 120 : ensemble accuracy: 86.90%
  1104. 2019-03-12 09:38:38,839 : >> epoch 120 : loss histogram: (0.00,0.24, 18),(0.24,0.48, 0),(0.48,0.72, 4),(0.72,0.96, 0),(0.96,1.20, 18),
  1105. 2019-03-12 09:38:38,839 : >> epoch 120 : accuracy histogram: (0.00,34.48, 18),(34.48,68.96, 0),(68.96,103.43, 4),(103.43,137.91, 0),(137.91,172.39, 18),
  1106. 2019-03-12 09:38:38,844 : >> epoch 120 : weight std histogram: (0.0000,1.6271, 465249),(1.6271,3.2541, 1480),(3.2541,4.8812, 157),(4.8812,6.5082, 17),(6.5082,8.1353, 3),
  1107. 2019-03-12 09:38:38,849 : >> epoch 120 : weight normalized std histogram: (0.0000,15699.2234, 466905),(15699.2234,31398.4469, 0),(31398.4469,47097.6703, 0),(47097.6703,62796.8938, 0),(62796.8938,78496.1172, 1),
  1108. 2019-03-12 09:38:38,849 : >> epoch 120 : dist histogram: (2005.5403,2073.9593, 5),(2073.9593,2142.3784, 4),(2142.3784,2210.7974, 4),(2210.7974,2279.2165, 4),(2279.2165,2347.6355, 3),
  1109. 2019-03-12 09:38:38,904 : >> epoch 120 : dist pair-wise
  1110. 2019-03-12 09:38:38,909 :
  1111. [[ 0. 137.872 320.43 154.485 213.316 176.182 257.302 342.177
  1112. 425.665 318.852 224.72 298.872 232.642 230.598 341.861 200.29
  1113. 391.924 168.866 276.932 263.613]
  1114. [ 137.872 0. 335.561 165.992 216.41 207.892 287.582 354.807
  1115. 431.262 328.533 228.02 315.346 245.115 232.493 360.819 214.809
  1116. 422.195 193.067 284.815 273.362]
  1117. [ 320.43 335.561 0. 259.413 245.609 284.61 261.456 244.836
  1118. 339.186 179.603 237.647 286.496 236.557 217.875 139.961 231.694
  1119. 262.499 238.971 324.014 177.564]
  1120. [ 154.485 165.992 259.413 0. 181.598 177.239 227.836 287.207
  1121. 363.686 248.07 185.276 260.29 195.798 162.443 291.324 157.67
  1122. 339.365 151.658 245.135 198.129]
  1123. [ 213.316 216.41 245.609 181.598 0. 199.648 224.402 237.669
  1124. 328.245 249.425 167.946 229.929 204.99 162.441 266.749 182.322
  1125. 323.169 166.09 252.241 216.482]
  1126. [ 176.182 207.892 284.61 177.239 199.648 0. 252.919 317.782
  1127. 394.478 301.285 219.644 299.274 166.774 203.182 308.794 155.71
  1128. 352.836 204.202 289.504 238.745]
  1129. [ 257.302 287.582 261.456 227.836 224.402 252.919 0. 209.481
  1130. 263.175 228.13 181.344 195.115 259.444 208.278 285.619 262.524
  1131. 264.963 213.42 215.728 224.749]
  1132. [ 342.177 354.807 244.836 287.207 237.669 317.782 209.481 0.
  1133. 250.957 191.373 212.415 212.307 265.001 217.143 239.799 302.932
  1134. 216.597 239.408 269.58 220.374]
  1135. [ 425.665 431.262 339.186 363.686 328.245 394.478 263.175 250.957
  1136. 0. 314.704 283.725 215.847 377.607 326.379 365.641 390.093
  1137. 325.84 341.252 245.196 343.725]
  1138. [ 318.852 328.533 179.603 248.07 249.425 301.285 228.13 191.373
  1139. 314.704 0. 228.251 261.645 237.298 190.669 174.023 253.355
  1140. 220.132 228.466 300.474 157.888]
  1141. [ 224.72 228.02 237.647 185.276 167.946 219.644 181.344 212.415
  1142. 283.725 228.251 0. 197.38 221.668 176.031 267.553 201.173
  1143. 305.07 163.773 216.698 200.882]
  1144. [ 298.872 315.346 286.496 260.29 229.929 299.274 195.115 212.307
  1145. 215.847 261.645 197.38 0. 276.386 239.177 311.859 300.006
  1146. 292.801 229.146 172.724 258.638]
  1147. [ 232.642 245.115 236.557 195.798 204.99 166.774 259.444 265.001
  1148. 377.607 237.298 221.668 276.386 0. 171.823 246.06 172.347
  1149. 294.233 208.799 283.434 181.672]
  1150. [ 230.598 232.493 217.875 162.443 162.441 203.182 208.278 217.143
  1151. 326.379 190.669 176.031 239.177 171.823 0. 241.65 159.552
  1152. 291.921 178.582 266.375 160.089]
  1153. [ 341.861 360.819 139.961 291.324 266.749 308.794 285.619 239.799
  1154. 365.641 174.023 267.553 311.859 246.06 241.65 0. 264.449
  1155. 236.725 260.367 345.947 181.773]
  1156. [ 200.29 214.809 231.694 157.67 182.322 155.71 262.524 302.932
  1157. 390.093 253.355 201.173 300.006 172.347 159.552 264.449 0.
  1158. 346.661 199.801 305.62 199.771]
  1159. [ 391.924 422.195 262.499 339.365 323.169 352.836 264.963 216.597
  1160. 325.84 220.132 305.07 292.801 294.233 291.921 236.725 346.661
  1161. 0. 314.081 344.614 235.934]
  1162. [ 168.866 193.067 238.971 151.658 166.09 204.202 213.42 239.408
  1163. 341.252 228.466 163.773 229.146 208.799 178.582 260.367 199.801
  1164. 314.081 0. 239.587 198.831]
  1165. [ 276.932 284.815 324.014 245.135 252.241 289.504 215.728 269.58
  1166. 245.196 300.474 216.698 172.724 283.434 266.375 345.947 305.62
  1167. 344.614 239.587 0. 283.24 ]
  1168. [ 263.613 273.362 177.564 198.129 216.482 238.745 224.749 220.374
  1169. 343.725 157.888 200.882 258.638 181.672 160.089 181.773 199.771
  1170. 235.934 198.831 283.24 0. ]]
  1171. 2019-03-12 09:38:40,267 : epoch 121 , step 0 loss0= 0.0000 loss1= 5.1167 loss2=-3561.1021 lr= 0.000100
  1172. 2019-03-12 09:39:14,018 : epoch 121 , step 40 loss0= 0.0000 loss1= 4.3040 loss2=-3519.9673 lr= 0.000100
  1173. 2019-03-12 09:39:48,048 : epoch 121 , step 80 loss0= 0.0000 loss1= 4.2904 loss2=-3551.4812 lr= 0.000100
  1174. 2019-03-12 09:40:21,814 : epoch 121 , step 120 loss0= 0.0000 loss1= 4.0215 loss2=-3322.7043 lr= 0.000100
  1175. 2019-03-12 09:40:56,318 : epoch 121 , step 160 loss0= 0.0000 loss1= 4.2619 loss2=-3519.2585 lr= 0.000100
  1176. 2019-03-12 09:41:27,279 : epoch 122 , step 0 loss0= 0.0000 loss1= 3.8475 loss2=-3618.1023 lr= 0.000100
  1177. 2019-03-12 09:42:00,651 : epoch 122 , step 40 loss0= 0.0000 loss1= 5.0761 loss2=-3477.4395 lr= 0.000100
  1178. 2019-03-12 09:42:34,113 : epoch 122 , step 80 loss0= 0.0000 loss1= 5.8496 loss2=-3607.2817 lr= 0.000100
  1179. 2019-03-12 09:43:07,486 : epoch 122 , step 120 loss0= 0.0000 loss1= 5.4510 loss2=-3828.8406 lr= 0.000100
  1180. 2019-03-12 09:43:41,298 : epoch 122 , step 160 loss0= 0.0000 loss1= 4.6695 loss2=-3644.2285 lr= 0.000100
  1181. 2019-03-12 09:44:12,092 : epoch 123 , step 0 loss0= 0.0000 loss1= 4.5054 loss2=-3494.2317 lr= 0.000100
  1182. 2019-03-12 09:44:45,754 : epoch 123 , step 40 loss0= 0.0000 loss1= 5.2400 loss2=-3335.6724 lr= 0.000100
  1183. 2019-03-12 09:45:19,237 : epoch 123 , step 80 loss0= 0.0000 loss1= 5.0450 loss2=-3481.1750 lr= 0.000100
  1184. 2019-03-12 09:45:52,922 : epoch 123 , step 120 loss0= 0.0000 loss1= 4.8028 loss2=-3497.8118 lr= 0.000100
  1185. 2019-03-12 09:46:26,351 : epoch 123 , step 160 loss0= 0.0000 loss1= 6.9149 loss2=-3484.5083 lr= 0.000100
  1186. 2019-03-12 09:46:57,312 : epoch 124 , step 0 loss0= 0.0000 loss1= 4.2392 loss2=-3499.4849 lr= 0.000100
  1187. 2019-03-12 09:47:30,698 : epoch 124 , step 40 loss0= 0.0000 loss1= 5.2025 loss2=-3552.4790 lr= 0.000100
  1188. 2019-03-12 09:48:04,101 : epoch 124 , step 80 loss0= 0.0000 loss1= 4.9097 loss2=-3552.3560 lr= 0.000100
  1189. 2019-03-12 09:48:37,547 : epoch 124 , step 120 loss0= 0.0000 loss1= 4.6740 loss2=-3440.9294 lr= 0.000100
  1190. 2019-03-12 09:49:11,004 : epoch 124 , step 160 loss0= 0.0000 loss1= 5.0002 loss2=-3702.7253 lr= 0.000100
  1191. 2019-03-12 09:49:41,227 : epoch 125 , step 0 loss0= 0.0000 loss1= 2.7821 loss2=-2812.5789 lr= 0.000100
  1192. 2019-03-12 09:50:15,246 : epoch 125 , step 40 loss0= 0.0000 loss1= 6.1781 loss2=-3684.1001 lr= 0.000100
  1193. 2019-03-12 09:50:48,868 : epoch 125 , step 80 loss0= 0.0000 loss1= 4.8788 loss2=-3461.7368 lr= 0.000100
  1194. 2019-03-12 09:51:22,212 : epoch 125 , step 120 loss0= 0.0000 loss1= 5.4780 loss2=-3563.0942 lr= 0.000100
  1195. 2019-03-12 09:51:56,302 : epoch 125 , step 160 loss0= 0.0000 loss1= 6.4020 loss2=-3398.0513 lr= 0.000100
  1196. 2019-03-12 09:52:27,313 : epoch 126 , step 0 loss0= 0.0000 loss1= 4.8205 loss2=-3553.9629 lr= 0.000100
  1197. 2019-03-12 09:53:00,928 : epoch 126 , step 40 loss0= 0.0000 loss1= 3.8760 loss2=-3253.4646 lr= 0.000100
  1198. 2019-03-12 09:53:34,445 : epoch 126 , step 80 loss0= 0.0000 loss1= 5.3373 loss2=-3446.6919 lr= 0.000100
  1199. 2019-03-12 09:54:07,636 : epoch 126 , step 120 loss0= 0.0000 loss1= 3.9305 loss2=-3387.0625 lr= 0.000100
  1200. 2019-03-12 09:54:40,817 : epoch 126 , step 160 loss0= 0.0000 loss1= 4.8564 loss2=-3599.8716 lr= 0.000100
  1201. 2019-03-12 09:55:11,637 : epoch 127 , step 0 loss0= 0.0000 loss1= 3.4119 loss2=-3119.5732 lr= 0.000100
  1202. 2019-03-12 09:55:44,874 : epoch 127 , step 40 loss0= 0.0000 loss1= 5.0022 loss2=-3848.4568 lr= 0.000100
  1203. 2019-03-12 09:56:18,570 : epoch 127 , step 80 loss0= 0.0000 loss1= 5.1102 loss2=-3796.2029 lr= 0.000100
  1204. 2019-03-12 09:56:52,671 : epoch 127 , step 120 loss0= 0.0000 loss1= 4.5078 loss2=-3535.5930 lr= 0.000100
  1205. 2019-03-12 09:57:26,801 : epoch 127 , step 160 loss0= 0.0000 loss1= 5.9139 loss2=-3718.7871 lr= 0.000100
  1206. 2019-03-12 09:57:57,324 : epoch 128 , step 0 loss0= 0.0000 loss1= 4.6931 loss2=-3495.9900 lr= 0.000100
  1207. 2019-03-12 09:58:30,651 : epoch 128 , step 40 loss0= 0.0000 loss1= 4.0468 loss2=-3344.3474 lr= 0.000100
  1208. 2019-03-12 09:59:03,944 : epoch 128 , step 80 loss0= 0.0000 loss1= 4.2266 loss2=-3420.6411 lr= 0.000100
  1209. 2019-03-12 09:59:37,562 : epoch 128 , step 120 loss0= 0.0000 loss1= 4.0722 loss2=-3226.2500 lr= 0.000100
  1210. 2019-03-12 10:00:11,167 : epoch 128 , step 160 loss0= 0.0000 loss1= 4.0742 loss2=-3558.4739 lr= 0.000100
  1211. 2019-03-12 10:00:41,761 : epoch 129 , step 0 loss0= 0.0000 loss1= 4.9345 loss2=-3665.3235 lr= 0.000100
  1212. 2019-03-12 10:01:14,955 : epoch 129 , step 40 loss0= 0.0000 loss1= 3.8154 loss2=-3563.4441 lr= 0.000100
  1213. 2019-03-12 10:01:49,106 : epoch 129 , step 80 loss0= 0.0000 loss1= 4.8644 loss2=-3285.3535 lr= 0.000100
  1214. 2019-03-12 10:02:23,752 : epoch 129 , step 120 loss0= 0.0000 loss1= 5.6798 loss2=-3907.5083 lr= 0.000100
  1215. 2019-03-12 10:02:58,069 : epoch 129 , step 160 loss0= 0.0000 loss1= 4.9739 loss2=-3354.0955 lr= 0.000100
  1216. 2019-03-12 10:03:28,471 : epoch 130 , step 0 loss0= 0.0000 loss1= 3.7473 loss2=-3241.1458 lr= 0.000100
  1217. 2019-03-12 10:04:02,917 : epoch 130 , step 40 loss0= 0.0000 loss1= 3.8804 loss2=-3296.3691 lr= 0.000100
  1218. 2019-03-12 10:04:37,168 : epoch 130 , step 80 loss0= 0.0000 loss1= 4.1641 loss2=-3534.1892 lr= 0.000100
  1219. 2019-03-12 10:05:11,245 : epoch 130 , step 120 loss0= 0.0000 loss1= 4.1039 loss2=-3435.9282 lr= 0.000100
  1220. 2019-03-12 10:05:45,205 : epoch 130 , step 160 loss0= 0.0000 loss1= 5.0858 loss2=-3228.4333 lr= 0.000100
  1221. 2019-03-12 10:06:16,505 : epoch 131 , step 0 loss0= 0.0000 loss1= 4.5692 loss2=-3422.0415 lr= 0.000100
  1222. 2019-03-12 10:06:50,702 : epoch 131 , step 40 loss0= 0.0000 loss1= 3.7150 loss2=-3376.7129 lr= 0.000100
  1223. 2019-03-12 10:07:24,740 : epoch 131 , step 80 loss0= 0.0000 loss1= 4.8421 loss2=-3187.6191 lr= 0.000100
  1224. 2019-03-12 10:07:59,036 : epoch 131 , step 120 loss0= 0.0000 loss1= 3.5269 loss2=-3204.8560 lr= 0.000100
  1225. 2019-03-12 10:08:32,914 : epoch 131 , step 160 loss0= 0.0000 loss1= 4.1606 loss2=-3491.8340 lr= 0.000100
  1226. 2019-03-12 10:09:03,468 : epoch 132 , step 0 loss0= 0.0000 loss1= 4.4498 loss2=-3229.9248 lr= 0.000100
  1227. 2019-03-12 10:09:37,367 : epoch 132 , step 40 loss0= 0.0000 loss1= 4.1871 loss2=-3277.9243 lr= 0.000100
  1228. 2019-03-12 10:10:11,726 : epoch 132 , step 80 loss0= 0.0000 loss1= 4.4681 loss2=-3497.2637 lr= 0.000100
  1229. 2019-03-12 10:10:46,773 : epoch 132 , step 120 loss0= 0.0000 loss1= 4.9389 loss2=-3316.6665 lr= 0.000100
  1230. 2019-03-12 10:11:21,414 : epoch 132 , step 160 loss0= 0.0000 loss1= 5.6746 loss2=-3741.9822 lr= 0.000100
  1231. 2019-03-12 10:11:53,473 : epoch 133 , step 0 loss0= 0.0000 loss1= 5.1364 loss2=-3497.9556 lr= 0.000100
  1232. 2019-03-12 10:12:27,888 : epoch 133 , step 40 loss0= 0.0000 loss1= 3.8807 loss2=-3198.2759 lr= 0.000100
  1233. 2019-03-12 10:13:02,226 : epoch 133 , step 80 loss0= 0.0000 loss1= 4.6623 loss2=-3604.6580 lr= 0.000100
  1234. 2019-03-12 10:13:36,279 : epoch 133 , step 120 loss0= 0.0000 loss1= 4.5624 loss2=-3392.6794 lr= 0.000100
  1235. 2019-03-12 10:14:11,245 : epoch 133 , step 160 loss0= 0.0000 loss1= 5.4618 loss2=-3513.9785 lr= 0.000100
  1236. 2019-03-12 10:14:42,528 : epoch 134 , step 0 loss0= 0.0000 loss1= 4.1475 loss2=-3361.9766 lr= 0.000100
  1237. 2019-03-12 10:15:16,798 : epoch 134 , step 40 loss0= 0.0000 loss1= 4.4891 loss2=-3518.2466 lr= 0.000100
  1238. 2019-03-12 10:15:50,864 : epoch 134 , step 80 loss0= 0.0000 loss1= 4.9014 loss2=-3528.3984 lr= 0.000100
  1239. 2019-03-12 10:16:24,499 : epoch 134 , step 120 loss0= 0.0000 loss1= 5.3632 loss2=-3560.6985 lr= 0.000100
  1240. 2019-03-12 10:16:58,379 : epoch 134 , step 160 loss0= 0.0000 loss1= 4.7355 loss2=-3699.1843 lr= 0.000100
  1241. 2019-03-12 10:17:28,842 : epoch 135 , step 0 loss0= 0.0000 loss1= 3.6124 loss2=-3348.5173 lr= 0.000100
  1242. 2019-03-12 10:18:02,768 : epoch 135 , step 40 loss0= 0.0000 loss1= 4.8723 loss2=-3831.9602 lr= 0.000100
  1243. 2019-03-12 10:18:36,119 : epoch 135 , step 80 loss0= 0.0000 loss1= 3.7126 loss2=-3560.3899 lr= 0.000100
  1244. 2019-03-12 10:19:09,827 : epoch 135 , step 120 loss0= 0.0000 loss1= 4.6474 loss2=-3491.9824 lr= 0.000100
  1245. 2019-03-12 10:19:43,656 : epoch 135 , step 160 loss0= 0.0000 loss1= 4.1992 loss2=-3432.9712 lr= 0.000100
  1246. 2019-03-12 10:20:14,284 : epoch 136 , step 0 loss0= 0.0000 loss1= 5.0539 loss2=-3786.2512 lr= 0.000100
  1247. 2019-03-12 10:20:48,036 : epoch 136 , step 40 loss0= 0.0000 loss1= 4.8174 loss2=-3606.2166 lr= 0.000100
  1248. 2019-03-12 10:21:21,641 : epoch 136 , step 80 loss0= 0.0000 loss1= 4.4544 loss2=-3469.3679 lr= 0.000100
  1249. 2019-03-12 10:21:55,454 : epoch 136 , step 120 loss0= 0.0000 loss1= 4.0122 loss2=-3409.1672 lr= 0.000100
  1250. 2019-03-12 10:22:29,764 : epoch 136 , step 160 loss0= 0.0000 loss1= 4.2928 loss2=-3593.1111 lr= 0.000100
  1251. 2019-03-12 10:23:00,839 : epoch 137 , step 0 loss0= 0.0000 loss1= 4.7958 loss2=-3745.0701 lr= 0.000100
  1252. 2019-03-12 10:23:34,547 : epoch 137 , step 40 loss0= 0.0000 loss1= 4.6972 loss2=-3690.6243 lr= 0.000100
  1253. 2019-03-12 10:24:08,433 : epoch 137 , step 80 loss0= 0.0000 loss1= 5.7022 loss2=-3901.7297 lr= 0.000100
  1254. 2019-03-12 10:24:42,946 : epoch 137 , step 120 loss0= 0.0000 loss1= 4.5768 loss2=-3621.8455 lr= 0.000100
  1255. 2019-03-12 10:25:17,094 : epoch 137 , step 160 loss0= 0.0000 loss1= 4.2630 loss2=-3699.7954 lr= 0.000100
  1256. 2019-03-12 10:25:47,966 : epoch 138 , step 0 loss0= 0.0000 loss1= 5.1470 loss2=-3454.2031 lr= 0.000100
  1257. 2019-03-12 10:26:21,830 : epoch 138 , step 40 loss0= 0.0000 loss1= 3.9369 loss2=-3694.0234 lr= 0.000100
  1258. 2019-03-12 10:26:55,052 : epoch 138 , step 80 loss0= 0.0000 loss1= 3.8010 loss2=-3399.2188 lr= 0.000100
  1259. 2019-03-12 10:27:28,261 : epoch 138 , step 120 loss0= 0.0000 loss1= 4.2826 loss2=-3503.2285 lr= 0.000100
  1260. 2019-03-12 10:28:01,455 : epoch 138 , step 160 loss0= 0.0000 loss1= 4.1953 loss2=-3393.4875 lr= 0.000100
  1261. 2019-03-12 10:28:31,961 : epoch 139 , step 0 loss0= 0.0000 loss1= 4.1593 loss2=-3471.4119 lr= 0.000100
  1262. 2019-03-12 10:29:05,340 : epoch 139 , step 40 loss0= 0.0000 loss1= 3.8827 loss2=-3457.2590 lr= 0.000100
  1263. 2019-03-12 10:29:38,561 : epoch 139 , step 80 loss0= 0.0000 loss1= 3.6728 loss2=-3317.1016 lr= 0.000100
  1264. 2019-03-12 10:30:11,833 : epoch 139 , step 120 loss0= 0.0000 loss1= 5.3881 loss2=-3344.4666 lr= 0.000100
  1265. 2019-03-12 10:30:45,289 : epoch 139 , step 160 loss0= 0.0000 loss1= 5.1957 loss2=-3524.0591 lr= 0.000100
  1266. 2019-03-12 10:31:15,187 : epoch 140 , step 0 loss0= 0.0000 loss1= 3.8909 loss2=-3404.0630 lr= 0.000100
  1267. 2019-03-12 10:31:48,088 : epoch 140 , step 40 loss0= 0.0000 loss1= 5.1547 loss2=-3729.3918 lr= 0.000100
  1268. 2019-03-12 10:32:21,060 : epoch 140 , step 80 loss0= 0.0000 loss1= 5.3906 loss2=-3647.0474 lr= 0.000100
  1269. 2019-03-12 10:32:54,010 : epoch 140 , step 120 loss0= 0.0000 loss1= 4.9608 loss2=-3464.8872 lr= 0.000100
  1270. 2019-03-12 10:33:26,975 : epoch 140 , step 160 loss0= 0.0000 loss1= 4.1416 loss2=-3276.0518 lr= 0.000100
  1271. 2019-03-12 10:33:55,374 : ## train ##
  1272. 2019-03-12 10:38:07,246 : >> epoch 140 : average_loss= 0.6495, average_accuracy=82.177%, mean_weight_std= 0.1798, mean_normalized_std= 0.7027 mean_dist=2.264e+03
  1273. 2019-03-12 10:38:07,257 : >> epoch 140 : median_loss= 0.6308, median_accuracy=82.935%, median_weight_std= 0.1171, median_normalized_std= 0.0686 median_dist=2.252e+03
  1274. 2019-03-12 10:38:07,257 : >> epoch 140 : ensemble accuracy: 84.26%
  1275. 2019-03-12 10:38:07,258 : >> epoch 140 : loss histogram: (0.00,0.28, 18),(0.28,0.56, 0),(0.56,0.84, 4),(0.84,1.12, 0),(1.12,1.39, 18),
  1276. 2019-03-12 10:38:07,258 : >> epoch 140 : accuracy histogram: (0.00,33.26, 18),(33.26,66.52, 0),(66.52,99.77, 4),(99.77,133.03, 0),(133.03,166.29, 18),
  1277. 2019-03-12 10:38:07,263 : >> epoch 140 : weight std histogram: (0.0000,1.7409, 465065),(1.7409,3.4818, 1646),(3.4818,5.2226, 172),(5.2226,6.9635, 21),(6.9635,8.7044, 2),
  1278. 2019-03-12 10:38:07,268 : >> epoch 140 : weight normalized std histogram: (0.0000,32107.6844, 466905),(32107.6844,64215.3688, 0),(64215.3688,96323.0531, 0),(96323.0531,128430.7375, 0),(128430.7375,160538.4219, 1),
  1279. 2019-03-12 10:38:07,268 : >> epoch 140 : dist histogram: (2126.6353,2229.2121, 7),(2229.2121,2331.7889, 11),(2331.7889,2434.3657, 0),(2434.3657,2536.9425, 1),(2536.9425,2639.5193, 1),
  1280. 2019-03-12 10:38:07,269 : ## eval ##
  1281. 2019-03-12 10:42:16,991 : >> epoch 140 : average_loss= 0.5470, average_accuracy=85.436%, mean_weight_std= 0.1768, mean_normalized_std= 0.4307 mean_dist=2.264e+03
  1282. 2019-03-12 10:42:17,001 : >> epoch 140 : median_loss= 0.5253, median_accuracy=85.775%, median_weight_std= 0.115, median_normalized_std= 0.0672 median_dist=2.252e+03
  1283. 2019-03-12 10:42:17,002 : >> epoch 140 : ensemble accuracy: 87.22%
  1284. 2019-03-12 10:42:17,002 : >> epoch 140 : loss histogram: (0.00,0.23, 18),(0.23,0.47, 0),(0.47,0.70, 4),(0.70,0.94, 0),(0.94,1.17, 18),
  1285. 2019-03-12 10:42:17,002 : >> epoch 140 : accuracy histogram: (0.00,34.44, 18),(34.44,68.89, 0),(68.89,103.33, 4),(103.33,137.78, 0),(137.78,172.22, 18),
  1286. 2019-03-12 10:42:17,008 : >> epoch 140 : weight std histogram: (0.0000,2.0378, 465541),(2.0378,4.0756, 1215),(4.0756,6.1133, 132),(6.1133,8.1511, 15),(8.1511,10.1889, 3),
  1287. 2019-03-12 10:42:17,012 : >> epoch 140 : weight normalized std histogram: (0.0000,7631.0453, 466904),(7631.0453,15262.0906, 1),(15262.0906,22893.1359, 0),(22893.1359,30524.1813, 0),(30524.1813,38155.2266, 1),
  1288. 2019-03-12 10:42:17,013 : >> epoch 140 : dist histogram: (2034.3345,2131.1500, 1),(2131.1500,2227.9656, 7),(2227.9656,2324.7812, 7),(2324.7812,2421.5968, 4),(2421.5968,2518.4124, 1),
  1289. 2019-03-12 10:42:17,068 : >> epoch 140 : dist pair-wise
  1290. 2019-03-12 10:42:17,074 :
  1291. [[ 0. 231.644 307.058 231.202 291.61 265.986 332.028 417.774
  1292. 203.795 239.064 286.682 195.712 228.025 385.51 200.588 289.942
  1293. 169.101 181.753 223.119 203.834]
  1294. [ 231.644 0. 237.757 202.017 301.783 221.201 229.94 362.786
  1295. 264.484 160.982 223.485 264.256 159.45 279.498 151.03 219.05
  1296. 242.81 195.361 208.182 277.334]
  1297. [ 307.058 237.757 0. 230.027 442.18 197.368 220.964 228.99
  1298. 386.555 202.096 262.47 366.548 299.633 237.659 222.648 367.907
  1299. 346.93 314.892 220.666 409.036]
  1300. [ 231.202 202.017 230.027 0. 376.761 198.563 277.976 311.25
  1301. 267.516 202.534 308.763 256.716 246.499 314.168 184.741 315.136
  1302. 285.681 253.264 179.046 316.54 ]
  1303. [ 291.61 301.783 442.18 376.761 0. 447.414 384.18 589.528
  1304. 283.778 341.126 302.867 289.22 284.785 447.187 321.744 284.343
  1305. 255.526 265.472 408.572 239.623]
  1306. [ 265.986 221.201 197.368 198.563 447.414 0. 287.718 267.532
  1307. 322.611 216.095 309.283 324.518 258.334 306.938 207.638 333.702
  1308. 304.743 263.731 131.985 355.131]
  1309. [ 332.028 229.94 220.964 277.976 384.18 287.718 0. 316.63
  1310. 358.489 210.028 222.278 355.652 295.28 160.228 267.702 320.301
  1311. 344.323 293.389 292.128 367.804]
  1312. [ 417.774 362.786 228.99 311.25 589.528 267.532 316.63 0.
  1313. 474.821 319.523 411.194 468.343 418.619 309.874 349.877 486.883
  1314. 470.415 438.872 292.2 524.407]
  1315. [ 203.795 264.484 386.555 267.516 283.778 322.611 358.489 474.821
  1316. 0. 296.173 342.7 165.176 227.772 408.8 251.723 253.222
  1317. 210.615 195.01 273.374 161.338]
  1318. [ 239.064 160.982 202.096 202.534 341.126 216.095 210.028 319.523
  1319. 296.173 0. 228.072 299.386 222.114 282.824 187.993 276.604
  1320. 272.488 209.309 201.878 319.669]
  1321. [ 286.682 223.485 262.47 308.763 302.867 309.283 222.278 411.194
  1322. 342.7 228.072 0. 342.686 259.293 266.503 226.912 268.226
  1323. 283.581 256.212 314.824 332.939]
  1324. [ 195.712 264.256 366.548 256.716 289.22 324.518 355.652 468.343
  1325. 165.176 299.386 342.686 0. 246.44 403.582 248.885 302.528
  1326. 221.553 222.33 276.342 178.646]
  1327. [ 228.025 159.45 299.633 246.499 284.785 258.334 295.28 418.619
  1328. 227.772 222.114 259.293 246.44 0. 337.264 173.009 229.601
  1329. 225.116 197.89 227.611 247.544]
  1330. [ 385.51 279.498 237.659 314.168 447.187 306.938 160.228 309.874
  1331. 408.8 282.824 266.503 403.582 337.264 0. 308.615 357.869
  1332. 395.487 361.576 330.492 418.868]
  1333. [ 200.588 151.03 222.648 184.741 321.744 207.638 267.702 349.877
  1334. 251.723 187.993 226.912 248.885 173.009 308.615 0. 260.518
  1335. 224.089 197.906 184.095 278.956]
  1336. [ 289.942 219.05 367.907 315.136 284.343 333.702 320.301 486.883
  1337. 253.222 276.604 268.226 302.528 229.601 357.869 260.518 0.
  1338. 273.099 218.985 325.677 269.1 ]
  1339. [ 169.101 242.81 346.93 285.681 255.526 304.743 344.323 470.415
  1340. 210.615 272.488 283.581 221.553 225.116 395.487 224.089 273.099
  1341. 0. 171.682 253.746 160.084]
  1342. [ 181.753 195.361 314.892 253.264 265.472 263.731 293.389 438.872
  1343. 195.01 209.309 256.212 222.33 197.89 361.576 197.906 218.985
  1344. 171.682 0. 228.427 185.435]
  1345. [ 223.119 208.182 220.666 179.046 408.572 131.985 292.128 292.2
  1346. 273.374 201.878 314.824 276.342 227.611 330.492 184.095 325.677
  1347. 253.746 228.427 0. 309.73 ]
  1348. [ 203.834 277.334 409.036 316.54 239.623 355.131 367.804 524.407
  1349. 161.338 319.669 332.939 178.646 247.544 418.868 278.956 269.1
  1350. 160.084 185.435 309.73 0. ]]
  1351. 2019-03-12 10:42:18,437 : epoch 141 , step 0 loss0= 0.0000 loss1= 5.5510 loss2=-3555.5691 lr= 0.000100
  1352. 2019-03-12 10:42:52,095 : epoch 141 , step 40 loss0= 0.0000 loss1= 5.0372 loss2=-3500.7141 lr= 0.000100
  1353. 2019-03-12 10:43:26,236 : epoch 141 , step 80 loss0= 0.0000 loss1= 3.6439 loss2=-3422.2778 lr= 0.000100
  1354. 2019-03-12 10:43:59,819 : epoch 141 , step 120 loss0= 0.0000 loss1= 3.9717 loss2=-3575.1501 lr= 0.000100
  1355. 2019-03-12 10:44:34,125 : epoch 141 , step 160 loss0= 0.0000 loss1= 4.6843 loss2=-3319.2458 lr= 0.000100
  1356. 2019-03-12 10:45:04,588 : epoch 142 , step 0 loss0= 0.0000 loss1= 4.1516 loss2=-3390.6309 lr= 0.000100
  1357. 2019-03-12 10:45:38,525 : epoch 142 , step 40 loss0= 0.0000 loss1= 4.9904 loss2=-3582.6938 lr= 0.000100
  1358. 2019-03-12 10:46:12,623 : epoch 142 , step 80 loss0= 0.0000 loss1= 4.3939 loss2=-3435.6699 lr= 0.000100
  1359. 2019-03-12 10:46:46,108 : epoch 142 , step 120 loss0= 0.0000 loss1= 5.4461 loss2=-3441.3835 lr= 0.000100
  1360. 2019-03-12 10:47:19,485 : epoch 142 , step 160 loss0= 0.0000 loss1= 4.1366 loss2=-3610.9329 lr= 0.000100
  1361. 2019-03-12 10:47:49,786 : epoch 143 , step 0 loss0= 0.0000 loss1= 4.1277 loss2=-3589.5649 lr= 0.000100
  1362. 2019-03-12 10:48:23,555 : epoch 143 , step 40 loss0= 0.0000 loss1= 4.7780 loss2=-3392.0396 lr= 0.000100
  1363. 2019-03-12 10:48:57,022 : epoch 143 , step 80 loss0= 0.0000 loss1= 3.5750 loss2=-3654.5845 lr= 0.000100
  1364. 2019-03-12 10:49:30,952 : epoch 143 , step 120 loss0= 0.0000 loss1= 4.7989 loss2=-3626.8562 lr= 0.000100
  1365. 2019-03-12 10:50:04,865 : epoch 143 , step 160 loss0= 0.0000 loss1= 4.7678 loss2=-3707.8613 lr= 0.000100
  1366. 2019-03-12 10:50:35,614 : epoch 144 , step 0 loss0= 0.0000 loss1= 3.4187 loss2=-3485.2192 lr= 0.000100
  1367. 2019-03-12 10:51:09,277 : epoch 144 , step 40 loss0= 0.0000 loss1= 4.7502 loss2=-3427.8457 lr= 0.000100
  1368. 2019-03-12 10:51:43,659 : epoch 144 , step 80 loss0= 0.0000 loss1= 4.7633 loss2=-3618.2598 lr= 0.000100
  1369. 2019-03-12 10:52:17,719 : epoch 144 , step 120 loss0= 0.0000 loss1= 4.4381 loss2=-3375.0046 lr= 0.000100
  1370. 2019-03-12 10:52:51,569 : epoch 144 , step 160 loss0= 0.0000 loss1= 3.7797 loss2=-3299.1013 lr= 0.000100
  1371. 2019-03-12 10:53:22,324 : epoch 145 , step 0 loss0= 0.0000 loss1= 3.0699 loss2=-3662.5645 lr= 0.000100
  1372. 2019-03-12 10:53:56,038 : epoch 145 , step 40 loss0= 0.0000 loss1= 4.5373 loss2=-3455.5447 lr= 0.000100
  1373. 2019-03-12 10:54:29,735 : epoch 145 , step 80 loss0= 0.0000 loss1= 3.8324 loss2=-3669.1548 lr= 0.000100
  1374. 2019-03-12 10:55:03,984 : epoch 145 , step 120 loss0= 0.0000 loss1= 4.6787 loss2=-3417.1594 lr= 0.000100
  1375. 2019-03-12 10:55:37,746 : epoch 145 , step 160 loss0= 0.0000 loss1= 3.5982 loss2=-3540.5857 lr= 0.000100
  1376. 2019-03-12 10:56:08,348 : epoch 146 , step 0 loss0= 0.0000 loss1= 3.4974 loss2=-3321.8813 lr= 0.000100
  1377. 2019-03-12 10:56:41,745 : epoch 146 , step 40 loss0= 0.0000 loss1= 3.8457 loss2=-3369.0024 lr= 0.000100
  1378. 2019-03-12 10:57:15,300 : epoch 146 , step 80 loss0= 0.0000 loss1= 3.2204 loss2=-3042.6685 lr= 0.000100
  1379. 2019-03-12 10:57:48,697 : epoch 146 , step 120 loss0= 0.0000 loss1= 3.5869 loss2=-3501.2732 lr= 0.000100
  1380. 2019-03-12 10:58:22,056 : epoch 146 , step 160 loss0= 0.0000 loss1= 5.1566 loss2=-3761.1296 lr= 0.000100
  1381. 2019-03-12 10:58:52,392 : epoch 147 , step 0 loss0= 0.0000 loss1= 4.9336 loss2=-3663.2810 lr= 0.000100
  1382. 2019-03-12 10:59:26,129 : epoch 147 , step 40 loss0= 0.0000 loss1= 3.3901 loss2=-3597.9172 lr= 0.000100
  1383. 2019-03-12 10:59:59,770 : epoch 147 , step 80 loss0= 0.0000 loss1= 3.4968 loss2=-3283.0344 lr= 0.000100
  1384. 2019-03-12 11:00:33,561 : epoch 147 , step 120 loss0= 0.0000 loss1= 3.7913 loss2=-3722.6582 lr= 0.000100
  1385. 2019-03-12 11:01:08,278 : epoch 147 , step 160 loss0= 0.0000 loss1= 4.0084 loss2=-3537.1091 lr= 0.000100
  1386. 2019-03-12 11:01:40,142 : epoch 148 , step 0 loss0= 0.0000 loss1= 4.2321 loss2=-3271.0603 lr= 0.000100
  1387. 2019-03-12 11:02:15,130 : epoch 148 , step 40 loss0= 0.0000 loss1= 4.8220 loss2=-3825.8943 lr= 0.000100
  1388. 2019-03-12 11:02:49,158 : epoch 148 , step 80 loss0= 0.0000 loss1= 2.7583 loss2=-3386.0708 lr= 0.000100
  1389. 2019-03-12 11:03:23,789 : epoch 148 , step 120 loss0= 0.0000 loss1= 3.9754 loss2=-3532.9524 lr= 0.000100
  1390. 2019-03-12 11:03:58,379 : epoch 148 , step 160 loss0= 0.0000 loss1= 5.0438 loss2=-3465.2351 lr= 0.000100
  1391. 2019-03-12 11:04:29,541 : epoch 149 , step 0 loss0= 0.0000 loss1= 3.9464 loss2=-3409.4146 lr= 0.000100
  1392. 2019-03-12 11:05:03,740 : epoch 149 , step 40 loss0= 0.0000 loss1= 4.4965 loss2=-3674.9446 lr= 0.000100
  1393. 2019-03-12 11:05:38,556 : epoch 149 , step 80 loss0= 0.0000 loss1= 4.9770 loss2=-3693.1169 lr= 0.000100
  1394. 2019-03-12 11:06:13,283 : epoch 149 , step 120 loss0= 0.0000 loss1= 4.9614 loss2=-3717.8540 lr= 0.000100
  1395. 2019-03-12 11:06:48,023 : epoch 149 , step 160 loss0= 0.0000 loss1= 3.1604 loss2=-3225.9268 lr= 0.000100
  1396. 2019-03-12 11:07:19,249 : epoch 150 , step 0 loss0= 0.0000 loss1= 2.9424 loss2=-3450.6782 lr= 0.000100
  1397. 2019-03-12 11:07:52,207 : epoch 150 , step 40 loss0= 0.0000 loss1= 5.5929 loss2=-4025.0964 lr= 0.000100
  1398. 2019-03-12 11:08:24,993 : epoch 150 , step 80 loss0= 0.0000 loss1= 4.9377 loss2=-3600.2271 lr= 0.000100
  1399. 2019-03-12 11:08:57,803 : epoch 150 , step 120 loss0= 0.0000 loss1= 3.3720 loss2=-3361.6960 lr= 0.000100
  1400. 2019-03-12 11:09:30,980 : epoch 150 , step 160 loss0= 0.0000 loss1= 3.9691 loss2=-3521.0352 lr= 0.000100
  1401. 2019-03-12 11:10:01,351 : epoch 151 , step 0 loss0= 0.0000 loss1= 3.9453 loss2=-3591.9895 lr= 0.000100
  1402. 2019-03-12 11:10:35,449 : epoch 151 , step 40 loss0= 0.0000 loss1= 3.9108 loss2=-3336.8418 lr= 0.000100
  1403. 2019-03-12 11:11:09,814 : epoch 151 , step 80 loss0= 0.0000 loss1= 4.9992 loss2=-3519.6418 lr= 0.000100
  1404. 2019-03-12 11:11:43,741 : epoch 151 , step 120 loss0= 0.0000 loss1= 5.0569 loss2=-3815.5818 lr= 0.000100
  1405. 2019-03-12 11:12:17,485 : epoch 151 , step 160 loss0= 0.0000 loss1= 4.6887 loss2=-3438.9563 lr= 0.000100
  1406. 2019-03-12 11:12:47,855 : epoch 152 , step 0 loss0= 0.0000 loss1= 3.1893 loss2=-3521.3652 lr= 0.000100
  1407. 2019-03-12 11:13:21,999 : epoch 152 , step 40 loss0= 0.0000 loss1= 3.4429 loss2=-3572.5735 lr= 0.000100
  1408. 2019-03-12 11:13:56,280 : epoch 152 , step 80 loss0= 0.0000 loss1= 4.6795 loss2=-3487.4966 lr= 0.000100
  1409. 2019-03-12 11:14:29,976 : epoch 152 , step 120 loss0= 0.0000 loss1= 4.8028 loss2=-3182.3762 lr= 0.000100
  1410. 2019-03-12 11:15:03,368 : epoch 152 , step 160 loss0= 0.0000 loss1= 4.7405 loss2=-3555.1160 lr= 0.000100
  1411. 2019-03-12 11:15:34,398 : epoch 153 , step 0 loss0= 0.0000 loss1= 3.6147 loss2=-3405.9958 lr= 0.000100
  1412. 2019-03-12 11:16:08,106 : epoch 153 , step 40 loss0= 0.0000 loss1= 3.5141 loss2=-3135.2336 lr= 0.000100
  1413. 2019-03-12 11:16:41,999 : epoch 153 , step 80 loss0= 0.0000 loss1= 4.8077 loss2=-3835.0605 lr= 0.000100
  1414. 2019-03-12 11:17:15,682 : epoch 153 , step 120 loss0= 0.0000 loss1= 4.2852 loss2=-3378.7976 lr= 0.000100
  1415. 2019-03-12 11:17:49,389 : epoch 153 , step 160 loss0= 0.0000 loss1= 3.5430 loss2=-3592.3638 lr= 0.000100
  1416. 2019-03-12 11:18:20,388 : epoch 154 , step 0 loss0= 0.0000 loss1= 4.5874 loss2=-3608.6958 lr= 0.000100
  1417. 2019-03-12 11:18:54,053 : epoch 154 , step 40 loss0= 0.0000 loss1= 3.5396 loss2=-3299.0154 lr= 0.000100
  1418. 2019-03-12 11:19:28,264 : epoch 154 , step 80 loss0= 0.0000 loss1= 3.5903 loss2=-3602.5422 lr= 0.000100
  1419. 2019-03-12 11:20:01,726 : epoch 154 , step 120 loss0= 0.0000 loss1= 4.9426 loss2=-3292.3118 lr= 0.000100
  1420. 2019-03-12 11:20:36,045 : epoch 154 , step 160 loss0= 0.0000 loss1= 4.8763 loss2=-3431.2922 lr= 0.000100
  1421. 2019-03-12 11:21:07,086 : epoch 155 , step 0 loss0= 0.0000 loss1= 3.9658 loss2=-3448.7605 lr= 0.000100
  1422. 2019-03-12 11:21:40,764 : epoch 155 , step 40 loss0= 0.0000 loss1= 2.9923 loss2=-3425.1313 lr= 0.000100
  1423. 2019-03-12 11:22:14,446 : epoch 155 , step 80 loss0= 0.0000 loss1= 4.9448 loss2=-3527.9341 lr= 0.000100
  1424. 2019-03-12 11:22:48,437 : epoch 155 , step 120 loss0= 0.0000 loss1= 3.9770 loss2=-3495.0676 lr= 0.000100
  1425. 2019-03-12 11:23:22,470 : epoch 155 , step 160 loss0= 0.0000 loss1= 4.8202 loss2=-3393.3110 lr= 0.000100
  1426. 2019-03-12 11:23:53,407 : epoch 156 , step 0 loss0= 0.0000 loss1= 4.4301 loss2=-3381.9719 lr= 0.000100
  1427. 2019-03-12 11:24:27,289 : epoch 156 , step 40 loss0= 0.0000 loss1= 4.5038 loss2=-3499.3718 lr= 0.000100
  1428. 2019-03-12 11:25:00,739 : epoch 156 , step 80 loss0= 0.0000 loss1= 2.3503 loss2=-3298.3086 lr= 0.000100
  1429. 2019-03-12 11:25:34,055 : epoch 156 , step 120 loss0= 0.0000 loss1= 3.7602 loss2=-3417.6899 lr= 0.000100
  1430. 2019-03-12 11:26:07,482 : epoch 156 , step 160 loss0= 0.0000 loss1= 4.9704 loss2=-3615.9939 lr= 0.000100
  1431. 2019-03-12 11:26:38,143 : epoch 157 , step 0 loss0= 0.0000 loss1= 3.8328 loss2=-3521.0339 lr= 0.000100
  1432. 2019-03-12 11:27:11,746 : epoch 157 , step 40 loss0= 0.0000 loss1= 3.7453 loss2=-3186.0884 lr= 0.000100
  1433. 2019-03-12 11:27:45,309 : epoch 157 , step 80 loss0= 0.0000 loss1= 5.5404 loss2=-3395.7195 lr= 0.000100
  1434. 2019-03-12 11:28:18,719 : epoch 157 , step 120 loss0= 0.0000 loss1= 5.0422 loss2=-3523.5696 lr= 0.000100
  1435. 2019-03-12 11:28:52,032 : epoch 157 , step 160 loss0= 0.0000 loss1= 4.3134 loss2=-3520.5566 lr= 0.000100
  1436. 2019-03-12 11:29:22,274 : epoch 158 , step 0 loss0= 0.0000 loss1= 4.3442 loss2=-3274.9951 lr= 0.000100
  1437. 2019-03-12 11:29:55,989 : epoch 158 , step 40 loss0= 0.0000 loss1= 4.2442 loss2=-3612.3013 lr= 0.000100
  1438. 2019-03-12 11:30:29,815 : epoch 158 , step 80 loss0= 0.0000 loss1= 3.7533 loss2=-3724.7458 lr= 0.000100
  1439. 2019-03-12 11:31:03,938 : epoch 158 , step 120 loss0= 0.0000 loss1= 5.0190 loss2=-3909.6724 lr= 0.000100
  1440. 2019-03-12 11:31:37,364 : epoch 158 , step 160 loss0= 0.0000 loss1= 3.3770 loss2=-3258.8623 lr= 0.000100
  1441. 2019-03-12 11:32:07,945 : epoch 159 , step 0 loss0= 0.0000 loss1= 3.2841 loss2=-3474.9973 lr= 0.000100
  1442. 2019-03-12 11:32:41,883 : epoch 159 , step 40 loss0= 0.0000 loss1= 4.8767 loss2=-3537.4438 lr= 0.000100
  1443. 2019-03-12 11:33:15,215 : epoch 159 , step 80 loss0= 0.0000 loss1= 4.6967 loss2=-3810.5181 lr= 0.000100
  1444. 2019-03-12 11:33:48,744 : epoch 159 , step 120 loss0= 0.0000 loss1= 4.3845 loss2=-3641.7688 lr= 0.000100
  1445. 2019-03-12 11:34:22,478 : epoch 159 , step 160 loss0= 0.0000 loss1= 2.9080 loss2=-3581.2063 lr= 0.000100
  1446. 2019-03-12 11:34:53,057 : epoch 160 , step 0 loss0= 0.0000 loss1= 4.0850 loss2=-3541.7859 lr= 0.000100
  1447. 2019-03-12 11:35:27,086 : epoch 160 , step 40 loss0= 0.0000 loss1= 3.9871 loss2=-3583.8521 lr= 0.000100
  1448. 2019-03-12 11:36:01,671 : epoch 160 , step 80 loss0= 0.0000 loss1= 2.8281 loss2=-3007.1716 lr= 0.000100
  1449. 2019-03-12 11:36:36,301 : epoch 160 , step 120 loss0= 0.0000 loss1= 4.6849 loss2=-3557.3860 lr= 0.000100
  1450. 2019-03-12 11:37:10,213 : epoch 160 , step 160 loss0= 0.0000 loss1= 3.1547 loss2=-3332.6191 lr= 0.000100
  1451. 2019-03-12 11:37:39,360 : ## train ##
  1452. 2019-03-12 11:41:54,746 : >> epoch 160 : average_loss= 0.6575, average_accuracy=82.519%, mean_weight_std= 0.1919, mean_normalized_std= 0.4219 mean_dist=2.357e+03
  1453. 2019-03-12 11:41:54,756 : >> epoch 160 : median_loss= 0.6449, median_accuracy=83.220%, median_weight_std= 0.1229, median_normalized_std= 0.0680 median_dist=2.319e+03
  1454. 2019-03-12 11:41:54,756 : >> epoch 160 : ensemble accuracy: 84.72%
  1455. 2019-03-12 11:41:54,756 : >> epoch 160 : loss histogram: (0.00,0.28, 18),(0.28,0.56, 0),(0.56,0.84, 4),(0.84,1.12, 0),(1.12,1.40, 18),
  1456. 2019-03-12 11:41:54,757 : >> epoch 160 : accuracy histogram: (0.00,33.42, 18),(33.42,66.84, 0),(66.84,100.26, 4),(100.26,133.68, 0),(133.68,167.10, 18),
  1457. 2019-03-12 11:41:54,762 : >> epoch 160 : weight std histogram: (0.0000,1.8991, 464953),(1.8991,3.7982, 1692),(3.7982,5.6972, 215),(5.6972,7.5963, 39),(7.5963,9.4954, 7),
  1458. 2019-03-12 11:41:54,767 : >> epoch 160 : weight normalized std histogram: (0.0000,3500.7715, 466901),(3500.7715,7001.5430, 3),(7001.5430,10502.3145, 0),(10502.3145,14003.0859, 0),(14003.0859,17503.8574, 2),
  1459. 2019-03-12 11:41:54,767 : >> epoch 160 : dist histogram: (2191.0540,2262.6255, 4),(2262.6255,2334.1970, 7),(2334.1970,2405.7686, 2),(2405.7686,2477.3401, 3),(2477.3401,2548.9116, 4),
  1460. 2019-03-12 11:41:54,767 : ## eval ##
  1461. 2019-03-12 11:46:09,378 : >> epoch 160 : average_loss= 0.5660, average_accuracy=85.670%, mean_weight_std= 0.1677, mean_normalized_std= 0.4401 mean_dist=2.358e+03
  1462. 2019-03-12 11:46:09,390 : >> epoch 160 : median_loss= 0.5504, median_accuracy=85.180%, median_weight_std= 0.1064, median_normalized_std= 0.0592 median_dist=2.388e+03
  1463. 2019-03-12 11:46:09,390 : >> epoch 160 : ensemble accuracy: 87.58%
  1464. 2019-03-12 11:46:09,390 : >> epoch 160 : loss histogram: (0.00,0.24, 18),(0.24,0.47, 0),(0.47,0.71, 4),(0.71,0.95, 0),(0.95,1.18, 18),
  1465. 2019-03-12 11:46:09,390 : >> epoch 160 : accuracy histogram: (0.00,34.61, 18),(34.61,69.22, 0),(69.22,103.84, 4),(103.84,138.45, 0),(138.45,173.06, 18),
  1466. 2019-03-12 11:46:09,396 : >> epoch 160 : weight std histogram: (0.0000,1.8932, 465064),(1.8932,3.7864, 1598),(3.7864,5.6796, 211),(5.6796,7.5728, 27),(7.5728,9.4660, 6),
  1467. 2019-03-12 11:46:09,401 : >> epoch 160 : weight normalized std histogram: (0.0000,7981.3742, 466904),(7981.3742,15962.7484, 0),(15962.7484,23944.1227, 1),(23944.1227,31925.4969, 0),(31925.4969,39906.8711, 1),
  1468. 2019-03-12 11:46:09,401 : >> epoch 160 : dist histogram: (2147.3071,2216.0311, 1),(2216.0311,2284.7550, 3),(2284.7550,2353.4789, 3),(2353.4789,2422.2028, 9),(2422.2028,2490.9268, 4),
  1469. 2019-03-12 11:46:09,455 : >> epoch 160 : dist pair-wise
  1470. 2019-03-12 11:46:09,461 :
  1471. [[ 0. 247.212 243.276 218.023 216.693 218.848 276.413 379.833
  1472. 232.121 402.063 225.753 376.311 269.22 357.155 235.708 321.794
  1473. 274.486 348.632 342.26 491.446]
  1474. [ 247.212 0. 293.952 224.416 199.298 224.164 286.084 245.057
  1475. 191.493 313.189 179.249 366.038 292.472 276.313 195.125 239.44
  1476. 317.131 240.046 324.46 380.848]
  1477. [ 243.276 293.952 0. 217.655 226.952 200.179 221.582 354.683
  1478. 263.609 397.468 243.868 263.602 214.848 339.175 278.245 298.477
  1479. 187.129 352.034 293.645 477.326]
  1480. [ 218.023 224.416 217.655 0. 179.187 174.101 210.304 299.942
  1481. 212.036 362.58 182.709 307.429 266.366 269.676 195.143 237.894
  1482. 238.903 270.257 257.797 399.549]
  1483. [ 216.693 199.298 226.952 179.187 0. 149.681 192.103 294.195
  1484. 217.147 352.033 169.712 295.295 240.952 271.643 175.707 230.569
  1485. 246.793 282.985 260.428 393.495]
  1486. [ 218.848 224.164 200.179 174.101 149.681 0. 220.178 294.034
  1487. 196.801 370.95 185.451 282.465 241.122 292.273 202.704 266.659
  1488. 218.588 291.192 242.96 422.717]
  1489. [ 276.413 286.084 221.582 210.304 192.103 220.178 0. 322.066
  1490. 260.751 396.642 250.636 224.552 238.649 251.838 237.144 219.202
  1491. 238.146 339.934 215.195 379.209]
  1492. [ 379.833 245.057 354.683 299.942 294.195 294.034 322.066 0.
  1493. 220.304 241.161 273.758 372.5 384.653 273.449 239.755 242.428
  1494. 378.707 202.808 314.019 294.267]
  1495. [ 232.121 191.493 263.609 212.036 217.147 196.801 260.751 220.304
  1496. 0. 311.447 206.448 338.968 308.173 246.971 186.877 232.887
  1497. 293.037 247.89 270.378 369.605]
  1498. [ 402.063 313.189 397.468 362.58 352.033 370.95 396.642 241.161
  1499. 311.447 0. 315.189 462.924 428.365 357.785 298.447 308.048
  1500. 432.907 200.632 438.549 348.38 ]
  1501. [ 225.753 179.249 243.868 182.709 169.712 185.451 250.636 273.758
  1502. 206.448 315.189 0. 343.162 280.936 293.169 175.866 241.522
  1503. 278.965 252.085 314.706 406.211]
  1504. [ 376.311 366.038 263.602 307.429 295.295 282.465 224.552 372.5
  1505. 338.968 462.924 343.162 0. 229.527 323.382 347.84 338.158
  1506. 197.931 416.232 211.888 424.016]
  1507. [ 269.22 292.472 214.848 266.366 240.952 241.122 238.649 384.653
  1508. 308.173 428.365 280.936 229.527 0. 363.48 311.062 342.542
  1509. 178.54 389.992 284.078 478.744]
  1510. [ 357.155 276.313 339.175 269.676 271.643 292.273 251.838 273.449
  1511. 246.971 357.785 293.169 323.382 363.48 0. 262.104 193.131
  1512. 338.849 286.988 247.581 250.96 ]
  1513. [ 235.708 195.125 278.245 195.143 175.707 202.704 237.144 239.755
  1514. 186.877 298.447 175.866 347.84 311.062 262.104 0. 206.521
  1515. 301.957 232.324 291.124 366.142]
  1516. [ 321.794 239.44 298.477 237.894 230.569 266.659 219.202 242.428
  1517. 232.887 308.048 241.522 338.158 342.542 193.131 206.521 0.
  1518. 330.404 248.154 291.254 288.834]
  1519. [ 274.486 317.131 187.129 238.903 246.793 218.588 238.146 378.707
  1520. 293.037 432.907 278.965 197.931 178.54 338.849 301.957 330.404
  1521. 0. 375.063 261.543 468.003]
  1522. [ 348.632 240.046 352.034 270.257 282.985 291.192 339.934 202.808
  1523. 247.89 200.632 252.085 416.232 389.992 286.988 232.324 248.154
  1524. 375.063 0. 372.999 304.423]
  1525. [ 342.26 324.46 293.645 257.797 260.428 242.96 215.195 314.019
  1526. 270.378 438.549 314.706 211.888 284.078 247.581 291.124 291.254
  1527. 261.543 372.999 0. 355.332]
  1528. [ 491.446 380.848 477.326 399.549 393.495 422.717 379.209 294.267
  1529. 369.605 348.38 406.211 424.016 478.744 250.96 366.142 288.834
  1530. 468.003 304.423 355.332 0. ]]
  1531. 2019-03-12 11:46:10,833 : epoch 161 , step 0 loss0= 0.0000 loss1= 3.4795 loss2=-3291.6626 lr= 0.000100
  1532. 2019-03-12 11:46:45,371 : epoch 161 , step 40 loss0= 0.0000 loss1= 4.1907 loss2=-3628.7227 lr= 0.000100
  1533. 2019-03-12 11:47:19,732 : epoch 161 , step 80 loss0= 0.0000 loss1= 3.6568 loss2=-3554.3755 lr= 0.000100
  1534. 2019-03-12 11:47:53,410 : epoch 161 , step 120 loss0= 0.0000 loss1= 4.4338 loss2=-3444.1125 lr= 0.000100
  1535. 2019-03-12 11:48:26,733 : epoch 161 , step 160 loss0= 0.0000 loss1= 4.3619 loss2=-3764.9707 lr= 0.000100
  1536. 2019-03-12 11:48:57,052 : epoch 162 , step 0 loss0= 0.0000 loss1= 4.0235 loss2=-3601.0344 lr= 0.000100
  1537. 2019-03-12 11:49:31,284 : epoch 162 , step 40 loss0= 0.0000 loss1= 3.9794 loss2=-3400.2781 lr= 0.000100
  1538. 2019-03-12 11:50:04,885 : epoch 162 , step 80 loss0= 0.0000 loss1= 4.9638 loss2=-3736.8640 lr= 0.000100
  1539. 2019-03-12 11:50:39,048 : epoch 162 , step 120 loss0= 0.0000 loss1= 4.0042 loss2=-3523.8904 lr= 0.000100
  1540. 2019-03-12 11:51:13,015 : epoch 162 , step 160 loss0= 0.0000 loss1= 2.6882 loss2=-3291.7556 lr= 0.000100
  1541. 2019-03-12 11:51:43,505 : epoch 163 , step 0 loss0= 0.0000 loss1= 3.4992 loss2=-3714.7168 lr= 0.000100
  1542. 2019-03-12 11:52:17,887 : epoch 163 , step 40 loss0= 0.0000 loss1= 4.2708 loss2=-3708.8162 lr= 0.000100
  1543. 2019-03-12 11:52:52,328 : epoch 163 , step 80 loss0= 0.0000 loss1= 3.3426 loss2=-3037.4216 lr= 0.000100
  1544. 2019-03-12 11:53:26,752 : epoch 163 , step 120 loss0= 0.0000 loss1= 3.8176 loss2=-3650.5879 lr= 0.000100
  1545. 2019-03-12 11:54:00,519 : epoch 163 , step 160 loss0= 0.0000 loss1= 3.5068 loss2=-3657.4392 lr= 0.000100
  1546. 2019-03-12 11:54:30,848 : epoch 164 , step 0 loss0= 0.0000 loss1= 4.4715 loss2=-3537.9399 lr= 0.000100
  1547. 2019-03-12 11:55:04,725 : epoch 164 , step 40 loss0= 0.0000 loss1= 3.7164 loss2=-3582.6594 lr= 0.000100
  1548. 2019-03-12 11:55:38,492 : epoch 164 , step 80 loss0= 0.0000 loss1= 4.3672 loss2=-3653.0974 lr= 0.000100
  1549. 2019-03-12 11:56:12,142 : epoch 164 , step 120 loss0= 0.0000 loss1= 4.6087 loss2=-3748.3989 lr= 0.000100
  1550. 2019-03-12 11:56:45,854 : epoch 164 , step 160 loss0= 0.0000 loss1= 3.4760 loss2=-3494.2361 lr= 0.000100
  1551. 2019-03-12 11:57:16,385 : epoch 165 , step 0 loss0= 0.0000 loss1= 4.5371 loss2=-3774.2493 lr= 0.000100
  1552. 2019-03-12 11:57:50,682 : epoch 165 , step 40 loss0= 0.0000 loss1= 4.1792 loss2=-3768.0984 lr= 0.000100
  1553. 2019-03-12 11:58:24,895 : epoch 165 , step 80 loss0= 0.0000 loss1= 4.0045 loss2=-3313.6360 lr= 0.000100
  1554. 2019-03-12 11:58:58,729 : epoch 165 , step 120 loss0= 0.0000 loss1= 4.9081 loss2=-3868.4316 lr= 0.000100
  1555. 2019-03-12 11:59:32,887 : epoch 165 , step 160 loss0= 0.0000 loss1= 4.0633 loss2=-3728.3474 lr= 0.000100
  1556. 2019-03-12 12:00:03,590 : epoch 166 , step 0 loss0= 0.0000 loss1= 3.1961 loss2=-3271.3357 lr= 0.000100
  1557. 2019-03-12 12:00:37,297 : epoch 166 , step 40 loss0= 0.0000 loss1= 3.0132 loss2=-3124.7361 lr= 0.000100
  1558. 2019-03-12 12:01:11,057 : epoch 166 , step 80 loss0= 0.0000 loss1= 3.8806 loss2=-3407.8308 lr= 0.000100
  1559. 2019-03-12 12:01:44,685 : epoch 166 , step 120 loss0= 0.0000 loss1= 3.2101 loss2=-3102.2466 lr= 0.000100
  1560. 2019-03-12 12:02:18,409 : epoch 166 , step 160 loss0= 0.0000 loss1= 3.4651 loss2=-3526.0786 lr= 0.000100
  1561. 2019-03-12 12:02:48,917 : epoch 167 , step 0 loss0= 0.0000 loss1= 3.4762 loss2=-3807.0356 lr= 0.000100
  1562. 2019-03-12 12:03:22,691 : epoch 167 , step 40 loss0= 0.0000 loss1= 5.1864 loss2=-3702.9539 lr= 0.000100
  1563. 2019-03-12 12:03:56,057 : epoch 167 , step 80 loss0= 0.0000 loss1= 4.0664 loss2=-3456.8750 lr= 0.000100
  1564. 2019-03-12 12:04:30,131 : epoch 167 , step 120 loss0= 0.0000 loss1= 3.7995 loss2=-3654.5701 lr= 0.000100
  1565. 2019-03-12 12:05:04,678 : epoch 167 , step 160 loss0= 0.0000 loss1= 4.3007 loss2=-3515.3352 lr= 0.000100
  1566. 2019-03-12 12:05:35,932 : epoch 168 , step 0 loss0= 0.0000 loss1= 3.0429 loss2=-3341.2283 lr= 0.000100
  1567. 2019-03-12 12:06:08,714 : epoch 168 , step 40 loss0= 0.0000 loss1= 4.9832 loss2=-3667.9028 lr= 0.000100
  1568. 2019-03-12 12:06:41,446 : epoch 168 , step 80 loss0= 0.0000 loss1= 3.3273 loss2=-3560.3796 lr= 0.000100
  1569. 2019-03-12 12:07:14,319 : epoch 168 , step 120 loss0= 0.0000 loss1= 5.0166 loss2=-3865.0142 lr= 0.000100
  1570. 2019-03-12 12:07:47,655 : epoch 168 , step 160 loss0= 0.0000 loss1= 4.0191 loss2=-3723.6326 lr= 0.000100
  1571. 2019-03-12 12:08:18,027 : epoch 169 , step 0 loss0= 0.0000 loss1= 4.6991 loss2=-3571.8027 lr= 0.000100
  1572. 2019-03-12 12:08:51,942 : epoch 169 , step 40 loss0= 0.0000 loss1= 3.6197 loss2=-3433.3884 lr= 0.000100
  1573. 2019-03-12 12:09:25,687 : epoch 169 , step 80 loss0= 0.0000 loss1= 3.4733 loss2=-3221.3418 lr= 0.000100
  1574. 2019-03-12 12:09:59,458 : epoch 169 , step 120 loss0= 0.0000 loss1= 4.2810 loss2=-3695.9900 lr= 0.000100
  1575. 2019-03-12 12:10:33,425 : epoch 169 , step 160 loss0= 0.0000 loss1= 4.6765 loss2=-3569.4688 lr= 0.000100
  1576. 2019-03-12 12:11:04,177 : epoch 170 , step 0 loss0= 0.0000 loss1= 4.8179 loss2=-3683.7200 lr= 0.000100
  1577. 2019-03-12 12:11:38,238 : epoch 170 , step 40 loss0= 0.0000 loss1= 3.9142 loss2=-3563.2888 lr= 0.000100
  1578. 2019-03-12 12:12:11,911 : epoch 170 , step 80 loss0= 0.0000 loss1= 4.2661 loss2=-3562.2576 lr= 0.000100
  1579. 2019-03-12 12:12:45,256 : epoch 170 , step 120 loss0= 0.0000 loss1= 4.3432 loss2=-3724.1255 lr= 0.000100
  1580. 2019-03-12 12:13:18,631 : epoch 170 , step 160 loss0= 0.0000 loss1= 3.8642 loss2=-3397.5305 lr= 0.000100
  1581. 2019-03-12 12:13:48,856 : epoch 171 , step 0 loss0= 0.0000 loss1= 3.3751 loss2=-3450.1313 lr= 0.000100
  1582. 2019-03-12 12:14:22,319 : epoch 171 , step 40 loss0= 0.0000 loss1= 3.1188 loss2=-3488.1548 lr= 0.000100
  1583. 2019-03-12 12:14:55,780 : epoch 171 , step 80 loss0= 0.0000 loss1= 4.7925 loss2=-3809.9548 lr= 0.000100
  1584. 2019-03-12 12:15:29,640 : epoch 171 , step 120 loss0= 0.0000 loss1= 3.2449 loss2=-3303.4810 lr= 0.000100
  1585. 2019-03-12 12:16:03,634 : epoch 171 , step 160 loss0= 0.0000 loss1= 4.2907 loss2=-3506.0710 lr= 0.000100
  1586. 2019-03-12 12:16:34,310 : epoch 172 , step 0 loss0= 0.0000 loss1= 3.4238 loss2=-3407.8950 lr= 0.000100
  1587. 2019-03-12 12:17:08,028 : epoch 172 , step 40 loss0= 0.0000 loss1= 2.9054 loss2=-3357.2793 lr= 0.000100
  1588. 2019-03-12 12:17:41,359 : epoch 172 , step 80 loss0= 0.0000 loss1= 3.6376 loss2=-3517.2532 lr= 0.000100
  1589. 2019-03-12 12:18:15,614 : epoch 172 , step 120 loss0= 0.0000 loss1= 3.7273 loss2=-3546.4751 lr= 0.000100
  1590. 2019-03-12 12:18:49,645 : epoch 172 , step 160 loss0= 0.0000 loss1= 4.1216 loss2=-3447.3574 lr= 0.000100
  1591. 2019-03-12 12:19:20,589 : epoch 173 , step 0 loss0= 0.0000 loss1= 3.3855 loss2=-3420.4353 lr= 0.000100
  1592. 2019-03-12 12:19:54,235 : epoch 173 , step 40 loss0= 0.0000 loss1= 3.8600 loss2=-3524.7114 lr= 0.000100
  1593. 2019-03-12 12:20:28,515 : epoch 173 , step 80 loss0= 0.0000 loss1= 3.1770 loss2=-3628.5610 lr= 0.000100
  1594. 2019-03-12 12:21:02,677 : epoch 173 , step 120 loss0= 0.0000 loss1= 4.6483 loss2=-3620.4817 lr= 0.000100
  1595. 2019-03-12 12:21:37,106 : epoch 173 , step 160 loss0= 0.0000 loss1= 4.9934 loss2=-3556.3613 lr= 0.000100
  1596. 2019-03-12 12:22:08,158 : epoch 174 , step 0 loss0= 0.0000 loss1= 3.6273 loss2=-3507.9172 lr= 0.000100
  1597. 2019-03-12 12:22:41,941 : epoch 174 , step 40 loss0= 0.0000 loss1= 4.7616 loss2=-3629.1790 lr= 0.000100
  1598. 2019-03-12 12:23:15,349 : epoch 174 , step 80 loss0= 0.0000 loss1= 2.7087 loss2=-3433.0701 lr= 0.000100
  1599. 2019-03-12 12:23:48,711 : epoch 174 , step 120 loss0= 0.0000 loss1= 3.8681 loss2=-3748.8123 lr= 0.000100
  1600. 2019-03-12 12:24:23,057 : epoch 174 , step 160 loss0= 0.0000 loss1= 3.1019 loss2=-3330.7781 lr= 0.000100
  1601. 2019-03-12 12:24:53,714 : epoch 175 , step 0 loss0= 0.0000 loss1= 3.8701 loss2=-3701.5024 lr= 0.000100
  1602. 2019-03-12 12:25:27,557 : epoch 175 , step 40 loss0= 0.0000 loss1= 2.2331 loss2=-3092.9626 lr= 0.000100
  1603. 2019-03-12 12:26:01,050 : epoch 175 , step 80 loss0= 0.0000 loss1= 3.7675 loss2=-3632.8950 lr= 0.000100
  1604. 2019-03-12 12:26:34,461 : epoch 175 , step 120 loss0= 0.0000 loss1= 4.8665 loss2=-3862.3716 lr= 0.000100
  1605. 2019-03-12 12:27:07,874 : epoch 175 , step 160 loss0= 0.0000 loss1= 4.5354 loss2=-3621.3477 lr= 0.000100
  1606. 2019-03-12 12:27:38,362 : epoch 176 , step 0 loss0= 0.0000 loss1= 2.7009 loss2=-3393.0950 lr= 0.000100
  1607. 2019-03-12 12:28:11,787 : epoch 176 , step 40 loss0= 0.0000 loss1= 2.5274 loss2=-3552.8748 lr= 0.000100
  1608. 2019-03-12 12:28:45,207 : epoch 176 , step 80 loss0= 0.0000 loss1= 3.6269 loss2=-3708.1990 lr= 0.000100
  1609. 2019-03-12 12:29:18,643 : epoch 176 , step 120 loss0= 0.0000 loss1= 5.1004 loss2=-3910.8950 lr= 0.000100
  1610. 2019-03-12 12:29:52,031 : epoch 176 , step 160 loss0= 0.0000 loss1= 3.8399 loss2=-3570.4758 lr= 0.000100
  1611. 2019-03-12 12:30:22,405 : epoch 177 , step 0 loss0= 0.0000 loss1= 3.0344 loss2=-3636.9766 lr= 0.000100
  1612. 2019-03-12 12:30:56,713 : epoch 177 , step 40 loss0= 0.0000 loss1= 4.3013 loss2=-3503.4797 lr= 0.000100
  1613. 2019-03-12 12:31:30,603 : epoch 177 , step 80 loss0= 0.0000 loss1= 4.6763 loss2=-3733.5134 lr= 0.000100
  1614. 2019-03-12 12:32:04,329 : epoch 177 , step 120 loss0= 0.0000 loss1= 2.4973 loss2=-3609.2754 lr= 0.000100
  1615. 2019-03-12 12:32:38,643 : epoch 177 , step 160 loss0= 0.0000 loss1= 3.8111 loss2=-3766.8521 lr= 0.000100
  1616. 2019-03-12 12:33:09,475 : epoch 178 , step 0 loss0= 0.0000 loss1= 3.3487 loss2=-3620.4641 lr= 0.000100
  1617. 2019-03-12 12:33:42,692 : epoch 178 , step 40 loss0= 0.0000 loss1= 3.6785 loss2=-3438.3982 lr= 0.000100
  1618. 2019-03-12 12:34:16,236 : epoch 178 , step 80 loss0= 0.0000 loss1= 2.7354 loss2=-3283.4822 lr= 0.000100
  1619. 2019-03-12 12:34:49,486 : epoch 178 , step 120 loss0= 0.0000 loss1= 4.9896 loss2=-3702.3462 lr= 0.000100
  1620. 2019-03-12 12:35:22,288 : epoch 178 , step 160 loss0= 0.0000 loss1= 4.4966 loss2=-3408.2598 lr= 0.000100
  1621. 2019-03-12 12:35:52,096 : epoch 179 , step 0 loss0= 0.0000 loss1= 2.8973 loss2=-3389.7791 lr= 0.000100
  1622. 2019-03-12 12:36:26,038 : epoch 179 , step 40 loss0= 0.0000 loss1= 2.9630 loss2=-3639.1980 lr= 0.000100
  1623. 2019-03-12 12:37:00,444 : epoch 179 , step 80 loss0= 0.0000 loss1= 3.7315 loss2=-3522.8672 lr= 0.000100
  1624. 2019-03-12 12:37:34,789 : epoch 179 , step 120 loss0= 0.0000 loss1= 5.4700 loss2=-3464.2715 lr= 0.000100
  1625. 2019-03-12 12:38:09,202 : epoch 179 , step 160 loss0= 0.0000 loss1= 3.7076 loss2=-3409.3757 lr= 0.000100
  1626. 2019-03-12 12:38:39,982 : epoch 180 , step 0 loss0= 0.0000 loss1= 3.8051 loss2=-3471.2739 lr= 0.000100
  1627. 2019-03-12 12:39:14,092 : epoch 180 , step 40 loss0= 0.0000 loss1= 3.5390 loss2=-3445.8572 lr= 0.000100
  1628. 2019-03-12 12:39:48,044 : epoch 180 , step 80 loss0= 0.0000 loss1= 4.0555 loss2=-3763.3618 lr= 0.000100
  1629. 2019-03-12 12:40:22,620 : epoch 180 , step 120 loss0= 0.0000 loss1= 3.0155 loss2=-3329.9927 lr= 0.000100
  1630. 2019-03-12 12:40:56,198 : epoch 180 , step 160 loss0= 0.0000 loss1= 4.4459 loss2=-3671.1716 lr= 0.000100
  1631. 2019-03-12 12:41:25,165 : ## train ##
  1632. 2019-03-12 12:45:40,048 : >> epoch 180 : average_loss= 0.6719, average_accuracy=82.472%, mean_weight_std= 0.1989, mean_normalized_std= 0.4613 mean_dist=2.439e+03
  1633. 2019-03-12 12:45:40,059 : >> epoch 180 : median_loss= 0.6929, median_accuracy=81.570%, median_weight_std= 0.1249, median_normalized_std= 0.0672 median_dist=2.407e+03
  1634. 2019-03-12 12:45:40,059 : >> epoch 180 : ensemble accuracy: 84.53%
  1635. 2019-03-12 12:45:40,059 : >> epoch 180 : loss histogram: (0.00,0.28, 18),(0.28,0.56, 0),(0.56,0.83, 4),(0.83,1.11, 0),(1.11,1.39, 18),
  1636. 2019-03-12 12:45:40,060 : >> epoch 180 : accuracy histogram: (0.00,33.42, 18),(33.42,66.84, 0),(66.84,100.27, 4),(100.27,133.69, 0),(133.69,167.11, 18),
  1637. 2019-03-12 12:45:40,065 : >> epoch 180 : weight std histogram: (0.0000,2.2352, 465084),(2.2352,4.4704, 1568),(4.4704,6.7056, 217),(6.7056,8.9409, 30),(8.9409,11.1761, 7),
  1638. 2019-03-12 12:45:40,070 : >> epoch 180 : weight normalized std histogram: (0.0000,6821.9773, 466904),(6821.9773,13643.9547, 1),(13643.9547,20465.9320, 0),(20465.9320,27287.9094, 0),(27287.9094,34109.8867, 1),
  1639. 2019-03-12 12:45:40,070 : >> epoch 180 : dist histogram: (2299.1443,2373.8495, 6),(2373.8495,2448.5546, 7),(2448.5546,2523.2598, 2),(2523.2598,2597.9650, 3),(2597.9650,2672.6702, 2),
  1640. 2019-03-12 12:45:40,070 : ## eval ##
  1641. 2019-03-12 12:49:54,113 : >> epoch 180 : average_loss= 0.5849, average_accuracy=85.648%, mean_weight_std= 0.2166, mean_normalized_std= 1.3610 mean_dist=2.438e+03
  1642. 2019-03-12 12:49:54,121 : >> epoch 180 : median_loss= 0.5669, median_accuracy=86.270%, median_weight_std= 0.1389, median_normalized_std= 0.0738 median_dist=2.397e+03
  1643. 2019-03-12 12:49:54,121 : >> epoch 180 : ensemble accuracy: 87.35%
  1644. 2019-03-12 12:49:54,121 : >> epoch 180 : loss histogram: (0.00,0.25, 18),(0.25,0.50, 0),(0.50,0.75, 4),(0.75,0.99, 0),(0.99,1.24, 18),
  1645. 2019-03-12 12:49:54,121 : >> epoch 180 : accuracy histogram: (0.00,34.64, 18),(34.64,69.28, 0),(69.28,103.92, 4),(103.92,138.56, 0),(138.56,173.20, 18),
  1646. 2019-03-12 12:49:54,127 : >> epoch 180 : weight std histogram: (0.0000,1.9514, 464411),(1.9514,3.9028, 2162),(3.9028,5.8542, 280),(5.8542,7.8057, 43),(7.8057,9.7571, 10),
  1647. 2019-03-12 12:49:54,132 : >> epoch 180 : weight normalized std histogram: (0.0000,92568.1000, 466905),(92568.1000,185136.2000, 0),(185136.2000,277704.3000, 0),(277704.3000,370272.4000, 0),(370272.4000,462840.5000, 1),
  1648. 2019-03-12 12:49:54,132 : >> epoch 180 : dist histogram: (2261.8892,2357.4566, 7),(2357.4566,2453.0240, 6),(2453.0240,2548.5915, 3),(2548.5915,2644.1589, 2),(2644.1589,2739.7263, 2),
  1649. 2019-03-12 12:49:54,185 : >> epoch 180 : dist pair-wise
  1650. 2019-03-12 12:49:54,190 :
  1651. [[ 0. 376.857 245.997 317.842 258.472 363.054 493.221 197.391
  1652. 349.061 460.021 206.393 195.316 248.607 250.672 504.719 270.624
  1653. 373.751 234.803 276.261 203.361]
  1654. [ 376.857 0. 282.325 427.169 421.293 389.898 454.286 357.455
  1655. 202.233 590.351 385.309 361.893 219.891 436.379 366.412 439.457
  1656. 260.166 255.998 291.03 313.441]
  1657. [ 245.997 282.325 0. 263.035 285.378 268.373 412.91 233.589
  1658. 264.974 502.102 260.268 241.889 187.408 300.698 403.05 339.527
  1659. 303.463 208.223 193.877 250.77 ]
  1660. [ 317.842 427.169 263.035 0. 325.715 208.742 308.52 264.423
  1661. 343.46 424.307 333.437 358.343 349.463 402.004 385.071 369.831
  1662. 337.22 326.648 314.108 338.215]
  1663. [ 258.472 421.293 285.378 325.715 0. 356.341 509.707 314.297
  1664. 392.769 352.627 199.382 203.628 347.412 223.166 544.223 157.575
  1665. 419.531 289.76 342.496 272.351]
  1666. [ 363.054 389.898 268.373 208.742 356.341 0. 335.249 300.866
  1667. 314.121 491.158 338.138 362.346 339.439 420.857 413.513 413.564
  1668. 339.288 320.483 297.733 344.064]
  1669. [ 493.221 454.286 412.91 308.52 509.707 335.249 0. 412.335
  1670. 363.407 534.385 516.318 535.614 462.692 600.309 321.583 528.904
  1671. 334.906 455.445 443.775 458.248]
  1672. [ 197.391 357.455 233.589 264.423 314.297 300.866 412.335 0.
  1673. 288.911 471.475 275.285 275.459 248.624 320.873 431.098 319.665
  1674. 313.744 235.146 261.878 198.858]
  1675. [ 349.061 202.233 264.974 343.46 392.769 314.121 363.407 288.911
  1676. 0. 538.8 370.477 355.024 245.717 435.648 324.354 402.861
  1677. 192.004 230.585 295.087 268.282]
  1678. [ 460.021 590.351 502.102 424.307 352.627 491.158 534.385 471.475
  1679. 538.8 0. 448.336 461.186 552.217 454.989 577.214 320.853
  1680. 548.564 522.788 534.758 481.39 ]
  1681. [ 206.393 385.309 260.268 333.437 199.382 338.138 516.318 275.285
  1682. 370.477 448.336 0. 145.38 278.388 215.069 550.126 241.832
  1683. 406.885 239.404 278.488 213.532]
  1684. [ 195.316 361.893 241.889 358.343 203.628 362.346 535.614 275.459
  1685. 355.024 461.186 145.38 0. 259.651 194.247 547.735 229.095
  1686. 409.057 229.669 288.907 214.472]
  1687. [ 248.607 219.891 187.408 349.463 347.412 339.439 462.692 248.624
  1688. 245.717 552.217 278.388 259.651 0. 331.713 405.048 374.771
  1689. 298.625 201.089 186.629 232.738]
  1690. [ 250.672 436.379 300.698 402.004 223.166 420.857 600.309 320.873
  1691. 435.648 454.989 215.069 194.247 331.713 0. 600.996 242.829
  1692. 483.308 313.52 353.577 287.718]
  1693. [ 504.719 366.412 403.05 385.071 544.223 413.513 321.583 431.098
  1694. 324.354 577.214 550.126 547.735 405.048 600.996 0. 566.659
  1695. 331.602 455.591 415.848 485.67 ]
  1696. [ 270.624 439.457 339.527 369.831 157.575 413.564 528.904 319.665
  1697. 402.861 320.853 241.832 229.095 374.771 242.829 566.659 0.
  1698. 430.507 312.775 382.055 262.468]
  1699. [ 373.751 260.166 303.463 337.22 419.531 339.288 334.906 313.744
  1700. 192.004 548.564 406.885 409.057 298.625 483.308 331.602 430.507
  1701. 0. 264.117 318.717 321.413]
  1702. [ 234.803 255.998 208.223 326.648 289.76 320.483 455.445 235.146
  1703. 230.585 522.788 239.404 229.669 201.089 313.52 455.591 312.775
  1704. 264.117 0. 246.879 185.254]
  1705. [ 276.261 291.03 193.877 314.108 342.496 297.733 443.775 261.878
  1706. 295.087 534.758 278.488 288.907 186.629 353.577 415.848 382.055
  1707. 318.717 246.879 0. 277.004]
  1708. [ 203.361 313.441 250.77 338.215 272.351 344.064 458.248 198.858
  1709. 268.282 481.39 213.532 214.472 232.738 287.718 485.67 262.468
  1710. 321.413 185.254 277.004 0. ]]
  1711. 2019-03-12 12:49:55,555 : epoch 181 , step 0 loss0= 0.0000 loss1= 3.4664 loss2=-3376.3279 lr= 0.000100
  1712. 2019-03-12 12:50:29,383 : epoch 181 , step 40 loss0= 0.0000 loss1= 3.4449 loss2=-3322.7798 lr= 0.000100
  1713. 2019-03-12 12:51:03,154 : epoch 181 , step 80 loss0= 0.0000 loss1= 2.8139 loss2=-3087.3877 lr= 0.000100
  1714. 2019-03-12 12:51:36,807 : epoch 181 , step 120 loss0= 0.0000 loss1= 3.2743 loss2=-3536.0527 lr= 0.000100
  1715. 2019-03-12 12:52:10,401 : epoch 181 , step 160 loss0= 0.0000 loss1= 2.7005 loss2=-3382.5027 lr= 0.000100
  1716. 2019-03-12 12:52:40,683 : epoch 182 , step 0 loss0= 0.0000 loss1= 3.6726 loss2=-3699.2009 lr= 0.000100
  1717. 2019-03-12 12:53:14,556 : epoch 182 , step 40 loss0= 0.0000 loss1= 3.7412 loss2=-3505.8066 lr= 0.000100
  1718. 2019-03-12 12:53:48,097 : epoch 182 , step 80 loss0= 0.0000 loss1= 2.5576 loss2=-3305.7361 lr= 0.000100
  1719. 2019-03-12 12:54:21,393 : epoch 182 , step 120 loss0= 0.0000 loss1= 3.7904 loss2=-3632.9570 lr= 0.000100
  1720. 2019-03-12 12:54:54,753 : epoch 182 , step 160 loss0= 0.0000 loss1= 3.9033 loss2=-3814.5762 lr= 0.000100
  1721. 2019-03-12 12:55:25,134 : epoch 183 , step 0 loss0= 0.0000 loss1= 4.4270 loss2=-3556.2129 lr= 0.000100
  1722. 2019-03-12 12:55:58,749 : epoch 183 , step 40 loss0= 0.0000 loss1= 5.1887 loss2=-3812.5396 lr= 0.000100
  1723. 2019-03-12 12:56:32,209 : epoch 183 , step 80 loss0= 0.0000 loss1= 3.4334 loss2=-3720.8875 lr= 0.000100
  1724. 2019-03-12 12:57:05,591 : epoch 183 , step 120 loss0= 0.0000 loss1= 3.9058 loss2=-3717.3149 lr= 0.000100
  1725. 2019-03-12 12:57:39,879 : epoch 183 , step 160 loss0= 0.0000 loss1= 3.3385 loss2=-3665.2615 lr= 0.000100
  1726. 2019-03-12 12:58:10,594 : epoch 184 , step 0 loss0= 0.0000 loss1= 4.6068 loss2=-3697.2810 lr= 0.000100
  1727. 2019-03-12 12:58:44,163 : epoch 184 , step 40 loss0= 0.0000 loss1= 4.4220 loss2=-3562.7175 lr= 0.000100
  1728. 2019-03-12 12:59:18,047 : epoch 184 , step 80 loss0= 0.0000 loss1= 3.8931 loss2=-3599.3845 lr= 0.000100
  1729. 2019-03-12 12:59:51,678 : epoch 184 , step 120 loss0= 0.0000 loss1= 4.5261 loss2=-3718.1602 lr= 0.000100
  1730. 2019-03-12 13:00:25,018 : epoch 184 , step 160 loss0= 0.0000 loss1= 4.4289 loss2=-3608.2668 lr= 0.000100
  1731. 2019-03-12 13:00:55,765 : epoch 185 , step 0 loss0= 0.0000 loss1= 3.9608 loss2=-3599.5696 lr= 0.000100
  1732. 2019-03-12 13:01:29,215 : epoch 185 , step 40 loss0= 0.0000 loss1= 3.2433 loss2=-3683.2078 lr= 0.000100
  1733. 2019-03-12 13:02:03,185 : epoch 185 , step 80 loss0= 0.0000 loss1= 3.4885 loss2=-3724.2556 lr= 0.000100
  1734. 2019-03-12 13:02:36,652 : epoch 185 , step 120 loss0= 0.0000 loss1= 3.3685 loss2=-3490.0876 lr= 0.000100
  1735. 2019-03-12 13:03:10,397 : epoch 185 , step 160 loss0= 0.0000 loss1= 4.2176 loss2=-3488.3455 lr= 0.000100
  1736. 2019-03-12 13:03:40,976 : epoch 186 , step 0 loss0= 0.0000 loss1= 3.8714 loss2=-3432.2192 lr= 0.000100
  1737. 2019-03-12 13:04:14,074 : epoch 186 , step 40 loss0= 0.0000 loss1= 3.7940 loss2=-3666.0298 lr= 0.000100
  1738. 2019-03-12 13:04:47,137 : epoch 186 , step 80 loss0= 0.0000 loss1= 3.7494 loss2=-3628.1418 lr= 0.000100
  1739. 2019-03-12 13:05:20,339 : epoch 186 , step 120 loss0= 0.0000 loss1= 2.4957 loss2=-3654.9187 lr= 0.000100
  1740. 2019-03-12 13:05:54,043 : epoch 186 , step 160 loss0= 0.0000 loss1= 2.9912 loss2=-3698.0984 lr= 0.000100
  1741. 2019-03-12 13:06:24,639 : epoch 187 , step 0 loss0= 0.0000 loss1= 2.4355 loss2=-3572.8118 lr= 0.000100
  1742. 2019-03-12 13:06:58,598 : epoch 187 , step 40 loss0= 0.0000 loss1= 3.2243 loss2=-3600.4109 lr= 0.000100
  1743. 2019-03-12 13:07:32,319 : epoch 187 , step 80 loss0= 0.0000 loss1= 2.9381 loss2=-3608.0037 lr= 0.000100
  1744. 2019-03-12 13:08:05,721 : epoch 187 , step 120 loss0= 0.0000 loss1= 5.0113 loss2=-3694.7349 lr= 0.000100
  1745. 2019-03-12 13:08:39,343 : epoch 187 , step 160 loss0= 0.0000 loss1= 2.4040 loss2=-3368.4607 lr= 0.000100
  1746. 2019-03-12 13:09:10,192 : epoch 188 , step 0 loss0= 0.0000 loss1= 4.5211 loss2=-3681.1704 lr= 0.000100
  1747. 2019-03-12 13:09:43,964 : epoch 188 , step 40 loss0= 0.0000 loss1= 3.8783 loss2=-3704.5369 lr= 0.000100
  1748. 2019-03-12 13:10:18,077 : epoch 188 , step 80 loss0= 0.0000 loss1= 4.8462 loss2=-3565.2312 lr= 0.000100
  1749. 2019-03-12 13:10:51,691 : epoch 188 , step 120 loss0= 0.0000 loss1= 4.6165 loss2=-3872.6531 lr= 0.000100
  1750. 2019-03-12 13:11:25,195 : epoch 188 , step 160 loss0= 0.0000 loss1= 3.3436 loss2=-3321.5681 lr= 0.000100
  1751. 2019-03-12 13:11:55,428 : epoch 189 , step 0 loss0= 0.0000 loss1= 4.7498 loss2=-3714.7625 lr= 0.000100
  1752. 2019-03-12 13:12:29,308 : epoch 189 , step 40 loss0= 0.0000 loss1= 3.5873 loss2=-3481.6853 lr= 0.000100
  1753. 2019-03-12 13:13:02,887 : epoch 189 , step 80 loss0= 0.0000 loss1= 2.7970 loss2=-3358.6560 lr= 0.000100
  1754. 2019-03-12 13:13:36,797 : epoch 189 , step 120 loss0= 0.0000 loss1= 3.1876 loss2=-3356.2876 lr= 0.000100
  1755. 2019-03-12 13:14:10,226 : epoch 189 , step 160 loss0= 0.0000 loss1= 4.2000 loss2=-3720.4490 lr= 0.000100
  1756. 2019-03-12 13:14:40,650 : epoch 190 , step 0 loss0= 0.0000 loss1= 3.3260 loss2=-3809.5879 lr= 0.000100
  1757. 2019-03-12 13:15:15,016 : epoch 190 , step 40 loss0= 0.0000 loss1= 2.7078 loss2=-3689.8025 lr= 0.000100
  1758. 2019-03-12 13:15:49,157 : epoch 190 , step 80 loss0= 0.0000 loss1= 3.4382 loss2=-3618.4890 lr= 0.000100
  1759. 2019-03-12 13:16:22,942 : epoch 190 , step 120 loss0= 0.0000 loss1= 3.2396 loss2=-3673.9531 lr= 0.000100
  1760. 2019-03-12 13:16:56,355 : epoch 190 , step 160 loss0= 0.0000 loss1= 4.7617 loss2=-3268.7888 lr= 0.000100
  1761. 2019-03-12 13:17:26,652 : epoch 191 , step 0 loss0= 0.0000 loss1= 4.1664 loss2=-3639.2488 lr= 0.000100
  1762. 2019-03-12 13:18:00,555 : epoch 191 , step 40 loss0= 0.0000 loss1= 3.7803 loss2=-3925.7610 lr= 0.000100
  1763. 2019-03-12 13:18:34,074 : epoch 191 , step 80 loss0= 0.0000 loss1= 2.5975 loss2=-3331.2585 lr= 0.000100
  1764. 2019-03-12 13:19:08,025 : epoch 191 , step 120 loss0= 0.0000 loss1= 2.8225 loss2=-3636.3181 lr= 0.000100
  1765. 2019-03-12 13:19:41,726 : epoch 191 , step 160 loss0= 0.0000 loss1= 2.8729 loss2=-3469.2766 lr= 0.000100
  1766. 2019-03-12 13:20:12,369 : epoch 192 , step 0 loss0= 0.0000 loss1= 4.9059 loss2=-3713.7546 lr= 0.000100
  1767. 2019-03-12 13:20:46,321 : epoch 192 , step 40 loss0= 0.0000 loss1= 4.8164 loss2=-3579.0559 lr= 0.000100
  1768. 2019-03-12 13:21:20,181 : epoch 192 , step 80 loss0= 0.0000 loss1= 3.4488 loss2=-3667.7878 lr= 0.000100
  1769. 2019-03-12 13:21:53,716 : epoch 192 , step 120 loss0= 0.0000 loss1= 3.2929 loss2=-3626.4304 lr= 0.000100
  1770. 2019-03-12 13:22:27,680 : epoch 192 , step 160 loss0= 0.0000 loss1= 3.4037 loss2=-3388.9041 lr= 0.000100
  1771. 2019-03-12 13:22:58,732 : epoch 193 , step 0 loss0= 0.0000 loss1= 3.7644 loss2=-3529.3066 lr= 0.000100
  1772. 2019-03-12 13:23:33,013 : epoch 193 , step 40 loss0= 0.0000 loss1= 3.0948 loss2=-3917.3142 lr= 0.000100
  1773. 2019-03-12 13:24:06,952 : epoch 193 , step 80 loss0= 0.0000 loss1= 2.8518 loss2=-3372.1189 lr= 0.000100
  1774. 2019-03-12 13:24:41,368 : epoch 193 , step 120 loss0= 0.0000 loss1= 3.5978 loss2=-3680.2583 lr= 0.000100
  1775. 2019-03-12 13:25:15,442 : epoch 193 , step 160 loss0= 0.0000 loss1= 2.5750 loss2=-3293.2461 lr= 0.000100
  1776. 2019-03-12 13:25:45,835 : epoch 194 , step 0 loss0= 0.0000 loss1= 3.0951 loss2=-3662.7693 lr= 0.000100
  1777. 2019-03-12 13:26:19,783 : epoch 194 , step 40 loss0= 0.0000 loss1= 3.3278 loss2=-3646.0532 lr= 0.000100
  1778. 2019-03-12 13:26:53,983 : epoch 194 , step 80 loss0= 0.0000 loss1= 3.5824 loss2=-3861.9724 lr= 0.000100
  1779. 2019-03-12 13:27:27,960 : epoch 194 , step 120 loss0= 0.0000 loss1= 4.7355 loss2=-3784.6523 lr= 0.000100
  1780. 2019-03-12 13:28:02,012 : epoch 194 , step 160 loss0= 0.0000 loss1= 3.6333 loss2=-3784.4016 lr= 0.000100
  1781. 2019-03-12 13:28:32,902 : epoch 195 , step 0 loss0= 0.0000 loss1= 3.5275 loss2=-3682.6660 lr= 0.000100
  1782. 2019-03-12 13:29:06,451 : epoch 195 , step 40 loss0= 0.0000 loss1= 3.8670 loss2=-3879.1353 lr= 0.000100
  1783. 2019-03-12 13:29:40,847 : epoch 195 , step 80 loss0= 0.0000 loss1= 3.9333 loss2=-3608.1099 lr= 0.000100
  1784. 2019-03-12 13:30:15,142 : epoch 195 , step 120 loss0= 0.0000 loss1= 4.5864 loss2=-3873.2136 lr= 0.000100
  1785. 2019-03-12 13:30:49,001 : epoch 195 , step 160 loss0= 0.0000 loss1= 2.9817 loss2=-3148.9812 lr= 0.000100
  1786. 2019-03-12 13:31:20,163 : epoch 196 , step 0 loss0= 0.0000 loss1= 3.7112 loss2=-3612.0618 lr= 0.000100
  1787. 2019-03-12 13:31:53,607 : epoch 196 , step 40 loss0= 0.0000 loss1= 3.4750 loss2=-3343.2195 lr= 0.000100
  1788. 2019-03-12 13:32:27,100 : epoch 196 , step 80 loss0= 0.0000 loss1= 2.9859 loss2=-3559.4041 lr= 0.000100
  1789. 2019-03-12 13:33:00,212 : epoch 196 , step 120 loss0= 0.0000 loss1= 3.6211 loss2=-3399.9812 lr= 0.000100
  1790. 2019-03-12 13:33:32,952 : epoch 196 , step 160 loss0= 0.0000 loss1= 3.6409 loss2=-3606.2285 lr= 0.000100
  1791. 2019-03-12 13:34:02,707 : epoch 197 , step 0 loss0= 0.0000 loss1= 3.4039 loss2=-3405.4333 lr= 0.000100
  1792. 2019-03-12 13:34:36,053 : epoch 197 , step 40 loss0= 0.0000 loss1= 4.9652 loss2=-3847.7446 lr= 0.000100
  1793. 2019-03-12 13:35:09,835 : epoch 197 , step 80 loss0= 0.0000 loss1= 4.2287 loss2=-3673.0789 lr= 0.000100
  1794. 2019-03-12 13:35:43,437 : epoch 197 , step 120 loss0= 0.0000 loss1= 2.8911 loss2=-3610.6423 lr= 0.000100
  1795. 2019-03-12 13:36:16,970 : epoch 197 , step 160 loss0= 0.0000 loss1= 4.1538 loss2=-3650.0930 lr= 0.000100
  1796. 2019-03-12 13:36:47,386 : epoch 198 , step 0 loss0= 0.0000 loss1= 3.6391 loss2=-3553.8494 lr= 0.000100
  1797. 2019-03-12 13:37:21,257 : epoch 198 , step 40 loss0= 0.0000 loss1= 3.9169 loss2=-3436.5618 lr= 0.000100
  1798. 2019-03-12 13:37:56,312 : epoch 198 , step 80 loss0= 0.0000 loss1= 3.5228 loss2=-3523.2148 lr= 0.000100
  1799. 2019-03-12 13:38:30,860 : epoch 198 , step 120 loss0= 0.0000 loss1= 3.6405 loss2=-3851.8469 lr= 0.000100
  1800. 2019-03-12 13:39:05,513 : epoch 198 , step 160 loss0= 0.0000 loss1= 4.0050 loss2=-3522.0662 lr= 0.000100
  1801. 2019-03-12 13:39:36,309 : epoch 199 , step 0 loss0= 0.0000 loss1= 3.4653 loss2=-3634.3047 lr= 0.000100
  1802. 2019-03-12 13:40:10,348 : epoch 199 , step 40 loss0= 0.0000 loss1= 3.3920 loss2=-3600.8728 lr= 0.000100
  1803. 2019-03-12 13:40:44,474 : epoch 199 , step 80 loss0= 0.0000 loss1= 2.5289 loss2=-3516.3091 lr= 0.000100
  1804. 2019-03-12 13:41:18,902 : epoch 199 , step 120 loss0= 0.0000 loss1= 3.0363 loss2=-3461.8455 lr= 0.000100
  1805. 2019-03-12 13:41:53,760 : epoch 199 , step 160 loss0= 0.0000 loss1= 4.6069 loss2=-3573.0525 lr= 0.000100
  1806. 2019-03-12 13:42:24,918 : epoch 200 , step 0 loss0= 0.0000 loss1= 4.0423 loss2=-3731.9692 lr= 0.000100
  1807. 2019-03-12 13:42:59,174 : epoch 200 , step 40 loss0= 0.0000 loss1= 3.4946 loss2=-3614.1724 lr= 0.000100
  1808. 2019-03-12 13:43:33,143 : epoch 200 , step 80 loss0= 0.0000 loss1= 3.2937 loss2=-3397.8660 lr= 0.000100
  1809. 2019-03-12 13:44:07,631 : epoch 200 , step 120 loss0= 0.0000 loss1= 2.7392 loss2=-3477.0837 lr= 0.000100
  1810. 2019-03-12 13:44:41,840 : epoch 200 , step 160 loss0= 0.0000 loss1= 4.1096 loss2=-3465.9531 lr= 0.000100
  1811. 2019-03-12 13:45:10,999 : ## train ##
  1812. 2019-03-12 13:49:26,513 : >> epoch 200 : average_loss= 0.7103, average_accuracy=82.582%, mean_weight_std= 0.1976, mean_normalized_std= 0.4263 mean_dist=2.507e+03
  1813. 2019-03-12 13:49:26,523 : >> epoch 200 : median_loss= 0.6997, median_accuracy=82.905%, median_weight_std= 0.1207, median_normalized_std= 0.0635 median_dist=2.538e+03
  1814. 2019-03-12 13:49:26,523 : >> epoch 200 : ensemble accuracy: 84.70%
  1815. 2019-03-12 13:49:26,523 : >> epoch 200 : loss histogram: (0.00,0.30, 18),(0.30,0.60, 0),(0.60,0.90, 4),(0.90,1.20, 0),(1.20,1.50, 18),
  1816. 2019-03-12 13:49:26,523 : >> epoch 200 : accuracy histogram: (0.00,33.50, 18),(33.50,67.00, 0),(67.00,100.49, 4),(100.49,133.99, 0),(133.99,167.49, 18),
  1817. 2019-03-12 13:49:26,529 : >> epoch 200 : weight std histogram: (0.0000,2.0917, 464638),(2.0917,4.1834, 1947),(4.1834,6.2752, 263),(6.2752,8.3669, 47),(8.3669,10.4586, 11),
  1818. 2019-03-12 13:49:26,534 : >> epoch 200 : weight normalized std histogram: (0.0000,3965.4984, 466903),(3965.4984,7930.9969, 1),(7930.9969,11896.4953, 0),(11896.4953,15861.9937, 1),(15861.9937,19827.4922, 1),
  1819. 2019-03-12 13:49:26,534 : >> epoch 200 : dist histogram: (2300.0098,2365.2688, 3),(2365.2688,2430.5278, 1),(2430.5278,2495.7869, 5),(2495.7869,2561.0459, 5),(2561.0459,2626.3049, 6),
  1820. 2019-03-12 13:49:26,534 : ## eval ##
  1821. 2019-03-12 13:53:42,898 : >> epoch 200 : average_loss= 0.5829, average_accuracy=86.189%, mean_weight_std= 0.2107, mean_normalized_std= 0.5217 mean_dist=2.506e+03
  1822. 2019-03-12 13:53:42,908 : >> epoch 200 : median_loss= 0.5750, median_accuracy=87.415%, median_weight_std= 0.1298, median_normalized_std= 0.0680 median_dist=2.499e+03
  1823. 2019-03-12 13:53:42,909 : >> epoch 200 : ensemble accuracy: 88.21%
  1824. 2019-03-12 13:53:42,909 : >> epoch 200 : loss histogram: (0.00,0.24, 18),(0.24,0.48, 0),(0.48,0.72, 4),(0.72,0.96, 0),(0.96,1.21, 18),
  1825. 2019-03-12 13:53:42,909 : >> epoch 200 : accuracy histogram: (0.00,34.89, 18),(34.89,69.77, 0),(69.77,104.66, 4),(104.66,139.54, 0),(139.54,174.43, 18),
  1826. 2019-03-12 13:53:42,915 : >> epoch 200 : weight std histogram: (0.0000,2.1000, 464524),(2.1000,4.2000, 2059),(4.2000,6.3000, 274),(6.3000,8.4000, 38),(8.4000,10.5001, 11),
  1827. 2019-03-12 13:53:42,919 : >> epoch 200 : weight normalized std histogram: (0.0000,13519.2828, 466905),(13519.2828,27038.5656, 0),(27038.5656,40557.8484, 0),(40557.8484,54077.1312, 0),(54077.1312,67596.4141, 1),
  1828. 2019-03-12 13:53:42,920 : >> epoch 200 : dist histogram: (2330.1902,2405.1584, 5),(2405.1584,2480.1266, 4),(2480.1266,2555.0948, 3),(2555.0948,2630.0630, 5),(2630.0630,2705.0312, 3),
  1829. 2019-03-12 13:53:42,974 : >> epoch 200 : dist pair-wise
  1830. 2019-03-12 13:53:42,980 :
  1831. [[ 0. 444.166 303.821 555.275 363.812 523.289 382.272 538.553
  1832. 336.496 488.309 379.562 410.477 474.172 386.85 270.495 488.154
  1833. 282.757 515.553 460.363 340.588]
  1834. [ 444.166 0. 328.146 289.9 357.731 219.551 256.448 434.679
  1835. 292.821 372.8 292.344 476.537 356.232 259.565 369.947 231.664
  1836. 322.379 334.287 245.208 371.074]
  1837. [ 303.821 328.146 0. 412.73 289.265 398.392 271.529 465.617
  1838. 262.908 408.531 288.902 412.807 409.282 324.631 175.338 369.088
  1839. 258.367 429.916 335.704 278.711]
  1840. [ 555.275 289.9 412.73 0. 361.378 223.161 294.204 366.702
  1841. 350.651 303.811 286.339 552.476 311.119 431.513 441.249 339.725
  1842. 409.268 467.152 321.27 376.133]
  1843. [ 363.812 357.731 289.265 361.378 0. 370.027 287.203 304.613
  1844. 297.079 269.249 256.322 380.034 278.303 396.007 253.307 375.44
  1845. 271.184 492.123 339.633 173.895]
  1846. [ 523.289 219.551 398.392 223.161 370.027 0. 324.961 421.471
  1847. 353.191 354.259 316.762 547.36 354.376 334.935 427.566 225.092
  1848. 384.31 346.586 216.7 393.92 ]
  1849. [ 382.272 256.448 271.529 294.204 287.203 324.961 0. 361.368
  1850. 184.663 297.97 175.314 405.1 271.36 367.042 298.615 361.342
  1851. 243.435 471.492 350.441 267.176]
  1852. [ 538.553 434.679 465.617 366.702 304.613 421.471 361.368 0.
  1853. 382.263 183.386 315.901 373.166 188.828 530.61 433.109 486.409
  1854. 366.704 605.669 456.431 279.703]
  1855. [ 336.496 292.821 262.908 350.651 297.079 353.191 184.663 382.263
  1856. 0. 313.057 161.744 353.052 300.078 384.988 261.868 408.339
  1857. 203.777 479.155 381.347 246.05 ]
  1858. [ 488.309 372.8 408.531 303.811 269.249 354.259 297.97 183.386
  1859. 313.057 0. 227.355 403.868 180.255 479.142 381.271 440.081
  1860. 315.329 559.055 406.077 250.264]
  1861. [ 379.562 292.344 288.902 286.339 256.322 316.762 175.314 315.901
  1862. 161.744 227.355 0. 393.007 244.227 396.413 288.368 383.733
  1863. 235.103 490.662 357.957 221.621]
  1864. [ 410.477 476.537 412.807 552.476 380.034 547.36 405.1 373.166
  1865. 353.052 403.868 393.007 0. 372.044 494.435 385.229 539.594
  1866. 338.445 586.332 516.168 308.329]
  1867. [ 474.172 356.232 409.282 311.119 278.303 354.376 271.36 188.828
  1868. 300.078 180.255 244.227 372.044 0. 468.743 385.944 431.162
  1869. 294.167 550.551 403.87 255.014]
  1870. [ 386.85 259.565 324.631 431.513 396.007 334.935 367.042 530.61
  1871. 384.988 479.142 396.413 494.435 468.743 0. 381.915 248.061
  1872. 375.386 293.085 228.2 401.376]
  1873. [ 270.495 369.947 175.338 441.249 253.307 427.566 298.615 433.109
  1874. 261.868 381.271 288.368 385.229 385.944 381.915 0. 419.073
  1875. 210.933 474.237 379.574 239.162]
  1876. [ 488.154 231.664 369.088 339.725 375.44 225.092 361.342 486.409
  1877. 408.339 440.081 383.733 539.594 431.162 248.061 419.073 0.
  1878. 412.694 247.922 134.218 418.964]
  1879. [ 282.757 322.379 258.367 409.268 271.184 384.31 243.435 366.704
  1880. 203.777 315.329 235.103 338.445 294.167 375.386 210.933 412.694
  1881. 0. 474.852 382.852 226.374]
  1882. [ 515.553 334.287 429.916 467.152 492.123 346.586 471.492 605.669
  1883. 479.155 559.055 490.662 586.332 550.551 293.085 474.237 247.922
  1884. 474.852 0. 268.585 514.503]
  1885. [ 460.363 245.208 335.704 321.27 339.633 216.7 350.441 456.431
  1886. 381.347 406.077 357.957 516.168 403.87 228.2 379.574 134.218
  1887. 382.852 268.585 0. 378.633]
  1888. [ 340.588 371.074 278.711 376.133 173.895 393.92 267.176 279.703
  1889. 246.05 250.264 221.621 308.329 255.014 401.376 239.162 418.964
  1890. 226.374 514.503 378.633 0. ]]
  1891. 2019-03-12 13:53:44,335 : epoch 201 , step 0 loss0= 0.0000 loss1= 4.0170 loss2=-3462.4875 lr= 0.000100
  1892. 2019-03-12 13:54:18,223 : epoch 201 , step 40 loss0= 0.0000 loss1= 3.2004 loss2=-3401.3113 lr= 0.000100
  1893. 2019-03-12 13:54:51,966 : epoch 201 , step 80 loss0= 0.0000 loss1= 2.9508 loss2=-3395.6992 lr= 0.000100
  1894. 2019-03-12 13:55:25,814 : epoch 201 , step 120 loss0= 0.0000 loss1= 3.9086 loss2=-3750.0859 lr= 0.000100
  1895. 2019-03-12 13:55:59,282 : epoch 201 , step 160 loss0= 0.0000 loss1= 3.0666 loss2=-3464.7617 lr= 0.000100
  1896. 2019-03-12 13:56:29,682 : epoch 202 , step 0 loss0= 0.0000 loss1= 3.5438 loss2=-3799.4871 lr= 0.000100
  1897. 2019-03-12 13:57:03,622 : epoch 202 , step 40 loss0= 0.0000 loss1= 3.2196 loss2=-3564.1541 lr= 0.000100
  1898. 2019-03-12 13:57:37,361 : epoch 202 , step 80 loss0= 0.0000 loss1= 3.3450 loss2=-3396.9219 lr= 0.000100
  1899. 2019-03-12 13:58:11,011 : epoch 202 , step 120 loss0= 0.0000 loss1= 3.2911 loss2=-3272.9048 lr= 0.000100
  1900. 2019-03-12 13:58:44,937 : epoch 202 , step 160 loss0= 0.0000 loss1= 3.7238 loss2=-3940.2595 lr= 0.000100
  1901. 2019-03-12 13:59:15,412 : epoch 203 , step 0 loss0= 0.0000 loss1= 2.8602 loss2=-3489.7180 lr= 0.000100
  1902. 2019-03-12 13:59:49,332 : epoch 203 , step 40 loss0= 0.0000 loss1= 4.1715 loss2=-3631.8887 lr= 0.000100
  1903. 2019-03-12 14:00:24,074 : epoch 203 , step 80 loss0= 0.0000 loss1= 2.7253 loss2=-3481.8984 lr= 0.000100
  1904. 2019-03-12 14:00:58,405 : epoch 203 , step 120 loss0= 0.0000 loss1= 3.3929 loss2=-3654.2949 lr= 0.000100
  1905. 2019-03-12 14:01:32,755 : epoch 203 , step 160 loss0= 0.0000 loss1= 2.5211 loss2=-3398.6951 lr= 0.000100
  1906. 2019-03-12 14:02:03,074 : epoch 204 , step 0 loss0= 0.0000 loss1= 3.1224 loss2=-3722.8708 lr= 0.000100
  1907. 2019-03-12 14:02:36,803 : epoch 204 , step 40 loss0= 0.0000 loss1= 2.8813 loss2=-3458.8289 lr= 0.000100
  1908. 2019-03-12 14:03:11,250 : epoch 204 , step 80 loss0= 0.0000 loss1= 2.8124 loss2=-3634.9868 lr= 0.000100
  1909. 2019-03-12 14:03:45,925 : epoch 204 , step 120 loss0= 0.0000 loss1= 2.4091 loss2=-3245.7771 lr= 0.000100
  1910. 2019-03-12 14:04:20,472 : epoch 204 , step 160 loss0= 0.0000 loss1= 3.1116 loss2=-3445.1816 lr= 0.000100
  1911. 2019-03-12 14:04:50,964 : epoch 205 , step 0 loss0= 0.0000 loss1= 3.8018 loss2=-3415.3235 lr= 0.000100
  1912. 2019-03-12 14:05:25,275 : epoch 205 , step 40 loss0= 0.0000 loss1= 3.0314 loss2=-3800.6345 lr= 0.000100
  1913. 2019-03-12 14:05:58,796 : epoch 205 , step 80 loss0= 0.0000 loss1= 4.3620 loss2=-3944.6277 lr= 0.000100
  1914. 2019-03-12 14:06:32,622 : epoch 205 , step 120 loss0= 0.0000 loss1= 4.1542 loss2=-3590.6309 lr= 0.000100
  1915. 2019-03-12 14:07:06,357 : epoch 205 , step 160 loss0= 0.0000 loss1= 4.9179 loss2=-3880.0681 lr= 0.000100
  1916. 2019-03-12 14:07:37,298 : epoch 206 , step 0 loss0= 0.0000 loss1= 2.3049 loss2=-3456.3572 lr= 0.000100
  1917. 2019-03-12 14:08:11,750 : epoch 206 , step 40 loss0= 0.0000 loss1= 3.0424 loss2=-3793.2205 lr= 0.000100
  1918. 2019-03-12 14:08:45,817 : epoch 206 , step 80 loss0= 0.0000 loss1= 3.6390 loss2=-3755.1372 lr= 0.000100
  1919. 2019-03-12 14:09:19,361 : epoch 206 , step 120 loss0= 0.0000 loss1= 3.7728 loss2=-3317.9822 lr= 0.000100
  1920. 2019-03-12 14:09:52,849 : epoch 206 , step 160 loss0= 0.0000 loss1= 2.1695 loss2=-3396.7402 lr= 0.000100
  1921. 2019-03-12 14:10:23,794 : epoch 207 , step 0 loss0= 0.0000 loss1= 2.8855 loss2=-3604.2312 lr= 0.000100
  1922. 2019-03-12 14:10:57,595 : epoch 207 , step 40 loss0= 0.0000 loss1= 2.3802 loss2=-3483.2231 lr= 0.000100
  1923. 2019-03-12 14:11:31,248 : epoch 207 , step 80 loss0= 0.0000 loss1= 2.6283 loss2=-3438.3464 lr= 0.000100
  1924. 2019-03-12 14:12:05,216 : epoch 207 , step 120 loss0= 0.0000 loss1= 2.8826 loss2=-3754.0493 lr= 0.000100
  1925. 2019-03-12 14:12:39,123 : epoch 207 , step 160 loss0= 0.0000 loss1= 3.3239 loss2=-3683.1411 lr= 0.000100
  1926. 2019-03-12 14:13:09,600 : epoch 208 , step 0 loss0= 0.0000 loss1= 2.4584 loss2=-3432.1614 lr= 0.000100
  1927. 2019-03-12 14:13:43,156 : epoch 208 , step 40 loss0= 0.0000 loss1= 4.0746 loss2=-3531.8489 lr= 0.000100
  1928. 2019-03-12 14:14:16,639 : epoch 208 , step 80 loss0= 0.0000 loss1= 2.9415 loss2=-3499.2388 lr= 0.000100
  1929. 2019-03-12 14:14:50,295 : epoch 208 , step 120 loss0= 0.0000 loss1= 3.6517 loss2=-3634.8223 lr= 0.000100
  1930. 2019-03-12 14:15:23,821 : epoch 208 , step 160 loss0= 0.0000 loss1= 3.6414 loss2=-3313.5005 lr= 0.000100
  1931. 2019-03-12 14:15:54,434 : epoch 209 , step 0 loss0= 0.0000 loss1= 4.0335 loss2=-3453.6672 lr= 0.000100
  1932. 2019-03-12 14:16:28,685 : epoch 209 , step 40 loss0= 0.0000 loss1= 2.6579 loss2=-3524.4441 lr= 0.000100
  1933. 2019-03-12 14:17:02,919 : epoch 209 , step 80 loss0= 0.0000 loss1= 2.3916 loss2=-3416.0955 lr= 0.000100
  1934. 2019-03-12 14:17:37,042 : epoch 209 , step 120 loss0= 0.0000 loss1= 2.8579 loss2=-3646.4126 lr= 0.000100
  1935. 2019-03-12 14:18:10,903 : epoch 209 , step 160 loss0= 0.0000 loss1= 3.3479 loss2=-3650.8445 lr= 0.000100
  1936. 2019-03-12 14:18:41,792 : epoch 210 , step 0 loss0= 0.0000 loss1= 3.3864 loss2=-3659.0349 lr= 0.000100
  1937. 2019-03-12 14:19:15,734 : epoch 210 , step 40 loss0= 0.0000 loss1= 3.1678 loss2=-3893.4119 lr= 0.000100
  1938. 2019-03-12 14:19:49,494 : epoch 210 , step 80 loss0= 0.0000 loss1= 2.6644 loss2=-3499.8750 lr= 0.000100
  1939. 2019-03-12 14:20:24,656 : epoch 210 , step 120 loss0= 0.0000 loss1= 3.2499 loss2=-3681.6099 lr= 0.000100
  1940. 2019-03-12 14:20:59,894 : epoch 210 , step 160 loss0= 0.0000 loss1= 3.1731 loss2=-3691.1042 lr= 0.000100
  1941. 2019-03-12 14:21:30,690 : epoch 211 , step 0 loss0= 0.0000 loss1= 2.2851 loss2=-3393.9531 lr= 0.000100
  1942. 2019-03-12 14:22:05,389 : epoch 211 , step 40 loss0= 0.0000 loss1= 3.3947 loss2=-3427.7937 lr= 0.000100
  1943. 2019-03-12 14:22:40,663 : epoch 211 , step 80 loss0= 0.0000 loss1= 4.3982 loss2=-3999.9773 lr= 0.000100
  1944. 2019-03-12 14:23:15,938 : epoch 211 , step 120 loss0= 0.0000 loss1= 2.9861 loss2=-3423.9458 lr= 0.000100
  1945. 2019-03-12 14:23:50,297 : epoch 211 , step 160 loss0= 0.0000 loss1= 3.4033 loss2=-3685.6274 lr= 0.000100
  1946. 2019-03-12 14:24:20,944 : epoch 212 , step 0 loss0= 0.0000 loss1= 2.3179 loss2=-3334.8953 lr= 0.000100
  1947. 2019-03-12 14:24:54,742 : epoch 212 , step 40 loss0= 0.0000 loss1= 3.2910 loss2=-3563.5942 lr= 0.000100
  1948. 2019-03-12 14:25:28,369 : epoch 212 , step 80 loss0= 0.0000 loss1= 3.5903 loss2=-3642.4407 lr= 0.000100
  1949. 2019-03-12 14:26:02,085 : epoch 212 , step 120 loss0= 0.0000 loss1= 2.4329 loss2=-3121.2012 lr= 0.000100
  1950. 2019-03-12 14:26:36,019 : epoch 212 , step 160 loss0= 0.0000 loss1= 2.6170 loss2=-3536.8867 lr= 0.000100
  1951. 2019-03-12 14:27:06,517 : epoch 213 , step 0 loss0= 0.0000 loss1= 2.4509 loss2=-3524.4614 lr= 0.000100
  1952. 2019-03-12 14:27:40,606 : epoch 213 , step 40 loss0= 0.0000 loss1= 2.9407 loss2=-3724.8279 lr= 0.000100
  1953. 2019-03-12 14:28:14,822 : epoch 213 , step 80 loss0= 0.0000 loss1= 2.9625 loss2=-3570.7231 lr= 0.000100
  1954. 2019-03-12 14:28:48,959 : epoch 213 , step 120 loss0= 0.0000 loss1= 4.0255 loss2=-3557.7517 lr= 0.000100
  1955. 2019-03-12 14:29:23,320 : epoch 213 , step 160 loss0= 0.0000 loss1= 2.9947 loss2=-3372.4048 lr= 0.000100
  1956. 2019-03-12 14:29:53,716 : epoch 214 , step 0 loss0= 0.0000 loss1= 2.8840 loss2=-3449.9900 lr= 0.000100
  1957. 2019-03-12 14:30:28,155 : epoch 214 , step 40 loss0= 0.0000 loss1= 2.7645 loss2=-3556.4255 lr= 0.000100
  1958. 2019-03-12 14:31:02,362 : epoch 214 , step 80 loss0= 0.0000 loss1= 3.5973 loss2=-3431.6372 lr= 0.000100
  1959. 2019-03-12 14:31:36,272 : epoch 214 , step 120 loss0= 0.0000 loss1= 5.1365 loss2=-3930.3750 lr= 0.000100
  1960. 2019-03-12 14:32:10,195 : epoch 214 , step 160 loss0= 0.0000 loss1= 2.9304 loss2=-3635.7195 lr= 0.000100
  1961. 2019-03-12 14:32:41,026 : epoch 215 , step 0 loss0= 0.0000 loss1= 3.0663 loss2=-3532.3665 lr= 0.000100
  1962. 2019-03-12 14:33:14,896 : epoch 215 , step 40 loss0= 0.0000 loss1= 1.7570 loss2=-3314.4751 lr= 0.000100
  1963. 2019-03-12 14:33:48,718 : epoch 215 , step 80 loss0= 0.0000 loss1= 3.6517 loss2=-3867.0435 lr= 0.000100
  1964. 2019-03-12 14:34:22,640 : epoch 215 , step 120 loss0= 0.0000 loss1= 2.8663 loss2=-3299.1497 lr= 0.000100
  1965. 2019-03-12 14:34:56,344 : epoch 215 , step 160 loss0= 0.0000 loss1= 2.9831 loss2=-3753.7642 lr= 0.000100
  1966. 2019-03-12 14:35:26,809 : epoch 216 , step 0 loss0= 0.0000 loss1= 3.0407 loss2=-3312.5916 lr= 0.000100
  1967. 2019-03-12 14:36:00,964 : epoch 216 , step 40 loss0= 0.0000 loss1= 2.1112 loss2=-3612.2302 lr= 0.000100
  1968. 2019-03-12 14:36:34,980 : epoch 216 , step 80 loss0= 0.0000 loss1= 3.0934 loss2=-3592.5125 lr= 0.000100
  1969. 2019-03-12 14:37:08,491 : epoch 216 , step 120 loss0= 0.0000 loss1= 2.9543 loss2=-3611.6677 lr= 0.000100
  1970. 2019-03-12 14:37:42,021 : epoch 216 , step 160 loss0= 0.0000 loss1= 3.8351 loss2=-3547.0669 lr= 0.000100
  1971. 2019-03-12 14:38:12,578 : epoch 217 , step 0 loss0= 0.0000 loss1= 2.0395 loss2=-3396.3455 lr= 0.000100
  1972. 2019-03-12 14:38:46,811 : epoch 217 , step 40 loss0= 0.0000 loss1= 3.5921 loss2=-3648.2207 lr= 0.000100
  1973. 2019-03-12 14:39:21,096 : epoch 217 , step 80 loss0= 0.0000 loss1= 2.4905 loss2=-3347.7368 lr= 0.000100
  1974. 2019-03-12 14:39:55,310 : epoch 217 , step 120 loss0= 0.0000 loss1= 2.6392 loss2=-3404.8718 lr= 0.000100
  1975. 2019-03-12 14:40:29,724 : epoch 217 , step 160 loss0= 0.0000 loss1= 3.5584 loss2=-3702.7205 lr= 0.000100
  1976. 2019-03-12 14:41:01,138 : epoch 218 , step 0 loss0= 0.0000 loss1= 3.4728 loss2=-3588.0037 lr= 0.000100
  1977. 2019-03-12 14:41:35,193 : epoch 218 , step 40 loss0= 0.0000 loss1= 2.2404 loss2=-3545.0876 lr= 0.000100
  1978. 2019-03-12 14:42:10,097 : epoch 218 , step 80 loss0= 0.0000 loss1= 2.8685 loss2=-3587.3677 lr= 0.000100
  1979. 2019-03-12 14:42:44,605 : epoch 218 , step 120 loss0= 0.0000 loss1= 3.5523 loss2=-3716.4988 lr= 0.000100
  1980. 2019-03-12 14:43:18,946 : epoch 218 , step 160 loss0= 0.0000 loss1= 4.6404 loss2=-3657.7114 lr= 0.000100
  1981. 2019-03-12 14:43:50,111 : epoch 219 , step 0 loss0= 0.0000 loss1= 3.9030 loss2=-3758.6150 lr= 0.000100
  1982. 2019-03-12 14:44:24,219 : epoch 219 , step 40 loss0= 0.0000 loss1= 2.3446 loss2=-3496.2239 lr= 0.000100
  1983. 2019-03-12 14:44:58,436 : epoch 219 , step 80 loss0= 0.0000 loss1= 2.5431 loss2=-3579.0786 lr= 0.000100
  1984. 2019-03-12 14:45:32,330 : epoch 219 , step 120 loss0= 0.0000 loss1= 2.5212 loss2=-3430.9915 lr= 0.000100
  1985. 2019-03-12 14:46:06,730 : epoch 219 , step 160 loss0= 0.0000 loss1= 3.1765 loss2=-3502.4661 lr= 0.000100
  1986. 2019-03-12 14:46:37,844 : epoch 220 , step 0 loss0= 0.0000 loss1= 2.7283 loss2=-3725.2969 lr= 0.000100
  1987. 2019-03-12 14:47:12,325 : epoch 220 , step 40 loss0= 0.0000 loss1= 2.7671 loss2=-3481.8679 lr= 0.000100
  1988. 2019-03-12 14:47:46,845 : epoch 220 , step 80 loss0= 0.0000 loss1= 4.4506 loss2=-3622.9302 lr= 0.000100
  1989. 2019-03-12 14:48:20,946 : epoch 220 , step 120 loss0= 0.0000 loss1= 2.8348 loss2=-3301.4587 lr= 0.000100
  1990. 2019-03-12 14:48:55,280 : epoch 220 , step 160 loss0= 0.0000 loss1= 3.7735 loss2=-3566.5554 lr= 0.000100
  1991. 2019-03-12 14:49:24,858 : ## train ##
  1992. 2019-03-12 14:53:46,269 : >> epoch 220 : average_loss= 0.6999, average_accuracy=83.294%, mean_weight_std= 0.2254, mean_normalized_std= 0.5980 mean_dist=2.572e+03
  1993. 2019-03-12 14:53:46,280 : >> epoch 220 : median_loss= 0.6876, median_accuracy=83.605%, median_weight_std= 0.1393, median_normalized_std= 0.0715 median_dist=2.576e+03
  1994. 2019-03-12 14:53:46,280 : >> epoch 220 : ensemble accuracy: 85.57%
  1995. 2019-03-12 14:53:46,280 : >> epoch 220 : loss histogram: (0.00,0.30, 18),(0.30,0.60, 0),(0.60,0.90, 4),(0.90,1.21, 0),(1.21,1.51, 18),
  1996. 2019-03-12 14:53:46,280 : >> epoch 220 : accuracy histogram: (0.00,33.95, 18),(33.95,67.89, 0),(67.89,101.84, 4),(101.84,135.78, 0),(135.78,169.73, 18),
  1997. 2019-03-12 14:53:46,286 : >> epoch 220 : weight std histogram: (0.0000,2.1005, 464247),(2.1005,4.2009, 2305),(4.2009,6.3014, 296),(6.3014,8.4018, 49),(8.4018,10.5023, 9),
  1998. 2019-03-12 14:53:46,291 : >> epoch 220 : weight normalized std histogram: (0.0000,19290.5281, 466905),(19290.5281,38581.0563, 0),(38581.0563,57871.5844, 0),(57871.5844,77162.1125, 0),(77162.1125,96452.6406, 1),
  1999. 2019-03-12 14:53:46,291 : >> epoch 220 : dist histogram: (2314.2444,2411.0652, 1),(2411.0652,2507.8861, 5),(2507.8861,2604.7069, 7),(2604.7069,2701.5278, 4),(2701.5278,2798.3486, 3),
  2000. 2019-03-12 14:53:46,291 : ## eval ##
  2001. 2019-03-12 14:58:09,577 : >> epoch 220 : average_loss= 0.6242, average_accuracy=86.117%, mean_weight_std= 0.2341, mean_normalized_std= 0.3830 mean_dist=2.572e+03
  2002. 2019-03-12 14:58:09,584 : >> epoch 220 : median_loss= 0.6302, median_accuracy=85.805%, median_weight_std= 0.1462, median_normalized_std= 0.0751 median_dist=2.543e+03
  2003. 2019-03-12 14:58:09,584 : >> epoch 220 : ensemble accuracy: 88.06%
  2004. 2019-03-12 14:58:09,585 : >> epoch 220 : loss histogram: (0.00,0.27, 18),(0.27,0.54, 0),(0.54,0.80, 4),(0.80,1.07, 0),(1.07,1.34, 18),
  2005. 2019-03-12 14:58:09,585 : >> epoch 220 : accuracy histogram: (0.00,34.85, 18),(34.85,69.70, 0),(69.70,104.55, 4),(104.55,139.40, 0),(139.40,174.25, 18),
  2006. 2019-03-12 14:58:09,591 : >> epoch 220 : weight std histogram: (0.0000,2.2041, 464590),(2.2041,4.4082, 2038),(4.4082,6.6124, 232),(6.6124,8.8165, 36),(8.8165,11.0206, 10),
  2007. 2019-03-12 14:58:09,596 : >> epoch 220 : weight normalized std histogram: (0.0000,1551.5112, 466899),(1551.5112,3103.0225, 5),(3103.0225,4654.5337, 1),(4654.5337,6206.0449, 0),(6206.0449,7757.5562, 1),
  2008. 2019-03-12 14:58:09,596 : >> epoch 220 : dist histogram: (2417.0466,2506.1227, 8),(2506.1227,2595.1987, 6),(2595.1987,2684.2747, 3),(2684.2747,2773.3507, 0),(2773.3507,2862.4268, 3),
  2009. 2019-03-12 14:58:09,654 : >> epoch 220 : dist pair-wise
  2010. 2019-03-12 14:58:09,660 :
  2011. [[ 0. 394.866 490.527 348.66 328.566 346.72 338.248 397.876
  2012. 264.708 491.19 253.28 326.752 369.544 275.197 424.592 406.039
  2013. 576.111 379.627 445.991 420.82 ]
  2014. [ 394.866 0. 338.577 227.545 576.845 363.39 324.119 331.847
  2015. 339.267 442.897 280.206 394.721 494.284 365.108 308.764 331.079
  2016. 543.126 303.351 201.261 313.292]
  2017. [ 490.527 338.577 0. 385.996 635.072 441.225 339.389 363.841
  2018. 399.538 253.229 440.999 530.343 521.195 469.512 223.516 399.85
  2019. 310.381 403.472 277.605 392.951]
  2020. [ 348.66 227.545 385.996 0. 485.853 283.4 236.7 291.698
  2021. 287.934 423.726 297.739 385.207 366.213 344.598 290.401 271.343
  2022. 525.895 204.024 307.8 298.84 ]
  2023. [ 328.566 576.845 635.072 485.853 0. 466.84 477.203 570.906
  2024. 450.914 590.679 475.103 433.344 347.713 433.681 571.746 575.802
  2025. 625.314 491.778 629.216 593.651]
  2026. [ 346.72 363.39 441.225 283.4 466.84 0. 253.719 239.924
  2027. 250.176 446.799 312.199 331.531 432.435 275.573 314.74 230.687
  2028. 564.52 245.887 348.846 274.764]
  2029. [ 338.248 324.119 339.389 236.7 477.203 253.719 0. 265.702
  2030. 249.858 302.685 311.706 406.214 336.982 336.173 223.113 232.144
  2031. 434.078 223.527 329.22 306.097]
  2032. [ 397.876 331.847 363.841 291.698 570.906 239.924 265.702 0.
  2033. 263.827 409.21 344.769 388.883 498.78 296.039 275.049 207.294
  2034. 531.539 239.86 295.377 233.725]
  2035. [ 264.708 339.267 399.538 287.934 450.914 250.176 249.858 263.827
  2036. 0. 403.609 242.339 329.944 412.396 263.109 300.145 271.83
  2037. 523.309 300.431 344.274 292.75 ]
  2038. [ 491.19 442.897 253.229 423.726 590.679 446.799 302.685 409.21
  2039. 403.609 0. 474.056 562.948 461.602 502.88 258.374 415.263
  2040. 231.729 412.271 395.086 443.948]
  2041. [ 253.28 280.206 440.999 297.739 475.103 312.199 311.706 344.769
  2042. 242.339 474.056 0. 298.359 468.321 293.155 356.096 317.306
  2043. 585.67 350.268 328.554 330.96 ]
  2044. [ 326.752 394.721 530.343 385.207 433.344 331.531 406.214 388.883
  2045. 329.944 562.948 298.359 0. 539.764 221.277 454.128 364.752
  2046. 671.188 390.749 416.359 335.144]
  2047. [ 369.544 494.284 521.195 366.213 347.713 432.435 336.982 498.78
  2048. 412.396 461.602 468.321 539.764 0. 475.267 456.893 503.148
  2049. 503.079 386.261 556.528 543.188]
  2050. [ 275.197 365.108 469.512 344.598 433.681 275.573 336.173 296.039
  2051. 263.109 502.88 293.155 221.277 475.267 0. 390.446 320.491
  2052. 617.289 311.736 373.599 297.349]
  2053. [ 424.592 308.764 223.516 290.401 571.746 314.74 223.113 275.049
  2054. 300.145 258.374 356.096 454.128 456.893 390.446 0. 262.746
  2055. 371.282 286.635 239.03 276.839]
  2056. [ 406.039 331.079 399.85 271.343 575.802 230.687 232.144 207.294
  2057. 271.83 415.263 317.306 364.752 503.148 320.491 262.746 0.
  2058. 552.287 243.265 303.823 195.792]
  2059. [ 576.111 543.126 310.381 525.895 625.314 564.52 434.078 531.539
  2060. 523.309 231.729 585.67 671.188 503.079 617.289 371.282 552.287
  2061. 0. 521.097 510.193 574.271]
  2062. [ 379.627 303.351 403.472 204.024 491.778 245.887 223.527 239.86
  2063. 300.431 412.271 350.268 390.749 386.261 311.736 286.635 243.265
  2064. 521.097 0. 332.94 270.243]
  2065. [ 445.991 201.261 277.605 307.8 629.216 348.846 329.22 295.377
  2066. 344.274 395.086 328.554 416.359 556.528 373.599 239.03 303.823
  2067. 510.193 332.94 0. 267.813]
  2068. [ 420.82 313.292 392.951 298.84 593.651 274.764 306.097 233.725
  2069. 292.75 443.948 330.96 335.144 543.188 297.349 276.839 195.792
  2070. 574.271 270.243 267.813 0. ]]
  2071. 2019-03-12 14:58:11,058 : epoch 221 , step 0 loss0= 0.0000 loss1= 2.6834 loss2=-3432.8164 lr= 0.000100
  2072. 2019-03-12 14:58:46,193 : epoch 221 , step 40 loss0= 0.0000 loss1= 2.3667 loss2=-3448.4929 lr= 0.000100
  2073. 2019-03-12 14:59:20,858 : epoch 221 , step 80 loss0= 0.0000 loss1= 2.9814 loss2=-3420.7375 lr= 0.000100
  2074. 2019-03-12 14:59:55,070 : epoch 221 , step 120 loss0= 0.0000 loss1= 3.3098 loss2=-3629.7517 lr= 0.000100
  2075. 2019-03-12 15:00:29,115 : epoch 221 , step 160 loss0= 0.0000 loss1= 2.8388 loss2=-3717.4387 lr= 0.000100
  2076. 2019-03-12 15:01:00,231 : epoch 222 , step 0 loss0= 0.0000 loss1= 3.1588 loss2=-3510.4998 lr= 0.000100
  2077. 2019-03-12 15:01:34,170 : epoch 222 , step 40 loss0= 0.0000 loss1= 3.7963 loss2=-3742.1387 lr= 0.000100
  2078. 2019-03-12 15:02:07,957 : epoch 222 , step 80 loss0= 0.0000 loss1= 4.0938 loss2=-3713.2661 lr= 0.000100
  2079. 2019-03-12 15:02:41,871 : epoch 222 , step 120 loss0= 0.0000 loss1= 3.9472 loss2=-3820.6919 lr= 0.000100
  2080. 2019-03-12 15:03:15,835 : epoch 222 , step 160 loss0= 0.0000 loss1= 3.7254 loss2=-3877.2810 lr= 0.000100
  2081. 2019-03-12 15:03:46,759 : epoch 223 , step 0 loss0= 0.0000 loss1= 2.6453 loss2=-3617.3860 lr= 0.000100
  2082. 2019-03-12 15:04:20,760 : epoch 223 , step 40 loss0= 0.0000 loss1= 3.3238 loss2=-3467.7168 lr= 0.000100
  2083. 2019-03-12 15:04:54,779 : epoch 223 , step 80 loss0= 0.0000 loss1= 3.6628 loss2=-3751.0024 lr= 0.000100
  2084. 2019-03-12 15:05:29,364 : epoch 223 , step 120 loss0= 0.0000 loss1= 2.9576 loss2=-3230.4385 lr= 0.000100
  2085. 2019-03-12 15:06:03,427 : epoch 223 , step 160 loss0= 0.0000 loss1= 2.7597 loss2=-3425.3860 lr= 0.000100
  2086. 2019-03-12 15:06:34,589 : epoch 224 , step 0 loss0= 0.0000 loss1= 3.2438 loss2=-3699.9446 lr= 0.000100
  2087. 2019-03-12 15:07:08,975 : epoch 224 , step 40 loss0= 0.0000 loss1= 3.2413 loss2=-3717.9978 lr= 0.000100
  2088. 2019-03-12 15:07:43,060 : epoch 224 , step 80 loss0= 0.0000 loss1= 2.6466 loss2=-3666.4714 lr= 0.000100
  2089. 2019-03-12 15:08:17,211 : epoch 224 , step 120 loss0= 0.0000 loss1= 2.8154 loss2=-3432.1138 lr= 0.000100
  2090. 2019-03-12 15:08:51,538 : epoch 224 , step 160 loss0= 0.0000 loss1= 3.1796 loss2=-3725.6541 lr= 0.000100
  2091. 2019-03-12 15:09:22,650 : epoch 225 , step 0 loss0= 0.0000 loss1= 2.8268 loss2=-3214.9438 lr= 0.000100
  2092. 2019-03-12 15:09:56,635 : epoch 225 , step 40 loss0= 0.0000 loss1= 4.1009 loss2=-4038.2051 lr= 0.000100
  2093. 2019-03-12 15:10:30,504 : epoch 225 , step 80 loss0= 0.0000 loss1= 2.3925 loss2=-3458.2935 lr= 0.000100
  2094. 2019-03-12 15:11:04,833 : epoch 225 , step 120 loss0= 0.0000 loss1= 4.5747 loss2=-3881.9978 lr= 0.000100
  2095. 2019-03-12 15:11:39,006 : epoch 225 , step 160 loss0= 0.0000 loss1= 3.0113 loss2=-3675.1680 lr= 0.000100
  2096. 2019-03-12 15:12:09,881 : epoch 226 , step 0 loss0= 0.0000 loss1= 2.6596 loss2=-3819.8601 lr= 0.000100
  2097. 2019-03-12 15:12:44,622 : epoch 226 , step 40 loss0= 0.0000 loss1= 2.7919 loss2=-3635.7844 lr= 0.000100
  2098. 2019-03-12 15:13:19,394 : epoch 226 , step 80 loss0= 0.0000 loss1= 2.9040 loss2=-3552.7810 lr= 0.000100
  2099. 2019-03-12 15:13:53,799 : epoch 226 , step 120 loss0= 0.0000 loss1= 3.8836 loss2=-3530.6453 lr= 0.000100
  2100. 2019-03-12 15:14:28,411 : epoch 226 , step 160 loss0= 0.0000 loss1= 4.5288 loss2=-3892.3025 lr= 0.000100
  2101. 2019-03-12 15:14:59,442 : epoch 227 , step 0 loss0= 0.0000 loss1= 3.3522 loss2=-3725.1738 lr= 0.000100
  2102. 2019-03-12 15:15:33,512 : epoch 227 , step 40 loss0= 0.0000 loss1= 3.2587 loss2=-3957.4609 lr= 0.000100
  2103. 2019-03-12 15:16:07,907 : epoch 227 , step 80 loss0= 0.0000 loss1= 3.4891 loss2=-3483.7871 lr= 0.000100
  2104. 2019-03-12 15:16:42,273 : epoch 227 , step 120 loss0= 0.0000 loss1= 3.4213 loss2=-3483.1853 lr= 0.000100
  2105. 2019-03-12 15:17:16,581 : epoch 227 , step 160 loss0= 0.0000 loss1= 2.4780 loss2=-3540.4426 lr= 0.000100
  2106. 2019-03-12 15:17:47,485 : epoch 228 , step 0 loss0= 0.0000 loss1= 2.1743 loss2=-3505.8228 lr= 0.000100
  2107. 2019-03-12 15:18:22,014 : epoch 228 , step 40 loss0= 0.0000 loss1= 2.7289 loss2=-3327.0376 lr= 0.000100
  2108. 2019-03-12 15:18:56,465 : epoch 228 , step 80 loss0= 0.0000 loss1= 2.0148 loss2=-3369.2649 lr= 0.000100
  2109. 2019-03-12 15:19:30,615 : epoch 228 , step 120 loss0= 0.0000 loss1= 2.9706 loss2=-3544.7988 lr= 0.000100
  2110. 2019-03-12 15:20:04,484 : epoch 228 , step 160 loss0= 0.0000 loss1= 3.1118 loss2=-3548.4868 lr= 0.000100
  2111. 2019-03-12 15:20:35,484 : epoch 229 , step 0 loss0= 0.0000 loss1= 2.4580 loss2=-3504.4773 lr= 0.000100
  2112. 2019-03-12 15:21:09,655 : epoch 229 , step 40 loss0= 0.0000 loss1= 2.4256 loss2=-3448.1125 lr= 0.000100
  2113. 2019-03-12 15:21:43,709 : epoch 229 , step 80 loss0= 0.0000 loss1= 2.6308 loss2=-3357.2656 lr= 0.000100
  2114. 2019-03-12 15:22:17,621 : epoch 229 , step 120 loss0= 0.0000 loss1= 2.7192 loss2=-3328.6477 lr= 0.000100
  2115. 2019-03-12 15:22:51,677 : epoch 229 , step 160 loss0= 0.0000 loss1= 2.0649 loss2=-3387.8665 lr= 0.000100
  2116. 2019-03-12 15:23:22,655 : epoch 230 , step 0 loss0= 0.0000 loss1= 3.9285 loss2=-3526.6782 lr= 0.000100
  2117. 2019-03-12 15:23:56,808 : epoch 230 , step 40 loss0= 0.0000 loss1= 4.2677 loss2=-3608.4875 lr= 0.000100
  2118. 2019-03-12 15:24:31,134 : epoch 230 , step 80 loss0= 0.0000 loss1= 2.7135 loss2=-3684.5020 lr= 0.000100
  2119. 2019-03-12 15:25:05,051 : epoch 230 , step 120 loss0= 0.0000 loss1= 2.5235 loss2=-3554.3552 lr= 0.000100
  2120. 2019-03-12 15:25:39,134 : epoch 230 , step 160 loss0= 0.0000 loss1= 2.9650 loss2=-3820.5916 lr= 0.000100
  2121. 2019-03-12 15:26:10,135 : epoch 231 , step 0 loss0= 0.0000 loss1= 3.3705 loss2=-3713.4180 lr= 0.000100
  2122. 2019-03-12 15:26:44,302 : epoch 231 , step 40 loss0= 0.0000 loss1= 3.6377 loss2=-3723.3481 lr= 0.000100
  2123. 2019-03-12 15:27:18,508 : epoch 231 , step 80 loss0= 0.0000 loss1= 3.2821 loss2=-3759.4695 lr= 0.000100
  2124. 2019-03-12 15:27:52,586 : epoch 231 , step 120 loss0= 0.0000 loss1= 2.8485 loss2=-3553.3977 lr= 0.000100
  2125. 2019-03-12 15:28:26,720 : epoch 231 , step 160 loss0= 0.0000 loss1= 2.8905 loss2=-3700.6294 lr= 0.000100
  2126. 2019-03-12 15:28:57,449 : epoch 232 , step 0 loss0= 0.0000 loss1= 3.4712 loss2=-3719.1951 lr= 0.000100
  2127. 2019-03-12 15:29:31,530 : epoch 232 , step 40 loss0= 0.0000 loss1= 2.5436 loss2=-3688.9158 lr= 0.000100
  2128. 2019-03-12 15:30:05,464 : epoch 232 , step 80 loss0= 0.0000 loss1= 4.0065 loss2=-3660.8938 lr= 0.000100
  2129. 2019-03-12 15:30:39,690 : epoch 232 , step 120 loss0= 0.0000 loss1= 2.5741 loss2=-3220.1936 lr= 0.000100
  2130. 2019-03-12 15:31:13,623 : epoch 232 , step 160 loss0= 0.0000 loss1= 3.6887 loss2=-3620.4243 lr= 0.000100
  2131. 2019-03-12 15:31:45,087 : epoch 233 , step 0 loss0= 0.0000 loss1= 2.6973 loss2=-3468.8259 lr= 0.000100
  2132. 2019-03-12 15:32:19,518 : epoch 233 , step 40 loss0= 0.0000 loss1= 3.1011 loss2=-3738.0984 lr= 0.000100
  2133. 2019-03-12 15:32:53,817 : epoch 233 , step 80 loss0= 0.0000 loss1= 4.6818 loss2=-3933.4121 lr= 0.000100
  2134. 2019-03-12 15:33:27,926 : epoch 233 , step 120 loss0= 0.0000 loss1= 3.6666 loss2=-3824.3464 lr= 0.000100
  2135. 2019-03-12 15:34:01,942 : epoch 233 , step 160 loss0= 0.0000 loss1= 3.0657 loss2=-3634.1243 lr= 0.000100
  2136. 2019-03-12 15:34:33,013 : epoch 234 , step 0 loss0= 0.0000 loss1= 3.7271 loss2=-3390.3215 lr= 0.000100
  2137. 2019-03-12 15:35:07,270 : epoch 234 , step 40 loss0= 0.0000 loss1= 4.0500 loss2=-3471.8677 lr= 0.000100
  2138. 2019-03-12 15:35:41,438 : epoch 234 , step 80 loss0= 0.0000 loss1= 2.7996 loss2=-3591.5720 lr= 0.000100
  2139. 2019-03-12 15:36:15,512 : epoch 234 , step 120 loss0= 0.0000 loss1= 3.2156 loss2=-3686.6138 lr= 0.000100
  2140. 2019-03-12 15:36:49,945 : epoch 234 , step 160 loss0= 0.0000 loss1= 2.6464 loss2=-3452.8345 lr= 0.000100
  2141. 2019-03-12 15:37:20,936 : epoch 235 , step 0 loss0= 0.0000 loss1= 2.1910 loss2=-3579.8945 lr= 0.000100
  2142. 2019-03-12 15:37:54,741 : epoch 235 , step 40 loss0= 0.0000 loss1= 2.3625 loss2=-3634.8162 lr= 0.000100
  2143. 2019-03-12 15:38:28,533 : epoch 235 , step 80 loss0= 0.0000 loss1= 2.1914 loss2=-3274.2424 lr= 0.000100
  2144. 2019-03-12 15:39:02,989 : epoch 235 , step 120 loss0= 0.0000 loss1= 2.7756 loss2=-3425.7466 lr= 0.000100
  2145. 2019-03-12 15:39:36,990 : epoch 235 , step 160 loss0= 0.0000 loss1= 2.2418 loss2=-3556.4753 lr= 0.000100
  2146. 2019-03-12 15:40:07,692 : epoch 236 , step 0 loss0= 0.0000 loss1= 2.4697 loss2=-3662.2812 lr= 0.000100
  2147. 2019-03-12 15:40:42,274 : epoch 236 , step 40 loss0= 0.0000 loss1= 3.3604 loss2=-3624.5669 lr= 0.000100
  2148. 2019-03-12 15:41:17,314 : epoch 236 , step 80 loss0= 0.0000 loss1= 2.1305 loss2=-3419.1965 lr= 0.000100
  2149. 2019-03-12 15:41:52,602 : epoch 236 , step 120 loss0= 0.0000 loss1= 3.7513 loss2=-3913.8228 lr= 0.000100
  2150. 2019-03-12 15:42:27,491 : epoch 236 , step 160 loss0= 0.0000 loss1= 3.6774 loss2=-3836.5813 lr= 0.000100
  2151. 2019-03-12 15:42:58,770 : epoch 237 , step 0 loss0= 0.0000 loss1= 2.7849 loss2=-3555.2559 lr= 0.000100
  2152. 2019-03-12 15:43:32,777 : epoch 237 , step 40 loss0= 0.0000 loss1= 2.6916 loss2=-3723.1587 lr= 0.000100
  2153. 2019-03-12 15:44:06,758 : epoch 237 , step 80 loss0= 0.0000 loss1= 2.3634 loss2=-3332.8223 lr= 0.000100
  2154. 2019-03-12 15:44:40,816 : epoch 237 , step 120 loss0= 0.0000 loss1= 3.0501 loss2=-3530.7136 lr= 0.000100
  2155. 2019-03-12 15:45:15,073 : epoch 237 , step 160 loss0= 0.0000 loss1= 4.4260 loss2=-3913.3748 lr= 0.000100
  2156. 2019-03-12 15:45:46,203 : epoch 238 , step 0 loss0= 0.0000 loss1= 2.3987 loss2=-3219.3418 lr= 0.000100
  2157. 2019-03-12 15:46:20,183 : epoch 238 , step 40 loss0= 0.0000 loss1= 3.7968 loss2=-4033.3005 lr= 0.000100
  2158. 2019-03-12 15:46:54,153 : epoch 238 , step 80 loss0= 0.0000 loss1= 2.6505 loss2=-3740.3926 lr= 0.000100
  2159. 2019-03-12 15:47:28,481 : epoch 238 , step 120 loss0= 0.0000 loss1= 3.3677 loss2=-3697.5984 lr= 0.000100
  2160. 2019-03-12 15:48:02,577 : epoch 238 , step 160 loss0= 0.0000 loss1= 3.6382 loss2=-3794.1406 lr= 0.000100
  2161. 2019-03-12 15:48:33,976 : epoch 239 , step 0 loss0= 0.0000 loss1= 3.3053 loss2=-3620.0632 lr= 0.000100
  2162. 2019-03-12 15:49:08,938 : epoch 239 , step 40 loss0= 0.0000 loss1= 2.9322 loss2=-3409.7844 lr= 0.000100
  2163. 2019-03-12 15:49:43,525 : epoch 239 , step 80 loss0= 0.0000 loss1= 3.2018 loss2=-3869.5020 lr= 0.000100
  2164. 2019-03-12 15:50:18,335 : epoch 239 , step 120 loss0= 0.0000 loss1= 3.5069 loss2=-3430.6853 lr= 0.000100
  2165. 2019-03-12 15:50:52,795 : epoch 239 , step 160 loss0= 0.0000 loss1= 2.2996 loss2=-3100.8281 lr= 0.000100
  2166. 2019-03-12 15:51:24,207 : epoch 240 , step 0 loss0= 0.0000 loss1= 2.7634 loss2=-3634.2524 lr= 0.000100
  2167. 2019-03-12 15:51:59,134 : epoch 240 , step 40 loss0= 0.0000 loss1= 3.6750 loss2=-3517.5095 lr= 0.000100
  2168. 2019-03-12 15:52:33,903 : epoch 240 , step 80 loss0= 0.0000 loss1= 3.0084 loss2=-3378.5005 lr= 0.000100
  2169. 2019-03-12 15:53:08,612 : epoch 240 , step 120 loss0= 0.0000 loss1= 2.4698 loss2=-3491.5017 lr= 0.000100
  2170. 2019-03-12 15:53:43,431 : epoch 240 , step 160 loss0= 0.0000 loss1= 2.0876 loss2=-3369.1978 lr= 0.000100
  2171. 2019-03-12 15:54:13,511 : ## train ##
  2172. 2019-03-12 15:58:46,963 : >> epoch 240 : average_loss= 0.7499, average_accuracy=82.064%, mean_weight_std= 0.276, mean_normalized_std= 0.4186 mean_dist=2.631e+03
  2173. 2019-03-12 15:58:46,972 : >> epoch 240 : median_loss= 0.7539, median_accuracy=82.125%, median_weight_std= 0.1711, median_normalized_std= 0.0856 median_dist=2.581e+03
  2174. 2019-03-12 15:58:46,972 : >> epoch 240 : ensemble accuracy: 84.66%
  2175. 2019-03-12 15:58:46,972 : >> epoch 240 : loss histogram: (0.00,0.32, 18),(0.32,0.64, 0),(0.64,0.95, 4),(0.95,1.27, 0),(1.27,1.59, 18),
  2176. 2019-03-12 15:58:46,973 : >> epoch 240 : accuracy histogram: (0.00,33.34, 18),(33.34,66.68, 0),(66.68,100.02, 4),(100.02,133.36, 0),(133.36,166.70, 18),
  2177. 2019-03-12 15:58:46,979 : >> epoch 240 : weight std histogram: (0.0000,2.3808, 464100),(2.3808,4.7616, 2434),(4.7616,7.1424, 313),(7.1424,9.5233, 52),(9.5233,11.9041, 7),
  2178. 2019-03-12 15:58:46,984 : >> epoch 240 : weight normalized std histogram: (0.0000,906.0647, 466892),(906.0647,1812.1295, 7),(1812.1295,2718.1942, 5),(2718.1942,3624.2590, 0),(3624.2590,4530.3237, 2),
  2179. 2019-03-12 15:58:46,984 : >> epoch 240 : dist histogram: (2392.2556,2511.0350, 5),(2511.0350,2629.8144, 8),(2629.8144,2748.5938, 3),(2748.5938,2867.3732, 1),(2867.3732,2986.1526, 3),
  2180. 2019-03-12 15:58:46,984 : ## eval ##
  2181. 2019-03-12 16:03:17,218 : >> epoch 240 : average_loss= 0.6380, average_accuracy=85.424%, mean_weight_std= 0.2592, mean_normalized_std= 0.5745 mean_dist=2.631e+03
  2182. 2019-03-12 16:03:17,227 : >> epoch 240 : median_loss= 0.6092, median_accuracy=85.835%, median_weight_std= 0.1606, median_normalized_std= 0.0805 median_dist=2.622e+03
  2183. 2019-03-12 16:03:17,227 : >> epoch 240 : ensemble accuracy: 87.78%
  2184. 2019-03-12 16:03:17,227 : >> epoch 240 : loss histogram: (0.00,0.29, 18),(0.29,0.58, 0),(0.58,0.87, 4),(0.87,1.17, 0),(1.17,1.46, 18),
  2185. 2019-03-12 16:03:17,227 : >> epoch 240 : accuracy histogram: (0.00,34.81, 18),(34.81,69.61, 0),(69.61,104.42, 4),(104.42,139.22, 0),(139.22,174.03, 18),
  2186. 2019-03-12 16:03:17,233 : >> epoch 240 : weight std histogram: (0.0000,2.4019, 464414),(2.4019,4.8038, 2224),(4.8038,7.2057, 240),(7.2057,9.6076, 23),(9.6076,12.0094, 5),
  2187. 2019-03-12 16:03:17,238 : >> epoch 240 : weight normalized std histogram: (0.0000,13313.5797, 466905),(13313.5797,26627.1594, 0),(26627.1594,39940.7391, 0),(39940.7391,53254.3187, 0),(53254.3187,66567.8984, 1),
  2188. 2019-03-12 16:03:17,238 : >> epoch 240 : dist histogram: (2309.9050,2417.7196, 1),(2417.7196,2525.5341, 4),(2525.5341,2633.3487, 5),(2633.3487,2741.1632, 6),(2741.1632,2848.9778, 4),
  2189. 2019-03-12 16:03:17,296 : >> epoch 240 : dist pair-wise
  2190. 2019-03-12 16:03:17,301 :
  2191. [[ 0. 491.687 389.706 506.038 601.578 508.448 227.649 329.785
  2192. 358.968 311.136 315.598 401.163 220.666 418.288 314.095 565.981
  2193. 298.934 369.473 404.819 437.906]
  2194. [ 491.687 0. 439.544 623.048 378.847 472.154 415.425 413.828
  2195. 475.642 436.107 460.934 439.368 496.022 461.221 478.036 538.188
  2196. 452.878 605.627 415.841 504.88 ]
  2197. [ 389.706 439.544 0. 338.015 577.044 534.701 372.336 315.856
  2198. 428.151 427.637 351.067 464.033 379.985 162.584 374.809 628.333
  2199. 348.701 341.989 432.251 474.866]
  2200. [ 506.038 623.048 338.015 0. 672.636 568.121 524.253 469.328
  2201. 554.243 561.189 453.357 557.475 444.597 349.915 514.39 674.997
  2202. 498.404 396.993 506.514 643.388]
  2203. [ 601.578 378.847 577.044 672.636 0. 383.974 513.483 513.907
  2204. 582.842 492.471 560.042 515.238 568.668 611.256 579.5 358.366
  2205. 570.125 692.921 424.533 657.452]
  2206. [ 508.448 472.154 534.701 568.121 383.974 0. 430.898 393.094
  2207. 478.664 381.415 414.22 368.511 463.431 570.71 481.58 319.605
  2208. 461.519 567.128 291.245 654.319]
  2209. [ 227.649 415.425 372.336 524.253 513.483 430.898 0. 279.851
  2210. 302.208 195.642 283.783 328.427 313.475 401.903 231.074 485.519
  2211. 229.052 368.618 338.681 373.665]
  2212. [ 329.785 413.828 315.856 469.328 513.907 393.094 279.851 0.
  2213. 346.751 258.767 292.655 314.496 309.645 385.541 322.891 493.433
  2214. 291.402 403.714 256.82 518.312]
  2215. [ 358.968 475.642 428.151 554.243 582.842 478.664 302.208 346.751
  2216. 0. 304.964 231.191 246.18 447.575 432.535 200.772 552.678
  2217. 236.728 364.294 375.212 347.516]
  2218. [ 311.136 436.107 427.637 561.189 492.471 381.415 195.642 258.767
  2219. 304.964 0. 296.996 277.583 365.076 464.398 249.81 416.239
  2220. 239.365 392.769 303.743 439.804]
  2221. [ 315.598 460.934 351.067 453.357 560.042 414.22 283.783 292.655
  2222. 231.191 296.996 0. 270.714 371.039 355.189 205.34 535.727
  2223. 204.207 292.595 298.853 390.741]
  2224. [ 401.163 439.368 464.033 557.475 515.238 368.511 328.427 314.496
  2225. 246.18 277.583 270.714 0. 436.46 481.599 303.424 446.48
  2226. 312.496 424.076 306.826 478.89 ]
  2227. [ 220.666 496.022 379.985 444.597 568.668 463.431 313.475 309.645
  2228. 447.575 365.076 371.039 436.46 0. 438.86 411.779 543.773
  2229. 385.015 432.186 365.41 562.022]
  2230. [ 418.288 461.221 162.584 349.915 611.256 570.71 401.903 385.541
  2231. 432.535 464.398 355.189 481.599 438.86 0. 377.003 675.08
  2232. 355.136 326.201 491.652 427.408]
  2233. [ 314.095 478.036 374.809 514.39 579.5 481.58 231.074 322.891
  2234. 200.772 249.81 205.34 303.424 411.779 377.003 0. 550.77
  2235. 144.789 293.594 369.239 305.19 ]
  2236. [ 565.981 538.188 628.333 674.997 358.366 319.605 485.519 493.433
  2237. 552.678 416.239 535.727 446.48 543.773 675.08 550.77 0.
  2238. 555.911 639.681 415.066 707.521]
  2239. [ 298.934 452.878 348.701 498.404 570.125 461.519 229.052 291.402
  2240. 236.728 239.365 204.207 312.496 385.015 355.136 144.789 555.911
  2241. 0. 290.147 356.896 343.738]
  2242. [ 369.473 605.627 341.989 396.993 692.921 567.128 368.618 403.714
  2243. 364.294 392.769 292.595 424.076 432.186 326.201 293.594 639.681
  2244. 290.147 0. 488.671 436.275]
  2245. [ 404.819 415.841 432.251 506.514 424.533 291.245 338.681 256.82
  2246. 375.212 303.743 298.853 306.826 365.41 491.652 369.239 415.066
  2247. 356.896 488.671 0. 559.464]
  2248. [ 437.906 504.88 474.866 643.388 657.452 654.319 373.665 518.312
  2249. 347.516 439.804 390.741 478.89 562.022 427.408 305.19 707.521
  2250. 343.738 436.275 559.464 0. ]]
  2251. 2019-03-12 16:03:18,696 : epoch 241 , step 0 loss0= 0.0000 loss1= 2.6711 loss2=-3796.6470 lr= 0.000100
  2252. 2019-03-12 16:03:53,457 : epoch 241 , step 40 loss0= 0.0000 loss1= 3.3583 loss2=-3712.8540 lr= 0.000100
  2253. 2019-03-12 16:04:27,998 : epoch 241 , step 80 loss0= 0.0000 loss1= 3.4040 loss2=-3703.8171 lr= 0.000100
  2254. 2019-03-12 16:05:02,504 : epoch 241 , step 120 loss0= 0.0000 loss1= 3.1248 loss2=-3396.3665 lr= 0.000100
  2255. 2019-03-12 16:05:37,180 : epoch 241 , step 160 loss0= 0.0000 loss1= 3.0588 loss2=-3450.3169 lr= 0.000100
  2256. 2019-03-12 16:06:08,005 : epoch 242 , step 0 loss0= 0.0000 loss1= 2.8491 loss2=-3753.2908 lr= 0.000100
  2257. 2019-03-12 16:06:41,812 : epoch 242 , step 40 loss0= 0.0000 loss1= 3.0934 loss2=-3775.7766 lr= 0.000100
  2258. 2019-03-12 16:07:16,549 : epoch 242 , step 80 loss0= 0.0000 loss1= 2.4751 loss2=-3684.4138 lr= 0.000100
  2259. 2019-03-12 16:07:50,617 : epoch 242 , step 120 loss0= 0.0000 loss1= 3.0794 loss2=-3897.4915 lr= 0.000100
  2260. 2019-03-12 16:08:24,217 : epoch 242 , step 160 loss0= 0.0000 loss1= 2.9568 loss2=-3583.9067 lr= 0.000100
  2261. 2019-03-12 16:08:55,429 : epoch 243 , step 0 loss0= 0.0000 loss1= 2.2702 loss2=-3712.5742 lr= 0.000100
  2262. 2019-03-12 16:09:29,898 : epoch 243 , step 40 loss0= 0.0000 loss1= 3.2735 loss2=-3517.1462 lr= 0.000100
  2263. 2019-03-12 16:10:04,885 : epoch 243 , step 80 loss0= 0.0000 loss1= 3.1303 loss2=-3842.2505 lr= 0.000100
  2264. 2019-03-12 16:10:39,491 : epoch 243 , step 120 loss0= 0.0000 loss1= 3.5277 loss2=-3752.8054 lr= 0.000100
  2265. 2019-03-12 16:11:13,873 : epoch 243 , step 160 loss0= 0.0000 loss1= 3.4487 loss2=-3670.8813 lr= 0.000100
  2266. 2019-03-12 16:11:44,804 : epoch 244 , step 0 loss0= 0.0000 loss1= 3.3219 loss2=-3433.6062 lr= 0.000100
  2267. 2019-03-12 16:12:19,045 : epoch 244 , step 40 loss0= 0.0000 loss1= 3.0504 loss2=-3325.5435 lr= 0.000100
  2268. 2019-03-12 16:12:53,069 : epoch 244 , step 80 loss0= 0.0000 loss1= 2.4588 loss2=-3199.2263 lr= 0.000100
  2269. 2019-03-12 16:13:26,894 : epoch 244 , step 120 loss0= 0.0000 loss1= 2.8685 loss2=-3729.8992 lr= 0.000100
  2270. 2019-03-12 16:14:00,994 : epoch 244 , step 160 loss0= 0.0000 loss1= 2.2336 loss2=-3118.9617 lr= 0.000100
  2271. 2019-03-12 16:14:31,944 : epoch 245 , step 0 loss0= 0.0000 loss1= 2.8557 loss2=-3455.5056 lr= 0.000100
  2272. 2019-03-12 16:15:05,989 : epoch 245 , step 40 loss0= 0.0000 loss1= 1.9222 loss2=-3510.0454 lr= 0.000100
  2273. 2019-03-12 16:15:40,418 : epoch 245 , step 80 loss0= 0.0000 loss1= 2.9761 loss2=-3459.2693 lr= 0.000100
  2274. 2019-03-12 16:16:14,923 : epoch 245 , step 120 loss0= 0.0000 loss1= 3.4069 loss2=-3824.7693 lr= 0.000100
  2275. 2019-03-12 16:16:49,533 : epoch 245 , step 160 loss0= 0.0000 loss1= 2.6899 loss2=-3783.6658 lr= 0.000100
  2276. 2019-03-12 16:17:20,754 : epoch 246 , step 0 loss0= 0.0000 loss1= 3.5620 loss2=-3953.2578 lr= 0.000100
  2277. 2019-03-12 16:17:54,834 : epoch 246 , step 40 loss0= 0.0000 loss1= 3.1130 loss2=-3515.2859 lr= 0.000100
  2278. 2019-03-12 16:18:28,996 : epoch 246 , step 80 loss0= 0.0000 loss1= 3.0016 loss2=-3475.4587 lr= 0.000100
  2279. 2019-03-12 16:19:03,210 : epoch 246 , step 120 loss0= 0.0000 loss1= 3.4312 loss2=-3671.1013 lr= 0.000100
  2280. 2019-03-12 16:19:37,476 : epoch 246 , step 160 loss0= 0.0000 loss1= 3.1348 loss2=-3678.1333 lr= 0.000100
  2281. 2019-03-12 16:20:08,829 : epoch 247 , step 0 loss0= 0.0000 loss1= 2.9387 loss2=-3195.7869 lr= 0.000100
  2282. 2019-03-12 16:20:43,228 : epoch 247 , step 40 loss0= 0.0000 loss1= 2.8906 loss2=-3816.4583 lr= 0.000100
  2283. 2019-03-12 16:21:17,755 : epoch 247 , step 80 loss0= 0.0000 loss1= 2.9153 loss2=-3824.5681 lr= 0.000100
  2284. 2019-03-12 16:21:51,937 : epoch 247 , step 120 loss0= 0.0000 loss1= 2.5095 loss2=-3813.3025 lr= 0.000100
  2285. 2019-03-12 16:22:26,449 : epoch 247 , step 160 loss0= 0.0000 loss1= 3.1259 loss2=-3752.6335 lr= 0.000100
  2286. 2019-03-12 16:22:57,834 : epoch 248 , step 0 loss0= 0.0000 loss1= 2.2065 loss2=-3472.0723 lr= 0.000100
  2287. 2019-03-12 16:23:32,294 : epoch 248 , step 40 loss0= 0.0000 loss1= 3.0833 loss2=-3696.3196 lr= 0.000100
  2288. 2019-03-12 16:24:06,355 : epoch 248 , step 80 loss0= 0.0000 loss1= 3.0416 loss2=-3848.0669 lr= 0.000100
  2289. 2019-03-12 16:24:40,510 : epoch 248 , step 120 loss0= 0.0000 loss1= 3.3928 loss2=-3585.5579 lr= 0.000100
  2290. 2019-03-12 16:25:14,355 : epoch 248 , step 160 loss0= 0.0000 loss1= 2.5299 loss2=-3702.3391 lr= 0.000100
  2291. 2019-03-12 16:25:45,541 : epoch 249 , step 0 loss0= 0.0000 loss1= 2.6254 loss2=-3710.9783 lr= 0.000100
  2292. 2019-03-12 16:26:20,202 : epoch 249 , step 40 loss0= 0.0000 loss1= 2.9560 loss2=-3471.2361 lr= 0.000100
  2293. 2019-03-12 16:26:54,887 : epoch 249 , step 80 loss0= 0.0000 loss1= 2.6822 loss2=-3515.8611 lr= 0.000100
  2294. 2019-03-12 16:27:28,599 : epoch 249 , step 120 loss0= 0.0000 loss1= 2.5623 loss2=-3445.2871 lr= 0.000100
  2295. 2019-03-12 16:28:02,980 : epoch 249 , step 160 loss0= 0.0000 loss1= 3.3041 loss2=-3688.2766 lr= 0.000100
  2296. 2019-03-12 16:28:34,263 : epoch 250 , step 0 loss0= 0.0000 loss1= 2.2923 loss2=-3545.9919 lr= 0.000100
  2297. 2019-03-12 16:29:08,745 : epoch 250 , step 40 loss0= 0.0000 loss1= 3.0643 loss2=-3645.2852 lr= 0.000100
  2298. 2019-03-12 16:29:43,469 : epoch 250 , step 80 loss0= 0.0000 loss1= 3.0032 loss2=-3778.6665 lr= 0.000100
  2299. 2019-03-12 16:30:18,432 : epoch 250 , step 120 loss0= 0.0000 loss1= 3.4743 loss2=-3304.3552 lr= 0.000100
  2300. 2019-03-12 16:30:52,787 : epoch 250 , step 160 loss0= 0.0000 loss1= 3.2139 loss2=-3799.3691 lr= 0.000100
  2301. 2019-03-12 16:31:23,940 : epoch 251 , step 0 loss0= 0.0000 loss1= 3.1811 loss2=-3802.0364 lr= 0.000100
  2302. 2019-03-12 16:31:58,507 : epoch 251 , step 40 loss0= 0.0000 loss1= 3.9945 loss2=-3816.8494 lr= 0.000100
  2303. 2019-03-12 16:32:32,879 : epoch 251 , step 80 loss0= 0.0000 loss1= 2.7328 loss2=-3469.1443 lr= 0.000100
  2304. 2019-03-12 16:33:07,191 : epoch 251 , step 120 loss0= 0.0000 loss1= 2.3283 loss2=-3575.1489 lr= 0.000100
  2305. 2019-03-12 16:33:41,504 : epoch 251 , step 160 loss0= 0.0000 loss1= 3.1417 loss2=-3564.3875 lr= 0.000100
  2306. 2019-03-12 16:34:12,574 : epoch 252 , step 0 loss0= 0.0000 loss1= 3.3788 loss2=-3678.8735 lr= 0.000100
  2307. 2019-03-12 16:34:47,124 : epoch 252 , step 40 loss0= 0.0000 loss1= 2.1627 loss2=-3460.6726 lr= 0.000100
  2308. 2019-03-12 16:35:21,042 : epoch 252 , step 80 loss0= 0.0000 loss1= 3.0123 loss2=-3693.8989 lr= 0.000100
  2309. 2019-03-12 16:35:55,662 : epoch 252 , step 120 loss0= 0.0000 loss1= 1.8813 loss2=-2962.4849 lr= 0.000100
  2310. 2019-03-12 16:36:30,340 : epoch 252 , step 160 loss0= 0.0000 loss1= 2.2555 loss2=-3352.3943 lr= 0.000100
  2311. 2019-03-12 16:37:01,851 : epoch 253 , step 0 loss0= 0.0000 loss1= 2.6516 loss2=-3561.5083 lr= 0.000100
  2312. 2019-03-12 16:37:36,265 : epoch 253 , step 40 loss0= 0.0000 loss1= 3.2118 loss2=-3765.1431 lr= 0.000100
  2313. 2019-03-12 16:38:11,278 : epoch 253 , step 80 loss0= 0.0000 loss1= 2.7033 loss2=-3082.2354 lr= 0.000100
  2314. 2019-03-12 16:38:45,961 : epoch 253 , step 120 loss0= 0.0000 loss1= 2.6406 loss2=-3794.9622 lr= 0.000100
  2315. 2019-03-12 16:39:20,660 : epoch 253 , step 160 loss0= 0.0000 loss1= 2.6990 loss2=-3543.7673 lr= 0.000100
  2316. 2019-03-12 16:39:52,282 : epoch 254 , step 0 loss0= 0.0000 loss1= 2.0417 loss2=-3600.9197 lr= 0.000100
  2317. 2019-03-12 16:40:26,371 : epoch 254 , step 40 loss0= 0.0000 loss1= 3.4947 loss2=-3410.4023 lr= 0.000100
  2318. 2019-03-12 16:41:00,490 : epoch 254 , step 80 loss0= 0.0000 loss1= 3.3057 loss2=-3598.0552 lr= 0.000100
  2319. 2019-03-12 16:41:34,631 : epoch 254 , step 120 loss0= 0.0000 loss1= 3.1558 loss2=-3815.8501 lr= 0.000100
  2320. 2019-03-12 16:42:09,463 : epoch 254 , step 160 loss0= 0.0000 loss1= 2.9015 loss2=-3897.7439 lr= 0.000100
  2321. 2019-03-12 16:42:40,562 : epoch 255 , step 0 loss0= 0.0000 loss1= 2.6038 loss2=-3528.0864 lr= 0.000100
  2322. 2019-03-12 16:43:15,170 : epoch 255 , step 40 loss0= 0.0000 loss1= 3.7475 loss2=-3941.3997 lr= 0.000100
  2323. 2019-03-12 16:43:49,563 : epoch 255 , step 80 loss0= 0.0000 loss1= 2.9459 loss2=-3847.1743 lr= 0.000100
  2324. 2019-03-12 16:44:24,345 : epoch 255 , step 120 loss0= 0.0000 loss1= 1.7487 loss2=-3168.1387 lr= 0.000100
  2325. 2019-03-12 16:44:58,686 : epoch 255 , step 160 loss0= 0.0000 loss1= 2.4473 loss2=-3414.7817 lr= 0.000100
  2326. 2019-03-12 16:45:29,779 : epoch 256 , step 0 loss0= 0.0000 loss1= 3.3373 loss2=-3796.8208 lr= 0.000100
  2327. 2019-03-12 16:46:04,418 : epoch 256 , step 40 loss0= 0.0000 loss1= 3.5865 loss2=-3701.6255 lr= 0.000100
  2328. 2019-03-12 16:46:38,894 : epoch 256 , step 80 loss0= 0.0000 loss1= 2.9546 loss2=-3689.0161 lr= 0.000100
  2329. 2019-03-12 16:47:13,686 : epoch 256 , step 120 loss0= 0.0000 loss1= 3.1865 loss2=-3474.6060 lr= 0.000100
  2330. 2019-03-12 16:47:48,444 : epoch 256 , step 160 loss0= 0.0000 loss1= 3.1794 loss2=-3552.0435 lr= 0.000100
  2331. 2019-03-12 16:48:20,126 : epoch 257 , step 0 loss0= 0.0000 loss1= 2.8895 loss2=-3627.5879 lr= 0.000100
  2332. 2019-03-12 16:48:54,689 : epoch 257 , step 40 loss0= 0.0000 loss1= 2.1634 loss2=-3560.1028 lr= 0.000100
  2333. 2019-03-12 16:49:29,567 : epoch 257 , step 80 loss0= 0.0000 loss1= 3.6073 loss2=-3625.2429 lr= 0.000100
  2334. 2019-03-12 16:50:04,024 : epoch 257 , step 120 loss0= 0.0000 loss1= 3.3191 loss2=-3669.8625 lr= 0.000100
  2335. 2019-03-12 16:50:38,081 : epoch 257 , step 160 loss0= 0.0000 loss1= 2.6977 loss2=-3525.8345 lr= 0.000100
  2336. 2019-03-12 16:51:09,117 : epoch 258 , step 0 loss0= 0.0000 loss1= 2.7386 loss2=-3270.4404 lr= 0.000100
  2337. 2019-03-12 16:51:43,234 : epoch 258 , step 40 loss0= 0.0000 loss1= 2.7568 loss2=-3577.4844 lr= 0.000100
  2338. 2019-03-12 16:52:17,166 : epoch 258 , step 80 loss0= 0.0000 loss1= 3.0678 loss2=-3659.7344 lr= 0.000100
  2339. 2019-03-12 16:52:51,737 : epoch 258 , step 120 loss0= 0.0000 loss1= 2.4301 loss2=-3747.8220 lr= 0.000100
  2340. 2019-03-12 16:53:25,872 : epoch 258 , step 160 loss0= 0.0000 loss1= 1.9784 loss2=-3348.7798 lr= 0.000100
  2341. 2019-03-12 16:53:57,222 : epoch 259 , step 0 loss0= 0.0000 loss1= 3.6972 loss2=-3805.3594 lr= 0.000100
  2342. 2019-03-12 16:54:31,307 : epoch 259 , step 40 loss0= 0.0000 loss1= 2.4629 loss2=-3446.6895 lr= 0.000100
  2343. 2019-03-12 16:55:05,364 : epoch 259 , step 80 loss0= 0.0000 loss1= 2.9188 loss2=-3416.3162 lr= 0.000100
  2344. 2019-03-12 16:55:39,168 : epoch 259 , step 120 loss0= 0.0000 loss1= 3.8073 loss2=-3590.8728 lr= 0.000100
  2345. 2019-03-12 16:56:12,926 : epoch 259 , step 160 loss0= 0.0000 loss1= 2.8900 loss2=-3569.7234 lr= 0.000100
  2346. 2019-03-12 16:56:43,564 : epoch 260 , step 0 loss0= 0.0000 loss1= 2.9220 loss2=-3455.5605 lr= 0.000100
  2347. 2019-03-12 16:57:17,394 : epoch 260 , step 40 loss0= 0.0000 loss1= 3.0527 loss2=-3698.1477 lr= 0.000100
  2348. 2019-03-12 16:57:51,028 : epoch 260 , step 80 loss0= 0.0000 loss1= 2.8340 loss2=-3442.8757 lr= 0.000100
  2349. 2019-03-12 16:58:24,801 : epoch 260 , step 120 loss0= 0.0000 loss1= 2.5010 loss2=-3640.0681 lr= 0.000100
  2350. 2019-03-12 16:58:58,568 : epoch 260 , step 160 loss0= 0.0000 loss1= 2.9194 loss2=-3547.0955 lr= 0.000100
  2351. 2019-03-12 16:59:27,950 : ## train ##
  2352. 2019-03-12 17:03:50,710 : >> epoch 260 : average_loss= 0.7744, average_accuracy=82.373%, mean_weight_std= 0.2597, mean_normalized_std= 0.4832 mean_dist=2.692e+03
  2353. 2019-03-12 17:03:50,721 : >> epoch 260 : median_loss= 0.7548, median_accuracy=82.520%, median_weight_std= 0.1596, median_normalized_std= 0.0783 median_dist=2.701e+03
  2354. 2019-03-12 17:03:50,721 : >> epoch 260 : ensemble accuracy: 84.58%
  2355. 2019-03-12 17:03:50,721 : >> epoch 260 : loss histogram: (0.00,0.34, 18),(0.34,0.67, 0),(0.67,1.01, 4),(1.01,1.35, 0),(1.35,1.68, 18),
  2356. 2019-03-12 17:03:50,721 : >> epoch 260 : accuracy histogram: (0.00,33.33, 18),(33.33,66.66, 0),(66.66,100.00, 4),(100.00,133.33, 0),(133.33,166.66, 18),
  2357. 2019-03-12 17:03:50,727 : >> epoch 260 : weight std histogram: (0.0000,2.2430, 463928),(2.2430,4.4861, 2593),(4.4861,6.7291, 317),(6.7291,8.9721, 62),(8.9721,11.2151, 6),
  2358. 2019-03-12 17:03:50,732 : >> epoch 260 : weight normalized std histogram: (0.0000,6477.7320, 466905),(6477.7320,12955.4641, 0),(12955.4641,19433.1961, 0),(19433.1961,25910.9281, 0),(25910.9281,32388.6602, 1),
  2359. 2019-03-12 17:03:50,733 : >> epoch 260 : dist histogram: (2394.3950,2509.7688, 1),(2509.7688,2625.1425, 6),(2625.1425,2740.5162, 8),(2740.5162,2855.8899, 3),(2855.8899,2971.2637, 2),
  2360. 2019-03-12 17:03:50,733 : ## eval ##
  2361. 2019-03-12 17:08:15,256 : >> epoch 260 : average_loss= 0.6678, average_accuracy=85.482%, mean_weight_std= 0.3361, mean_normalized_std= 0.4786 mean_dist=2.689e+03
  2362. 2019-03-12 17:08:15,266 : >> epoch 260 : median_loss= 0.6613, median_accuracy=84.605%, median_weight_std= 0.2098, median_normalized_std= 0.1034 median_dist=2.644e+03
  2363. 2019-03-12 17:08:15,266 : >> epoch 260 : ensemble accuracy: 87.69%
  2364. 2019-03-12 17:08:15,266 : >> epoch 260 : loss histogram: (0.00,0.28, 18),(0.28,0.56, 0),(0.56,0.85, 4),(0.85,1.13, 0),(1.13,1.41, 18),
  2365. 2019-03-12 17:08:15,266 : >> epoch 260 : accuracy histogram: (0.00,34.84, 18),(34.84,69.68, 0),(69.68,104.52, 4),(104.52,139.36, 0),(139.36,174.20, 18),
  2366. 2019-03-12 17:08:15,272 : >> epoch 260 : weight std histogram: (0.0000,2.4348, 463285),(2.4348,4.8696, 3127),(4.8696,7.3045, 418),(7.3045,9.7393, 69),(9.7393,12.1741, 7),
  2367. 2019-03-12 17:08:15,277 : >> epoch 260 : weight normalized std histogram: (0.0000,871.8414, 466890),(871.8414,1743.6828, 9),(1743.6828,2615.5242, 3),(2615.5242,3487.3656, 1),(3487.3656,4359.2070, 3),
  2368. 2019-03-12 17:08:15,277 : >> epoch 260 : dist histogram: (2390.1094,2569.9257, 7),(2569.9257,2749.7421, 8),(2749.7421,2929.5584, 3),(2929.5584,3109.3748, 0),(3109.3748,3289.1912, 2),
  2369. 2019-03-12 17:08:15,333 : >> epoch 260 : dist pair-wise
  2370. 2019-03-12 17:08:15,339 :
  2371. [[ 0. 420.44 229.019 249.097 1057.963 364.715 462.553
  2372. 430.51 282.891 354.539 259.939 358.751 358.927 266.16
  2373. 362.081 411.212 339.851 451.008 843.546 381.529]
  2374. [ 420.44 0. 442.403 535.639 975.541 296.996 525.739
  2375. 503.292 403.513 445.374 310.25 472.281 479.631 432.93
  2376. 386.315 489.526 415.042 539.994 969.376 523.281]
  2377. [ 229.019 442.403 0. 288.276 1074.309 365.667 515.627
  2378. 412.746 280.728 301.367 236.13 287.754 269.922 217.312
  2379. 402.415 365.464 319.076 419.323 920.646 365.508]
  2380. [ 249.097 535.639 288.276 0. 1202.693 484.383 639.928
  2381. 578.171 455.682 472.533 352.152 407.319 415.981 285.71
  2382. 554.632 468.339 453.566 543.042 973.22 490.846]
  2383. [ 1057.963 975.541 1074.309 1202.693 0. 931.894 749.425
  2384. 890.233 939.436 943.682 1055.345 984.922 1025.563 1117.721
  2385. 847.756 908.724 895.731 943.171 932.87 960.532]
  2386. [ 364.715 296.996 365.667 484.383 931.894 0. 466.996
  2387. 431.058 307.59 395.125 301.709 343.508 391.13 342.827
  2388. 310.26 344.702 263.735 431.561 959.854 423.325]
  2389. [ 462.553 525.739 515.627 639.928 749.425 466.996 0.
  2390. 370.466 370.205 471.024 521.153 512.746 542.577 562.444
  2391. 306.696 485.17 389.157 442.282 673.651 413.295]
  2392. [ 430.51 503.292 412.746 578.171 890.233 431.058 370.466
  2393. 0. 251.845 405.709 409.728 428.244 426.031 463.877
  2394. 308.227 480.464 396.679 338.326 756.926 278.555]
  2395. [ 282.891 403.513 280.728 455.682 939.436 307.59 370.205
  2396. 251.845 0. 306.943 276.543 343.396 334.782 352.571
  2397. 221.212 395.249 282.603 389.908 806.844 322.047]
  2398. [ 354.539 445.374 301.367 472.533 943.682 395.125 471.024
  2399. 405.709 306.943 0. 331.5 303.506 229.283 362.127
  2400. 346.04 386.672 368.336 449.105 853.585 418.142]
  2401. [ 259.939 310.25 236.13 352.152 1055.345 301.709 521.153
  2402. 409.728 276.543 331.5 0. 325.134 321.712 272.374
  2403. 347.024 422.067 350.932 470.453 937.006 425.734]
  2404. [ 358.751 472.281 287.754 407.319 984.922 343.508 512.746
  2405. 428.244 343.396 303.506 325.134 0. 203.912 279.418
  2406. 407.651 225.432 258.002 392.516 991.47 373.335]
  2407. [ 358.927 479.631 269.922 415.981 1025.563 391.13 542.577
  2408. 426.031 334.782 229.283 321.712 203.912 0. 272.763
  2409. 420.229 339.807 333.822 446.972 966.028 403.509]
  2410. [ 266.16 432.93 217.312 285.71 1117.721 342.827 562.444
  2411. 463.877 352.571 362.127 272.374 279.418 272.763 0.
  2412. 437.543 367.837 332.385 431.326 995.631 378.11 ]
  2413. [ 362.081 386.315 402.415 554.632 847.756 310.26 306.696
  2414. 308.227 221.212 346.04 347.024 407.651 420.229 437.543
  2415. 0. 441.692 319.626 445.105 768.56 394.566]
  2416. [ 411.212 489.526 365.464 468.339 908.724 344.702 485.17
  2417. 480.464 395.249 386.672 422.067 225.432 339.807 367.837
  2418. 441.692 0. 217.967 382.593 995.924 401.732]
  2419. [ 339.851 415.042 319.076 453.566 895.731 263.735 389.157
  2420. 396.679 282.603 368.336 350.932 258.002 333.822 332.385
  2421. 319.626 217.967 0. 384.575 927.812 351.058]
  2422. [ 451.008 539.994 419.323 543.042 943.171 431.561 442.282
  2423. 338.326 389.908 449.105 470.453 392.516 446.972 431.326
  2424. 445.105 382.593 384.575 0. 875.904 193.861]
  2425. [ 843.546 969.376 920.646 973.22 932.87 959.854 673.651
  2426. 756.926 806.844 853.585 937.006 991.47 966.028 995.631
  2427. 768.56 995.924 927.812 875.904 0. 829.181]
  2428. [ 381.529 523.281 365.508 490.846 960.532 423.325 413.295
  2429. 278.555 322.047 418.142 425.734 373.335 403.509 378.11
  2430. 394.566 401.732 351.058 193.861 829.181 0. ]]
  2431. 2019-03-12 17:08:16,713 : epoch 261 , step 0 loss0= 0.0000 loss1= 3.3546 loss2=-3581.7207 lr= 0.000100
  2432. 2019-03-12 17:08:50,712 : epoch 261 , step 40 loss0= 0.0000 loss1= 2.3779 loss2=-3610.5591 lr= 0.000100
  2433. 2019-03-12 17:09:24,916 : epoch 261 , step 80 loss0= 0.0000 loss1= 2.2464 loss2=-3574.6367 lr= 0.000100
  2434. 2019-03-12 17:09:59,419 : epoch 261 , step 120 loss0= 0.0000 loss1= 2.3221 loss2=-3507.1345 lr= 0.000100
  2435. 2019-03-12 17:10:34,062 : epoch 261 , step 160 loss0= 0.0000 loss1= 2.1184 loss2=-3420.0085 lr= 0.000100
  2436. 2019-03-12 17:11:05,157 : epoch 262 , step 0 loss0= 0.0000 loss1= 2.3774 loss2=-3855.1414 lr= 0.000100
  2437. 2019-03-12 17:11:39,500 : epoch 262 , step 40 loss0= 0.0000 loss1= 2.1084 loss2=-3353.7571 lr= 0.000100
  2438. 2019-03-12 17:12:13,956 : epoch 262 , step 80 loss0= 0.0000 loss1= 3.0262 loss2=-3603.9607 lr= 0.000100
  2439. 2019-03-12 17:12:48,652 : epoch 262 , step 120 loss0= 0.0000 loss1= 2.7462 loss2=-3738.3328 lr= 0.000100
  2440. 2019-03-12 17:13:23,225 : epoch 262 , step 160 loss0= 0.0000 loss1= 3.6259 loss2=-3802.4636 lr= 0.000100
  2441. 2019-03-12 17:13:54,450 : epoch 263 , step 0 loss0= 0.0000 loss1= 2.9034 loss2=-3491.3445 lr= 0.000100
  2442. 2019-03-12 17:14:28,882 : epoch 263 , step 40 loss0= 0.0000 loss1= 2.7198 loss2=-3341.2739 lr= 0.000100
  2443. 2019-03-12 17:15:03,528 : epoch 263 , step 80 loss0= 0.0000 loss1= 3.3319 loss2=-3604.5251 lr= 0.000100
  2444. 2019-03-12 17:15:37,780 : epoch 263 , step 120 loss0= 0.0000 loss1= 2.6162 loss2=-3367.8845 lr= 0.000100
  2445. 2019-03-12 17:16:12,604 : epoch 263 , step 160 loss0= 0.0000 loss1= 3.6806 loss2=-3706.1560 lr= 0.000100
  2446. 2019-03-12 17:16:43,411 : epoch 264 , step 0 loss0= 0.0000 loss1= 3.0369 loss2=-3267.0742 lr= 0.000100
  2447. 2019-03-12 17:17:17,483 : epoch 264 , step 40 loss0= 0.0000 loss1= 3.2419 loss2=-3728.2642 lr= 0.000100
  2448. 2019-03-12 17:17:51,462 : epoch 264 , step 80 loss0= 0.0000 loss1= 2.1510 loss2=-3634.8599 lr= 0.000100
  2449. 2019-03-12 17:18:25,831 : epoch 264 , step 120 loss0= 0.0000 loss1= 2.2662 loss2=-3709.9758 lr= 0.000100
  2450. 2019-03-12 17:18:59,870 : epoch 264 , step 160 loss0= 0.0000 loss1= 2.9797 loss2=-3628.5845 lr= 0.000100
  2451. 2019-03-12 17:19:30,840 : epoch 265 , step 0 loss0= 0.0000 loss1= 2.1802 loss2=-3529.2361 lr= 0.000100
  2452. 2019-03-12 17:20:05,655 : epoch 265 , step 40 loss0= 0.0000 loss1= 1.9973 loss2=-3521.5083 lr= 0.000100
  2453. 2019-03-12 17:20:40,808 : epoch 265 , step 80 loss0= 0.0000 loss1= 3.4103 loss2=-3792.7585 lr= 0.000100
  2454. 2019-03-12 17:21:15,380 : epoch 265 , step 120 loss0= 0.0000 loss1= 3.1894 loss2=-3651.4500 lr= 0.000100
  2455. 2019-03-12 17:21:49,589 : epoch 265 , step 160 loss0= 0.0000 loss1= 3.6168 loss2=-3698.1360 lr= 0.000100
  2456. 2019-03-12 17:22:20,969 : epoch 266 , step 0 loss0= 0.0000 loss1= 1.9150 loss2=-3610.1875 lr= 0.000100
  2457. 2019-03-12 17:22:55,570 : epoch 266 , step 40 loss0= 0.0000 loss1= 1.7732 loss2=-3379.4380 lr= 0.000100
  2458. 2019-03-12 17:23:30,039 : epoch 266 , step 80 loss0= 0.0000 loss1= 2.7640 loss2=-3707.8430 lr= 0.000100
  2459. 2019-03-12 17:24:04,275 : epoch 266 , step 120 loss0= 0.0000 loss1= 2.9667 loss2=-3730.2629 lr= 0.000100
  2460. 2019-03-12 17:24:38,331 : epoch 266 , step 160 loss0= 0.0000 loss1= 2.8184 loss2=-3419.0149 lr= 0.000100
  2461. 2019-03-12 17:25:10,100 : epoch 267 , step 0 loss0= 0.0000 loss1= 2.6087 loss2=-3233.0532 lr= 0.000100
  2462. 2019-03-12 17:25:43,976 : epoch 267 , step 40 loss0= 0.0000 loss1= 3.0311 loss2=-3822.3638 lr= 0.000100
  2463. 2019-03-12 17:26:17,711 : epoch 267 , step 80 loss0= 0.0000 loss1= 2.2620 loss2=-3536.8345 lr= 0.000100
  2464. 2019-03-12 17:26:51,807 : epoch 267 , step 120 loss0= 0.0000 loss1= 3.1591 loss2=-3564.9910 lr= 0.000100
  2465. 2019-03-12 17:27:26,733 : epoch 267 , step 160 loss0= 0.0000 loss1= 2.5843 loss2=-3571.4568 lr= 0.000100
  2466. 2019-03-12 17:27:58,471 : epoch 268 , step 0 loss0= 0.0000 loss1= 4.1677 loss2=-3671.3064 lr= 0.000100
  2467. 2019-03-12 17:28:32,714 : epoch 268 , step 40 loss0= 0.0000 loss1= 2.9680 loss2=-3689.5632 lr= 0.000100
  2468. 2019-03-12 17:29:07,239 : epoch 268 , step 80 loss0= 0.0000 loss1= 2.8882 loss2=-3637.3848 lr= 0.000100
  2469. 2019-03-12 17:29:41,678 : epoch 268 , step 120 loss0= 0.0000 loss1= 3.0026 loss2=-3807.8357 lr= 0.000100
  2470. 2019-03-12 17:30:16,043 : epoch 268 , step 160 loss0= 0.0000 loss1= 2.6950 loss2=-3637.8000 lr= 0.000100
  2471. 2019-03-12 17:30:47,575 : epoch 269 , step 0 loss0= 0.0000 loss1= 2.4441 loss2=-3947.8242 lr= 0.000100
  2472. 2019-03-12 17:31:22,117 : epoch 269 , step 40 loss0= 0.0000 loss1= 3.1375 loss2=-3879.2478 lr= 0.000100
  2473. 2019-03-12 17:31:56,828 : epoch 269 , step 80 loss0= 0.0000 loss1= 3.4335 loss2=-3701.0618 lr= 0.000100
  2474. 2019-03-12 17:32:31,808 : epoch 269 , step 120 loss0= 0.0000 loss1= 2.5126 loss2=-3290.0630 lr= 0.000100
  2475. 2019-03-12 17:33:06,864 : epoch 269 , step 160 loss0= 0.0000 loss1= 3.6131 loss2=-3756.2585 lr= 0.000100
  2476. 2019-03-12 17:33:38,269 : epoch 270 , step 0 loss0= 0.0000 loss1= 2.6538 loss2=-3715.4199 lr= 0.000100
  2477. 2019-03-12 17:34:12,511 : epoch 270 , step 40 loss0= 0.0000 loss1= 2.1516 loss2=-3445.5142 lr= 0.000100
  2478. 2019-03-12 17:34:46,614 : epoch 270 , step 80 loss0= 0.0000 loss1= 2.9794 loss2=-3946.5317 lr= 0.000100
  2479. 2019-03-12 17:35:20,808 : epoch 270 , step 120 loss0= 0.0000 loss1= 3.4380 loss2=-3607.2332 lr= 0.000100
  2480. 2019-03-12 17:35:55,006 : epoch 270 , step 160 loss0= 0.0000 loss1= 2.3324 loss2=-3698.7715 lr= 0.000100
  2481. 2019-03-12 17:36:26,129 : epoch 271 , step 0 loss0= 0.0000 loss1= 3.3397 loss2=-3403.4407 lr= 0.000100
  2482. 2019-03-12 17:37:00,794 : epoch 271 , step 40 loss0= 0.0000 loss1= 1.8457 loss2=-3506.1816 lr= 0.000100
  2483. 2019-03-12 17:37:35,632 : epoch 271 , step 80 loss0= 0.0000 loss1= 1.8858 loss2=-3434.5005 lr= 0.000100
  2484. 2019-03-12 17:38:10,065 : epoch 271 , step 120 loss0= 0.0000 loss1= 2.5289 loss2=-3681.3113 lr= 0.000100
  2485. 2019-03-12 17:38:45,010 : epoch 271 , step 160 loss0= 0.0000 loss1= 2.8293 loss2=-3454.2349 lr= 0.000100
  2486. 2019-03-12 17:39:16,391 : epoch 272 , step 0 loss0= 0.0000 loss1= 1.9363 loss2=-3335.6157 lr= 0.000100
  2487. 2019-03-12 17:39:51,203 : epoch 272 , step 40 loss0= 0.0000 loss1= 2.6562 loss2=-3607.5801 lr= 0.000100
  2488. 2019-03-12 17:40:26,452 : epoch 272 , step 80 loss0= 0.0000 loss1= 2.6820 loss2=-3586.8965 lr= 0.000100
  2489. 2019-03-12 17:41:01,087 : epoch 272 , step 120 loss0= 0.0000 loss1= 1.7943 loss2=-3562.9050 lr= 0.000100
  2490. 2019-03-12 17:41:35,547 : epoch 272 , step 160 loss0= 0.0000 loss1= 2.4192 loss2=-3582.4509 lr= 0.000100
  2491. 2019-03-12 17:42:07,171 : epoch 273 , step 0 loss0= 0.0000 loss1= 2.7718 loss2=-3568.5688 lr= 0.000100
  2492. 2019-03-12 17:42:41,838 : epoch 273 , step 40 loss0= 0.0000 loss1= 2.4204 loss2=-3606.0508 lr= 0.000100
  2493. 2019-03-12 17:43:17,225 : epoch 273 , step 80 loss0= 0.0000 loss1= 2.9449 loss2=-3596.5840 lr= 0.000100
  2494. 2019-03-12 17:43:51,478 : epoch 273 , step 120 loss0= 0.0000 loss1= 2.3047 loss2=-3618.5903 lr= 0.000100
  2495. 2019-03-12 17:44:25,306 : epoch 273 , step 160 loss0= 0.0000 loss1= 3.4812 loss2=-3914.0044 lr= 0.000100
  2496. 2019-03-12 17:44:56,132 : epoch 274 , step 0 loss0= 0.0000 loss1= 2.6650 loss2=-3658.0071 lr= 0.000100
  2497. 2019-03-12 17:45:30,643 : epoch 274 , step 40 loss0= 0.0000 loss1= 2.0072 loss2=-3449.6189 lr= 0.000100
  2498. 2019-03-12 17:46:04,908 : epoch 274 , step 80 loss0= 0.0000 loss1= 2.2788 loss2=-3224.8469 lr= 0.000100
  2499. 2019-03-12 17:46:39,147 : epoch 274 , step 120 loss0= 0.0000 loss1= 2.9365 loss2=-3450.4021 lr= 0.000100
  2500. 2019-03-12 17:47:12,974 : epoch 274 , step 160 loss0= 0.0000 loss1= 2.2702 loss2=-3689.3325 lr= 0.000100
  2501. 2019-03-12 17:47:43,736 : epoch 275 , step 0 loss0= 0.0000 loss1= 2.7542 loss2=-3576.6431 lr= 0.000100
  2502. 2019-03-12 17:48:18,400 : epoch 275 , step 40 loss0= 0.0000 loss1= 3.1138 loss2=-3996.0469 lr= 0.000100
  2503. 2019-03-12 17:48:52,802 : epoch 275 , step 80 loss0= 0.0000 loss1= 1.9105 loss2=-3844.9160 lr= 0.000100
  2504. 2019-03-12 17:49:27,653 : epoch 275 , step 120 loss0= 0.0000 loss1= 2.3079 loss2=-3317.6067 lr= 0.000100
  2505. 2019-03-12 17:50:02,265 : epoch 275 , step 160 loss0= 0.0000 loss1= 2.7591 loss2=-3422.7466 lr= 0.000100
  2506. 2019-03-12 17:50:33,453 : epoch 276 , step 0 loss0= 0.0000 loss1= 2.4920 loss2=-3717.5618 lr= 0.000100
  2507. 2019-03-12 17:51:07,673 : epoch 276 , step 40 loss0= 0.0000 loss1= 2.3597 loss2=-3785.8391 lr= 0.000100
  2508. 2019-03-12 17:51:41,754 : epoch 276 , step 80 loss0= 0.0000 loss1= 3.0397 loss2=-3487.0039 lr= 0.000100
  2509. 2019-03-12 17:52:16,083 : epoch 276 , step 120 loss0= 0.0000 loss1= 2.4335 loss2=-3296.5750 lr= 0.000100
  2510. 2019-03-12 17:52:50,220 : epoch 276 , step 160 loss0= 0.0000 loss1= 2.9097 loss2=-3633.9478 lr= 0.000100
  2511. 2019-03-12 17:53:21,048 : epoch 277 , step 0 loss0= 0.0000 loss1= 3.1835 loss2=-3981.3054 lr= 0.000100
  2512. 2019-03-12 17:53:55,803 : epoch 277 , step 40 loss0= 0.0000 loss1= 2.5471 loss2=-3647.1802 lr= 0.000100
  2513. 2019-03-12 17:54:30,093 : epoch 277 , step 80 loss0= 0.0000 loss1= 2.1588 loss2=-2948.4626 lr= 0.000100
  2514. 2019-03-12 17:55:04,330 : epoch 277 , step 120 loss0= 0.0000 loss1= 3.2509 loss2=-3691.1921 lr= 0.000100
  2515. 2019-03-12 17:55:38,433 : epoch 277 , step 160 loss0= 0.0000 loss1= 3.0310 loss2=-3458.0437 lr= 0.000100
  2516. 2019-03-12 17:56:09,122 : epoch 278 , step 0 loss0= 0.0000 loss1= 1.9763 loss2=-3529.5376 lr= 0.000100
  2517. 2019-03-12 17:56:43,024 : epoch 278 , step 40 loss0= 0.0000 loss1= 2.6250 loss2=-3559.4031 lr= 0.000100
  2518. 2019-03-12 17:57:17,367 : epoch 278 , step 80 loss0= 0.0000 loss1= 3.6249 loss2=-3722.4324 lr= 0.000100
  2519. 2019-03-12 17:57:52,398 : epoch 278 , step 120 loss0= 0.0000 loss1= 2.5510 loss2=-3501.2251 lr= 0.000100
  2520. 2019-03-12 17:58:26,806 : epoch 278 , step 160 loss0= 0.0000 loss1= 2.0496 loss2=-3234.1873 lr= 0.000100
  2521. 2019-03-12 17:58:57,833 : epoch 279 , step 0 loss0= 0.0000 loss1= 2.7960 loss2=-3830.1755 lr= 0.000100
  2522. 2019-03-12 17:59:32,246 : epoch 279 , step 40 loss0= 0.0000 loss1= 2.3995 loss2=-3463.5884 lr= 0.000100
  2523. 2019-03-12 18:00:06,660 : epoch 279 , step 80 loss0= 0.0000 loss1= 2.1825 loss2=-3520.4946 lr= 0.000100
  2524. 2019-03-12 18:00:41,093 : epoch 279 , step 120 loss0= 0.0000 loss1= 2.8689 loss2=-3759.4536 lr= 0.000100
  2525. 2019-03-12 18:01:14,985 : epoch 279 , step 160 loss0= 0.0000 loss1= 2.8490 loss2=-3729.8274 lr= 0.000100
  2526. 2019-03-12 18:01:46,016 : epoch 280 , step 0 loss0= 0.0000 loss1= 2.5231 loss2=-3710.2446 lr= 0.000100
  2527. 2019-03-12 18:02:20,423 : epoch 280 , step 40 loss0= 0.0000 loss1= 2.5433 loss2=-3610.1978 lr= 0.000100
  2528. 2019-03-12 18:02:54,132 : epoch 280 , step 80 loss0= 0.0000 loss1= 3.1085 loss2=-3771.8391 lr= 0.000100
  2529. 2019-03-12 18:03:27,978 : epoch 280 , step 120 loss0= 0.0000 loss1= 2.9280 loss2=-3534.5005 lr= 0.000100
  2530. 2019-03-12 18:04:02,033 : epoch 280 , step 160 loss0= 0.0000 loss1= 2.0198 loss2=-3426.0579 lr= 0.000100
  2531. 2019-03-12 18:04:31,909 : ## train ##
  2532. 2019-03-12 18:08:57,181 : >> epoch 280 : average_loss= 0.7533, average_accuracy=82.784%, mean_weight_std= 0.3271, mean_normalized_std= 0.4952 mean_dist=2.75e+03
  2533. 2019-03-12 18:08:57,189 : >> epoch 280 : median_loss= 0.7497, median_accuracy=83.450%, median_weight_std= 0.2024, median_normalized_std= 0.0979 median_dist=2.739e+03
  2534. 2019-03-12 18:08:57,190 : >> epoch 280 : ensemble accuracy: 85.40%
  2535. 2019-03-12 18:08:57,190 : >> epoch 280 : loss histogram: (0.00,0.32, 18),(0.32,0.65, 0),(0.65,0.97, 4),(0.97,1.29, 0),(1.29,1.62, 18),
  2536. 2019-03-12 18:08:57,190 : >> epoch 280 : accuracy histogram: (0.00,33.96, 18),(33.96,67.93, 0),(67.93,101.89, 4),(101.89,135.86, 0),(135.86,169.82, 18),
  2537. 2019-03-12 18:08:57,196 : >> epoch 280 : weight std histogram: (0.0000,2.2617, 462642),(2.2617,4.5235, 3679),(4.5235,6.7852, 492),(6.7852,9.0469, 76),(9.0469,11.3086, 17),
  2538. 2019-03-12 18:08:57,201 : >> epoch 280 : weight normalized std histogram: (0.0000,1655.1416, 466899),(1655.1416,3310.2832, 3),(3310.2832,4965.4248, 0),(4965.4248,6620.5664, 2),(6620.5664,8275.7080, 2),
  2539. 2019-03-12 18:08:57,201 : >> epoch 280 : dist histogram: (2494.2964,2635.7198, 8),(2635.7198,2777.1433, 4),(2777.1433,2918.5667, 4),(2918.5667,3059.9901, 2),(3059.9901,3201.4136, 2),
  2540. 2019-03-12 18:08:57,201 : ## eval ##
  2541. 2019-03-12 18:13:24,022 : >> epoch 280 : average_loss= 0.6496, average_accuracy=86.088%, mean_weight_std= 0.3116, mean_normalized_std= 0.4740 mean_dist=2.751e+03
  2542. 2019-03-12 18:13:24,030 : >> epoch 280 : median_loss= 0.6520, median_accuracy=85.815%, median_weight_std= 0.1892, median_normalized_std= 0.0924 median_dist=2.756e+03
  2543. 2019-03-12 18:13:24,030 : >> epoch 280 : ensemble accuracy: 88.44%
  2544. 2019-03-12 18:13:24,030 : >> epoch 280 : loss histogram: (0.00,0.28, 18),(0.28,0.56, 0),(0.56,0.84, 4),(0.84,1.13, 0),(1.13,1.41, 18),
  2545. 2019-03-12 18:13:24,031 : >> epoch 280 : accuracy histogram: (0.00,35.11, 18),(35.11,70.22, 0),(70.22,105.32, 4),(105.32,140.43, 0),(140.43,175.54, 18),
  2546. 2019-03-12 18:13:24,037 : >> epoch 280 : weight std histogram: (0.0000,2.3627, 463066),(2.3627,4.7253, 3290),(4.7253,7.0880, 469),(7.0880,9.4506, 64),(9.4506,11.8133, 17),
  2547. 2019-03-12 18:13:24,042 : >> epoch 280 : weight normalized std histogram: (0.0000,4369.8180, 466904),(4369.8180,8739.6359, 1),(8739.6359,13109.4539, 0),(13109.4539,17479.2719, 0),(17479.2719,21849.0898, 1),
  2548. 2019-03-12 18:13:24,042 : >> epoch 280 : dist histogram: (2524.6714,2683.5041, 7),(2683.5041,2842.3368, 10),(2842.3368,3001.1695, 1),(3001.1695,3160.0022, 1),(3160.0022,3318.8350, 1),
  2549. 2019-03-12 18:13:24,098 : >> epoch 280 : dist pair-wise
  2550. 2019-03-12 18:13:24,103 :
  2551. [[ 0. 316.258 1053.3 369.195 323.274 229.681 377.324
  2552. 448.13 623.069 222.528 226.879 244.841 265.62 338.247
  2553. 391.556 435.71 413.001 297.896 450.424 425.756]
  2554. [ 316.258 0. 1014.299 417.636 431.287 396.105 451.05
  2555. 511.909 500.79 341.839 314.547 397.314 356.368 417.177
  2556. 427.349 531.007 488.263 340.413 486.26 270.908]
  2557. [ 1053.3 1014.299 0. 805.83 858.51 1099.204 902.572
  2558. 822.11 974.67 1036.587 1081.893 976.212 1112.611 873.123
  2559. 947.032 791.107 937.019 1075.75 893.284 961.371]
  2560. [ 369.195 417.636 805.83 0. 252.883 453.403 401.098
  2561. 380.856 656.711 377.791 424.884 342.006 456.514 344.671
  2562. 479.63 309.6 412.284 481.888 346.394 499.361]
  2563. [ 323.274 431.287 858.51 252.883 0. 392.594 291.843
  2564. 297.114 608.675 376.353 398.183 220.77 461.448 275.549
  2565. 461.827 291.625 451.397 489.47 372.475 519.079]
  2566. [ 229.681 396.105 1099.204 453.403 392.594 0. 408.512
  2567. 532.608 695.522 286.676 267.573 293.424 253.298 419.158
  2568. 454.885 507.317 413.574 340.964 475.476 509.173]
  2569. [ 377.324 451.05 902.572 401.098 291.843 408.512 0.
  2570. 265.008 521.033 439.271 448.506 297.634 516.005 252.392
  2571. 454.492 471.716 485.365 524.415 494.788 506.035]
  2572. [ 448.13 511.909 822.11 380.856 297.114 532.608 265.008
  2573. 0. 524.577 502.143 544.697 351.009 601.373 276.263
  2574. 489.829 436.749 521.41 593.697 566.113 537.755]
  2575. [ 623.069 500.79 974.67 656.711 608.675 695.522 521.033
  2576. 524.577 0. 687.727 693.259 598.625 711.478 592.716
  2577. 606.193 718.24 708.555 694.835 798.514 436.116]
  2578. [ 222.528 341.839 1036.587 377.791 376.353 286.676 439.271
  2579. 502.143 687.727 0. 216.455 305.254 217.867 347.722
  2580. 341.886 451.651 393.023 227.323 419.681 425.625]
  2581. [ 226.879 314.547 1081.893 424.884 398.183 267.573 448.506
  2582. 544.697 693.259 216.455 0. 332.137 285.919 388.7
  2583. 399.37 476.54 470.756 265.863 403.351 445.041]
  2584. [ 244.841 397.314 976.212 342.006 220.77 293.424 297.634
  2585. 351.009 598.625 305.254 332.137 0. 369.04 309.071
  2586. 428.339 396.494 428.303 427.72 434.568 492.106]
  2587. [ 265.62 356.368 1112.611 456.514 461.448 253.298 516.005
  2588. 601.373 711.478 217.867 285.919 369.04 0. 470.282
  2589. 421.066 531.9 409.22 263.513 482.787 449.93 ]
  2590. [ 338.247 417.177 873.123 344.671 275.549 419.158 252.392
  2591. 276.263 592.716 347.722 388.7 309.071 470.282 0.
  2592. 331.223 413.844 455.786 420.327 450.985 456.362]
  2593. [ 391.556 427.349 947.032 479.63 461.827 454.885 454.492
  2594. 489.829 606.193 341.886 399.37 428.339 421.066 331.223
  2595. 0. 526.556 477.636 307.869 557.396 368.995]
  2596. [ 435.71 531.007 791.107 309.6 291.625 507.317 471.716
  2597. 436.749 718.24 451.651 476.54 396.494 531.9 413.844
  2598. 526.556 0. 561.911 531.368 364.958 587.537]
  2599. [ 413.001 488.263 937.019 412.284 451.397 413.574 485.365
  2600. 521.41 708.555 393.023 470.756 428.303 409.22 455.786
  2601. 477.636 561.911 0. 459.376 563.228 539.263]
  2602. [ 297.896 340.413 1075.75 481.888 489.47 340.964 524.415
  2603. 593.697 694.835 227.323 265.863 427.72 263.513 420.327
  2604. 307.869 531.368 459.376 0. 502.349 378.544]
  2605. [ 450.424 486.26 893.284 346.394 372.475 475.476 494.788
  2606. 566.113 798.514 419.681 403.351 434.568 482.787 450.985
  2607. 557.396 364.958 563.228 502.349 0. 592.087]
  2608. [ 425.756 270.908 961.371 499.361 519.079 509.173 506.035
  2609. 537.755 436.116 425.625 445.041 492.106 449.93 456.362
  2610. 368.995 587.537 539.263 378.544 592.087 0. ]]
  2611. 2019-03-12 18:13:25,517 : epoch 281 , step 0 loss0= 0.0000 loss1= 2.8669 loss2=-3905.8220 lr= 0.000100
  2612. 2019-03-12 18:13:59,803 : epoch 281 , step 40 loss0= 0.0000 loss1= 3.0259 loss2=-3752.6169 lr= 0.000100
  2613. 2019-03-12 18:14:34,616 : epoch 281 , step 80 loss0= 0.0000 loss1= 2.3300 loss2=-3705.0156 lr= 0.000100
  2614. 2019-03-12 18:15:09,059 : epoch 281 , step 120 loss0= 0.0000 loss1= 3.6648 loss2=-4009.0449 lr= 0.000100
  2615. 2019-03-12 18:15:43,889 : epoch 281 , step 160 loss0= 0.0000 loss1= 2.6441 loss2=-3693.2468 lr= 0.000100
  2616. 2019-03-12 18:16:15,750 : epoch 282 , step 0 loss0= 0.0000 loss1= 2.1473 loss2=-3549.3357 lr= 0.000100
  2617. 2019-03-12 18:16:50,347 : epoch 282 , step 40 loss0= 0.0000 loss1= 3.2434 loss2=-3770.7083 lr= 0.000100
  2618. 2019-03-12 18:17:25,111 : epoch 282 , step 80 loss0= 0.0000 loss1= 3.0872 loss2=-3936.2117 lr= 0.000100
  2619. 2019-03-12 18:18:00,190 : epoch 282 , step 120 loss0= 0.0000 loss1= 3.4288 loss2=-3579.9817 lr= 0.000100
  2620. 2019-03-12 18:18:34,747 : epoch 282 , step 160 loss0= 0.0000 loss1= 3.1774 loss2=-3713.0864 lr= 0.000100
  2621. 2019-03-12 18:19:05,939 : epoch 283 , step 0 loss0= 0.0000 loss1= 1.9374 loss2=-3418.0403 lr= 0.000100
  2622. 2019-03-12 18:19:41,168 : epoch 283 , step 40 loss0= 0.0000 loss1= 2.3463 loss2=-3731.3313 lr= 0.000100
  2623. 2019-03-12 18:20:16,539 : epoch 283 , step 80 loss0= 0.0000 loss1= 2.7148 loss2=-3701.5071 lr= 0.000100
  2624. 2019-03-12 18:20:51,619 : epoch 283 , step 120 loss0= 0.0000 loss1= 2.0080 loss2=-3609.4727 lr= 0.000100
  2625. 2019-03-12 18:21:26,739 : epoch 283 , step 160 loss0= 0.0000 loss1= 1.7542 loss2=-3175.4470 lr= 0.000100
  2626. 2019-03-12 18:21:58,618 : epoch 284 , step 0 loss0= 0.0000 loss1= 1.7157 loss2=-3250.8044 lr= 0.000100
  2627. 2019-03-12 18:22:33,791 : epoch 284 , step 40 loss0= 0.0000 loss1= 2.4023 loss2=-3552.0632 lr= 0.000100
  2628. 2019-03-12 18:23:08,744 : epoch 284 , step 80 loss0= 0.0000 loss1= 3.1463 loss2=-3713.6360 lr= 0.000100
  2629. 2019-03-12 18:23:44,007 : epoch 284 , step 120 loss0= 0.0000 loss1= 2.5663 loss2=-3659.2656 lr= 0.000100
  2630. 2019-03-12 18:24:18,592 : epoch 284 , step 160 loss0= 0.0000 loss1= 1.9278 loss2=-3268.8474 lr= 0.000100
  2631. 2019-03-12 18:24:49,742 : epoch 285 , step 0 loss0= 0.0000 loss1= 2.3761 loss2=-3670.6152 lr= 0.000100
  2632. 2019-03-12 18:25:24,045 : epoch 285 , step 40 loss0= 0.0000 loss1= 1.9543 loss2=-3550.2488 lr= 0.000100
  2633. 2019-03-12 18:25:59,383 : epoch 285 , step 80 loss0= 0.0000 loss1= 3.1461 loss2=-3896.9109 lr= 0.000100
  2634. 2019-03-12 18:26:34,787 : epoch 285 , step 120 loss0= 0.0000 loss1= 1.7374 loss2=-3295.9185 lr= 0.000100
  2635. 2019-03-12 18:27:09,560 : epoch 285 , step 160 loss0= 0.0000 loss1= 2.8901 loss2=-3623.8435 lr= 0.000100
  2636. 2019-03-12 18:27:41,091 : epoch 286 , step 0 loss0= 0.0000 loss1= 2.6312 loss2=-3704.3398 lr= 0.000100
  2637. 2019-03-12 18:28:15,793 : epoch 286 , step 40 loss0= 0.0000 loss1= 1.8600 loss2=-3749.0515 lr= 0.000100
  2638. 2019-03-12 18:28:50,277 : epoch 286 , step 80 loss0= 0.0000 loss1= 2.2753 loss2=-3583.5222 lr= 0.000100
  2639. 2019-03-12 18:29:24,544 : epoch 286 , step 120 loss0= 0.0000 loss1= 2.0003 loss2=-3610.0559 lr= 0.000100
  2640. 2019-03-12 18:29:59,236 : epoch 286 , step 160 loss0= 0.0000 loss1= 2.9339 loss2=-3219.8020 lr= 0.000100
  2641. 2019-03-12 18:30:30,060 : epoch 287 , step 0 loss0= 0.0000 loss1= 2.3457 loss2=-3757.9609 lr= 0.000100
  2642. 2019-03-12 18:31:04,581 : epoch 287 , step 40 loss0= 0.0000 loss1= 4.4862 loss2=-3878.2800 lr= 0.000100
  2643. 2019-03-12 18:31:38,966 : epoch 287 , step 80 loss0= 0.0000 loss1= 2.5170 loss2=-3762.1860 lr= 0.000100
  2644. 2019-03-12 18:32:12,902 : epoch 287 , step 120 loss0= 0.0000 loss1= 1.8725 loss2=-3370.8201 lr= 0.000100
  2645. 2019-03-12 18:32:47,248 : epoch 287 , step 160 loss0= 0.0000 loss1= 1.9180 loss2=-3546.3240 lr= 0.000100
  2646. 2019-03-12 18:33:18,462 : epoch 288 , step 0 loss0= 0.0000 loss1= 3.0275 loss2=-4002.5203 lr= 0.000100
  2647. 2019-03-12 18:33:52,709 : epoch 288 , step 40 loss0= 0.0000 loss1= 2.5582 loss2=-3556.1638 lr= 0.000100
  2648. 2019-03-12 18:34:26,756 : epoch 288 , step 80 loss0= 0.0000 loss1= 2.1537 loss2=-3772.6582 lr= 0.000100
  2649. 2019-03-12 18:35:00,660 : epoch 288 , step 120 loss0= 0.0000 loss1= 1.8629 loss2=-3340.5085 lr= 0.000100
  2650. 2019-03-12 18:35:34,552 : epoch 288 , step 160 loss0= 0.0000 loss1= 2.3087 loss2=-3412.1235 lr= 0.000100
  2651. 2019-03-12 18:36:05,374 : epoch 289 , step 0 loss0= 0.0000 loss1= 2.5917 loss2=-3309.2908 lr= 0.000100
  2652. 2019-03-12 18:36:39,557 : epoch 289 , step 40 loss0= 0.0000 loss1= 2.6129 loss2=-3746.4851 lr= 0.000100
  2653. 2019-03-12 18:37:13,876 : epoch 289 , step 80 loss0= 0.0000 loss1= 2.5143 loss2=-3501.4993 lr= 0.000100
  2654. 2019-03-12 18:37:48,474 : epoch 289 , step 120 loss0= 0.0000 loss1= 3.6227 loss2=-3819.1387 lr= 0.000100
  2655. 2019-03-12 18:38:22,678 : epoch 289 , step 160 loss0= 0.0000 loss1= 2.4909 loss2=-3673.5720 lr= 0.000100
  2656. 2019-03-12 18:38:53,682 : epoch 290 , step 0 loss0= 0.0000 loss1= 2.3334 loss2=-3625.1411 lr= 0.000100
  2657. 2019-03-12 18:39:27,960 : epoch 290 , step 40 loss0= 0.0000 loss1= 2.4471 loss2=-3436.4199 lr= 0.000100
  2658. 2019-03-12 18:40:02,069 : epoch 290 , step 80 loss0= 0.0000 loss1= 3.8164 loss2=-3842.4050 lr= 0.000100
  2659. 2019-03-12 18:40:36,225 : epoch 290 , step 120 loss0= 0.0000 loss1= 2.5967 loss2=-3610.7844 lr= 0.000100
  2660. 2019-03-12 18:41:10,779 : epoch 290 , step 160 loss0= 0.0000 loss1= 3.8768 loss2=-3785.6060 lr= 0.000100
  2661. 2019-03-12 18:41:41,888 : epoch 291 , step 0 loss0= 0.0000 loss1= 2.2879 loss2=-3622.3835 lr= 0.000100
  2662. 2019-03-12 18:42:16,480 : epoch 291 , step 40 loss0= 0.0000 loss1= 3.3011 loss2=-3945.0071 lr= 0.000100
  2663. 2019-03-12 18:42:50,668 : epoch 291 , step 80 loss0= 0.0000 loss1= 3.4413 loss2=-3692.5134 lr= 0.000100
  2664. 2019-03-12 18:43:25,452 : epoch 291 , step 120 loss0= 0.0000 loss1= 2.3881 loss2=-3656.1111 lr= 0.000100
  2665. 2019-03-12 18:44:00,095 : epoch 291 , step 160 loss0= 0.0000 loss1= 3.1437 loss2=-3752.6521 lr= 0.000100
  2666. 2019-03-12 18:44:31,423 : epoch 292 , step 0 loss0= 0.0000 loss1= 1.6788 loss2=-3565.1641 lr= 0.000100
  2667. 2019-03-12 18:45:05,664 : epoch 292 , step 40 loss0= 0.0000 loss1= 1.7815 loss2=-3236.7175 lr= 0.000100
  2668. 2019-03-12 18:45:40,040 : epoch 292 , step 80 loss0= 0.0000 loss1= 2.1910 loss2=-3557.8445 lr= 0.000100
  2669. 2019-03-12 18:46:14,273 : epoch 292 , step 120 loss0= 0.0000 loss1= 1.6514 loss2=-3180.2224 lr= 0.000100
  2670. 2019-03-12 18:46:48,482 : epoch 292 , step 160 loss0= 0.0000 loss1= 3.1516 loss2=-3591.9080 lr= 0.000100
  2671. 2019-03-12 18:47:19,603 : epoch 293 , step 0 loss0= 0.0000 loss1= 4.1658 loss2=-3847.3220 lr= 0.000100
  2672. 2019-03-12 18:47:53,545 : epoch 293 , step 40 loss0= 0.0000 loss1= 2.1812 loss2=-3554.6704 lr= 0.000100
  2673. 2019-03-12 18:48:27,787 : epoch 293 , step 80 loss0= 0.0000 loss1= 2.7226 loss2=-3417.5391 lr= 0.000100
  2674. 2019-03-12 18:49:02,219 : epoch 293 , step 120 loss0= 0.0000 loss1= 3.2037 loss2=-3898.5330 lr= 0.000100
  2675. 2019-03-12 18:49:36,576 : epoch 293 , step 160 loss0= 0.0000 loss1= 3.6803 loss2=-3907.5371 lr= 0.000100
  2676. 2019-03-12 18:50:08,081 : epoch 294 , step 0 loss0= 0.0000 loss1= 2.4107 loss2=-3750.9138 lr= 0.000100
  2677. 2019-03-12 18:50:42,434 : epoch 294 , step 40 loss0= 0.0000 loss1= 2.4811 loss2=-4031.3340 lr= 0.000100
  2678. 2019-03-12 18:51:16,290 : epoch 294 , step 80 loss0= 0.0000 loss1= 3.5033 loss2=-3597.2727 lr= 0.000100
  2679. 2019-03-12 18:51:50,067 : epoch 294 , step 120 loss0= 0.0000 loss1= 2.5730 loss2=-3784.4880 lr= 0.000100
  2680. 2019-03-12 18:52:23,892 : epoch 294 , step 160 loss0= 0.0000 loss1= 5.4150 loss2=-3733.6665 lr= 0.000100
  2681. 2019-03-12 18:52:55,058 : epoch 295 , step 0 loss0= 0.0000 loss1= 1.7315 loss2=-3566.0391 lr= 0.000100
  2682. 2019-03-12 18:53:29,556 : epoch 295 , step 40 loss0= 0.0000 loss1= 2.6448 loss2=-3559.3579 lr= 0.000100
  2683. 2019-03-12 18:54:04,079 : epoch 295 , step 80 loss0= 0.0000 loss1= 2.3905 loss2=-3581.2673 lr= 0.000100
  2684. 2019-03-12 18:54:37,788 : epoch 295 , step 120 loss0= 0.0000 loss1= 2.4616 loss2=-3775.0442 lr= 0.000100
  2685. 2019-03-12 18:55:12,494 : epoch 295 , step 160 loss0= 0.0000 loss1= 3.3246 loss2=-3965.6619 lr= 0.000100
  2686. 2019-03-12 18:55:43,753 : epoch 296 , step 0 loss0= 0.0000 loss1= 2.6169 loss2=-3588.8035 lr= 0.000100
  2687. 2019-03-12 18:56:18,015 : epoch 296 , step 40 loss0= 0.0000 loss1= 3.2159 loss2=-3731.3787 lr= 0.000100
  2688. 2019-03-12 18:56:52,533 : epoch 296 , step 80 loss0= 0.0000 loss1= 1.8695 loss2=-3396.3259 lr= 0.000100
  2689. 2019-03-12 18:57:26,771 : epoch 296 , step 120 loss0= 0.0000 loss1= 2.8800 loss2=-3680.4641 lr= 0.000100
  2690. 2019-03-12 18:58:01,001 : epoch 296 , step 160 loss0= 0.0000 loss1= 2.7373 loss2=-3830.9902 lr= 0.000100
  2691. 2019-03-12 18:58:31,924 : epoch 297 , step 0 loss0= 0.0000 loss1= 2.2852 loss2=-3589.6641 lr= 0.000100
  2692. 2019-03-12 18:59:06,138 : epoch 297 , step 40 loss0= 0.0000 loss1= 2.5682 loss2=-3399.6951 lr= 0.000100
  2693. 2019-03-12 18:59:40,031 : epoch 297 , step 80 loss0= 0.0000 loss1= 2.4999 loss2=-3379.8247 lr= 0.000100
  2694. 2019-03-12 19:00:13,820 : epoch 297 , step 120 loss0= 0.0000 loss1= 2.7094 loss2=-3759.9001 lr= 0.000100
  2695. 2019-03-12 19:00:49,029 : epoch 297 , step 160 loss0= 0.0000 loss1= 2.2526 loss2=-3590.2090 lr= 0.000100
  2696. 2019-03-12 19:01:20,060 : epoch 298 , step 0 loss0= 0.0000 loss1= 2.5522 loss2=-3550.4531 lr= 0.000100
  2697. 2019-03-12 19:01:54,218 : epoch 298 , step 40 loss0= 0.0000 loss1= 2.5699 loss2=-3927.3210 lr= 0.000100
  2698. 2019-03-12 19:02:28,528 : epoch 298 , step 80 loss0= 0.0000 loss1= 2.6971 loss2=-3683.0461 lr= 0.000100
  2699. 2019-03-12 19:03:02,648 : epoch 298 , step 120 loss0= 0.0000 loss1= 1.5540 loss2=-3204.8008 lr= 0.000100
  2700. 2019-03-12 19:03:36,503 : epoch 298 , step 160 loss0= 0.0000 loss1= 2.3136 loss2=-3509.3208 lr= 0.000100
  2701. 2019-03-12 19:04:07,316 : epoch 299 , step 0 loss0= 0.0000 loss1= 2.4533 loss2=-3847.2871 lr= 0.000100
  2702. 2019-03-12 19:04:41,980 : epoch 299 , step 40 loss0= 0.0000 loss1= 2.8459 loss2=-3876.7234 lr= 0.000100
  2703. 2019-03-12 19:05:15,923 : epoch 299 , step 80 loss0= 0.0000 loss1= 1.7639 loss2=-3635.8191 lr= 0.000100
  2704. 2019-03-12 19:05:50,184 : epoch 299 , step 120 loss0= 0.0000 loss1= 3.3098 loss2=-3846.7563 lr= 0.000100
  2705. 2019-03-12 19:06:24,050 : epoch 299 , step 160 loss0= 0.0000 loss1= 2.3498 loss2=-3820.3118 lr= 0.000100
  2706. 2019-03-12 19:06:55,190 : epoch 300 , step 0 loss0= 0.0000 loss1= 2.1416 loss2=-3641.6716 lr= 0.000100
  2707. 2019-03-12 19:07:29,431 : epoch 300 , step 40 loss0= 0.0000 loss1= 2.9906 loss2=-3799.9871 lr= 0.000100
  2708. 2019-03-12 19:08:03,484 : epoch 300 , step 80 loss0= 0.0000 loss1= 1.7678 loss2=-3356.1179 lr= 0.000100
  2709. 2019-03-12 19:08:37,989 : epoch 300 , step 120 loss0= 0.0000 loss1= 2.4732 loss2=-3577.7493 lr= 0.000100
  2710. 2019-03-12 19:09:12,444 : epoch 300 , step 160 loss0= 0.0000 loss1= 3.5970 loss2=-3823.9438 lr= 0.000100
  2711. 2019-03-12 19:09:42,118 : ## train ##
  2712. 2019-03-12 19:14:04,605 : >> epoch 300 : average_loss= 0.7611, average_accuracy=83.035%, mean_weight_std= 0.2576, mean_normalized_std= 0.5807 mean_dist=2.805e+03
  2713. 2019-03-12 19:14:04,615 : >> epoch 300 : median_loss= 0.7641, median_accuracy=84.070%, median_weight_std= 0.1455, median_normalized_std= 0.0707 median_dist=2.774e+03
  2714. 2019-03-12 19:14:04,615 : >> epoch 300 : ensemble accuracy: 85.53%
  2715. 2019-03-12 19:14:04,616 : >> epoch 300 : loss histogram: (0.00,0.32, 18),(0.32,0.63, 0),(0.63,0.95, 4),(0.95,1.27, 0),(1.27,1.58, 18),
  2716. 2019-03-12 19:14:04,616 : >> epoch 300 : accuracy histogram: (0.00,33.80, 18),(33.80,67.60, 0),(67.60,101.39, 4),(101.39,135.19, 0),(135.19,168.99, 18),
  2717. 2019-03-12 19:14:04,621 : >> epoch 300 : weight std histogram: (0.0000,2.7235, 464425),(2.7235,5.4469, 2132),(5.4469,8.1704, 298),(8.1704,10.8938, 41),(10.8938,13.6173, 10),
  2718. 2019-03-12 19:14:04,626 : >> epoch 300 : weight normalized std histogram: (0.0000,11400.1047, 466904),(11400.1047,22800.2094, 1),(22800.2094,34200.3141, 0),(34200.3141,45600.4187, 0),(45600.4187,57000.5234, 1),
  2719. 2019-03-12 19:14:04,626 : >> epoch 300 : dist histogram: (2646.4397,2718.9073, 5),(2718.9073,2791.3749, 6),(2791.3749,2863.8424, 1),(2863.8424,2936.3100, 5),(2936.3100,3008.7776, 3),
  2720. 2019-03-12 19:14:04,626 : ## eval ##
  2721. 2019-03-12 19:18:26,108 : >> epoch 300 : average_loss= 0.6737, average_accuracy=86.265%, mean_weight_std= 0.3058, mean_normalized_std= 0.4288 mean_dist=2.802e+03
  2722. 2019-03-12 19:18:26,117 : >> epoch 300 : median_loss= 0.6571, median_accuracy=86.695%, median_weight_std= 0.1839, median_normalized_std= 0.0878 median_dist=2.789e+03
  2723. 2019-03-12 19:18:26,117 : >> epoch 300 : ensemble accuracy: 88.43%
  2724. 2019-03-12 19:18:26,118 : >> epoch 300 : loss histogram: (0.00,0.28, 18),(0.28,0.56, 0),(0.56,0.84, 4),(0.84,1.12, 0),(1.12,1.40, 18),
  2725. 2019-03-12 19:18:26,118 : >> epoch 300 : accuracy histogram: (0.00,35.14, 18),(35.14,70.28, 0),(70.28,105.42, 4),(105.42,140.56, 0),(140.56,175.70, 18),
  2726. 2019-03-12 19:18:26,124 : >> epoch 300 : weight std histogram: (0.0000,2.5785, 463914),(2.5785,5.1570, 2571),(5.1570,7.7354, 357),(7.7354,10.3139, 53),(10.3139,12.8924, 11),
  2727. 2019-03-12 19:18:26,128 : >> epoch 300 : weight normalized std histogram: (0.0000,2243.3941, 466904),(2243.3941,4486.7883, 1),(4486.7883,6730.1824, 0),(6730.1824,8973.5766, 0),(8973.5766,11216.9707, 1),
  2728. 2019-03-12 19:18:26,129 : >> epoch 300 : dist histogram: (2431.3003,2563.2488, 2),(2563.2488,2695.1974, 0),(2695.1974,2827.1459, 11),(2827.1459,2959.0944, 3),(2959.0944,3091.0430, 4),
  2729. 2019-03-12 19:18:26,186 : >> epoch 300 : dist pair-wise
  2730. 2019-03-12 19:18:26,191 :
  2731. [[ 0. 315.607 433.33 352.231 272.831 297.571 550.865 239.554
  2732. 562.846 222.997 431.777 477.172 490.777 403.173 391.71 521.297
  2733. 473.95 441.728 438.954 604.778]
  2734. [ 315.607 0. 336.295 401.667 292.21 457.084 607.398 289.114
  2735. 650.456 388.963 577.374 466.831 505.653 526.043 379.867 589.052
  2736. 549.56 296.067 548.75 652.948]
  2737. [ 433.33 336.295 0. 488.1 308.619 510.423 653.317 344.512
  2738. 759.9 538.105 597.011 586.182 417.009 644.903 316.027 521.274
  2739. 580.457 217.991 575.119 553.633]
  2740. [ 352.231 401.667 488.1 0. 399.587 431.663 483.863 332.143
  2741. 650.975 365.728 405.937 648.773 465.076 480.36 341.534 547.616
  2742. 412.74 520.522 387.44 500.662]
  2743. [ 272.831 292.21 308.619 399.587 0. 374.103 551.027 249.817
  2744. 668.417 362.149 490.634 515.165 382.831 476.846 301.99 466.738
  2745. 465.238 335.489 461.601 509.66 ]
  2746. [ 297.571 457.084 510.423 431.663 374.103 0. 580.637 316.058
  2747. 576.553 324.015 470.096 581.939 512.838 451.853 404.341 364.545
  2748. 493.953 541.515 487.196 603.249]
  2749. [ 550.865 607.398 653.317 483.863 551.027 580.637 0. 523.316
  2750. 799.368 521.236 467.253 793.966 570.685 344.989 597.761 619.83
  2751. 244.529 695.294 270.064 530.382]
  2752. [ 239.554 289.114 344.512 332.143 249.817 316.058 523.316 0.
  2753. 636.447 336.383 457.059 557.693 423.282 444.154 303.277 462.609
  2754. 421.625 392.627 412.788 523.655]
  2755. [ 562.846 650.456 759.9 650.975 668.417 576.553 799.368 636.447
  2756. 0. 586.94 522.217 441.15 824.911 684.504 740.522 754.618
  2757. 779.245 716.498 732.24 888.548]
  2758. [ 222.997 388.963 538.105 365.728 362.149 324.015 521.236 336.383
  2759. 586.94 0. 483.553 529.313 585.114 339.521 454.725 560.059
  2760. 463.647 550.72 458.306 677.682]
  2761. [ 431.777 577.374 597.011 405.937 490.634 470.096 467.253 457.059
  2762. 522.217 483.553 0. 606.467 517.316 455.826 544.215 558.912
  2763. 411.873 627.605 333.947 487.339]
  2764. [ 477.172 466.831 586.182 648.773 515.165 581.939 793.966 557.693
  2765. 441.15 529.313 606.467 0. 746.592 659.571 665.662 751.536
  2766. 767.055 500.441 735.94 872.478]
  2767. [ 490.777 505.653 417.009 465.076 382.831 512.838 570.685 423.282
  2768. 824.911 585.114 517.316 746.592 0. 621.799 374.437 422.396
  2769. 459.913 502.006 482.425 265.329]
  2770. [ 403.173 526.043 644.903 480.36 476.846 451.853 344.989 444.154
  2771. 684.504 339.521 455.826 659.571 621.799 0. 588.825 605.2
  2772. 316.27 660.382 325.849 651.704]
  2773. [ 391.71 379.867 316.027 341.534 301.99 404.341 597.761 303.277
  2774. 740.522 454.725 544.215 665.662 374.437 588.825 0. 415.081
  2775. 504.028 403.505 513.704 480.405]
  2776. [ 521.297 589.052 521.274 547.616 466.738 364.545 619.83 462.609
  2777. 754.618 560.059 558.912 751.536 422.396 605.2 415.081 0.
  2778. 514.602 604.955 558.667 494.714]
  2779. [ 473.95 549.56 580.457 412.74 465.238 493.953 244.529 421.625
  2780. 779.245 463.647 411.873 767.055 459.913 316.27 504.028 514.602
  2781. 0. 638.462 213.173 425.499]
  2782. [ 441.728 296.067 217.991 520.522 335.489 541.515 695.294 392.627
  2783. 716.498 550.72 627.605 500.441 502.006 660.382 403.505 604.955
  2784. 638.462 0. 630.116 645.296]
  2785. [ 438.954 548.75 575.119 387.44 461.601 487.196 270.064 412.788
  2786. 732.24 458.306 333.947 735.94 482.425 325.849 513.704 558.667
  2787. 213.173 630.116 0. 433.515]
  2788. [ 604.778 652.948 553.633 500.662 509.66 603.249 530.382 523.655
  2789. 888.548 677.682 487.339 872.478 265.329 651.704 480.405 494.714
  2790. 425.499 645.296 433.515 0. ]]
  2791. 2019-03-12 19:18:27,578 : epoch 301 , step 0 loss0= 0.0000 loss1= 2.5536 loss2=-3569.8689 lr= 0.000100
  2792. 2019-03-12 19:19:02,502 : epoch 301 , step 40 loss0= 0.0000 loss1= 2.6593 loss2=-3937.1770 lr= 0.000100
  2793. 2019-03-12 19:19:37,189 : epoch 301 , step 80 loss0= 0.0000 loss1= 2.0373 loss2=-3573.6606 lr= 0.000100
  2794. 2019-03-12 19:20:12,224 : epoch 301 , step 120 loss0= 0.0000 loss1= 3.4324 loss2=-3840.6040 lr= 0.000100
  2795. 2019-03-12 19:20:47,446 : epoch 301 , step 160 loss0= 0.0000 loss1= 2.5479 loss2=-3695.8809 lr= 0.000100
  2796. 2019-03-12 19:21:19,269 : epoch 302 , step 0 loss0= 0.0000 loss1= 2.1963 loss2=-3642.7356 lr= 0.000100
  2797. 2019-03-12 19:21:53,819 : epoch 302 , step 40 loss0= 0.0000 loss1= 2.2443 loss2=-3533.4822 lr= 0.000100
  2798. 2019-03-12 19:22:28,212 : epoch 302 , step 80 loss0= 0.0000 loss1= 2.2159 loss2=-3668.9829 lr= 0.000100
  2799. 2019-03-12 19:23:02,494 : epoch 302 , step 120 loss0= 0.0000 loss1= 2.9216 loss2=-3629.2043 lr= 0.000100
  2800. 2019-03-12 19:23:36,530 : epoch 302 , step 160 loss0= 0.0000 loss1= 2.7002 loss2=-3730.6824 lr= 0.000100
  2801. 2019-03-12 19:24:07,827 : epoch 303 , step 0 loss0= 0.0000 loss1= 2.5336 loss2=-3463.4714 lr= 0.000100
  2802. 2019-03-12 19:24:41,723 : epoch 303 , step 40 loss0= 0.0000 loss1= 2.1066 loss2=-3429.9570 lr= 0.000100
  2803. 2019-03-12 19:25:16,203 : epoch 303 , step 80 loss0= 0.0000 loss1= 2.4913 loss2=-3504.8730 lr= 0.000100
  2804. 2019-03-12 19:25:50,789 : epoch 303 , step 120 loss0= 0.0000 loss1= 3.5252 loss2=-3975.5703 lr= 0.000100
  2805. 2019-03-12 19:26:25,607 : epoch 303 , step 160 loss0= 0.0000 loss1= 2.6972 loss2=-4020.8430 lr= 0.000100
  2806. 2019-03-12 19:26:56,853 : epoch 304 , step 0 loss0= 0.0000 loss1= 1.9739 loss2=-3216.8416 lr= 0.000100
  2807. 2019-03-12 19:27:31,660 : epoch 304 , step 40 loss0= 0.0000 loss1= 2.0595 loss2=-3767.4265 lr= 0.000100
  2808. 2019-03-12 19:28:05,698 : epoch 304 , step 80 loss0= 0.0000 loss1= 1.5123 loss2=-3217.0667 lr= 0.000100
  2809. 2019-03-12 19:28:40,756 : epoch 304 , step 120 loss0= 0.0000 loss1= 2.2339 loss2=-3556.7000 lr= 0.000100
  2810. 2019-03-12 19:29:15,269 : epoch 304 , step 160 loss0= 0.0000 loss1= 2.8451 loss2=-3564.1543 lr= 0.000100
  2811. 2019-03-12 19:29:45,948 : epoch 305 , step 0 loss0= 0.0000 loss1= 2.7784 loss2=-3798.1404 lr= 0.000100
  2812. 2019-03-12 19:30:20,008 : epoch 305 , step 40 loss0= 0.0000 loss1= 2.5474 loss2=-3584.9771 lr= 0.000100
  2813. 2019-03-12 19:30:54,851 : epoch 305 , step 80 loss0= 0.0000 loss1= 3.1870 loss2=-4057.6130 lr= 0.000100
  2814. 2019-03-12 19:31:29,859 : epoch 305 , step 120 loss0= 0.0000 loss1= 2.8530 loss2=-3782.8013 lr= 0.000100
  2815. 2019-03-12 19:32:04,853 : epoch 305 , step 160 loss0= 0.0000 loss1= 3.2083 loss2=-3939.2739 lr= 0.000100
  2816. 2019-03-12 19:32:36,215 : epoch 306 , step 0 loss0= 0.0000 loss1= 2.9559 loss2=-3520.2466 lr= 0.000100
  2817. 2019-03-12 19:33:10,378 : epoch 306 , step 40 loss0= 0.0000 loss1= 1.6268 loss2=-3292.7642 lr= 0.000100
  2818. 2019-03-12 19:33:44,207 : epoch 306 , step 80 loss0= 0.0000 loss1= 2.3401 loss2=-3827.9265 lr= 0.000100
  2819. 2019-03-12 19:34:18,070 : epoch 306 , step 120 loss0= 0.0000 loss1= 2.1295 loss2=-3746.6021 lr= 0.000100
  2820. 2019-03-12 19:34:51,546 : epoch 306 , step 160 loss0= 0.0000 loss1= 2.2197 loss2=-3627.5005 lr= 0.000100
  2821. 2019-03-12 19:35:22,038 : epoch 307 , step 0 loss0= 0.0000 loss1= 2.9105 loss2=-3637.7263 lr= 0.000100
  2822. 2019-03-12 19:35:55,801 : epoch 307 , step 40 loss0= 0.0000 loss1= 2.6489 loss2=-3785.3560 lr= 0.000100
  2823. 2019-03-12 19:36:29,602 : epoch 307 , step 80 loss0= 0.0000 loss1= 2.3122 loss2=-3995.0857 lr= 0.000100
  2824. 2019-03-12 19:37:05,614 : epoch 307 , step 120 loss0= 0.0000 loss1= 1.7651 loss2=-3412.7812 lr= 0.000100
  2825. 2019-03-12 19:37:41,187 : epoch 307 , step 160 loss0= 0.0000 loss1= 2.6125 loss2=-3831.5017 lr= 0.000100
  2826. 2019-03-12 19:38:13,416 : epoch 308 , step 0 loss0= 0.0000 loss1= 2.1651 loss2=-3480.4949 lr= 0.000100
  2827. 2019-03-12 19:38:48,942 : epoch 308 , step 40 loss0= 0.0000 loss1= 2.0638 loss2=-3502.7188 lr= 0.000100
  2828. 2019-03-12 19:39:24,237 : epoch 308 , step 80 loss0= 0.0000 loss1= 2.5226 loss2=-3776.6543 lr= 0.000100
  2829. 2019-03-12 19:39:59,481 : epoch 308 , step 120 loss0= 0.0000 loss1= 2.6951 loss2=-3772.3123 lr= 0.000100
  2830. 2019-03-12 19:40:34,847 : epoch 308 , step 160 loss0= 0.0000 loss1= 2.7921 loss2=-3644.9607 lr= 0.000100
  2831. 2019-03-12 19:41:07,081 : epoch 309 , step 0 loss0= 0.0000 loss1= 2.9587 loss2=-3974.9785 lr= 0.000100
  2832. 2019-03-12 19:41:42,450 : epoch 309 , step 40 loss0= 0.0000 loss1= 3.3639 loss2=-3853.9622 lr= 0.000100
  2833. 2019-03-12 19:42:18,291 : epoch 309 , step 80 loss0= 0.0000 loss1= 3.2139 loss2=-3516.1394 lr= 0.000100
  2834. 2019-03-12 19:42:53,756 : epoch 309 , step 120 loss0= 0.0000 loss1= 2.6676 loss2=-3571.1873 lr= 0.000100
  2835. 2019-03-12 19:43:29,091 : epoch 309 , step 160 loss0= 0.0000 loss1= 1.4630 loss2=-3307.5251 lr= 0.000100
  2836. 2019-03-12 19:44:01,625 : epoch 310 , step 0 loss0= 0.0000 loss1= 2.4898 loss2=-3693.2986 lr= 0.000100
  2837. 2019-03-12 19:44:37,502 : epoch 310 , step 40 loss0= 0.0000 loss1= 2.6838 loss2=-3666.0989 lr= 0.000100
  2838. 2019-03-12 19:45:13,150 : epoch 310 , step 80 loss0= 0.0000 loss1= 3.3428 loss2=-3785.0376 lr= 0.000100
  2839. 2019-03-12 19:45:48,428 : epoch 310 , step 120 loss0= 0.0000 loss1= 2.7191 loss2=-3468.7283 lr= 0.000100
  2840. 2019-03-12 19:46:23,834 : epoch 310 , step 160 loss0= 0.0000 loss1= 2.2439 loss2=-3520.6179 lr= 0.000100
  2841. 2019-03-12 19:46:55,893 : epoch 311 , step 0 loss0= 0.0000 loss1= 2.4643 loss2=-3552.9302 lr= 0.000100
  2842. 2019-03-12 19:47:32,043 : epoch 311 , step 40 loss0= 0.0000 loss1= 2.4133 loss2=-3477.8665 lr= 0.000100
  2843. 2019-03-12 19:48:08,275 : epoch 311 , step 80 loss0= 0.0000 loss1= 2.1476 loss2=-3625.1060 lr= 0.000100
  2844. 2019-03-12 19:48:43,983 : epoch 311 , step 120 loss0= 0.0000 loss1= 3.0141 loss2=-3678.7781 lr= 0.000100
  2845. 2019-03-12 19:49:19,622 : epoch 311 , step 160 loss0= 0.0000 loss1= 2.2128 loss2=-3618.0071 lr= 0.000100
  2846. 2019-03-12 19:49:52,176 : epoch 312 , step 0 loss0= 0.0000 loss1= 2.6422 loss2=-3507.3892 lr= 0.000100
  2847. 2019-03-12 19:50:27,958 : epoch 312 , step 40 loss0= 0.0000 loss1= 2.5592 loss2=-3623.2747 lr= 0.000100
  2848. 2019-03-12 19:51:03,690 : epoch 312 , step 80 loss0= 0.0000 loss1= 2.0603 loss2=-3678.8914 lr= 0.000100
  2849. 2019-03-12 19:51:38,937 : epoch 312 , step 120 loss0= 0.0000 loss1= 3.2231 loss2=-3853.9907 lr= 0.000100
  2850. 2019-03-12 19:52:15,481 : epoch 312 , step 160 loss0= 0.0000 loss1= 2.4284 loss2=-3199.1489 lr= 0.000100
  2851. 2019-03-12 19:52:48,224 : epoch 313 , step 0 loss0= 0.0000 loss1= 2.7544 loss2=-3747.2434 lr= 0.000100
  2852. 2019-03-12 19:53:23,687 : epoch 313 , step 40 loss0= 0.0000 loss1= 2.0210 loss2=-3525.0728 lr= 0.000100
  2853. 2019-03-12 19:53:59,898 : epoch 313 , step 80 loss0= 0.0000 loss1= 2.2411 loss2=-3516.2798 lr= 0.000100
  2854. 2019-03-12 19:54:36,207 : epoch 313 , step 120 loss0= 0.0000 loss1= 2.4984 loss2=-3650.3884 lr= 0.000100
  2855. 2019-03-12 19:55:12,096 : epoch 313 , step 160 loss0= 0.0000 loss1= 3.5734 loss2=-3692.8652 lr= 0.000100
  2856. 2019-03-12 19:55:44,670 : epoch 314 , step 0 loss0= 0.0000 loss1= 2.5715 loss2=-3875.5164 lr= 0.000100
  2857. 2019-03-12 19:56:20,238 : epoch 314 , step 40 loss0= 0.0000 loss1= 2.7638 loss2=-3644.1211 lr= 0.000100
  2858. 2019-03-12 19:56:55,568 : epoch 314 , step 80 loss0= 0.0000 loss1= 2.3362 loss2=-3488.3430 lr= 0.000100
  2859. 2019-03-12 19:57:31,023 : epoch 314 , step 120 loss0= 0.0000 loss1= 2.7851 loss2=-3659.4238 lr= 0.000100
  2860. 2019-03-12 19:58:06,314 : epoch 314 , step 160 loss0= 0.0000 loss1= 2.5996 loss2=-3699.6804 lr= 0.000100
  2861. 2019-03-12 19:58:38,600 : epoch 315 , step 0 loss0= 0.0000 loss1= 2.5625 loss2=-3379.1145 lr= 0.000100
  2862. 2019-03-12 19:59:14,313 : epoch 315 , step 40 loss0= 0.0000 loss1= 2.5200 loss2=-3834.8489 lr= 0.000100
  2863. 2019-03-12 19:59:50,228 : epoch 315 , step 80 loss0= 0.0000 loss1= 2.2879 loss2=-3902.3601 lr= 0.000100
  2864. 2019-03-12 20:00:26,139 : epoch 315 , step 120 loss0= 0.0000 loss1= 2.6714 loss2=-3577.1938 lr= 0.000100
  2865. 2019-03-12 20:01:02,076 : epoch 315 , step 160 loss0= 0.0000 loss1= 3.2816 loss2=-3654.9954 lr= 0.000100
  2866. 2019-03-12 20:01:34,712 : epoch 316 , step 0 loss0= 0.0000 loss1= 2.4494 loss2=-3459.1013 lr= 0.000100
  2867. 2019-03-12 20:02:10,243 : epoch 316 , step 40 loss0= 0.0000 loss1= 2.7983 loss2=-3733.3657 lr= 0.000100
  2868. 2019-03-12 20:02:46,280 : epoch 316 , step 80 loss0= 0.0000 loss1= 1.9010 loss2=-3453.1985 lr= 0.000100
  2869. 2019-03-12 20:03:22,268 : epoch 316 , step 120 loss0= 0.0000 loss1= 1.9295 loss2=-3718.1570 lr= 0.000100
  2870. 2019-03-12 20:03:58,126 : epoch 316 , step 160 loss0= 0.0000 loss1= 3.1074 loss2=-3428.6189 lr= 0.000100
  2871. 2019-03-12 20:04:30,274 : epoch 317 , step 0 loss0= 0.0000 loss1= 2.4207 loss2=-3794.4021 lr= 0.000100
  2872. 2019-03-12 20:05:05,980 : epoch 317 , step 40 loss0= 0.0000 loss1= 2.3248 loss2=-3817.8181 lr= 0.000100
  2873. 2019-03-12 20:05:41,178 : epoch 317 , step 80 loss0= 0.0000 loss1= 2.5351 loss2=-3613.9067 lr= 0.000100
  2874. 2019-03-12 20:06:16,398 : epoch 317 , step 120 loss0= 0.0000 loss1= 2.8451 loss2=-3487.4734 lr= 0.000100
  2875. 2019-03-12 20:06:51,742 : epoch 317 , step 160 loss0= 0.0000 loss1= 3.1802 loss2=-3895.0662 lr= 0.000100
  2876. 2019-03-12 20:07:24,302 : epoch 318 , step 0 loss0= 0.0000 loss1= 1.6675 loss2=-3179.7842 lr= 0.000100
  2877. 2019-03-12 20:08:00,382 : epoch 318 , step 40 loss0= 0.0000 loss1= 2.0436 loss2=-3588.8201 lr= 0.000100
  2878. 2019-03-12 20:08:36,580 : epoch 318 , step 80 loss0= 0.0000 loss1= 2.4743 loss2=-3895.9048 lr= 0.000100
  2879. 2019-03-12 20:09:12,238 : epoch 318 , step 120 loss0= 0.0000 loss1= 2.3193 loss2=-3619.8215 lr= 0.000100
  2880. 2019-03-12 20:09:48,541 : epoch 318 , step 160 loss0= 0.0000 loss1= 3.1779 loss2=-3740.4426 lr= 0.000100
  2881. 2019-03-12 20:10:21,167 : epoch 319 , step 0 loss0= 0.0000 loss1= 1.8586 loss2=-3706.5398 lr= 0.000100
  2882. 2019-03-12 20:10:57,046 : epoch 319 , step 40 loss0= 0.0000 loss1= 3.0248 loss2=-3770.4646 lr= 0.000100
  2883. 2019-03-12 20:11:33,383 : epoch 319 , step 80 loss0= 0.0000 loss1= 2.3205 loss2=-3599.9185 lr= 0.000100
  2884. 2019-03-12 20:12:09,920 : epoch 319 , step 120 loss0= 0.0000 loss1= 3.0748 loss2=-4166.8291 lr= 0.000100
  2885. 2019-03-12 20:12:46,248 : epoch 319 , step 160 loss0= 0.0000 loss1= 3.1118 loss2=-3823.1072 lr= 0.000100
  2886. 2019-03-12 20:13:19,460 : epoch 320 , step 0 loss0= 0.0000 loss1= 1.5678 loss2=-3386.2090 lr= 0.000100
  2887. 2019-03-12 20:13:55,233 : epoch 320 , step 40 loss0= 0.0000 loss1= 1.9030 loss2=-3430.0508 lr= 0.000100
  2888. 2019-03-12 20:14:31,161 : epoch 320 , step 80 loss0= 0.0000 loss1= 2.1794 loss2=-3835.6296 lr= 0.000100
  2889. 2019-03-12 20:15:07,230 : epoch 320 , step 120 loss0= 0.0000 loss1= 2.8832 loss2=-3780.3989 lr= 0.000100
  2890. 2019-03-12 20:15:43,482 : epoch 320 , step 160 loss0= 0.0000 loss1= 3.2032 loss2=-3869.3423 lr= 0.000100
  2891. 2019-03-12 20:16:15,033 : ## train ##
  2892. 2019-03-12 20:21:09,683 : >> epoch 320 : average_loss= 0.7995, average_accuracy=82.522%, mean_weight_std= 0.3337, mean_normalized_std= 0.5783 mean_dist=2.849e+03
  2893. 2019-03-12 20:21:09,691 : >> epoch 320 : median_loss= 0.7922, median_accuracy=82.280%, median_weight_std= 0.1974, median_normalized_std= 0.0932 median_dist=2.819e+03
  2894. 2019-03-12 20:21:09,692 : >> epoch 320 : ensemble accuracy: 84.97%
  2895. 2019-03-12 20:21:09,692 : >> epoch 320 : loss histogram: (0.00,0.33, 18),(0.33,0.66, 0),(0.66,1.00, 4),(1.00,1.33, 0),(1.33,1.66, 18),
  2896. 2019-03-12 20:21:09,692 : >> epoch 320 : accuracy histogram: (0.00,33.56, 18),(33.56,67.13, 0),(67.13,100.69, 4),(100.69,134.26, 0),(134.26,167.82, 18),
  2897. 2019-03-12 20:21:09,698 : >> epoch 320 : weight std histogram: (0.0000,3.0406, 464300),(3.0406,6.0812, 2321),(6.0812,9.1219, 245),(9.1219,12.1625, 36),(12.1625,15.2031, 4),
  2898. 2019-03-12 20:21:09,703 : >> epoch 320 : weight normalized std histogram: (0.0000,1767.3227, 466892),(1767.3227,3534.6453, 5),(3534.6453,5301.9680, 5),(5301.9680,7069.2906, 2),(7069.2906,8836.6133, 2),
  2899. 2019-03-12 20:21:09,703 : >> epoch 320 : dist histogram: (2584.9353,2748.6829, 7),(2748.6829,2912.4305, 7),(2912.4305,3076.1781, 4),(3076.1781,3239.9257, 1),(3239.9257,3403.6733, 1),
  2900. 2019-03-12 20:21:09,703 : ## eval ##
  2901. 2019-03-12 20:26:03,407 : >> epoch 320 : average_loss= 0.6813, average_accuracy=86.112%, mean_weight_std= 0.3897, mean_normalized_std= 0.7821 mean_dist=2.848e+03
  2902. 2019-03-12 20:26:03,416 : >> epoch 320 : median_loss= 0.6707, median_accuracy=87.365%, median_weight_std= 0.2364, median_normalized_std= 0.1104 median_dist=2.779e+03
  2903. 2019-03-12 20:26:03,416 : >> epoch 320 : ensemble accuracy: 88.09%
  2904. 2019-03-12 20:26:03,416 : >> epoch 320 : loss histogram: (0.00,0.29, 18),(0.29,0.59, 0),(0.59,0.88, 4),(0.88,1.18, 0),(1.18,1.47, 18),
  2905. 2019-03-12 20:26:03,417 : >> epoch 320 : accuracy histogram: (0.00,34.96, 18),(34.96,69.92, 0),(69.92,104.89, 4),(104.89,139.85, 0),(139.85,174.81, 18),
  2906. 2019-03-12 20:26:03,423 : >> epoch 320 : weight std histogram: (0.0000,3.0268, 463596),(3.0268,6.0536, 2913),(6.0536,9.0805, 359),(9.0805,12.1073, 34),(12.1073,15.1341, 4),
  2907. 2019-03-12 20:26:03,428 : >> epoch 320 : weight normalized std histogram: (0.0000,6967.8781, 466900),(6967.8781,13935.7563, 2),(13935.7563,20903.6344, 1),(20903.6344,27871.5125, 1),(27871.5125,34839.3906, 2),
  2908. 2019-03-12 20:26:03,428 : >> epoch 320 : dist histogram: (2565.6921,2745.8429, 9),(2745.8429,2925.9937, 6),(2925.9937,3106.1445, 2),(3106.1445,3286.2953, 1),(3286.2953,3466.4460, 2),
  2909. 2019-03-12 20:26:03,484 : >> epoch 320 : dist pair-wise
  2910. 2019-03-12 20:26:03,490 :
  2911. [[ 0. 649.692 511.767 564.668 1102.45 928.995 626.827
  2912. 546.916 601.705 423.867 395.76 724.524 444.021 657.167
  2913. 601.688 573.484 557.073 311.209 569.771 536.655]
  2914. [ 649.692 0. 452.799 589.891 1232.657 1135.529 252.835
  2915. 563.402 245.345 396.697 527.587 813.738 478.041 172.613
  2916. 269.497 445.896 553.167 673.939 289.567 312.413]
  2917. [ 511.767 452.799 0. 454.77 1269.348 1063.458 515.3
  2918. 500.385 470.083 336.568 496.812 776.295 409.176 478.361
  2919. 434.123 405.033 450.326 533.519 372.631 348.66 ]
  2920. [ 564.668 589.891 454.77 0. 1150.033 927.946 641.718
  2921. 276.878 514.047 461.803 501.216 423.995 479.737 627.984
  2922. 579.05 567.042 278.917 542.744 402.233 500.826]
  2923. [ 1102.45 1232.657 1269.348 1150.033 0. 1030.421 1077.276
  2924. 1084.066 1174.948 1117.004 999.471 1067.17 1036.362 1242.391
  2925. 1112.197 1063.012 1092.114 948.665 1176.997 1165.848]
  2926. [ 928.995 1135.529 1063.458 927.946 1030.421 0. 1059.735
  2927. 832.709 1015.505 1025.585 848.403 773.88 925.961 1131.774
  2928. 1072.844 1094.097 965.584 924.338 980.238 964.09 ]
  2929. [ 626.827 252.835 515.3 641.718 1077.276 1059.735 0.
  2930. 571.866 289.2 396.589 479.6 831.828 443.337 283.008
  2931. 223.63 407.563 568.285 608.243 355.062 325.515]
  2932. [ 546.916 563.402 500.385 276.878 1084.066 832.709 571.866
  2933. 0. 433.57 437.942 386.434 464.105 411.471 603.29
  2934. 531.527 550.258 320.874 528.761 350.837 437.35 ]
  2935. [ 601.705 245.345 470.083 514.047 1174.948 1015.505 289.2
  2936. 433.57 0. 397.47 440.185 705.29 415.584 301.47
  2937. 301.765 467.01 496.898 634.208 205.561 250.802]
  2938. [ 423.867 396.697 336.568 461.803 1117.004 1025.585 396.589
  2939. 437.942 397.47 0. 322.835 747.644 249.149 437.776
  2940. 332.928 307.765 384.208 429.516 346.95 312.907]
  2941. [ 395.76 527.587 496.812 501.216 999.471 848.403 479.6
  2942. 386.434 440.185 322.835 0. 656.589 220.697 548.766
  2943. 424.678 419.492 484.505 407.172 406.235 373.676]
  2944. [ 724.524 813.738 776.295 423.995 1067.17 773.88 831.828
  2945. 464.105 705.29 747.644 656.589 0. 711.545 821.406
  2946. 810.393 836.828 557.878 712.458 635.429 732.277]
  2947. [ 444.021 478.041 409.176 479.737 1036.362 925.961 443.337
  2948. 411.471 415.584 249.149 220.697 711.545 0. 513.415
  2949. 372.338 308.496 431.32 429.142 371.653 320.947]
  2950. [ 657.167 172.613 478.361 627.984 1242.391 1131.774 283.008
  2951. 603.29 301.47 437.776 548.766 821.406 513.415 0.
  2952. 291.897 474.425 595.572 696.272 335.601 338.147]
  2953. [ 601.688 269.497 434.123 579.05 1112.197 1072.844 223.63
  2954. 531.527 301.765 332.928 424.678 810.393 372.338 291.897
  2955. 0. 288.747 523.71 584.797 309.319 266.604]
  2956. [ 573.484 445.896 405.033 567.042 1063.012 1094.097 407.563
  2957. 550.258 467.01 307.765 419.492 836.828 308.496 474.425
  2958. 288.747 0. 494.438 502.727 427.848 371.957]
  2959. [ 557.073 553.167 450.326 278.917 1092.114 965.584 568.285
  2960. 320.874 496.898 384.208 484.505 557.878 431.32 595.572
  2961. 523.71 494.438 0. 505.203 400.627 463.771]
  2962. [ 311.209 673.939 533.519 542.744 948.665 924.338 608.243
  2963. 528.761 634.208 429.516 407.172 712.458 429.142 696.272
  2964. 584.797 502.727 505.203 0. 587.28 573.048]
  2965. [ 569.771 289.567 372.631 402.233 1176.997 980.238 355.062
  2966. 350.837 205.561 346.95 406.235 635.429 371.653 335.601
  2967. 309.319 427.848 400.627 587.28 0. 206.002]
  2968. [ 536.655 312.413 348.66 500.826 1165.848 964.09 325.515
  2969. 437.35 250.802 312.907 373.676 732.277 320.947 338.147
  2970. 266.604 371.957 463.771 573.048 206.002 0. ]]
  2971. 2019-03-12 20:26:04,951 : epoch 321 , step 0 loss0= 0.0000 loss1= 1.6304 loss2=-3691.5664 lr= 0.000100
  2972. 2019-03-12 20:26:40,574 : epoch 321 , step 40 loss0= 0.0000 loss1= 2.7272 loss2=-3799.8425 lr= 0.000100
  2973. 2019-03-12 20:27:17,014 : epoch 321 , step 80 loss0= 0.0000 loss1= 2.7884 loss2=-3960.2754 lr= 0.000100
  2974. 2019-03-12 20:27:53,377 : epoch 321 , step 120 loss0= 0.0000 loss1= 2.4136 loss2=-3837.4458 lr= 0.000100
  2975. 2019-03-12 20:28:29,862 : epoch 321 , step 160 loss0= 0.0000 loss1= 2.7633 loss2=-3648.1724 lr= 0.000100
  2976. 2019-03-12 20:29:02,910 : epoch 322 , step 0 loss0= 0.0000 loss1= 2.4373 loss2=-3919.2615 lr= 0.000100
  2977. 2019-03-12 20:29:39,145 : epoch 322 , step 40 loss0= 0.0000 loss1= 2.0239 loss2=-3777.3840 lr= 0.000100
  2978. 2019-03-12 20:30:15,475 : epoch 322 , step 80 loss0= 0.0000 loss1= 2.4958 loss2=-3591.0681 lr= 0.000100
  2979. 2019-03-12 20:30:50,838 : epoch 322 , step 120 loss0= 0.0000 loss1= 1.7061 loss2=-3330.7664 lr= 0.000100
  2980. 2019-03-12 20:31:26,229 : epoch 322 , step 160 loss0= 0.0000 loss1= 2.9062 loss2=-3594.8728 lr= 0.000100
  2981. 2019-03-12 20:31:58,363 : epoch 323 , step 0 loss0= 0.0000 loss1= 2.7975 loss2=-3707.2153 lr= 0.000100
  2982. 2019-03-12 20:32:33,756 : epoch 323 , step 40 loss0= 0.0000 loss1= 2.4468 loss2=-3519.3403 lr= 0.000100
  2983. 2019-03-12 20:33:10,072 : epoch 323 , step 80 loss0= 0.0000 loss1= 3.0072 loss2=-3872.4238 lr= 0.000100
  2984. 2019-03-12 20:33:46,383 : epoch 323 , step 120 loss0= 0.0000 loss1= 1.6778 loss2=-3601.8601 lr= 0.000100
  2985. 2019-03-12 20:34:22,501 : epoch 323 , step 160 loss0= 0.0000 loss1= 2.3744 loss2=-3395.0593 lr= 0.000100
  2986. 2019-03-12 20:34:55,246 : epoch 324 , step 0 loss0= 0.0000 loss1= 2.3474 loss2=-3686.3162 lr= 0.000100
  2987. 2019-03-12 20:35:30,877 : epoch 324 , step 40 loss0= 0.0000 loss1= 2.5407 loss2=-4027.2878 lr= 0.000100
  2988. 2019-03-12 20:36:06,970 : epoch 324 , step 80 loss0= 0.0000 loss1= 1.8204 loss2=-3640.0083 lr= 0.000100
  2989. 2019-03-12 20:36:42,174 : epoch 324 , step 120 loss0= 0.0000 loss1= 1.9953 loss2=-3434.9946 lr= 0.000100
  2990. 2019-03-12 20:37:18,012 : epoch 324 , step 160 loss0= 0.0000 loss1= 2.3756 loss2=-3715.5623 lr= 0.000100
  2991. 2019-03-12 20:37:50,382 : epoch 325 , step 0 loss0= 0.0000 loss1= 2.1120 loss2=-3372.7024 lr= 0.000100
  2992. 2019-03-12 20:38:26,340 : epoch 325 , step 40 loss0= 0.0000 loss1= 2.8591 loss2=-3806.5291 lr= 0.000100
  2993. 2019-03-12 20:39:02,216 : epoch 325 , step 80 loss0= 0.0000 loss1= 2.7276 loss2=-3721.6169 lr= 0.000100
  2994. 2019-03-12 20:39:38,170 : epoch 325 , step 120 loss0= 0.0000 loss1= 2.1962 loss2=-3502.1582 lr= 0.000100
  2995. 2019-03-12 20:40:13,868 : epoch 325 , step 160 loss0= 0.0000 loss1= 2.4085 loss2=-3840.0845 lr= 0.000100
  2996. 2019-03-12 20:40:46,040 : epoch 326 , step 0 loss0= 0.0000 loss1= 1.9694 loss2=-3475.6868 lr= 0.000100
  2997. 2019-03-12 20:41:21,670 : epoch 326 , step 40 loss0= 0.0000 loss1= 2.5284 loss2=-3582.8521 lr= 0.000100
  2998. 2019-03-12 20:41:56,870 : epoch 326 , step 80 loss0= 0.0000 loss1= 2.8324 loss2=-3357.6841 lr= 0.000100
  2999. 2019-03-12 20:42:32,347 : epoch 326 , step 120 loss0= 0.0000 loss1= 1.6186 loss2=-3484.5376 lr= 0.000100
  3000. 2019-03-12 20:43:08,238 : epoch 326 , step 160 loss0= 0.0000 loss1= 2.5054 loss2=-3679.7117 lr= 0.000100
  3001. 2019-03-12 20:43:40,431 : epoch 327 , step 0 loss0= 0.0000 loss1= 2.2765 loss2=-3782.3833 lr= 0.000100
  3002. 2019-03-12 20:44:16,316 : epoch 327 , step 40 loss0= 0.0000 loss1= 2.3206 loss2=-3578.9109 lr= 0.000100
  3003. 2019-03-12 20:44:52,073 : epoch 327 , step 80 loss0= 0.0000 loss1= 2.2349 loss2=-3650.6882 lr= 0.000100
  3004. 2019-03-12 20:45:27,437 : epoch 327 , step 120 loss0= 0.0000 loss1= 2.2055 loss2=-3664.3679 lr= 0.000100
  3005. 2019-03-12 20:46:03,557 : epoch 327 , step 160 loss0= 0.0000 loss1= 1.9852 loss2=-3326.7603 lr= 0.000100
  3006. 2019-03-12 20:46:35,839 : epoch 328 , step 0 loss0= 0.0000 loss1= 2.2550 loss2=-3440.2305 lr= 0.000100
  3007. 2019-03-12 20:47:11,583 : epoch 328 , step 40 loss0= 0.0000 loss1= 2.1147 loss2=-3717.6160 lr= 0.000100
  3008. 2019-03-12 20:47:46,912 : epoch 328 , step 80 loss0= 0.0000 loss1= 3.0344 loss2=-3759.7800 lr= 0.000100
  3009. 2019-03-12 20:48:22,194 : epoch 328 , step 120 loss0= 0.0000 loss1= 2.3136 loss2=-3335.9712 lr= 0.000100
  3010. 2019-03-12 20:48:57,625 : epoch 328 , step 160 loss0= 0.0000 loss1= 1.8253 loss2=-3215.9514 lr= 0.000100
  3011. 2019-03-12 20:49:29,779 : epoch 329 , step 0 loss0= 0.0000 loss1= 1.7839 loss2=-3529.0063 lr= 0.000100
  3012. 2019-03-12 20:50:05,588 : epoch 329 , step 40 loss0= 0.0000 loss1= 2.6537 loss2=-3700.3501 lr= 0.000100
  3013. 2019-03-12 20:50:41,802 : epoch 329 , step 80 loss0= 0.0000 loss1= 2.2616 loss2=-3228.2412 lr= 0.000100
  3014. 2019-03-12 20:51:17,929 : epoch 329 , step 120 loss0= 0.0000 loss1= 2.1970 loss2=-3515.4590 lr= 0.000100
  3015. 2019-03-12 20:51:53,580 : epoch 329 , step 160 loss0= 0.0000 loss1= 2.4434 loss2=-3608.3020 lr= 0.000100
  3016. 2019-03-12 20:52:25,776 : epoch 330 , step 0 loss0= 0.0000 loss1= 2.9227 loss2=-3221.9968 lr= 0.000100
  3017. 2019-03-12 20:53:01,176 : epoch 330 , step 40 loss0= 0.0000 loss1= 2.4198 loss2=-3871.3032 lr= 0.000100
  3018. 2019-03-12 20:53:37,061 : epoch 330 , step 80 loss0= 0.0000 loss1= 2.9949 loss2=-3793.1833 lr= 0.000100
  3019. 2019-03-12 20:54:12,493 : epoch 330 , step 120 loss0= 0.0000 loss1= 2.5947 loss2=-3397.3601 lr= 0.000100
  3020. 2019-03-12 20:54:47,989 : epoch 330 , step 160 loss0= 0.0000 loss1= 2.9465 loss2=-3617.0493 lr= 0.000100
  3021. 2019-03-12 20:55:20,188 : epoch 331 , step 0 loss0= 0.0000 loss1= 2.6171 loss2=-3665.0720 lr= 0.000100
  3022. 2019-03-12 20:55:55,765 : epoch 331 , step 40 loss0= 0.0000 loss1= 3.3736 loss2=-4021.6985 lr= 0.000100
  3023. 2019-03-12 20:56:31,267 : epoch 331 , step 80 loss0= 0.0000 loss1= 2.9560 loss2=-3963.4302 lr= 0.000100
  3024. 2019-03-12 20:57:07,626 : epoch 331 , step 120 loss0= 0.0000 loss1= 1.9232 loss2=-3481.8125 lr= 0.000100
  3025. 2019-03-12 20:57:43,800 : epoch 331 , step 160 loss0= 0.0000 loss1= 2.7587 loss2=-3270.3733 lr= 0.000100
  3026. 2019-03-12 20:58:16,768 : epoch 332 , step 0 loss0= 0.0000 loss1= 2.5920 loss2=-3781.0547 lr= 0.000100
  3027. 2019-03-12 20:58:52,196 : epoch 332 , step 40 loss0= 0.0000 loss1= 2.5602 loss2=-3744.7312 lr= 0.000100
  3028. 2019-03-12 20:59:27,907 : epoch 332 , step 80 loss0= 0.0000 loss1= 3.2027 loss2=-4039.8757 lr= 0.000100
  3029. 2019-03-12 21:00:03,260 : epoch 332 , step 120 loss0= 0.0000 loss1= 2.5328 loss2=-3827.8320 lr= 0.000100
  3030. 2019-03-12 21:00:38,645 : epoch 332 , step 160 loss0= 0.0000 loss1= 2.9696 loss2=-3652.6868 lr= 0.000100
  3031. 2019-03-12 21:01:10,930 : epoch 333 , step 0 loss0= 0.0000 loss1= 1.6622 loss2=-3594.4539 lr= 0.000100
  3032. 2019-03-12 21:01:46,590 : epoch 333 , step 40 loss0= 0.0000 loss1= 3.3403 loss2=-3824.4021 lr= 0.000100
  3033. 2019-03-12 21:02:22,245 : epoch 333 , step 80 loss0= 0.0000 loss1= 2.1950 loss2=-3502.1296 lr= 0.000100
  3034. 2019-03-12 21:02:58,574 : epoch 333 , step 120 loss0= 0.0000 loss1= 2.5657 loss2=-3817.3579 lr= 0.000100
  3035. 2019-03-12 21:03:34,705 : epoch 333 , step 160 loss0= 0.0000 loss1= 2.3320 loss2=-3454.1140 lr= 0.000100
  3036. 2019-03-12 21:04:07,119 : epoch 334 , step 0 loss0= 0.0000 loss1= 2.1704 loss2=-3713.2498 lr= 0.000100
  3037. 2019-03-12 21:04:43,051 : epoch 334 , step 40 loss0= 0.0000 loss1= 2.6261 loss2=-3687.4570 lr= 0.000100
  3038. 2019-03-12 21:05:18,930 : epoch 334 , step 80 loss0= 0.0000 loss1= 2.3776 loss2=-3646.5251 lr= 0.000100
  3039. 2019-03-12 21:05:54,654 : epoch 334 , step 120 loss0= 0.0000 loss1= 1.7899 loss2=-3509.8689 lr= 0.000100
  3040. 2019-03-12 21:06:31,167 : epoch 334 , step 160 loss0= 0.0000 loss1= 1.8333 loss2=-3572.6392 lr= 0.000100
  3041. 2019-03-12 21:07:04,233 : epoch 335 , step 0 loss0= 0.0000 loss1= 2.3494 loss2=-3785.4939 lr= 0.000100
  3042. 2019-03-12 21:07:39,948 : epoch 335 , step 40 loss0= 0.0000 loss1= 3.2230 loss2=-3982.1677 lr= 0.000100
  3043. 2019-03-12 21:08:15,273 : epoch 335 , step 80 loss0= 0.0000 loss1= 2.4215 loss2=-3660.2273 lr= 0.000100
  3044. 2019-03-12 21:08:50,564 : epoch 335 , step 120 loss0= 0.0000 loss1= 2.4790 loss2=-3620.4216 lr= 0.000100
  3045. 2019-03-12 21:09:26,045 : epoch 335 , step 160 loss0= 0.0000 loss1= 2.4467 loss2=-4044.2969 lr= 0.000100
  3046. 2019-03-12 21:09:58,976 : epoch 336 , step 0 loss0= 0.0000 loss1= 2.6781 loss2=-3763.8835 lr= 0.000100
  3047. 2019-03-12 21:10:34,577 : epoch 336 , step 40 loss0= 0.0000 loss1= 2.1708 loss2=-3362.5837 lr= 0.000100
  3048. 2019-03-12 21:11:10,032 : epoch 336 , step 80 loss0= 0.0000 loss1= 3.6656 loss2=-3756.7566 lr= 0.000100
  3049. 2019-03-12 21:11:46,334 : epoch 336 , step 120 loss0= 0.0000 loss1= 2.5624 loss2=-3829.2849 lr= 0.000100
  3050. 2019-03-12 21:12:21,748 : epoch 336 , step 160 loss0= 0.0000 loss1= 2.7042 loss2=-3460.5852 lr= 0.000100
  3051. 2019-03-12 21:12:53,936 : epoch 337 , step 0 loss0= 0.0000 loss1= 2.5647 loss2=-3837.1978 lr= 0.000100
  3052. 2019-03-12 21:13:29,134 : epoch 337 , step 40 loss0= 0.0000 loss1= 1.7622 loss2=-3475.1587 lr= 0.000100
  3053. 2019-03-12 21:14:04,809 : epoch 337 , step 80 loss0= 0.0000 loss1= 2.4351 loss2=-3865.8455 lr= 0.000100
  3054. 2019-03-12 21:14:40,661 : epoch 337 , step 120 loss0= 0.0000 loss1= 2.3550 loss2=-3639.4395 lr= 0.000100
  3055. 2019-03-12 21:15:15,955 : epoch 337 , step 160 loss0= 0.0000 loss1= 2.6628 loss2=-3732.7114 lr= 0.000100
  3056. 2019-03-12 21:15:48,345 : epoch 338 , step 0 loss0= 0.0000 loss1= 1.9201 loss2=-3517.3262 lr= 0.000100
  3057. 2019-03-12 21:16:24,510 : epoch 338 , step 40 loss0= 0.0000 loss1= 2.0900 loss2=-3551.1392 lr= 0.000100
  3058. 2019-03-12 21:17:00,616 : epoch 338 , step 80 loss0= 0.0000 loss1= 2.4504 loss2=-3745.0610 lr= 0.000100
  3059. 2019-03-12 21:17:36,758 : epoch 338 , step 120 loss0= 0.0000 loss1= 2.1421 loss2=-3672.5540 lr= 0.000100
  3060. 2019-03-12 21:18:12,342 : epoch 338 , step 160 loss0= 0.0000 loss1= 1.8261 loss2=-3238.0720 lr= 0.000100
  3061. 2019-03-12 21:18:44,658 : epoch 339 , step 0 loss0= 0.0000 loss1= 1.7417 loss2=-3491.4666 lr= 0.000100
  3062. 2019-03-12 21:19:20,457 : epoch 339 , step 40 loss0= 0.0000 loss1= 2.0884 loss2=-3560.1121 lr= 0.000100
  3063. 2019-03-12 21:19:56,323 : epoch 339 , step 80 loss0= 0.0000 loss1= 1.6369 loss2=-3467.7458 lr= 0.000100
  3064. 2019-03-12 21:20:31,614 : epoch 339 , step 120 loss0= 0.0000 loss1= 2.2135 loss2=-3399.0740 lr= 0.000100
  3065. 2019-03-12 21:21:08,068 : epoch 339 , step 160 loss0= 0.0000 loss1= 3.4426 loss2=-3919.6384 lr= 0.000100
  3066. 2019-03-12 21:21:40,818 : epoch 340 , step 0 loss0= 0.0000 loss1= 2.3602 loss2=-3315.3689 lr= 0.000100
  3067. 2019-03-12 21:22:16,501 : epoch 340 , step 40 loss0= 0.0000 loss1= 1.7953 loss2=-3513.7649 lr= 0.000100
  3068. 2019-03-12 21:22:52,495 : epoch 340 , step 80 loss0= 0.0000 loss1= 2.3130 loss2=-3595.3435 lr= 0.000100
  3069. 2019-03-12 21:23:28,479 : epoch 340 , step 120 loss0= 0.0000 loss1= 2.2401 loss2=-3633.8730 lr= 0.000100
  3070. 2019-03-12 21:24:04,243 : epoch 340 , step 160 loss0= 0.0000 loss1= 2.2835 loss2=-3648.4185 lr= 0.000100
  3071. 2019-03-12 21:24:35,661 : ## train ##
  3072. 2019-03-12 21:29:32,192 : >> epoch 340 : average_loss= 0.7750, average_accuracy=82.871%, mean_weight_std= 0.3503, mean_normalized_std= 0.4640 mean_dist=2.894e+03
  3073. 2019-03-12 21:29:32,201 : >> epoch 340 : median_loss= 0.7529, median_accuracy=83.090%, median_weight_std= 0.2107, median_normalized_std= 0.0976 median_dist=2.89e+03
  3074. 2019-03-12 21:29:32,201 : >> epoch 340 : ensemble accuracy: 85.66%
  3075. 2019-03-12 21:29:32,201 : >> epoch 340 : loss histogram: (0.00,0.34, 18),(0.34,0.69, 0),(0.69,1.03, 4),(1.03,1.37, 0),(1.37,1.71, 18),
  3076. 2019-03-12 21:29:32,201 : >> epoch 340 : accuracy histogram: (0.00,34.03, 18),(34.03,68.06, 0),(68.06,102.10, 4),(102.10,136.13, 0),(136.13,170.16, 18),
  3077. 2019-03-12 21:29:32,208 : >> epoch 340 : weight std histogram: (0.0000,3.2435, 464555),(3.2435,6.4870, 2094),(6.4870,9.7305, 233),(9.7305,12.9740, 20),(12.9740,16.2175, 4),
  3078. 2019-03-12 21:29:32,213 : >> epoch 340 : weight normalized std histogram: (0.0000,1851.6586, 466900),(1851.6586,3703.3172, 5),(3703.3172,5554.9758, 0),(5554.9758,7406.6344, 0),(7406.6344,9258.2930, 1),
  3079. 2019-03-12 21:29:32,214 : >> epoch 340 : dist histogram: (2525.2776,2668.5296, 2),(2668.5296,2811.7817, 5),(2811.7817,2955.0337, 6),(2955.0337,3098.2858, 4),(3098.2858,3241.5378, 3),
  3080. 2019-03-12 21:29:32,214 : ## eval ##
  3081. 2019-03-12 21:34:27,259 : >> epoch 340 : average_loss= 0.6970, average_accuracy=85.944%, mean_weight_std= 0.3622, mean_normalized_std= 0.5841 mean_dist=2.893e+03
  3082. 2019-03-12 21:34:27,268 : >> epoch 340 : median_loss= 0.7002, median_accuracy=86.055%, median_weight_std= 0.2195, median_normalized_std= 0.1015 median_dist=2.869e+03
  3083. 2019-03-12 21:34:27,268 : >> epoch 340 : ensemble accuracy: 88.48%
  3084. 2019-03-12 21:34:27,269 : >> epoch 340 : loss histogram: (0.00,0.31, 18),(0.31,0.63, 0),(0.63,0.94, 4),(0.94,1.26, 0),(1.26,1.57, 18),
  3085. 2019-03-12 21:34:27,269 : >> epoch 340 : accuracy histogram: (0.00,35.15, 18),(35.15,70.31, 0),(70.31,105.46, 4),(105.46,140.62, 0),(140.62,175.77, 18),
  3086. 2019-03-12 21:34:27,280 : >> epoch 340 : weight std histogram: (0.0000,2.9012, 463779),(2.9012,5.8024, 2758),(5.8024,8.7035, 329),(8.7035,11.6047, 35),(11.6047,14.5059, 5),
  3087. 2019-03-12 21:34:27,286 : >> epoch 340 : weight normalized std histogram: (0.0000,9771.6562, 466904),(9771.6562,19543.3125, 1),(19543.3125,29314.9688, 0),(29314.9688,39086.6250, 0),(39086.6250,48858.2812, 1),
  3088. 2019-03-12 21:34:27,286 : >> epoch 340 : dist histogram: (2553.5315,2728.8182, 4),(2728.8182,2904.1049, 9),(2904.1049,3079.3917, 4),(3079.3917,3254.6784, 1),(3254.6784,3429.9651, 2),
  3089. 2019-03-12 21:34:27,349 : >> epoch 340 : dist pair-wise
  3090. 2019-03-12 21:34:27,355 :
  3091. [[ 0. 359.274 468.758 391.075 535.422 401.483 775.917
  3092. 341.176 657.278 443.387 438.558 710.416 450.638 382.396
  3093. 642.949 383.71 874.781 575.017 268.19 476.334]
  3094. [ 359.274 0. 687.375 589.621 685.472 556.008 821.69
  3095. 438.115 753.688 648.188 623.81 770.424 644.435 522.704
  3096. 766.191 497.804 900.197 722.82 353.682 649.573]
  3097. [ 468.758 687.375 0. 393.192 454.915 378.397 727.096
  3098. 506.336 545.424 255.928 341.238 781.21 301.255 391.778
  3099. 487.383 504.384 932.118 493.722 523.165 338.337]
  3100. [ 391.075 589.621 393.192 0. 423.95 349.758 582.443
  3101. 484.839 496.108 366.316 279.161 591.745 293.643 350.166
  3102. 507.395 481.612 760.418 558.823 531.04 487.92 ]
  3103. [ 535.422 685.472 454.915 423.95 0. 534.449 445.406
  3104. 631.19 634.947 305.272 495.768 782.888 483.913 491.909
  3105. 585.687 630.944 787.794 650.458 590.289 429.449]
  3106. [ 401.483 556.008 378.397 349.758 534.449 0. 730.547
  3107. 325. 469.915 413.435 289.773 758.422 304.33 316.199
  3108. 393.947 294.936 966.815 332.341 482.2 413.512]
  3109. [ 775.917 821.69 727.096 582.443 445.406 730.547 0.
  3110. 817.924 645.598 634.436 643.785 771.586 662.518 704.522
  3111. 698.154 844.897 672.681 849.113 830.227 748.972]
  3112. [ 341.176 438.115 506.336 484.839 631.19 325. 817.924
  3113. 0. 555.913 540.341 409.834 806.345 467.872 439.57
  3114. 516.419 268.36 1018.073 421.825 400.695 517.317]
  3115. [ 657.278 753.688 545.424 496.108 634.947 469.915 645.598
  3116. 555.913 0. 599.689 353.997 722.176 468.584 539.371
  3117. 348.886 645.457 865.802 547.83 727.771 662.709]
  3118. [ 443.387 648.188 255.928 366.316 305.272 413.435 634.436
  3119. 540.341 599.689 0. 410.366 738.284 351.566 379.189
  3120. 543.035 515.536 861.633 559.122 471.671 240.881]
  3121. [ 438.558 623.81 341.238 279.161 495.768 289.773 643.785
  3122. 409.834 353.997 410.366 0. 640.853 247.818 370.497
  3123. 398.411 464.159 834.372 470.665 554.175 488.203]
  3124. [ 710.416 770.424 781.21 591.745 782.888 758.422 771.586
  3125. 806.345 722.176 738.284 640.853 0. 680.397 660.783
  3126. 924.033 881.019 680.969 1008.839 734.617 842.682]
  3127. [ 450.638 644.435 301.255 293.643 483.913 304.33 662.518
  3128. 467.872 468.584 351.566 247.818 680.397 0. 400.537
  3129. 462.821 444.75 841.851 485.936 555.236 445.942]
  3130. [ 382.396 522.704 391.778 350.166 491.909 316.199 704.522
  3131. 439.57 539.371 379.189 370.497 660.783 400.537 0.
  3132. 516.031 463.896 906.263 502.9 433.176 397.007]
  3133. [ 642.949 766.191 487.383 507.395 585.687 393.947 698.154
  3134. 516.419 348.886 543.035 398.411 924.033 462.821 516.031
  3135. 0. 520.729 1032.995 316.743 725.177 554.775]
  3136. [ 383.71 497.804 504.384 481.612 630.944 294.936 844.897
  3137. 268.36 645.457 515.536 464.159 881.019 444.75 463.896
  3138. 520.729 0. 1079.006 374.998 452.026 475.044]
  3139. [ 874.781 900.197 932.118 760.418 787.794 966.815 672.681
  3140. 1018.073 865.802 861.633 834.372 680.969 841.851 906.263
  3141. 1032.995 1079.006 0. 1162.146 929.612 997.861]
  3142. [ 575.017 722.82 493.722 558.823 650.458 332.341 849.113
  3143. 421.825 547.83 559.122 470.665 1008.839 485.936 502.9
  3144. 316.743 374.998 1162.146 0. 653.336 507.322]
  3145. [ 268.19 353.682 523.165 531.04 590.289 482.2 830.227
  3146. 400.695 727.771 471.671 554.175 734.617 555.236 433.176
  3147. 725.177 452.026 929.612 653.336 0. 464.526]
  3148. [ 476.334 649.573 338.337 487.92 429.449 413.512 748.972
  3149. 517.317 662.709 240.881 488.203 842.682 445.942 397.007
  3150. 554.775 475.044 997.861 507.322 464.526 0. ]]
  3151. 2019-03-12 21:34:28,882 : epoch 341 , step 0 loss0= 0.0000 loss1= 2.8076 loss2=-3775.8723 lr= 0.000100
  3152. 2019-03-12 21:35:04,379 : epoch 341 , step 40 loss0= 0.0000 loss1= 2.1070 loss2=-3627.2537 lr= 0.000100
  3153. 2019-03-12 21:35:39,856 : epoch 341 , step 80 loss0= 0.0000 loss1= 2.5300 loss2=-3744.6978 lr= 0.000100
  3154. 2019-03-12 21:36:15,398 : epoch 341 , step 120 loss0= 0.0000 loss1= 2.9869 loss2=-3691.4863 lr= 0.000100
  3155. 2019-03-12 21:36:51,584 : epoch 341 , step 160 loss0= 0.0000 loss1= 2.0958 loss2=-3793.2654 lr= 0.000100
  3156. 2019-03-12 21:37:24,105 : epoch 342 , step 0 loss0= 0.0000 loss1= 2.9665 loss2=-3522.5291 lr= 0.000100
  3157. 2019-03-12 21:37:59,725 : epoch 342 , step 40 loss0= 0.0000 loss1= 3.2546 loss2=-3507.1335 lr= 0.000100
  3158. 2019-03-12 21:38:35,828 : epoch 342 , step 80 loss0= 0.0000 loss1= 2.4928 loss2=-3744.0437 lr= 0.000100
  3159. 2019-03-12 21:39:12,290 : epoch 342 , step 120 loss0= 0.0000 loss1= 2.0887 loss2=-3906.4968 lr= 0.000100
  3160. 2019-03-12 21:39:48,590 : epoch 342 , step 160 loss0= 0.0000 loss1= 1.8697 loss2=-3649.1399 lr= 0.000100
  3161. 2019-03-12 21:40:21,297 : epoch 343 , step 0 loss0= 0.0000 loss1= 1.8419 loss2=-3578.5801 lr= 0.000100
  3162. 2019-03-12 21:40:57,313 : epoch 343 , step 40 loss0= 0.0000 loss1= 2.0431 loss2=-3611.2915 lr= 0.000100
  3163. 2019-03-12 21:41:33,335 : epoch 343 , step 80 loss0= 0.0000 loss1= 1.4780 loss2=-3524.5083 lr= 0.000100
  3164. 2019-03-12 21:42:08,766 : epoch 343 , step 120 loss0= 0.0000 loss1= 1.8749 loss2=-3712.9910 lr= 0.000100
  3165. 2019-03-12 21:42:44,772 : epoch 343 , step 160 loss0= 0.0000 loss1= 1.9829 loss2=-3461.6802 lr= 0.000100
  3166. 2019-03-12 21:43:17,332 : epoch 344 , step 0 loss0= 0.0000 loss1= 2.1032 loss2=-3661.8835 lr= 0.000100
  3167. 2019-03-12 21:43:52,814 : epoch 344 , step 40 loss0= 0.0000 loss1= 1.9239 loss2=-3565.7715 lr= 0.000100
  3168. 2019-03-12 21:44:28,420 : epoch 344 , step 80 loss0= 0.0000 loss1= 2.7511 loss2=-3809.8145 lr= 0.000100
  3169. 2019-03-12 21:45:04,350 : epoch 344 , step 120 loss0= 0.0000 loss1= 2.9312 loss2=-3699.7927 lr= 0.000100
  3170. 2019-03-12 21:45:40,321 : epoch 344 , step 160 loss0= 0.0000 loss1= 2.6989 loss2=-3788.7871 lr= 0.000100
  3171. 2019-03-12 21:46:12,836 : epoch 345 , step 0 loss0= 0.0000 loss1= 2.9326 loss2=-3519.0273 lr= 0.000100
  3172. 2019-03-12 21:46:49,281 : epoch 345 , step 40 loss0= 0.0000 loss1= 2.7719 loss2=-3631.7688 lr= 0.000100
  3173. 2019-03-12 21:47:25,140 : epoch 345 , step 80 loss0= 0.0000 loss1= 2.9767 loss2=-3700.7532 lr= 0.000100
  3174. 2019-03-12 21:48:01,079 : epoch 345 , step 120 loss0= 0.0000 loss1= 1.8835 loss2=-3836.5603 lr= 0.000100
  3175. 2019-03-12 21:48:36,661 : epoch 345 , step 160 loss0= 0.0000 loss1= 2.5231 loss2=-3770.0156 lr= 0.000100
  3176. 2019-03-12 21:49:08,895 : epoch 346 , step 0 loss0= 0.0000 loss1= 2.0451 loss2=-3563.7610 lr= 0.000100
  3177. 2019-03-12 21:49:45,333 : epoch 346 , step 40 loss0= 0.0000 loss1= 2.0784 loss2=-3678.5779 lr= 0.000100
  3178. 2019-03-12 21:50:20,850 : epoch 346 , step 80 loss0= 0.0000 loss1= 2.1966 loss2=-3882.9778 lr= 0.000100
  3179. 2019-03-12 21:50:56,621 : epoch 346 , step 120 loss0= 0.0000 loss1= 1.8026 loss2=-3482.6731 lr= 0.000100
  3180. 2019-03-12 21:51:31,778 : epoch 346 , step 160 loss0= 0.0000 loss1= 2.0947 loss2=-3449.7576 lr= 0.000100
  3181. 2019-03-12 21:52:04,032 : epoch 347 , step 0 loss0= 0.0000 loss1= 1.9783 loss2=-3662.4492 lr= 0.000100
  3182. 2019-03-12 21:52:39,714 : epoch 347 , step 40 loss0= 0.0000 loss1= 1.8393 loss2=-3503.9478 lr= 0.000100
  3183. 2019-03-12 21:53:15,259 : epoch 347 , step 80 loss0= 0.0000 loss1= 3.2473 loss2=-4127.8066 lr= 0.000100
  3184. 2019-03-12 21:53:50,566 : epoch 347 , step 120 loss0= 0.0000 loss1= 1.7776 loss2=-3487.3911 lr= 0.000100
  3185. 2019-03-12 21:54:26,646 : epoch 347 , step 160 loss0= 0.0000 loss1= 2.0492 loss2=-3725.2427 lr= 0.000100
  3186. 2019-03-12 21:54:58,833 : epoch 348 , step 0 loss0= 0.0000 loss1= 2.0654 loss2=-3610.9883 lr= 0.000100
  3187. 2019-03-12 21:55:34,577 : epoch 348 , step 40 loss0= 0.0000 loss1= 1.4796 loss2=-3527.5391 lr= 0.000100
  3188. 2019-03-12 21:56:10,320 : epoch 348 , step 80 loss0= 0.0000 loss1= 2.4942 loss2=-3574.9302 lr= 0.000100
  3189. 2019-03-12 21:56:45,795 : epoch 348 , step 120 loss0= 0.0000 loss1= 2.7176 loss2=-3559.1152 lr= 0.000100
  3190. 2019-03-12 21:57:21,183 : epoch 348 , step 160 loss0= 0.0000 loss1= 2.7071 loss2=-3851.9966 lr= 0.000100
  3191. 2019-03-12 21:57:53,405 : epoch 349 , step 0 loss0= 0.0000 loss1= 2.1067 loss2=-3661.2122 lr= 0.000100
  3192. 2019-03-12 21:58:29,009 : epoch 349 , step 40 loss0= 0.0000 loss1= 2.7383 loss2=-3980.3782 lr= 0.000100
  3193. 2019-03-12 21:59:04,309 : epoch 349 , step 80 loss0= 0.0000 loss1= 2.3896 loss2=-3763.2102 lr= 0.000100
  3194. 2019-03-12 21:59:39,905 : epoch 349 , step 120 loss0= 0.0000 loss1= 3.3005 loss2=-3779.8298 lr= 0.000100
  3195. 2019-03-12 22:00:15,697 : epoch 349 , step 160 loss0= 0.0000 loss1= 3.0085 loss2=-3635.4797 lr= 0.000100
  3196. 2019-03-12 22:00:48,073 : epoch 350 , step 0 loss0= 0.0000 loss1= 3.0734 loss2=-3655.4509 lr= 0.000100
  3197. 2019-03-12 22:01:23,338 : epoch 350 , step 40 loss0= 0.0000 loss1= 2.0279 loss2=-3678.2615 lr= 0.000100
  3198. 2019-03-12 22:01:58,632 : epoch 350 , step 80 loss0= 0.0000 loss1= 2.4419 loss2=-3755.2207 lr= 0.000100
  3199. 2019-03-12 22:02:34,374 : epoch 350 , step 120 loss0= 0.0000 loss1= 1.3201 loss2=-3158.6145 lr= 0.000100
  3200. 2019-03-12 22:03:09,940 : epoch 350 , step 160 loss0= 0.0000 loss1= 2.7871 loss2=-3819.8274 lr= 0.000100
  3201. 2019-03-12 22:03:42,707 : epoch 351 , step 0 loss0= 0.0000 loss1= 1.9289 loss2=-2985.3154 lr= 0.000100
  3202. 2019-03-12 22:04:18,409 : epoch 351 , step 40 loss0= 0.0000 loss1= 2.5820 loss2=-3591.1589 lr= 0.000100
  3203. 2019-03-12 22:04:53,825 : epoch 351 , step 80 loss0= 0.0000 loss1= 3.0626 loss2=-3765.4531 lr= 0.000100
  3204. 2019-03-12 22:05:29,266 : epoch 351 , step 120 loss0= 0.0000 loss1= 2.2214 loss2=-3281.8579 lr= 0.000100
  3205. 2019-03-12 22:06:04,819 : epoch 351 , step 160 loss0= 0.0000 loss1= 2.5185 loss2=-3444.8669 lr= 0.000100
  3206. 2019-03-12 22:06:37,020 : epoch 352 , step 0 loss0= 0.0000 loss1= 1.8610 loss2=-3447.7468 lr= 0.000100
  3207. 2019-03-12 22:07:12,744 : epoch 352 , step 40 loss0= 0.0000 loss1= 2.2974 loss2=-3862.0161 lr= 0.000100
  3208. 2019-03-12 22:07:48,444 : epoch 352 , step 80 loss0= 0.0000 loss1= 2.9375 loss2=-3761.2739 lr= 0.000100
  3209. 2019-03-12 22:08:24,096 : epoch 352 , step 120 loss0= 0.0000 loss1= 3.6205 loss2=-3845.5681 lr= 0.000100
  3210. 2019-03-12 22:08:59,912 : epoch 352 , step 160 loss0= 0.0000 loss1= 2.9677 loss2=-3865.7703 lr= 0.000100
  3211. 2019-03-12 22:09:32,081 : epoch 353 , step 0 loss0= 0.0000 loss1= 1.7558 loss2=-2846.1326 lr= 0.000100
  3212. 2019-03-12 22:10:07,808 : epoch 353 , step 40 loss0= 0.0000 loss1= 2.7772 loss2=-3806.9590 lr= 0.000100
  3213. 2019-03-12 22:10:43,081 : epoch 353 , step 80 loss0= 0.0000 loss1= 1.7518 loss2=-3298.3630 lr= 0.000100
  3214. 2019-03-12 22:11:18,882 : epoch 353 , step 120 loss0= 0.0000 loss1= 2.6827 loss2=-3590.8110 lr= 0.000100
  3215. 2019-03-12 22:11:54,567 : epoch 353 , step 160 loss0= 0.0000 loss1= 2.8271 loss2=-3856.9133 lr= 0.000100
  3216. 2019-03-12 22:12:27,117 : epoch 354 , step 0 loss0= 0.0000 loss1= 2.2981 loss2=-3548.5095 lr= 0.000100
  3217. 2019-03-12 22:13:03,200 : epoch 354 , step 40 loss0= 0.0000 loss1= 1.7654 loss2=-3620.7583 lr= 0.000100
  3218. 2019-03-12 22:13:39,329 : epoch 354 , step 80 loss0= 0.0000 loss1= 2.6829 loss2=-3894.9382 lr= 0.000100
  3219. 2019-03-12 22:14:15,118 : epoch 354 , step 120 loss0= 0.0000 loss1= 1.8939 loss2=-3732.2273 lr= 0.000100
  3220. 2019-03-12 22:14:50,725 : epoch 354 , step 160 loss0= 0.0000 loss1= 2.6258 loss2=-3666.3867 lr= 0.000100
  3221. 2019-03-12 22:15:22,979 : epoch 355 , step 0 loss0= 0.0000 loss1= 2.7411 loss2=-3850.6208 lr= 0.000100
  3222. 2019-03-12 22:15:59,014 : epoch 355 , step 40 loss0= 0.0000 loss1= 1.9544 loss2=-3697.8796 lr= 0.000100
  3223. 2019-03-12 22:16:34,486 : epoch 355 , step 80 loss0= 0.0000 loss1= 2.6984 loss2=-3725.2961 lr= 0.000100
  3224. 2019-03-12 22:17:09,942 : epoch 355 , step 120 loss0= 0.0000 loss1= 2.6803 loss2=-3825.1743 lr= 0.000100
  3225. 2019-03-12 22:17:45,752 : epoch 355 , step 160 loss0= 0.0000 loss1= 2.1069 loss2=-3729.9160 lr= 0.000100
  3226. 2019-03-12 22:18:17,870 : epoch 356 , step 0 loss0= 0.0000 loss1= 1.5900 loss2=-3426.4341 lr= 0.000100
  3227. 2019-03-12 22:18:53,554 : epoch 356 , step 40 loss0= 0.0000 loss1= 2.3310 loss2=-3646.8970 lr= 0.000100
  3228. 2019-03-12 22:19:29,024 : epoch 356 , step 80 loss0= 0.0000 loss1= 1.9054 loss2=-3481.2341 lr= 0.000100
  3229. 2019-03-12 22:20:04,334 : epoch 356 , step 120 loss0= 0.0000 loss1= 1.9540 loss2=-3618.6348 lr= 0.000100
  3230. 2019-03-12 22:20:39,972 : epoch 356 , step 160 loss0= 0.0000 loss1= 3.4224 loss2=-3672.9641 lr= 0.000100
  3231. 2019-03-12 22:21:12,164 : epoch 357 , step 0 loss0= 0.0000 loss1= 2.2525 loss2=-3737.9927 lr= 0.000100
  3232. 2019-03-12 22:21:47,492 : epoch 357 , step 40 loss0= 0.0000 loss1= 2.7056 loss2=-3855.7283 lr= 0.000100
  3233. 2019-03-12 22:22:23,294 : epoch 357 , step 80 loss0= 0.0000 loss1= 2.5810 loss2=-3650.6814 lr= 0.000100
  3234. 2019-03-12 22:22:59,898 : epoch 357 , step 120 loss0= 0.0000 loss1= 2.0956 loss2=-3836.0767 lr= 0.000100
  3235. 2019-03-12 22:23:36,453 : epoch 357 , step 160 loss0= 0.0000 loss1= 2.9439 loss2=-3921.8872 lr= 0.000100
  3236. 2019-03-12 22:24:09,240 : epoch 358 , step 0 loss0= 0.0000 loss1= 2.6984 loss2=-3936.5710 lr= 0.000100
  3237. 2019-03-12 22:24:45,137 : epoch 358 , step 40 loss0= 0.0000 loss1= 2.2661 loss2=-3766.4016 lr= 0.000100
  3238. 2019-03-12 22:25:20,515 : epoch 358 , step 80 loss0= 0.0000 loss1= 2.6231 loss2=-3951.1445 lr= 0.000100
  3239. 2019-03-12 22:25:56,394 : epoch 358 , step 120 loss0= 0.0000 loss1= 2.9256 loss2=-3814.4998 lr= 0.000100
  3240. 2019-03-12 22:26:32,590 : epoch 358 , step 160 loss0= 0.0000 loss1= 2.7808 loss2=-4090.8293 lr= 0.000100
  3241. 2019-03-12 22:27:05,505 : epoch 359 , step 0 loss0= 0.0000 loss1= 1.8618 loss2=-3498.2207 lr= 0.000100
  3242. 2019-03-12 22:27:41,206 : epoch 359 , step 40 loss0= 0.0000 loss1= 2.0018 loss2=-3587.3884 lr= 0.000100
  3243. 2019-03-12 22:28:16,874 : epoch 359 , step 80 loss0= 0.0000 loss1= 1.7723 loss2=-3316.2349 lr= 0.000100
  3244. 2019-03-12 22:28:52,593 : epoch 359 , step 120 loss0= 0.0000 loss1= 2.4127 loss2=-3608.6062 lr= 0.000100
  3245. 2019-03-12 22:29:28,790 : epoch 359 , step 160 loss0= 0.0000 loss1= 2.2581 loss2=-4033.1255 lr= 0.000100
  3246. 2019-03-12 22:30:02,065 : epoch 360 , step 0 loss0= 0.0000 loss1= 2.7273 loss2=-3679.3840 lr= 0.000100
  3247. 2019-03-12 22:30:38,339 : epoch 360 , step 40 loss0= 0.0000 loss1= 2.6506 loss2=-3599.7078 lr= 0.000100
  3248. 2019-03-12 22:31:14,580 : epoch 360 , step 80 loss0= 0.0000 loss1= 2.5961 loss2=-3810.3867 lr= 0.000100
  3249. 2019-03-12 22:31:50,894 : epoch 360 , step 120 loss0= 0.0000 loss1= 2.6541 loss2=-3743.4758 lr= 0.000100
  3250. 2019-03-12 22:32:26,983 : epoch 360 , step 160 loss0= 0.0000 loss1= 1.8382 loss2=-3635.6882 lr= 0.000100
  3251. 2019-03-12 22:32:57,988 : ## train ##
  3252. 2019-03-12 22:37:52,469 : >> epoch 360 : average_loss= 0.8125, average_accuracy=82.302%, mean_weight_std= 0.3418, mean_normalized_std= 0.4692 mean_dist=2.94e+03
  3253. 2019-03-12 22:37:52,477 : >> epoch 360 : median_loss= 0.7880, median_accuracy=83.670%, median_weight_std= 0.2009, median_normalized_std= 0.0918 median_dist=2.909e+03
  3254. 2019-03-12 22:37:52,477 : >> epoch 360 : ensemble accuracy: 85.26%
  3255. 2019-03-12 22:37:52,478 : >> epoch 360 : loss histogram: (0.00,0.34, 18),(0.34,0.68, 0),(0.68,1.03, 4),(1.03,1.37, 0),(1.37,1.71, 18),
  3256. 2019-03-12 22:37:52,478 : >> epoch 360 : accuracy histogram: (0.00,33.61, 18),(33.61,67.22, 0),(67.22,100.83, 4),(100.83,134.44, 0),(134.44,168.05, 18),
  3257. 2019-03-12 22:37:52,484 : >> epoch 360 : weight std histogram: (0.0000,2.8433, 463695),(2.8433,5.6865, 2832),(5.6865,8.5298, 321),(8.5298,11.3731, 52),(11.3731,14.2163, 6),
  3258. 2019-03-12 22:37:52,489 : >> epoch 360 : weight normalized std histogram: (0.0000,1300.3690, 466896),(1300.3690,2600.7381, 5),(2600.7381,3901.1071, 2),(3901.1071,5201.4762, 1),(5201.4762,6501.8452, 2),
  3259. 2019-03-12 22:37:52,490 : >> epoch 360 : dist histogram: (2609.8345,2747.0021, 2),(2747.0021,2884.1696, 7),(2884.1696,3021.3372, 4),(3021.3372,3158.5048, 5),(3158.5048,3295.6724, 2),
  3260. 2019-03-12 22:37:52,490 : ## eval ##
  3261. 2019-03-12 22:42:44,388 : >> epoch 360 : average_loss= 0.7028, average_accuracy=85.891%, mean_weight_std= 0.3353, mean_normalized_std= 0.5597 mean_dist=2.941e+03
  3262. 2019-03-12 22:42:44,397 : >> epoch 360 : median_loss= 0.6887, median_accuracy=86.875%, median_weight_std= 0.1948, median_normalized_std= 0.0890 median_dist=2.954e+03
  3263. 2019-03-12 22:42:44,397 : >> epoch 360 : ensemble accuracy: 88.59%
  3264. 2019-03-12 22:42:44,397 : >> epoch 360 : loss histogram: (0.00,0.30, 18),(0.30,0.60, 0),(0.60,0.91, 4),(0.91,1.21, 0),(1.21,1.51, 18),
  3265. 2019-03-12 22:42:44,398 : >> epoch 360 : accuracy histogram: (0.00,34.96, 18),(34.96,69.91, 0),(69.91,104.87, 4),(104.87,139.82, 0),(139.82,174.78, 18),
  3266. 2019-03-12 22:42:44,409 : >> epoch 360 : weight std histogram: (0.0000,3.1676, 464413),(3.1676,6.3351, 2234),(6.3351,9.5027, 230),(9.5027,12.6703, 27),(12.6703,15.8378, 2),
  3267. 2019-03-12 22:42:44,414 : >> epoch 360 : weight normalized std histogram: (0.0000,3468.6895, 466902),(3468.6895,6937.3789, 0),(6937.3789,10406.0684, 2),(10406.0684,13874.7578, 1),(13874.7578,17343.4473, 1),
  3268. 2019-03-12 22:42:44,415 : >> epoch 360 : dist histogram: (2720.9929,2832.0394, 7),(2832.0394,2943.0859, 2),(2943.0859,3054.1324, 8),(3054.1324,3165.1789, 0),(3165.1789,3276.2253, 3),
  3269. 2019-03-12 22:42:44,470 : >> epoch 360 : dist pair-wise
  3270. 2019-03-12 22:42:44,476 :
  3271. [[ 0. 930.47 472.178 293.349 513.913 443.842 507.124 509.327
  3272. 721.131 401.311 250.625 565.383 881.464 419.099 559.582 307.566
  3273. 367.262 689.75 432.893 425.101]
  3274. [ 930.47 0. 809.885 940.77 805.98 717.074 971.338 767.192
  3275. 783.317 895.384 932.166 886.404 905.957 738.268 833.964 951.914
  3276. 832.938 792.843 761.65 818.124]
  3277. [ 472.178 809.885 0. 512.189 328.347 556.024 637.253 313.325
  3278. 600.113 490.334 426.011 492.545 746.528 260.819 290.509 358.263
  3279. 510.889 578.628 396.444 324.102]
  3280. [ 293.349 940.77 512.189 0. 455.932 425.5 388.418 531.018
  3281. 703.046 287.58 269.514 607.499 778.369 488.067 528.502 359.844
  3282. 342.551 729.122 513.639 513.913]
  3283. [ 513.913 805.98 328.347 455.932 0. 569.018 603.062 290.91
  3284. 523.114 364.437 423.353 539.649 522.545 387.673 370.147 435.58
  3285. 510.307 586.224 428.581 401.019]
  3286. [ 443.842 717.074 556.024 425.5 569.018 0. 476.907 549.503
  3287. 680.094 473.346 452.101 698.833 862.739 492.155 561.614 545.17
  3288. 334.345 748.637 578.446 584.456]
  3289. [ 507.124 971.338 637.253 388.418 603.062 476.907 0. 632.013
  3290. 740.359 402.729 432.244 712.514 818.318 653.496 575.844 508.971
  3291. 443.059 809.782 738.222 703.149]
  3292. [ 509.327 767.192 313.325 531.018 290.91 549.503 632.013 0.
  3293. 405.097 422.106 431.023 513.151 590.308 384.11 357.208 430.57
  3294. 460.522 539.046 465.381 414.276]
  3295. [ 721.131 783.317 600.113 703.046 523.114 680.094 740.359 405.097
  3296. 0. 614.194 650.963 568.713 535.557 645.33 554.24 633.512
  3297. 590.908 500.749 687.556 684.042]
  3298. [ 401.311 895.384 490.334 287.58 364.437 473.346 402.729 422.106
  3299. 614.194 0. 288.755 634.202 641.12 497.008 472.694 393.183
  3300. 364.686 726.529 552.047 526.609]
  3301. [ 250.625 932.166 426.011 269.514 423.353 452.101 432.244 431.023
  3302. 650.963 288.755 0. 549.595 766.812 421.288 468.255 263.708
  3303. 369.099 684.329 469.033 427.706]
  3304. [ 565.383 886.404 492.545 607.499 539.649 698.833 712.514 513.151
  3305. 568.713 634.202 549.595 0. 732.383 482.662 513.061 471.782
  3306. 621.692 285.359 490.962 498.339]
  3307. [ 881.464 905.957 746.528 778.369 522.545 862.739 818.318 590.308
  3308. 535.557 641.12 766.812 732.383 0. 794.915 680.98 799.281
  3309. 792.727 677.329 801.004 793.488]
  3310. [ 419.099 738.268 260.819 488.067 387.673 492.155 653.496 384.11
  3311. 645.33 497.008 421.288 482.662 794.915 0. 376.905 392.362
  3312. 470.785 570.228 313.319 330.528]
  3313. [ 559.582 833.964 290.509 528.502 370.147 561.614 575.844 357.208
  3314. 554.24 472.694 468.255 513.061 680.98 376.905 0. 395.787
  3315. 496.356 614.85 573.304 532.449]
  3316. [ 307.566 951.914 358.263 359.844 435.58 545.17 508.971 430.57
  3317. 633.512 393.183 263.708 471.782 799.281 392.362 395.787 0.
  3318. 431.539 628.849 478.098 435.702]
  3319. [ 367.262 832.938 510.889 342.551 510.307 334.345 443.059 460.522
  3320. 590.908 364.686 369.099 621.692 792.727 470.785 496.356 431.539
  3321. 0. 701.962 578.396 576.934]
  3322. [ 689.75 792.843 578.628 729.122 586.224 748.637 809.782 539.046
  3323. 500.749 726.529 684.329 285.359 677.329 570.228 614.85 628.849
  3324. 701.962 0. 547.457 568.915]
  3325. [ 432.893 761.65 396.444 513.639 428.581 578.446 738.222 465.381
  3326. 687.556 552.047 469.033 490.962 801.004 313.319 573.304 478.098
  3327. 578.396 547.457 0. 248.484]
  3328. [ 425.101 818.124 324.102 513.913 401.019 584.456 703.149 414.276
  3329. 684.042 526.609 427.706 498.339 793.488 330.528 532.449 435.702
  3330. 576.934 568.915 248.484 0. ]]
  3331. 2019-03-12 22:42:46,003 : epoch 361 , step 0 loss0= 0.0000 loss1= 2.9391 loss2=-3765.0125 lr= 0.000100
  3332. 2019-03-12 22:43:21,469 : epoch 361 , step 40 loss0= 0.0000 loss1= 3.0618 loss2=-3506.3828 lr= 0.000100
  3333. 2019-03-12 22:43:56,884 : epoch 361 , step 80 loss0= 0.0000 loss1= 1.7703 loss2=-3665.1189 lr= 0.000100
  3334. 2019-03-12 22:44:32,639 : epoch 361 , step 120 loss0= 0.0000 loss1= 1.7267 loss2=-3641.2415 lr= 0.000100
  3335. 2019-03-12 22:45:08,826 : epoch 361 , step 160 loss0= 0.0000 loss1= 2.6641 loss2=-3899.4204 lr= 0.000100
  3336. 2019-03-12 22:45:41,257 : epoch 362 , step 0 loss0= 0.0000 loss1= 1.8476 loss2=-3454.2126 lr= 0.000100
  3337. 2019-03-12 22:46:17,324 : epoch 362 , step 40 loss0= 0.0000 loss1= 3.1280 loss2=-3668.9958 lr= 0.000100
  3338. 2019-03-12 22:46:53,401 : epoch 362 , step 80 loss0= 0.0000 loss1= 2.3810 loss2=-3926.8853 lr= 0.000100
  3339. 2019-03-12 22:47:28,799 : epoch 362 , step 120 loss0= 0.0000 loss1= 2.5222 loss2=-3906.3298 lr= 0.000100
  3340. 2019-03-12 22:48:04,997 : epoch 362 , step 160 loss0= 0.0000 loss1= 2.0068 loss2=-3696.6340 lr= 0.000100
  3341. 2019-03-12 22:48:38,300 : epoch 363 , step 0 loss0= 0.0000 loss1= 2.1960 loss2=-3657.0115 lr= 0.000100
  3342. 2019-03-12 22:49:14,361 : epoch 363 , step 40 loss0= 0.0000 loss1= 1.2480 loss2=-3078.1685 lr= 0.000100
  3343. 2019-03-12 22:49:50,177 : epoch 363 , step 80 loss0= 0.0000 loss1= 1.8461 loss2=-3567.7400 lr= 0.000100
  3344. 2019-03-12 22:50:26,041 : epoch 363 , step 120 loss0= 0.0000 loss1= 1.5472 loss2=-3329.6821 lr= 0.000100
  3345. 2019-03-12 22:51:01,818 : epoch 363 , step 160 loss0= 0.0000 loss1= 2.1677 loss2=-3300.5715 lr= 0.000100
  3346. 2019-03-12 22:51:34,909 : epoch 364 , step 0 loss0= 0.0000 loss1= 2.1239 loss2=-3271.7859 lr= 0.000100
  3347. 2019-03-12 22:52:11,419 : epoch 364 , step 40 loss0= 0.0000 loss1= 2.3295 loss2=-3791.3796 lr= 0.000100
  3348. 2019-03-12 22:52:48,177 : epoch 364 , step 80 loss0= 0.0000 loss1= 2.8107 loss2=-3653.8262 lr= 0.000100
  3349. 2019-03-12 22:53:25,057 : epoch 364 , step 120 loss0= 0.0000 loss1= 1.6555 loss2=-3635.0884 lr= 0.000100
  3350. 2019-03-12 22:54:01,376 : epoch 364 , step 160 loss0= 0.0000 loss1= 2.7821 loss2=-3828.2537 lr= 0.000100
  3351. 2019-03-12 22:54:34,301 : epoch 365 , step 0 loss0= 0.0000 loss1= 2.2061 loss2=-3858.7478 lr= 0.000100
  3352. 2019-03-12 22:55:10,250 : epoch 365 , step 40 loss0= 0.0000 loss1= 3.9515 loss2=-3912.2485 lr= 0.000100
  3353. 2019-03-12 22:55:46,090 : epoch 365 , step 80 loss0= 0.0000 loss1= 2.5327 loss2=-4016.8450 lr= 0.000100
  3354. 2019-03-12 22:56:21,666 : epoch 365 , step 120 loss0= 0.0000 loss1= 1.8194 loss2=-3554.2954 lr= 0.000100
  3355. 2019-03-12 22:56:57,128 : epoch 365 , step 160 loss0= 0.0000 loss1= 2.1478 loss2=-3881.8242 lr= 0.000100
  3356. 2019-03-12 22:57:29,399 : epoch 366 , step 0 loss0= 0.0000 loss1= 2.0183 loss2=-3519.6704 lr= 0.000100
  3357. 2019-03-12 22:58:05,367 : epoch 366 , step 40 loss0= 0.0000 loss1= 1.8474 loss2=-3487.8555 lr= 0.000100
  3358. 2019-03-12 22:58:41,211 : epoch 366 , step 80 loss0= 0.0000 loss1= 1.9310 loss2=-3875.7556 lr= 0.000100
  3359. 2019-03-12 22:59:16,912 : epoch 366 , step 120 loss0= 0.0000 loss1= 3.1585 loss2=-3848.5344 lr= 0.000100
  3360. 2019-03-12 22:59:52,859 : epoch 366 , step 160 loss0= 0.0000 loss1= 2.6807 loss2=-3619.8735 lr= 0.000100
  3361. 2019-03-12 23:00:25,584 : epoch 367 , step 0 loss0= 0.0000 loss1= 2.2966 loss2=-3549.1829 lr= 0.000100
  3362. 2019-03-12 23:01:01,480 : epoch 367 , step 40 loss0= 0.0000 loss1= 2.1416 loss2=-3868.7410 lr= 0.000100
  3363. 2019-03-12 23:01:37,003 : epoch 367 , step 80 loss0= 0.0000 loss1= 2.9549 loss2=-3790.0063 lr= 0.000100
  3364. 2019-03-12 23:02:13,029 : epoch 367 , step 120 loss0= 0.0000 loss1= 2.0886 loss2=-3938.8579 lr= 0.000100
  3365. 2019-03-12 23:02:49,572 : epoch 367 , step 160 loss0= 0.0000 loss1= 2.9442 loss2=-3906.9314 lr= 0.000100
  3366. 2019-03-12 23:03:22,378 : epoch 368 , step 0 loss0= 0.0000 loss1= 2.3570 loss2=-3839.3093 lr= 0.000100
  3367. 2019-03-12 23:03:58,358 : epoch 368 , step 40 loss0= 0.0000 loss1= 2.4329 loss2=-3635.8572 lr= 0.000100
  3368. 2019-03-12 23:04:34,739 : epoch 368 , step 80 loss0= 0.0000 loss1= 1.9558 loss2=-3658.8318 lr= 0.000100
  3369. 2019-03-12 23:05:10,600 : epoch 368 , step 120 loss0= 0.0000 loss1= 1.7201 loss2=-3629.8474 lr= 0.000100
  3370. 2019-03-12 23:05:46,038 : epoch 368 , step 160 loss0= 0.0000 loss1= 2.2573 loss2=-3709.3894 lr= 0.000100
  3371. 2019-03-12 23:06:18,446 : epoch 369 , step 0 loss0= 0.0000 loss1= 1.8684 loss2=-3603.0007 lr= 0.000100
  3372. 2019-03-12 23:06:54,903 : epoch 369 , step 40 loss0= 0.0000 loss1= 3.0850 loss2=-3675.4055 lr= 0.000100
  3373. 2019-03-12 23:07:30,689 : epoch 369 , step 80 loss0= 0.0000 loss1= 3.1140 loss2=-3742.3552 lr= 0.000100
  3374. 2019-03-12 23:08:06,498 : epoch 369 , step 120 loss0= 0.0000 loss1= 2.2954 loss2=-3800.7825 lr= 0.000100
  3375. 2019-03-12 23:08:42,548 : epoch 369 , step 160 loss0= 0.0000 loss1= 2.0552 loss2=-3633.0430 lr= 0.000100
  3376. 2019-03-12 23:09:14,992 : epoch 370 , step 0 loss0= 0.0000 loss1= 2.4283 loss2=-3400.8918 lr= 0.000100
  3377. 2019-03-12 23:09:50,431 : epoch 370 , step 40 loss0= 0.0000 loss1= 2.1040 loss2=-3454.8403 lr= 0.000100
  3378. 2019-03-12 23:10:25,786 : epoch 370 , step 80 loss0= 0.0000 loss1= 1.9171 loss2=-3542.6074 lr= 0.000100
  3379. 2019-03-12 23:11:01,234 : epoch 370 , step 120 loss0= 0.0000 loss1= 2.9211 loss2=-3751.3430 lr= 0.000100
  3380. 2019-03-12 23:11:36,890 : epoch 370 , step 160 loss0= 0.0000 loss1= 2.8009 loss2=-3333.7649 lr= 0.000100
  3381. 2019-03-12 23:12:09,438 : epoch 371 , step 0 loss0= 0.0000 loss1= 1.9161 loss2=-3717.7615 lr= 0.000100
  3382. 2019-03-12 23:12:45,736 : epoch 371 , step 40 loss0= 0.0000 loss1= 1.9552 loss2=-3752.4458 lr= 0.000100
  3383. 2019-03-12 23:13:22,083 : epoch 371 , step 80 loss0= 0.0000 loss1= 3.0257 loss2=-3508.5168 lr= 0.000100
  3384. 2019-03-12 23:13:58,608 : epoch 371 , step 120 loss0= 0.0000 loss1= 1.8954 loss2=-3638.9705 lr= 0.000100
  3385. 2019-03-12 23:14:34,308 : epoch 371 , step 160 loss0= 0.0000 loss1= 2.3799 loss2=-3567.2349 lr= 0.000100
  3386. 2019-03-12 23:15:06,761 : epoch 372 , step 0 loss0= 0.0000 loss1= 1.9446 loss2=-3189.4829 lr= 0.000100
  3387. 2019-03-12 23:15:42,702 : epoch 372 , step 40 loss0= 0.0000 loss1= 2.0056 loss2=-3140.9783 lr= 0.000100
  3388. 2019-03-12 23:16:18,694 : epoch 372 , step 80 loss0= 0.0000 loss1= 2.6334 loss2=-3894.0273 lr= 0.000100
  3389. 2019-03-12 23:16:54,990 : epoch 372 , step 120 loss0= 0.0000 loss1= 2.2439 loss2=-3805.2957 lr= 0.000100
  3390. 2019-03-12 23:17:31,305 : epoch 372 , step 160 loss0= 0.0000 loss1= 2.3789 loss2=-3731.3501 lr= 0.000100
  3391. 2019-03-12 23:18:03,663 : epoch 373 , step 0 loss0= 0.0000 loss1= 1.7563 loss2=-3511.4727 lr= 0.000100
  3392. 2019-03-12 23:18:39,927 : epoch 373 , step 40 loss0= 0.0000 loss1= 2.4723 loss2=-3645.6724 lr= 0.000100
  3393. 2019-03-12 23:19:16,706 : epoch 373 , step 80 loss0= 0.0000 loss1= 1.7347 loss2=-3457.1250 lr= 0.000100
  3394. 2019-03-12 23:19:52,582 : epoch 373 , step 120 loss0= 0.0000 loss1= 2.5803 loss2=-3338.9146 lr= 0.000100
  3395. 2019-03-12 23:20:28,310 : epoch 373 , step 160 loss0= 0.0000 loss1= 1.8080 loss2=-3662.2441 lr= 0.000100
  3396. 2019-03-12 23:21:00,772 : epoch 374 , step 0 loss0= 0.0000 loss1= 2.7741 loss2=-3870.8132 lr= 0.000100
  3397. 2019-03-12 23:21:36,357 : epoch 374 , step 40 loss0= 0.0000 loss1= 2.3982 loss2=-3470.9707 lr= 0.000100
  3398. 2019-03-12 23:22:11,919 : epoch 374 , step 80 loss0= 0.0000 loss1= 2.1935 loss2=-3730.6606 lr= 0.000100
  3399. 2019-03-12 23:22:47,499 : epoch 374 , step 120 loss0= 0.0000 loss1= 2.3477 loss2=-3660.1235 lr= 0.000100
  3400. 2019-03-12 23:23:23,143 : epoch 374 , step 160 loss0= 0.0000 loss1= 1.8656 loss2=-3423.1653 lr= 0.000100
  3401. 2019-03-12 23:23:55,611 : epoch 375 , step 0 loss0= 0.0000 loss1= 2.2375 loss2=-3715.7739 lr= 0.000100
  3402. 2019-03-12 23:24:31,703 : epoch 375 , step 40 loss0= 0.0000 loss1= 1.5323 loss2=-3391.8696 lr= 0.000100
  3403. 2019-03-12 23:25:07,460 : epoch 375 , step 80 loss0= 0.0000 loss1= 2.0948 loss2=-3669.7922 lr= 0.000100
  3404. 2019-03-12 23:25:43,382 : epoch 375 , step 120 loss0= 0.0000 loss1= 1.9007 loss2=-3627.1543 lr= 0.000100
  3405. 2019-03-12 23:26:20,072 : epoch 375 , step 160 loss0= 0.0000 loss1= 2.5467 loss2=-3513.9360 lr= 0.000100
  3406. 2019-03-12 23:26:52,884 : epoch 376 , step 0 loss0= 0.0000 loss1= 1.7098 loss2=-3429.3672 lr= 0.000100
  3407. 2019-03-12 23:27:28,933 : epoch 376 , step 40 loss0= 0.0000 loss1= 1.7135 loss2=-3463.2837 lr= 0.000100
  3408. 2019-03-12 23:28:04,814 : epoch 376 , step 80 loss0= 0.0000 loss1= 1.9601 loss2=-3862.8938 lr= 0.000100
  3409. 2019-03-12 23:28:40,797 : epoch 376 , step 120 loss0= 0.0000 loss1= 1.5108 loss2=-3615.6797 lr= 0.000100
  3410. 2019-03-12 23:29:16,679 : epoch 376 , step 160 loss0= 0.0000 loss1= 1.7064 loss2=-3290.8743 lr= 0.000100
  3411. 2019-03-12 23:29:49,556 : epoch 377 , step 0 loss0= 0.0000 loss1= 2.5130 loss2=-3759.8247 lr= 0.000100
  3412. 2019-03-12 23:30:25,300 : epoch 377 , step 40 loss0= 0.0000 loss1= 2.2805 loss2=-3766.6387 lr= 0.000100
  3413. 2019-03-12 23:31:00,905 : epoch 377 , step 80 loss0= 0.0000 loss1= 2.0089 loss2=-3589.2759 lr= 0.000100
  3414. 2019-03-12 23:31:36,598 : epoch 377 , step 120 loss0= 0.0000 loss1= 3.1006 loss2=-3686.3416 lr= 0.000100
  3415. 2019-03-12 23:32:12,581 : epoch 377 , step 160 loss0= 0.0000 loss1= 2.2161 loss2=-4079.6743 lr= 0.000100
  3416. 2019-03-12 23:32:45,574 : epoch 378 , step 0 loss0= 0.0000 loss1= 2.0636 loss2=-3847.7224 lr= 0.000100
  3417. 2019-03-12 23:33:22,101 : epoch 378 , step 40 loss0= 0.0000 loss1= 2.2671 loss2=-3710.9531 lr= 0.000100
  3418. 2019-03-12 23:33:57,580 : epoch 378 , step 80 loss0= 0.0000 loss1= 2.2198 loss2=-3695.9036 lr= 0.000100
  3419. 2019-03-12 23:34:33,310 : epoch 378 , step 120 loss0= 0.0000 loss1= 1.7687 loss2=-3679.2156 lr= 0.000100
  3420. 2019-03-12 23:35:10,058 : epoch 378 , step 160 loss0= 0.0000 loss1= 2.4533 loss2=-3864.2935 lr= 0.000100
  3421. 2019-03-12 23:35:43,363 : epoch 379 , step 0 loss0= 0.0000 loss1= 2.5629 loss2=-3860.1843 lr= 0.000100
  3422. 2019-03-12 23:36:19,455 : epoch 379 , step 40 loss0= 0.0000 loss1= 2.3454 loss2=-3958.5454 lr= 0.000100
  3423. 2019-03-12 23:36:55,431 : epoch 379 , step 80 loss0= 0.0000 loss1= 2.7938 loss2=-3424.7468 lr= 0.000100
  3424. 2019-03-12 23:37:31,211 : epoch 379 , step 120 loss0= 0.0000 loss1= 2.4935 loss2=-3758.5515 lr= 0.000100
  3425. 2019-03-12 23:38:07,485 : epoch 379 , step 160 loss0= 0.0000 loss1= 1.9178 loss2=-3296.1619 lr= 0.000100
  3426. 2019-03-12 23:38:40,226 : epoch 380 , step 0 loss0= 0.0000 loss1= 3.4940 loss2=-4097.5044 lr= 0.000100
  3427. 2019-03-12 23:39:17,062 : epoch 380 , step 40 loss0= 0.0000 loss1= 2.0913 loss2=-3515.7766 lr= 0.000100
  3428. 2019-03-12 23:39:53,701 : epoch 380 , step 80 loss0= 0.0000 loss1= 2.0039 loss2=-3673.3110 lr= 0.000100
  3429. 2019-03-12 23:40:30,412 : epoch 380 , step 120 loss0= 0.0000 loss1= 1.5363 loss2=-3289.5173 lr= 0.000100
  3430. 2019-03-12 23:41:06,778 : epoch 380 , step 160 loss0= 0.0000 loss1= 2.0567 loss2=-3769.9094 lr= 0.000100
  3431. 2019-03-12 23:41:37,878 : ## train ##
  3432. 2019-03-12 23:46:34,739 : >> epoch 380 : average_loss= 0.8134, average_accuracy=82.370%, mean_weight_std= 0.3737, mean_normalized_std= 0.5466 mean_dist=2.979e+03
  3433. 2019-03-12 23:46:34,748 : >> epoch 380 : median_loss= 0.8216, median_accuracy=82.490%, median_weight_std= 0.2221, median_normalized_std= 0.1009 median_dist=2.978e+03
  3434. 2019-03-12 23:46:34,748 : >> epoch 380 : ensemble accuracy: 85.68%
  3435. 2019-03-12 23:46:34,748 : >> epoch 380 : loss histogram: (0.00,0.35, 18),(0.35,0.70, 0),(0.70,1.05, 4),(1.05,1.40, 0),(1.40,1.75, 18),
  3436. 2019-03-12 23:46:34,748 : >> epoch 380 : accuracy histogram: (0.00,33.87, 18),(33.87,67.74, 0),(67.74,101.62, 4),(101.62,135.49, 0),(135.49,169.36, 18),
  3437. 2019-03-12 23:46:34,754 : >> epoch 380 : weight std histogram: (0.0000,2.6932, 462633),(2.6932,5.3864, 3763),(5.3864,8.0795, 436),(8.0795,10.7727, 66),(10.7727,13.4659, 8),
  3438. 2019-03-12 23:46:34,760 : >> epoch 380 : weight normalized std histogram: (0.0000,3897.2160, 466901),(3897.2160,7794.4320, 4),(7794.4320,11691.6480, 0),(11691.6480,15588.8641, 0),(15588.8641,19486.0801, 1),
  3439. 2019-03-12 23:46:34,760 : >> epoch 380 : dist histogram: (2577.9053,2741.3880, 2),(2741.3880,2904.8708, 5),(2904.8708,3068.3536, 6),(3068.3536,3231.8363, 5),(3231.8363,3395.3191, 2),
  3440. 2019-03-12 23:46:34,760 : ## eval ##
  3441. 2019-03-12 23:51:28,913 : >> epoch 380 : average_loss= 0.7333, average_accuracy=85.886%, mean_weight_std= 0.3924, mean_normalized_std= 0.6131 mean_dist=2.98e+03
  3442. 2019-03-12 23:51:28,921 : >> epoch 380 : median_loss= 0.7233, median_accuracy=86.630%, median_weight_std= 0.2273, median_normalized_std= 0.1044 median_dist=2.984e+03
  3443. 2019-03-12 23:51:28,921 : >> epoch 380 : ensemble accuracy: 88.86%
  3444. 2019-03-12 23:51:28,922 : >> epoch 380 : loss histogram: (0.00,0.31, 18),(0.31,0.63, 0),(0.63,0.94, 4),(0.94,1.25, 0),(1.25,1.57, 18),
  3445. 2019-03-12 23:51:28,922 : >> epoch 380 : accuracy histogram: (0.00,35.15, 18),(35.15,70.29, 0),(70.29,105.44, 4),(105.44,140.58, 0),(140.58,175.73, 18),
  3446. 2019-03-12 23:51:28,928 : >> epoch 380 : weight std histogram: (0.0000,3.5314, 464126),(3.5314,7.0629, 2498),(7.0629,10.5943, 253),(10.5943,14.1258, 27),(14.1258,17.6572, 2),
  3447. 2019-03-12 23:51:28,933 : >> epoch 380 : weight normalized std histogram: (0.0000,4374.6594, 466901),(4374.6594,8749.3188, 2),(8749.3188,13123.9781, 2),(13123.9781,17498.6375, 0),(17498.6375,21873.2969, 1),
  3448. 2019-03-12 23:51:28,933 : >> epoch 380 : dist histogram: (2690.6570,2904.0657, 8),(2904.0657,3117.4745, 8),(3117.4745,3330.8832, 3),(3330.8832,3544.2919, 0),(3544.2919,3757.7007, 1),
  3449. 2019-03-12 23:51:28,989 : >> epoch 380 : dist pair-wise
  3450. 2019-03-12 23:51:28,995 :
  3451. [[ 0. 605.63 375.501 656.434 799.422 456.482 694.868
  3452. 474.311 676.863 811.517 576.648 979.959 569.13 1158.063
  3453. 800.651 834.394 658.428 628.479 816.619 672.407]
  3454. [ 605.63 0. 471.698 330.028 540.147 395.323 368.451
  3455. 416.508 518.654 540.627 322.638 796.591 495.827 1134.903
  3456. 578.873 554.311 494.097 547.738 564.917 558.835]
  3457. [ 375.501 471.698 0. 504.859 588.219 344.437 503.224
  3458. 275.638 497.419 685.657 374.806 854.963 522.629 1282.403
  3459. 603.636 681.377 627.798 556.026 636.391 558.183]
  3460. [ 656.434 330.028 504.859 0. 563.811 461.095 445.208
  3461. 462.288 405.142 361.395 266.273 804.029 496.778 1277.706
  3462. 502.632 383.804 509.683 544.322 399.265 508.84 ]
  3463. [ 799.422 540.147 588.219 563.811 0. 560.236 425.354
  3464. 548.671 563.611 653.392 532.846 503.685 626.044 1283.72
  3465. 426.402 626.076 642.763 647.237 594.678 614.002]
  3466. [ 456.482 395.323 344.437 461.095 560.236 0. 478.37
  3467. 384.655 464.516 587.454 406.309 881.888 296.948 1197.
  3468. 557.865 585.199 476.411 366.446 560.866 507.41 ]
  3469. [ 694.868 368.451 503.224 445.208 425.354 478.37 0.
  3470. 440.239 532.612 617.951 376.072 708.66 563.543 1218.108
  3471. 484.917 601.593 577.3 549.285 576.005 589.019]
  3472. [ 474.311 416.508 275.638 462.288 548.671 384.655 440.239
  3473. 0. 442.528 616.343 337.241 833.057 514.026 1314.101
  3474. 551.411 627.51 609.724 527.9 577.454 528.362]
  3475. [ 676.863 518.654 497.419 405.142 563.611 464.516 532.612
  3476. 442.528 0. 412.36 368.351 896. 430.608 1442.745
  3477. 324.655 418.555 480.443 442.179 275.497 293.163]
  3478. [ 811.517 540.627 685.657 361.395 653.392 587.454 617.951
  3479. 616.343 412.36 0. 500.251 898.621 520.346 1386.909
  3480. 484.295 208.259 493.311 538.517 281.944 487.467]
  3481. [ 576.648 322.638 374.806 266.273 532.846 406.309 376.072
  3482. 337.241 368.351 500.251 0. 813.664 481.695 1298.435
  3483. 490.378 509.411 534.416 509.029 450.509 487.48 ]
  3484. [ 979.959 796.591 854.963 804.029 503.685 881.888 708.66
  3485. 833.057 896. 898.621 813.664 0. 948.919 1221.183
  3486. 738.978 885.224 866.311 971.224 910.961 884.953]
  3487. [ 569.13 495.827 522.629 496.778 626.044 296.948 563.543
  3488. 514.026 430.608 520.346 481.695 948.919 0. 1233.479
  3489. 546.817 524.647 429.947 205.666 486.936 494.539]
  3490. [ 1158.063 1134.903 1282.403 1277.706 1283.72 1197. 1218.108
  3491. 1314.101 1442.745 1386.909 1298.435 1221.183 1233.479 0.
  3492. 1406.912 1416.166 1145.82 1298.647 1471.786 1347.154]
  3493. [ 800.651 578.873 603.636 502.632 426.402 557.865 484.917
  3494. 551.411 324.655 484.295 490.378 738.978 546.817 1406.912
  3495. 0. 445.729 487.448 521.641 363.584 340.596]
  3496. [ 834.394 554.311 681.377 383.804 626.076 585.199 601.593
  3497. 627.51 418.555 208.259 509.411 885.224 524.647 1416.166
  3498. 445.729 0. 541.723 527.278 274.75 511.502]
  3499. [ 658.428 494.097 627.798 509.683 642.763 476.411 577.3
  3500. 609.724 480.443 493.311 534.416 866.311 429.947 1145.82
  3501. 487.448 541.723 0. 463.851 511.926 344.181]
  3502. [ 628.479 547.738 556.026 544.322 647.237 366.446 549.285
  3503. 527.9 442.179 538.517 509.029 971.224 205.666 1298.647
  3504. 521.641 527.278 463.851 0. 476.568 503.245]
  3505. [ 816.619 564.917 636.391 399.265 594.678 560.866 576.005
  3506. 577.454 275.497 281.944 450.509 910.961 486.936 1471.786
  3507. 363.584 274.75 511.926 476.568 0. 418.052]
  3508. [ 672.407 558.835 558.183 508.84 614.002 507.41 589.019
  3509. 528.362 293.163 487.467 487.48 884.953 494.539 1347.154
  3510. 340.596 511.502 344.181 503.245 418.052 0. ]]
  3511. 2019-03-12 23:51:30,513 : epoch 381 , step 0 loss0= 0.0000 loss1= 2.6787 loss2=-3796.5649 lr= 0.000100
  3512. 2019-03-12 23:52:06,252 : epoch 381 , step 40 loss0= 0.0000 loss1= 2.1305 loss2=-3341.2949 lr= 0.000100
  3513. 2019-03-12 23:52:42,746 : epoch 381 , step 80 loss0= 0.0000 loss1= 2.1231 loss2=-3743.4219 lr= 0.000100
  3514. 2019-03-12 23:53:18,617 : epoch 381 , step 120 loss0= 0.0000 loss1= 1.6737 loss2=-3579.4441 lr= 0.000100
  3515. 2019-03-12 23:53:54,700 : epoch 381 , step 160 loss0= 0.0000 loss1= 1.6265 loss2=-3260.6392 lr= 0.000100
  3516. 2019-03-12 23:54:27,446 : epoch 382 , step 0 loss0= 0.0000 loss1= 2.4865 loss2=-3648.4031 lr= 0.000100
  3517. 2019-03-12 23:55:03,296 : epoch 382 , step 40 loss0= 0.0000 loss1= 2.0676 loss2=-3851.5508 lr= 0.000100
  3518. 2019-03-12 23:55:39,583 : epoch 382 , step 80 loss0= 0.0000 loss1= 1.7065 loss2=-3478.8743 lr= 0.000100
  3519. 2019-03-12 23:56:15,539 : epoch 382 , step 120 loss0= 0.0000 loss1= 2.2100 loss2=-3685.9524 lr= 0.000100
  3520. 2019-03-12 23:56:51,096 : epoch 382 , step 160 loss0= 0.0000 loss1= 2.0435 loss2=-3577.9109 lr= 0.000100
  3521. 2019-03-12 23:57:23,427 : epoch 383 , step 0 loss0= 0.0000 loss1= 1.9985 loss2=-3334.5591 lr= 0.000100
  3522. 2019-03-12 23:57:59,059 : epoch 383 , step 40 loss0= 0.0000 loss1= 1.7832 loss2=-3471.5212 lr= 0.000100
  3523. 2019-03-12 23:58:34,838 : epoch 383 , step 80 loss0= 0.0000 loss1= 2.4861 loss2=-3935.3540 lr= 0.000100
  3524. 2019-03-12 23:59:11,707 : epoch 383 , step 120 loss0= 0.0000 loss1= 1.7591 loss2=-3920.8013 lr= 0.000100
  3525. 2019-03-12 23:59:48,262 : epoch 383 , step 160 loss0= 0.0000 loss1= 2.1135 loss2=-3762.3032 lr= 0.000100
  3526. 2019-03-13 00:00:20,809 : epoch 384 , step 0 loss0= 0.0000 loss1= 2.3367 loss2=-3900.8574 lr= 0.000100
  3527. 2019-03-13 00:00:57,163 : epoch 384 , step 40 loss0= 0.0000 loss1= 2.6029 loss2=-3885.9504 lr= 0.000100
  3528. 2019-03-13 00:01:32,777 : epoch 384 , step 80 loss0= 0.0000 loss1= 1.8815 loss2=-3718.9438 lr= 0.000100
  3529. 2019-03-13 00:02:09,255 : epoch 384 , step 120 loss0= 0.0000 loss1= 1.7340 loss2=-3717.6384 lr= 0.000100
  3530. 2019-03-13 00:02:46,067 : epoch 384 , step 160 loss0= 0.0000 loss1= 1.9367 loss2=-3706.7688 lr= 0.000100
  3531. 2019-03-13 00:03:18,985 : epoch 385 , step 0 loss0= 0.0000 loss1= 2.0560 loss2=-3675.0493 lr= 0.000100
  3532. 2019-03-13 00:03:55,498 : epoch 385 , step 40 loss0= 0.0000 loss1= 2.9968 loss2=-3833.5388 lr= 0.000100
  3533. 2019-03-13 00:04:31,190 : epoch 385 , step 80 loss0= 0.0000 loss1= 2.0760 loss2=-3380.7546 lr= 0.000100
  3534. 2019-03-13 00:05:07,458 : epoch 385 , step 120 loss0= 0.0000 loss1= 2.0909 loss2=-3612.8367 lr= 0.000100
  3535. 2019-03-13 00:05:43,466 : epoch 385 , step 160 loss0= 0.0000 loss1= 2.3796 loss2=-3901.8582 lr= 0.000100
  3536. 2019-03-13 00:06:16,038 : epoch 386 , step 0 loss0= 0.0000 loss1= 2.8059 loss2=-3295.7642 lr= 0.000100
  3537. 2019-03-13 00:06:51,400 : epoch 386 , step 40 loss0= 0.0000 loss1= 2.1626 loss2=-3516.1367 lr= 0.000100
  3538. 2019-03-13 00:07:26,925 : epoch 386 , step 80 loss0= 0.0000 loss1= 1.9486 loss2=-3789.6609 lr= 0.000100
  3539. 2019-03-13 00:08:03,372 : epoch 386 , step 120 loss0= 0.0000 loss1= 1.6921 loss2=-3569.1047 lr= 0.000100
  3540. 2019-03-13 00:08:39,495 : epoch 386 , step 160 loss0= 0.0000 loss1= 1.9816 loss2=-3717.7383 lr= 0.000100
  3541. 2019-03-13 00:09:12,603 : epoch 387 , step 0 loss0= 0.0000 loss1= 2.3689 loss2=-3823.9907 lr= 0.000100
  3542. 2019-03-13 00:09:48,124 : epoch 387 , step 40 loss0= 0.0000 loss1= 2.0012 loss2=-3988.5020 lr= 0.000100
  3543. 2019-03-13 00:10:23,431 : epoch 387 , step 80 loss0= 0.0000 loss1= 2.2276 loss2=-3728.5437 lr= 0.000100
  3544. 2019-03-13 00:10:58,675 : epoch 387 , step 120 loss0= 0.0000 loss1= 2.5643 loss2=-3368.6521 lr= 0.000100
  3545. 2019-03-13 00:11:34,633 : epoch 387 , step 160 loss0= 0.0000 loss1= 2.3294 loss2=-3734.2310 lr= 0.000100
  3546. 2019-03-13 00:12:07,487 : epoch 388 , step 0 loss0= 0.0000 loss1= 1.6335 loss2=-3685.2185 lr= 0.000100
  3547. 2019-03-13 00:12:43,612 : epoch 388 , step 40 loss0= 0.0000 loss1= 2.4173 loss2=-3838.7271 lr= 0.000100
  3548. 2019-03-13 00:13:19,339 : epoch 388 , step 80 loss0= 0.0000 loss1= 2.5207 loss2=-3762.6274 lr= 0.000100
  3549. 2019-03-13 00:13:54,886 : epoch 388 , step 120 loss0= 0.0000 loss1= 2.7123 loss2=-3945.9211 lr= 0.000100
  3550. 2019-03-13 00:14:30,933 : epoch 388 , step 160 loss0= 0.0000 loss1= 1.9777 loss2=-3544.9009 lr= 0.000100
  3551. 2019-03-13 00:15:03,207 : epoch 389 , step 0 loss0= 0.0000 loss1= 2.3770 loss2=-3408.5076 lr= 0.000100
  3552. 2019-03-13 00:15:39,291 : epoch 389 , step 40 loss0= 0.0000 loss1= 1.9999 loss2=-3693.6345 lr= 0.000100
  3553. 2019-03-13 00:16:15,368 : epoch 389 , step 80 loss0= 0.0000 loss1= 1.8369 loss2=-3933.7036 lr= 0.000100
  3554. 2019-03-13 00:16:51,980 : epoch 389 , step 120 loss0= 0.0000 loss1= 1.4195 loss2=-3102.0078 lr= 0.000100
  3555. 2019-03-13 00:17:27,984 : epoch 389 , step 160 loss0= 0.0000 loss1= 2.2371 loss2=-3402.6072 lr= 0.000100
  3556. 2019-03-13 00:18:01,020 : epoch 390 , step 0 loss0= 0.0000 loss1= 2.1159 loss2=-3799.4688 lr= 0.000100
  3557. 2019-03-13 00:18:36,726 : epoch 390 , step 40 loss0= 0.0000 loss1= 2.3437 loss2=-3734.3054 lr= 0.000100
  3558. 2019-03-13 00:19:12,806 : epoch 390 , step 80 loss0= 0.0000 loss1= 2.9004 loss2=-3713.4966 lr= 0.000100
  3559. 2019-03-13 00:19:48,538 : epoch 390 , step 120 loss0= 0.0000 loss1= 1.7395 loss2=-3795.4841 lr= 0.000100
  3560. 2019-03-13 00:20:24,891 : epoch 390 , step 160 loss0= 0.0000 loss1= 2.1971 loss2=-3759.8228 lr= 0.000100
  3561. 2019-03-13 00:20:57,322 : epoch 391 , step 0 loss0= 0.0000 loss1= 2.0767 loss2=-3599.6165 lr= 0.000100
  3562. 2019-03-13 00:21:33,738 : epoch 391 , step 40 loss0= 0.0000 loss1= 2.3843 loss2=-3871.3972 lr= 0.000100
  3563. 2019-03-13 00:22:09,724 : epoch 391 , step 80 loss0= 0.0000 loss1= 2.7261 loss2=-3878.7947 lr= 0.000100
  3564. 2019-03-13 00:22:45,785 : epoch 391 , step 120 loss0= 0.0000 loss1= 2.8229 loss2=-3782.8125 lr= 0.000100
  3565. 2019-03-13 00:23:21,851 : epoch 391 , step 160 loss0= 0.0000 loss1= 2.3524 loss2=-4048.0408 lr= 0.000100
  3566. 2019-03-13 00:23:54,343 : epoch 392 , step 0 loss0= 0.0000 loss1= 1.1010 loss2=-3063.9353 lr= 0.000100
  3567. 2019-03-13 00:24:30,443 : epoch 392 , step 40 loss0= 0.0000 loss1= 2.0139 loss2=-3493.9028 lr= 0.000100
  3568. 2019-03-13 00:25:06,969 : epoch 392 , step 80 loss0= 0.0000 loss1= 2.2305 loss2=-3530.5828 lr= 0.000100
  3569. 2019-03-13 00:25:43,754 : epoch 392 , step 120 loss0= 0.0000 loss1= 3.3125 loss2=-3754.9204 lr= 0.000100
  3570. 2019-03-13 00:26:20,393 : epoch 392 , step 160 loss0= 0.0000 loss1= 2.4429 loss2=-4188.8198 lr= 0.000100
  3571. 2019-03-13 00:26:53,252 : epoch 393 , step 0 loss0= 0.0000 loss1= 1.4227 loss2=-3226.5896 lr= 0.000100
  3572. 2019-03-13 00:27:29,123 : epoch 393 , step 40 loss0= 0.0000 loss1= 2.6957 loss2=-4008.5903 lr= 0.000100
  3573. 2019-03-13 00:28:04,548 : epoch 393 , step 80 loss0= 0.0000 loss1= 1.9650 loss2=-3903.8650 lr= 0.000100
  3574. 2019-03-13 00:28:40,167 : epoch 393 , step 120 loss0= 0.0000 loss1= 2.3388 loss2=-3912.3977 lr= 0.000100
  3575. 2019-03-13 00:29:16,473 : epoch 393 , step 160 loss0= 0.0000 loss1= 2.1270 loss2=-3459.7285 lr= 0.000100
  3576. 2019-03-13 00:29:49,287 : epoch 394 , step 0 loss0= 0.0000 loss1= 2.7078 loss2=-3912.6443 lr= 0.000100
  3577. 2019-03-13 00:30:25,062 : epoch 394 , step 40 loss0= 0.0000 loss1= 1.9449 loss2=-3396.8711 lr= 0.000100
  3578. 2019-03-13 00:31:00,874 : epoch 394 , step 80 loss0= 0.0000 loss1= 1.8513 loss2=-3684.0271 lr= 0.000100
  3579. 2019-03-13 00:31:36,895 : epoch 394 , step 120 loss0= 0.0000 loss1= 2.6826 loss2=-4035.7341 lr= 0.000100
  3580. 2019-03-13 00:32:12,517 : epoch 394 , step 160 loss0= 0.0000 loss1= 1.9948 loss2=-3619.5142 lr= 0.000100
  3581. 2019-03-13 00:32:45,567 : epoch 395 , step 0 loss0= 0.0000 loss1= 1.9800 loss2=-3932.7629 lr= 0.000100
  3582. 2019-03-13 00:33:22,099 : epoch 395 , step 40 loss0= 0.0000 loss1= 2.6042 loss2=-3842.5903 lr= 0.000100
  3583. 2019-03-13 00:33:57,724 : epoch 395 , step 80 loss0= 0.0000 loss1= 2.2677 loss2=-3732.7173 lr= 0.000100
  3584. 2019-03-13 00:34:33,316 : epoch 395 , step 120 loss0= 0.0000 loss1= 1.5835 loss2=-3595.1816 lr= 0.000100
  3585. 2019-03-13 00:35:08,782 : epoch 395 , step 160 loss0= 0.0000 loss1= 1.9945 loss2=-3791.9407 lr= 0.000100
  3586. 2019-03-13 00:35:41,071 : epoch 396 , step 0 loss0= 0.0000 loss1= 2.2201 loss2=-3397.3376 lr= 0.000100
  3587. 2019-03-13 00:36:17,297 : epoch 396 , step 40 loss0= 0.0000 loss1= 2.0092 loss2=-3708.5747 lr= 0.000100
  3588. 2019-03-13 00:36:53,253 : epoch 396 , step 80 loss0= 0.0000 loss1= 3.0019 loss2=-4033.3411 lr= 0.000100
  3589. 2019-03-13 00:37:29,446 : epoch 396 , step 120 loss0= 0.0000 loss1= 3.0196 loss2=-3796.3435 lr= 0.000100
  3590. 2019-03-13 00:38:05,429 : epoch 396 , step 160 loss0= 0.0000 loss1= 2.6940 loss2=-3844.9336 lr= 0.000100
  3591. 2019-03-13 00:38:37,862 : epoch 397 , step 0 loss0= 0.0000 loss1= 2.1057 loss2=-3708.2654 lr= 0.000100
  3592. 2019-03-13 00:39:13,267 : epoch 397 , step 40 loss0= 0.0000 loss1= 2.2202 loss2=-3556.9875 lr= 0.000100
  3593. 2019-03-13 00:39:48,666 : epoch 397 , step 80 loss0= 0.0000 loss1= 2.0407 loss2=-3586.3652 lr= 0.000100
  3594. 2019-03-13 00:40:24,501 : epoch 397 , step 120 loss0= 0.0000 loss1= 1.9192 loss2=-3616.8364 lr= 0.000100
  3595. 2019-03-13 00:41:00,220 : epoch 397 , step 160 loss0= 0.0000 loss1= 2.0983 loss2=-3395.2676 lr= 0.000100
  3596. 2019-03-13 00:41:32,563 : epoch 398 , step 0 loss0= 0.0000 loss1= 2.8091 loss2=-3811.0930 lr= 0.000100
  3597. 2019-03-13 00:42:08,837 : epoch 398 , step 40 loss0= 0.0000 loss1= 2.1860 loss2=-4011.4543 lr= 0.000100
  3598. 2019-03-13 00:42:45,174 : epoch 398 , step 80 loss0= 0.0000 loss1= 2.5264 loss2=-3779.0359 lr= 0.000100
  3599. 2019-03-13 00:43:21,519 : epoch 398 , step 120 loss0= 0.0000 loss1= 2.1799 loss2=-3757.3228 lr= 0.000100
  3600. 2019-03-13 00:43:57,972 : epoch 398 , step 160 loss0= 0.0000 loss1= 1.8496 loss2=-3512.3093 lr= 0.000100
  3601. 2019-03-13 00:44:30,699 : epoch 399 , step 0 loss0= 0.0000 loss1= 1.6626 loss2=-3665.0359 lr= 0.000100
  3602. 2019-03-13 00:45:07,146 : epoch 399 , step 40 loss0= 0.0000 loss1= 2.2220 loss2=-3746.3516 lr= 0.000100
  3603. 2019-03-13 00:45:43,771 : epoch 399 , step 80 loss0= 0.0000 loss1= 2.1358 loss2=-3840.1262 lr= 0.000100
  3604. 2019-03-13 00:46:20,266 : epoch 399 , step 120 loss0= 0.0000 loss1= 2.0789 loss2=-3573.7993 lr= 0.000100
  3605. 2019-03-13 00:46:56,938 : epoch 399 , step 160 loss0= 0.0000 loss1= 2.2236 loss2=-3731.1667 lr= 0.000100
  3606. 2019-03-13 00:47:30,050 : epoch 400 , step 0 loss0= 0.0000 loss1= 2.0070 loss2=-3706.8279 lr= 0.000100
  3607. 2019-03-13 00:48:05,515 : epoch 400 , step 40 loss0= 0.0000 loss1= 2.1975 loss2=-3828.0200 lr= 0.000100
  3608. 2019-03-13 00:48:41,266 : epoch 400 , step 80 loss0= 0.0000 loss1= 2.8048 loss2=-3738.0291 lr= 0.000100
  3609. 2019-03-13 00:49:17,583 : epoch 400 , step 120 loss0= 0.0000 loss1= 2.0336 loss2=-3527.0278 lr= 0.000100
  3610. 2019-03-13 00:49:53,723 : epoch 400 , step 160 loss0= 0.0000 loss1= 1.4479 loss2=-3251.7351 lr= 0.000100
  3611. 2019-03-13 00:50:25,031 : ## train ##
  3612. 2019-03-13 00:55:21,948 : >> epoch 400 : average_loss= 0.8349, average_accuracy=82.299%, mean_weight_std= 0.425, mean_normalized_std= 0.7275 mean_dist=3.021e+03
  3613. 2019-03-13 00:55:21,957 : >> epoch 400 : median_loss= 0.8063, median_accuracy=84.205%, median_weight_std= 0.2527, median_normalized_std= 0.1125 median_dist=2.968e+03
  3614. 2019-03-13 00:55:21,957 : >> epoch 400 : ensemble accuracy: 85.42%
  3615. 2019-03-13 00:55:21,958 : >> epoch 400 : loss histogram: (0.00,0.36, 18),(0.36,0.71, 0),(0.71,1.07, 4),(1.07,1.43, 0),(1.43,1.78, 18),
  3616. 2019-03-13 00:55:21,958 : >> epoch 400 : accuracy histogram: (0.00,33.68, 18),(33.68,67.36, 0),(67.36,101.05, 4),(101.05,134.73, 0),(134.73,168.41, 18),
  3617. 2019-03-13 00:55:21,964 : >> epoch 400 : weight std histogram: (0.0000,3.0776, 462737),(3.0776,6.1553, 3625),(6.1553,9.2329, 459),(9.2329,12.3106, 73),(12.3106,15.3882, 12),
  3618. 2019-03-13 00:55:21,969 : >> epoch 400 : weight normalized std histogram: (0.0000,9212.7633, 466904),(9212.7633,18425.5266, 0),(18425.5266,27638.2898, 0),(27638.2898,36851.0531, 0),(36851.0531,46063.8164, 2),
  3619. 2019-03-13 00:55:21,969 : >> epoch 400 : dist histogram: (2624.0444,2816.0251, 3),(2816.0251,3008.0058, 9),(3008.0058,3199.9864, 5),(3199.9864,3391.9671, 0),(3391.9671,3583.9478, 3),
  3620. 2019-03-13 00:55:21,970 : ## eval ##
  3621. 2019-03-13 01:00:16,875 : >> epoch 400 : average_loss= 0.7286, average_accuracy=85.613%, mean_weight_std= 0.35, mean_normalized_std= 1.0292 mean_dist=3.024e+03
  3622. 2019-03-13 01:00:16,890 : >> epoch 400 : median_loss= 0.7216, median_accuracy=85.960%, median_weight_std= 0.1978, median_normalized_std= 0.0898 median_dist=3.005e+03
  3623. 2019-03-13 01:00:16,890 : >> epoch 400 : ensemble accuracy: 88.46%
  3624. 2019-03-13 01:00:16,890 : >> epoch 400 : loss histogram: (0.00,0.31, 18),(0.31,0.63, 0),(0.63,0.94, 4),(0.94,1.26, 0),(1.26,1.57, 18),
  3625. 2019-03-13 01:00:16,891 : >> epoch 400 : accuracy histogram: (0.00,35.14, 18),(35.14,70.29, 0),(70.29,105.43, 4),(105.43,140.58, 0),(140.58,175.72, 18),
  3626. 2019-03-13 01:00:16,897 : >> epoch 400 : weight std histogram: (0.0000,3.0509, 463741),(3.0509,6.1019, 2760),(6.1019,9.1528, 346),(9.1528,12.2038, 52),(12.2038,15.2547, 7),
  3627. 2019-03-13 01:00:16,902 : >> epoch 400 : weight normalized std histogram: (0.0000,32373.4344, 466904),(32373.4344,64746.8688, 1),(64746.8688,97120.3031, 0),(97120.3031,129493.7375, 0),(129493.7375,161867.1719, 1),
  3628. 2019-03-13 01:00:16,902 : >> epoch 400 : dist histogram: (2666.8921,2821.8792, 1),(2821.8792,2976.8664, 7),(2976.8664,3131.8536, 8),(3131.8536,3286.8407, 3),(3286.8407,3441.8279, 1),
  3629. 2019-03-13 01:00:16,968 : >> epoch 400 : dist pair-wise
  3630. 2019-03-13 01:00:16,974 :
  3631. [[ 0. 483.678 640.248 416.031 511.131 292.344 518.375 403.743
  3632. 276.39 417.386 649.557 525.777 478.118 638.891 491.088 378.488
  3633. 534.086 338.568 511.533 593.496]
  3634. [ 483.678 0. 881.137 691.409 289.339 593.966 567.251 693.002
  3635. 447.381 459.717 766.479 701.688 251.853 771.595 251.564 618.558
  3636. 670.495 665.009 726.649 854.215]
  3637. [ 640.248 881.137 0. 783.155 859.005 758.131 962.65 687.44
  3638. 578.15 550.162 686.474 655.69 892.251 655.346 871.863 504.247
  3639. 749.114 625.088 516.83 819.732]
  3640. [ 416.031 691.409 783.155 0. 695.066 307.971 548.552 407.772
  3641. 569.239 684.977 648.042 592.528 686.82 630.746 709.481 629.931
  3642. 586.391 433.318 593.489 471.424]
  3643. [ 511.131 289.339 859.005 695.066 0. 626.645 503.138 658.3
  3644. 417.359 488.153 739.942 606.68 310.679 760.008 328.448 613.058
  3645. 562.145 687.544 720.613 842.747]
  3646. [ 292.344 593.966 758.131 307.971 626.645 0. 580.885 351.572
  3647. 449.049 577.078 736.205 630.687 572.628 723.921 602.326 566.613
  3648. 620.746 364.976 639.88 439.344]
  3649. [ 518.375 567.251 962.65 548.552 503.138 580.885 0. 582.882
  3650. 564.09 724.987 702.779 535.304 576.693 716.487 603.932 711.429
  3651. 452.818 690.903 714.045 825.604]
  3652. [ 403.743 693.002 687.44 407.772 658.3 351.572 582.882 0.
  3653. 494.783 623.586 678.122 475.112 691.545 679.963 704.44 561.088
  3654. 470.525 413.272 595.366 498.473]
  3655. [ 276.39 447.381 578.15 569.239 417.359 449.049 564.09 494.783
  3656. 0. 326.533 691.429 538.814 436.568 677.239 442.923 409.604
  3657. 551.413 478.855 570.992 694.518]
  3658. [ 417.386 459.717 550.162 684.977 488.153 577.078 724.987 623.586
  3659. 326.533 0. 702.078 616.019 516.996 677.429 503.908 437.376
  3660. 633.112 560.445 536.735 783.599]
  3661. [ 649.557 766.479 686.474 648.042 739.942 736.205 702.779 678.122
  3662. 691.429 702.078 0. 545.945 783.262 271.443 778.365 601.866
  3663. 615.64 646.469 385.551 797.748]
  3664. [ 525.777 701.688 655.69 592.528 606.68 630.687 535.304 475.112
  3665. 538.814 616.019 545.945 0. 731.776 538.143 725.4 520.404
  3666. 225.229 589.761 475.338 780.119]
  3667. [ 478.118 251.853 892.251 686.82 310.679 572.628 576.693 691.545
  3668. 436.568 516.996 783.262 731.776 0. 804.277 161.213 604.888
  3669. 707.112 631.861 747.664 790.349]
  3670. [ 638.891 771.595 655.346 630.746 760.008 723.921 716.487 679.963
  3671. 677.239 677.429 271.443 538.143 804.277 0. 802.29 632.181
  3672. 616.506 677.815 345.751 825.72 ]
  3673. [ 491.088 251.564 871.863 709.481 328.448 602.326 603.932 704.44
  3674. 442.923 503.908 778.365 725.4 161.213 802.29 0. 595.968
  3675. 710.514 639.897 733.786 817.505]
  3676. [ 378.488 618.558 504.247 629.931 613.058 566.613 711.429 561.088
  3677. 409.604 437.376 601.866 520.404 604.888 632.181 595.968 0.
  3678. 598.286 369.749 424.305 718.296]
  3679. [ 534.086 670.495 749.114 586.391 562.145 620.746 452.818 470.525
  3680. 551.413 633.112 615.64 225.229 707.112 616.506 710.514 598.286
  3681. 0. 629.042 571.899 780.579]
  3682. [ 338.568 665.009 625.088 433.318 687.544 364.976 690.903 413.272
  3683. 478.855 560.445 646.469 589.761 631.861 677.815 639.897 369.749
  3684. 629.042 0. 511.977 473.226]
  3685. [ 511.533 726.649 516.83 593.489 720.613 639.88 714.045 595.366
  3686. 570.992 536.735 385.551 475.338 747.664 345.751 733.786 424.305
  3687. 571.899 511.977 0. 757.55 ]
  3688. [ 593.496 854.215 819.732 471.424 842.747 439.344 825.604 498.473
  3689. 694.518 783.599 797.748 780.119 790.349 825.72 817.505 718.296
  3690. 780.579 473.226 757.55 0. ]]
  3691. 2019-03-13 01:00:18,440 : epoch 401 , step 0 loss0= 0.0000 loss1= 2.1341 loss2=-3816.2805 lr= 0.000100
  3692. 2019-03-13 01:00:54,349 : epoch 401 , step 40 loss0= 0.0000 loss1= 2.5280 loss2=-3489.6804 lr= 0.000100
  3693. 2019-03-13 01:01:30,127 : epoch 401 , step 80 loss0= 0.0000 loss1= 1.9208 loss2=-3761.6023 lr= 0.000100
  3694. 2019-03-13 01:02:06,409 : epoch 401 , step 120 loss0= 0.0000 loss1= 2.5987 loss2=-3709.4758 lr= 0.000100
  3695. 2019-03-13 01:02:42,171 : epoch 401 , step 160 loss0= 0.0000 loss1= 2.4248 loss2=-3764.7388 lr= 0.000100
  3696. 2019-03-13 01:03:14,748 : epoch 402 , step 0 loss0= 0.0000 loss1= 2.1290 loss2=-3715.8677 lr= 0.000100
  3697. 2019-03-13 01:03:50,606 : epoch 402 , step 40 loss0= 0.0000 loss1= 1.7970 loss2=-3562.5447 lr= 0.000100
  3698. 2019-03-13 01:04:26,745 : epoch 402 , step 80 loss0= 0.0000 loss1= 2.8838 loss2=-3779.1453 lr= 0.000100
  3699. 2019-03-13 01:05:03,146 : epoch 402 , step 120 loss0= 0.0000 loss1= 1.8520 loss2=-3447.8914 lr= 0.000100
  3700. 2019-03-13 01:05:39,840 : epoch 402 , step 160 loss0= 0.0000 loss1= 1.9553 loss2=-3855.5828 lr= 0.000100
  3701. 2019-03-13 01:06:12,516 : epoch 403 , step 0 loss0= 0.0000 loss1= 2.3991 loss2=-3886.7849 lr= 0.000100
  3702. 2019-03-13 01:06:48,191 : epoch 403 , step 40 loss0= 0.0000 loss1= 1.7988 loss2=-3246.7693 lr= 0.000100
  3703. 2019-03-13 01:07:24,098 : epoch 403 , step 80 loss0= 0.0000 loss1= 2.6742 loss2=-3699.1946 lr= 0.000100
  3704. 2019-03-13 01:08:00,664 : epoch 403 , step 120 loss0= 0.0000 loss1= 2.2537 loss2=-3662.4333 lr= 0.000100
  3705. 2019-03-13 01:08:36,767 : epoch 403 , step 160 loss0= 0.0000 loss1= 2.5082 loss2=-4038.4324 lr= 0.000100
  3706. 2019-03-13 01:09:08,912 : epoch 404 , step 0 loss0= 0.0000 loss1= 2.2389 loss2=-3774.9453 lr= 0.000100
  3707. 2019-03-13 01:09:45,363 : epoch 404 , step 40 loss0= 0.0000 loss1= 2.3247 loss2=-3665.7751 lr= 0.000100
  3708. 2019-03-13 01:10:21,866 : epoch 404 , step 80 loss0= 0.0000 loss1= 2.8341 loss2=-3850.8704 lr= 0.000100
  3709. 2019-03-13 01:10:57,279 : epoch 404 , step 120 loss0= 0.0000 loss1= 2.5379 loss2=-3779.7820 lr= 0.000100
  3710. 2019-03-13 01:11:32,713 : epoch 404 , step 160 loss0= 0.0000 loss1= 2.2406 loss2=-3384.0312 lr= 0.000100
  3711. 2019-03-13 01:12:04,796 : epoch 405 , step 0 loss0= 0.0000 loss1= 1.9090 loss2=-3521.4126 lr= 0.000100
  3712. 2019-03-13 01:12:40,922 : epoch 405 , step 40 loss0= 0.0000 loss1= 1.7660 loss2=-3262.3162 lr= 0.000100
  3713. 2019-03-13 01:13:17,149 : epoch 405 , step 80 loss0= 0.0000 loss1= 2.4962 loss2=-3687.2429 lr= 0.000100
  3714. 2019-03-13 01:13:53,018 : epoch 405 , step 120 loss0= 0.0000 loss1= 2.2022 loss2=-3773.7329 lr= 0.000100
  3715. 2019-03-13 01:14:28,842 : epoch 405 , step 160 loss0= 0.0000 loss1= 2.2267 loss2=-3880.6501 lr= 0.000100
  3716. 2019-03-13 01:15:01,527 : epoch 406 , step 0 loss0= 0.0000 loss1= 2.4400 loss2=-3543.2517 lr= 0.000100
  3717. 2019-03-13 01:15:37,590 : epoch 406 , step 40 loss0= 0.0000 loss1= 2.1471 loss2=-3930.0063 lr= 0.000100
  3718. 2019-03-13 01:16:13,558 : epoch 406 , step 80 loss0= 0.0000 loss1= 1.7342 loss2=-3633.6880 lr= 0.000100
  3719. 2019-03-13 01:16:49,294 : epoch 406 , step 120 loss0= 0.0000 loss1= 2.5669 loss2=-3972.6990 lr= 0.000100
  3720. 2019-03-13 01:17:24,896 : epoch 406 , step 160 loss0= 0.0000 loss1= 2.5196 loss2=-3698.5376 lr= 0.000100
  3721. 2019-03-13 01:17:58,014 : epoch 407 , step 0 loss0= 0.0000 loss1= 1.6145 loss2=-3598.8718 lr= 0.000100
  3722. 2019-03-13 01:18:33,638 : epoch 407 , step 40 loss0= 0.0000 loss1= 2.2627 loss2=-3746.1392 lr= 0.000100
  3723. 2019-03-13 01:19:09,443 : epoch 407 , step 80 loss0= 0.0000 loss1= 1.7095 loss2=-3296.2156 lr= 0.000100
  3724. 2019-03-13 01:19:45,701 : epoch 407 , step 120 loss0= 0.0000 loss1= 1.6357 loss2=-3528.5837 lr= 0.000100
  3725. 2019-03-13 01:20:21,419 : epoch 407 , step 160 loss0= 0.0000 loss1= 2.8036 loss2=-3757.3494 lr= 0.000100
  3726. 2019-03-13 01:20:54,392 : epoch 408 , step 0 loss0= 0.0000 loss1= 1.7307 loss2=-3171.7275 lr= 0.000100
  3727. 2019-03-13 01:21:30,199 : epoch 408 , step 40 loss0= 0.0000 loss1= 1.8314 loss2=-3644.0586 lr= 0.000100
  3728. 2019-03-13 01:22:05,908 : epoch 408 , step 80 loss0= 0.0000 loss1= 1.8880 loss2=-3692.3533 lr= 0.000100
  3729. 2019-03-13 01:22:41,878 : epoch 408 , step 120 loss0= 0.0000 loss1= 2.3718 loss2=-3891.4329 lr= 0.000100
  3730. 2019-03-13 01:23:18,547 : epoch 408 , step 160 loss0= 0.0000 loss1= 2.6221 loss2=-4005.5398 lr= 0.000100
  3731. 2019-03-13 01:23:51,680 : epoch 409 , step 0 loss0= 0.0000 loss1= 1.7746 loss2=-3785.3950 lr= 0.000100
  3732. 2019-03-13 01:24:27,852 : epoch 409 , step 40 loss0= 0.0000 loss1= 2.1201 loss2=-3532.0623 lr= 0.000100
  3733. 2019-03-13 01:25:03,568 : epoch 409 , step 80 loss0= 0.0000 loss1= 2.2173 loss2=-3665.9524 lr= 0.000100
  3734. 2019-03-13 01:25:39,350 : epoch 409 , step 120 loss0= 0.0000 loss1= 2.2240 loss2=-3747.0310 lr= 0.000100
  3735. 2019-03-13 01:26:15,382 : epoch 409 , step 160 loss0= 0.0000 loss1= 1.8257 loss2=-3532.5383 lr= 0.000100
  3736. 2019-03-13 01:26:47,853 : epoch 410 , step 0 loss0= 0.0000 loss1= 3.4378 loss2=-3827.6335 lr= 0.000100
  3737. 2019-03-13 01:27:23,723 : epoch 410 , step 40 loss0= 0.0000 loss1= 2.1772 loss2=-3310.8533 lr= 0.000100
  3738. 2019-03-13 01:27:59,596 : epoch 410 , step 80 loss0= 0.0000 loss1= 2.1452 loss2=-3884.8418 lr= 0.000100
  3739. 2019-03-13 01:28:35,944 : epoch 410 , step 120 loss0= 0.0000 loss1= 1.9825 loss2=-3772.2146 lr= 0.000100
  3740. 2019-03-13 01:29:12,680 : epoch 410 , step 160 loss0= 0.0000 loss1= 2.3623 loss2=-3763.8755 lr= 0.000100
  3741. 2019-03-13 01:29:45,855 : epoch 411 , step 0 loss0= 0.0000 loss1= 2.5123 loss2=-3768.1367 lr= 0.000100
  3742. 2019-03-13 01:30:21,673 : epoch 411 , step 40 loss0= 0.0000 loss1= 2.4595 loss2=-3929.6345 lr= 0.000100
  3743. 2019-03-13 01:30:57,582 : epoch 411 , step 80 loss0= 0.0000 loss1= 1.5524 loss2=-3289.6257 lr= 0.000100
  3744. 2019-03-13 01:31:33,851 : epoch 411 , step 120 loss0= 0.0000 loss1= 2.4967 loss2=-3609.4968 lr= 0.000100
  3745. 2019-03-13 01:32:09,809 : epoch 411 , step 160 loss0= 0.0000 loss1= 1.9282 loss2=-3843.0181 lr= 0.000100
  3746. 2019-03-13 01:32:42,484 : epoch 412 , step 0 loss0= 0.0000 loss1= 2.4611 loss2=-3776.5195 lr= 0.000100
  3747. 2019-03-13 01:33:18,233 : epoch 412 , step 40 loss0= 0.0000 loss1= 2.0592 loss2=-3818.1648 lr= 0.000100
  3748. 2019-03-13 01:33:53,967 : epoch 412 , step 80 loss0= 0.0000 loss1= 1.8012 loss2=-3738.6985 lr= 0.000100
  3749. 2019-03-13 01:34:30,270 : epoch 412 , step 120 loss0= 0.0000 loss1= 1.9870 loss2=-3785.5759 lr= 0.000100
  3750. 2019-03-13 01:35:06,371 : epoch 412 , step 160 loss0= 0.0000 loss1= 2.0442 loss2=-3900.7000 lr= 0.000100
  3751. 2019-03-13 01:35:39,178 : epoch 413 , step 0 loss0= 0.0000 loss1= 1.6742 loss2=-3582.1482 lr= 0.000100
  3752. 2019-03-13 01:36:15,274 : epoch 413 , step 40 loss0= 0.0000 loss1= 2.0870 loss2=-3510.7947 lr= 0.000100
  3753. 2019-03-13 01:36:51,474 : epoch 413 , step 80 loss0= 0.0000 loss1= 2.1724 loss2=-3880.0156 lr= 0.000100
  3754. 2019-03-13 01:37:27,403 : epoch 413 , step 120 loss0= 0.0000 loss1= 2.4145 loss2=-3554.2524 lr= 0.000100
  3755. 2019-03-13 01:38:03,139 : epoch 413 , step 160 loss0= 0.0000 loss1= 2.0353 loss2=-3631.9539 lr= 0.000100
  3756. 2019-03-13 01:38:36,227 : epoch 414 , step 0 loss0= 0.0000 loss1= 1.8484 loss2=-3411.8892 lr= 0.000100
  3757. 2019-03-13 01:39:11,598 : epoch 414 , step 40 loss0= 0.0000 loss1= 2.2655 loss2=-3174.6665 lr= 0.000100
  3758. 2019-03-13 01:39:47,217 : epoch 414 , step 80 loss0= 0.0000 loss1= 1.5902 loss2=-3492.8123 lr= 0.000100
  3759. 2019-03-13 01:40:22,471 : epoch 414 , step 120 loss0= 0.0000 loss1= 2.2241 loss2=-3802.6345 lr= 0.000100
  3760. 2019-03-13 01:40:57,930 : epoch 414 , step 160 loss0= 0.0000 loss1= 2.1577 loss2=-3845.9255 lr= 0.000100
  3761. 2019-03-13 01:41:30,320 : epoch 415 , step 0 loss0= 0.0000 loss1= 2.0602 loss2=-3767.5024 lr= 0.000100
  3762. 2019-03-13 01:42:05,890 : epoch 415 , step 40 loss0= 0.0000 loss1= 2.9245 loss2=-4084.3906 lr= 0.000100
  3763. 2019-03-13 01:42:42,049 : epoch 415 , step 80 loss0= 0.0000 loss1= 1.4778 loss2=-3623.1873 lr= 0.000100
  3764. 2019-03-13 01:43:18,433 : epoch 415 , step 120 loss0= 0.0000 loss1= 2.2982 loss2=-3534.4656 lr= 0.000100
  3765. 2019-03-13 01:43:54,706 : epoch 415 , step 160 loss0= 0.0000 loss1= 2.8515 loss2=-3955.2244 lr= 0.000100
  3766. 2019-03-13 01:44:27,872 : epoch 416 , step 0 loss0= 0.0000 loss1= 2.1335 loss2=-3691.7114 lr= 0.000100
  3767. 2019-03-13 01:45:04,612 : epoch 416 , step 40 loss0= 0.0000 loss1= 1.9529 loss2=-3469.9929 lr= 0.000100
  3768. 2019-03-13 01:45:41,553 : epoch 416 , step 80 loss0= 0.0000 loss1= 1.2710 loss2=-3398.1033 lr= 0.000100
  3769. 2019-03-13 01:46:18,098 : epoch 416 , step 120 loss0= 0.0000 loss1= 2.1811 loss2=-3465.5759 lr= 0.000100
  3770. 2019-03-13 01:46:54,105 : epoch 416 , step 160 loss0= 0.0000 loss1= 2.2387 loss2=-3583.0942 lr= 0.000100
  3771. 2019-03-13 01:47:27,063 : epoch 417 , step 0 loss0= 0.0000 loss1= 1.3423 loss2=-3391.4102 lr= 0.000100
  3772. 2019-03-13 01:48:03,018 : epoch 417 , step 40 loss0= 0.0000 loss1= 2.1848 loss2=-3489.6301 lr= 0.000100
  3773. 2019-03-13 01:48:38,661 : epoch 417 , step 80 loss0= 0.0000 loss1= 2.2553 loss2=-3861.8816 lr= 0.000100
  3774. 2019-03-13 01:49:15,017 : epoch 417 , step 120 loss0= 0.0000 loss1= 1.7016 loss2=-3435.2546 lr= 0.000100
  3775. 2019-03-13 01:49:51,523 : epoch 417 , step 160 loss0= 0.0000 loss1= 1.6412 loss2=-3459.1321 lr= 0.000100
  3776. 2019-03-13 01:50:24,421 : epoch 418 , step 0 loss0= 0.0000 loss1= 2.2794 loss2=-4062.5320 lr= 0.000100
  3777. 2019-03-13 01:51:00,042 : epoch 418 , step 40 loss0= 0.0000 loss1= 3.2360 loss2=-4055.2461 lr= 0.000100
  3778. 2019-03-13 01:51:35,507 : epoch 418 , step 80 loss0= 0.0000 loss1= 2.1998 loss2=-3602.6492 lr= 0.000100
  3779. 2019-03-13 01:52:11,102 : epoch 418 , step 120 loss0= 0.0000 loss1= 1.7908 loss2=-3848.7864 lr= 0.000100
  3780. 2019-03-13 01:52:46,697 : epoch 418 , step 160 loss0= 0.0000 loss1= 1.6682 loss2=-3603.9700 lr= 0.000100
  3781. 2019-03-13 01:53:19,560 : epoch 419 , step 0 loss0= 0.0000 loss1= 2.4781 loss2=-3895.4578 lr= 0.000100
  3782. 2019-03-13 01:53:55,759 : epoch 419 , step 40 loss0= 0.0000 loss1= 1.7410 loss2=-3556.8379 lr= 0.000100
  3783. 2019-03-13 01:54:31,878 : epoch 419 , step 80 loss0= 0.0000 loss1= 2.1284 loss2=-3889.7244 lr= 0.000100
  3784. 2019-03-13 01:55:08,050 : epoch 419 , step 120 loss0= 0.0000 loss1= 3.0311 loss2=-3867.2844 lr= 0.000100
  3785. 2019-03-13 01:55:43,668 : epoch 419 , step 160 loss0= 0.0000 loss1= 1.7796 loss2=-3540.0857 lr= 0.000100
  3786. 2019-03-13 01:56:16,624 : epoch 420 , step 0 loss0= 0.0000 loss1= 1.5733 loss2=-3592.9419 lr= 0.000100
  3787. 2019-03-13 01:56:52,716 : epoch 420 , step 40 loss0= 0.0000 loss1= 2.3166 loss2=-3465.9119 lr= 0.000100
  3788. 2019-03-13 01:57:29,315 : epoch 420 , step 80 loss0= 0.0000 loss1= 2.0198 loss2=-3803.5657 lr= 0.000100
  3789. 2019-03-13 01:58:05,245 : epoch 420 , step 120 loss0= 0.0000 loss1= 2.0263 loss2=-3463.0369 lr= 0.000100
  3790. 2019-03-13 01:58:41,252 : epoch 420 , step 160 loss0= 0.0000 loss1= 2.1449 loss2=-3965.1609 lr= 0.000100
  3791. 2019-03-13 01:59:12,957 : ## train ##
  3792. 2019-03-13 02:04:09,311 : >> epoch 420 : average_loss= 0.8216, average_accuracy=82.849%, mean_weight_std= 0.4544, mean_normalized_std= 0.6635 mean_dist=3.058e+03
  3793. 2019-03-13 02:04:09,321 : >> epoch 420 : median_loss= 0.8125, median_accuracy=83.010%, median_weight_std= 0.2727, median_normalized_std= 0.1201 median_dist=2.983e+03
  3794. 2019-03-13 02:04:09,321 : >> epoch 420 : ensemble accuracy: 85.51%
  3795. 2019-03-13 02:04:09,321 : >> epoch 420 : loss histogram: (0.00,0.36, 18),(0.36,0.73, 0),(0.73,1.09, 4),(1.09,1.46, 0),(1.46,1.82, 18),
  3796. 2019-03-13 02:04:09,322 : >> epoch 420 : accuracy histogram: (0.00,33.89, 18),(33.89,67.78, 0),(67.78,101.66, 4),(101.66,135.55, 0),(135.55,169.44, 18),
  3797. 2019-03-13 02:04:09,328 : >> epoch 420 : weight std histogram: (0.0000,3.5669, 463616),(3.5669,7.1337, 2901),(7.1337,10.7006, 334),(10.7006,14.2674, 47),(14.2674,17.8343, 8),
  3798. 2019-03-13 02:04:09,333 : >> epoch 420 : weight normalized std histogram: (0.0000,8469.6313, 466904),(8469.6313,16939.2625, 0),(16939.2625,25408.8938, 1),(25408.8938,33878.5250, 0),(33878.5250,42348.1562, 1),
  3799. 2019-03-13 02:04:09,334 : >> epoch 420 : dist histogram: (2654.2017,2896.7432, 6),(2896.7432,3139.2847, 10),(3139.2847,3381.8262, 1),(3381.8262,3624.3677, 1),(3624.3677,3866.9092, 2),
  3800. 2019-03-13 02:04:09,334 : ## eval ##
  3801. 2019-03-13 02:09:03,502 : >> epoch 420 : average_loss= 0.7124, average_accuracy=86.208%, mean_weight_std= 0.4513, mean_normalized_std= 0.9677 mean_dist=3.061e+03
  3802. 2019-03-13 02:09:03,512 : >> epoch 420 : median_loss= 0.6958, median_accuracy=86.685%, median_weight_std= 0.2603, median_normalized_std= 0.1148 median_dist=2.995e+03
  3803. 2019-03-13 02:09:03,512 : >> epoch 420 : ensemble accuracy: 89.27%
  3804. 2019-03-13 02:09:03,512 : >> epoch 420 : loss histogram: (0.00,0.32, 18),(0.32,0.63, 0),(0.63,0.95, 4),(0.95,1.26, 0),(1.26,1.58, 18),
  3805. 2019-03-13 02:09:03,512 : >> epoch 420 : accuracy histogram: (0.00,35.27, 18),(35.27,70.55, 0),(70.55,105.82, 4),(105.82,141.10, 0),(141.10,176.37, 18),
  3806. 2019-03-13 02:09:03,518 : >> epoch 420 : weight std histogram: (0.0000,3.7200, 463531),(3.7200,7.4400, 2988),(7.4400,11.1600, 350),(11.1600,14.8800, 33),(14.8800,18.6000, 4),
  3807. 2019-03-13 02:09:03,523 : >> epoch 420 : weight normalized std histogram: (0.0000,28630.5250, 466905),(28630.5250,57261.0500, 0),(57261.0500,85891.5750, 0),(85891.5750,114522.1000, 0),(114522.1000,143152.6250, 1),
  3808. 2019-03-13 02:09:03,524 : >> epoch 420 : dist histogram: (2690.6958,2927.6133, 8),(2927.6133,3164.5308, 6),(3164.5308,3401.4482, 4),(3401.4482,3638.3657, 1),(3638.3657,3875.2832, 1),
  3809. 2019-03-13 02:09:03,579 : >> epoch 420 : dist pair-wise
  3810. 2019-03-13 02:09:03,585 :
  3811. [[ 0. 1555.815 1403.661 1337.083 1253.664 1275.844 1349.579
  3812. 1366.872 1534.149 1335.205 1341.478 1392.329 1580.721 1233.528
  3813. 1290.276 1486.946 1464.662 1373.431 1559.57 1451.658]
  3814. [ 1555.815 0. 323.975 529.809 638.369 799.633 469.505
  3815. 673.364 255.389 1062.446 841.735 776.074 355.364 532.164
  3816. 554.565 254.965 467.494 415.613 340.623 297.985]
  3817. [ 1403.661 323.975 0. 414.505 492.3 661.294 430.014
  3818. 568.199 308.275 915.814 766.785 767.385 512.28 410.271
  3819. 377.164 337.278 472.853 335.995 402.371 270.803]
  3820. [ 1337.083 529.809 414.505 0. 547.17 661.873 582.135
  3821. 471.69 518.532 756.957 866.696 891.39 708.507 500.501
  3822. 453.287 539.635 608.43 410.968 567.461 492.415]
  3823. [ 1253.664 638.369 492.3 547.17 0. 286.948 519.83
  3824. 638.198 627.056 881.958 749.506 784.34 710.916 518.063
  3825. 487.65 592.929 682.687 553.735 692.846 543.55 ]
  3826. [ 1275.844 799.633 661.294 661.873 286.948 0. 678.824
  3827. 777.756 784.709 934.456 830.174 883.765 854.13 686.671
  3828. 618.007 764.089 787.707 738.549 863.816 723.091]
  3829. [ 1349.579 469.505 430.014 582.135 519.83 678.824 0.
  3830. 668.015 494.531 1038.08 967.169 925.616 627.874 311.683
  3831. 543.2 514.119 583.889 398.327 377.948 458.151]
  3832. [ 1366.872 673.364 568.199 471.69 638.198 777.756 668.015
  3833. 0. 639.775 567.908 929.502 933.899 771.039 621.225
  3834. 645.624 663.266 787.498 465.186 599.107 602.731]
  3835. [ 1534.149 255.389 308.275 518.532 627.056 784.709 494.531
  3836. 639.775 0. 1005.06 800.495 747.39 399.205 560.408
  3837. 579.726 347.817 401.587 392.912 383.949 360.663]
  3838. [ 1335.205 1062.446 915.814 756.957 881.958 934.456 1038.08
  3839. 567.908 1005.06 0. 1047.814 1103.214 1142.855 941.242
  3840. 878.17 1029.604 1054.992 865.25 1026.101 975.971]
  3841. [ 1341.478 841.735 766.785 866.696 749.506 830.174 967.169
  3842. 929.502 800.495 1047.814 0. 265.101 707.725 912.007
  3843. 804.406 737.751 834.648 905.796 1018.447 755.914]
  3844. [ 1392.329 776.074 767.385 891.39 784.34 883.765 925.616
  3845. 933.899 747.39 1103.214 265.101 0. 593.511 917.393
  3846. 841.904 690.971 781.938 882.542 961.202 744.181]
  3847. [ 1580.721 355.364 512.28 708.507 710.916 854.13 627.874
  3848. 771.039 399.205 1142.855 707.725 593.511 0. 704.507
  3849. 709.646 364.197 570.774 604.394 551.41 449.098]
  3850. [ 1233.528 532.164 410.271 500.501 518.063 686.671 311.683
  3851. 621.225 560.408 941.242 912.007 917.393 704.507 0.
  3852. 391.069 496.305 663.009 359.741 462.433 416.517]
  3853. [ 1290.276 554.565 377.164 453.287 487.65 618.007 543.2
  3854. 645.624 579.726 878.17 804.406 841.904 709.646 391.069
  3855. 0. 489.494 634.462 464.621 599.07 446.878]
  3856. [ 1486.946 254.965 337.278 539.635 592.929 764.089 514.119
  3857. 663.266 347.817 1029.604 737.751 690.971 364.197 496.305
  3858. 489.494 0. 564.292 436.566 436.545 231.414]
  3859. [ 1464.662 467.494 472.853 608.43 682.687 787.707 583.889
  3860. 787.498 401.587 1054.992 834.648 781.938 570.774 663.009
  3861. 634.462 564.292 0. 582.255 612.844 565.977]
  3862. [ 1373.431 415.613 335.995 410.968 553.735 738.549 398.327
  3863. 465.186 392.912 865.25 905.796 882.542 604.394 359.741
  3864. 464.621 436.566 582.255 0. 318.182 394.668]
  3865. [ 1559.57 340.623 402.371 567.461 692.846 863.816 377.948
  3866. 599.107 383.949 1026.101 1018.447 961.202 551.41 462.433
  3867. 599.07 436.545 612.844 318.182 0. 401.566]
  3868. [ 1451.658 297.985 270.803 492.415 543.55 723.091 458.151
  3869. 602.731 360.663 975.971 755.914 744.181 449.098 416.517
  3870. 446.878 231.414 565.977 394.668 401.566 0. ]]
  3871. 2019-03-13 02:09:05,052 : epoch 421 , step 0 loss0= 0.0000 loss1= 2.0244 loss2=-3732.4316 lr= 0.000100
  3872. 2019-03-13 02:09:41,084 : epoch 421 , step 40 loss0= 0.0000 loss1= 1.6526 loss2=-3519.2693 lr= 0.000100
  3873. 2019-03-13 02:10:16,856 : epoch 421 , step 80 loss0= 0.0000 loss1= 1.8399 loss2=-3704.1345 lr= 0.000100
  3874. 2019-03-13 02:10:52,886 : epoch 421 , step 120 loss0= 0.0000 loss1= 1.6058 loss2=-3429.2773 lr= 0.000100
  3875. 2019-03-13 02:11:28,360 : epoch 421 , step 160 loss0= 0.0000 loss1= 2.6451 loss2=-3625.3723 lr= 0.000100
  3876. 2019-03-13 02:12:00,598 : epoch 422 , step 0 loss0= 0.0000 loss1= 2.0969 loss2=-3563.8105 lr= 0.000100
  3877. 2019-03-13 02:12:36,977 : epoch 422 , step 40 loss0= 0.0000 loss1= 1.6361 loss2=-3550.4790 lr= 0.000100
  3878. 2019-03-13 02:13:13,554 : epoch 422 , step 80 loss0= 0.0000 loss1= 2.7159 loss2=-3754.1282 lr= 0.000100
  3879. 2019-03-13 02:13:49,632 : epoch 422 , step 120 loss0= 0.0000 loss1= 1.7438 loss2=-3698.6899 lr= 0.000100
  3880. 2019-03-13 02:14:25,392 : epoch 422 , step 160 loss0= 0.0000 loss1= 2.9176 loss2=-3643.6775 lr= 0.000100
  3881. 2019-03-13 02:14:57,895 : epoch 423 , step 0 loss0= 0.0000 loss1= 1.8976 loss2=-3713.5515 lr= 0.000100
  3882. 2019-03-13 02:15:33,492 : epoch 423 , step 40 loss0= 0.0000 loss1= 1.5915 loss2=-3127.2485 lr= 0.000100
  3883. 2019-03-13 02:16:09,027 : epoch 423 , step 80 loss0= 0.0000 loss1= 2.3398 loss2=-3610.8130 lr= 0.000100
  3884. 2019-03-13 02:16:45,006 : epoch 423 , step 120 loss0= 0.0000 loss1= 1.7238 loss2=-3684.1736 lr= 0.000100
  3885. 2019-03-13 02:17:20,870 : epoch 423 , step 160 loss0= 0.0000 loss1= 2.4561 loss2=-3565.3306 lr= 0.000100
  3886. 2019-03-13 02:17:53,368 : epoch 424 , step 0 loss0= 0.0000 loss1= 2.2005 loss2=-4007.3191 lr= 0.000100
  3887. 2019-03-13 02:18:29,546 : epoch 424 , step 40 loss0= 0.0000 loss1= 1.7594 loss2=-3677.9773 lr= 0.000100
  3888. 2019-03-13 02:19:05,947 : epoch 424 , step 80 loss0= 0.0000 loss1= 2.5280 loss2=-3773.3035 lr= 0.000100
  3889. 2019-03-13 02:19:42,513 : epoch 424 , step 120 loss0= 0.0000 loss1= 2.1097 loss2=-3786.9478 lr= 0.000100
  3890. 2019-03-13 02:20:19,050 : epoch 424 , step 160 loss0= 0.0000 loss1= 2.7376 loss2=-3872.2009 lr= 0.000100
  3891. 2019-03-13 02:20:51,914 : epoch 425 , step 0 loss0= 0.0000 loss1= 2.4934 loss2=-3843.0278 lr= 0.000100
  3892. 2019-03-13 02:21:27,899 : epoch 425 , step 40 loss0= 0.0000 loss1= 1.9543 loss2=-3673.7517 lr= 0.000100
  3893. 2019-03-13 02:22:03,877 : epoch 425 , step 80 loss0= 0.0000 loss1= 1.8375 loss2=-3741.5115 lr= 0.000100
  3894. 2019-03-13 02:22:39,747 : epoch 425 , step 120 loss0= 0.0000 loss1= 1.0550 loss2=-3289.8689 lr= 0.000100
  3895. 2019-03-13 02:23:15,917 : epoch 425 , step 160 loss0= 0.0000 loss1= 2.2110 loss2=-3264.7256 lr= 0.000100
  3896. 2019-03-13 02:23:48,852 : epoch 426 , step 0 loss0= 0.0000 loss1= 2.3225 loss2=-3840.4055 lr= 0.000100
  3897. 2019-03-13 02:24:25,195 : epoch 426 , step 40 loss0= 0.0000 loss1= 2.2649 loss2=-3573.1306 lr= 0.000100
  3898. 2019-03-13 02:25:01,022 : epoch 426 , step 80 loss0= 0.0000 loss1= 2.4468 loss2=-3744.0779 lr= 0.000100
  3899. 2019-03-13 02:25:36,680 : epoch 426 , step 120 loss0= 0.0000 loss1= 1.7284 loss2=-3619.4705 lr= 0.000100
  3900. 2019-03-13 02:26:12,261 : epoch 426 , step 160 loss0= 0.0000 loss1= 2.2382 loss2=-4016.1079 lr= 0.000100
  3901. 2019-03-13 02:26:44,766 : epoch 427 , step 0 loss0= 0.0000 loss1= 1.6874 loss2=-3713.7063 lr= 0.000100
  3902. 2019-03-13 02:27:20,986 : epoch 427 , step 40 loss0= 0.0000 loss1= 2.1524 loss2=-3836.4626 lr= 0.000100
  3903. 2019-03-13 02:27:57,576 : epoch 427 , step 80 loss0= 0.0000 loss1= 1.7577 loss2=-3845.3242 lr= 0.000100
  3904. 2019-03-13 02:28:33,942 : epoch 427 , step 120 loss0= 0.0000 loss1= 2.0842 loss2=-3639.7935 lr= 0.000100
  3905. 2019-03-13 02:29:10,345 : epoch 427 , step 160 loss0= 0.0000 loss1= 2.0028 loss2=-3549.6821 lr= 0.000100
  3906. 2019-03-13 02:29:42,773 : epoch 428 , step 0 loss0= 0.0000 loss1= 2.5709 loss2=-3674.2771 lr= 0.000100
  3907. 2019-03-13 02:30:18,957 : epoch 428 , step 40 loss0= 0.0000 loss1= 2.4112 loss2=-3903.9470 lr= 0.000100
  3908. 2019-03-13 02:30:55,271 : epoch 428 , step 80 loss0= 0.0000 loss1= 2.2949 loss2=-3889.3379 lr= 0.000100
  3909. 2019-03-13 02:31:31,754 : epoch 428 , step 120 loss0= 0.0000 loss1= 2.7571 loss2=-3572.4851 lr= 0.000100
  3910. 2019-03-13 02:32:07,414 : epoch 428 , step 160 loss0= 0.0000 loss1= 2.4151 loss2=-3853.1990 lr= 0.000100
  3911. 2019-03-13 02:32:40,366 : epoch 429 , step 0 loss0= 0.0000 loss1= 1.9434 loss2=-3797.1633 lr= 0.000100
  3912. 2019-03-13 02:33:16,958 : epoch 429 , step 40 loss0= 0.0000 loss1= 1.8707 loss2=-3655.9275 lr= 0.000100
  3913. 2019-03-13 02:33:53,528 : epoch 429 , step 80 loss0= 0.0000 loss1= 2.2352 loss2=-3844.6033 lr= 0.000100
  3914. 2019-03-13 02:34:29,698 : epoch 429 , step 120 loss0= 0.0000 loss1= 2.7861 loss2=-3826.4192 lr= 0.000100
  3915. 2019-03-13 02:35:05,253 : epoch 429 , step 160 loss0= 0.0000 loss1= 1.7028 loss2=-3800.9353 lr= 0.000100
  3916. 2019-03-13 02:35:38,473 : epoch 430 , step 0 loss0= 0.0000 loss1= 1.7538 loss2=-3400.4204 lr= 0.000100
  3917. 2019-03-13 02:36:14,324 : epoch 430 , step 40 loss0= 0.0000 loss1= 2.2878 loss2=-3807.4668 lr= 0.000100
  3918. 2019-03-13 02:36:50,052 : epoch 430 , step 80 loss0= 0.0000 loss1= 1.6420 loss2=-3719.5239 lr= 0.000100
  3919. 2019-03-13 02:37:26,443 : epoch 430 , step 120 loss0= 0.0000 loss1= 1.6798 loss2=-3657.4138 lr= 0.000100
  3920. 2019-03-13 02:38:03,119 : epoch 430 , step 160 loss0= 0.0000 loss1= 2.3448 loss2=-3840.5632 lr= 0.000100
  3921. 2019-03-13 02:38:36,220 : epoch 431 , step 0 loss0= 0.0000 loss1= 1.5192 loss2=-3319.9539 lr= 0.000100
  3922. 2019-03-13 02:39:12,660 : epoch 431 , step 40 loss0= 0.0000 loss1= 2.0985 loss2=-3659.1648 lr= 0.000100
  3923. 2019-03-13 02:39:48,880 : epoch 431 , step 80 loss0= 0.0000 loss1= 1.1901 loss2=-3320.6570 lr= 0.000100
  3924. 2019-03-13 02:40:24,160 : epoch 431 , step 120 loss0= 0.0000 loss1= 2.2288 loss2=-3955.4148 lr= 0.000100
  3925. 2019-03-13 02:40:59,735 : epoch 431 , step 160 loss0= 0.0000 loss1= 1.9373 loss2=-3848.9275 lr= 0.000100
  3926. 2019-03-13 02:41:32,519 : epoch 432 , step 0 loss0= 0.0000 loss1= 1.9868 loss2=-3656.7446 lr= 0.000100
  3927. 2019-03-13 02:42:08,194 : epoch 432 , step 40 loss0= 0.0000 loss1= 2.0357 loss2=-3628.7166 lr= 0.000100
  3928. 2019-03-13 02:42:44,076 : epoch 432 , step 80 loss0= 0.0000 loss1= 1.8913 loss2=-3573.2478 lr= 0.000100
  3929. 2019-03-13 02:43:20,010 : epoch 432 , step 120 loss0= 0.0000 loss1= 3.4707 loss2=-4214.0708 lr= 0.000100
  3930. 2019-03-13 02:43:55,965 : epoch 432 , step 160 loss0= 0.0000 loss1= 2.4928 loss2=-3588.6238 lr= 0.000100
  3931. 2019-03-13 02:44:28,516 : epoch 433 , step 0 loss0= 0.0000 loss1= 1.8581 loss2=-3689.1152 lr= 0.000100
  3932. 2019-03-13 02:45:04,085 : epoch 433 , step 40 loss0= 0.0000 loss1= 2.3616 loss2=-3935.3496 lr= 0.000100
  3933. 2019-03-13 02:45:39,568 : epoch 433 , step 80 loss0= 0.0000 loss1= 1.8121 loss2=-3634.1003 lr= 0.000100
  3934. 2019-03-13 02:46:15,031 : epoch 433 , step 120 loss0= 0.0000 loss1= 1.9211 loss2=-3785.7656 lr= 0.000100
  3935. 2019-03-13 02:46:50,568 : epoch 433 , step 160 loss0= 0.0000 loss1= 2.0174 loss2=-3446.6790 lr= 0.000100
  3936. 2019-03-13 02:47:23,060 : epoch 434 , step 0 loss0= 0.0000 loss1= 2.3483 loss2=-3674.1079 lr= 0.000100
  3937. 2019-03-13 02:47:59,216 : epoch 434 , step 40 loss0= 0.0000 loss1= 2.0968 loss2=-3585.4607 lr= 0.000100
  3938. 2019-03-13 02:48:35,028 : epoch 434 , step 80 loss0= 0.0000 loss1= 2.0283 loss2=-3672.7922 lr= 0.000100
  3939. 2019-03-13 02:49:10,965 : epoch 434 , step 120 loss0= 0.0000 loss1= 1.6309 loss2=-3615.3774 lr= 0.000100
  3940. 2019-03-13 02:49:47,045 : epoch 434 , step 160 loss0= 0.0000 loss1= 1.9269 loss2=-3858.8926 lr= 0.000100
  3941. 2019-03-13 02:50:19,677 : epoch 435 , step 0 loss0= 0.0000 loss1= 1.8757 loss2=-3907.5583 lr= 0.000100
  3942. 2019-03-13 02:50:55,293 : epoch 435 , step 40 loss0= 0.0000 loss1= 2.3392 loss2=-3888.3274 lr= 0.000100
  3943. 2019-03-13 02:51:30,920 : epoch 435 , step 80 loss0= 0.0000 loss1= 1.6900 loss2=-3683.0818 lr= 0.000100
  3944. 2019-03-13 02:52:06,776 : epoch 435 , step 120 loss0= 0.0000 loss1= 2.8211 loss2=-3830.3677 lr= 0.000100
  3945. 2019-03-13 02:52:43,231 : epoch 435 , step 160 loss0= 0.0000 loss1= 2.1405 loss2=-3983.2976 lr= 0.000100
  3946. 2019-03-13 02:53:16,121 : epoch 436 , step 0 loss0= 0.0000 loss1= 3.2390 loss2=-3794.5281 lr= 0.000100
  3947. 2019-03-13 02:53:52,009 : epoch 436 , step 40 loss0= 0.0000 loss1= 2.2402 loss2=-3777.1912 lr= 0.000100
  3948. 2019-03-13 02:54:27,660 : epoch 436 , step 80 loss0= 0.0000 loss1= 2.5177 loss2=-3759.2336 lr= 0.000100
  3949. 2019-03-13 02:55:04,255 : epoch 436 , step 120 loss0= 0.0000 loss1= 1.9814 loss2=-3856.3364 lr= 0.000100
  3950. 2019-03-13 02:55:40,922 : epoch 436 , step 160 loss0= 0.0000 loss1= 2.3881 loss2=-3874.3328 lr= 0.000100
  3951. 2019-03-13 02:56:14,089 : epoch 437 , step 0 loss0= 0.0000 loss1= 1.4634 loss2=-3670.5332 lr= 0.000100
  3952. 2019-03-13 02:56:49,640 : epoch 437 , step 40 loss0= 0.0000 loss1= 2.4281 loss2=-3773.9500 lr= 0.000100
  3953. 2019-03-13 02:57:25,481 : epoch 437 , step 80 loss0= 0.0000 loss1= 1.9662 loss2=-3590.6111 lr= 0.000100
  3954. 2019-03-13 02:58:02,143 : epoch 437 , step 120 loss0= 0.0000 loss1= 1.7278 loss2=-3626.6501 lr= 0.000100
  3955. 2019-03-13 02:58:38,540 : epoch 437 , step 160 loss0= 0.0000 loss1= 1.5221 loss2=-3525.1855 lr= 0.000100
  3956. 2019-03-13 02:59:11,067 : epoch 438 , step 0 loss0= 0.0000 loss1= 2.8181 loss2=-3628.2439 lr= 0.000100
  3957. 2019-03-13 02:59:47,258 : epoch 438 , step 40 loss0= 0.0000 loss1= 1.8889 loss2=-4010.0625 lr= 0.000100
  3958. 2019-03-13 03:00:23,046 : epoch 438 , step 80 loss0= 0.0000 loss1= 2.4745 loss2=-3742.4199 lr= 0.000100
  3959. 2019-03-13 03:00:58,676 : epoch 438 , step 120 loss0= 0.0000 loss1= 1.8889 loss2=-3435.2454 lr= 0.000100
  3960. 2019-03-13 03:01:34,261 : epoch 438 , step 160 loss0= 0.0000 loss1= 2.2130 loss2=-3879.0903 lr= 0.000100
  3961. 2019-03-13 03:02:06,808 : epoch 439 , step 0 loss0= 0.0000 loss1= 2.6056 loss2=-3837.0566 lr= 0.000100
  3962. 2019-03-13 03:02:43,242 : epoch 439 , step 40 loss0= 0.0000 loss1= 2.1951 loss2=-3779.7168 lr= 0.000100
  3963. 2019-03-13 03:03:19,530 : epoch 439 , step 80 loss0= 0.0000 loss1= 2.5574 loss2=-3798.4438 lr= 0.000100
  3964. 2019-03-13 03:03:55,766 : epoch 439 , step 120 loss0= 0.0000 loss1= 2.9637 loss2=-4171.1621 lr= 0.000100
  3965. 2019-03-13 03:04:32,095 : epoch 439 , step 160 loss0= 0.0000 loss1= 1.8396 loss2=-3923.3755 lr= 0.000100
  3966. 2019-03-13 03:05:05,120 : epoch 440 , step 0 loss0= 0.0000 loss1= 2.4439 loss2=-3714.4185 lr= 0.000100
  3967. 2019-03-13 03:05:41,069 : epoch 440 , step 40 loss0= 0.0000 loss1= 2.6888 loss2=-3362.2507 lr= 0.000100
  3968. 2019-03-13 03:06:16,506 : epoch 440 , step 80 loss0= 0.0000 loss1= 1.7513 loss2=-3821.2324 lr= 0.000100
  3969. 2019-03-13 03:06:52,013 : epoch 440 , step 120 loss0= 0.0000 loss1= 1.6182 loss2=-3553.4978 lr= 0.000100
  3970. 2019-03-13 03:07:28,094 : epoch 440 , step 160 loss0= 0.0000 loss1= 1.6841 loss2=-3839.1099 lr= 0.000100
  3971. 2019-03-13 03:07:59,128 : ## train ##
  3972. 2019-03-13 03:12:54,801 : >> epoch 440 : average_loss= 0.8496, average_accuracy=82.906%, mean_weight_std= 0.4673, mean_normalized_std= 0.9335 mean_dist=3.102e+03
  3973. 2019-03-13 03:12:54,810 : >> epoch 440 : median_loss= 0.8448, median_accuracy=83.355%, median_weight_std= 0.2765, median_normalized_std= 0.1207 median_dist=3.074e+03
  3974. 2019-03-13 03:12:54,810 : >> epoch 440 : ensemble accuracy: 85.43%
  3975. 2019-03-13 03:12:54,811 : >> epoch 440 : loss histogram: (0.00,0.36, 18),(0.36,0.72, 0),(0.72,1.08, 4),(1.08,1.44, 0),(1.44,1.80, 18),
  3976. 2019-03-13 03:12:54,811 : >> epoch 440 : accuracy histogram: (0.00,33.90, 18),(33.90,67.80, 0),(67.80,101.70, 4),(101.70,135.60, 0),(135.60,169.50, 18),
  3977. 2019-03-13 03:12:54,817 : >> epoch 440 : weight std histogram: (0.0000,3.7444, 463487),(3.7444,7.4887, 2990),(7.4887,11.2331, 363),(11.2331,14.9775, 58),(14.9775,18.7219, 8),
  3978. 2019-03-13 03:12:54,822 : >> epoch 440 : weight normalized std histogram: (0.0000,24316.3969, 466905),(24316.3969,48632.7937, 0),(48632.7937,72949.1906, 0),(72949.1906,97265.5875, 0),(97265.5875,121581.9844, 1),
  3979. 2019-03-13 03:12:54,823 : >> epoch 440 : dist histogram: (2641.4397,2902.1132, 5),(2902.1132,3162.7867, 10),(3162.7867,3423.4602, 2),(3423.4602,3684.1336, 2),(3684.1336,3944.8071, 1),
  3980. 2019-03-13 03:12:54,823 : ## eval ##
  3981. 2019-03-13 03:17:49,428 : >> epoch 440 : average_loss= 0.7398, average_accuracy=85.987%, mean_weight_std= 0.4187, mean_normalized_std= 0.7546 mean_dist=3.104e+03
  3982. 2019-03-13 03:17:49,437 : >> epoch 440 : median_loss= 0.7159, median_accuracy=87.175%, median_weight_std= 0.2293, median_normalized_std= 0.1012 median_dist=3.032e+03
  3983. 2019-03-13 03:17:49,437 : >> epoch 440 : ensemble accuracy: 88.90%
  3984. 2019-03-13 03:17:49,438 : >> epoch 440 : loss histogram: (0.00,0.32, 18),(0.32,0.63, 0),(0.63,0.95, 4),(0.95,1.26, 0),(1.26,1.58, 18),
  3985. 2019-03-13 03:17:49,438 : >> epoch 440 : accuracy histogram: (0.00,35.02, 18),(35.02,70.03, 0),(70.03,105.05, 4),(105.05,140.06, 0),(140.06,175.08, 18),
  3986. 2019-03-13 03:17:49,444 : >> epoch 440 : weight std histogram: (0.0000,3.5947, 463218),(3.5947,7.1895, 3251),(7.1895,10.7842, 374),(10.7842,14.3789, 58),(14.3789,17.9737, 5),
  3987. 2019-03-13 03:17:49,449 : >> epoch 440 : weight normalized std histogram: (0.0000,6977.7234, 466899),(6977.7234,13955.4469, 5),(13955.4469,20933.1703, 1),(20933.1703,27910.8937, 0),(27910.8937,34888.6172, 1),
  3988. 2019-03-13 03:17:49,450 : >> epoch 440 : dist histogram: (2783.6245,2945.5105, 5),(2945.5105,3107.3966, 7),(3107.3966,3269.2826, 4),(3269.2826,3431.1687, 2),(3431.1687,3593.0547, 2),
  3989. 2019-03-13 03:17:49,506 : >> epoch 440 : dist pair-wise
  3990. 2019-03-13 03:17:49,512 :
  3991. [[ 0. 389.901 714.428 1157.449 501.76 812.3 846.242
  3992. 925.015 643.943 307.729 497.45 372.182 591.512 284.802
  3993. 678.025 950.094 556.129 1307.835 519.561 845.22 ]
  3994. [ 389.901 0. 529.504 1110.576 458.784 749.203 737.648
  3995. 719.541 586.602 391.977 454.568 473.42 783.246 348.321
  3996. 668.691 977.965 511.961 1140.7 416.117 628.424]
  3997. [ 714.428 529.504 0. 918.082 461.767 734.835 544.955
  3998. 490.512 520.285 661.944 452.932 604.448 897.903 636.478
  3999. 668.906 853.889 598.172 929.482 520.483 470.544]
  4000. [ 1157.449 1110.576 918.082 0. 996.547 1045.657 859.404
  4001. 921.8 1108.321 1041.199 972.829 983.434 1016.169 1064.447
  4002. 1084.652 1040.322 1029.708 1027.124 991.108 906.698]
  4003. [ 501.76 458.784 461.767 996.547 0. 634.199 638.607
  4004. 776.181 337.676 531.323 175.64 414.464 801.364 548.36
  4005. 422.965 722.659 395.964 1164.979 379.023 711.655]
  4006. [ 812.3 749.203 734.835 1045.657 634.199 0. 590.319
  4007. 862.842 694.192 725.617 617.474 681.684 923.677 786.023
  4008. 685.029 874.102 376.942 1059.513 495.649 816.658]
  4009. [ 846.242 737.648 544.955 859.404 638.607 590.319 0.
  4010. 675.846 698.408 673.083 584.952 580.376 827.055 708.31
  4011. 767.977 866.603 580.297 950.996 507.206 603.503]
  4012. [ 925.015 719.541 490.512 921.8 776.181 862.842 675.846
  4013. 0. 747.495 821.731 729.537 831.643 991.324 779.719
  4014. 857.503 939.907 781.708 592.447 721.119 262.471]
  4015. [ 643.943 586.602 520.285 1108.321 337.676 694.192 698.408
  4016. 747.495 0. 675.971 328.81 588.423 916.126 667.128
  4017. 356.873 616.874 486.09 1146.959 531.627 735.89 ]
  4018. [ 307.729 391.977 661.944 1041.199 531.323 725.617 673.083
  4019. 821.731 675.971 0. 508.251 287.553 500.493 204.625
  4020. 709.369 937.582 512.979 1176.91 432.003 721.129]
  4021. [ 497.45 454.568 452.932 972.829 175.64 617.474 584.952
  4022. 729.537 328.81 508.251 0. 395.758 775.99 518.952
  4023. 449.646 730.222 356.235 1130.076 341.156 673.64 ]
  4024. [ 372.182 473.42 604.448 983.434 414.464 681.684 580.376
  4025. 831.643 588.423 287.553 395.758 0. 519.713 337.438
  4026. 611.691 811.769 469.256 1189.847 380.843 741.665]
  4027. [ 591.512 783.246 897.903 1016.169 801.364 923.677 827.055
  4028. 991.324 916.126 500.493 775.99 519.713 0. 520.345
  4029. 878.342 950.723 803.306 1227.644 736.233 924.139]
  4030. [ 284.802 348.321 636.478 1064.447 548.36 786.023 708.31
  4031. 779.719 667.128 204.625 518.952 337.438 520.345 0.
  4032. 719.221 947.482 562.706 1164.059 472.846 692.716]
  4033. [ 678.025 668.691 668.906 1084.652 422.965 685.029 767.977
  4034. 857.503 356.873 709.369 449.646 611.691 878.342 719.221
  4035. 0. 472.432 525.049 1172.132 580.363 829.158]
  4036. [ 950.094 977.965 853.889 1040.322 722.659 874.102 866.603
  4037. 939.907 616.874 937.582 730.222 811.769 950.723 947.482
  4038. 472.432 0. 791.29 1169.203 847.53 957.998]
  4039. [ 556.129 511.961 598.172 1029.708 395.964 376.942 580.297
  4040. 781.708 486.09 512.979 356.235 469.256 803.306 562.706
  4041. 525.049 791.29 0. 1111.948 298.098 733.679]
  4042. [ 1307.835 1140.7 929.482 1027.124 1164.979 1059.513 950.996
  4043. 592.447 1146.959 1176.91 1130.076 1189.847 1227.644 1164.059
  4044. 1172.132 1169.203 1111.948 0. 1069.794 641.555]
  4045. [ 519.561 416.117 520.483 991.108 379.023 495.649 507.206
  4046. 721.119 531.627 432.003 341.156 380.843 736.233 472.846
  4047. 580.363 847.53 298.098 1069.794 0. 641.159]
  4048. [ 845.22 628.424 470.544 906.698 711.655 816.658 603.503
  4049. 262.471 735.89 721.129 673.64 741.665 924.139 692.716
  4050. 829.158 957.998 733.679 641.555 641.159 0. ]]
  4051. 2019-03-13 03:17:51,000 : epoch 441 , step 0 loss0= 0.0000 loss1= 1.8552 loss2=-3683.4373 lr= 0.000100
  4052. 2019-03-13 03:18:27,184 : epoch 441 , step 40 loss0= 0.0000 loss1= 2.2350 loss2=-4240.8472 lr= 0.000100
  4053. 2019-03-13 03:19:02,987 : epoch 441 , step 80 loss0= 0.0000 loss1= 2.0323 loss2=-3798.6438 lr= 0.000100
  4054. 2019-03-13 03:19:38,570 : epoch 441 , step 120 loss0= 0.0000 loss1= 2.7686 loss2=-3910.3708 lr= 0.000100
  4055. 2019-03-13 03:20:14,845 : epoch 441 , step 160 loss0= 0.0000 loss1= 1.6343 loss2=-3685.2976 lr= 0.000100
  4056. 2019-03-13 03:20:47,387 : epoch 442 , step 0 loss0= 0.0000 loss1= 2.1444 loss2=-4017.3281 lr= 0.000100
  4057. 2019-03-13 03:21:23,809 : epoch 442 , step 40 loss0= 0.0000 loss1= 2.3991 loss2=-3445.4849 lr= 0.000100
  4058. 2019-03-13 03:22:00,416 : epoch 442 , step 80 loss0= 0.0000 loss1= 3.7087 loss2=-4172.4644 lr= 0.000100
  4059. 2019-03-13 03:22:36,480 : epoch 442 , step 120 loss0= 0.0000 loss1= 2.2931 loss2=-3830.2141 lr= 0.000100
  4060. 2019-03-13 03:23:12,789 : epoch 442 , step 160 loss0= 0.0000 loss1= 2.2371 loss2=-3689.0117 lr= 0.000100
  4061. 2019-03-13 03:23:45,264 : epoch 443 , step 0 loss0= 0.0000 loss1= 2.1354 loss2=-3785.5825 lr= 0.000100
  4062. 2019-03-13 03:24:21,396 : epoch 443 , step 40 loss0= 0.0000 loss1= 2.5003 loss2=-3679.2969 lr= 0.000100
  4063. 2019-03-13 03:24:56,983 : epoch 443 , step 80 loss0= 0.0000 loss1= 1.8532 loss2=-3743.3242 lr= 0.000100
  4064. 2019-03-13 03:25:32,519 : epoch 443 , step 120 loss0= 0.0000 loss1= 1.9021 loss2=-3546.0330 lr= 0.000100
  4065. 2019-03-13 03:26:07,961 : epoch 443 , step 160 loss0= 0.0000 loss1= 1.3098 loss2=-3181.1384 lr= 0.000100
  4066. 2019-03-13 03:26:40,309 : epoch 444 , step 0 loss0= 0.0000 loss1= 2.6344 loss2=-3830.9712 lr= 0.000100
  4067. 2019-03-13 03:27:16,087 : epoch 444 , step 40 loss0= 0.0000 loss1= 1.5742 loss2=-3589.2180 lr= 0.000100
  4068. 2019-03-13 03:27:51,886 : epoch 444 , step 80 loss0= 0.0000 loss1= 3.1632 loss2=-3658.2720 lr= 0.000100
  4069. 2019-03-13 03:28:27,847 : epoch 444 , step 120 loss0= 0.0000 loss1= 1.7490 loss2=-3721.0017 lr= 0.000100
  4070. 2019-03-13 03:29:04,546 : epoch 444 , step 160 loss0= 0.0000 loss1= 1.8972 loss2=-3279.1606 lr= 0.000100
  4071. 2019-03-13 03:29:37,077 : epoch 445 , step 0 loss0= 0.0000 loss1= 1.6966 loss2=-3712.5466 lr= 0.000100
  4072. 2019-03-13 03:30:12,858 : epoch 445 , step 40 loss0= 0.0000 loss1= 1.1932 loss2=-3451.4673 lr= 0.000100
  4073. 2019-03-13 03:30:48,748 : epoch 445 , step 80 loss0= 0.0000 loss1= 1.7701 loss2=-3458.4646 lr= 0.000100
  4074. 2019-03-13 03:31:24,835 : epoch 445 , step 120 loss0= 0.0000 loss1= 1.7489 loss2=-3628.3494 lr= 0.000100
  4075. 2019-03-13 03:32:01,217 : epoch 445 , step 160 loss0= 0.0000 loss1= 1.7262 loss2=-3827.4106 lr= 0.000100
  4076. 2019-03-13 03:32:33,890 : epoch 446 , step 0 loss0= 0.0000 loss1= 2.0701 loss2=-3865.7883 lr= 0.000100
  4077. 2019-03-13 03:33:10,311 : epoch 446 , step 40 loss0= 0.0000 loss1= 1.5963 loss2=-3749.1531 lr= 0.000100
  4078. 2019-03-13 03:33:46,861 : epoch 446 , step 80 loss0= 0.0000 loss1= 2.3753 loss2=-3954.9626 lr= 0.000100
  4079. 2019-03-13 03:34:23,226 : epoch 446 , step 120 loss0= 0.0000 loss1= 1.5237 loss2=-3585.9431 lr= 0.000100
  4080. 2019-03-13 03:34:59,216 : epoch 446 , step 160 loss0= 0.0000 loss1= 1.7345 loss2=-3546.8494 lr= 0.000100
  4081. 2019-03-13 03:35:31,883 : epoch 447 , step 0 loss0= 0.0000 loss1= 1.9058 loss2=-3712.5962 lr= 0.000100
  4082. 2019-03-13 03:36:07,578 : epoch 447 , step 40 loss0= 0.0000 loss1= 2.1256 loss2=-3740.3223 lr= 0.000100
  4083. 2019-03-13 03:36:44,124 : epoch 447 , step 80 loss0= 0.0000 loss1= 2.8154 loss2=-4125.0083 lr= 0.000100
  4084. 2019-03-13 03:37:20,034 : epoch 447 , step 120 loss0= 0.0000 loss1= 2.1530 loss2=-3399.0142 lr= 0.000100
  4085. 2019-03-13 03:37:55,516 : epoch 447 , step 160 loss0= 0.0000 loss1= 1.9484 loss2=-4021.9067 lr= 0.000100
  4086. 2019-03-13 03:38:27,628 : epoch 448 , step 0 loss0= 0.0000 loss1= 1.6335 loss2=-3427.3032 lr= 0.000100
  4087. 2019-03-13 03:39:03,692 : epoch 448 , step 40 loss0= 0.0000 loss1= 1.5757 loss2=-3729.5305 lr= 0.000100
  4088. 2019-03-13 03:39:39,866 : epoch 448 , step 80 loss0= 0.0000 loss1= 2.2782 loss2=-3736.2205 lr= 0.000100
  4089. 2019-03-13 03:40:16,370 : epoch 448 , step 120 loss0= 0.0000 loss1= 1.9605 loss2=-3916.6470 lr= 0.000100
  4090. 2019-03-13 03:40:52,765 : epoch 448 , step 160 loss0= 0.0000 loss1= 1.7671 loss2=-3721.7075 lr= 0.000100
  4091. 2019-03-13 03:41:25,601 : epoch 449 , step 0 loss0= 0.0000 loss1= 2.0776 loss2=-3599.0918 lr= 0.000100
  4092. 2019-03-13 03:42:01,134 : epoch 449 , step 40 loss0= 0.0000 loss1= 1.5981 loss2=-3493.6550 lr= 0.000100
  4093. 2019-03-13 03:42:36,535 : epoch 449 , step 80 loss0= 0.0000 loss1= 2.3912 loss2=-3819.9177 lr= 0.000100
  4094. 2019-03-13 03:43:11,983 : epoch 449 , step 120 loss0= 0.0000 loss1= 1.7833 loss2=-3644.5220 lr= 0.000100
  4095. 2019-03-13 03:43:48,002 : epoch 449 , step 160 loss0= 0.0000 loss1= 2.3811 loss2=-3775.4255 lr= 0.000100
  4096. 2019-03-13 03:44:20,834 : epoch 450 , step 0 loss0= 0.0000 loss1= 1.9508 loss2=-3696.5930 lr= 0.000100
  4097. 2019-03-13 03:44:56,584 : epoch 450 , step 40 loss0= 0.0000 loss1= 3.3078 loss2=-3770.4863 lr= 0.000100
  4098. 2019-03-13 03:45:33,246 : epoch 450 , step 80 loss0= 0.0000 loss1= 2.2864 loss2=-3651.5818 lr= 0.000100
  4099. 2019-03-13 03:46:09,407 : epoch 450 , step 120 loss0= 0.0000 loss1= 2.6419 loss2=-3782.3391 lr= 0.000100
  4100. 2019-03-13 03:46:45,357 : epoch 450 , step 160 loss0= 0.0000 loss1= 2.2289 loss2=-3620.2571 lr= 0.000100
  4101. 2019-03-13 03:47:17,823 : epoch 451 , step 0 loss0= 0.0000 loss1= 2.2557 loss2=-3921.0500 lr= 0.000100
  4102. 2019-03-13 03:47:53,885 : epoch 451 , step 40 loss0= 0.0000 loss1= 2.0811 loss2=-3622.3875 lr= 0.000100
  4103. 2019-03-13 03:48:29,668 : epoch 451 , step 80 loss0= 0.0000 loss1= 1.9394 loss2=-3687.9490 lr= 0.000100
  4104. 2019-03-13 03:49:05,673 : epoch 451 , step 120 loss0= 0.0000 loss1= 2.4903 loss2=-3835.7981 lr= 0.000100
  4105. 2019-03-13 03:49:42,461 : epoch 451 , step 160 loss0= 0.0000 loss1= 1.7636 loss2=-3660.5129 lr= 0.000100
  4106. 2019-03-13 03:50:15,015 : epoch 452 , step 0 loss0= 0.0000 loss1= 1.8597 loss2=-3677.0593 lr= 0.000100
  4107. 2019-03-13 03:50:51,364 : epoch 452 , step 40 loss0= 0.0000 loss1= 1.8753 loss2=-3710.3718 lr= 0.000100
  4108. 2019-03-13 03:51:27,216 : epoch 452 , step 80 loss0= 0.0000 loss1= 1.8280 loss2=-3502.1060 lr= 0.000100
  4109. 2019-03-13 03:52:03,461 : epoch 452 , step 120 loss0= 0.0000 loss1= 1.2225 loss2=-3323.6758 lr= 0.000100
  4110. 2019-03-13 03:52:40,003 : epoch 452 , step 160 loss0= 0.0000 loss1= 1.6115 loss2=-3867.7849 lr= 0.000100
  4111. 2019-03-13 03:53:12,609 : epoch 453 , step 0 loss0= 0.0000 loss1= 2.5100 loss2=-3760.8914 lr= 0.000100
  4112. 2019-03-13 03:53:48,624 : epoch 453 , step 40 loss0= 0.0000 loss1= 1.7446 loss2=-3836.6016 lr= 0.000100
  4113. 2019-03-13 03:54:24,497 : epoch 453 , step 80 loss0= 0.0000 loss1= 2.1178 loss2=-3921.6399 lr= 0.000100
  4114. 2019-03-13 03:55:00,254 : epoch 453 , step 120 loss0= 0.0000 loss1= 2.8617 loss2=-3948.4160 lr= 0.000100
  4115. 2019-03-13 03:55:35,849 : epoch 453 , step 160 loss0= 0.0000 loss1= 1.6350 loss2=-3628.0364 lr= 0.000100
  4116. 2019-03-13 03:56:08,887 : epoch 454 , step 0 loss0= 0.0000 loss1= 1.4193 loss2=-3140.7373 lr= 0.000100
  4117. 2019-03-13 03:56:44,818 : epoch 454 , step 40 loss0= 0.0000 loss1= 2.2097 loss2=-3779.5278 lr= 0.000100
  4118. 2019-03-13 03:57:20,695 : epoch 454 , step 80 loss0= 0.0000 loss1= 1.4880 loss2=-3680.2700 lr= 0.000100
  4119. 2019-03-13 03:57:57,023 : epoch 454 , step 120 loss0= 0.0000 loss1= 2.3385 loss2=-3674.5376 lr= 0.000100
  4120. 2019-03-13 03:58:33,995 : epoch 454 , step 160 loss0= 0.0000 loss1= 1.5769 loss2=-3699.8679 lr= 0.000100
  4121. 2019-03-13 03:59:07,172 : epoch 455 , step 0 loss0= 0.0000 loss1= 1.3225 loss2=-3380.6953 lr= 0.000100
  4122. 2019-03-13 03:59:43,397 : epoch 455 , step 40 loss0= 0.0000 loss1= 1.4925 loss2=-3500.5027 lr= 0.000100
  4123. 2019-03-13 04:00:20,078 : epoch 455 , step 80 loss0= 0.0000 loss1= 2.5745 loss2=-3806.3013 lr= 0.000100
  4124. 2019-03-13 04:00:56,263 : epoch 455 , step 120 loss0= 0.0000 loss1= 2.0938 loss2=-3680.5806 lr= 0.000100
  4125. 2019-03-13 04:01:32,094 : epoch 455 , step 160 loss0= 0.0000 loss1= 1.8610 loss2=-3888.8943 lr= 0.000100
  4126. 2019-03-13 04:02:04,881 : epoch 456 , step 0 loss0= 0.0000 loss1= 2.3035 loss2=-3875.9900 lr= 0.000100
  4127. 2019-03-13 04:02:40,551 : epoch 456 , step 40 loss0= 0.0000 loss1= 2.3187 loss2=-3844.4797 lr= 0.000100
  4128. 2019-03-13 04:03:16,147 : epoch 456 , step 80 loss0= 0.0000 loss1= 1.9173 loss2=-3735.0232 lr= 0.000100
  4129. 2019-03-13 04:03:52,352 : epoch 456 , step 120 loss0= 0.0000 loss1= 2.0577 loss2=-3674.6511 lr= 0.000100
  4130. 2019-03-13 04:04:28,233 : epoch 456 , step 160 loss0= 0.0000 loss1= 2.0711 loss2=-3639.4353 lr= 0.000100
  4131. 2019-03-13 04:05:01,052 : epoch 457 , step 0 loss0= 0.0000 loss1= 1.8410 loss2=-3785.9602 lr= 0.000100
  4132. 2019-03-13 04:05:37,279 : epoch 457 , step 40 loss0= 0.0000 loss1= 2.3622 loss2=-3755.8474 lr= 0.000100
  4133. 2019-03-13 04:06:13,631 : epoch 457 , step 80 loss0= 0.0000 loss1= 1.9875 loss2=-3868.4512 lr= 0.000100
  4134. 2019-03-13 04:06:49,703 : epoch 457 , step 120 loss0= 0.0000 loss1= 2.3550 loss2=-3679.4868 lr= 0.000100
  4135. 2019-03-13 04:07:26,264 : epoch 457 , step 160 loss0= 0.0000 loss1= 2.0690 loss2=-3609.7537 lr= 0.000100
  4136. 2019-03-13 04:07:59,137 : epoch 458 , step 0 loss0= 0.0000 loss1= 1.5551 loss2=-3499.5962 lr= 0.000100
  4137. 2019-03-13 04:08:35,226 : epoch 458 , step 40 loss0= 0.0000 loss1= 2.4103 loss2=-3441.9277 lr= 0.000100
  4138. 2019-03-13 04:09:10,994 : epoch 458 , step 80 loss0= 0.0000 loss1= 2.7166 loss2=-3965.7544 lr= 0.000100
  4139. 2019-03-13 04:09:46,668 : epoch 458 , step 120 loss0= 0.0000 loss1= 2.0630 loss2=-3597.0027 lr= 0.000100
  4140. 2019-03-13 04:10:22,688 : epoch 458 , step 160 loss0= 0.0000 loss1= 2.7664 loss2=-3746.9304 lr= 0.000100
  4141. 2019-03-13 04:10:55,186 : epoch 459 , step 0 loss0= 0.0000 loss1= 1.8566 loss2=-3314.5454 lr= 0.000100
  4142. 2019-03-13 04:11:31,509 : epoch 459 , step 40 loss0= 0.0000 loss1= 1.6197 loss2=-3757.1555 lr= 0.000100
  4143. 2019-03-13 04:12:07,614 : epoch 459 , step 80 loss0= 0.0000 loss1= 1.6365 loss2=-3464.4165 lr= 0.000100
  4144. 2019-03-13 04:12:43,643 : epoch 459 , step 120 loss0= 0.0000 loss1= 2.5817 loss2=-4041.6509 lr= 0.000100
  4145. 2019-03-13 04:13:19,847 : epoch 459 , step 160 loss0= 0.0000 loss1= 1.7352 loss2=-3754.2798 lr= 0.000100
  4146. 2019-03-13 04:13:52,991 : epoch 460 , step 0 loss0= 0.0000 loss1= 1.6265 loss2=-3418.4688 lr= 0.000100
  4147. 2019-03-13 04:14:29,584 : epoch 460 , step 40 loss0= 0.0000 loss1= 2.2103 loss2=-3656.7195 lr= 0.000100
  4148. 2019-03-13 04:15:05,489 : epoch 460 , step 80 loss0= 0.0000 loss1= 1.6263 loss2=-3601.7380 lr= 0.000100
  4149. 2019-03-13 04:15:41,198 : epoch 460 , step 120 loss0= 0.0000 loss1= 1.8920 loss2=-3540.8813 lr= 0.000100
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!