API tools faq
paste
Guest User

Untitled

a guest
Feb 13th, 2018
84
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 766.58 KB | None | 0 0
raw download clone embed print report
  1. $ python run_job.py -n 5 -g 60 -c 12 --simulator_procs 10 --use_sync --name breakout --short
  2. args.offline: False
  3. ('bash command: ', 'srun -A luna -N 5 -n 5 -c 12 -t 55:00 distributed_tensorpack_mkl.sh 61216 30065 Breakout-v0 adam 1 "3nodes 12cores" "breakout_1517329773.87" 0.00015 128 60 0 None 256 10 1 uniform normal False . True 1 False /net/archive/groups/plggluna/intel_2/logs/ 1e-08 0.9 0.999 0 False False False False 120 False')
  4. srun: job 9511741 queued and waiting for resources
  5. srun: job 9511741 has been allocated resources
  6. SLURM_JOB_ID 9511741 ; SLURM_JOB_NAME distributed_tensorpack_mkl.sh ; SLURM_JOB_NODELIST p[0111-0112,0115,0574,0576] ; SLURMD_NODENAME p0112 ; SLURM_JOB_NUM_NODES 5
  7. SLURM_JOB_ID 9511741 ; SLURM_JOB_NAME distributed_tensorpack_mkl.sh ; SLURM_JOB_NODELIST p[0111-0112,0115,0574,0576] ; SLURMD_NODENAME p0111 ; SLURM_JOB_NUM_NODES 5
  8. SLURM_JOB_ID 9511741 ; SLURM_JOB_NAME distributed_tensorpack_mkl.sh ; SLURM_JOB_NODELIST p[0111-0112,0115,0574,0576] ; SLURMD_NODENAME p0115 ; SLURM_JOB_NUM_NODES 5
  9. mkdir: cannot create directory ‘/net/archive/groups/plggluna/adam/experiments/breakout_1517329773.87’: File exists
  10. mkdir: cannot create directory ‘/net/archive/groups/plggluna/adam/experiments/breakout_1517329773.87’: File exists
  11. mkdir: cannot create directory ‘/net/archive/groups/plggluna/adam/experiments/breakout_1517329773.87/models/’: File exists
  12. mkdir: cannot create directory ‘/net/archive/groups/plggluna/adam/experiments/breakout_1517329773.87/models/’: File exists
  13. mkdir: cannot create directory ‘/net/archive/groups/plggluna/adam/experiments/breakout_1517329773.87/storage/’: File exists
  14. mkdir: cannot create directory ‘/net/archive/groups/plggluna/adam/experiments/breakout_1517329773.87/storage/’: File exists
  15. SLURM_JOB_ID 9511741 ; SLURM_JOB_NAME distributed_tensorpack_mkl.sh ; SLURM_JOB_NODELIST p[0111-0112,0115,0574,0576] ; SLURMD_NODENAME p0574 ; SLURM_JOB_NUM_NODES 5
  16. SLURM_JOB_ID 9511741 ; SLURM_JOB_NAME distributed_tensorpack_mkl.sh ; SLURM_JOB_NODELIST p[0111-0112,0115,0574,0576] ; SLURMD_NODENAME p0576 ; SLURM_JOB_NUM_NODES 5
  17. plgrid/libs/qt/5.4.1 loaded.
  18. plgrid/libs/qt/5.4.1 loaded.
  19. plgrid/libs/qt/5.4.1 loaded.
  20. plgrid/libs/mkl/11.3.1 loaded.
  21. plgrid/libs/mkl/11.3.1 loaded.
  22. plgrid/libs/mkl/11.3.1 loaded.
  23. plgrid/tools/gcc/4.9.2 loaded.
  24. plgrid/tools/gcc/4.9.2 loaded.
  25. plgrid/tools/intel/15.0.2 loaded.
  26. plgrid/tools/intel/15.0.2 loaded.
  27. plgrid/tools/gcc/4.9.2 loaded.
  28. plgrid/tools/intel/15.0.2 loaded.
  29. plgrid/tools/tcltk/8.5.19-threads loaded.
  30. plgrid/tools/tcltk/8.5.19-threads loaded.
  31. plgrid/tools/python/2.7.13 loaded.
  32. plgrid/tools/python/2.7.13 loaded.
  33. plgrid/tools/tcltk/8.5.19-threads loaded.
  34. plgrid/tools/python/2.7.13 loaded.
  35. plgrid/libs/qt/5.4.1 loaded.
  36. plgrid/libs/mkl/11.3.1 loaded.
  37. plgrid/tools/gcc/4.9.2 loaded.
  38. plgrid/tools/intel/15.0.2 loaded.
  39. plgrid/tools/tcltk/8.5.19-threads loaded.
  40. plgrid/tools/python/2.7.13 loaded.
  41. plgrid/libs/qt/5.4.1 loaded.
  42. plgrid/libs/mkl/11.3.1 loaded.
  43. plgrid/tools/gcc/4.9.2 loaded.
  44. plgrid/tools/intel/15.0.2 loaded.
  45. plgrid/tools/tcltk/8.5.19-threads loaded.
  46. plgrid/tools/python/2.7.13 loaded.
  47. plgrid/libs/mkl/11.3.1 unloaded.
  48. plgrid/libs/mkl/2017.0.0 loaded.
  49.  
  50. The following have been reloaded with a version change:
  51. 1) plgrid/libs/mkl/11.3.1 => plgrid/libs/mkl/2017.0.0
  52.  
  53. plgrid/libs/mkl/11.3.1 unloaded.
  54. plgrid/libs/mkl/2017.0.0 loaded.
  55.  
  56. The following have been reloaded with a version change:
  57. 1) plgrid/libs/mkl/11.3.1 => plgrid/libs/mkl/2017.0.0
  58.  
  59. plgrid/libs/mkl/11.3.1 unloaded.
  60. plgrid/libs/mkl/2017.0.0 loaded.
  61.  
  62. The following have been reloaded with a version change:
  63. 1) plgrid/libs/mkl/11.3.1 => plgrid/libs/mkl/2017.0.0
  64.  
  65. tools/gcc/6.2.0 loaded.
  66. tools/gcc/6.2.0 loaded.
  67. tools/gcc/6.2.0 loaded.
  68. plgrid/libs/mkl/11.3.1 unloaded.
  69. plgrid/libs/mkl/2017.0.0 loaded.
  70.  
  71. The following have been reloaded with a version change:
  72. 1) plgrid/libs/mkl/11.3.1 => plgrid/libs/mkl/2017.0.0
  73.  
  74. plgrid/libs/mkl/11.3.1 unloaded.
  75. plgrid/libs/mkl/2017.0.0 loaded.
  76.  
  77. The following have been reloaded with a version change:
  78. 1) plgrid/libs/mkl/11.3.1 => plgrid/libs/mkl/2017.0.0
  79.  
  80. tools/gcc/6.2.0 loaded.
  81. tools/gcc/6.2.0 loaded.
  82. PROGRAM_ARGS: --mkl 0 --dummy 0 --sync 0 --cpu 1 --artificial_slowdown 0 --queue_size 1 --my_sim_master_queue 1 --train_log_path /net/archive/groups/plggluna/adam/experiments/breakout_1517329773.87/storage//atari_trainlog/ --predict_batch_size 16 --dummy_predictor 0 --do_train 1 --simulator_procs 10 --env Breakout-v0 --nr_towers 1 --nr_predict_towers 3 --steps_per_epoch 1000 --fc_neurons 256 --batch_size 128 --learning_rate 0.00015 --port 61216 --tf_port 30065 --optimizer adam --use_sync_opt 1 --num_grad 60 --early_stopping None --ps 1 --fc_init uniform --conv_init normal --replace_with_conv True --fc_splits 1 --debug_charts False --epsilon 1e-08 --beta1 0.9 --beta2 0.999 --save_every 0 --models_dir /net/archive/groups/plggluna/adam/experiments/breakout_1517329773.87/models/ --experiment_dir /net/archive/groups/plggluna/adam/experiments/breakout_1517329773.87 --adam_debug False --eval_node False --record_node False --schedule_hyper False
  83. OFFLINE: False
  84. PROGRAM_ARGS: --mkl 0 --dummy 0 --sync 0 --cpu 1 --artificial_slowdown 0 --queue_size 1 --my_sim_master_queue 1 --train_log_path /net/archive/groups/plggluna/adam/experiments/breakout_1517329773.87/storage//atari_trainlog/ --predict_batch_size 16 --dummy_predictor 0 --do_train 1 --simulator_procs 10 --env Breakout-v0 --nr_towers 1 --nr_predict_towers 3 --steps_per_epoch 1000 --fc_neurons 256 --batch_size 128 --learning_rate 0.00015 --port 61216 --tf_port 30065 --optimizer adam --use_sync_opt 1 --num_grad 60 --early_stopping None --ps 1 --fc_init uniform --conv_init normal --replace_with_conv True --fc_splits 1 --debug_charts False --epsilon 1e-08 --beta1 0.9 --beta2 0.999 --save_every 0 --models_dir /net/archive/groups/plggluna/adam/experiments/breakout_1517329773.87/models/ --experiment_dir /net/archive/groups/plggluna/adam/experiments/breakout_1517329773.87 --adam_debug False --eval_node False --record_node False --schedule_hyper False
  85. OFFLINE: False
  86. PROGRAM_ARGS: --mkl 0 --dummy 0 --sync 0 --cpu 1 --artificial_slowdown 0 --queue_size 1 --my_sim_master_queue 1 --train_log_path /net/archive/groups/plggluna/adam/experiments/breakout_1517329773.87/storage//atari_trainlog/ --predict_batch_size 16 --dummy_predictor 0 --do_train 1 --simulator_procs 10 --env Breakout-v0 --nr_towers 1 --nr_predict_towers 3 --steps_per_epoch 1000 --fc_neurons 256 --batch_size 128 --learning_rate 0.00015 --port 61216 --tf_port 30065 --optimizer adam --use_sync_opt 1 --num_grad 60 --early_stopping None --ps 1 --fc_init uniform --conv_init normal --replace_with_conv True --fc_splits 1 --debug_charts False --epsilon 1e-08 --beta1 0.9 --beta2 0.999 --save_every 0 --models_dir /net/archive/groups/plggluna/adam/experiments/breakout_1517329773.87/models/ --experiment_dir /net/archive/groups/plggluna/adam/experiments/breakout_1517329773.87 --adam_debug False --eval_node False --record_node False --schedule_hyper False
  87. OFFLINE: False
  88. 2018-01-30 17:29:48.435736: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
  89. 2018-01-30 17:29:48.435772: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
  90. 2018-01-30 17:29:48.435780: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
  91. 2018-01-30 17:29:48.435788: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
  92. 2018-01-30 17:29:48.435795: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
  93. 2018-01-30 17:29:48.445459: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:215] Initialize GrpcChannelCache for job ps -> {0 -> localhost:30065}
  94. 2018-01-30 17:29:48.445499: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:215] Initialize GrpcChannelCache for job worker -> {0 -> p0112:30066, 1 -> p0115:30066, 2 -> p0574:30066, 3 -> p0576:30066}
  95. 2018-01-30 17:29:48.446966: I tensorflow/core/distributed_runtime/rpc/grpc_server_lib.cc:316] Started server with target: grpc://localhost:30065
  96. {'ps': ['p0111:30065'], 'worker': ['p0112:30066', 'p0115:30066', 'p0574:30066', 'p0576:30066']}
  97. [ps:0] Starting the TF server
  98. [0130 17:29:48 @train.py:84] [ps:0] joining the server.
  99. 2018-01-30 17:29:48.492019: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
  100. 2018-01-30 17:29:48.492076: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
  101. 2018-01-30 17:29:48.492084: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
  102. 2018-01-30 17:29:48.492091: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
  103. 2018-01-30 17:29:48.492097: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
  104. 2018-01-30 17:29:48.502455: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:215] Initialize GrpcChannelCache for job ps -> {0 -> p0111:30065}
  105. 2018-01-30 17:29:48.502505: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:215] Initialize GrpcChannelCache for job worker -> {0 -> localhost:30066, 1 -> p0115:30066, 2 -> p0574:30066, 3 -> p0576:30066}
  106. 2018-01-30 17:29:48.504127: I tensorflow/core/distributed_runtime/rpc/grpc_server_lib.cc:316] Started server with target: grpc://localhost:30066
  107. [2018-01-30 17:29:48,505] Making new env: Breakout-v0
  108. {'ps': ['p0111:30065'], 'worker': ['p0112:30066', 'p0115:30066', 'p0574:30066', 'p0576:30066']}
  109. [worker:0] Starting the TF server
  110. args.mkl == 0
  111. using tensorflow convolution
  112. [2018-01-30 17:29:48,609] Making new env: Breakout-v0
  113. [2018-01-30 17:29:48,614] Making new env: Breakout-v0
  114. [2018-01-30 17:29:48,620] Making new env: Breakout-v0
  115. [2018-01-30 17:29:48,625] Making new env: Breakout-v0
  116. [2018-01-30 17:29:48,631] Making new env: Breakout-v0
  117. [2018-01-30 17:29:48,637] Making new env: Breakout-v0
  118. [2018-01-30 17:29:48,643] Making new env: Breakout-v0
  119. [2018-01-30 17:29:48,649] Making new env: Breakout-v0
  120. [2018-01-30 17:29:48,655] Making new env: Breakout-v0
  121. [2018-01-30 17:29:48,661] Making new env: Breakout-v0
  122. None <type 'NoneType'>
  123.  
  124.  
  125. worker host: grpc://localhost:30066
  126.  
  127.  
  128. [0130 17:29:48 @train.py:711] [BA3C] Train on gpu 0 and infer on gpu 0,0,0
  129. [0130 17:29:48 @train.py:717] using async version
  130. DUMMY PREDICTOR 0
  131. MultiGPUTrainer __init__ dummy = 0
  132. [0130 17:29:48 @multigpu.py:57] Training a model of 1 tower
  133. [0130 17:29:48 @multigpu.py:67] Building graph for training tower 0..., /cpu:0
  134. 12
  135. [0130 17:29:48 @_common.py:61] conv0 input: [None, 84, 84, 16]
  136. Tensor("tower0/conv0/output:0", shape=(?, 80, 80, 32), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  137. [0130 17:29:48 @_common.py:70] conv0 output: [None, 80, 80, 32]
  138. [0130 17:29:48 @_common.py:61] pool0 input: [None, 80, 80, 32]
  139. Tensor("tower0/pool0/MaxPool:0", shape=(?, 40, 40, 32), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  140. [0130 17:29:48 @_common.py:70] pool0 output: [None, 40, 40, 32]
  141. [0130 17:29:48 @_common.py:61] conv1 input: [None, 40, 40, 32]
  142. Tensor("tower0/conv1/output:0", shape=(?, 36, 36, 32), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  143. [0130 17:29:48 @_common.py:70] conv1 output: [None, 36, 36, 32]
  144. [0130 17:29:48 @_common.py:61] pool1 input: [None, 36, 36, 32]
  145. Tensor("tower0/pool1/MaxPool:0", shape=(?, 18, 18, 32), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  146. [0130 17:29:48 @_common.py:70] pool1 output: [None, 18, 18, 32]
  147. [0130 17:29:48 @_common.py:61] conv2 input: [None, 18, 18, 32]
  148. Tensor("tower0/conv2/output:0", shape=(?, 14, 14, 64), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  149. [0130 17:29:48 @_common.py:70] conv2 output: [None, 14, 14, 64]
  150. [0130 17:29:48 @_common.py:61] pool2 input: [None, 14, 14, 64]
  151. Tensor("tower0/pool2/MaxPool:0", shape=(?, 7, 7, 64), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  152. [0130 17:29:48 @_common.py:70] pool2 output: [None, 7, 7, 64]
  153. [0130 17:29:48 @_common.py:61] conv3 input: [None, 7, 7, 64]
  154. Tensor("tower0/conv3/output:0", shape=(?, 5, 5, 64), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  155. [0130 17:29:48 @_common.py:70] conv3 output: [None, 5, 5, 64]
  156. [0130 17:29:48 @_common.py:61] fc1_0 input: [None, 5, 5, 64]
  157. Tensor("tower0/fc1_0/output:0", shape=(?, 1, 1, 256), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  158. [0130 17:29:48 @_common.py:70] fc1_0 output: [None, 1, 1, 256]
  159. [0130 17:29:48 @_common.py:61] fc-pi input: [None, 256]
  160. Tensor("tower0/fc-pi/output:0", shape=(?, 6), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  161. [0130 17:29:48 @_common.py:70] fc-pi output: [None, 6]
  162. [0130 17:29:48 @_common.py:61] fc-v input: [None, 256]
  163. PROGRAM_ARGS: --mkl 0 --dummy 0 --sync 0 --cpu 1 --artificial_slowdown 0 --queue_size 1 --my_sim_master_queue 1 --train_log_path /net/archive/groups/plggluna/adam/experiments/breakout_1517329773.87/storage//atari_trainlog/ --predict_batch_size 16 --dummy_predictor 0 --do_train 1 --simulator_procs 10 --env Breakout-v0 --nr_towers 1 --nr_predict_towers 3 --steps_per_epoch 1000 --fc_neurons 256 --batch_size 128 --learning_rate 0.00015 --port 61216 --tf_port 30065 --optimizer adam --use_sync_opt 1 --num_grad 60 --early_stopping None --ps 1 --fc_init uniform --conv_init normal --replace_with_conv True --fc_splits 1 --debug_charts False --epsilon 1e-08 --beta1 0.9 --beta2 0.999 --save_every 0 --models_dir /net/archive/groups/plggluna/adam/experiments/breakout_1517329773.87/models/ --experiment_dir /net/archive/groups/plggluna/adam/experiments/breakout_1517329773.87 --adam_debug False --eval_node False --record_node False --schedule_hyper False
  164. OFFLINE: False
  165. PROGRAM_ARGS: --mkl 0 --dummy 0 --sync 0 --cpu 1 --artificial_slowdown 0 --queue_size 1 --my_sim_master_queue 1 --train_log_path /net/archive/groups/plggluna/adam/experiments/breakout_1517329773.87/storage//atari_trainlog/ --predict_batch_size 16 --dummy_predictor 0 --do_train 1 --simulator_procs 10 --env Breakout-v0 --nr_towers 1 --nr_predict_towers 3 --steps_per_epoch 1000 --fc_neurons 256 --batch_size 128 --learning_rate 0.00015 --port 61216 --tf_port 30065 --optimizer adam --use_sync_opt 1 --num_grad 60 --early_stopping None --ps 1 --fc_init uniform --conv_init normal --replace_with_conv True --fc_splits 1 --debug_charts False --epsilon 1e-08 --beta1 0.9 --beta2 0.999 --save_every 0 --models_dir /net/archive/groups/plggluna/adam/experiments/breakout_1517329773.87/models/ --experiment_dir /net/archive/groups/plggluna/adam/experiments/breakout_1517329773.87 --adam_debug False --eval_node False --record_node False --schedule_hyper False
  166. OFFLINE: False
  167. 2018-01-30 17:29:48.970453: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
  168. 2018-01-30 17:29:48.970511: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
  169. 2018-01-30 17:29:48.970531: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
  170. 2018-01-30 17:29:48.970539: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
  171. 2018-01-30 17:29:48.970546: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
  172. 2018-01-30 17:29:48.981347: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:215] Initialize GrpcChannelCache for job ps -> {0 -> p0111:30065}
  173. 2018-01-30 17:29:48.981385: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:215] Initialize GrpcChannelCache for job worker -> {0 -> p0112:30066, 1 -> localhost:30066, 2 -> p0574:30066, 3 -> p0576:30066}
  174. 2018-01-30 17:29:48.983013: I tensorflow/core/distributed_runtime/rpc/grpc_server_lib.cc:316] Started server with target: grpc://localhost:30066
  175. [2018-01-30 17:29:48,984] Making new env: Breakout-v0
  176. {'ps': ['p0111:30065'], 'worker': ['p0112:30066', 'p0115:30066', 'p0574:30066', 'p0576:30066']}
  177. [worker:1] Starting the TF server
  178. args.mkl == 0
  179. Tensor("tower0/fc-v/output:0", shape=(?, 1), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  180. [0130 17:29:49 @_common.py:70] fc-v output: [None, 1]
  181. using tensorflow convolution
  182. [2018-01-30 17:29:49,097] Making new env: Breakout-v0
  183. [2018-01-30 17:29:49,103] Making new env: Breakout-v0
  184. [2018-01-30 17:29:49,109] Making new env: Breakout-v0
  185. [2018-01-30 17:29:49,114] Making new env: Breakout-v0
  186. [2018-01-30 17:29:49,120] Making new env: Breakout-v0
  187. [2018-01-30 17:29:49,126] Making new env: Breakout-v0
  188. [2018-01-30 17:29:49,132] Making new env: Breakout-v0
  189. [2018-01-30 17:29:49,138] Making new env: Breakout-v0
  190. [2018-01-30 17:29:49,145] Making new env: Breakout-v0
  191. [2018-01-30 17:29:49,151] Making new env: Breakout-v0
  192. None <type 'NoneType'>
  193.  
  194.  
  195. worker host: grpc://localhost:30066
  196.  
  197.  
  198. [0130 17:29:49 @train.py:711] [BA3C] Train on gpu 0 and infer on gpu 0,0,0
  199. [0130 17:29:49 @train.py:717] using async version
  200. DUMMY PREDICTOR 0
  201. MultiGPUTrainer __init__ dummy = 0
  202. [0130 17:29:49 @multigpu.py:57] Training a model of 1 tower
  203. [0130 17:29:49 @multigpu.py:67] Building graph for training tower 0..., /cpu:0
  204. 12
  205. [0130 17:29:49 @_common.py:61] conv0 input: [None, 84, 84, 16]
  206. Tensor("tower0/conv0/output:0", shape=(?, 80, 80, 32), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  207. [0130 17:29:49 @_common.py:70] conv0 output: [None, 80, 80, 32]
  208. [0130 17:29:49 @_common.py:61] pool0 input: [None, 80, 80, 32]
  209. Tensor("tower0/pool0/MaxPool:0", shape=(?, 40, 40, 32), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  210. [0130 17:29:49 @_common.py:70] pool0 output: [None, 40, 40, 32]
  211. [0130 17:29:49 @_common.py:61] conv1 input: [None, 40, 40, 32]
  212. Tensor("tower0/conv1/output:0", shape=(?, 36, 36, 32), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  213. [0130 17:29:49 @_common.py:70] conv1 output: [None, 36, 36, 32]
  214. [0130 17:29:49 @_common.py:61] pool1 input: [None, 36, 36, 32]
  215. Tensor("tower0/pool1/MaxPool:0", shape=(?, 18, 18, 32), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  216. [0130 17:29:49 @_common.py:70] pool1 output: [None, 18, 18, 32]
  217. [0130 17:29:49 @_common.py:61] conv2 input: [None, 18, 18, 32]
  218. Tensor("tower0/conv2/output:0", shape=(?, 14, 14, 64), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  219. [0130 17:29:49 @_common.py:70] conv2 output: [None, 14, 14, 64]
  220. [0130 17:29:49 @_common.py:61] pool2 input: [None, 14, 14, 64]
  221. Tensor("tower0/pool2/MaxPool:0", shape=(?, 7, 7, 64), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  222. [0130 17:29:49 @_common.py:70] pool2 output: [None, 7, 7, 64]
  223. [0130 17:29:49 @_common.py:61] conv3 input: [None, 7, 7, 64]
  224. Tensor("tower0/conv3/output:0", shape=(?, 5, 5, 64), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  225. [0130 17:29:49 @_common.py:70] conv3 output: [None, 5, 5, 64]
  226. [0130 17:29:49 @_common.py:61] fc1_0 input: [None, 5, 5, 64]
  227. Tensor("tower0/fc1_0/output:0", shape=(?, 1, 1, 256), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  228. [0130 17:29:49 @_common.py:70] fc1_0 output: [None, 1, 1, 256]
  229. [0130 17:29:49 @_common.py:61] fc-pi input: [None, 256]
  230. Tensor("tower0/fc-pi/output:0", shape=(?, 6), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  231. [0130 17:29:49 @_common.py:70] fc-pi output: [None, 6]
  232. [0130 17:29:49 @_common.py:61] fc-v input: [None, 256]
  233. Tensor("tower0/fc-v/output:0", shape=(?, 1), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  234. [0130 17:29:49 @_common.py:70] fc-v output: [None, 1]
  235. MOVING_SUMMARY_VARIABLES
  236. []
  237. [0130 17:29:49 @modelutils.py:22] Model Parameters:
  238. conv0/W:0: shape=[5, 5, 16, 32], dim=12800
  239. conv1/W:0: shape=[5, 5, 32, 32], dim=25600
  240. conv2/W:0: shape=[5, 5, 32, 64], dim=51200
  241. conv3/W:0: shape=[3, 3, 64, 64], dim=36864
  242. fc1_0/W:0: shape=[5, 5, 64, 256], dim=409600
  243. fc-pi/W:0: shape=[256, 6], dim=1536
  244. fc-pi/b:0: shape=[6], dim=6
  245. fc-v/W:0: shape=[256, 1], dim=256
  246. fc-v/b:0: shape=[1], dim=1
  247. Total param=537863 (2.051785 MB assuming all float32)
  248. MOVING_SUMMARY_VARIABLES
  249. []
  250. [0130 17:29:50 @modelutils.py:22] Model Parameters:
  251. conv0/W:0: shape=[5, 5, 16, 32], dim=12800
  252. conv1/W:0: shape=[5, 5, 32, 32], dim=25600
  253. conv2/W:0: shape=[5, 5, 32, 64], dim=51200
  254. conv3/W:0: shape=[3, 3, 64, 64], dim=36864
  255. fc1_0/W:0: shape=[5, 5, 64, 256], dim=409600
  256. fc-pi/W:0: shape=[256, 6], dim=1536
  257. fc-pi/b:0: shape=[6], dim=6
  258. fc-v/W:0: shape=[256, 1], dim=256
  259. fc-v/b:0: shape=[1], dim=1
  260. Total param=537863 (2.051785 MB assuming all float32)
  261. [0130 17:29:50 @multigpu.py:228] Setup callbacks ...
  262. Creating Predictorfactor 0
  263. [0130 17:29:50 @base.py:132] Building graph for predictor tower 0...
  264. 12
  265. Tensor("towerp0/conv0/output:0", shape=(?, 80, 80, 32), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  266. Tensor("towerp0/pool0/MaxPool:0", shape=(?, 40, 40, 32), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  267. Tensor("towerp0/conv1/output:0", shape=(?, 36, 36, 32), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  268. Tensor("towerp0/pool1/MaxPool:0", shape=(?, 18, 18, 32), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  269. Tensor("towerp0/conv2/output:0", shape=(?, 14, 14, 64), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  270. Tensor("towerp0/pool2/MaxPool:0", shape=(?, 7, 7, 64), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  271. Tensor("towerp0/conv3/output:0", shape=(?, 5, 5, 64), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  272. Tensor("towerp0/fc1_0/output:0", shape=(?, 1, 1, 256), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  273. Tensor("towerp0/fc-pi/output:0", shape=(?, 6), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  274. Tensor("towerp0/fc-v/output:0", shape=(?, 1), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  275. [0130 17:29:51 @base.py:132] Building graph for predictor tower 0...
  276. 12
  277. Tensor("towerp0_1/conv0/output:0", shape=(?, 80, 80, 32), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  278. Tensor("towerp0_1/pool0/MaxPool:0", shape=(?, 40, 40, 32), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  279. Tensor("towerp0_1/conv1/output:0", shape=(?, 36, 36, 32), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  280. Tensor("towerp0_1/pool1/MaxPool:0", shape=(?, 18, 18, 32), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  281. Tensor("towerp0_1/conv2/output:0", shape=(?, 14, 14, 64), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  282. Tensor("towerp0_1/pool2/MaxPool:0", shape=(?, 7, 7, 64), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  283. Tensor("towerp0_1/conv3/output:0", shape=(?, 5, 5, 64), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  284. Tensor("towerp0_1/fc1_0/output:0", shape=(?, 1, 1, 256), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  285. Tensor("towerp0_1/fc-pi/output:0", shape=(?, 6), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  286. Tensor("towerp0_1/fc-v/output:0", shape=(?, 1), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  287. [0130 17:29:51 @base.py:132] Building graph for predictor tower 0...
  288. 12
  289. Tensor("towerp0_2/conv0/output:0", shape=(?, 80, 80, 32), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  290. Tensor("towerp0_2/pool0/MaxPool:0", shape=(?, 40, 40, 32), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  291. Tensor("towerp0_2/conv1/output:0", shape=(?, 36, 36, 32), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  292. Tensor("towerp0_2/pool1/MaxPool:0", shape=(?, 18, 18, 32), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  293. Tensor("towerp0_2/conv2/output:0", shape=(?, 14, 14, 64), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  294. Tensor("towerp0_2/pool2/MaxPool:0", shape=(?, 7, 7, 64), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  295. Tensor("towerp0_2/conv3/output:0", shape=(?, 5, 5, 64), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  296. Tensor("towerp0_2/fc1_0/output:0", shape=(?, 1, 1, 256), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  297. Tensor("towerp0_2/fc-pi/output:0", shape=(?, 6), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  298. Tensor("towerp0_2/fc-v/output:0", shape=(?, 1), dtype=float32, device=/job:worker/task:0/device:CPU:0)
  299. [0130 17:29:51 @base.py:177] ===============================================================
  300. [0130 17:29:51 @base.py:179] CHIEF!
  301. [0130 17:29:51 @base.py:180] [p0112] Creating the session
  302. [0130 17:29:51 @base.py:181] ===============================================================
  303. [0130 17:29:51 @multigpu.py:228] Setup callbacks ...
  304. Creating Predictorfactor 0
  305. [0130 17:29:51 @base.py:132] Building graph for predictor tower 0...
  306. 12
  307. Tensor("towerp0/conv0/output:0", shape=(?, 80, 80, 32), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  308. Tensor("towerp0/pool0/MaxPool:0", shape=(?, 40, 40, 32), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  309. Tensor("towerp0/conv1/output:0", shape=(?, 36, 36, 32), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  310. Tensor("towerp0/pool1/MaxPool:0", shape=(?, 18, 18, 32), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  311. Tensor("towerp0/conv2/output:0", shape=(?, 14, 14, 64), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  312. Tensor("towerp0/pool2/MaxPool:0", shape=(?, 7, 7, 64), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  313. Tensor("towerp0/conv3/output:0", shape=(?, 5, 5, 64), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  314. Tensor("towerp0/fc1_0/output:0", shape=(?, 1, 1, 256), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  315. Tensor("towerp0/fc-pi/output:0", shape=(?, 6), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  316. Tensor("towerp0/fc-v/output:0", shape=(?, 1), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  317. [0130 17:29:51 @base.py:132] Building graph for predictor tower 0...
  318. 12
  319. Tensor("towerp0_1/conv0/output:0", shape=(?, 80, 80, 32), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  320. Tensor("towerp0_1/pool0/MaxPool:0", shape=(?, 40, 40, 32), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  321. Tensor("towerp0_1/conv1/output:0", shape=(?, 36, 36, 32), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  322. Tensor("towerp0_1/pool1/MaxPool:0", shape=(?, 18, 18, 32), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  323. Tensor("towerp0_1/conv2/output:0", shape=(?, 14, 14, 64), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  324. Tensor("towerp0_1/pool2/MaxPool:0", shape=(?, 7, 7, 64), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  325. Tensor("towerp0_1/conv3/output:0", shape=(?, 5, 5, 64), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  326. Tensor("towerp0_1/fc1_0/output:0", shape=(?, 1, 1, 256), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  327. Tensor("towerp0_1/fc-pi/output:0", shape=(?, 6), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  328. Tensor("towerp0_1/fc-v/output:0", shape=(?, 1), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  329. [0130 17:29:51 @base.py:132] Building graph for predictor tower 0...
  330. 12
  331. Tensor("towerp0_2/conv0/output:0", shape=(?, 80, 80, 32), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  332. Tensor("towerp0_2/pool0/MaxPool:0", shape=(?, 40, 40, 32), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  333. Tensor("towerp0_2/conv1/output:0", shape=(?, 36, 36, 32), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  334. Tensor("towerp0_2/pool1/MaxPool:0", shape=(?, 18, 18, 32), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  335. Tensor("towerp0_2/conv2/output:0", shape=(?, 14, 14, 64), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  336. Tensor("towerp0_2/pool2/MaxPool:0", shape=(?, 7, 7, 64), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  337. Tensor("towerp0_2/conv3/output:0", shape=(?, 5, 5, 64), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  338. Tensor("towerp0_2/fc1_0/output:0", shape=(?, 1, 1, 256), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  339. Tensor("towerp0_2/fc-pi/output:0", shape=(?, 6), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  340. Tensor("towerp0_2/fc-v/output:0", shape=(?, 1), dtype=float32, device=/job:worker/task:1/device:CPU:0)
  341. [0130 17:29:51 @base.py:177] ===============================================================
  342. [0130 17:29:51 @base.py:180] [p0115] Creating the session
  343. [0130 17:29:51 @base.py:181] ===============================================================
  344. 2018-01-30 17:29:57.665912: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
  345. 2018-01-30 17:29:57.665949: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
  346. 2018-01-30 17:29:57.665958: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
  347. 2018-01-30 17:29:57.665965: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
  348. 2018-01-30 17:29:57.665973: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
  349. 2018-01-30 17:29:57.670158: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
  350. 2018-01-30 17:29:57.670198: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
  351. 2018-01-30 17:29:57.670207: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
  352. 2018-01-30 17:29:57.670215: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
  353. 2018-01-30 17:29:57.670222: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
  354. 2018-01-30 17:29:57.676919: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:215] Initialize GrpcChannelCache for job ps -> {0 -> p0111:30065}
  355. 2018-01-30 17:29:57.676959: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:215] Initialize GrpcChannelCache for job worker -> {0 -> p0112:30066, 1 -> p0115:30066, 2 -> p0574:30066, 3 -> localhost:30066}
  356. 2018-01-30 17:29:57.678659: I tensorflow/core/distributed_runtime/rpc/grpc_server_lib.cc:316] Started server with target: grpc://localhost:30066
  357. 2018-01-30 17:29:57.680658: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:215] Initialize GrpcChannelCache for job ps -> {0 -> p0111:30065}
  358. [2018-01-30 17:29:57,679] Making new env: Breakout-v0
  359. 2018-01-30 17:29:57.680702: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:215] Initialize GrpcChannelCache for job worker -> {0 -> p0112:30066, 1 -> p0115:30066, 2 -> localhost:30066, 3 -> p0576:30066}
  360. 2018-01-30 17:29:57.682393: I tensorflow/core/distributed_runtime/rpc/grpc_server_lib.cc:316] Started server with target: grpc://localhost:30066
  361. [2018-01-30 17:29:57,683] Making new env: Breakout-v0
  362. {'ps': ['p0111:30065'], 'worker': ['p0112:30066', 'p0115:30066', 'p0574:30066', 'p0576:30066']}
  363. [worker:3] Starting the TF server
  364. args.mkl == 0
  365. {'ps': ['p0111:30065'], 'worker': ['p0112:30066', 'p0115:30066', 'p0574:30066', 'p0576:30066']}
  366. [worker:2] Starting the TF server
  367. args.mkl == 0
  368. using tensorflow convolution
  369. using tensorflow convolution
  370. [2018-01-30 17:29:57,918] Making new env: Breakout-v0
  371. [2018-01-30 17:29:57,917] Making new env: Breakout-v0
  372. [2018-01-30 17:29:57,922] Making new env: Breakout-v0
  373. [2018-01-30 17:29:57,923] Making new env: Breakout-v0
  374. [2018-01-30 17:29:57,928] Making new env: Breakout-v0
  375. [2018-01-30 17:29:57,929] Making new env: Breakout-v0
  376. [2018-01-30 17:29:57,934] Making new env: Breakout-v0
  377. [2018-01-30 17:29:57,935] Making new env: Breakout-v0
  378. [2018-01-30 17:29:57,939] Making new env: Breakout-v0
  379. [2018-01-30 17:29:57,940] Making new env: Breakout-v0
  380. [2018-01-30 17:29:57,952] Making new env: Breakout-v0
  381. [2018-01-30 17:29:57,956] Making new env: Breakout-v0
  382. [2018-01-30 17:29:57,957] Making new env: Breakout-v0
  383. [2018-01-30 17:29:57,959] Making new env: Breakout-v0
  384. [2018-01-30 17:29:57,961] Making new env: Breakout-v0
  385. [2018-01-30 17:29:57,964] Making new env: Breakout-v0
  386. [2018-01-30 17:29:57,965] Making new env: Breakout-v0
  387. [2018-01-30 17:29:57,971] Making new env: Breakout-v0
  388. [2018-01-30 17:29:57,978] Making new env: Breakout-v0
  389. [2018-01-30 17:29:57,978] Making new env: Breakout-v0
  390. None <type 'NoneType'>
  391.  
  392.  
  393. worker host: grpc://localhost:30066
  394.  
  395.  
  396. [0130 17:29:57 @train.py:711] [BA3C] Train on gpu 0 and infer on gpu 0,0,0
  397. [0130 17:29:57 @train.py:717] using async version
  398. DUMMY PREDICTOR 0
  399. MultiGPUTrainer __init__ dummy = 0
  400. [0130 17:29:57 @multigpu.py:57] Training a model of 1 tower
  401. [0130 17:29:57 @multigpu.py:67] Building graph for training tower 0..., /cpu:0
  402. None <type 'NoneType'>
  403.  
  404.  
  405. worker host: grpc://localhost:30066
  406.  
  407.  
  408. [0130 17:29:58 @train.py:711] [BA3C] Train on gpu 0 and infer on gpu 0,0,0
  409. [0130 17:29:58 @train.py:717] using async version
  410. 12
  411. DUMMY PREDICTOR 0
  412. MultiGPUTrainer __init__ dummy = 0
  413. [0130 17:29:58 @multigpu.py:57] Training a model of 1 tower
  414. [0130 17:29:58 @multigpu.py:67] Building graph for training tower 0..., /cpu:0
  415. [0130 17:29:58 @_common.py:61] conv0 input: [None, 84, 84, 16]
  416. 12
  417. Tensor("tower0/conv0/output:0", shape=(?, 80, 80, 32), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  418. [0130 17:29:58 @_common.py:70] conv0 output: [None, 80, 80, 32]
  419. [0130 17:29:58 @_common.py:61] pool0 input: [None, 80, 80, 32]
  420. Tensor("tower0/pool0/MaxPool:0", shape=(?, 40, 40, 32), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  421. [0130 17:29:58 @_common.py:70] pool0 output: [None, 40, 40, 32]
  422. [0130 17:29:58 @_common.py:61] conv1 input: [None, 40, 40, 32]
  423. [0130 17:29:58 @_common.py:61] conv0 input: [None, 84, 84, 16]
  424. Tensor("tower0/conv1/output:0", shape=(?, 36, 36, 32), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  425. [0130 17:29:58 @_common.py:70] conv1 output: [None, 36, 36, 32]
  426. [0130 17:29:58 @_common.py:61] pool1 input: [None, 36, 36, 32]
  427. Tensor("tower0/pool1/MaxPool:0", shape=(?, 18, 18, 32), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  428. [0130 17:29:58 @_common.py:70] pool1 output: [None, 18, 18, 32]
  429. [0130 17:29:58 @_common.py:61] conv2 input: [None, 18, 18, 32]
  430. Tensor("tower0/conv0/output:0", shape=(?, 80, 80, 32), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  431. [0130 17:29:58 @_common.py:70] conv0 output: [None, 80, 80, 32]
  432. [0130 17:29:58 @_common.py:61] pool0 input: [None, 80, 80, 32]
  433. Tensor("tower0/conv2/output:0", shape=(?, 14, 14, 64), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  434. [0130 17:29:58 @_common.py:70] conv2 output: [None, 14, 14, 64]
  435. Tensor("tower0/pool0/MaxPool:0", shape=(?, 40, 40, 32), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  436. [0130 17:29:58 @_common.py:70] pool0 output: [None, 40, 40, 32]
  437. [0130 17:29:58 @_common.py:61] conv1 input: [None, 40, 40, 32]
  438. [0130 17:29:58 @_common.py:61] pool2 input: [None, 14, 14, 64]
  439. Tensor("tower0/pool2/MaxPool:0", shape=(?, 7, 7, 64), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  440. [0130 17:29:58 @_common.py:70] pool2 output: [None, 7, 7, 64]
  441. [0130 17:29:58 @_common.py:61] conv3 input: [None, 7, 7, 64]
  442. Tensor("tower0/conv1/output:0", shape=(?, 36, 36, 32), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  443. [0130 17:29:58 @_common.py:70] conv1 output: [None, 36, 36, 32]
  444. Tensor("tower0/conv3/output:0", shape=(?, 5, 5, 64), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  445. [0130 17:29:58 @_common.py:70] conv3 output: [None, 5, 5, 64]
  446. [0130 17:29:58 @_common.py:61] pool1 input: [None, 36, 36, 32]
  447. Tensor("tower0/pool1/MaxPool:0", shape=(?, 18, 18, 32), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  448. [0130 17:29:58 @_common.py:70] pool1 output: [None, 18, 18, 32]
  449. [0130 17:29:58 @_common.py:61] conv2 input: [None, 18, 18, 32]
  450. [0130 17:29:58 @_common.py:61] fc1_0 input: [None, 5, 5, 64]
  451. Tensor("tower0/conv2/output:0", shape=(?, 14, 14, 64), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  452. [0130 17:29:58 @_common.py:70] conv2 output: [None, 14, 14, 64]
  453. Tensor("tower0/fc1_0/output:0", shape=(?, 1, 1, 256), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  454. [0130 17:29:58 @_common.py:70] fc1_0 output: [None, 1, 1, 256]
  455. [0130 17:29:58 @_common.py:61] pool2 input: [None, 14, 14, 64]
  456. Tensor("tower0/pool2/MaxPool:0", shape=(?, 7, 7, 64), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  457. [0130 17:29:58 @_common.py:70] pool2 output: [None, 7, 7, 64]
  458. [0130 17:29:58 @_common.py:61] conv3 input: [None, 7, 7, 64]
  459. [0130 17:29:58 @_common.py:61] fc-pi input: [None, 256]
  460. Tensor("tower0/conv3/output:0", shape=(?, 5, 5, 64), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  461. [0130 17:29:58 @_common.py:70] conv3 output: [None, 5, 5, 64]
  462. [0130 17:29:58 @_common.py:61] fc1_0 input: [None, 5, 5, 64]
  463. Tensor("tower0/fc1_0/output:0", shape=(?, 1, 1, 256), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  464. [0130 17:29:58 @_common.py:70] fc1_0 output: [None, 1, 1, 256]
  465. Tensor("tower0/fc-pi/output:0", shape=(?, 6), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  466. [0130 17:29:58 @_common.py:70] fc-pi output: [None, 6]
  467. [0130 17:29:58 @_common.py:61] fc-v input: [None, 256]
  468. [0130 17:29:58 @_common.py:61] fc-pi input: [None, 256]
  469. Tensor("tower0/fc-pi/output:0", shape=(?, 6), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  470. [0130 17:29:58 @_common.py:70] fc-pi output: [None, 6]
  471. [0130 17:29:58 @_common.py:61] fc-v input: [None, 256]
  472. Tensor("tower0/fc-v/output:0", shape=(?, 1), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  473. [0130 17:29:58 @_common.py:70] fc-v output: [None, 1]
  474. Tensor("tower0/fc-v/output:0", shape=(?, 1), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  475. [0130 17:29:58 @_common.py:70] fc-v output: [None, 1]
  476. MOVING_SUMMARY_VARIABLES
  477. []
  478. [0130 17:29:58 @modelutils.py:22] Model Parameters:
  479. conv0/W:0: shape=[5, 5, 16, 32], dim=12800
  480. conv1/W:0: shape=[5, 5, 32, 32], dim=25600
  481. conv2/W:0: shape=[5, 5, 32, 64], dim=51200
  482. conv3/W:0: shape=[3, 3, 64, 64], dim=36864
  483. fc1_0/W:0: shape=[5, 5, 64, 256], dim=409600
  484. fc-pi/W:0: shape=[256, 6], dim=1536
  485. fc-pi/b:0: shape=[6], dim=6
  486. fc-v/W:0: shape=[256, 1], dim=256
  487. fc-v/b:0: shape=[1], dim=1
  488. Total param=537863 (2.051785 MB assuming all float32)
  489. MOVING_SUMMARY_VARIABLES
  490. []
  491. [0130 17:29:59 @modelutils.py:22] Model Parameters:
  492. conv0/W:0: shape=[5, 5, 16, 32], dim=12800
  493. conv1/W:0: shape=[5, 5, 32, 32], dim=25600
  494. conv2/W:0: shape=[5, 5, 32, 64], dim=51200
  495. conv3/W:0: shape=[3, 3, 64, 64], dim=36864
  496. fc1_0/W:0: shape=[5, 5, 64, 256], dim=409600
  497. fc-pi/W:0: shape=[256, 6], dim=1536
  498. fc-pi/b:0: shape=[6], dim=6
  499. fc-v/W:0: shape=[256, 1], dim=256
  500. fc-v/b:0: shape=[1], dim=1
  501. Total param=537863 (2.051785 MB assuming all float32)
  502. [0130 17:30:00 @multigpu.py:228] Setup callbacks ...
  503. Creating Predictorfactor 0
  504. [0130 17:30:00 @base.py:132] Building graph for predictor tower 0...
  505. 2018-01-30 17:30:00.118372: I tensorflow/core/distributed_runtime/master_session.cc:999] Start master session f133859301d86503 with config:
  506.  
  507. 12
  508. Tensor("towerp0/conv0/output:0", shape=(?, 80, 80, 32), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  509. [0130 17:30:00 @multigpu.py:228] Setup callbacks ...
  510. Tensor("towerp0/pool0/MaxPool:0", shape=(?, 40, 40, 32), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  511. Creating Predictorfactor 0
  512. [0130 17:30:00 @base.py:132] Building graph for predictor tower 0...
  513. Tensor("towerp0/conv1/output:0", shape=(?, 36, 36, 32), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  514. Tensor("towerp0/pool1/MaxPool:0", shape=(?, 18, 18, 32), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  515. Tensor("towerp0/conv2/output:0", shape=(?, 14, 14, 64), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  516. Tensor("towerp0/pool2/MaxPool:0", shape=(?, 7, 7, 64), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  517. 12
  518. Tensor("towerp0/conv3/output:0", shape=(?, 5, 5, 64), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  519. Tensor("towerp0/fc1_0/output:0", shape=(?, 1, 1, 256), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  520. Tensor("towerp0/conv0/output:0", shape=(?, 80, 80, 32), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  521. Tensor("towerp0/fc-pi/output:0", shape=(?, 6), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  522. Tensor("towerp0/pool0/MaxPool:0", shape=(?, 40, 40, 32), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  523. Tensor("towerp0/conv1/output:0", shape=(?, 36, 36, 32), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  524. Tensor("towerp0/pool1/MaxPool:0", shape=(?, 18, 18, 32), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  525. Tensor("towerp0/fc-v/output:0", shape=(?, 1), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  526. Tensor("towerp0/conv2/output:0", shape=(?, 14, 14, 64), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  527. Tensor("towerp0/pool2/MaxPool:0", shape=(?, 7, 7, 64), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  528. Tensor("towerp0/conv3/output:0", shape=(?, 5, 5, 64), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  529. Tensor("towerp0/fc1_0/output:0", shape=(?, 1, 1, 256), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  530. Tensor("towerp0/fc-pi/output:0", shape=(?, 6), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  531. Tensor("towerp0/fc-v/output:0", shape=(?, 1), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  532. [0130 17:30:00 @base.py:132] Building graph for predictor tower 0...
  533. 12
  534. Tensor("towerp0_1/conv0/output:0", shape=(?, 80, 80, 32), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  535. Tensor("towerp0_1/pool0/MaxPool:0", shape=(?, 40, 40, 32), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  536. Tensor("towerp0_1/conv1/output:0", shape=(?, 36, 36, 32), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  537. [0130 17:30:00 @base.py:132] Building graph for predictor tower 0...
  538. Tensor("towerp0_1/pool1/MaxPool:0", shape=(?, 18, 18, 32), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  539. Tensor("towerp0_1/conv2/output:0", shape=(?, 14, 14, 64), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  540. Tensor("towerp0_1/pool2/MaxPool:0", shape=(?, 7, 7, 64), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  541. Tensor("towerp0_1/conv3/output:0", shape=(?, 5, 5, 64), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  542. 12
  543. Tensor("towerp0_1/fc1_0/output:0", shape=(?, 1, 1, 256), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  544. Tensor("towerp0_1/fc-pi/output:0", shape=(?, 6), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  545. Tensor("towerp0_1/conv0/output:0", shape=(?, 80, 80, 32), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  546. Tensor("towerp0_1/pool0/MaxPool:0", shape=(?, 40, 40, 32), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  547. Tensor("towerp0_1/fc-v/output:0", shape=(?, 1), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  548. Tensor("towerp0_1/conv1/output:0", shape=(?, 36, 36, 32), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  549. Tensor("towerp0_1/pool1/MaxPool:0", shape=(?, 18, 18, 32), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  550. Tensor("towerp0_1/conv2/output:0", shape=(?, 14, 14, 64), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  551. Tensor("towerp0_1/pool2/MaxPool:0", shape=(?, 7, 7, 64), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  552. Tensor("towerp0_1/conv3/output:0", shape=(?, 5, 5, 64), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  553. Tensor("towerp0_1/fc1_0/output:0", shape=(?, 1, 1, 256), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  554. [0130 17:30:00 @base.py:132] Building graph for predictor tower 0...
  555. Tensor("towerp0_1/fc-pi/output:0", shape=(?, 6), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  556. Tensor("towerp0_1/fc-v/output:0", shape=(?, 1), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  557. 12
  558. Tensor("towerp0_2/conv0/output:0", shape=(?, 80, 80, 32), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  559. Tensor("towerp0_2/pool0/MaxPool:0", shape=(?, 40, 40, 32), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  560. Tensor("towerp0_2/conv1/output:0", shape=(?, 36, 36, 32), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  561. Tensor("towerp0_2/pool1/MaxPool:0", shape=(?, 18, 18, 32), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  562. Tensor("towerp0_2/conv2/output:0", shape=(?, 14, 14, 64), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  563. Tensor("towerp0_2/pool2/MaxPool:0", shape=(?, 7, 7, 64), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  564. [0130 17:30:00 @base.py:132] Building graph for predictor tower 0...
  565. Tensor("towerp0_2/conv3/output:0", shape=(?, 5, 5, 64), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  566. Tensor("towerp0_2/fc1_0/output:0", shape=(?, 1, 1, 256), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  567. 12
  568. Tensor("towerp0_2/fc-pi/output:0", shape=(?, 6), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  569. Tensor("towerp0_2/fc-v/output:0", shape=(?, 1), dtype=float32, device=/job:worker/task:3/device:CPU:0)
  570. Tensor("towerp0_2/conv0/output:0", shape=(?, 80, 80, 32), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  571. Tensor("towerp0_2/pool0/MaxPool:0", shape=(?, 40, 40, 32), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  572. Tensor("towerp0_2/conv1/output:0", shape=(?, 36, 36, 32), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  573. Tensor("towerp0_2/pool1/MaxPool:0", shape=(?, 18, 18, 32), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  574. Tensor("towerp0_2/conv2/output:0", shape=(?, 14, 14, 64), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  575. Tensor("towerp0_2/pool2/MaxPool:0", shape=(?, 7, 7, 64), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  576. Tensor("towerp0_2/conv3/output:0", shape=(?, 5, 5, 64), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  577. Tensor("towerp0_2/fc1_0/output:0", shape=(?, 1, 1, 256), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  578. [0130 17:30:00 @base.py:177] ===============================================================
  579. [0130 17:30:00 @base.py:180] [p0576] Creating the session
  580. [0130 17:30:00 @base.py:181] ===============================================================
  581. Tensor("towerp0_2/fc-pi/output:0", shape=(?, 6), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  582. Tensor("towerp0_2/fc-v/output:0", shape=(?, 1), dtype=float32, device=/job:worker/task:2/device:CPU:0)
  583. [0130 17:30:00 @base.py:177] ===============================================================
  584. [0130 17:30:00 @base.py:180] [p0574] Creating the session
  585. [0130 17:30:00 @base.py:181] ===============================================================
  586. 2018-01-30 17:30:00.790835: I tensorflow/core/distributed_runtime/master_session.cc:999] Start master session ce42849ba70813b5 with config:
  587.  
  588. [0130 17:30:01 @base.py:189] ===============================================================
  589. [0130 17:30:01 @base.py:190] [p0112] Session created
  590. [0130 17:30:01 @base.py:191] ===============================================================
  591. [0130 17:30:01 @base.py:112] [p0112] Initializing graph variables ...
  592. [0130 17:30:01 @base.py:119] [p0112] Starting concurrency...
  593. [0130 17:30:01 @base.py:198] Starting all threads & procs ...
  594. [0130 17:30:01 @base.py:122] [p0112] Setting default session
  595. [0130 17:30:01 @base.py:125] [p0112] Getting global step
  596. [0130 17:30:01 @base.py:127] [p0112] Start training with global_step=0
  597. server main loop
  598. before socket bind... tcp://*:61216
  599. receiving
  600. 2018-01-30 17:30:01.209105: I tensorflow/core/distributed_runtime/master_session.cc:999] Start master session 2eefc7caccbf1c89 with config:
  601.  
  602. 2018-01-30 17:30:01.276788: I tensorflow/core/distributed_runtime/master_session.cc:999] Start master session e774dcd3d1ac3165 with config:
  603.  
  604. [0130 17:30:01 @base.py:189] ===============================================================
  605. [0130 17:30:01 @base.py:190] [p0576] Session created
  606. [0130 17:30:01 @base.py:191] ===============================================================
  607. [0130 17:30:01 @base.py:112] [p0576] Initializing graph variables ...
  608. [0130 17:30:01 @base.py:119] [p0576] Starting concurrency...
  609. [0130 17:30:01 @base.py:198] Starting all threads & procs ...
  610. [0130 17:30:01 @base.py:122] [p0576] Setting default session
  611. [0130 17:30:01 @base.py:125] [p0576] Getting global step
  612. [0130 17:30:01 @base.py:127] [p0576] Start training with global_step=0
  613. [0130 17:30:01 @base.py:189] ===============================================================
  614. [0130 17:30:01 @base.py:190] [p0574] Session created
  615. [0130 17:30:01 @base.py:191] ===============================================================
  616. [0130 17:30:01 @base.py:112] [p0574] Initializing graph variables ...
  617. [0130 17:30:01 @base.py:119] [p0574] Starting concurrency...
  618. [0130 17:30:01 @base.py:198] Starting all threads & procs ...
  619. [0130 17:30:01 @base.py:122] [p0574] Setting default session
  620. [0130 17:30:01 @base.py:125] [p0574] Getting global step
  621. [0130 17:30:01 @base.py:127] [p0574] Start training with global_step=0
  622. [0130 17:30:03 @multigpu.py:323] [p0576] step: count(1), step_time 4762.69, mean_step_time 4762.69, it/s 0.21
  623. [0130 17:30:03 @multigpu.py:323] [p0574] step: count(1), step_time 4870.79, mean_step_time 4870.79, it/s 0.21
  624. [0130 17:30:05 @multigpu.py:323] [p0576] step: count(2), step_time 1437.01, mean_step_time 3099.85, it/s 0.32
  625. [0130 17:30:05 @multigpu.py:323] [p0574] step: count(2), step_time 1388.9, mean_step_time 3129.85, it/s 0.32
  626. [0130 17:30:06 @multigpu.py:323] [p0576] step: count(3), step_time 1392.92, mean_step_time 2530.87, it/s 0.4
  627. [0130 17:30:06 @multigpu.py:323] [p0574] step: count(3), step_time 1417.79, mean_step_time 2559.16, it/s 0.39
  628. [0130 17:30:07 @multigpu.py:323] [p0576] step: count(4), step_time 1381.3, mean_step_time 2243.48, it/s 0.45
  629. [0130 17:30:08 @multigpu.py:323] [p0574] step: count(4), step_time 1380.92, mean_step_time 2264.6, it/s 0.44
  630. [0130 17:30:09 @multigpu.py:323] [p0576] step: count(5), step_time 1413.79, mean_step_time 2077.54, it/s 0.48
  631. [0130 17:30:09 @multigpu.py:323] [p0574] step: count(5), step_time 1414.17, mean_step_time 2094.51, it/s 0.48
  632. [0130 17:30:10 @multigpu.py:323] [p0576] step: count(6), step_time 1387.62, mean_step_time 1962.56, it/s 0.51
  633. [0130 17:30:10 @multigpu.py:323] [p0574] step: count(6), step_time 1367.55, mean_step_time 1973.35, it/s 0.51
  634. [0130 17:30:12 @multigpu.py:323] [p0576] step: count(7), step_time 1363.13, mean_step_time 1876.92, it/s 0.53
  635. [0130 17:30:12 @multigpu.py:323] [p0574] step: count(7), step_time 1377.66, mean_step_time 1888.25, it/s 0.53
  636. [0130 17:30:13 @multigpu.py:323] [p0574] step: count(8), step_time 1369.67, mean_step_time 1823.43, it/s 0.55
  637. [0130 17:30:13 @multigpu.py:323] [p0576] step: count(8), step_time 1444.03, mean_step_time 1822.81, it/s 0.55
  638. [0130 17:30:14 @multigpu.py:323] [p0574] step: count(9), step_time 1376.32, mean_step_time 1773.75, it/s 0.56
  639. [0130 17:30:15 @multigpu.py:323] [p0576] step: count(9), step_time 1402.34, mean_step_time 1776.09, it/s 0.56
  640. [0130 17:30:16 @multigpu.py:323] [p0574] step: count(10), step_time 1358.5, mean_step_time 1732.23, it/s 0.58
  641. [0130 17:30:16 @multigpu.py:323] [p0576] step: count(10), step_time 1372.5, mean_step_time 1735.73, it/s 0.58
  642. [0130 17:30:17 @multigpu.py:323] [p0574] step: count(11), step_time 1402.19, mean_step_time 1702.22, it/s 0.59
  643. [0130 17:30:17 @multigpu.py:323] [p0576] step: count(11), step_time 1415.18, mean_step_time 1706.59, it/s 0.59
  644. [0130 17:30:19 @multigpu.py:323] [p0574] step: count(12), step_time 1360.35, mean_step_time 1673.73, it/s 0.6
  645. [0130 17:30:19 @multigpu.py:323] [p0576] step: count(12), step_time 1386.91, mean_step_time 1679.95, it/s 0.6
  646. [0130 17:30:20 @multigpu.py:323] [p0574] step: count(13), step_time 1434.23, mean_step_time 1655.31, it/s 0.6
  647. [0130 17:30:20 @multigpu.py:323] [p0576] step: count(13), step_time 1391.04, mean_step_time 1657.73, it/s 0.6
  648. [0130 17:30:21 @multigpu.py:323] [p0574] step: count(14), step_time 1350.67, mean_step_time 1633.55, it/s 0.61
  649. [0130 17:30:22 @multigpu.py:323] [p0576] step: count(14), step_time 1425.22, mean_step_time 1641.12, it/s 0.61
  650. [0130 17:30:23 @multigpu.py:323] [p0574] step: count(15), step_time 1427.02, mean_step_time 1619.78, it/s 0.62
  651. [0130 17:30:23 @multigpu.py:323] [p0576] step: count(15), step_time 1376.45, mean_step_time 1623.48, it/s 0.62
  652. [0130 17:30:24 @multigpu.py:323] [p0574] step: count(16), step_time 1363.51, mean_step_time 1603.76, it/s 0.62
  653. [0130 17:30:24 @multigpu.py:323] [p0576] step: count(16), step_time 1441.44, mean_step_time 1612.1, it/s 0.62
  654. [0130 17:30:26 @multigpu.py:323] [p0574] step: count(17), step_time 1414.29, mean_step_time 1592.62, it/s 0.63
  655. [0130 17:30:26 @multigpu.py:323] [p0576] step: count(17), step_time 1441.84, mean_step_time 1602.08, it/s 0.62
  656. [0130 17:30:27 @multigpu.py:323] [p0574] step: count(18), step_time 1425.53, mean_step_time 1583.34, it/s 0.63
  657. [0130 17:30:27 @multigpu.py:323] [p0576] step: count(18), step_time 1347.63, mean_step_time 1587.95, it/s 0.63
  658. [0130 17:30:28 @multigpu.py:323] [p0574] step: count(19), step_time 1392.45, mean_step_time 1573.29, it/s 0.64
  659. [0130 17:30:29 @multigpu.py:323] [p0576] step: count(19), step_time 1404.59, mean_step_time 1578.3, it/s 0.63
  660. [0130 17:30:30 @multigpu.py:323] [p0574] step: count(20), step_time 1396.35, mean_step_time 1564.44, it/s 0.64
  661. [0130 17:30:30 @multigpu.py:323] [p0576] step: count(20), step_time 1363.71, mean_step_time 1567.57, it/s 0.64
  662. 2018-01-30 17:30:30.682429: I tensorflow/core/distributed_runtime/master_session.cc:999] Start master session 8454efd2be83293e with config:
  663.  
  664. [0130 17:30:30 @base.py:189] ===============================================================
  665. [0130 17:30:30 @base.py:190] [p0115] Session created
  666. [0130 17:30:30 @base.py:191] ===============================================================
  667. [0130 17:30:30 @base.py:112] [p0115] Initializing graph variables ...
  668. [0130 17:30:30 @base.py:119] [p0115] Starting concurrency...
  669. [0130 17:30:30 @base.py:198] Starting all threads & procs ...
  670. [0130 17:30:30 @base.py:122] [p0115] Setting default session
  671. [0130 17:30:30 @base.py:125] [p0115] Getting global step
  672. [0130 17:30:30 @base.py:127] [p0115] Start training with global_step=0
  673. [0130 17:30:31 @multigpu.py:323] [p0574] step: count(21), step_time 1395.6, mean_step_time 1390.68, it/s 0.72
  674. [0130 17:30:31 @multigpu.py:323] [p0576] step: count(21), step_time 1391.65, mean_step_time 1399.02, it/s 0.71
  675. sending to address tcp://p0112:61216
  676. ##### Sending to neptune: online_score : 0.00865667495463 , 1.2 #####
  677. [u'online', 1.2]
  678. receiving
  679. [0130 17:30:33 @multigpu.py:323] [p0574] step: count(22), step_time 1402.65, mean_step_time 1391.37, it/s 0.72
  680. [0130 17:30:33 @multigpu.py:323] [p0576] step: count(22), step_time 1399.34, mean_step_time 1397.13, it/s 0.72
  681. [0130 17:30:33 @multigpu.py:323] [p0115] step: count(1), step_time 42890.76, mean_step_time 42890.76, it/s 0.02
  682. [0130 17:30:34 @multigpu.py:323] [p0574] step: count(23), step_time 1389.17, mean_step_time 1389.94, it/s 0.72
  683. [0130 17:30:34 @multigpu.py:323] [p0576] step: count(23), step_time 1378.85, mean_step_time 1396.43, it/s 0.72
  684. [0130 17:30:34 @multigpu.py:323] [p0115] step: count(2), step_time 1448.3, mean_step_time 22169.53, it/s 0.05
  685. [0130 17:30:35 @multigpu.py:323] [p0574] step: count(24), step_time 1366.48, mean_step_time 1389.22, it/s 0.72
  686. [0130 17:30:35 @multigpu.py:323] [p0576] step: count(24), step_time 1417.93, mean_step_time 1398.26, it/s 0.72
  687. [0130 17:30:36 @multigpu.py:323] [p0115] step: count(3), step_time 1356.88, mean_step_time 15231.98, it/s 0.07
  688. [0130 17:30:37 @multigpu.py:323] [p0574] step: count(25), step_time 1368.03, mean_step_time 1386.91, it/s 0.72
  689. [0130 17:30:37 @multigpu.py:323] [p0576] step: count(25), step_time 1394.76, mean_step_time 1397.31, it/s 0.72
  690. [0130 17:30:37 @multigpu.py:323] [p0115] step: count(4), step_time 1482.5, mean_step_time 11794.61, it/s 0.08
  691. [0130 17:30:38 @multigpu.py:323] [p0574] step: count(26), step_time 1432.12, mean_step_time 1390.14, it/s 0.72
  692. [0130 17:30:38 @multigpu.py:323] [p0576] step: count(26), step_time 1385.95, mean_step_time 1397.23, it/s 0.72
  693. [0130 17:30:38 @multigpu.py:323] [p0115] step: count(5), step_time 1416.5, mean_step_time 9718.99, it/s 0.1
  694. [0130 17:30:40 @multigpu.py:323] [p0574] step: count(27), step_time 1389.58, mean_step_time 1390.73, it/s 0.72
  695. [0130 17:30:40 @multigpu.py:323] [p0576] step: count(27), step_time 1469.08, mean_step_time 1402.52, it/s 0.71
  696. [0130 17:30:40 @multigpu.py:323] [p0115] step: count(6), step_time 1373.32, mean_step_time 8328.04, it/s 0.12
  697. [0130 17:30:41 @multigpu.py:323] [p0574] step: count(28), step_time 1414.32, mean_step_time 1392.97, it/s 0.72
  698. [0130 17:30:41 @multigpu.py:323] [p0576] step: count(28), step_time 1390.28, mean_step_time 1399.83, it/s 0.71
  699. [0130 17:30:41 @multigpu.py:323] [p0115] step: count(7), step_time 1393.66, mean_step_time 7337.41, it/s 0.14
  700. [0130 17:30:42 @multigpu.py:323] [p0574] step: count(29), step_time 1395.04, mean_step_time 1393.9, it/s 0.72
  701. sending to address tcp://p0112:61216
  702. ##### Sending to neptune: online_score : 0.0116006455157 , 1.9 #####
  703. [u'online', 1.9]
  704. receiving
  705. [0130 17:30:43 @multigpu.py:323] [p0576] step: count(29), step_time 1390.76, mean_step_time 1399.26, it/s 0.71
  706. [0130 17:30:43 @multigpu.py:323] [p0115] step: count(8), step_time 1377.49, mean_step_time 6592.42, it/s 0.15
  707. [0130 17:30:44 @multigpu.py:323] [p0574] step: count(30), step_time 1366.0, mean_step_time 1394.28, it/s 0.72
  708. [0130 17:30:44 @multigpu.py:323] [p0576] step: count(30), step_time 1366.41, mean_step_time 1398.95, it/s 0.71
  709. [0130 17:30:44 @multigpu.py:323] [p0115] step: count(9), step_time 1394.2, mean_step_time 6014.84, it/s 0.17
  710. [0130 17:30:45 @multigpu.py:323] [p0574] step: count(31), step_time 1413.87, mean_step_time 1394.86, it/s 0.72
  711. [0130 17:30:45 @multigpu.py:323] [p0576] step: count(31), step_time 1402.14, mean_step_time 1398.3, it/s 0.72
  712. [0130 17:30:45 @multigpu.py:323] [p0115] step: count(10), step_time 1394.34, mean_step_time 5552.79, it/s 0.18
  713. [0130 17:30:47 @multigpu.py:323] [p0574] step: count(32), step_time 1385.71, mean_step_time 1396.13, it/s 0.72
  714. [0130 17:30:47 @multigpu.py:323] [p0576] step: count(32), step_time 1385.07, mean_step_time 1398.21, it/s 0.72
  715. [0130 17:30:47 @multigpu.py:323] [p0115] step: count(11), step_time 1371.74, mean_step_time 5172.7, it/s 0.19
  716. [0130 17:30:48 @multigpu.py:323] [p0574] step: count(33), step_time 1395.79, mean_step_time 1394.21, it/s 0.72
  717. [0130 17:30:48 @multigpu.py:323] [p0576] step: count(33), step_time 1399.75, mean_step_time 1398.64, it/s 0.71
  718. [0130 17:30:48 @multigpu.py:323] [p0115] step: count(12), step_time 1378.85, mean_step_time 4856.54, it/s 0.21
  719. [0130 17:30:49 @multigpu.py:323] [p0574] step: count(34), step_time 1391.1, mean_step_time 1396.23, it/s 0.72
  720. [0130 17:30:49 @multigpu.py:323] [p0576] step: count(34), step_time 1391.81, mean_step_time 1396.97, it/s 0.72
  721. [0130 17:30:50 @multigpu.py:323] [p0115] step: count(13), step_time 1400.51, mean_step_time 4590.69, it/s 0.22
  722. [0130 17:30:51 @multigpu.py:323] [p0574] step: count(35), step_time 1359.27, mean_step_time 1392.84, it/s 0.72
  723. [0130 17:30:51 @multigpu.py:323] [p0576] step: count(35), step_time 1379.89, mean_step_time 1397.15, it/s 0.72
  724. [0130 17:30:51 @multigpu.py:323] [p0115] step: count(14), step_time 1362.87, mean_step_time 4360.14, it/s 0.23
  725. [0130 17:30:52 @multigpu.py:323] [p0574] step: count(36), step_time 1472.95, mean_step_time 1398.32, it/s 0.72
  726. [0130 17:30:52 @multigpu.py:323] [p0576] step: count(36), step_time 1389.22, mean_step_time 1394.53, it/s 0.72
  727. [0130 17:30:52 @multigpu.py:323] [p0115] step: count(15), step_time 1390.51, mean_step_time 4162.16, it/s 0.24
  728. [0130 17:30:54 @multigpu.py:323] [p0574] step: count(37), step_time 1416.91, mean_step_time 1398.45, it/s 0.72
  729. [0130 17:30:54 @multigpu.py:323] [p0576] step: count(37), step_time 1372.71, mean_step_time 1391.08, it/s 0.72
  730. [0130 17:30:54 @multigpu.py:323] [p0115] step: count(16), step_time 1361.38, mean_step_time 3987.11, it/s 0.25
  731. [0130 17:30:55 @multigpu.py:323] [p0576] step: count(38), step_time 1411.73, mean_step_time 1394.28, it/s 0.72
  732. [0130 17:30:55 @multigpu.py:323] [p0115] step: count(17), step_time 1413.07, mean_step_time 3835.7, it/s 0.26
  733. [0130 17:30:55 @multigpu.py:323] [p0574] step: count(38), step_time 1520.18, mean_step_time 1403.18, it/s 0.71
  734. [0130 17:30:56 @multigpu.py:323] [p0115] step: count(18), step_time 1368.26, mean_step_time 3698.62, it/s 0.27
  735. [0130 17:30:56 @multigpu.py:323] [p0576] step: count(39), step_time 1408.37, mean_step_time 1394.47, it/s 0.72
  736. [0130 17:30:57 @multigpu.py:323] [p0574] step: count(39), step_time 1564.09, mean_step_time 1411.76, it/s 0.71
  737. [0130 17:30:58 @multigpu.py:323] [p0115] step: count(19), step_time 1387.09, mean_step_time 3576.96, it/s 0.28
  738. [0130 17:30:58 @multigpu.py:323] [p0576] step: count(40), step_time 1374.96, mean_step_time 1395.03, it/s 0.72
  739. [0130 17:30:58 @multigpu.py:323] [p0574] step: count(40), step_time 1608.55, mean_step_time 1422.37, it/s 0.7
  740. [0130 17:30:59 @multigpu.py:323] [p0576] step: count(41), step_time 1359.14, mean_step_time 1393.41, it/s 0.72
  741. [0130 17:30:59 @multigpu.py:323] [p0115] step: count(20), step_time 1390.59, mean_step_time 3467.64, it/s 0.29
  742. [0130 17:31:00 @multigpu.py:323] [p0574] step: count(41), step_time 1480.41, mean_step_time 1426.61, it/s 0.7
  743. [0130 17:31:01 @multigpu.py:323] [p0576] step: count(42), step_time 1370.21, mean_step_time 1391.95, it/s 0.72
  744. [0130 17:31:01 @multigpu.py:323] [p0115] step: count(21), step_time 1360.8, mean_step_time 1391.14, it/s 0.72
  745. [0130 17:31:01 @multigpu.py:323] [p0574] step: count(42), step_time 1462.71, mean_step_time 1429.61, it/s 0.7
  746. [0130 17:31:02 @multigpu.py:323] [p0576] step: count(43), step_time 1372.3, mean_step_time 1391.62, it/s 0.72
  747. [0130 17:31:02 @multigpu.py:323] [p0115] step: count(22), step_time 1454.88, mean_step_time 1391.47, it/s 0.72
  748. sending to address tcp://p0112:61216
  749. ##### Sending to neptune: online_score : 0.0171053697003 , 1.2 #####
  750. [u'online', 1.2]
  751. ##### Sending to neptune: active_workers : 0.0171054766575 , 2 #####
  752. receiving
  753. [0130 17:31:03 @multigpu.py:323] [p0574] step: count(43), step_time 1458.14, mean_step_time 1433.06, it/s 0.7
  754. [0130 17:31:03 @multigpu.py:323] [p0576] step: count(44), step_time 1351.7, mean_step_time 1388.31, it/s 0.72
  755. [0130 17:31:03 @multigpu.py:323] [p0115] step: count(23), step_time 1368.19, mean_step_time 1392.04, it/s 0.72
  756. [0130 17:31:04 @multigpu.py:323] [p0574] step: count(44), step_time 1500.34, mean_step_time 1439.76, it/s 0.69
  757. [0130 17:31:05 @multigpu.py:323] [p0576] step: count(45), step_time 1414.58, mean_step_time 1389.3, it/s 0.72
  758. [0130 17:31:05 @multigpu.py:323] [p0115] step: count(24), step_time 1397.49, mean_step_time 1387.79, it/s 0.72
  759. [0130 17:31:06 @multigpu.py:323] [p0574] step: count(45), step_time 1476.14, mean_step_time 1445.16, it/s 0.69
  760. [0130 17:31:06 @multigpu.py:323] [p0576] step: count(46), step_time 1353.29, mean_step_time 1387.67, it/s 0.72
  761. [0130 17:31:06 @multigpu.py:323] [p0115] step: count(25), step_time 1354.62, mean_step_time 1384.69, it/s 0.72
  762. [0130 17:31:07 @multigpu.py:323] [p0574] step: count(46), step_time 1478.86, mean_step_time 1447.5, it/s 0.69
  763. [0130 17:31:07 @multigpu.py:323] [p0576] step: count(47), step_time 1360.59, mean_step_time 1382.25, it/s 0.72
  764. [0130 17:31:08 @multigpu.py:323] [p0115] step: count(26), step_time 1414.51, mean_step_time 1386.75, it/s 0.72
  765. [0130 17:31:09 @multigpu.py:323] [p0574] step: count(47), step_time 1581.36, mean_step_time 1457.09, it/s 0.69
  766. [0130 17:31:10 @multigpu.py:323] [p0574] step: count(48), step_time 1608.18, mean_step_time 1466.78, it/s 0.68
  767. [0130 17:31:10 @multigpu.py:323] [p0576] step: count(48), step_time 2931.45, mean_step_time 1459.3, it/s 0.69
  768. [0130 17:31:10 @multigpu.py:323] [p0115] step: count(27), step_time 2777.06, mean_step_time 1455.92, it/s 0.69
  769. sending to address tcp://p0112:61216
  770. ##### Sending to neptune: online_score : 0.0195416252481 , 1.9 #####
  771. [u'online', 1.9]
  772. receiving
  773. [0130 17:31:12 @multigpu.py:323] [p0576] step: count(49), step_time 1407.16, mean_step_time 1460.12, it/s 0.68
  774. [0130 17:31:12 @multigpu.py:323] [p0115] step: count(28), step_time 1437.32, mean_step_time 1458.91, it/s 0.69
  775. [0130 17:31:12 @multigpu.py:323] [p0574] step: count(49), step_time 1500.8, mean_step_time 1472.07, it/s 0.68
  776. [0130 17:31:13 @multigpu.py:323] [p0576] step: count(50), step_time 1344.68, mean_step_time 1459.04, it/s 0.69
  777. [0130 17:31:13 @multigpu.py:323] [p0115] step: count(29), step_time 1368.68, mean_step_time 1457.64, it/s 0.69
  778. [0130 17:31:13 @multigpu.py:323] [p0574] step: count(50), step_time 1415.66, mean_step_time 1474.55, it/s 0.68
  779. [0130 17:31:14 @multigpu.py:323] [p0576] step: count(51), step_time 1377.45, mean_step_time 1457.8, it/s 0.69
  780. [0130 17:31:14 @multigpu.py:323] [p0115] step: count(30), step_time 1373.92, mean_step_time 1456.62, it/s 0.69
  781. [0130 17:31:15 @multigpu.py:323] [p0574] step: count(51), step_time 1430.28, mean_step_time 1475.37, it/s 0.68
  782. sending to address tcp://p0112:61216
  783. ##### Sending to neptune: online_score : 0.0207704544067 , 2.4 #####
  784. [u'online', 2.4]
  785. receiving
  786. [0130 17:31:16 @multigpu.py:323] [p0576] step: count(52), step_time 1360.11, mean_step_time 1456.56, it/s 0.69
  787. [0130 17:31:16 @multigpu.py:323] [p0115] step: count(31), step_time 1392.93, mean_step_time 1457.68, it/s 0.69
  788. [0130 17:31:16 @multigpu.py:323] [p0574] step: count(52), step_time 1571.52, mean_step_time 1484.66, it/s 0.67
  789. [0130 17:31:17 @multigpu.py:323] [p0576] step: count(53), step_time 1366.86, mean_step_time 1454.91, it/s 0.69
  790. [0130 17:31:17 @multigpu.py:323] [p0115] step: count(32), step_time 1387.37, mean_step_time 1458.1, it/s 0.69
  791. [0130 17:31:18 @multigpu.py:323] [p0574] step: count(53), step_time 1407.6, mean_step_time 1485.25, it/s 0.67
  792. [0130 17:31:19 @multigpu.py:323] [p0576] step: count(54), step_time 1391.97, mean_step_time 1454.92, it/s 0.69
  793. [0130 17:31:19 @multigpu.py:323] [p0115] step: count(33), step_time 1400.33, mean_step_time 1458.09, it/s 0.69
  794. [0130 17:31:19 @multigpu.py:323] [p0574] step: count(54), step_time 1387.15, mean_step_time 1485.06, it/s 0.67
  795. [0130 17:31:20 @multigpu.py:323] [p0576] step: count(55), step_time 1381.99, mean_step_time 1455.02, it/s 0.69
  796. [0130 17:31:20 @multigpu.py:323] [p0115] step: count(34), step_time 1369.56, mean_step_time 1458.43, it/s 0.69
  797. [0130 17:31:20 @multigpu.py:323] [p0574] step: count(55), step_time 1407.24, mean_step_time 1487.45, it/s 0.67
  798. [0130 17:31:21 @multigpu.py:323] [p0576] step: count(56), step_time 1388.41, mean_step_time 1454.98, it/s 0.69
  799. [0130 17:31:21 @multigpu.py:323] [p0115] step: count(35), step_time 1419.57, mean_step_time 1459.88, it/s 0.68
  800. [0130 17:31:22 @multigpu.py:323] [p0574] step: count(56), step_time 1437.67, mean_step_time 1485.69, it/s 0.67
  801. [0130 17:31:23 @multigpu.py:323] [p0576] step: count(57), step_time 1392.69, mean_step_time 1455.98, it/s 0.69
  802. [0130 17:31:23 @multigpu.py:323] [p0115] step: count(36), step_time 1377.4, mean_step_time 1460.68, it/s 0.68
  803. [0130 17:31:23 @multigpu.py:323] [p0574] step: count(57), step_time 1357.55, mean_step_time 1482.72, it/s 0.67
  804. [0130 17:31:24 @multigpu.py:323] [p0576] step: count(58), step_time 1387.9, mean_step_time 1454.79, it/s 0.69
  805. [0130 17:31:24 @multigpu.py:323] [p0115] step: count(37), step_time 1381.1, mean_step_time 1459.08, it/s 0.69
  806. sending to address tcp://p0112:61216
  807. ##### Sending to neptune: online_score : 0.0233404641019 , 1.1 #####
  808. [u'online', 1.1]
  809. receiving
  810. [0130 17:31:25 @multigpu.py:323] [p0574] step: count(58), step_time 1408.59, mean_step_time 1477.14, it/s 0.68
  811. [0130 17:31:26 @multigpu.py:323] [p0576] step: count(59), step_time 1427.41, mean_step_time 1455.74, it/s 0.69
  812. [0130 17:31:26 @multigpu.py:323] [p0115] step: count(38), step_time 1400.34, mean_step_time 1460.69, it/s 0.68
  813. [0130 17:31:26 @multigpu.py:323] [p0574] step: count(59), step_time 1421.86, mean_step_time 1470.03, it/s 0.68
  814. [0130 17:31:27 @multigpu.py:323] [p0576] step: count(60), step_time 1393.25, mean_step_time 1456.66, it/s 0.69
  815. [0130 17:31:27 @multigpu.py:323] [p0115] step: count(39), step_time 1392.08, mean_step_time 1460.94, it/s 0.68
  816. [0130 17:31:27 @multigpu.py:323] [p0574] step: count(60), step_time 1420.09, mean_step_time 1460.61, it/s 0.68
  817. [0130 17:31:28 @multigpu.py:323] [p0576] step: count(61), step_time 1368.35, mean_step_time 1457.12, it/s 0.69
  818. [0130 17:31:28 @multigpu.py:323] [p0115] step: count(40), step_time 1355.33, mean_step_time 1459.17, it/s 0.69
  819. [0130 17:31:29 @multigpu.py:323] [p0574] step: count(61), step_time 1562.18, mean_step_time 1464.7, it/s 0.68
  820. [0130 17:31:30 @multigpu.py:323] [p0576] step: count(62), step_time 1390.44, mean_step_time 1458.13, it/s 0.69
  821. [0130 17:31:30 @multigpu.py:323] [p0115] step: count(41), step_time 1372.64, mean_step_time 1459.77, it/s 0.69
  822. [0130 17:31:31 @multigpu.py:323] [p0574] step: count(62), step_time 1453.26, mean_step_time 1464.22, it/s 0.68
  823. [0130 17:31:31 @multigpu.py:323] [p0576] step: count(63), step_time 1373.3, mean_step_time 1458.18, it/s 0.69
  824. [0130 17:31:31 @multigpu.py:323] [p0115] step: count(42), step_time 1391.55, mean_step_time 1456.6, it/s 0.69
  825. [0130 17:31:32 @multigpu.py:323] [p0574] step: count(63), step_time 1447.01, mean_step_time 1463.67, it/s 0.68
  826. [0130 17:31:32 @multigpu.py:323] [p0576] step: count(64), step_time 1352.92, mean_step_time 1458.24, it/s 0.69
  827. [0130 17:31:32 @multigpu.py:323] [p0115] step: count(43), step_time 1351.92, mean_step_time 1455.79, it/s 0.69
  828. [0130 17:31:34 @multigpu.py:323] [p0574] step: count(64), step_time 1567.27, mean_step_time 1467.01, it/s 0.68
  829. [0130 17:31:34 @multigpu.py:323] [p0576] step: count(65), step_time 1410.33, mean_step_time 1458.03, it/s 0.69
  830. [0130 17:31:34 @multigpu.py:323] [p0115] step: count(44), step_time 1417.63, mean_step_time 1456.79, it/s 0.69
  831. [0130 17:31:35 @multigpu.py:323] [p0574] step: count(65), step_time 1457.16, mean_step_time 1466.06, it/s 0.68
  832. [0130 17:31:35 @multigpu.py:323] [p0576] step: count(66), step_time 1330.76, mean_step_time 1456.9, it/s 0.69
  833. [0130 17:31:35 @multigpu.py:323] [p0115] step: count(45), step_time 1425.71, mean_step_time 1460.35, it/s 0.68
  834. sending to address tcp://p0112:61216
  835. ##### Sending to neptune: online_score : 0.0265158143971 , 0.4 #####
  836. [u'online', 0.4]
  837. receiving
  838. [0130 17:31:36 @multigpu.py:323] [p0574] step: count(66), step_time 1510.81, mean_step_time 1467.66, it/s 0.68
  839. [0130 17:31:37 @multigpu.py:323] [p0576] step: count(67), step_time 1398.99, mean_step_time 1458.82, it/s 0.69
  840. [0130 17:31:37 @multigpu.py:323] [p0115] step: count(46), step_time 1376.59, mean_step_time 1458.45, it/s 0.69
  841. [0130 17:31:38 @multigpu.py:323] [p0576] step: count(68), step_time 1353.56, mean_step_time 1379.93, it/s 0.72
  842. [0130 17:31:39 @multigpu.py:323] [p0574] step: count(67), step_time 2849.34, mean_step_time 1531.06, it/s 0.65
  843. [0130 17:31:39 @multigpu.py:323] [p0576] step: count(69), step_time 1413.17, mean_step_time 1380.23, it/s 0.72
  844. [0130 17:31:39 @multigpu.py:323] [p0115] step: count(47), step_time 2621.15, mean_step_time 1450.66, it/s 0.69
  845. [0130 17:31:41 @multigpu.py:323] [p0576] step: count(70), step_time 1364.24, mean_step_time 1381.21, it/s 0.72
  846. [0130 17:31:41 @multigpu.py:323] [p0574] step: count(68), step_time 1413.99, mean_step_time 1521.35, it/s 0.66
  847. [0130 17:31:41 @multigpu.py:323] [p0115] step: count(48), step_time 1445.55, mean_step_time 1451.07, it/s 0.69
  848. sending to address tcp://p0112:61216
  849. ##### Sending to neptune: online_score : 0.0280429810948 , 1.2 #####
  850. [u'online', 1.2]
  851. receiving
  852. [0130 17:31:42 @multigpu.py:323] [p0576] step: count(71), step_time 1389.69, mean_step_time 1381.82, it/s 0.72
  853. [0130 17:31:42 @multigpu.py:323] [p0574] step: count(69), step_time 1360.72, mean_step_time 1514.35, it/s 0.66
  854. [0130 17:31:42 @multigpu.py:323] [p0115] step: count(49), step_time 1398.37, mean_step_time 1452.55, it/s 0.69
  855. [0130 17:31:43 @multigpu.py:323] [p0576] step: count(72), step_time 1337.14, mean_step_time 1380.67, it/s 0.72
  856. [0130 17:31:44 @multigpu.py:323] [p0574] step: count(70), step_time 1395.54, mean_step_time 1513.34, it/s 0.66
  857. [0130 17:31:44 @multigpu.py:323] [p0115] step: count(50), step_time 1376.98, mean_step_time 1452.7, it/s 0.69
  858. [0130 17:31:45 @multigpu.py:323] [p0576] step: count(73), step_time 1400.4, mean_step_time 1382.35, it/s 0.72
  859. [0130 17:31:45 @multigpu.py:323] [p0574] step: count(71), step_time 1398.99, mean_step_time 1511.78, it/s 0.66
  860. [0130 17:31:45 @multigpu.py:323] [p0115] step: count(51), step_time 1356.44, mean_step_time 1450.88, it/s 0.69
  861. [0130 17:31:46 @multigpu.py:323] [p0576] step: count(74), step_time 1366.03, mean_step_time 1381.05, it/s 0.72
  862. [0130 17:31:46 @multigpu.py:323] [p0574] step: count(72), step_time 1349.11, mean_step_time 1500.66, it/s 0.67
  863. [0130 17:31:46 @multigpu.py:323] [p0115] step: count(52), step_time 1407.75, mean_step_time 1451.9, it/s 0.69
  864. [0130 17:31:48 @multigpu.py:323] [p0574] step: count(73), step_time 1410.12, mean_step_time 1500.78, it/s 0.67
  865. [0130 17:31:48 @multigpu.py:323] [p0576] step: count(75), step_time 1482.05, mean_step_time 1386.05, it/s 0.72
  866. [0130 17:31:48 @multigpu.py:323] [p0115] step: count(53), step_time 1391.0, mean_step_time 1451.43, it/s 0.69
  867. [0130 17:31:49 @multigpu.py:323] [p0115] step: count(54), step_time 1359.58, mean_step_time 1450.93, it/s 0.69
  868. [0130 17:31:49 @multigpu.py:323] [p0576] step: count(76), step_time 1401.94, mean_step_time 1386.73, it/s 0.72
  869. [0130 17:31:49 @multigpu.py:323] [p0574] step: count(74), step_time 1418.6, mean_step_time 1502.36, it/s 0.67
  870. sending to address tcp://p0112:61216
  871. ##### Sending to neptune: online_score : 0.0303178360727 , 1.0 #####
  872. [u'online', 1.0]
  873. receiving
  874. [0130 17:31:50 @multigpu.py:323] [p0576] step: count(77), step_time 1349.13, mean_step_time 1384.55, it/s 0.72
  875. [0130 17:31:50 @multigpu.py:323] [p0574] step: count(75), step_time 1366.17, mean_step_time 1500.3, it/s 0.67
  876. [0130 17:31:50 @multigpu.py:323] [p0115] step: count(55), step_time 1408.07, mean_step_time 1450.36, it/s 0.69
  877. [0130 17:31:52 @multigpu.py:323] [p0574] step: count(76), step_time 1361.83, mean_step_time 1496.51, it/s 0.67
  878. [0130 17:31:52 @multigpu.py:323] [p0576] step: count(78), step_time 1419.72, mean_step_time 1386.14, it/s 0.72
  879. [0130 17:31:52 @multigpu.py:323] [p0115] step: count(56), step_time 1383.6, mean_step_time 1450.67, it/s 0.69
  880. [0130 17:31:53 @multigpu.py:323] [p0574] step: count(77), step_time 1408.0, mean_step_time 1499.03, it/s 0.67
  881. [0130 17:31:53 @multigpu.py:323] [p0576] step: count(79), step_time 1385.7, mean_step_time 1384.06, it/s 0.72
  882. [0130 17:31:53 @multigpu.py:323] [p0115] step: count(57), step_time 1368.43, mean_step_time 1450.04, it/s 0.69
  883. [0130 17:31:55 @multigpu.py:323] [p0115] step: count(58), step_time 1342.06, mean_step_time 1447.12, it/s 0.69
  884. [0130 17:31:55 @multigpu.py:323] [p0574] step: count(78), step_time 1385.83, mean_step_time 1497.89, it/s 0.67
  885. [0130 17:31:55 @multigpu.py:323] [p0576] step: count(80), step_time 1390.21, mean_step_time 1383.9, it/s 0.72
  886. [0130 17:31:56 @multigpu.py:323] [p0115] step: count(59), step_time 1394.26, mean_step_time 1447.23, it/s 0.69
  887. [0130 17:31:56 @multigpu.py:323] [p0576] step: count(81), step_time 1366.46, mean_step_time 1383.81, it/s 0.72
  888. [0130 17:31:56 @multigpu.py:323] [p0574] step: count(79), step_time 1432.96, mean_step_time 1498.45, it/s 0.67
  889. sending to address tcp://p0112:61216
  890. ##### Sending to neptune: online_score : 0.0322564772103 , 0.7 #####
  891. [u'online', 0.7]
  892. receiving
  893. [0130 17:31:57 @multigpu.py:323] [p0576] step: count(82), step_time 1393.91, mean_step_time 1383.98, it/s 0.72
  894. [0130 17:31:57 @multigpu.py:323] [p0115] step: count(60), step_time 1417.24, mean_step_time 1450.33, it/s 0.69
  895. [0130 17:31:57 @multigpu.py:323] [p0574] step: count(80), step_time 1353.39, mean_step_time 1495.11, it/s 0.67
  896. [0130 17:31:59 @multigpu.py:323] [p0576] step: count(83), step_time 1352.62, mean_step_time 1382.95, it/s 0.72
  897. [0130 17:31:59 @multigpu.py:323] [p0115] step: count(61), step_time 1369.22, mean_step_time 1450.16, it/s 0.69
  898. [0130 17:31:59 @multigpu.py:323] [p0574] step: count(81), step_time 1419.05, mean_step_time 1487.96, it/s 0.67
  899. [0130 17:32:00 @multigpu.py:323] [p0576] step: count(84), step_time 1376.94, mean_step_time 1384.15, it/s 0.72
  900. [0130 17:32:00 @multigpu.py:323] [p0115] step: count(62), step_time 1392.11, mean_step_time 1450.18, it/s 0.69
  901. [0130 17:32:00 @multigpu.py:323] [p0574] step: count(82), step_time 1567.51, mean_step_time 1493.67, it/s 0.67
  902. [0130 17:32:01 @multigpu.py:323] [p0576] step: count(85), step_time 1364.35, mean_step_time 1381.85, it/s 0.72
  903. [0130 17:32:02 @multigpu.py:323] [p0115] step: count(63), step_time 1402.84, mean_step_time 1452.73, it/s 0.69
  904. [0130 17:32:02 @multigpu.py:323] [p0574] step: count(83), step_time 1388.35, mean_step_time 1490.74, it/s 0.67
  905. [0130 17:32:03 @multigpu.py:323] [p0576] step: count(86), step_time 1388.53, mean_step_time 1384.74, it/s 0.72
  906. [0130 17:32:03 @multigpu.py:323] [p0115] step: count(64), step_time 1359.18, mean_step_time 1449.81, it/s 0.69
  907. [0130 17:32:03 @multigpu.py:323] [p0574] step: count(84), step_time 1357.99, mean_step_time 1480.27, it/s 0.68
  908. [0130 17:32:04 @multigpu.py:323] [p0576] step: count(87), step_time 1357.1, mean_step_time 1382.64, it/s 0.72
  909. [0130 17:32:04 @multigpu.py:323] [p0115] step: count(65), step_time 1433.11, mean_step_time 1450.18, it/s 0.69
  910. [0130 17:32:05 @multigpu.py:323] [p0574] step: count(85), step_time 1411.06, mean_step_time 1477.97, it/s 0.68
  911. [0130 17:32:06 @multigpu.py:323] [p0576] step: count(88), step_time 1413.38, mean_step_time 1385.63, it/s 0.72
  912. [0130 17:32:06 @multigpu.py:323] [p0115] step: count(66), step_time 1376.56, mean_step_time 1450.18, it/s 0.69
  913. [0130 17:32:06 @multigpu.py:323] [p0574] step: count(86), step_time 1371.07, mean_step_time 1470.98, it/s 0.68
  914. sending to address tcp://p0112:61216
  915. ##### Sending to neptune: online_score : 0.0349453416136 , 1.0 #####
  916. [u'online', 1.0]
  917. ##### Sending to neptune: active_workers : 0.034945417775 , 3 #####
  918. receiving
  919. [0130 17:32:07 @multigpu.py:323] [p0574] step: count(87), step_time 1381.74, mean_step_time 1397.6, it/s 0.72
  920. [0130 17:32:07 @multigpu.py:323] [p0576] step: count(89), step_time 1651.44, mean_step_time 1397.55, it/s 0.72
  921. [0130 17:32:07 @multigpu.py:323] [p0115] step: count(67), step_time 1568.35, mean_step_time 1397.54, it/s 0.72
  922. [0130 17:32:09 @multigpu.py:323] [p0574] step: count(88), step_time 1388.4, mean_step_time 1396.32, it/s 0.72
  923. [0130 17:32:09 @multigpu.py:323] [p0115] step: count(68), step_time 1399.14, mean_step_time 1395.21, it/s 0.72
  924. [0130 17:32:09 @multigpu.py:323] [p0576] step: count(90), step_time 1415.9, mean_step_time 1400.13, it/s 0.71
  925. [0130 17:32:10 @multigpu.py:323] [p0574] step: count(89), step_time 1346.75, mean_step_time 1395.62, it/s 0.72
  926. [0130 17:32:10 @multigpu.py:323] [p0576] step: count(91), step_time 1336.87, mean_step_time 1397.49, it/s 0.72
  927. [0130 17:32:10 @multigpu.py:323] [p0115] step: count(69), step_time 1472.22, mean_step_time 1398.91, it/s 0.71
  928. [0130 17:32:11 @multigpu.py:323] [p0574] step: count(90), step_time 1396.78, mean_step_time 1395.68, it/s 0.72
  929. [0130 17:32:12 @multigpu.py:323] [p0576] step: count(92), step_time 1474.83, mean_step_time 1404.37, it/s 0.71
  930. [0130 17:32:12 @multigpu.py:323] [p0115] step: count(70), step_time 1359.43, mean_step_time 1398.03, it/s 0.72
  931. [0130 17:32:13 @multigpu.py:323] [p0574] step: count(91), step_time 1389.08, mean_step_time 1395.19, it/s 0.72
  932. [0130 17:32:13 @multigpu.py:323] [p0576] step: count(93), step_time 1346.17, mean_step_time 1401.66, it/s 0.71
  933. [0130 17:32:13 @multigpu.py:323] [p0115] step: count(71), step_time 1420.7, mean_step_time 1401.24, it/s 0.71
  934. [0130 17:32:14 @multigpu.py:323] [p0574] step: count(92), step_time 1377.89, mean_step_time 1396.63, it/s 0.72
  935. [0130 17:32:14 @multigpu.py:323] [p0576] step: count(94), step_time 1409.36, mean_step_time 1403.83, it/s 0.71
  936. [0130 17:32:14 @multigpu.py:323] [p0115] step: count(72), step_time 1387.85, mean_step_time 1400.25, it/s 0.71
  937. sending to address tcp://p0112:61216
  938. ##### Sending to neptune: online_score : 0.0373082438442 , 1.3 #####
  939. [u'online', 1.3]
  940. receiving
  941. [0130 17:32:16 @multigpu.py:323] [p0574] step: count(93), step_time 1398.81, mean_step_time 1396.06, it/s 0.72
  942. [0130 17:32:16 @multigpu.py:323] [p0576] step: count(95), step_time 1378.33, mean_step_time 1398.64, it/s 0.71
  943. [0130 17:32:16 @multigpu.py:323] [p0115] step: count(73), step_time 1430.54, mean_step_time 1402.23, it/s 0.71
  944. [0130 17:32:17 @multigpu.py:323] [p0576] step: count(96), step_time 1360.46, mean_step_time 1396.57, it/s 0.72
  945. [0130 17:32:17 @multigpu.py:323] [p0574] step: count(94), step_time 1466.82, mean_step_time 1398.47, it/s 0.72
  946. [0130 17:32:17 @multigpu.py:323] [p0115] step: count(74), step_time 1409.39, mean_step_time 1404.72, it/s 0.71
  947. [0130 17:32:18 @multigpu.py:323] [p0576] step: count(97), step_time 1391.39, mean_step_time 1398.68, it/s 0.71
  948. [0130 17:32:19 @multigpu.py:323] [p0574] step: count(95), step_time 1440.57, mean_step_time 1402.19, it/s 0.71
  949. [0130 17:32:19 @multigpu.py:323] [p0115] step: count(75), step_time 1384.89, mean_step_time 1403.56, it/s 0.71
  950. [0130 17:32:20 @multigpu.py:323] [p0576] step: count(98), step_time 1356.32, mean_step_time 1395.51, it/s 0.72
  951. [0130 17:32:20 @multigpu.py:323] [p0115] step: count(76), step_time 1322.07, mean_step_time 1400.48, it/s 0.71
  952. [0130 17:32:20 @multigpu.py:323] [p0574] step: count(96), step_time 1402.93, mean_step_time 1404.25, it/s 0.71
  953. [0130 17:32:21 @multigpu.py:323] [p0576] step: count(99), step_time 1364.49, mean_step_time 1394.45, it/s 0.72
  954. [0130 17:32:21 @multigpu.py:323] [p0115] step: count(77), step_time 1393.59, mean_step_time 1401.74, it/s 0.71
  955. [0130 17:32:21 @multigpu.py:323] [p0574] step: count(97), step_time 1396.28, mean_step_time 1403.66, it/s 0.71
  956. [0130 17:32:23 @multigpu.py:323] [p0576] step: count(100), step_time 1392.71, mean_step_time 1394.58, it/s 0.72
  957. sending debugging info...
  958. sending to address tcp://p0112:61216
  959. ##### Sending to neptune: mean_delay : 0.0394246213304 , 0.0 #####
  960. sending to address tcp://p0112:61216
  961. ##### Sending to neptune: max_delay : 0.0394246213304 , -0.0 #####
  962. ##### Sending to neptune: min_delay : 0.0394246213304 , -0.0 #####
  963. [u'delays', [0.0, -0.0, -0.0]]
  964. receiving
  965. ##### Sending to neptune: cost : 0.0394251494275 , -0.0056879054755 #####
  966. sending to address tcp://p0112:61216
  967. ##### Sending to neptune: policy_loss : 0.0394251494275 , 0.399972140789 #####
  968. ##### Sending to neptune: xentropy_loss : 0.0394251494275 , -2.29181861877 #####
  969. ##### Sending to neptune: value_loss : 0.0394251494275 , 1.16379439831 #####
  970. ##### Sending to neptune: advantage : 0.0394251494275 , -0.00176788156386 #####
  971. ##### Sending to neptune: pred_reward : 0.0394251494275 , 0.261878997087 #####
  972. ##### Sending to neptune: max_logit : 0.0394251494275 , 0.181682661176 #####
  973. [u'loss', -0.005687905475497246, 0.399972140789032, -2.291818618774414, 1.1637943983078003, -0.0017678815638646483, 0.26187899708747864, 0.18168266117572784]
  974. receiving
  975. ##### Sending to neptune: active_relus : 0.0394261499908 , 9003946.08 #####
  976. ##### Sending to neptune: dp_per_s : 0.0394261499908 , 86.5037380059 #####
  977. [u'other', 9003946.08, 86.50373800587406]
  978. receiving
  979. [0130 17:32:23 @multigpu.py:323] [p0115] step: count(78), step_time 1362.33, mean_step_time 1402.75, it/s 0.71
  980. [0130 17:32:23 @multigpu.py:323] [p0574] step: count(98), step_time 1373.19, mean_step_time 1403.03, it/s 0.71
  981. [0130 17:32:24 @multigpu.py:323] [p0576] step: count(101), step_time 1420.15, mean_step_time 1397.26, it/s 0.72
  982. [0130 17:32:24 @multigpu.py:323] [p0115] step: count(79), step_time 1413.0, mean_step_time 1403.69, it/s 0.71
  983. [0130 17:32:24 @multigpu.py:323] [p0574] step: count(99), step_time 1382.8, mean_step_time 1400.52, it/s 0.71
  984. sending to address tcp://p0112:61216
  985. ##### Sending to neptune: online_score : 0.040045074953 , 0.8 #####
  986. [u'online', 0.8]
  987. receiving
  988. [0130 17:32:25 @multigpu.py:323] [p0576] step: count(102), step_time 1406.42, mean_step_time 1397.89, it/s 0.72
  989. [0130 17:32:25 @multigpu.py:323] [p0115] step: count(80), step_time 1359.02, mean_step_time 1400.78, it/s 0.71
  990. [0130 17:32:25 @multigpu.py:323] [p0574] step: count(100), step_time 1389.18, mean_step_time 1402.31, it/s 0.71
  991. sending debugging info...
  992. sending to address tcp://p0112:61216
  993. ##### Sending to neptune: mean_delay : 0.0402365121577 , 0.0 #####
  994. ##### Sending to neptune: max_delay : 0.0402365121577 , -0.0 #####
  995. sending to address tcp://p0112:61216
  996. ##### Sending to neptune: min_delay : 0.0402365121577 , -0.0 #####
  997. [u'delays', [0.0, -0.0, -0.0]]
  998. receiving
  999. ##### Sending to neptune: cost : 0.0402370121744 , 0.00216898717918 #####
  1000. ##### Sending to neptune: policy_loss : 0.0402370121744 , 1.23316562176 #####
  1001. sending to address tcp://p0112:61216
  1002. ##### Sending to neptune: xentropy_loss : 0.0402370121744 , -2.29180669785 #####
  1003. ##### Sending to neptune: value_loss : 0.0402370121744 , 1.3362711668 #####
  1004. ##### Sending to neptune: advantage : 0.0402370121744 , -0.00537997484207 #####
  1005. ##### Sending to neptune: pred_reward : 0.0402370121744 , 0.260432839394 #####
  1006. ##### Sending to neptune: max_logit : 0.0402370121744 , 0.181708931923 #####
  1007. [u'loss', 0.0021689871791750193, 1.2331656217575073, -2.291806697845459, 1.3362711668014526, -0.005379974842071533, 0.2604328393936157, 0.1817089319229126]
  1008. receiving
  1009. ##### Sending to neptune: active_relus : 0.0402375141117 , 8991974.32 #####
  1010. ##### Sending to neptune: dp_per_s : 0.0402375141117 , 84.6942909111 #####
  1011. [u'other', 8991974.32, 84.69429091109998]
  1012. receiving
  1013. [0130 17:32:27 @multigpu.py:323] [p0576] step: count(103), step_time 1386.27, mean_step_time 1399.57, it/s 0.71
  1014. [0130 17:32:27 @multigpu.py:323] [p0115] step: count(81), step_time 1420.97, mean_step_time 1403.37, it/s 0.71
  1015. [0130 17:32:27 @multigpu.py:323] [p0574] step: count(101), step_time 1417.69, mean_step_time 1402.24, it/s 0.71
  1016. [0130 17:32:28 @multigpu.py:323] [p0576] step: count(104), step_time 1378.61, mean_step_time 1399.65, it/s 0.71
  1017. [0130 17:32:28 @multigpu.py:323] [p0574] step: count(102), step_time 1342.28, mean_step_time 1390.98, it/s 0.72
  1018. [0130 17:32:28 @multigpu.py:323] [p0115] step: count(82), step_time 1416.15, mean_step_time 1404.57, it/s 0.71
  1019. [0130 17:32:30 @multigpu.py:323] [p0576] step: count(105), step_time 1376.33, mean_step_time 1400.25, it/s 0.71
  1020. [0130 17:32:30 @multigpu.py:323] [p0574] step: count(103), step_time 1393.23, mean_step_time 1391.23, it/s 0.72
  1021. [0130 17:32:30 @multigpu.py:323] [p0115] step: count(83), step_time 1367.98, mean_step_time 1402.82, it/s 0.71
  1022. [0130 17:32:31 @multigpu.py:323] [p0576] step: count(106), step_time 1405.28, mean_step_time 1401.09, it/s 0.71
  1023. [0130 17:32:31 @multigpu.py:323] [p0574] step: count(104), step_time 1371.23, mean_step_time 1391.89, it/s 0.72
  1024. [0130 17:32:31 @multigpu.py:323] [p0115] step: count(84), step_time 1419.1, mean_step_time 1405.82, it/s 0.71
  1025. [0130 17:32:32 @multigpu.py:323] [p0576] step: count(107), step_time 1391.55, mean_step_time 1402.81, it/s 0.71
  1026. [0130 17:32:32 @multigpu.py:323] [p0574] step: count(105), step_time 1418.55, mean_step_time 1392.26, it/s 0.72
  1027. [0130 17:32:32 @multigpu.py:323] [p0115] step: count(85), step_time 1394.13, mean_step_time 1403.87, it/s 0.71
  1028. [0130 17:32:34 @multigpu.py:323] [p0576] step: count(108), step_time 1447.96, mean_step_time 1404.54, it/s 0.71
  1029. [0130 17:32:34 @multigpu.py:323] [p0574] step: count(106), step_time 1361.46, mean_step_time 1391.78, it/s 0.72
  1030. [0130 17:32:34 @multigpu.py:323] [p0115] step: count(86), step_time 1395.83, mean_step_time 1404.84, it/s 0.71
  1031. sending to address tcp://p0112:61216
  1032. ##### Sending to neptune: online_score : 0.0427628719144 , 2.1 #####
  1033. [u'online', 2.1]
  1034. receiving
  1035. [0130 17:32:35 @multigpu.py:323] [p0574] step: count(107), step_time 1429.53, mean_step_time 1394.17, it/s 0.72
  1036. [0130 17:32:35 @multigpu.py:323] [p0576] step: count(109), step_time 1442.03, mean_step_time 1394.07, it/s 0.72
  1037. [0130 17:32:35 @multigpu.py:323] [p0115] step: count(87), step_time 1353.86, mean_step_time 1394.11, it/s 0.72
  1038. [0130 17:32:37 @multigpu.py:323] [p0576] step: count(110), step_time 1348.97, mean_step_time 1390.72, it/s 0.72
  1039. [0130 17:32:37 @multigpu.py:323] [p0574] step: count(108), step_time 1386.77, mean_step_time 1394.09, it/s 0.72
  1040. [0130 17:32:37 @multigpu.py:323] [p0115] step: count(88), step_time 1412.35, mean_step_time 1394.77, it/s 0.72
  1041. [0130 17:32:38 @multigpu.py:323] [p0576] step: count(111), step_time 1377.92, mean_step_time 1392.78, it/s 0.72
  1042. [0130 17:32:38 @multigpu.py:323] [p0115] step: count(89), step_time 1370.59, mean_step_time 1389.69, it/s 0.72
  1043. [0130 17:32:38 @multigpu.py:323] [p0574] step: count(109), step_time 1411.26, mean_step_time 1397.32, it/s 0.72
  1044. [0130 17:32:39 @multigpu.py:323] [p0576] step: count(112), step_time 1386.66, mean_step_time 1388.37, it/s 0.72
  1045. [0130 17:32:39 @multigpu.py:323] [p0115] step: count(90), step_time 1351.87, mean_step_time 1389.31, it/s 0.72
  1046. [0130 17:32:39 @multigpu.py:323] [p0574] step: count(110), step_time 1452.75, mean_step_time 1400.11, it/s 0.71
  1047. [0130 17:32:41 @multigpu.py:323] [p0115] step: count(91), step_time 1359.65, mean_step_time 1386.26, it/s 0.72
  1048. [0130 17:32:41 @multigpu.py:323] [p0576] step: count(113), step_time 1427.16, mean_step_time 1392.42, it/s 0.72
  1049. [0130 17:32:41 @multigpu.py:323] [p0574] step: count(111), step_time 1353.86, mean_step_time 1398.35, it/s 0.72
  1050. sending to address tcp://p0112:61216
  1051. ##### Sending to neptune: online_score : 0.0445858110984 , 1.5 #####
  1052. [u'online', 1.5]
  1053. receiving
  1054. [0130 17:32:42 @multigpu.py:323] [p0576] step: count(114), step_time 1351.34, mean_step_time 1389.52, it/s 0.72
  1055. [0130 17:32:42 @multigpu.py:323] [p0115] step: count(92), step_time 1456.38, mean_step_time 1389.68, it/s 0.72
  1056. [0130 17:32:42 @multigpu.py:323] [p0574] step: count(112), step_time 1408.49, mean_step_time 1399.88, it/s 0.71
  1057. [0130 17:32:44 @multigpu.py:323] [p0115] step: count(93), step_time 1363.97, mean_step_time 1386.36, it/s 0.72
  1058. [0130 17:32:44 @multigpu.py:323] [p0576] step: count(115), step_time 1460.08, mean_step_time 1393.61, it/s 0.72
  1059. [0130 17:32:44 @multigpu.py:323] [p0574] step: count(113), step_time 1432.72, mean_step_time 1401.58, it/s 0.71
  1060. [0130 17:32:45 @multigpu.py:323] [p0115] step: count(94), step_time 1370.03, mean_step_time 1384.39, it/s 0.72
  1061. [0130 17:32:45 @multigpu.py:323] [p0576] step: count(116), step_time 1421.28, mean_step_time 1396.65, it/s 0.72
  1062. [0130 17:32:45 @multigpu.py:323] [p0574] step: count(114), step_time 1433.08, mean_step_time 1399.89, it/s 0.71
  1063. [0130 17:32:46 @multigpu.py:323] [p0115] step: count(95), step_time 1378.92, mean_step_time 1384.09, it/s 0.72
  1064. [0130 17:32:46 @multigpu.py:323] [p0576] step: count(117), step_time 1382.6, mean_step_time 1396.21, it/s 0.72
  1065. [0130 17:32:47 @multigpu.py:323] [p0574] step: count(115), step_time 1430.45, mean_step_time 1399.39, it/s 0.71
  1066. [0130 17:32:48 @multigpu.py:323] [p0115] step: count(96), step_time 1401.86, mean_step_time 1388.08, it/s 0.72
  1067. [0130 17:32:48 @multigpu.py:323] [p0576] step: count(118), step_time 1369.23, mean_step_time 1396.85, it/s 0.72
  1068. [0130 17:32:48 @multigpu.py:323] [p0574] step: count(116), step_time 1421.34, mean_step_time 1400.31, it/s 0.71
  1069. [0130 17:32:49 @multigpu.py:323] [p0115] step: count(97), step_time 1382.13, mean_step_time 1387.51, it/s 0.72
  1070. [0130 17:32:49 @multigpu.py:323] [p0576] step: count(119), step_time 1389.47, mean_step_time 1398.1, it/s 0.72
  1071. [0130 17:32:49 @multigpu.py:323] [p0574] step: count(117), step_time 1401.27, mean_step_time 1400.56, it/s 0.71
  1072. [0130 17:32:50 @multigpu.py:323] [p0115] step: count(98), step_time 1443.72, mean_step_time 1391.58, it/s 0.72
  1073. [0130 17:32:51 @multigpu.py:323] [p0576] step: count(120), step_time 1429.82, mean_step_time 1399.96, it/s 0.71
  1074. [0130 17:32:51 @multigpu.py:323] [p0574] step: count(118), step_time 1454.38, mean_step_time 1404.62, it/s 0.71
  1075. [0130 17:32:52 @multigpu.py:323] [p0115] step: count(99), step_time 1387.8, mean_step_time 1390.31, it/s 0.72
  1076. [0130 17:32:52 @multigpu.py:323] [p0576] step: count(121), step_time 1392.97, mean_step_time 1398.6, it/s 0.72
  1077. [0130 17:32:52 @multigpu.py:323] [p0574] step: count(119), step_time 1527.24, mean_step_time 1411.84, it/s 0.71
  1078. [0130 17:32:53 @multigpu.py:323] [p0115] step: count(100), step_time 1381.26, mean_step_time 1391.43, it/s 0.72
  1079. sending debugging info...
  1080. sending to address tcp://p0112:61216
  1081. ##### Sending to neptune: mean_delay : 0.047960005535 , 0.0 #####
  1082. ##### Sending to neptune: max_delay : 0.047960005535 , -0.0 #####
  1083. sending to address tcp://p0112:61216
  1084. ##### Sending to neptune: min_delay : 0.047960005535 , -0.0 #####
  1085. [u'delays', [0.0, -0.0, -0.0]]
  1086. receiving
  1087. ##### Sending to neptune: cost : 0.0479606341653 , -0.0105593772605 #####
  1088. sending to address tcp://p0112:61216
  1089. ##### Sending to neptune: policy_loss : 0.0479606341653 , -0.167208120227 #####
  1090. ##### Sending to neptune: xentropy_loss : 0.0479606341653 , -2.29150485992 #####
  1091. ##### Sending to neptune: value_loss : 0.0479606341653 , 1.10711240768 #####
  1092. ##### Sending to neptune: advantage : 0.0479606341653 , 0.000706825871021 #####
  1093. ##### Sending to neptune: pred_reward : 0.0479606341653 , 0.30842140317 #####
  1094. ##### Sending to neptune: max_logit : 0.0479606341653 , 0.18301834166 #####
  1095. [u'loss', -0.010559377260506153, -0.16720812022686005, -2.2915048599243164, 1.1071124076843262, 0.0007068258710205555, 0.30842140316963196, 0.18301834166049957]
  1096. receiving
  1097. ##### Sending to neptune: active_relus : 0.0479611127244 , 8962460.69 #####
  1098. ##### Sending to neptune: dp_per_s : 0.0479611127244 , 76.3888871862 #####
  1099. [u'other', 8962460.69, 76.38888718623758]
  1100. receiving
  1101. [0130 17:32:53 @multigpu.py:323] [p0576] step: count(122), step_time 1392.28, mean_step_time 1397.89, it/s 0.72
  1102. [0130 17:32:54 @multigpu.py:323] [p0574] step: count(120), step_time 1530.71, mean_step_time 1418.91, it/s 0.7
  1103. sending to address tcp://p0112:61216
  1104. ##### Sending to neptune: online_score : 0.0481446746985 , 2.2 #####
  1105. [u'online', 2.2]
  1106. receiving
  1107. [0130 17:32:55 @multigpu.py:323] [p0115] step: count(101), step_time 1410.22, mean_step_time 1390.89, it/s 0.72
  1108. [0130 17:32:55 @multigpu.py:323] [p0576] step: count(123), step_time 1348.5, mean_step_time 1396.0, it/s 0.72
  1109. [0130 17:32:55 @multigpu.py:323] [p0574] step: count(121), step_time 1452.44, mean_step_time 1420.65, it/s 0.7
  1110. [0130 17:32:56 @multigpu.py:323] [p0115] step: count(102), step_time 1357.53, mean_step_time 1387.96, it/s 0.72
  1111. [0130 17:32:56 @multigpu.py:323] [p0576] step: count(124), step_time 1363.25, mean_step_time 1395.23, it/s 0.72
  1112. [0130 17:32:57 @multigpu.py:323] [p0574] step: count(122), step_time 1581.99, mean_step_time 1432.64, it/s 0.7
  1113. [0130 17:32:57 @multigpu.py:323] [p0115] step: count(103), step_time 1383.49, mean_step_time 1388.73, it/s 0.72
  1114. [0130 17:32:57 @multigpu.py:323] [p0576] step: count(125), step_time 1443.28, mean_step_time 1398.58, it/s 0.72
  1115. [0130 17:32:58 @multigpu.py:323] [p0574] step: count(123), step_time 1531.42, mean_step_time 1439.55, it/s 0.69
  1116. [0130 17:32:59 @multigpu.py:323] [p0115] step: count(104), step_time 1381.62, mean_step_time 1386.86, it/s 0.72
  1117. [0130 17:32:59 @multigpu.py:323] [p0576] step: count(126), step_time 1381.94, mean_step_time 1397.42, it/s 0.72
  1118. [0130 17:33:00 @multigpu.py:323] [p0574] step: count(124), step_time 1630.48, mean_step_time 1452.51, it/s 0.69
  1119. [0130 17:33:00 @multigpu.py:323] [p0115] step: count(105), step_time 1369.54, mean_step_time 1385.63, it/s 0.72
  1120. [0130 17:33:00 @multigpu.py:323] [p0576] step: count(127), step_time 1395.76, mean_step_time 1397.63, it/s 0.72
  1121. [0130 17:33:02 @multigpu.py:323] [p0115] step: count(106), step_time 1406.69, mean_step_time 1386.17, it/s 0.72
  1122. [0130 17:33:02 @multigpu.py:323] [p0574] step: count(125), step_time 1546.76, mean_step_time 1458.92, it/s 0.69
  1123. [0130 17:33:02 @multigpu.py:323] [p0576] step: count(128), step_time 1399.48, mean_step_time 1395.2, it/s 0.72
  1124. [0130 17:33:03 @multigpu.py:323] [p0576] step: count(129), step_time 1363.1, mean_step_time 1391.25, it/s 0.72
  1125. [0130 17:33:04 @multigpu.py:323] [p0576] step: count(130), step_time 1338.32, mean_step_time 1390.72, it/s 0.72
  1126. [0130 17:33:04 @multigpu.py:323] [p0115] step: count(107), step_time 2796.9, mean_step_time 1458.32, it/s 0.69
  1127. [0130 17:33:04 @multigpu.py:323] [p0574] step: count(126), step_time 2779.41, mean_step_time 1529.82, it/s 0.65
  1128. sending to address tcp://p0112:61216
  1129. ##### Sending to neptune: online_score : 0.0510955774784 , 1.3 #####
  1130. [u'online', 1.3]
  1131. receiving
  1132. [0130 17:33:06 @multigpu.py:323] [p0115] step: count(108), step_time 1399.23, mean_step_time 1457.67, it/s 0.69
  1133. [0130 17:33:06 @multigpu.py:323] [p0576] step: count(131), step_time 1472.37, mean_step_time 1395.44, it/s 0.72
  1134. [0130 17:33:06 @multigpu.py:323] [p0574] step: count(127), step_time 1573.33, mean_step_time 1537.01, it/s 0.65
  1135. [0130 17:33:07 @multigpu.py:323] [p0115] step: count(109), step_time 1380.55, mean_step_time 1458.17, it/s 0.69
  1136. [0130 17:33:07 @multigpu.py:323] [p0576] step: count(132), step_time 1427.56, mean_step_time 1397.49, it/s 0.72
  1137. [0130 17:33:08 @multigpu.py:323] [p0574] step: count(128), step_time 1619.61, mean_step_time 1548.65, it/s 0.65
  1138. [0130 17:33:09 @multigpu.py:323] [p0115] step: count(110), step_time 1370.05, mean_step_time 1459.08, it/s 0.69
  1139. [0130 17:33:09 @multigpu.py:323] [p0576] step: count(133), step_time 1399.82, mean_step_time 1396.12, it/s 0.72
  1140. sending to address tcp://p0112:61216
  1141. ##### Sending to neptune: online_score : 0.0522849238581 , 1.6 #####
  1142. [u'online', 1.6]
  1143. ##### Sending to neptune: active_workers : 0.0522849566407 , 3 #####
  1144. receiving
  1145. [0130 17:33:09 @multigpu.py:323] [p0574] step: count(129), step_time 1624.04, mean_step_time 1559.29, it/s 0.64
  1146. [0130 17:33:10 @multigpu.py:323] [p0115] step: count(111), step_time 1376.21, mean_step_time 1459.9, it/s 0.68
  1147. [0130 17:33:10 @multigpu.py:323] [p0576] step: count(134), step_time 1361.15, mean_step_time 1396.61, it/s 0.72
  1148. [0130 17:33:11 @multigpu.py:323] [p0574] step: count(130), step_time 1590.44, mean_step_time 1566.17, it/s 0.64
  1149. [0130 17:33:11 @multigpu.py:323] [p0115] step: count(112), step_time 1365.93, mean_step_time 1455.38, it/s 0.69
  1150. [0130 17:33:11 @multigpu.py:323] [p0576] step: count(135), step_time 1388.74, mean_step_time 1393.05, it/s 0.72
  1151. [0130 17:33:13 @multigpu.py:323] [p0574] step: count(131), step_time 1731.61, mean_step_time 1585.06, it/s 0.63
  1152. [0130 17:33:13 @multigpu.py:323] [p0115] step: count(113), step_time 1383.15, mean_step_time 1456.34, it/s 0.69
  1153. [0130 17:33:13 @multigpu.py:323] [p0576] step: count(136), step_time 1343.67, mean_step_time 1389.17, it/s 0.72
  1154. [0130 17:33:14 @multigpu.py:323] [p0115] step: count(114), step_time 1378.8, mean_step_time 1456.78, it/s 0.69
  1155. [0130 17:33:14 @multigpu.py:323] [p0574] step: count(132), step_time 1601.35, mean_step_time 1594.7, it/s 0.63
  1156. [0130 17:33:14 @multigpu.py:323] [p0576] step: count(137), step_time 1405.46, mean_step_time 1390.31, it/s 0.72
  1157. [0130 17:33:15 @multigpu.py:323] [p0115] step: count(115), step_time 1399.48, mean_step_time 1457.81, it/s 0.69
  1158. [0130 17:33:16 @multigpu.py:323] [p0576] step: count(138), step_time 1388.43, mean_step_time 1391.27, it/s 0.72
  1159. [0130 17:33:16 @multigpu.py:323] [p0574] step: count(133), step_time 1626.81, mean_step_time 1604.41, it/s 0.62
  1160. [0130 17:33:17 @multigpu.py:323] [p0115] step: count(116), step_time 1407.25, mean_step_time 1458.08, it/s 0.69
  1161. [0130 17:33:17 @multigpu.py:323] [p0576] step: count(139), step_time 1346.21, mean_step_time 1389.11, it/s 0.72
  1162. [0130 17:33:17 @multigpu.py:323] [p0574] step: count(134), step_time 1527.52, mean_step_time 1609.13, it/s 0.62
  1163. [0130 17:33:18 @multigpu.py:323] [p0115] step: count(117), step_time 1409.14, mean_step_time 1459.43, it/s 0.69
  1164. [0130 17:33:18 @multigpu.py:323] [p0576] step: count(140), step_time 1372.22, mean_step_time 1386.23, it/s 0.72
  1165. [0130 17:33:19 @multigpu.py:323] [p0574] step: count(135), step_time 1426.88, mean_step_time 1608.95, it/s 0.62
  1166. [0130 17:33:20 @multigpu.py:323] [p0115] step: count(118), step_time 1375.3, mean_step_time 1456.01, it/s 0.69
  1167. [0130 17:33:20 @multigpu.py:323] [p0576] step: count(141), step_time 1366.67, mean_step_time 1384.91, it/s 0.72
  1168. [0130 17:33:20 @multigpu.py:323] [p0574] step: count(136), step_time 1466.03, mean_step_time 1611.19, it/s 0.62
  1169. [0130 17:33:21 @multigpu.py:323] [p0576] step: count(142), step_time 1389.33, mean_step_time 1384.76, it/s 0.72
  1170. [0130 17:33:21 @multigpu.py:323] [p0115] step: count(119), step_time 1424.17, mean_step_time 1457.83, it/s 0.69
  1171. [0130 17:33:22 @multigpu.py:323] [p0574] step: count(137), step_time 1382.97, mean_step_time 1610.27, it/s 0.62
  1172. [0130 17:33:22 @multigpu.py:323] [p0576] step: count(143), step_time 1315.06, mean_step_time 1383.09, it/s 0.72
  1173. [0130 17:33:22 @multigpu.py:323] [p0115] step: count(120), step_time 1378.73, mean_step_time 1457.7, it/s 0.69
  1174. [0130 17:33:23 @multigpu.py:323] [p0574] step: count(138), step_time 1377.13, mean_step_time 1606.41, it/s 0.62
  1175. [0130 17:33:24 @multigpu.py:323] [p0576] step: count(144), step_time 1410.72, mean_step_time 1385.46, it/s 0.72
  1176. [0130 17:33:24 @multigpu.py:323] [p0115] step: count(121), step_time 1428.49, mean_step_time 1458.61, it/s 0.69
  1177. [0130 17:33:24 @multigpu.py:323] [p0574] step: count(139), step_time 1379.38, mean_step_time 1599.02, it/s 0.63
  1178. sending to address tcp://p0112:61216
  1179. ##### Sending to neptune: online_score : 0.0566255952252 , 1.6 #####
  1180. [u'online', 1.6]
  1181. receiving
  1182. sending to address tcp://p0112:61216
  1183. ##### Sending to neptune: online_score : 0.0567920777533 , 1.3 #####
  1184. [u'online', 1.3]
  1185. receiving
  1186. [0130 17:33:25 @multigpu.py:323] [p0576] step: count(145), step_time 1331.44, mean_step_time 1379.87, it/s 0.72
  1187. [0130 17:33:25 @multigpu.py:323] [p0115] step: count(122), step_time 1338.18, mean_step_time 1457.65, it/s 0.69
  1188. [0130 17:33:26 @multigpu.py:323] [p0574] step: count(140), step_time 1408.26, mean_step_time 1592.89, it/s 0.63
  1189. [0130 17:33:27 @multigpu.py:323] [p0576] step: count(146), step_time 1414.04, mean_step_time 1381.48, it/s 0.72
  1190. [0130 17:33:27 @multigpu.py:323] [p0115] step: count(123), step_time 1422.33, mean_step_time 1459.59, it/s 0.69
  1191. [0130 17:33:27 @multigpu.py:323] [p0574] step: count(141), step_time 1368.26, mean_step_time 1588.68, it/s 0.63
  1192. [0130 17:33:28 @multigpu.py:323] [p0576] step: count(147), step_time 1339.09, mean_step_time 1378.64, it/s 0.73
  1193. [0130 17:33:28 @multigpu.py:323] [p0115] step: count(124), step_time 1376.01, mean_step_time 1459.31, it/s 0.69
  1194. [0130 17:33:29 @multigpu.py:323] [p0574] step: count(142), step_time 1417.7, mean_step_time 1580.47, it/s 0.63
  1195. [0130 17:33:29 @multigpu.py:323] [p0576] step: count(148), step_time 1386.21, mean_step_time 1377.98, it/s 0.73
  1196. [0130 17:33:29 @multigpu.py:323] [p0115] step: count(125), step_time 1357.21, mean_step_time 1458.69, it/s 0.69
  1197. [0130 17:33:30 @multigpu.py:323] [p0574] step: count(143), step_time 1351.42, mean_step_time 1571.47, it/s 0.64
  1198. [0130 17:33:31 @multigpu.py:323] [p0576] step: count(149), step_time 1354.23, mean_step_time 1377.54, it/s 0.73
  1199. [0130 17:33:31 @multigpu.py:323] [p0115] step: count(126), step_time 1383.09, mean_step_time 1457.51, it/s 0.69
  1200. [0130 17:33:31 @multigpu.py:323] [p0574] step: count(144), step_time 1374.31, mean_step_time 1558.66, it/s 0.64
  1201. [0130 17:33:32 @multigpu.py:323] [p0576] step: count(150), step_time 1377.74, mean_step_time 1379.51, it/s 0.72
  1202. [0130 17:33:33 @multigpu.py:323] [p0576] step: count(151), step_time 1383.28, mean_step_time 1375.05, it/s 0.73
  1203. [0130 17:33:33 @multigpu.py:323] [p0574] step: count(145), step_time 2126.63, mean_step_time 1587.65, it/s 0.63
  1204. [0130 17:33:33 @multigpu.py:323] [p0115] step: count(127), step_time 2620.68, mean_step_time 1448.7, it/s 0.69
  1205. sending to address tcp://p0112:61216
  1206. ##### Sending to neptune: online_score : 0.0592391738627 , 1.6 #####
  1207. [u'online', 1.6]
  1208. receiving
  1209. [0130 17:33:35 @multigpu.py:323] [p0574] step: count(146), step_time 1375.08, mean_step_time 1517.44, it/s 0.66
  1210. [0130 17:33:35 @multigpu.py:323] [p0576] step: count(152), step_time 1375.58, mean_step_time 1372.45, it/s 0.73
  1211. [0130 17:33:35 @multigpu.py:323] [p0115] step: count(128), step_time 1440.45, mean_step_time 1450.76, it/s 0.69
  1212. [0130 17:33:36 @multigpu.py:323] [p0576] step: count(153), step_time 1407.13, mean_step_time 1372.82, it/s 0.73
  1213. [0130 17:33:36 @multigpu.py:323] [p0115] step: count(129), step_time 1366.56, mean_step_time 1450.06, it/s 0.69
  1214. [0130 17:33:36 @multigpu.py:323] [p0574] step: count(147), step_time 1436.0, mean_step_time 1510.57, it/s 0.66
  1215. [0130 17:33:37 @multigpu.py:323] [p0576] step: count(154), step_time 1322.97, mean_step_time 1370.91, it/s 0.73
  1216. [0130 17:33:38 @multigpu.py:323] [p0574] step: count(148), step_time 1352.32, mean_step_time 1497.21, it/s 0.67
  1217. [0130 17:33:38 @multigpu.py:323] [p0115] step: count(130), step_time 1406.66, mean_step_time 1451.89, it/s 0.69
  1218. [0130 17:33:39 @multigpu.py:323] [p0576] step: count(155), step_time 1419.88, mean_step_time 1372.47, it/s 0.73
  1219. [0130 17:33:39 @multigpu.py:323] [p0574] step: count(149), step_time 1408.69, mean_step_time 1486.44, it/s 0.67
  1220. [0130 17:33:39 @multigpu.py:323] [p0115] step: count(131), step_time 1452.33, mean_step_time 1455.7, it/s 0.69
  1221. [0130 17:33:40 @multigpu.py:323] [p0576] step: count(156), step_time 1347.77, mean_step_time 1372.67, it/s 0.73
  1222. [0130 17:33:40 @multigpu.py:323] [p0574] step: count(150), step_time 1384.36, mean_step_time 1476.13, it/s 0.68
  1223. [0130 17:33:40 @multigpu.py:323] [p0115] step: count(132), step_time 1370.15, mean_step_time 1455.91, it/s 0.69
  1224. [0130 17:33:42 @multigpu.py:323] [p0576] step: count(157), step_time 1357.76, mean_step_time 1370.29, it/s 0.73
  1225. [0130 17:33:42 @multigpu.py:323] [p0574] step: count(151), step_time 1339.38, mean_step_time 1456.52, it/s 0.69
  1226. [0130 17:33:42 @multigpu.py:323] [p0115] step: count(133), step_time 1349.21, mean_step_time 1454.21, it/s 0.69
  1227. [0130 17:33:43 @multigpu.py:323] [p0576] step: count(158), step_time 1393.96, mean_step_time 1370.56, it/s 0.73
  1228. [0130 17:33:43 @multigpu.py:323] [p0574] step: count(152), step_time 1389.87, mean_step_time 1445.95, it/s 0.69
  1229. [0130 17:33:43 @multigpu.py:323] [p0115] step: count(134), step_time 1368.66, mean_step_time 1453.7, it/s 0.69
  1230. sending to address tcp://p0112:61216
  1231. ##### Sending to neptune: online_score : 0.0620354730553 , 1.4 #####
  1232. [u'online', 1.4]
  1233. receiving
  1234. [0130 17:33:44 @multigpu.py:323] [p0576] step: count(159), step_time 1365.54, mean_step_time 1371.53, it/s 0.73
  1235. [0130 17:33:44 @multigpu.py:323] [p0574] step: count(153), step_time 1363.79, mean_step_time 1432.8, it/s 0.7
  1236. [0130 17:33:44 @multigpu.py:323] [p0115] step: count(135), step_time 1364.54, mean_step_time 1451.96, it/s 0.69
  1237. [0130 17:33:46 @multigpu.py:323] [p0576] step: count(160), step_time 1351.72, mean_step_time 1370.5, it/s 0.73
  1238. [0130 17:33:46 @multigpu.py:323] [p0574] step: count(154), step_time 1415.48, mean_step_time 1427.2, it/s 0.7
  1239. [0130 17:33:46 @multigpu.py:323] [p0115] step: count(136), step_time 1433.35, mean_step_time 1453.26, it/s 0.69
  1240. [0130 17:33:47 @multigpu.py:323] [p0576] step: count(161), step_time 1397.89, mean_step_time 1372.07, it/s 0.73
  1241. [0130 17:33:47 @multigpu.py:323] [p0574] step: count(155), step_time 1358.27, mean_step_time 1423.77, it/s 0.7
  1242. [0130 17:33:47 @multigpu.py:323] [p0115] step: count(137), step_time 1386.5, mean_step_time 1452.13, it/s 0.69
  1243. [0130 17:33:48 @multigpu.py:323] [p0576] step: count(162), step_time 1338.21, mean_step_time 1369.51, it/s 0.73
  1244. [0130 17:33:49 @multigpu.py:323] [p0574] step: count(156), step_time 1386.11, mean_step_time 1419.77, it/s 0.7
  1245. [0130 17:33:49 @multigpu.py:323] [p0115] step: count(138), step_time 1378.19, mean_step_time 1452.27, it/s 0.69
  1246. [0130 17:33:50 @multigpu.py:323] [p0576] step: count(163), step_time 1400.41, mean_step_time 1373.78, it/s 0.73
  1247. [0130 17:33:50 @multigpu.py:323] [p0574] step: count(157), step_time 1369.1, mean_step_time 1419.08, it/s 0.7
  1248. [0130 17:33:50 @multigpu.py:323] [p0115] step: count(139), step_time 1378.65, mean_step_time 1450.0, it/s 0.69
  1249. [0130 17:33:51 @multigpu.py:323] [p0576] step: count(164), step_time 1331.14, mean_step_time 1369.8, it/s 0.73
  1250. [0130 17:33:51 @multigpu.py:323] [p0574] step: count(158), step_time 1411.4, mean_step_time 1420.79, it/s 0.7
  1251. [0130 17:33:51 @multigpu.py:323] [p0115] step: count(140), step_time 1361.03, mean_step_time 1449.11, it/s 0.69
  1252. [0130 17:33:53 @multigpu.py:323] [p0576] step: count(165), step_time 1387.46, mean_step_time 1372.6, it/s 0.73
  1253. [0130 17:33:53 @multigpu.py:323] [p0115] step: count(141), step_time 1417.65, mean_step_time 1448.57, it/s 0.69
  1254. [0130 17:33:53 @multigpu.py:323] [p0574] step: count(159), step_time 1508.85, mean_step_time 1427.26, it/s 0.7
  1255. [0130 17:33:54 @multigpu.py:323] [p0576] step: count(166), step_time 1382.32, mean_step_time 1371.01, it/s 0.73
  1256. [0130 17:33:54 @multigpu.py:323] [p0115] step: count(142), step_time 1374.83, mean_step_time 1450.4, it/s 0.69
  1257. [0130 17:33:54 @multigpu.py:323] [p0574] step: count(160), step_time 1371.93, mean_step_time 1425.45, it/s 0.7
  1258. [0130 17:33:55 @multigpu.py:323] [p0576] step: count(167), step_time 1386.89, mean_step_time 1373.4, it/s 0.73
  1259. [0130 17:33:56 @multigpu.py:323] [p0115] step: count(143), step_time 1362.51, mean_step_time 1447.41, it/s 0.69
  1260. [0130 17:33:56 @multigpu.py:323] [p0574] step: count(161), step_time 1391.61, mean_step_time 1426.61, it/s 0.7
  1261. [0130 17:33:57 @multigpu.py:323] [p0576] step: count(168), step_time 1337.04, mean_step_time 1370.95, it/s 0.73
  1262. [0130 17:33:57 @multigpu.py:323] [p0574] step: count(162), step_time 1362.97, mean_step_time 1423.88, it/s 0.7
  1263. [0130 17:33:57 @multigpu.py:323] [p0115] step: count(144), step_time 1451.4, mean_step_time 1451.18, it/s 0.69
  1264. sending to address tcp://p0112:61216
  1265. ##### Sending to neptune: online_score : 0.0657542652554 , 0.9 #####
  1266. [u'online', 0.9]
  1267. receiving
  1268. [0130 17:33:58 @multigpu.py:323] [p0576] step: count(169), step_time 1383.69, mean_step_time 1372.42, it/s 0.73
  1269. [0130 17:33:58 @multigpu.py:323] [p0574] step: count(163), step_time 1399.8, mean_step_time 1426.3, it/s 0.7
  1270. [0130 17:33:58 @multigpu.py:323] [p0115] step: count(145), step_time 1387.85, mean_step_time 1452.71, it/s 0.69
  1271. sending to address tcp://p0112:61216
  1272. ##### Sending to neptune: online_score : 0.0660777313842 , 1.3 #####
  1273. [u'online', 1.3]
  1274. receiving
  1275. [0130 17:33:59 @multigpu.py:323] [p0576] step: count(170), step_time 1392.35, mean_step_time 1373.15, it/s 0.73
  1276. [0130 17:34:00 @multigpu.py:323] [p0115] step: count(146), step_time 1359.71, mean_step_time 1451.55, it/s 0.69
  1277. [0130 17:34:00 @multigpu.py:323] [p0574] step: count(164), step_time 1395.34, mean_step_time 1427.35, it/s 0.7
  1278. [0130 17:34:01 @multigpu.py:323] [p0115] step: count(147), step_time 1373.86, mean_step_time 1389.2, it/s 0.72
  1279. [0130 17:34:01 @multigpu.py:323] [p0574] step: count(165), step_time 1364.25, mean_step_time 1389.23, it/s 0.72
  1280. [0130 17:34:01 @multigpu.py:323] [p0576] step: count(171), step_time 1705.46, mean_step_time 1389.26, it/s 0.72
  1281. [0130 17:34:03 @multigpu.py:323] [p0115] step: count(148), step_time 1351.29, mean_step_time 1384.75, it/s 0.72
  1282. [0130 17:34:03 @multigpu.py:323] [p0576] step: count(172), step_time 1396.38, mean_step_time 1390.3, it/s 0.72
  1283. [0130 17:34:03 @multigpu.py:323] [p0574] step: count(166), step_time 1407.4, mean_step_time 1390.85, it/s 0.72
  1284. [0130 17:34:04 @multigpu.py:323] [p0115] step: count(149), step_time 1391.96, mean_step_time 1386.02, it/s 0.72
  1285. [0130 17:34:04 @multigpu.py:323] [p0574] step: count(167), step_time 1387.59, mean_step_time 1388.43, it/s 0.72
  1286. [0130 17:34:04 @multigpu.py:323] [p0576] step: count(173), step_time 1418.44, mean_step_time 1390.86, it/s 0.72
  1287. [0130 17:34:05 @multigpu.py:323] [p0115] step: count(150), step_time 1357.6, mean_step_time 1383.56, it/s 0.72
  1288. [0130 17:34:05 @multigpu.py:323] [p0574] step: count(168), step_time 1366.03, mean_step_time 1389.11, it/s 0.72
  1289. [0130 17:34:05 @multigpu.py:323] [p0576] step: count(174), step_time 1388.43, mean_step_time 1394.14, it/s 0.72
  1290. [0130 17:34:07 @multigpu.py:323] [p0115] step: count(151), step_time 1400.82, mean_step_time 1380.99, it/s 0.72
  1291. [0130 17:34:07 @multigpu.py:323] [p0574] step: count(169), step_time 1367.17, mean_step_time 1387.04, it/s 0.72
  1292. [0130 17:34:07 @multigpu.py:323] [p0576] step: count(175), step_time 1413.54, mean_step_time 1393.82, it/s 0.72
  1293. [0130 17:34:08 @multigpu.py:323] [p0115] step: count(152), step_time 1361.39, mean_step_time 1380.55, it/s 0.72
  1294. [0130 17:34:08 @multigpu.py:323] [p0574] step: count(170), step_time 1396.26, mean_step_time 1387.63, it/s 0.72
  1295. [0130 17:34:08 @multigpu.py:323] [p0576] step: count(176), step_time 1380.22, mean_step_time 1395.44, it/s 0.72
  1296. [0130 17:34:09 @multigpu.py:323] [p0115] step: count(153), step_time 1385.38, mean_step_time 1382.36, it/s 0.72
  1297. [0130 17:34:09 @multigpu.py:323] [p0574] step: count(171), step_time 1366.67, mean_step_time 1388.99, it/s 0.72
  1298. [0130 17:34:10 @multigpu.py:323] [p0576] step: count(177), step_time 1360.44, mean_step_time 1395.58, it/s 0.72
  1299. sending to address tcp://p0112:61216
  1300. ##### Sending to neptune: online_score : 0.0692923605442 , 0.7 #####
  1301. [u'online', 0.7]
  1302. ##### Sending to neptune: active_workers : 0.0692924124665 , 3 #####
  1303. receiving
  1304. [0130 17:34:11 @multigpu.py:323] [p0115] step: count(154), step_time 1383.39, mean_step_time 1383.09, it/s 0.72
  1305. [0130 17:34:11 @multigpu.py:323] [p0574] step: count(172), step_time 1391.29, mean_step_time 1389.07, it/s 0.72
  1306. [0130 17:34:11 @multigpu.py:323] [p0576] step: count(178), step_time 1413.78, mean_step_time 1396.57, it/s 0.72
  1307. [0130 17:34:12 @multigpu.py:323] [p0115] step: count(155), step_time 1360.79, mean_step_time 1382.91, it/s 0.72
  1308. [0130 17:34:12 @multigpu.py:323] [p0574] step: count(173), step_time 1400.33, mean_step_time 1390.89, it/s 0.72
  1309. [0130 17:34:12 @multigpu.py:323] [p0576] step: count(179), step_time 1355.16, mean_step_time 1396.05, it/s 0.72
  1310. [0130 17:34:14 @multigpu.py:323] [p0115] step: count(156), step_time 1419.04, mean_step_time 1382.19, it/s 0.72
  1311. [0130 17:34:14 @multigpu.py:323] [p0574] step: count(174), step_time 1371.01, mean_step_time 1388.67, it/s 0.72
  1312. [0130 17:34:14 @multigpu.py:323] [p0576] step: count(180), step_time 1350.09, mean_step_time 1395.97, it/s 0.72
  1313. [0130 17:34:15 @multigpu.py:323] [p0115] step: count(157), step_time 1368.62, mean_step_time 1381.3, it/s 0.72
  1314. [0130 17:34:15 @multigpu.py:323] [p0574] step: count(175), step_time 1396.0, mean_step_time 1390.56, it/s 0.72
  1315. [0130 17:34:15 @multigpu.py:323] [p0576] step: count(181), step_time 1375.06, mean_step_time 1394.83, it/s 0.72
  1316. [0130 17:34:16 @multigpu.py:323] [p0115] step: count(158), step_time 1412.57, mean_step_time 1383.02, it/s 0.72
  1317. [0130 17:34:16 @multigpu.py:323] [p0574] step: count(176), step_time 1350.15, mean_step_time 1388.76, it/s 0.72
  1318. [0130 17:34:16 @multigpu.py:323] [p0576] step: count(182), step_time 1365.4, mean_step_time 1396.19, it/s 0.72
  1319. [0130 17:34:18 @multigpu.py:323] [p0115] step: count(159), step_time 1386.58, mean_step_time 1383.41, it/s 0.72
  1320. [0130 17:34:18 @multigpu.py:323] [p0576] step: count(183), step_time 1362.43, mean_step_time 1394.29, it/s 0.72
  1321. [0130 17:34:18 @multigpu.py:323] [p0574] step: count(177), step_time 1407.9, mean_step_time 1390.7, it/s 0.72
  1322. [0130 17:34:19 @multigpu.py:323] [p0576] step: count(184), step_time 1366.59, mean_step_time 1396.06, it/s 0.72
  1323. [0130 17:34:19 @multigpu.py:323] [p0115] step: count(160), step_time 1374.1, mean_step_time 1384.07, it/s 0.72
  1324. [0130 17:34:19 @multigpu.py:323] [p0574] step: count(178), step_time 1409.13, mean_step_time 1390.58, it/s 0.72
  1325. [0130 17:34:20 @multigpu.py:323] [p0576] step: count(185), step_time 1375.44, mean_step_time 1395.46, it/s 0.72
  1326. [0130 17:34:20 @multigpu.py:323] [p0574] step: count(179), step_time 1321.83, mean_step_time 1381.23, it/s 0.72
  1327. [0130 17:34:21 @multigpu.py:323] [p0115] step: count(161), step_time 1406.45, mean_step_time 1383.51, it/s 0.72
  1328. [0130 17:34:22 @multigpu.py:323] [p0576] step: count(186), step_time 1394.41, mean_step_time 1396.06, it/s 0.72
  1329. [0130 17:34:22 @multigpu.py:323] [p0574] step: count(180), step_time 1405.91, mean_step_time 1382.93, it/s 0.72
  1330. [0130 17:34:22 @multigpu.py:323] [p0115] step: count(162), step_time 1402.49, mean_step_time 1384.89, it/s 0.72
  1331. [0130 17:34:23 @multigpu.py:323] [p0576] step: count(187), step_time 1335.38, mean_step_time 1393.49, it/s 0.72
  1332. [0130 17:34:23 @multigpu.py:323] [p0115] step: count(163), step_time 1373.02, mean_step_time 1385.41, it/s 0.72
  1333. [0130 17:34:23 @multigpu.py:323] [p0574] step: count(181), step_time 1410.48, mean_step_time 1383.88, it/s 0.72
  1334. [0130 17:34:25 @multigpu.py:323] [p0576] step: count(188), step_time 1389.98, mean_step_time 1396.13, it/s 0.72
  1335. [0130 17:34:25 @multigpu.py:323] [p0574] step: count(182), step_time 1353.91, mean_step_time 1383.42, it/s 0.72
  1336. [0130 17:34:25 @multigpu.py:323] [p0115] step: count(164), step_time 1392.64, mean_step_time 1382.48, it/s 0.72
  1337. sending to address tcp://p0112:61216
  1338. ##### Sending to neptune: online_score : 0.0735821829902 , 1.7 #####
  1339. [u'online', 1.7]
  1340. receiving
  1341. [0130 17:34:26 @multigpu.py:323] [p0576] step: count(189), step_time 1328.79, mean_step_time 1393.39, it/s 0.72
  1342. [0130 17:34:26 @multigpu.py:323] [p0115] step: count(165), step_time 1361.66, mean_step_time 1381.17, it/s 0.72
  1343. [0130 17:34:26 @multigpu.py:323] [p0574] step: count(183), step_time 1382.06, mean_step_time 1382.54, it/s 0.72
  1344. [0130 17:34:27 @multigpu.py:323] [p0576] step: count(190), step_time 1381.09, mean_step_time 1392.83, it/s 0.72
  1345. [0130 17:34:27 @multigpu.py:323] [p0574] step: count(184), step_time 1377.0, mean_step_time 1381.62, it/s 0.72
  1346. [0130 17:34:27 @multigpu.py:323] [p0115] step: count(166), step_time 1383.95, mean_step_time 1382.38, it/s 0.72
  1347. [0130 17:34:29 @multigpu.py:323] [p0115] step: count(167), step_time 1391.89, mean_step_time 1383.28, it/s 0.72
  1348. [0130 17:34:29 @multigpu.py:323] [p0574] step: count(185), step_time 1398.33, mean_step_time 1383.32, it/s 0.72
  1349. [0130 17:34:29 @multigpu.py:323] [p0576] step: count(191), step_time 1513.07, mean_step_time 1383.21, it/s 0.72
  1350. [0130 17:34:30 @multigpu.py:323] [p0115] step: count(168), step_time 1358.74, mean_step_time 1383.65, it/s 0.72
  1351. [0130 17:34:30 @multigpu.py:323] [p0574] step: count(186), step_time 1370.96, mean_step_time 1381.5, it/s 0.72
  1352. [0130 17:34:30 @multigpu.py:323] [p0576] step: count(192), step_time 1385.18, mean_step_time 1382.65, it/s 0.72
  1353. [0130 17:34:32 @multigpu.py:323] [p0576] step: count(193), step_time 1358.67, mean_step_time 1379.66, it/s 0.72
  1354. sending to address tcp://p0112:61216
  1355. ##### Sending to neptune: online_score : 0.0752706305186 , 0.8 #####
  1356. [u'online', 0.8]
  1357. receiving
  1358. [0130 17:34:32 @multigpu.py:323] [p0574] step: count(187), step_time 1392.0, mean_step_time 1381.72, it/s 0.72
  1359. [0130 17:34:32 @multigpu.py:323] [p0115] step: count(169), step_time 1417.02, mean_step_time 1384.91, it/s 0.72
  1360. [0130 17:34:33 @multigpu.py:323] [p0115] step: count(170), step_time 1332.3, mean_step_time 1383.64, it/s 0.72
  1361. [0130 17:34:33 @multigpu.py:323] [p0574] step: count(188), step_time 1353.15, mean_step_time 1381.08, it/s 0.72
  1362. [0130 17:34:33 @multigpu.py:323] [p0576] step: count(194), step_time 1381.28, mean_step_time 1379.3, it/s 0.73
  1363. [0130 17:34:34 @multigpu.py:323] [p0576] step: count(195), step_time 1359.54, mean_step_time 1376.6, it/s 0.73
  1364. [0130 17:34:34 @multigpu.py:323] [p0574] step: count(189), step_time 1392.51, mean_step_time 1382.34, it/s 0.72
  1365. [0130 17:34:34 @multigpu.py:323] [p0115] step: count(171), step_time 1418.38, mean_step_time 1384.52, it/s 0.72
  1366. [0130 17:34:36 @multigpu.py:323] [p0576] step: count(196), step_time 1368.99, mean_step_time 1376.04, it/s 0.73
  1367. [0130 17:34:36 @multigpu.py:323] [p0574] step: count(190), step_time 1355.72, mean_step_time 1380.32, it/s 0.72
  1368. [0130 17:34:36 @multigpu.py:323] [p0115] step: count(172), step_time 1368.87, mean_step_time 1384.89, it/s 0.72
  1369. [0130 17:34:37 @multigpu.py:323] [p0576] step: count(197), step_time 1351.55, mean_step_time 1375.59, it/s 0.73
  1370. [0130 17:34:37 @multigpu.py:323] [p0574] step: count(191), step_time 1382.27, mean_step_time 1381.1, it/s 0.72
  1371. [0130 17:34:37 @multigpu.py:323] [p0115] step: count(173), step_time 1384.74, mean_step_time 1384.86, it/s 0.72
  1372. sending to address tcp://p0112:61216
  1373. ##### Sending to neptune: online_score : 0.0769521447023 , 1.2 #####
  1374. [u'online', 1.2]
  1375. receiving
  1376. [0130 17:34:38 @multigpu.py:323] [p0576] step: count(198), step_time 1382.29, mean_step_time 1374.02, it/s 0.73
  1377. [0130 17:34:38 @multigpu.py:323] [p0574] step: count(192), step_time 1367.78, mean_step_time 1379.92, it/s 0.72
  1378. [0130 17:34:38 @multigpu.py:323] [p0115] step: count(174), step_time 1381.63, mean_step_time 1384.77, it/s 0.72
  1379. [0130 17:34:40 @multigpu.py:323] [p0576] step: count(199), step_time 1398.43, mean_step_time 1376.18, it/s 0.73
  1380. [0130 17:34:40 @multigpu.py:323] [p0574] step: count(193), step_time 1392.39, mean_step_time 1379.52, it/s 0.72
  1381. [0130 17:34:40 @multigpu.py:323] [p0115] step: count(175), step_time 1386.47, mean_step_time 1386.06, it/s 0.72
  1382. [0130 17:34:41 @multigpu.py:323] [p0576] step: count(200), step_time 1342.74, mean_step_time 1375.81, it/s 0.73
  1383. sending debugging info...
  1384. sending to address tcp://p0112:61216
  1385. sending to address tcp://p0112:61216
  1386. ##### Sending to neptune: mean_delay : 0.0779323197073 , 0.0 #####
  1387. ##### Sending to neptune: max_delay : 0.0779323197073 , -0.0 #####
  1388. ##### Sending to neptune: min_delay : 0.0779323197073 , -0.0 #####
  1389. [u'delays', [0.0, -0.0, -0.0]]
  1390. receiving
  1391. ##### Sending to neptune: cost : 0.0779328760836 , -0.00780312530696 #####
  1392. sending to address tcp://p0112:61216
  1393. ##### Sending to neptune: policy_loss : 0.0779328760836 , 0.118199490011 #####
  1394. ##### Sending to neptune: xentropy_loss : 0.0779328760836 , -2.29004001617 #####
  1395. ##### Sending to neptune: value_loss : 0.0779328760836 , 1.17304027081 #####
  1396. ##### Sending to neptune: advantage : 0.0779328760836 , -0.000583324115723 #####
  1397. ##### Sending to neptune: pred_reward : 0.0779328760836 , 0.359293758869 #####
  1398. ##### Sending to neptune: max_logit : 0.0779328760836 , 0.189353853464 #####
  1399. [u'loss', -0.007803125306963921, 0.11819949001073837, -2.2900400161743164, 1.1730402708053589, -0.0005833241157233715, 0.35929375886917114, 0.1893538534641266]
  1400. receiving
  1401. ##### Sending to neptune: active_relus : 0.0779333224561 , 8862300.92 #####
  1402. ##### Sending to neptune: dp_per_s : 0.0779333224561 , 92.2689357559 #####
  1403. [u'other', 8862300.92, 92.26893575586253]
  1404. receiving
  1405. [0130 17:34:41 @multigpu.py:323] [p0574] step: count(194), step_time 1399.89, mean_step_time 1380.97, it/s 0.72
  1406. [0130 17:34:41 @multigpu.py:323] [p0115] step: count(176), step_time 1364.66, mean_step_time 1383.34, it/s 0.72
  1407. [0130 17:34:43 @multigpu.py:323] [p0576] step: count(201), step_time 1375.37, mean_step_time 1375.83, it/s 0.73
  1408. [0130 17:34:43 @multigpu.py:323] [p0115] step: count(177), step_time 1421.45, mean_step_time 1385.98, it/s 0.72
  1409. [0130 17:34:43 @multigpu.py:323] [p0574] step: count(195), step_time 1460.79, mean_step_time 1384.21, it/s 0.72
  1410. [0130 17:34:44 @multigpu.py:323] [p0576] step: count(202), step_time 1389.25, mean_step_time 1377.02, it/s 0.73
  1411. [0130 17:34:44 @multigpu.py:323] [p0115] step: count(178), step_time 1326.35, mean_step_time 1381.67, it/s 0.72
  1412. [0130 17:34:44 @multigpu.py:323] [p0574] step: count(196), step_time 1371.53, mean_step_time 1385.28, it/s 0.72
  1413. [0130 17:34:45 @multigpu.py:323] [p0576] step: count(203), step_time 1359.61, mean_step_time 1376.88, it/s 0.73
  1414. [0130 17:34:45 @multigpu.py:323] [p0115] step: count(179), step_time 1378.42, mean_step_time 1381.26, it/s 0.72
  1415. [0130 17:34:46 @multigpu.py:323] [p0574] step: count(197), step_time 1484.98, mean_step_time 1389.13, it/s 0.72
  1416. [0130 17:34:47 @multigpu.py:323] [p0576] step: count(204), step_time 1380.84, mean_step_time 1377.59, it/s 0.73
  1417. [0130 17:34:47 @multigpu.py:323] [p0115] step: count(180), step_time 1379.61, mean_step_time 1381.54, it/s 0.72
  1418. [0130 17:34:47 @multigpu.py:323] [p0574] step: count(198), step_time 1512.61, mean_step_time 1394.3, it/s 0.72
  1419. [0130 17:34:48 @multigpu.py:323] [p0576] step: count(205), step_time 1359.08, mean_step_time 1376.78, it/s 0.73
  1420. [0130 17:34:48 @multigpu.py:323] [p0115] step: count(181), step_time 1355.4, mean_step_time 1378.99, it/s 0.73
  1421. [0130 17:34:48 @multigpu.py:323] [p0574] step: count(199), step_time 1361.12, mean_step_time 1396.27, it/s 0.72
  1422. [0130 17:34:49 @multigpu.py:323] [p0576] step: count(206), step_time 1383.91, mean_step_time 1376.25, it/s 0.73
  1423. [0130 17:34:50 @multigpu.py:323] [p0115] step: count(182), step_time 1399.66, mean_step_time 1378.84, it/s 0.73
  1424. [0130 17:34:50 @multigpu.py:323] [p0574] step: count(200), step_time 1434.78, mean_step_time 1397.71, it/s 0.72
  1425. sending debugging info...
  1426. sending to address tcp://p0112:61216
  1427. ##### Sending to neptune: mean_delay : 0.0803513783216 , 0.0 #####
  1428. ##### Sending to neptune: max_delay : 0.0803513783216 , -0.0 #####
  1429. ##### Sending to neptune: min_delay : 0.0803513783216 , -0.0 #####
  1430. [u'delays', [0.0, -0.0, -0.0]]
  1431. sending to address tcp://p0112:61216
  1432. receiving
  1433. ##### Sending to neptune: cost : 0.080351874431 , -0.00596463959664 #####
  1434. ##### Sending to neptune: policy_loss : 0.080351874431 , 0.299598842859 #####
  1435. ##### Sending to neptune: xentropy_loss : 0.080351874431 , -2.28987956047 #####
  1436. ##### Sending to neptune: value_loss : 0.080351874431 , 1.22680675983 #####
  1437. sending to address tcp://p0112:61216
  1438. ##### Sending to neptune: advantage : 0.080351874431 , -0.00136904569808 #####
  1439. ##### Sending to neptune: pred_reward : 0.080351874431 , 0.362240970135 #####
  1440. ##### Sending to neptune: max_logit : 0.080351874431 , 0.190133050084 #####
  1441. [u'loss', -0.005964639596641064, 0.2995988428592682, -2.289879560470581, 1.2268067598342896, -0.0013690456980839372, 0.3622409701347351, 0.19013305008411407]
  1442. receiving
  1443. ##### Sending to neptune: active_relus : 0.0803523513344 , 8882302.82 #####
  1444. ##### Sending to neptune: dp_per_s : 0.0803523513344 , 88.9797109993 #####
  1445. [u'other', 8882302.82, 88.97971099925284]
  1446. receiving
  1447. [0130 17:34:51 @multigpu.py:323] [p0576] step: count(207), step_time 1366.96, mean_step_time 1377.83, it/s 0.73
  1448. [0130 17:34:51 @multigpu.py:323] [p0115] step: count(183), step_time 1397.31, mean_step_time 1380.06, it/s 0.72
  1449. [0130 17:34:51 @multigpu.py:323] [p0574] step: count(201), step_time 1405.29, mean_step_time 1397.45, it/s 0.72
  1450. sending to address tcp://p0112:61216
  1451. ##### Sending to neptune: online_score : 0.0809522196982 , 1.4 #####
  1452. [u'online', 1.4]
  1453. receiving
  1454. [0130 17:34:52 @multigpu.py:323] [p0576] step: count(208), step_time 1430.94, mean_step_time 1379.88, it/s 0.72
  1455. [0130 17:34:52 @multigpu.py:323] [p0115] step: count(184), step_time 1378.93, mean_step_time 1379.37, it/s 0.72
  1456. [0130 17:34:53 @multigpu.py:323] [p0574] step: count(202), step_time 1402.51, mean_step_time 1399.88, it/s 0.71
  1457. [0130 17:34:54 @multigpu.py:323] [p0576] step: count(209), step_time 1332.2, mean_step_time 1380.05, it/s 0.72
  1458. [0130 17:34:54 @multigpu.py:323] [p0115] step: count(185), step_time 1397.63, mean_step_time 1381.17, it/s 0.72
  1459. [0130 17:34:54 @multigpu.py:323] [p0574] step: count(203), step_time 1388.81, mean_step_time 1400.22, it/s 0.71
  1460. [0130 17:34:55 @multigpu.py:323] [p0576] step: count(210), step_time 1379.6, mean_step_time 1379.97, it/s 0.72
  1461. [0130 17:34:55 @multigpu.py:323] [p0115] step: count(186), step_time 1369.04, mean_step_time 1380.43, it/s 0.72
  1462. [0130 17:34:55 @multigpu.py:323] [p0574] step: count(204), step_time 1403.95, mean_step_time 1401.57, it/s 0.71
  1463. [0130 17:34:57 @multigpu.py:323] [p0574] step: count(205), step_time 1470.88, mean_step_time 1405.2, it/s 0.71
  1464. [0130 17:34:57 @multigpu.py:323] [p0115] step: count(187), step_time 1890.06, mean_step_time 1405.33, it/s 0.71
  1465. [0130 17:34:57 @multigpu.py:323] [p0576] step: count(211), step_time 2018.29, mean_step_time 1405.24, it/s 0.71
  1466. sending to address tcp://p0112:61216
  1467. ##### Sending to neptune: online_score : 0.0823914271593 , 1.3 #####
  1468. [u'online', 1.3]
  1469. receiving
  1470. [0130 17:34:58 @multigpu.py:323] [p0576] step: count(212), step_time 1364.15, mean_step_time 1404.18, it/s 0.71
  1471. [0130 17:34:58 @multigpu.py:323] [p0115] step: count(188), step_time 1385.2, mean_step_time 1406.66, it/s 0.71
  1472. [0130 17:34:58 @multigpu.py:323] [p0574] step: count(206), step_time 1431.93, mean_step_time 1408.24, it/s 0.71
  1473. sending to address tcp://p0112:61216
  1474. ##### Sending to neptune: online_score : 0.0828070782953 , 1.7 #####
  1475. [u'online', 1.7]
  1476. receiving
  1477. [0130 17:35:00 @multigpu.py:323] [p0576] step: count(213), step_time 1372.01, mean_step_time 1404.85, it/s 0.71
  1478. [0130 17:35:00 @multigpu.py:323] [p0115] step: count(189), step_time 1385.77, mean_step_time 1405.09, it/s 0.71
  1479. [0130 17:35:00 @multigpu.py:323] [p0574] step: count(207), step_time 1425.89, mean_step_time 1409.94, it/s 0.71
  1480. [0130 17:35:01 @multigpu.py:323] [p0115] step: count(190), step_time 1344.78, mean_step_time 1405.72, it/s 0.71
  1481. [0130 17:35:01 @multigpu.py:323] [p0576] step: count(214), step_time 1387.69, mean_step_time 1405.17, it/s 0.71
  1482. [0130 17:35:01 @multigpu.py:323] [p0574] step: count(208), step_time 1364.84, mean_step_time 1410.52, it/s 0.71
  1483. [0130 17:35:02 @multigpu.py:323] [p0576] step: count(215), step_time 1356.17, mean_step_time 1405.0, it/s 0.71
  1484. [0130 17:35:02 @multigpu.py:323] [p0115] step: count(191), step_time 1402.47, mean_step_time 1404.92, it/s 0.71
  1485. [0130 17:35:03 @multigpu.py:323] [p0574] step: count(209), step_time 1413.0, mean_step_time 1411.55, it/s 0.71
  1486. [0130 17:35:04 @multigpu.py:323] [p0576] step: count(216), step_time 1396.33, mean_step_time 1406.37, it/s 0.71
  1487. [0130 17:35:04 @multigpu.py:323] [p0115] step: count(192), step_time 1363.41, mean_step_time 1404.65, it/s 0.71
  1488. [0130 17:35:04 @multigpu.py:323] [p0574] step: count(210), step_time 1400.8, mean_step_time 1413.8, it/s 0.71
  1489. [0130 17:35:05 @multigpu.py:323] [p0576] step: count(217), step_time 1388.98, mean_step_time 1408.24, it/s 0.71
  1490. [0130 17:35:05 @multigpu.py:323] [p0115] step: count(193), step_time 1392.09, mean_step_time 1405.02, it/s 0.71
  1491. [0130 17:35:05 @multigpu.py:323] [p0574] step: count(211), step_time 1408.62, mean_step_time 1415.12, it/s 0.71
  1492. [0130 17:35:07 @multigpu.py:323] [p0576] step: count(218), step_time 1369.08, mean_step_time 1407.58, it/s 0.71
  1493. [0130 17:35:07 @multigpu.py:323] [p0115] step: count(194), step_time 1404.0, mean_step_time 1406.14, it/s 0.71
  1494. [0130 17:35:07 @multigpu.py:323] [p0574] step: count(212), step_time 1398.96, mean_step_time 1416.68, it/s 0.71
  1495. [0130 17:35:08 @multigpu.py:323] [p0115] step: count(195), step_time 1346.86, mean_step_time 1404.16, it/s 0.71
  1496. [0130 17:35:08 @multigpu.py:323] [p0576] step: count(219), step_time 1433.47, mean_step_time 1409.33, it/s 0.71
  1497. [0130 17:35:08 @multigpu.py:323] [p0574] step: count(213), step_time 1362.13, mean_step_time 1415.17, it/s 0.71
  1498. [0130 17:35:09 @multigpu.py:323] [p0115] step: count(196), step_time 1389.49, mean_step_time 1405.4, it/s 0.71
  1499. [0130 17:35:09 @multigpu.py:323] [p0576] step: count(220), step_time 1391.26, mean_step_time 1411.76, it/s 0.71
  1500. [0130 17:35:10 @multigpu.py:323] [p0574] step: count(214), step_time 1386.5, mean_step_time 1414.5, it/s 0.71
  1501. [0130 17:35:11 @multigpu.py:323] [p0576] step: count(221), step_time 1364.82, mean_step_time 1411.23, it/s 0.71
  1502. [0130 17:35:11 @multigpu.py:323] [p0115] step: count(197), step_time 1424.18, mean_step_time 1405.53, it/s 0.71
  1503. [0130 17:35:11 @multigpu.py:323] [p0574] step: count(215), step_time 1408.09, mean_step_time 1411.86, it/s 0.71
  1504. [0130 17:35:12 @multigpu.py:323] [p0576] step: count(222), step_time 1351.14, mean_step_time 1409.33, it/s 0.71
  1505. [0130 17:35:12 @multigpu.py:323] [p0115] step: count(198), step_time 1369.54, mean_step_time 1407.69, it/s 0.71
  1506. [0130 17:35:12 @multigpu.py:323] [p0574] step: count(216), step_time 1432.83, mean_step_time 1414.93, it/s 0.71
  1507. [0130 17:35:14 @multigpu.py:323] [p0576] step: count(223), step_time 1396.41, mean_step_time 1411.17, it/s 0.71
  1508. [0130 17:35:14 @multigpu.py:323] [p0115] step: count(199), step_time 1409.2, mean_step_time 1409.23, it/s 0.71
  1509. [0130 17:35:14 @multigpu.py:323] [p0574] step: count(217), step_time 1347.35, mean_step_time 1408.04, it/s 0.71
  1510. [0130 17:35:15 @multigpu.py:323] [p0115] step: count(200), step_time 1346.51, mean_step_time 1407.58, it/s 0.71
  1511. sending debugging info...
  1512. sending to address tcp://p0112:61216
  1513. ##### Sending to neptune: mean_delay : 0.0873081782791 , 0.0 #####
  1514. sending to address tcp://p0112:61216
  1515. ##### Sending to neptune: max_delay : 0.0873081782791 , -0.0 #####
  1516. ##### Sending to neptune: min_delay : 0.0873081782791 , -0.0 #####
  1517. [u'delays', [0.0, -0.0, -0.0]]
  1518. ##### Sending to neptune: active_workers : 0.0873083741135 , 3 #####
  1519. receiving
  1520. ##### Sending to neptune: cost : 0.0873086949852 , -0.0134393870831 #####
  1521. sending to address tcp://p0112:61216
  1522. ##### Sending to neptune: policy_loss : 0.0873086949852 , -0.510861754417 #####
  1523. ##### Sending to neptune: xentropy_loss : 0.0873086949852 , -2.28939032555 #####
  1524. ##### Sending to neptune: value_loss : 0.0873086949852 , 1.08001065254 #####
  1525. ##### Sending to neptune: advantage : 0.0873086949852 , 0.00229038670659 #####
  1526. ##### Sending to neptune: pred_reward : 0.0873086949852 , 0.375006139278 #####
  1527. ##### Sending to neptune: max_logit : 0.0873086949852 , 0.191750437021 #####
  1528. [u'loss', -0.013439387083053589, -0.5108617544174194, -2.2893903255462646, 1.0800106525421143, 0.0022903867065906525, 0.37500613927841187, 0.1917504370212555]
  1529. receiving
  1530. ##### Sending to neptune: active_relus : 0.0873091855314 , 8887920.51 #####
  1531. ##### Sending to neptune: dp_per_s : 0.0873091855314 , 90.4954598592 #####
  1532. [u'other', 8887920.51, 90.49545985924865]
  1533. receiving
  1534. [0130 17:35:15 @multigpu.py:323] [p0576] step: count(224), step_time 1399.52, mean_step_time 1412.1, it/s 0.71
  1535. [0130 17:35:15 @multigpu.py:323] [p0574] step: count(218), step_time 1376.75, mean_step_time 1401.25, it/s 0.71
  1536. [0130 17:35:16 @multigpu.py:323] [p0115] step: count(201), step_time 1388.62, mean_step_time 1409.24, it/s 0.71
  1537. [0130 17:35:16 @multigpu.py:323] [p0576] step: count(225), step_time 1385.96, mean_step_time 1413.44, it/s 0.71
  1538. [0130 17:35:16 @multigpu.py:323] [p0574] step: count(219), step_time 1376.69, mean_step_time 1402.03, it/s 0.71
  1539. [0130 17:35:18 @multigpu.py:323] [p0115] step: count(202), step_time 1362.17, mean_step_time 1407.36, it/s 0.71
  1540. [0130 17:35:18 @multigpu.py:323] [p0576] step: count(226), step_time 1362.54, mean_step_time 1412.38, it/s 0.71
  1541. [0130 17:35:18 @multigpu.py:323] [p0574] step: count(220), step_time 1402.37, mean_step_time 1400.41, it/s 0.71
  1542. sending to address tcp://p0112:61216
  1543. ##### Sending to neptune: online_score : 0.0882514672147 , 1.4 #####
  1544. [u'online', 1.4]
  1545. receiving
  1546. [0130 17:35:19 @multigpu.py:323] [p0576] step: count(227), step_time 1344.1, mean_step_time 1411.23, it/s 0.71
  1547. [0130 17:35:19 @multigpu.py:323] [p0115] step: count(203), step_time 1401.89, mean_step_time 1407.59, it/s 0.71
  1548. [0130 17:35:19 @multigpu.py:323] [p0574] step: count(221), step_time 1395.37, mean_step_time 1399.91, it/s 0.71
  1549. [0130 17:35:20 @multigpu.py:323] [p0576] step: count(228), step_time 1371.69, mean_step_time 1408.27, it/s 0.71
  1550. [0130 17:35:20 @multigpu.py:323] [p0115] step: count(204), step_time 1370.23, mean_step_time 1407.16, it/s 0.71
  1551. [0130 17:35:21 @multigpu.py:323] [p0574] step: count(222), step_time 1453.52, mean_step_time 1402.46, it/s 0.71
  1552. sending to address tcp://p0112:61216
  1553. ##### Sending to neptune: online_score : 0.0889305941264 , 1.3 #####
  1554. [u'online', 1.3]
  1555. receiving
  1556. [0130 17:35:22 @multigpu.py:323] [p0576] step: count(229), step_time 1400.69, mean_step_time 1411.7, it/s 0.71
  1557. [0130 17:35:22 @multigpu.py:323] [p0115] step: count(205), step_time 1375.32, mean_step_time 1406.04, it/s 0.71
  1558. [0130 17:35:22 @multigpu.py:323] [p0574] step: count(223), step_time 1446.75, mean_step_time 1405.36, it/s 0.71
  1559. [0130 17:35:23 @multigpu.py:323] [p0576] step: count(230), step_time 1384.77, mean_step_time 1411.95, it/s 0.71
  1560. [0130 17:35:23 @multigpu.py:323] [p0115] step: count(206), step_time 1368.15, mean_step_time 1406.0, it/s 0.71
  1561. [0130 17:35:24 @multigpu.py:323] [p0574] step: count(224), step_time 1409.12, mean_step_time 1405.62, it/s 0.71
  1562. [0130 17:35:25 @multigpu.py:323] [p0574] step: count(225), step_time 1367.4, mean_step_time 1400.45, it/s 0.71
  1563. [0130 17:35:25 @multigpu.py:323] [p0115] step: count(207), step_time 1781.69, mean_step_time 1400.58, it/s 0.71
  1564. [0130 17:35:25 @multigpu.py:323] [p0576] step: count(231), step_time 1789.81, mean_step_time 1400.53, it/s 0.71
  1565. [0130 17:35:26 @multigpu.py:323] [p0576] step: count(232), step_time 1419.14, mean_step_time 1403.28, it/s 0.71
  1566. [0130 17:35:26 @multigpu.py:323] [p0574] step: count(226), step_time 1427.12, mean_step_time 1400.2, it/s 0.71
  1567. [0130 17:35:26 @multigpu.py:323] [p0115] step: count(208), step_time 1432.61, mean_step_time 1402.95, it/s 0.71
  1568. [0130 17:35:28 @multigpu.py:323] [p0574] step: count(227), step_time 1343.76, mean_step_time 1396.1, it/s 0.72
  1569. [0130 17:35:28 @multigpu.py:323] [p0576] step: count(233), step_time 1364.21, mean_step_time 1402.89, it/s 0.71
  1570. [0130 17:35:28 @multigpu.py:323] [p0115] step: count(209), step_time 1398.28, mean_step_time 1403.58, it/s 0.71
  1571. sending to address tcp://p0112:61216
  1572. ##### Sending to neptune: online_score : 0.0910588008165 , 1.4 #####
  1573. [u'online', 1.4]
  1574. receiving
  1575. [0130 17:35:29 @multigpu.py:323] [p0574] step: count(228), step_time 1385.26, mean_step_time 1397.12, it/s 0.72
  1576. [0130 17:35:29 @multigpu.py:323] [p0576] step: count(234), step_time 1381.8, mean_step_time 1402.59, it/s 0.71
  1577. [0130 17:35:29 @multigpu.py:323] [p0115] step: count(210), step_time 1369.82, mean_step_time 1404.83, it/s 0.71
  1578. [0130 17:35:30 @multigpu.py:323] [p0574] step: count(229), step_time 1344.25, mean_step_time 1393.68, it/s 0.72
  1579. [0130 17:35:31 @multigpu.py:323] [p0576] step: count(235), step_time 1426.86, mean_step_time 1406.13, it/s 0.71
  1580. [0130 17:35:31 @multigpu.py:323] [p0115] step: count(211), step_time 1409.13, mean_step_time 1405.16, it/s 0.71
  1581. [0130 17:35:32 @multigpu.py:323] [p0574] step: count(230), step_time 1422.48, mean_step_time 1394.77, it/s 0.72
  1582. [0130 17:35:32 @multigpu.py:323] [p0115] step: count(212), step_time 1342.63, mean_step_time 1404.12, it/s 0.71
  1583. [0130 17:35:32 @multigpu.py:323] [p0576] step: count(236), step_time 1360.8, mean_step_time 1404.35, it/s 0.71
  1584. [0130 17:35:33 @multigpu.py:323] [p0574] step: count(231), step_time 1366.62, mean_step_time 1392.67, it/s 0.72
  1585. [0130 17:35:33 @multigpu.py:323] [p0576] step: count(237), step_time 1404.87, mean_step_time 1405.15, it/s 0.71
  1586. [0130 17:35:33 @multigpu.py:323] [p0115] step: count(213), step_time 1440.57, mean_step_time 1406.55, it/s 0.71
  1587. [0130 17:35:35 @multigpu.py:323] [p0576] step: count(238), step_time 1372.38, mean_step_time 1405.31, it/s 0.71
  1588. [0130 17:35:35 @multigpu.py:323] [p0574] step: count(232), step_time 1442.58, mean_step_time 1394.85, it/s 0.72
  1589. [0130 17:35:35 @multigpu.py:323] [p0115] step: count(214), step_time 1366.34, mean_step_time 1404.66, it/s 0.71
  1590. [0130 17:35:36 @multigpu.py:323] [p0576] step: count(239), step_time 1396.97, mean_step_time 1403.49, it/s 0.71
  1591. [0130 17:35:36 @multigpu.py:323] [p0115] step: count(215), step_time 1435.04, mean_step_time 1409.07, it/s 0.71
  1592. [0130 17:35:36 @multigpu.py:323] [p0574] step: count(233), step_time 1562.81, mean_step_time 1404.88, it/s 0.71
  1593. [0130 17:35:37 @multigpu.py:323] [p0576] step: count(240), step_time 1401.66, mean_step_time 1404.01, it/s 0.71
  1594. [0130 17:35:38 @multigpu.py:323] [p0115] step: count(216), step_time 1426.44, mean_step_time 1410.92, it/s 0.71
  1595. [0130 17:35:38 @multigpu.py:323] [p0574] step: count(234), step_time 1530.0, mean_step_time 1412.06, it/s 0.71
  1596. [0130 17:35:39 @multigpu.py:323] [p0576] step: count(241), step_time 1389.36, mean_step_time 1405.23, it/s 0.71
  1597. [0130 17:35:39 @multigpu.py:323] [p0115] step: count(217), step_time 1348.05, mean_step_time 1407.11, it/s 0.71
  1598. [0130 17:35:39 @multigpu.py:323] [p0574] step: count(235), step_time 1505.13, mean_step_time 1416.91, it/s 0.71
  1599. [0130 17:35:40 @multigpu.py:323] [p0576] step: count(242), step_time 1439.39, mean_step_time 1409.65, it/s 0.71
  1600. [0130 17:35:40 @multigpu.py:323] [p0115] step: count(218), step_time 1401.83, mean_step_time 1408.73, it/s 0.71
  1601. [0130 17:35:41 @multigpu.py:323] [p0574] step: count(236), step_time 1410.48, mean_step_time 1415.79, it/s 0.71
  1602. [0130 17:35:42 @multigpu.py:323] [p0576] step: count(243), step_time 1347.01, mean_step_time 1407.18, it/s 0.71
  1603. [0130 17:35:42 @multigpu.py:323] [p0115] step: count(219), step_time 1380.97, mean_step_time 1407.31, it/s 0.71
  1604. [0130 17:35:42 @multigpu.py:323] [p0574] step: count(237), step_time 1423.83, mean_step_time 1419.62, it/s 0.7
  1605. [0130 17:35:43 @multigpu.py:323] [p0576] step: count(244), step_time 1387.0, mean_step_time 1406.55, it/s 0.71
  1606. [0130 17:35:43 @multigpu.py:323] [p0115] step: count(220), step_time 1371.53, mean_step_time 1408.57, it/s 0.71
  1607. [0130 17:35:44 @multigpu.py:323] [p0574] step: count(238), step_time 1374.06, mean_step_time 1419.48, it/s 0.7
  1608. [0130 17:35:44 @multigpu.py:323] [p0576] step: count(245), step_time 1384.53, mean_step_time 1406.48, it/s 0.71
  1609. [0130 17:35:44 @multigpu.py:323] [p0115] step: count(221), step_time 1386.56, mean_step_time 1408.46, it/s 0.71
  1610. [0130 17:35:45 @multigpu.py:323] [p0574] step: count(239), step_time 1363.55, mean_step_time 1418.82, it/s 0.7
  1611. [0130 17:35:46 @multigpu.py:323] [p0576] step: count(246), step_time 1337.92, mean_step_time 1405.25, it/s 0.71
  1612. [0130 17:35:46 @multigpu.py:323] [p0115] step: count(222), step_time 1363.66, mean_step_time 1408.54, it/s 0.71
  1613. [0130 17:35:46 @multigpu.py:323] [p0574] step: count(240), step_time 1375.6, mean_step_time 1417.48, it/s 0.71
  1614. [0130 17:35:47 @multigpu.py:323] [p0576] step: count(247), step_time 1425.29, mean_step_time 1409.31, it/s 0.71
  1615. [0130 17:35:47 @multigpu.py:323] [p0115] step: count(223), step_time 1375.84, mean_step_time 1407.23, it/s 0.71
  1616. [0130 17:35:48 @multigpu.py:323] [p0574] step: count(241), step_time 1392.03, mean_step_time 1417.32, it/s 0.71
  1617. sending to address tcp://p0112:61216
  1618. ##### Sending to neptune: online_score : 0.0964728083213 , 0.7 #####
  1619. [u'online', 0.7]
  1620. receiving
  1621. sending to address tcp://p0112:61216
  1622. ##### Sending to neptune: online_score : 0.0965966761112 , 2.3 #####
  1623. [u'online', 2.3]
  1624. receiving
  1625. [0130 17:35:49 @multigpu.py:323] [p0576] step: count(248), step_time 1370.39, mean_step_time 1409.24, it/s 0.71
  1626. [0130 17:35:49 @multigpu.py:323] [p0115] step: count(224), step_time 1380.98, mean_step_time 1407.77, it/s 0.71
  1627. [0130 17:35:49 @multigpu.py:323] [p0574] step: count(242), step_time 1367.97, mean_step_time 1413.04, it/s 0.71
  1628. [0130 17:35:50 @multigpu.py:323] [p0576] step: count(249), step_time 1371.62, mean_step_time 1407.79, it/s 0.71
  1629. [0130 17:35:50 @multigpu.py:323] [p0115] step: count(225), step_time 1399.34, mean_step_time 1408.97, it/s 0.71
  1630. [0130 17:35:50 @multigpu.py:323] [p0574] step: count(243), step_time 1377.29, mean_step_time 1409.57, it/s 0.71
  1631. [0130 17:35:51 @multigpu.py:323] [p0576] step: count(250), step_time 1401.31, mean_step_time 1408.62, it/s 0.71
  1632. [0130 17:35:51 @multigpu.py:323] [p0115] step: count(226), step_time 1427.54, mean_step_time 1411.94, it/s 0.71
  1633. [0130 17:35:52 @multigpu.py:323] [p0574] step: count(244), step_time 1402.78, mean_step_time 1409.25, it/s 0.71
  1634. [0130 17:35:53 @multigpu.py:323] [p0574] step: count(245), step_time 1371.89, mean_step_time 1409.47, it/s 0.71
  1635. [0130 17:35:53 @multigpu.py:323] [p0576] step: count(251), step_time 1805.83, mean_step_time 1409.42, it/s 0.71
  1636. [0130 17:35:53 @multigpu.py:323] [p0115] step: count(227), step_time 1732.07, mean_step_time 1409.46, it/s 0.71
  1637. sending to address tcp://p0112:61216
  1638. ##### Sending to neptune: online_score : 0.0980394785934 , 1.9 #####
  1639. [u'online', 1.9]
  1640. receiving
  1641. [0130 17:35:55 @multigpu.py:323] [p0115] step: count(228), step_time 1372.59, mean_step_time 1406.46, it/s 0.71
  1642. [0130 17:35:55 @multigpu.py:323] [p0574] step: count(246), step_time 1373.57, mean_step_time 1406.8, it/s 0.71
  1643. [0130 17:35:55 @multigpu.py:323] [p0576] step: count(252), step_time 1411.17, mean_step_time 1409.02, it/s 0.71
  1644. [0130 17:35:56 @multigpu.py:323] [p0115] step: count(229), step_time 1372.77, mean_step_time 1405.18, it/s 0.71
  1645. [0130 17:35:56 @multigpu.py:323] [p0574] step: count(247), step_time 1372.83, mean_step_time 1408.25, it/s 0.71
  1646. [0130 17:35:56 @multigpu.py:323] [p0576] step: count(253), step_time 1386.74, mean_step_time 1410.14, it/s 0.71
  1647. [0130 17:35:57 @multigpu.py:323] [p0115] step: count(230), step_time 1386.84, mean_step_time 1406.04, it/s 0.71
  1648. [0130 17:35:57 @multigpu.py:323] [p0574] step: count(248), step_time 1412.52, mean_step_time 1409.61, it/s 0.71
  1649. [0130 17:35:57 @multigpu.py:323] [p0576] step: count(254), step_time 1386.09, mean_step_time 1410.36, it/s 0.71
  1650. [0130 17:35:59 @multigpu.py:323] [p0574] step: count(249), step_time 1337.01, mean_step_time 1409.25, it/s 0.71
  1651. [0130 17:35:59 @multigpu.py:323] [p0115] step: count(231), step_time 1365.18, mean_step_time 1403.84, it/s 0.71
  1652. [0130 17:35:59 @multigpu.py:323] [p0576] step: count(255), step_time 1402.17, mean_step_time 1409.13, it/s 0.71
  1653. [0130 17:36:00 @multigpu.py:323] [p0115] step: count(232), step_time 1371.18, mean_step_time 1405.27, it/s 0.71
  1654. [0130 17:36:00 @multigpu.py:323] [p0574] step: count(250), step_time 1394.7, mean_step_time 1407.86, it/s 0.71
  1655. [0130 17:36:00 @multigpu.py:323] [p0576] step: count(256), step_time 1370.45, mean_step_time 1409.61, it/s 0.71
  1656. [0130 17:36:01 @multigpu.py:323] [p0115] step: count(233), step_time 1407.6, mean_step_time 1403.62, it/s 0.71
  1657. [0130 17:36:01 @multigpu.py:323] [p0574] step: count(251), step_time 1392.27, mean_step_time 1409.14, it/s 0.71
  1658. [0130 17:36:02 @multigpu.py:323] [p0576] step: count(257), step_time 1404.48, mean_step_time 1409.59, it/s 0.71
  1659. [0130 17:36:03 @multigpu.py:323] [p0115] step: count(234), step_time 1388.9, mean_step_time 1404.75, it/s 0.71
  1660. [0130 17:36:03 @multigpu.py:323] [p0574] step: count(252), step_time 1392.01, mean_step_time 1406.62, it/s 0.71
  1661. [0130 17:36:03 @multigpu.py:323] [p0576] step: count(258), step_time 1382.35, mean_step_time 1410.09, it/s 0.71
  1662. [0130 17:36:04 @multigpu.py:323] [p0574] step: count(253), step_time 1358.53, mean_step_time 1396.4, it/s 0.72
  1663. [0130 17:36:04 @multigpu.py:323] [p0115] step: count(235), step_time 1397.92, mean_step_time 1402.89, it/s 0.71
  1664. [0130 17:36:04 @multigpu.py:323] [p0576] step: count(259), step_time 1351.43, mean_step_time 1407.81, it/s 0.71
  1665. [0130 17:36:06 @multigpu.py:323] [p0115] step: count(236), step_time 1348.7, mean_step_time 1399.0, it/s 0.71
  1666. [0130 17:36:06 @multigpu.py:323] [p0576] step: count(260), step_time 1386.78, mean_step_time 1407.07, it/s 0.71
  1667. [0130 17:36:06 @multigpu.py:323] [p0574] step: count(254), step_time 1552.29, mean_step_time 1397.52, it/s 0.72
  1668. [0130 17:36:07 @multigpu.py:323] [p0115] step: count(237), step_time 1384.47, mean_step_time 1400.82, it/s 0.71
  1669. [0130 17:36:07 @multigpu.py:323] [p0576] step: count(261), step_time 1396.65, mean_step_time 1407.43, it/s 0.71
  1670. [0130 17:36:07 @multigpu.py:323] [p0574] step: count(255), step_time 1380.3, mean_step_time 1391.28, it/s 0.72
  1671. [0130 17:36:08 @multigpu.py:323] [p0115] step: count(238), step_time 1420.84, mean_step_time 1401.77, it/s 0.71
  1672. [0130 17:36:08 @multigpu.py:323] [p0576] step: count(262), step_time 1414.32, mean_step_time 1406.18, it/s 0.71
  1673. [0130 17:36:09 @multigpu.py:323] [p0574] step: count(256), step_time 1380.86, mean_step_time 1389.79, it/s 0.72
  1674. [0130 17:36:10 @multigpu.py:323] [p0115] step: count(239), step_time 1345.4, mean_step_time 1400.0, it/s 0.71
  1675. [0130 17:36:10 @multigpu.py:323] [p0576] step: count(263), step_time 1396.73, mean_step_time 1408.66, it/s 0.71
  1676. [0130 17:36:10 @multigpu.py:323] [p0574] step: count(257), step_time 1388.26, mean_step_time 1388.02, it/s 0.72
  1677. sending to address tcp://p0112:61216
  1678. ##### Sending to neptune: online_score : 0.102663866348 , 1.3 #####
  1679. [u'online', 1.3]
  1680. receiving
  1681. [0130 17:36:11 @multigpu.py:323] [p0115] step: count(240), step_time 1385.78, mean_step_time 1400.71, it/s 0.71
  1682. [0130 17:36:11 @multigpu.py:323] [p0576] step: count(264), step_time 1350.94, mean_step_time 1406.86, it/s 0.71
  1683. [0130 17:36:11 @multigpu.py:323] [p0574] step: count(258), step_time 1439.78, mean_step_time 1391.3, it/s 0.72
  1684. sending to address tcp://p0112:61216
  1685. ##### Sending to neptune: online_score : 0.103130486078 , 1.1 #####
  1686. [u'online', 1.1]
  1687. receiving
  1688. [0130 17:36:12 @multigpu.py:323] [p0115] step: count(241), step_time 1377.61, mean_step_time 1400.26, it/s 0.71
  1689. [0130 17:36:13 @multigpu.py:323] [p0576] step: count(265), step_time 1359.71, mean_step_time 1405.62, it/s 0.71
  1690. [0130 17:36:13 @multigpu.py:323] [p0574] step: count(259), step_time 1362.89, mean_step_time 1391.27, it/s 0.72
  1691. [0130 17:36:14 @multigpu.py:323] [p0115] step: count(242), step_time 1394.91, mean_step_time 1401.82, it/s 0.71
  1692. [0130 17:36:14 @multigpu.py:323] [p0576] step: count(266), step_time 1413.71, mean_step_time 1409.41, it/s 0.71
  1693. [0130 17:36:14 @multigpu.py:323] [p0574] step: count(260), step_time 1389.64, mean_step_time 1391.97, it/s 0.72
  1694. [0130 17:36:15 @multigpu.py:323] [p0115] step: count(243), step_time 1373.49, mean_step_time 1401.7, it/s 0.71
  1695. [0130 17:36:15 @multigpu.py:323] [p0576] step: count(267), step_time 1374.35, mean_step_time 1406.86, it/s 0.71
  1696. [0130 17:36:15 @multigpu.py:323] [p0574] step: count(261), step_time 1348.26, mean_step_time 1389.78, it/s 0.72
  1697. [0130 17:36:17 @multigpu.py:323] [p0115] step: count(244), step_time 1386.47, mean_step_time 1401.98, it/s 0.71
  1698. [0130 17:36:17 @multigpu.py:323] [p0576] step: count(268), step_time 1384.0, mean_step_time 1407.54, it/s 0.71
  1699. [0130 17:36:17 @multigpu.py:323] [p0574] step: count(262), step_time 1349.67, mean_step_time 1388.87, it/s 0.72
  1700. [0130 17:36:18 @multigpu.py:323] [p0115] step: count(245), step_time 1383.07, mean_step_time 1401.17, it/s 0.71
  1701. [0130 17:36:18 @multigpu.py:323] [p0576] step: count(269), step_time 1380.73, mean_step_time 1408.0, it/s 0.71
  1702. [0130 17:36:18 @multigpu.py:323] [p0574] step: count(263), step_time 1389.54, mean_step_time 1389.48, it/s 0.72
  1703. [0130 17:36:19 @multigpu.py:323] [p0115] step: count(246), step_time 1361.13, mean_step_time 1397.85, it/s 0.72
  1704. [0130 17:36:20 @multigpu.py:323] [p0574] step: count(264), step_time 1324.5, mean_step_time 1385.57, it/s 0.72
  1705. [0130 17:36:20 @multigpu.py:323] [p0576] step: count(270), step_time 1399.82, mean_step_time 1407.92, it/s 0.71
  1706. sending to address tcp://p0112:61216
  1707. ##### Sending to neptune: online_score : 0.10532262471 , 0.9 #####
  1708. [u'online', 0.9]
  1709. ##### Sending to neptune: active_workers : 0.105322658552 , 3 #####
  1710. receiving
  1711. [0130 17:36:21 @multigpu.py:323] [p0574] step: count(265), step_time 1493.54, mean_step_time 1391.65, it/s 0.72
  1712. [0130 17:36:21 @multigpu.py:323] [p0576] step: count(271), step_time 1479.04, mean_step_time 1391.58, it/s 0.72
  1713. [0130 17:36:21 @multigpu.py:323] [p0115] step: count(247), step_time 1609.22, mean_step_time 1391.7, it/s 0.72
  1714. [0130 17:36:22 @multigpu.py:323] [p0115] step: count(248), step_time 1364.46, mean_step_time 1391.3, it/s 0.72
  1715. [0130 17:36:22 @multigpu.py:323] [p0576] step: count(272), step_time 1373.2, mean_step_time 1389.69, it/s 0.72
  1716. [0130 17:36:23 @multigpu.py:323] [p0574] step: count(266), step_time 1546.98, mean_step_time 1400.32, it/s 0.71
  1717. [0130 17:36:24 @multigpu.py:323] [p0115] step: count(249), step_time 1400.73, mean_step_time 1392.69, it/s 0.72
  1718. [0130 17:36:24 @multigpu.py:323] [p0576] step: count(273), step_time 1402.29, mean_step_time 1390.46, it/s 0.72
  1719. [0130 17:36:24 @multigpu.py:323] [p0574] step: count(267), step_time 1500.75, mean_step_time 1406.72, it/s 0.71
  1720. [0130 17:36:25 @multigpu.py:323] [p0115] step: count(250), step_time 1340.03, mean_step_time 1390.35, it/s 0.72
  1721. [0130 17:36:25 @multigpu.py:323] [p0576] step: count(274), step_time 1363.4, mean_step_time 1389.33, it/s 0.72
  1722. [0130 17:36:26 @multigpu.py:323] [p0574] step: count(268), step_time 1638.14, mean_step_time 1418.0, it/s 0.71
  1723. [0130 17:36:27 @multigpu.py:323] [p0115] step: count(251), step_time 1383.45, mean_step_time 1391.27, it/s 0.72
  1724. [0130 17:36:27 @multigpu.py:323] [p0576] step: count(275), step_time 1434.35, mean_step_time 1390.94, it/s 0.72
  1725. [0130 17:36:27 @multigpu.py:323] [p0574] step: count(269), step_time 1532.01, mean_step_time 1427.75, it/s 0.7
  1726. [0130 17:36:28 @multigpu.py:323] [p0115] step: count(252), step_time 1385.99, mean_step_time 1392.01, it/s 0.72
  1727. [0130 17:36:28 @multigpu.py:323] [p0576] step: count(276), step_time 1401.97, mean_step_time 1392.51, it/s 0.72
  1728. [0130 17:36:29 @multigpu.py:323] [p0574] step: count(270), step_time 1592.14, mean_step_time 1437.62, it/s 0.7
  1729. [0130 17:36:29 @multigpu.py:323] [p0115] step: count(253), step_time 1385.17, mean_step_time 1390.89, it/s 0.72
  1730. [0130 17:36:29 @multigpu.py:323] [p0576] step: count(277), step_time 1378.67, mean_step_time 1391.22, it/s 0.72
  1731. [0130 17:36:30 @multigpu.py:323] [p0574] step: count(271), step_time 1486.68, mean_step_time 1442.34, it/s 0.69
  1732. [0130 17:36:31 @multigpu.py:323] [p0115] step: count(254), step_time 1391.45, mean_step_time 1391.01, it/s 0.72
  1733. [0130 17:36:31 @multigpu.py:323] [p0576] step: count(278), step_time 1465.1, mean_step_time 1395.36, it/s 0.72
  1734. [0130 17:36:32 @multigpu.py:323] [p0574] step: count(272), step_time 1466.54, mean_step_time 1446.07, it/s 0.69
  1735. [0130 17:36:32 @multigpu.py:323] [p0115] step: count(255), step_time 1385.32, mean_step_time 1390.38, it/s 0.72
  1736. [0130 17:36:32 @multigpu.py:323] [p0576] step: count(279), step_time 1373.63, mean_step_time 1396.47, it/s 0.72
  1737. sending to address tcp://p0112:61216
  1738. ##### Sending to neptune: online_score : 0.108991423051 , 1.3 #####
  1739. [u'online', 1.3]
  1740. receiving
  1741. [0130 17:36:33 @multigpu.py:323] [p0574] step: count(273), step_time 1636.7, mean_step_time 1459.97, it/s 0.68
  1742. [0130 17:36:33 @multigpu.py:323] [p0115] step: count(256), step_time 1384.76, mean_step_time 1392.19, it/s 0.72
  1743. [0130 17:36:34 @multigpu.py:323] [p0576] step: count(280), step_time 1397.0, mean_step_time 1396.98, it/s 0.72
  1744. [0130 17:36:35 @multigpu.py:323] [p0115] step: count(257), step_time 1385.74, mean_step_time 1392.25, it/s 0.72
  1745. [0130 17:36:35 @multigpu.py:323] [p0576] step: count(281), step_time 1343.67, mean_step_time 1394.33, it/s 0.72
  1746. [0130 17:36:35 @multigpu.py:323] [p0574] step: count(274), step_time 1662.91, mean_step_time 1465.5, it/s 0.68
  1747. [0130 17:36:36 @multigpu.py:323] [p0115] step: count(258), step_time 1358.49, mean_step_time 1389.13, it/s 0.72
  1748. [0130 17:36:36 @multigpu.py:323] [p0576] step: count(282), step_time 1396.08, mean_step_time 1393.42, it/s 0.72
  1749. [0130 17:36:37 @multigpu.py:323] [p0574] step: count(275), step_time 1619.75, mean_step_time 1477.48, it/s 0.68
  1750. [0130 17:36:38 @multigpu.py:323] [p0115] step: count(259), step_time 1387.44, mean_step_time 1391.24, it/s 0.72
  1751. [0130 17:36:38 @multigpu.py:323] [p0576] step: count(283), step_time 1457.86, mean_step_time 1396.48, it/s 0.72
  1752. [0130 17:36:38 @multigpu.py:323] [p0574] step: count(276), step_time 1383.84, mean_step_time 1477.63, it/s 0.68
  1753. [0130 17:36:39 @multigpu.py:323] [p0115] step: count(260), step_time 1375.11, mean_step_time 1390.7, it/s 0.72
  1754. [0130 17:36:39 @multigpu.py:323] [p0576] step: count(284), step_time 1445.71, mean_step_time 1401.21, it/s 0.71
  1755. [0130 17:36:39 @multigpu.py:323] [p0574] step: count(277), step_time 1414.63, mean_step_time 1478.94, it/s 0.68
  1756. sending to address tcp://p0112:61216
  1757. ##### Sending to neptune: online_score : 0.110936076641 , 1.3 #####
  1758. [u'online', 1.3]
  1759. receiving
  1760. [0130 17:36:40 @multigpu.py:323] [p0115] step: count(261), step_time 1376.47, mean_step_time 1390.64, it/s 0.72
  1761. [0130 17:36:41 @multigpu.py:323] [p0576] step: count(285), step_time 1379.43, mean_step_time 1402.2, it/s 0.71
  1762. [0130 17:36:41 @multigpu.py:323] [p0574] step: count(278), step_time 1411.53, mean_step_time 1477.53, it/s 0.68
  1763. [0130 17:36:42 @multigpu.py:323] [p0115] step: count(262), step_time 1371.64, mean_step_time 1389.48, it/s 0.72
  1764. [0130 17:36:42 @multigpu.py:323] [p0576] step: count(286), step_time 1378.01, mean_step_time 1400.42, it/s 0.71
  1765. [0130 17:36:42 @multigpu.py:323] [p0574] step: count(279), step_time 1407.81, mean_step_time 1479.78, it/s 0.68
  1766. [0130 17:36:43 @multigpu.py:323] [p0115] step: count(263), step_time 1485.85, mean_step_time 1395.1, it/s 0.72
  1767. [0130 17:36:43 @multigpu.py:323] [p0576] step: count(287), step_time 1401.66, mean_step_time 1401.78, it/s 0.71
  1768. [0130 17:36:44 @multigpu.py:323] [p0574] step: count(280), step_time 1467.43, mean_step_time 1483.67, it/s 0.67
  1769. [0130 17:36:45 @multigpu.py:323] [p0115] step: count(264), step_time 1356.22, mean_step_time 1393.59, it/s 0.72
  1770. sending to address tcp://p0112:61216
  1771. ##### Sending to neptune: online_score : 0.112287428313 , 1.3 #####
  1772. [u'online', 1.3]
  1773. receiving
  1774. [0130 17:36:45 @multigpu.py:323] [p0576] step: count(288), step_time 1476.2, mean_step_time 1406.39, it/s 0.71
  1775. [0130 17:36:45 @multigpu.py:323] [p0574] step: count(281), step_time 1594.94, mean_step_time 1496.0, it/s 0.67
  1776. [0130 17:36:46 @multigpu.py:323] [p0115] step: count(265), step_time 1363.91, mean_step_time 1392.63, it/s 0.72
  1777. [0130 17:36:46 @multigpu.py:323] [p0576] step: count(289), step_time 1398.59, mean_step_time 1407.28, it/s 0.71
  1778. [0130 17:36:47 @multigpu.py:323] [p0574] step: count(282), step_time 1580.49, mean_step_time 1507.54, it/s 0.66
  1779. [0130 17:36:47 @multigpu.py:323] [p0115] step: count(266), step_time 1397.36, mean_step_time 1394.44, it/s 0.72
  1780. [0130 17:36:48 @multigpu.py:323] [p0576] step: count(290), step_time 1325.89, mean_step_time 1403.59, it/s 0.71
  1781. [0130 17:36:48 @multigpu.py:323] [p0574] step: count(283), step_time 1467.9, mean_step_time 1511.46, it/s 0.66
  1782. [0130 17:36:49 @multigpu.py:323] [p0115] step: count(267), step_time 1365.89, mean_step_time 1382.27, it/s 0.72
  1783. [0130 17:36:50 @multigpu.py:323] [p0115] step: count(268), step_time 1384.45, mean_step_time 1383.27, it/s 0.72
  1784. [0130 17:36:50 @multigpu.py:323] [p0574] step: count(284), step_time 1619.19, mean_step_time 1526.2, it/s 0.66
  1785. [0130 17:36:50 @multigpu.py:323] [p0576] step: count(291), step_time 2436.1, mean_step_time 1451.44, it/s 0.69
  1786. [0130 17:36:51 @multigpu.py:323] [p0576] step: count(292), step_time 1357.52, mean_step_time 1450.66, it/s 0.69
  1787. [0130 17:36:51 @multigpu.py:323] [p0115] step: count(269), step_time 1398.63, mean_step_time 1383.17, it/s 0.72
  1788. [0130 17:36:52 @multigpu.py:323] [p0574] step: count(285), step_time 1613.86, mean_step_time 1532.21, it/s 0.65
  1789. [0130 17:36:53 @multigpu.py:323] [p0115] step: count(270), step_time 1342.82, mean_step_time 1383.31, it/s 0.72
  1790. [0130 17:36:53 @multigpu.py:323] [p0576] step: count(293), step_time 1395.05, mean_step_time 1450.29, it/s 0.69
  1791. [0130 17:36:53 @multigpu.py:323] [p0574] step: count(286), step_time 1630.58, mean_step_time 1536.39, it/s 0.65
  1792. [0130 17:36:54 @multigpu.py:323] [p0115] step: count(271), step_time 1308.8, mean_step_time 1379.58, it/s 0.72
  1793. [0130 17:36:54 @multigpu.py:323] [p0576] step: count(294), step_time 1346.21, mean_step_time 1449.44, it/s 0.69
  1794. [0130 17:36:55 @multigpu.py:323] [p0574] step: count(287), step_time 1521.75, mean_step_time 1537.44, it/s 0.65
  1795. [0130 17:36:56 @multigpu.py:323] [p0115] step: count(272), step_time 1412.48, mean_step_time 1380.9, it/s 0.72
  1796. [0130 17:36:56 @multigpu.py:323] [p0576] step: count(295), step_time 1369.98, mean_step_time 1446.22, it/s 0.69
  1797. [0130 17:36:56 @multigpu.py:323] [p0574] step: count(288), step_time 1358.74, mean_step_time 1523.47, it/s 0.66
  1798. [0130 17:36:57 @multigpu.py:323] [p0115] step: count(273), step_time 1386.04, mean_step_time 1380.94, it/s 0.72
  1799. [0130 17:36:57 @multigpu.py:323] [p0576] step: count(296), step_time 1424.16, mean_step_time 1447.33, it/s 0.69
  1800. [0130 17:36:58 @multigpu.py:323] [p0574] step: count(289), step_time 1388.73, mean_step_time 1516.31, it/s 0.66
  1801. sending to address tcp://p0112:61216
  1802. ##### Sending to neptune: online_score : 0.115880426632 , 1.2 #####
  1803. [u'online', 1.2]
  1804. receiving
  1805. [0130 17:36:58 @multigpu.py:323] [p0115] step: count(274), step_time 1380.34, mean_step_time 1380.39, it/s 0.72
  1806. [0130 17:36:58 @multigpu.py:323] [p0576] step: count(297), step_time 1368.9, mean_step_time 1446.84, it/s 0.69
  1807. [0130 17:36:59 @multigpu.py:323] [p0574] step: count(290), step_time 1394.5, mean_step_time 1506.43, it/s 0.66
  1808. [0130 17:37:00 @multigpu.py:323] [p0115] step: count(275), step_time 1379.69, mean_step_time 1380.11, it/s 0.72
  1809. [0130 17:37:00 @multigpu.py:323] [p0576] step: count(298), step_time 1394.81, mean_step_time 1443.32, it/s 0.69
  1810. [0130 17:37:00 @multigpu.py:323] [p0574] step: count(291), step_time 1387.45, mean_step_time 1501.46, it/s 0.67
  1811. [0130 17:37:01 @multigpu.py:323] [p0115] step: count(276), step_time 1387.1, mean_step_time 1380.22, it/s 0.72
  1812. [0130 17:37:01 @multigpu.py:323] [p0576] step: count(299), step_time 1385.73, mean_step_time 1443.93, it/s 0.69
  1813. [0130 17:37:02 @multigpu.py:323] [p0574] step: count(292), step_time 1369.93, mean_step_time 1496.63, it/s 0.67
  1814. [0130 17:37:02 @multigpu.py:323] [p0115] step: count(277), step_time 1383.91, mean_step_time 1380.13, it/s 0.72
  1815. [0130 17:37:03 @multigpu.py:323] [p0576] step: count(300), step_time 1462.3, mean_step_time 1447.19, it/s 0.69
  1816. sending debugging info...
  1817. sending to address tcp://p0112:61216
  1818. ##### Sending to neptune: mean_delay : 0.117210962176 , 0.0 #####
  1819. sending to address tcp://p0112:61216
  1820. ##### Sending to neptune: max_delay : 0.117210962176 , -0.0 #####
  1821. ##### Sending to neptune: min_delay : 0.117210962176 , -0.0 #####
  1822. [u'delays', [0.0, -0.0, -0.0]]
  1823. receiving
  1824. ##### Sending to neptune: cost : 0.117211548554 , -0.0115024279803 #####
  1825. sending to address tcp://p0112:61216
  1826. ##### Sending to neptune: policy_loss : 0.117211548554 , -0.317066699266 #####
  1827. ##### Sending to neptune: xentropy_loss : 0.117211548554 , -2.28820109367 #####
  1828. ##### Sending to neptune: value_loss : 0.117211548554 , 1.13295674324 #####
  1829. ##### Sending to neptune: advantage : 0.117211548554 , 0.00142617744859 #####
  1830. ##### Sending to neptune: pred_reward : 0.117211548554 , 0.396612882614 #####
  1831. ##### Sending to neptune: max_logit : 0.117211548554 , 0.193715110421 #####
  1832. [u'loss', -0.011502427980303764, -0.3170666992664337, -2.288201093673706, 1.1329567432403564, 0.00142617744859308, 0.39661288261413574, 0.19371511042118073]
  1833. receiving
  1834. ##### Sending to neptune: active_relus : 0.117212024993 , 8889058.0 #####
  1835. ##### Sending to neptune: dp_per_s : 0.117212024993 , 91.0211410016 #####
  1836. [u'other', 8889058.0, 91.02114100159582]
  1837. receiving
  1838. [0130 17:37:03 @multigpu.py:323] [p0574] step: count(293), step_time 1378.74, mean_step_time 1483.74, it/s 0.67
  1839. [0130 17:37:04 @multigpu.py:323] [p0115] step: count(278), step_time 1361.57, mean_step_time 1380.29, it/s 0.72
  1840. sending to address tcp://p0112:61216
  1841. ##### Sending to neptune: online_score : 0.117604462769 , 1.4 #####
  1842. [u'online', 1.4]
  1843. receiving
  1844. [0130 17:37:04 @multigpu.py:323] [p0576] step: count(301), step_time 1422.54, mean_step_time 1451.14, it/s 0.69
  1845. [0130 17:37:05 @multigpu.py:323] [p0574] step: count(294), step_time 1524.22, mean_step_time 1476.8, it/s 0.68
  1846. [0130 17:37:05 @multigpu.py:323] [p0115] step: count(279), step_time 1390.55, mean_step_time 1380.44, it/s 0.72
  1847. [0130 17:37:05 @multigpu.py:323] [p0576] step: count(302), step_time 1373.9, mean_step_time 1450.03, it/s 0.69
  1848. [0130 17:37:06 @multigpu.py:323] [p0574] step: count(295), step_time 1396.08, mean_step_time 1465.62, it/s 0.68
  1849. [0130 17:37:07 @multigpu.py:323] [p0115] step: count(280), step_time 1366.44, mean_step_time 1380.01, it/s 0.72
  1850. [0130 17:37:07 @multigpu.py:323] [p0576] step: count(303), step_time 1444.09, mean_step_time 1449.34, it/s 0.69
  1851. sending to address tcp://p0112:61216
  1852. ##### Sending to neptune: online_score : 0.118510236078 , 1.2 #####
  1853. [u'online', 1.2]
  1854. receiving
  1855. [0130 17:37:08 @multigpu.py:323] [p0574] step: count(296), step_time 1630.41, mean_step_time 1477.95, it/s 0.68
  1856. [0130 17:37:08 @multigpu.py:323] [p0115] step: count(281), step_time 1372.06, mean_step_time 1379.79, it/s 0.72
  1857. [0130 17:37:08 @multigpu.py:323] [p0576] step: count(304), step_time 1350.36, mean_step_time 1444.57, it/s 0.69
  1858. [0130 17:37:09 @multigpu.py:323] [p0574] step: count(297), step_time 1582.59, mean_step_time 1486.34, it/s 0.67
  1859. [0130 17:37:09 @multigpu.py:323] [p0115] step: count(282), step_time 1378.39, mean_step_time 1380.13, it/s 0.72
  1860. [0130 17:37:10 @multigpu.py:323] [p0576] step: count(305), step_time 1426.73, mean_step_time 1446.94, it/s 0.69
  1861. [0130 17:37:11 @multigpu.py:323] [p0574] step: count(298), step_time 1373.21, mean_step_time 1484.43, it/s 0.67
  1862. [0130 17:37:11 @multigpu.py:323] [p0115] step: count(283), step_time 1393.71, mean_step_time 1375.52, it/s 0.73
  1863. [0130 17:37:11 @multigpu.py:323] [p0576] step: count(306), step_time 1448.18, mean_step_time 1450.44, it/s 0.69
  1864. [0130 17:37:12 @multigpu.py:323] [p0574] step: count(299), step_time 1393.04, mean_step_time 1483.69, it/s 0.67
  1865. [0130 17:37:12 @multigpu.py:323] [p0115] step: count(284), step_time 1408.19, mean_step_time 1378.12, it/s 0.73
  1866. [0130 17:37:12 @multigpu.py:323] [p0576] step: count(307), step_time 1367.6, mean_step_time 1448.74, it/s 0.69
  1867. [0130 17:37:13 @multigpu.py:323] [p0574] step: count(300), step_time 1382.7, mean_step_time 1479.45, it/s 0.68
  1868. sending debugging info...
  1869. sending to address tcp://p0112:61216
  1870. ##### Sending to neptune: mean_delay : 0.120217489137 , 0.0 #####
  1871. ##### Sending to neptune: max_delay : 0.120217489137 , -0.0 #####
  1872. sending to address tcp://p0112:61216
  1873. ##### Sending to neptune: min_delay : 0.120217489137 , -0.0 #####
  1874. [u'delays', [0.0, -0.0, -0.0]]
  1875. receiving
  1876. ##### Sending to neptune: cost : 0.120218040546 , -0.0103861913085 #####
  1877. ##### Sending to neptune: policy_loss : 0.120218040546 , -0.244617804885 #####
  1878. sending to address tcp://p0112:61216
  1879. ##### Sending to neptune: xentropy_loss : 0.120218040546 , -2.28823947906 #####
  1880. ##### Sending to neptune: value_loss : 0.120218040546 , 1.20342481136 #####
  1881. ##### Sending to neptune: advantage : 0.120218040546 , 0.00103269040119 #####
  1882. ##### Sending to neptune: pred_reward : 0.120218040546 , 0.395009964705 #####
  1883. ##### Sending to neptune: max_logit : 0.120218040546 , 0.193438783288 #####
  1884. [u'loss', -0.010386191308498383, -0.24461780488491058, -2.2882394790649414, 1.2034248113632202, 0.0010326904011890292, 0.39500996470451355, 0.19343878328800201]
  1885. receiving
  1886. ##### Sending to neptune: active_relus : 0.120218533013 , 8887224.41 #####
  1887. ##### Sending to neptune: dp_per_s : 0.120218533013 , 89.5399337807 #####
  1888. [u'other', 8887224.41, 89.53993378070659]
  1889. receiving
  1890. [0130 17:37:13 @multigpu.py:323] [p0115] step: count(285), step_time 1365.26, mean_step_time 1378.19, it/s 0.73
  1891. [0130 17:37:14 @multigpu.py:323] [p0576] step: count(308), step_time 1443.34, mean_step_time 1447.1, it/s 0.69
  1892. [0130 17:37:15 @multigpu.py:323] [p0574] step: count(301), step_time 1380.97, mean_step_time 1468.75, it/s 0.68
  1893. [0130 17:37:15 @multigpu.py:323] [p0115] step: count(286), step_time 1412.84, mean_step_time 1378.96, it/s 0.73
  1894. [0130 17:37:15 @multigpu.py:323] [p0576] step: count(309), step_time 1374.83, mean_step_time 1445.91, it/s 0.69
  1895. [0130 17:37:16 @multigpu.py:323] [p0574] step: count(302), step_time 1371.75, mean_step_time 1458.32, it/s 0.69
  1896. [0130 17:37:16 @multigpu.py:323] [p0115] step: count(287), step_time 1411.56, mean_step_time 1381.24, it/s 0.72
  1897. [0130 17:37:17 @multigpu.py:323] [p0576] step: count(310), step_time 1376.81, mean_step_time 1448.46, it/s 0.69
  1898. [0130 17:37:18 @multigpu.py:323] [p0574] step: count(303), step_time 1409.68, mean_step_time 1455.41, it/s 0.69
  1899. [0130 17:37:19 @multigpu.py:323] [p0574] step: count(304), step_time 1364.59, mean_step_time 1442.68, it/s 0.69
  1900. [0130 17:37:19 @multigpu.py:323] [p0115] step: count(288), step_time 2615.34, mean_step_time 1442.79, it/s 0.69
  1901. [0130 17:37:19 @multigpu.py:323] [p0576] step: count(311), step_time 2319.64, mean_step_time 1442.63, it/s 0.69
  1902. [0130 17:37:20 @multigpu.py:323] [p0115] step: count(289), step_time 1383.5, mean_step_time 1442.03, it/s 0.69
  1903. [0130 17:37:20 @multigpu.py:323] [p0576] step: count(312), step_time 1439.56, mean_step_time 1446.74, it/s 0.69
  1904. [0130 17:37:20 @multigpu.py:323] [p0574] step: count(305), step_time 1487.43, mean_step_time 1436.35, it/s 0.7
  1905. [0130 17:37:22 @multigpu.py:323] [p0115] step: count(290), step_time 1381.82, mean_step_time 1443.98, it/s 0.69
  1906. [0130 17:37:22 @multigpu.py:323] [p0576] step: count(313), step_time 1420.89, mean_step_time 1448.03, it/s 0.69
  1907. [0130 17:37:22 @multigpu.py:323] [p0574] step: count(306), step_time 1430.76, mean_step_time 1426.36, it/s 0.7
  1908. [0130 17:37:23 @multigpu.py:323] [p0115] step: count(291), step_time 1380.89, mean_step_time 1447.58, it/s 0.69
  1909. [0130 17:37:23 @multigpu.py:323] [p0576] step: count(314), step_time 1397.24, mean_step_time 1450.58, it/s 0.69
  1910. [0130 17:37:23 @multigpu.py:323] [p0574] step: count(307), step_time 1395.27, mean_step_time 1420.04, it/s 0.7
  1911. [0130 17:37:24 @multigpu.py:323] [p0115] step: count(292), step_time 1402.06, mean_step_time 1447.06, it/s 0.69
  1912. [0130 17:37:25 @multigpu.py:323] [p0576] step: count(315), step_time 1441.95, mean_step_time 1454.18, it/s 0.69
  1913. sending to address tcp://p0112:61216
  1914. ##### Sending to neptune: online_score : 0.123340421054 , 1.1 #####
  1915. [u'online', 1.1]
  1916. ##### Sending to neptune: active_workers : 0.123340500792 , 3 #####
  1917. receiving
  1918. [0130 17:37:25 @multigpu.py:323] [p0574] step: count(308), step_time 1530.98, mean_step_time 1428.65, it/s 0.7
  1919. [0130 17:37:26 @multigpu.py:323] [p0115] step: count(293), step_time 1416.82, mean_step_time 1448.6, it/s 0.69
  1920. [0130 17:37:26 @multigpu.py:323] [p0576] step: count(316), step_time 1384.3, mean_step_time 1452.18, it/s 0.69
  1921. [0130 17:37:26 @multigpu.py:323] [p0574] step: count(309), step_time 1488.91, mean_step_time 1433.66, it/s 0.7
  1922. [0130 17:37:27 @multigpu.py:323] [p0115] step: count(294), step_time 1349.49, mean_step_time 1447.06, it/s 0.69
  1923. [0130 17:37:27 @multigpu.py:323] [p0576] step: count(317), step_time 1357.59, mean_step_time 1451.62, it/s 0.69
  1924. [0130 17:37:28 @multigpu.py:323] [p0574] step: count(310), step_time 1442.14, mean_step_time 1436.04, it/s 0.7
  1925. [0130 17:37:29 @multigpu.py:323] [p0115] step: count(295), step_time 1392.08, mean_step_time 1447.68, it/s 0.69
  1926. [0130 17:37:29 @multigpu.py:323] [p0576] step: count(318), step_time 1462.51, mean_step_time 1455.0, it/s 0.69
  1927. [0130 17:37:29 @multigpu.py:323] [p0574] step: count(311), step_time 1438.97, mean_step_time 1438.62, it/s 0.7
  1928. [0130 17:37:30 @multigpu.py:323] [p0115] step: count(296), step_time 1361.63, mean_step_time 1446.41, it/s 0.69
  1929. [0130 17:37:30 @multigpu.py:323] [p0576] step: count(319), step_time 1387.51, mean_step_time 1455.09, it/s 0.69
  1930. [0130 17:37:31 @multigpu.py:323] [p0574] step: count(312), step_time 1377.31, mean_step_time 1438.99, it/s 0.69
  1931. [0130 17:37:31 @multigpu.py:323] [p0115] step: count(297), step_time 1405.26, mean_step_time 1447.47, it/s 0.69
  1932. [0130 17:37:32 @multigpu.py:323] [p0576] step: count(320), step_time 1379.68, mean_step_time 1450.96, it/s 0.69
  1933. [0130 17:37:32 @multigpu.py:323] [p0574] step: count(313), step_time 1417.67, mean_step_time 1440.93, it/s 0.69
  1934. [0130 17:37:33 @multigpu.py:323] [p0115] step: count(298), step_time 1410.03, mean_step_time 1449.9, it/s 0.69
  1935. [0130 17:37:33 @multigpu.py:323] [p0576] step: count(321), step_time 1376.52, mean_step_time 1448.66, it/s 0.69
  1936. [0130 17:37:33 @multigpu.py:323] [p0574] step: count(314), step_time 1339.04, mean_step_time 1431.68, it/s 0.7
  1937. [0130 17:37:34 @multigpu.py:323] [p0115] step: count(299), step_time 1389.03, mean_step_time 1449.82, it/s 0.69
  1938. [0130 17:37:34 @multigpu.py:323] [p0576] step: count(322), step_time 1386.44, mean_step_time 1449.29, it/s 0.69
  1939. [0130 17:37:35 @multigpu.py:323] [p0574] step: count(315), step_time 1398.39, mean_step_time 1431.79, it/s 0.7
  1940. [0130 17:37:36 @multigpu.py:323] [p0115] step: count(300), step_time 1443.03, mean_step_time 1453.65, it/s 0.69
  1941. sending debugging info...
  1942. sending to address tcp://p0112:61216
  1943. ##### Sending to neptune: mean_delay : 0.126398886376 , 0.0 #####
  1944. sending to address tcp://p0112:61216
  1945. ##### Sending to neptune: max_delay : 0.126398886376 , -0.0 #####
  1946. ##### Sending to neptune: min_delay : 0.126398886376 , -0.0 #####
  1947. [u'delays', [0.0, -0.0, -0.0]]
  1948. receiving
  1949. ##### Sending to neptune: cost : 0.126399410831 , -0.0113957431167 #####
  1950. sending to address tcp://p0112:61216
  1951. ##### Sending to neptune: policy_loss : 0.126399410831 , -0.226604878902 #####
  1952. ##### Sending to neptune: xentropy_loss : 0.126399410831 , -2.28844261169 #####
  1953. ##### Sending to neptune: value_loss : 0.126399410831 , 1.05639243126 #####
  1954. ##### Sending to neptune: advantage : 0.126399410831 , 0.00103650894016 #####
  1955. ##### Sending to neptune: pred_reward : 0.126399410831 , 0.386131823063 #####
  1956. ##### Sending to neptune: max_logit : 0.126399410831 , 0.19203697145 #####
  1957. [u'loss', -0.011395743116736412, -0.2266048789024353, -2.288442611694336, 1.0563924312591553, 0.0010365089401602745, 0.38613182306289673, 0.192036971449852]
  1958. receiving
  1959. ##### Sending to neptune: active_relus : 0.126399871376 , 8856618.33 #####
  1960. ##### Sending to neptune: dp_per_s : 0.126399871376 , 91.271358836 #####
  1961. [u'other', 8856618.33, 91.27135883601044]
  1962. receiving
  1963. [0130 17:37:36 @multigpu.py:323] [p0576] step: count(323), step_time 1405.49, mean_step_time 1447.36, it/s 0.69
  1964. [0130 17:37:36 @multigpu.py:323] [p0574] step: count(316), step_time 1407.92, mean_step_time 1420.67, it/s 0.7
  1965. [0130 17:37:37 @multigpu.py:323] [p0115] step: count(301), step_time 1387.53, mean_step_time 1454.42, it/s 0.69
  1966. [0130 17:37:37 @multigpu.py:323] [p0576] step: count(324), step_time 1378.84, mean_step_time 1448.78, it/s 0.69
  1967. [0130 17:37:37 @multigpu.py:323] [p0574] step: count(317), step_time 1393.47, mean_step_time 1411.21, it/s 0.71
  1968. sending to address tcp://p0112:61216
  1969. ##### Sending to neptune: online_score : 0.12702402691 , 0.9 #####
  1970. [u'online', 0.9]
  1971. receiving
  1972. sending to address tcp://p0112:61216
  1973. ##### Sending to neptune: online_score : 0.127058637738 , 1.1 #####
  1974. [u'online', 1.1]
  1975. receiving
  1976. [0130 17:37:38 @multigpu.py:323] [p0115] step: count(302), step_time 1390.81, mean_step_time 1455.04, it/s 0.69
  1977. [0130 17:37:39 @multigpu.py:323] [p0576] step: count(325), step_time 1451.99, mean_step_time 1450.05, it/s 0.69
  1978. [0130 17:37:39 @multigpu.py:323] [p0574] step: count(318), step_time 1425.59, mean_step_time 1413.83, it/s 0.71
  1979. [0130 17:37:40 @multigpu.py:323] [p0115] step: count(303), step_time 1408.57, mean_step_time 1455.79, it/s 0.69
  1980. [0130 17:37:40 @multigpu.py:323] [p0576] step: count(326), step_time 1362.75, mean_step_time 1445.77, it/s 0.69
  1981. [0130 17:37:40 @multigpu.py:323] [p0574] step: count(319), step_time 1372.05, mean_step_time 1412.78, it/s 0.71
  1982. [0130 17:37:41 @multigpu.py:323] [p0115] step: count(304), step_time 1345.33, mean_step_time 1452.64, it/s 0.69
  1983. [0130 17:37:41 @multigpu.py:323] [p0576] step: count(327), step_time 1445.58, mean_step_time 1449.67, it/s 0.69
  1984. [0130 17:37:42 @multigpu.py:323] [p0574] step: count(320), step_time 1361.26, mean_step_time 1411.71, it/s 0.71
  1985. [0130 17:37:43 @multigpu.py:323] [p0115] step: count(305), step_time 1437.42, mean_step_time 1456.25, it/s 0.69
  1986. [0130 17:37:43 @multigpu.py:323] [p0576] step: count(328), step_time 1383.55, mean_step_time 1446.68, it/s 0.69
  1987. [0130 17:37:43 @multigpu.py:323] [p0574] step: count(321), step_time 1381.44, mean_step_time 1411.73, it/s 0.71
  1988. [0130 17:37:44 @multigpu.py:323] [p0115] step: count(306), step_time 1359.95, mean_step_time 1453.61, it/s 0.69
  1989. [0130 17:37:44 @multigpu.py:323] [p0576] step: count(329), step_time 1418.87, mean_step_time 1448.89, it/s 0.69
  1990. [0130 17:37:44 @multigpu.py:323] [p0574] step: count(322), step_time 1415.14, mean_step_time 1413.9, it/s 0.71
  1991. [0130 17:37:45 @multigpu.py:323] [p0115] step: count(307), step_time 1396.46, mean_step_time 1452.85, it/s 0.69
  1992. [0130 17:37:46 @multigpu.py:323] [p0576] step: count(330), step_time 1455.6, mean_step_time 1452.83, it/s 0.69
  1993. [0130 17:37:46 @multigpu.py:323] [p0574] step: count(323), step_time 1414.92, mean_step_time 1414.16, it/s 0.71
  1994. [0130 17:37:47 @multigpu.py:323] [p0574] step: count(324), step_time 1423.54, mean_step_time 1417.11, it/s 0.71
  1995. [0130 17:37:47 @multigpu.py:323] [p0115] step: count(308), step_time 1900.99, mean_step_time 1417.14, it/s 0.71
  1996. [0130 17:37:47 @multigpu.py:323] [p0576] step: count(331), step_time 1604.34, mean_step_time 1417.06, it/s 0.71
  1997. [0130 17:37:49 @multigpu.py:323] [p0115] step: count(309), step_time 1365.43, mean_step_time 1416.23, it/s 0.71
  1998. [0130 17:37:49 @multigpu.py:323] [p0574] step: count(325), step_time 1368.14, mean_step_time 1411.15, it/s 0.71
  1999. [0130 17:37:49 @multigpu.py:323] [p0576] step: count(332), step_time 1407.4, mean_step_time 1415.45, it/s 0.71
  2000. [0130 17:37:50 @multigpu.py:323] [p0574] step: count(326), step_time 1396.46, mean_step_time 1409.43, it/s 0.71
  2001. [0130 17:37:50 @multigpu.py:323] [p0115] step: count(310), step_time 1424.3, mean_step_time 1418.36, it/s 0.71
  2002. [0130 17:37:50 @multigpu.py:323] [p0576] step: count(333), step_time 1397.06, mean_step_time 1414.26, it/s 0.71
  2003. [0130 17:37:51 @multigpu.py:323] [p0574] step: count(327), step_time 1358.35, mean_step_time 1407.58, it/s 0.71
  2004. [0130 17:37:51 @multigpu.py:323] [p0115] step: count(311), step_time 1356.57, mean_step_time 1417.14, it/s 0.71
  2005. [0130 17:37:51 @multigpu.py:323] [p0576] step: count(334), step_time 1354.64, mean_step_time 1412.13, it/s 0.71
  2006. [0130 17:37:53 @multigpu.py:323] [p0574] step: count(328), step_time 1380.41, mean_step_time 1400.06, it/s 0.71
  2007. [0130 17:37:53 @multigpu.py:323] [p0115] step: count(312), step_time 1374.11, mean_step_time 1415.74, it/s 0.71
  2008. [0130 17:37:53 @multigpu.py:323] [p0576] step: count(335), step_time 1438.54, mean_step_time 1411.96, it/s 0.71
  2009. [0130 17:37:54 @multigpu.py:323] [p0574] step: count(329), step_time 1412.16, mean_step_time 1396.22, it/s 0.72
  2010. [0130 17:37:54 @multigpu.py:323] [p0115] step: count(313), step_time 1416.59, mean_step_time 1415.73, it/s 0.71
  2011. [0130 17:37:54 @multigpu.py:323] [p0576] step: count(336), step_time 1447.87, mean_step_time 1415.14, it/s 0.71
  2012. sending to address tcp://p0112:61216
  2013. ##### Sending to neptune: online_score : 0.131745723883 , 2.1 #####
  2014. [u'online', 2.1]
  2015. receiving
  2016. [0130 17:37:56 @multigpu.py:323] [p0574] step: count(330), step_time 1360.8, mean_step_time 1392.15, it/s 0.72
  2017. [0130 17:37:56 @multigpu.py:323] [p0115] step: count(314), step_time 1370.28, mean_step_time 1416.77, it/s 0.71
  2018. [0130 17:37:56 @multigpu.py:323] [p0576] step: count(337), step_time 1417.26, mean_step_time 1418.12, it/s 0.71
  2019. [0130 17:37:57 @multigpu.py:323] [p0574] step: count(331), step_time 1391.51, mean_step_time 1389.78, it/s 0.72
  2020. [0130 17:37:57 @multigpu.py:323] [p0115] step: count(315), step_time 1387.7, mean_step_time 1416.55, it/s 0.71
  2021. [0130 17:37:57 @multigpu.py:323] [p0576] step: count(338), step_time 1444.41, mean_step_time 1417.22, it/s 0.71
  2022. [0130 17:37:58 @multigpu.py:323] [p0574] step: count(332), step_time 1374.04, mean_step_time 1389.62, it/s 0.72
  2023. [0130 17:37:58 @multigpu.py:323] [p0115] step: count(316), step_time 1388.04, mean_step_time 1417.87, it/s 0.71
  2024. [0130 17:37:59 @multigpu.py:323] [p0576] step: count(339), step_time 1442.65, mean_step_time 1419.97, it/s 0.7
  2025. sending to address tcp://p0112:61216
  2026. ##### Sending to neptune: online_score : 0.132890012993 , 1.4 #####
  2027. [u'online', 1.4]
  2028. receiving
  2029. [0130 17:38:00 @multigpu.py:323] [p0574] step: count(333), step_time 1416.93, mean_step_time 1389.58, it/s 0.72
  2030. [0130 17:38:00 @multigpu.py:323] [p0115] step: count(317), step_time 1383.89, mean_step_time 1416.8, it/s 0.71
  2031. [0130 17:38:00 @multigpu.py:323] [p0576] step: count(340), step_time 1449.19, mean_step_time 1423.45, it/s 0.7
  2032. [0130 17:38:01 @multigpu.py:323] [p0574] step: count(334), step_time 1360.92, mean_step_time 1390.67, it/s 0.72
  2033. [0130 17:38:01 @multigpu.py:323] [p0115] step: count(318), step_time 1384.03, mean_step_time 1415.5, it/s 0.71
  2034. [0130 17:38:01 @multigpu.py:323] [p0576] step: count(341), step_time 1400.62, mean_step_time 1424.65, it/s 0.7
  2035. sending to address tcp://p0112:61216
  2036. ##### Sending to neptune: online_score : 0.133605820537 , 1.9 #####
  2037. [u'online', 1.9]
  2038. receiving
  2039. [0130 17:38:02 @multigpu.py:323] [p0115] step: count(319), step_time 1350.75, mean_step_time 1413.59, it/s 0.71
  2040. [0130 17:38:03 @multigpu.py:323] [p0574] step: count(335), step_time 1407.67, mean_step_time 1391.14, it/s 0.72
  2041. [0130 17:38:03 @multigpu.py:323] [p0576] step: count(342), step_time 1447.8, mean_step_time 1427.72, it/s 0.7
  2042. [0130 17:38:04 @multigpu.py:323] [p0574] step: count(336), step_time 1384.51, mean_step_time 1389.97, it/s 0.72
  2043. [0130 17:38:04 @multigpu.py:323] [p0115] step: count(320), step_time 1447.59, mean_step_time 1413.82, it/s 0.71
  2044. [0130 17:38:04 @multigpu.py:323] [p0576] step: count(343), step_time 1418.6, mean_step_time 1428.38, it/s 0.7
  2045. [0130 17:38:05 @multigpu.py:323] [p0574] step: count(337), step_time 1368.34, mean_step_time 1388.71, it/s 0.72
  2046. [0130 17:38:05 @multigpu.py:323] [p0115] step: count(321), step_time 1407.05, mean_step_time 1414.79, it/s 0.71
  2047. [0130 17:38:06 @multigpu.py:323] [p0576] step: count(344), step_time 1410.33, mean_step_time 1429.95, it/s 0.7
  2048. [0130 17:38:07 @multigpu.py:323] [p0574] step: count(338), step_time 1393.57, mean_step_time 1387.11, it/s 0.72
  2049. [0130 17:38:07 @multigpu.py:323] [p0115] step: count(322), step_time 1391.66, mean_step_time 1414.83, it/s 0.71
  2050. [0130 17:38:07 @multigpu.py:323] [p0576] step: count(345), step_time 1421.3, mean_step_time 1428.42, it/s 0.7
  2051. [0130 17:38:08 @multigpu.py:323] [p0574] step: count(339), step_time 1349.97, mean_step_time 1386.0, it/s 0.72
  2052. [0130 17:38:08 @multigpu.py:323] [p0115] step: count(323), step_time 1379.38, mean_step_time 1413.38, it/s 0.71
  2053. [0130 17:38:09 @multigpu.py:323] [p0576] step: count(346), step_time 1406.67, mean_step_time 1430.61, it/s 0.7
  2054. [0130 17:38:09 @multigpu.py:323] [p0574] step: count(340), step_time 1452.74, mean_step_time 1390.58, it/s 0.72
  2055. [0130 17:38:09 @multigpu.py:323] [p0115] step: count(324), step_time 1370.54, mean_step_time 1414.64, it/s 0.71
  2056. [0130 17:38:10 @multigpu.py:323] [p0576] step: count(347), step_time 1397.1, mean_step_time 1428.19, it/s 0.7
  2057. [0130 17:38:11 @multigpu.py:323] [p0115] step: count(325), step_time 1345.95, mean_step_time 1410.06, it/s 0.71
  2058. [0130 17:38:11 @multigpu.py:323] [p0574] step: count(341), step_time 1485.56, mean_step_time 1395.78, it/s 0.72
  2059. [0130 17:38:11 @multigpu.py:323] [p0576] step: count(348), step_time 1434.95, mean_step_time 1430.76, it/s 0.7
  2060. [0130 17:38:12 @multigpu.py:323] [p0115] step: count(326), step_time 1372.42, mean_step_time 1410.69, it/s 0.71
  2061. [0130 17:38:12 @multigpu.py:323] [p0574] step: count(342), step_time 1347.73, mean_step_time 1392.41, it/s 0.72
  2062. [0130 17:38:13 @multigpu.py:323] [p0576] step: count(349), step_time 1477.29, mean_step_time 1433.68, it/s 0.7
  2063. [0130 17:38:14 @multigpu.py:323] [p0115] step: count(327), step_time 1419.62, mean_step_time 1411.84, it/s 0.71
  2064. [0130 17:38:14 @multigpu.py:323] [p0574] step: count(343), step_time 1402.41, mean_step_time 1391.79, it/s 0.72
  2065. [0130 17:38:14 @multigpu.py:323] [p0576] step: count(350), step_time 1375.61, mean_step_time 1429.68, it/s 0.7
  2066. [0130 17:38:16 @multigpu.py:323] [p0576] step: count(351), step_time 1423.33, mean_step_time 1420.63, it/s 0.7
  2067. [0130 17:38:16 @multigpu.py:323] [p0574] step: count(344), step_time 2003.9, mean_step_time 1420.81, it/s 0.7
  2068. [0130 17:38:16 @multigpu.py:323] [p0115] step: count(328), step_time 2080.96, mean_step_time 1420.84, it/s 0.7
  2069. [0130 17:38:17 @multigpu.py:323] [p0115] step: count(329), step_time 1369.82, mean_step_time 1421.06, it/s 0.7
  2070. [0130 17:38:17 @multigpu.py:323] [p0574] step: count(345), step_time 1407.39, mean_step_time 1422.77, it/s 0.7
  2071. [0130 17:38:17 @multigpu.py:323] [p0576] step: count(352), step_time 1426.08, mean_step_time 1421.56, it/s 0.7
  2072. [0130 17:38:18 @multigpu.py:323] [p0574] step: count(346), step_time 1367.47, mean_step_time 1421.32, it/s 0.7
  2073. [0130 17:38:18 @multigpu.py:323] [p0115] step: count(330), step_time 1413.16, mean_step_time 1420.5, it/s 0.7
  2074. [0130 17:38:19 @multigpu.py:323] [p0576] step: count(353), step_time 1400.77, mean_step_time 1421.75, it/s 0.7
  2075. sending to address tcp://p0112:61216
  2076. ##### Sending to neptune: online_score : 0.138402113583 , 0.7 #####
  2077. [u'online', 0.7]
  2078. receiving
  2079. [0130 17:38:20 @multigpu.py:323] [p0574] step: count(347), step_time 1365.51, mean_step_time 1421.68, it/s 0.7
  2080. [0130 17:38:20 @multigpu.py:323] [p0115] step: count(331), step_time 1395.58, mean_step_time 1422.46, it/s 0.7
  2081. [0130 17:38:20 @multigpu.py:323] [p0576] step: count(354), step_time 1377.34, mean_step_time 1422.89, it/s 0.7
  2082. [0130 17:38:21 @multigpu.py:323] [p0574] step: count(348), step_time 1433.04, mean_step_time 1424.31, it/s 0.7
  2083. [0130 17:38:21 @multigpu.py:323] [p0115] step: count(332), step_time 1404.26, mean_step_time 1423.96, it/s 0.7
  2084. [0130 17:38:21 @multigpu.py:323] [p0576] step: count(355), step_time 1390.11, mean_step_time 1420.46, it/s 0.7
  2085. [0130 17:38:23 @multigpu.py:323] [p0115] step: count(333), step_time 1366.41, mean_step_time 1421.45, it/s 0.7
  2086. [0130 17:38:23 @multigpu.py:323] [p0574] step: count(349), step_time 1398.66, mean_step_time 1423.63, it/s 0.7
  2087. [0130 17:38:23 @multigpu.py:323] [p0576] step: count(356), step_time 1410.44, mean_step_time 1418.59, it/s 0.7
  2088. [0130 17:38:24 @multigpu.py:323] [p0574] step: count(350), step_time 1343.51, mean_step_time 1422.77, it/s 0.7
  2089. [0130 17:38:24 @multigpu.py:323] [p0115] step: count(334), step_time 1460.58, mean_step_time 1425.97, it/s 0.7
  2090. [0130 17:38:24 @multigpu.py:323] [p0576] step: count(357), step_time 1432.73, mean_step_time 1419.37, it/s 0.7
  2091. sending to address tcp://p0112:61216
  2092. ##### Sending to neptune: online_score : 0.139930734701 , 0.2 #####
  2093. [u'online', 0.2]
  2094. receiving
  2095. [0130 17:38:25 @multigpu.py:323] [p0574] step: count(351), step_time 1363.59, mean_step_time 1421.37, it/s 0.7
  2096. [0130 17:38:25 @multigpu.py:323] [p0115] step: count(335), step_time 1355.52, mean_step_time 1424.36, it/s 0.7
  2097. [0130 17:38:26 @multigpu.py:323] [p0576] step: count(358), step_time 1470.23, mean_step_time 1420.66, it/s 0.7
  2098. [0130 17:38:27 @multigpu.py:323] [p0574] step: count(352), step_time 1403.23, mean_step_time 1422.83, it/s 0.7
  2099. [0130 17:38:27 @multigpu.py:323] [p0115] step: count(336), step_time 1392.58, mean_step_time 1424.59, it/s 0.7
  2100. [0130 17:38:27 @multigpu.py:323] [p0576] step: count(359), step_time 1365.14, mean_step_time 1416.78, it/s 0.71
  2101. sending to address tcp://p0112:61216
  2102. ##### Sending to neptune: online_score : 0.140695282751 , 1.5 #####
  2103. [u'online', 1.5]
  2104. ##### Sending to neptune: active_workers : 0.140695316063 , 3 #####
  2105. receiving
  2106. [0130 17:38:28 @multigpu.py:323] [p0574] step: count(353), step_time 1368.95, mean_step_time 1420.43, it/s 0.7
  2107. [0130 17:38:28 @multigpu.py:323] [p0115] step: count(337), step_time 1372.64, mean_step_time 1424.03, it/s 0.7
  2108. [0130 17:38:28 @multigpu.py:323] [p0576] step: count(360), step_time 1449.26, mean_step_time 1416.78, it/s 0.71
  2109. [0130 17:38:30 @multigpu.py:323] [p0574] step: count(354), step_time 1349.47, mean_step_time 1419.86, it/s 0.7
  2110. [0130 17:38:30 @multigpu.py:323] [p0115] step: count(338), step_time 1391.57, mean_step_time 1424.4, it/s 0.7
  2111. [0130 17:38:30 @multigpu.py:323] [p0576] step: count(361), step_time 1403.23, mean_step_time 1416.92, it/s 0.71
  2112. [0130 17:38:31 @multigpu.py:323] [p0574] step: count(355), step_time 1500.65, mean_step_time 1424.51, it/s 0.7
  2113. [0130 17:38:31 @multigpu.py:323] [p0115] step: count(339), step_time 1386.29, mean_step_time 1426.18, it/s 0.7
  2114. [0130 17:38:31 @multigpu.py:323] [p0576] step: count(362), step_time 1423.1, mean_step_time 1415.68, it/s 0.71
  2115. [0130 17:38:32 @multigpu.py:323] [p0574] step: count(356), step_time 1366.82, mean_step_time 1423.63, it/s 0.7
  2116. [0130 17:38:32 @multigpu.py:323] [p0115] step: count(340), step_time 1403.96, mean_step_time 1424.0, it/s 0.7
  2117. [0130 17:38:33 @multigpu.py:323] [p0576] step: count(363), step_time 1424.65, mean_step_time 1415.98, it/s 0.71
  2118. [0130 17:38:34 @multigpu.py:323] [p0115] step: count(341), step_time 1344.14, mean_step_time 1420.85, it/s 0.7
  2119. [0130 17:38:34 @multigpu.py:323] [p0574] step: count(357), step_time 1401.42, mean_step_time 1425.28, it/s 0.7
  2120. [0130 17:38:34 @multigpu.py:323] [p0576] step: count(364), step_time 1440.01, mean_step_time 1417.47, it/s 0.71
  2121. [0130 17:38:35 @multigpu.py:323] [p0115] step: count(342), step_time 1358.81, mean_step_time 1419.21, it/s 0.7
  2122. [0130 17:38:35 @multigpu.py:323] [p0574] step: count(358), step_time 1376.03, mean_step_time 1424.4, it/s 0.7
  2123. [0130 17:38:36 @multigpu.py:323] [p0576] step: count(365), step_time 1424.0, mean_step_time 1417.6, it/s 0.71
  2124. [0130 17:38:37 @multigpu.py:323] [p0574] step: count(359), step_time 1362.57, mean_step_time 1425.03, it/s 0.7
  2125. [0130 17:38:37 @multigpu.py:323] [p0115] step: count(343), step_time 1408.86, mean_step_time 1420.68, it/s 0.7
  2126. [0130 17:38:37 @multigpu.py:323] [p0576] step: count(366), step_time 1437.07, mean_step_time 1419.12, it/s 0.7
  2127. [0130 17:38:38 @multigpu.py:323] [p0574] step: count(360), step_time 1373.87, mean_step_time 1421.09, it/s 0.7
  2128. [0130 17:38:38 @multigpu.py:323] [p0115] step: count(344), step_time 1415.34, mean_step_time 1422.92, it/s 0.7
  2129. [0130 17:38:38 @multigpu.py:323] [p0576] step: count(367), step_time 1466.44, mean_step_time 1422.59, it/s 0.7
  2130. [0130 17:38:39 @multigpu.py:323] [p0574] step: count(361), step_time 1423.31, mean_step_time 1417.98, it/s 0.71
  2131. [0130 17:38:39 @multigpu.py:323] [p0115] step: count(345), step_time 1372.49, mean_step_time 1424.25, it/s 0.7
  2132. [0130 17:38:40 @multigpu.py:323] [p0576] step: count(368), step_time 1395.91, mean_step_time 1420.64, it/s 0.7
  2133. [0130 17:38:41 @multigpu.py:323] [p0115] step: count(346), step_time 1396.56, mean_step_time 1425.46, it/s 0.7
  2134. [0130 17:38:41 @multigpu.py:323] [p0574] step: count(362), step_time 1432.84, mean_step_time 1422.23, it/s 0.7
  2135. [0130 17:38:41 @multigpu.py:323] [p0576] step: count(369), step_time 1425.34, mean_step_time 1418.04, it/s 0.71
  2136. [0130 17:38:42 @multigpu.py:323] [p0574] step: count(363), step_time 1337.44, mean_step_time 1418.98, it/s 0.7
  2137. [0130 17:38:42 @multigpu.py:323] [p0115] step: count(347), step_time 1367.95, mean_step_time 1422.87, it/s 0.7
  2138. [0130 17:38:43 @multigpu.py:323] [p0576] step: count(370), step_time 1406.6, mean_step_time 1419.59, it/s 0.7
  2139. [0130 17:38:44 @multigpu.py:323] [p0576] step: count(371), step_time 1513.52, mean_step_time 1424.1, it/s 0.7
  2140. [0130 17:38:44 @multigpu.py:323] [p0115] step: count(348), step_time 2106.69, mean_step_time 1424.16, it/s 0.7
  2141. [0130 17:38:44 @multigpu.py:323] [p0574] step: count(364), step_time 2107.26, mean_step_time 1424.15, it/s 0.7
  2142. sending to address tcp://p0112:61216
  2143. ##### Sending to neptune: online_score : 0.145541653037 , 2.1 #####
  2144. [u'online', 2.1]
  2145. receiving
  2146. [0130 17:38:46 @multigpu.py:323] [p0574] step: count(365), step_time 1374.01, mean_step_time 1422.48, it/s 0.7
  2147. [0130 17:38:46 @multigpu.py:323] [p0576] step: count(372), step_time 1403.78, mean_step_time 1422.98, it/s 0.7
  2148. [0130 17:38:46 @multigpu.py:323] [p0115] step: count(349), step_time 1428.43, mean_step_time 1427.09, it/s 0.7
  2149. [0130 17:38:47 @multigpu.py:323] [p0115] step: count(350), step_time 1381.07, mean_step_time 1425.49, it/s 0.7
  2150. [0130 17:38:47 @multigpu.py:323] [p0574] step: count(366), step_time 1463.64, mean_step_time 1427.29, it/s 0.7
  2151. [0130 17:38:47 @multigpu.py:323] [p0576] step: count(373), step_time 1461.16, mean_step_time 1426.0, it/s 0.7
  2152. [0130 17:38:48 @multigpu.py:323] [p0115] step: count(351), step_time 1376.54, mean_step_time 1424.53, it/s 0.7
  2153. [0130 17:38:48 @multigpu.py:323] [p0574] step: count(367), step_time 1424.99, mean_step_time 1430.27, it/s 0.7
  2154. [0130 17:38:48 @multigpu.py:323] [p0576] step: count(374), step_time 1417.18, mean_step_time 1427.99, it/s 0.7
  2155. sending to address tcp://p0112:61216
  2156. ##### Sending to neptune: online_score : 0.146764651073 , 1.8 #####
  2157. [u'online', 1.8]
  2158. receiving
  2159. sending to address tcp://p0112:61216
  2160. ##### Sending to neptune: online_score : 0.146777135266 , 0.9 #####
  2161. [u'online', 0.9]
  2162. receiving
  2163. [0130 17:38:50 @multigpu.py:323] [p0115] step: count(352), step_time 1377.77, mean_step_time 1423.21, it/s 0.7
  2164. [0130 17:38:50 @multigpu.py:323] [p0574] step: count(368), step_time 1360.8, mean_step_time 1426.65, it/s 0.7
  2165. [0130 17:38:50 @multigpu.py:323] [p0576] step: count(375), step_time 1404.25, mean_step_time 1428.7, it/s 0.7
  2166. [0130 17:38:51 @multigpu.py:323] [p0115] step: count(353), step_time 1405.07, mean_step_time 1425.14, it/s 0.7
  2167. [0130 17:38:51 @multigpu.py:323] [p0574] step: count(369), step_time 1383.47, mean_step_time 1425.89, it/s 0.7
  2168. [0130 17:38:51 @multigpu.py:323] [p0576] step: count(376), step_time 1468.44, mean_step_time 1431.6, it/s 0.7
  2169. [0130 17:38:53 @multigpu.py:323] [p0115] step: count(354), step_time 1378.63, mean_step_time 1421.05, it/s 0.7
  2170. [0130 17:38:53 @multigpu.py:323] [p0574] step: count(370), step_time 1375.83, mean_step_time 1427.51, it/s 0.7
  2171. [0130 17:38:53 @multigpu.py:323] [p0576] step: count(377), step_time 1402.5, mean_step_time 1430.09, it/s 0.7
  2172. [0130 17:38:54 @multigpu.py:323] [p0574] step: count(371), step_time 1368.96, mean_step_time 1427.78, it/s 0.7
  2173. [0130 17:38:54 @multigpu.py:323] [p0115] step: count(355), step_time 1424.46, mean_step_time 1424.49, it/s 0.7
  2174. [0130 17:38:54 @multigpu.py:323] [p0576] step: count(378), step_time 1438.33, mean_step_time 1428.5, it/s 0.7
  2175. [0130 17:38:55 @multigpu.py:323] [p0115] step: count(356), step_time 1400.95, mean_step_time 1424.91, it/s 0.7
  2176. [0130 17:38:55 @multigpu.py:323] [p0574] step: count(372), step_time 1506.86, mean_step_time 1432.96, it/s 0.7
  2177. [0130 17:38:56 @multigpu.py:323] [p0576] step: count(379), step_time 1412.34, mean_step_time 1430.86, it/s 0.7
  2178. [0130 17:38:57 @multigpu.py:323] [p0115] step: count(357), step_time 1399.37, mean_step_time 1426.25, it/s 0.7
  2179. [0130 17:38:57 @multigpu.py:323] [p0574] step: count(373), step_time 1384.96, mean_step_time 1433.76, it/s 0.7
  2180. [0130 17:38:57 @multigpu.py:323] [p0576] step: count(380), step_time 1378.29, mean_step_time 1427.31, it/s 0.7
  2181. [0130 17:38:58 @multigpu.py:323] [p0115] step: count(358), step_time 1350.07, mean_step_time 1424.17, it/s 0.7
  2182. [0130 17:38:58 @multigpu.py:323] [p0574] step: count(374), step_time 1464.46, mean_step_time 1439.51, it/s 0.69
  2183. [0130 17:38:58 @multigpu.py:323] [p0576] step: count(381), step_time 1442.18, mean_step_time 1429.25, it/s 0.7
  2184. [0130 17:39:00 @multigpu.py:323] [p0115] step: count(359), step_time 1409.5, mean_step_time 1425.33, it/s 0.7
  2185. [0130 17:39:00 @multigpu.py:323] [p0574] step: count(375), step_time 1375.47, mean_step_time 1433.25, it/s 0.7
  2186. [0130 17:39:00 @multigpu.py:323] [p0576] step: count(382), step_time 1397.03, mean_step_time 1427.95, it/s 0.7
  2187. [0130 17:39:01 @multigpu.py:323] [p0115] step: count(360), step_time 1412.0, mean_step_time 1425.74, it/s 0.7
  2188. [0130 17:39:01 @multigpu.py:323] [p0574] step: count(376), step_time 1413.74, mean_step_time 1435.6, it/s 0.7
  2189. [0130 17:39:01 @multigpu.py:323] [p0576] step: count(383), step_time 1439.07, mean_step_time 1428.67, it/s 0.7
  2190. [0130 17:39:02 @multigpu.py:323] [p0115] step: count(361), step_time 1429.66, mean_step_time 1430.01, it/s 0.7
  2191. [0130 17:39:02 @multigpu.py:323] [p0574] step: count(377), step_time 1388.78, mean_step_time 1434.96, it/s 0.7
  2192. [0130 17:39:03 @multigpu.py:323] [p0576] step: count(384), step_time 1395.49, mean_step_time 1426.45, it/s 0.7
  2193. [0130 17:39:04 @multigpu.py:323] [p0115] step: count(362), step_time 1366.82, mean_step_time 1430.41, it/s 0.7
  2194. [0130 17:39:04 @multigpu.py:323] [p0574] step: count(378), step_time 1380.51, mean_step_time 1435.19, it/s 0.7
  2195. [0130 17:39:04 @multigpu.py:323] [p0576] step: count(385), step_time 1405.33, mean_step_time 1425.51, it/s 0.7
  2196. [0130 17:39:05 @multigpu.py:323] [p0115] step: count(363), step_time 1430.37, mean_step_time 1431.49, it/s 0.7
  2197. [0130 17:39:05 @multigpu.py:323] [p0574] step: count(379), step_time 1398.73, mean_step_time 1437.0, it/s 0.7
  2198. [0130 17:39:06 @multigpu.py:323] [p0576] step: count(386), step_time 1445.15, mean_step_time 1425.92, it/s 0.7
  2199. sending to address tcp://p0112:61216
  2200. ##### Sending to neptune: online_score : 0.151504749722 , 0.8 #####
  2201. [u'online', 0.8]
  2202. receiving
  2203. [0130 17:39:07 @multigpu.py:323] [p0115] step: count(364), step_time 1380.69, mean_step_time 1429.75, it/s 0.7
  2204. [0130 17:39:07 @multigpu.py:323] [p0574] step: count(380), step_time 1397.96, mean_step_time 1438.2, it/s 0.7
  2205. [0130 17:39:07 @multigpu.py:323] [p0576] step: count(387), step_time 1396.97, mean_step_time 1422.44, it/s 0.7
  2206. [0130 17:39:08 @multigpu.py:323] [p0115] step: count(365), step_time 1369.8, mean_step_time 1429.62, it/s 0.7
  2207. [0130 17:39:08 @multigpu.py:323] [p0574] step: count(381), step_time 1393.46, mean_step_time 1436.71, it/s 0.7
  2208. [0130 17:39:08 @multigpu.py:323] [p0576] step: count(388), step_time 1398.0, mean_step_time 1422.55, it/s 0.7
  2209. [0130 17:39:09 @multigpu.py:323] [p0115] step: count(366), step_time 1370.75, mean_step_time 1428.33, it/s 0.7
  2210. [0130 17:39:09 @multigpu.py:323] [p0574] step: count(382), step_time 1404.33, mean_step_time 1435.28, it/s 0.7
  2211. [0130 17:39:10 @multigpu.py:323] [p0576] step: count(389), step_time 1382.74, mean_step_time 1420.42, it/s 0.7
  2212. [0130 17:39:11 @multigpu.py:323] [p0115] step: count(367), step_time 1404.09, mean_step_time 1430.14, it/s 0.7
  2213. [0130 17:39:11 @multigpu.py:323] [p0574] step: count(383), step_time 1458.87, mean_step_time 1441.35, it/s 0.69
  2214. [0130 17:39:11 @multigpu.py:323] [p0576] step: count(390), step_time 1433.6, mean_step_time 1421.77, it/s 0.7
  2215. [0130 17:39:13 @multigpu.py:323] [p0574] step: count(384), step_time 1579.52, mean_step_time 1414.97, it/s 0.71
  2216. [0130 17:39:13 @multigpu.py:323] [p0576] step: count(391), step_time 1378.32, mean_step_time 1415.01, it/s 0.71
  2217. [0130 17:39:13 @multigpu.py:323] [p0115] step: count(368), step_time 1803.48, mean_step_time 1414.97, it/s 0.71
  2218. [0130 17:39:14 @multigpu.py:323] [p0115] step: count(369), step_time 1356.61, mean_step_time 1411.38, it/s 0.71
  2219. [0130 17:39:14 @multigpu.py:323] [p0576] step: count(392), step_time 1430.82, mean_step_time 1416.36, it/s 0.71
  2220. [0130 17:39:14 @multigpu.py:323] [p0574] step: count(385), step_time 1488.43, mean_step_time 1420.69, it/s 0.7
  2221. [0130 17:39:15 @multigpu.py:323] [p0115] step: count(370), step_time 1395.35, mean_step_time 1412.1, it/s 0.71
  2222. [0130 17:39:15 @multigpu.py:323] [p0576] step: count(393), step_time 1378.79, mean_step_time 1412.24, it/s 0.71
  2223. [0130 17:39:15 @multigpu.py:323] [p0574] step: count(386), step_time 1410.11, mean_step_time 1418.01, it/s 0.71
  2224. sending to address tcp://p0112:61216
  2225. ##### Sending to neptune: online_score : 0.154189984997 , 1.0 #####
  2226. [u'online', 1.0]
  2227. receiving
  2228. sending to address tcp://p0112:61216
  2229. ##### Sending to neptune: online_score : 0.154235092733 , 1.1 #####
  2230. [u'online', 1.1]
  2231. receiving
  2232. [0130 17:39:17 @multigpu.py:323] [p0115] step: count(371), step_time 1412.42, mean_step_time 1413.89, it/s 0.71
  2233. [0130 17:39:17 @multigpu.py:323] [p0576] step: count(394), step_time 1456.54, mean_step_time 1414.21, it/s 0.71
  2234. [0130 17:39:17 @multigpu.py:323] [p0574] step: count(387), step_time 1381.87, mean_step_time 1415.86, it/s 0.71
  2235. [0130 17:39:18 @multigpu.py:323] [p0115] step: count(372), step_time 1387.79, mean_step_time 1414.39, it/s 0.71
  2236. [0130 17:39:18 @multigpu.py:323] [p0574] step: count(388), step_time 1394.24, mean_step_time 1417.53, it/s 0.71
  2237. [0130 17:39:18 @multigpu.py:323] [p0576] step: count(395), step_time 1440.95, mean_step_time 1416.04, it/s 0.71
  2238. [0130 17:39:19 @multigpu.py:323] [p0115] step: count(373), step_time 1417.49, mean_step_time 1415.01, it/s 0.71
  2239. [0130 17:39:20 @multigpu.py:323] [p0574] step: count(389), step_time 1441.07, mean_step_time 1420.41, it/s 0.7
  2240. [0130 17:39:20 @multigpu.py:323] [p0576] step: count(396), step_time 1436.47, mean_step_time 1414.45, it/s 0.71
  2241. [0130 17:39:21 @multigpu.py:323] [p0115] step: count(374), step_time 1368.12, mean_step_time 1414.49, it/s 0.71
  2242. [0130 17:39:21 @multigpu.py:323] [p0574] step: count(390), step_time 1398.35, mean_step_time 1421.53, it/s 0.7
  2243. [0130 17:39:21 @multigpu.py:323] [p0576] step: count(397), step_time 1397.34, mean_step_time 1414.19, it/s 0.71
  2244. [0130 17:39:22 @multigpu.py:323] [p0115] step: count(375), step_time 1369.1, mean_step_time 1411.72, it/s 0.71
  2245. [0130 17:39:22 @multigpu.py:323] [p0574] step: count(391), step_time 1369.83, mean_step_time 1421.58, it/s 0.7
  2246. [0130 17:39:23 @multigpu.py:323] [p0576] step: count(398), step_time 1483.03, mean_step_time 1416.42, it/s 0.71
  2247. [0130 17:39:24 @multigpu.py:323] [p0115] step: count(376), step_time 1407.21, mean_step_time 1412.03, it/s 0.71
  2248. [0130 17:39:24 @multigpu.py:323] [p0574] step: count(392), step_time 1344.19, mean_step_time 1413.44, it/s 0.71
  2249. [0130 17:39:24 @multigpu.py:323] [p0576] step: count(399), step_time 1428.4, mean_step_time 1417.23, it/s 0.71
  2250. [0130 17:39:25 @multigpu.py:323] [p0115] step: count(377), step_time 1352.72, mean_step_time 1409.7, it/s 0.71
  2251. [0130 17:39:25 @multigpu.py:323] [p0574] step: count(393), step_time 1421.02, mean_step_time 1415.25, it/s 0.71
  2252. [0130 17:39:25 @multigpu.py:323] [p0576] step: count(400), step_time 1398.51, mean_step_time 1418.24, it/s 0.71
  2253. sending debugging info...
  2254. sending to address tcp://p0112:61216
  2255. ##### Sending to neptune: mean_delay : 0.156876291103 , 0.0 #####
  2256. ##### Sending to neptune: max_delay : 0.156876291103 , -0.0 #####
  2257. sending to address tcp://p0112:61216
  2258. ##### Sending to neptune: min_delay : 0.156876291103 , -0.0 #####
  2259. [u'delays', [0.0, -0.0, -0.0]]
  2260. receiving
  2261. ##### Sending to neptune: cost : 0.156876804696 , -0.00845347996801 #####
  2262. sending to address tcp://p0112:61216
  2263. ##### Sending to neptune: policy_loss : 0.156876804696 , 0.143131583929 #####
  2264. ##### Sending to neptune: xentropy_loss : 0.156876804696 , -2.290143013 #####
  2265. ##### Sending to neptune: value_loss : 0.156876804696 , 1.06496596336 #####
  2266. ##### Sending to neptune: advantage : 0.156876804696 , -0.000597213045694 #####
  2267. ##### Sending to neptune: pred_reward : 0.156876804696 , 0.332899808884 #####
  2268. ##### Sending to neptune: max_logit : 0.156876804696 , 0.185626909137 #####
  2269. [u'loss', -0.00845347996801138, 0.1431315839290619, -2.2901430130004883, 1.0649659633636475, -0.000597213045693934, 0.332899808883667, 0.18562690913677216]
  2270. receiving
  2271. ##### Sending to neptune: active_relus : 0.15687728107 , 8742840.19 #####
  2272. ##### Sending to neptune: dp_per_s : 0.15687728107 , 89.5178593025 #####
  2273. [u'other', 8742840.19, 89.51785930253774]
  2274. receiving
  2275. [0130 17:39:26 @multigpu.py:323] [p0115] step: count(378), step_time 1391.41, mean_step_time 1411.77, it/s 0.71
  2276. [0130 17:39:27 @multigpu.py:323] [p0574] step: count(394), step_time 1364.77, mean_step_time 1410.26, it/s 0.71
  2277. [0130 17:39:27 @multigpu.py:323] [p0576] step: count(401), step_time 1528.51, mean_step_time 1422.55, it/s 0.7
  2278. [0130 17:39:28 @multigpu.py:323] [p0115] step: count(379), step_time 1352.05, mean_step_time 1408.9, it/s 0.71
  2279. [0130 17:39:28 @multigpu.py:323] [p0574] step: count(395), step_time 1368.2, mean_step_time 1409.9, it/s 0.71
  2280. [0130 17:39:28 @multigpu.py:323] [p0576] step: count(402), step_time 1385.77, mean_step_time 1421.99, it/s 0.7
  2281. [0130 17:39:29 @multigpu.py:323] [p0115] step: count(380), step_time 1376.06, mean_step_time 1407.1, it/s 0.71
  2282. [0130 17:39:29 @multigpu.py:323] [p0574] step: count(396), step_time 1420.37, mean_step_time 1410.23, it/s 0.71
  2283. [0130 17:39:30 @multigpu.py:323] [p0576] step: count(403), step_time 1407.63, mean_step_time 1420.42, it/s 0.7
  2284. [0130 17:39:30 @multigpu.py:323] [p0115] step: count(381), step_time 1404.98, mean_step_time 1405.86, it/s 0.71
  2285. [0130 17:39:31 @multigpu.py:323] [p0574] step: count(397), step_time 1370.97, mean_step_time 1409.34, it/s 0.71
  2286. [0130 17:39:31 @multigpu.py:323] [p0576] step: count(404), step_time 1454.24, mean_step_time 1423.36, it/s 0.7
  2287. [0130 17:39:32 @multigpu.py:323] [p0115] step: count(382), step_time 1387.83, mean_step_time 1406.92, it/s 0.71
  2288. [0130 17:39:32 @multigpu.py:323] [p0574] step: count(398), step_time 1396.96, mean_step_time 1410.16, it/s 0.71
  2289. [0130 17:39:33 @multigpu.py:323] [p0576] step: count(405), step_time 1429.75, mean_step_time 1424.58, it/s 0.7
  2290. [0130 17:39:33 @multigpu.py:323] [p0115] step: count(383), step_time 1429.27, mean_step_time 1406.86, it/s 0.71
  2291. [0130 17:39:34 @multigpu.py:323] [p0574] step: count(399), step_time 1423.11, mean_step_time 1411.38, it/s 0.71
  2292. [0130 17:39:34 @multigpu.py:323] [p0576] step: count(406), step_time 1410.93, mean_step_time 1422.87, it/s 0.7
  2293. [0130 17:39:35 @multigpu.py:323] [p0115] step: count(384), step_time 1370.8, mean_step_time 1406.37, it/s 0.71
  2294. [0130 17:39:35 @multigpu.py:323] [p0574] step: count(400), step_time 1392.73, mean_step_time 1411.12, it/s 0.71
  2295. sending debugging info...
  2296. sending to address tcp://p0112:61216
  2297. ##### Sending to neptune: mean_delay : 0.159525871608 , 0.0 #####
  2298. sending to address tcp://p0112:61216
  2299. ##### Sending to neptune: max_delay : 0.159525871608 , -0.0 #####
  2300. ##### Sending to neptune: min_delay : 0.159525871608 , -0.0 #####
  2301. [u'delays', [0.0, -0.0, -0.0]]
  2302. ##### Sending to neptune: active_workers : 0.159526103603 , 3 #####
  2303. receiving
  2304. ##### Sending to neptune: cost : 0.159526361095 , -0.00946252513677 #####
  2305. sending to address tcp://p0112:61216
  2306. ##### Sending to neptune: policy_loss : 0.159526361095 , 0.028890747577 #####
  2307. ##### Sending to neptune: xentropy_loss : 0.159526361095 , -2.29028081894 #####
  2308. ##### Sending to neptune: value_loss : 0.159526361095 , 1.05018687248 #####
  2309. ##### Sending to neptune: advantage : 0.159526361095 , -0.000191484694369 #####
  2310. ##### Sending to neptune: pred_reward : 0.159526361095 , 0.328690111637 #####
  2311. ##### Sending to neptune: max_logit : 0.159526361095 , 0.185199126601 #####
  2312. [u'loss', -0.009462525136768818, 0.02889074757695198, -2.290280818939209, 1.0501868724822998, -0.0001914846943691373, 0.3286901116371155, 0.18519912660121918]
  2313. receiving
  2314. ##### Sending to neptune: active_relus : 0.159526825216 , 8762594.07 #####
  2315. ##### Sending to neptune: dp_per_s : 0.159526825216 , 90.186045923 #####
  2316. [u'other', 8762594.07, 90.18604592303011]
  2317. receiving
  2318. [0130 17:39:35 @multigpu.py:323] [p0576] step: count(407), step_time 1431.07, mean_step_time 1424.57, it/s 0.7
  2319. sending to address tcp://p0112:61216
  2320. ##### Sending to neptune: online_score : 0.15983586106 , 0.9 #####
  2321. [u'online', 0.9]
  2322. receiving
  2323. [0130 17:39:36 @multigpu.py:323] [p0115] step: count(385), step_time 1370.4, mean_step_time 1406.4, it/s 0.71
  2324. [0130 17:39:36 @multigpu.py:323] [p0574] step: count(401), step_time 1408.05, mean_step_time 1411.85, it/s 0.71
  2325. [0130 17:39:37 @multigpu.py:323] [p0576] step: count(408), step_time 1494.54, mean_step_time 1429.4, it/s 0.7
  2326. [0130 17:39:38 @multigpu.py:323] [p0115] step: count(386), step_time 1452.71, mean_step_time 1410.49, it/s 0.71
  2327. [0130 17:39:38 @multigpu.py:323] [p0574] step: count(402), step_time 1369.83, mean_step_time 1410.12, it/s 0.71
  2328. [0130 17:39:38 @multigpu.py:323] [p0576] step: count(409), step_time 1383.21, mean_step_time 1429.42, it/s 0.7
  2329. [0130 17:39:39 @multigpu.py:323] [p0115] step: count(387), step_time 1360.87, mean_step_time 1408.33, it/s 0.71
  2330. [0130 17:39:39 @multigpu.py:323] [p0574] step: count(403), step_time 1416.83, mean_step_time 1408.02, it/s 0.71
  2331. [0130 17:39:40 @multigpu.py:323] [p0576] step: count(410), step_time 1437.49, mean_step_time 1429.62, it/s 0.7
  2332. [0130 17:39:41 @multigpu.py:323] [p0576] step: count(411), step_time 1399.93, mean_step_time 1430.7, it/s 0.7
  2333. [0130 17:39:41 @multigpu.py:323] [p0574] step: count(404), step_time 2033.89, mean_step_time 1430.74, it/s 0.7
  2334. [0130 17:39:41 @multigpu.py:323] [p0115] step: count(388), step_time 2251.8, mean_step_time 1430.75, it/s 0.7
  2335. sending to address tcp://p0112:61216
  2336. ##### Sending to neptune: online_score : 0.161397429109 , 1.8 #####
  2337. [u'online', 1.8]
  2338. receiving
  2339. [0130 17:39:43 @multigpu.py:323] [p0576] step: count(412), step_time 1401.71, mean_step_time 1429.24, it/s 0.7
  2340. [0130 17:39:43 @multigpu.py:323] [p0115] step: count(389), step_time 1446.82, mean_step_time 1435.26, it/s 0.7
  2341. [0130 17:39:43 @multigpu.py:323] [p0574] step: count(405), step_time 1498.12, mean_step_time 1431.22, it/s 0.7
  2342. sending to address tcp://p0112:61216
  2343. ##### Sending to neptune: online_score : 0.161699656049 , 1.6 #####
  2344. [u'online', 1.6]
  2345. receiving
  2346. [0130 17:39:44 @multigpu.py:323] [p0576] step: count(413), step_time 1411.56, mean_step_time 1430.88, it/s 0.7
  2347. [0130 17:39:44 @multigpu.py:323] [p0115] step: count(390), step_time 1435.59, mean_step_time 1437.27, it/s 0.7
  2348. [0130 17:39:44 @multigpu.py:323] [p0574] step: count(406), step_time 1426.94, mean_step_time 1432.07, it/s 0.7
  2349. [0130 17:39:45 @multigpu.py:323] [p0576] step: count(414), step_time 1422.95, mean_step_time 1429.2, it/s 0.7
  2350. [0130 17:39:45 @multigpu.py:323] [p0115] step: count(391), step_time 1368.46, mean_step_time 1435.07, it/s 0.7
  2351. [0130 17:39:45 @multigpu.py:323] [p0574] step: count(407), step_time 1412.92, mean_step_time 1433.62, it/s 0.7
  2352. [0130 17:39:47 @multigpu.py:323] [p0576] step: count(415), step_time 1430.9, mean_step_time 1428.7, it/s 0.7
  2353. [0130 17:39:47 @multigpu.py:323] [p0115] step: count(392), step_time 1425.59, mean_step_time 1436.96, it/s 0.7
  2354. [0130 17:39:47 @multigpu.py:323] [p0574] step: count(408), step_time 1395.29, mean_step_time 1433.67, it/s 0.7
  2355. [0130 17:39:48 @multigpu.py:323] [p0115] step: count(393), step_time 1358.57, mean_step_time 1434.02, it/s 0.7
  2356. [0130 17:39:48 @multigpu.py:323] [p0576] step: count(416), step_time 1435.64, mean_step_time 1428.66, it/s 0.7
  2357. [0130 17:39:48 @multigpu.py:323] [p0574] step: count(409), step_time 1427.05, mean_step_time 1432.97, it/s 0.7
  2358. [0130 17:39:50 @multigpu.py:323] [p0115] step: count(394), step_time 1398.43, mean_step_time 1435.53, it/s 0.7
  2359. [0130 17:39:50 @multigpu.py:323] [p0576] step: count(417), step_time 1410.86, mean_step_time 1429.33, it/s 0.7
  2360. [0130 17:39:50 @multigpu.py:323] [p0574] step: count(410), step_time 1422.03, mean_step_time 1434.16, it/s 0.7
  2361. [0130 17:39:51 @multigpu.py:323] [p0115] step: count(395), step_time 1386.76, mean_step_time 1436.42, it/s 0.7
  2362. [0130 17:39:51 @multigpu.py:323] [p0576] step: count(418), step_time 1420.38, mean_step_time 1426.2, it/s 0.7
  2363. [0130 17:39:51 @multigpu.py:323] [p0574] step: count(411), step_time 1371.37, mean_step_time 1434.23, it/s 0.7
  2364. [0130 17:39:52 @multigpu.py:323] [p0115] step: count(396), step_time 1371.74, mean_step_time 1434.64, it/s 0.7
  2365. [0130 17:39:53 @multigpu.py:323] [p0574] step: count(412), step_time 1423.33, mean_step_time 1438.19, it/s 0.7
  2366. [0130 17:39:53 @multigpu.py:323] [p0576] step: count(419), step_time 1508.1, mean_step_time 1430.18, it/s 0.7
  2367. [0130 17:39:54 @multigpu.py:323] [p0115] step: count(397), step_time 1408.93, mean_step_time 1437.45, it/s 0.7
  2368. [0130 17:39:54 @multigpu.py:323] [p0574] step: count(413), step_time 1415.17, mean_step_time 1437.9, it/s 0.7
  2369. [0130 17:39:54 @multigpu.py:323] [p0576] step: count(420), step_time 1399.22, mean_step_time 1430.22, it/s 0.7
  2370. [0130 17:39:55 @multigpu.py:323] [p0115] step: count(398), step_time 1413.22, mean_step_time 1438.54, it/s 0.7
  2371. [0130 17:39:55 @multigpu.py:323] [p0574] step: count(414), step_time 1410.68, mean_step_time 1440.19, it/s 0.69
  2372. [0130 17:39:56 @multigpu.py:323] [p0576] step: count(421), step_time 1536.37, mean_step_time 1430.61, it/s 0.7
  2373. [0130 17:39:57 @multigpu.py:323] [p0115] step: count(399), step_time 1444.12, mean_step_time 1443.15, it/s 0.69
  2374. [0130 17:39:57 @multigpu.py:323] [p0574] step: count(415), step_time 1390.2, mean_step_time 1441.29, it/s 0.69
  2375. [0130 17:39:57 @multigpu.py:323] [p0576] step: count(422), step_time 1412.54, mean_step_time 1431.95, it/s 0.7
  2376. [0130 17:39:58 @multigpu.py:323] [p0115] step: count(400), step_time 1427.37, mean_step_time 1445.71, it/s 0.69
  2377. sending debugging info...
  2378. sending to address tcp://p0112:61216
  2379. ##### Sending to neptune: mean_delay : 0.165948813889 , 0.0 #####
  2380. ##### Sending to neptune: max_delay : 0.165948813889 , -0.0 #####
  2381. sending to address tcp://p0112:61216
  2382. ##### Sending to neptune: min_delay : 0.165948813889 , -0.0 #####
  2383. [u'delays', [0.0, -0.0, -0.0]]
  2384. receiving
  2385. ##### Sending to neptune: cost : 0.165949346357 , -0.00503255799413 #####
  2386. ##### Sending to neptune: policy_loss : 0.165949346357 , 0.491833776236 #####
  2387. sending to address tcp://p0112:61216
  2388. ##### Sending to neptune: xentropy_loss : 0.165949346357 , -2.29054236412 #####
  2389. ##### Sending to neptune: value_loss : 0.165949346357 , 1.15454125404 #####
  2390. ##### Sending to neptune: advantage : 0.165949346357 , -0.00217221933417 #####
  2391. ##### Sending to neptune: pred_reward : 0.165949346357 , 0.325955986977 #####
  2392. ##### Sending to neptune: max_logit : 0.165949346357 , 0.184464037418 #####
  2393. [u'loss', -0.0050325579941272736, 0.49183377623558044, -2.2905423641204834, 1.154541254043579, -0.0021722193341702223, 0.32595598697662354, 0.18446403741836548]
  2394. receiving
  2395. ##### Sending to neptune: active_relus : 0.165949839685 , 8744566.58 #####
  2396. ##### Sending to neptune: dp_per_s : 0.165949839685 , 89.9175458849 #####
  2397. [u'other', 8744566.58, 89.91754588494861]
  2398. receiving
  2399. [0130 17:39:58 @multigpu.py:323] [p0574] step: count(416), step_time 1385.62, mean_step_time 1439.55, it/s 0.69
  2400. [0130 17:39:58 @multigpu.py:323] [p0576] step: count(423), step_time 1421.15, mean_step_time 1432.63, it/s 0.7
  2401. [0130 17:39:59 @multigpu.py:323] [p0115] step: count(401), step_time 1441.04, mean_step_time 1447.52, it/s 0.69
  2402. [0130 17:39:59 @multigpu.py:323] [p0574] step: count(417), step_time 1377.82, mean_step_time 1439.9, it/s 0.69
  2403. [0130 17:40:00 @multigpu.py:323] [p0576] step: count(424), step_time 1440.47, mean_step_time 1431.94, it/s 0.7
  2404. [0130 17:40:01 @multigpu.py:323] [p0115] step: count(402), step_time 1358.7, mean_step_time 1446.06, it/s 0.69
  2405. [0130 17:40:01 @multigpu.py:323] [p0574] step: count(418), step_time 1439.77, mean_step_time 1442.04, it/s 0.69
  2406. [0130 17:40:01 @multigpu.py:323] [p0576] step: count(425), step_time 1406.44, mean_step_time 1430.77, it/s 0.7
  2407. [0130 17:40:02 @multigpu.py:323] [p0115] step: count(403), step_time 1423.27, mean_step_time 1445.76, it/s 0.69
  2408. [0130 17:40:02 @multigpu.py:323] [p0574] step: count(419), step_time 1406.12, mean_step_time 1441.19, it/s 0.69
  2409. [0130 17:40:03 @multigpu.py:323] [p0576] step: count(426), step_time 1398.42, mean_step_time 1430.15, it/s 0.7
  2410. [0130 17:40:04 @multigpu.py:323] [p0115] step: count(404), step_time 1408.34, mean_step_time 1447.64, it/s 0.69
  2411. [0130 17:40:04 @multigpu.py:323] [p0574] step: count(420), step_time 1404.68, mean_step_time 1441.79, it/s 0.69
  2412. [0130 17:40:04 @multigpu.py:323] [p0576] step: count(427), step_time 1447.22, mean_step_time 1430.95, it/s 0.7
  2413. [0130 17:40:05 @multigpu.py:323] [p0115] step: count(405), step_time 1429.56, mean_step_time 1450.6, it/s 0.69
  2414. [0130 17:40:05 @multigpu.py:323] [p0574] step: count(421), step_time 1381.4, mean_step_time 1440.45, it/s 0.69
  2415. [0130 17:40:05 @multigpu.py:323] [p0576] step: count(428), step_time 1413.83, mean_step_time 1426.92, it/s 0.7
  2416. [0130 17:40:06 @multigpu.py:323] [p0115] step: count(406), step_time 1383.02, mean_step_time 1447.11, it/s 0.69
  2417. [0130 17:40:07 @multigpu.py:323] [p0574] step: count(422), step_time 1432.03, mean_step_time 1443.56, it/s 0.69
  2418. [0130 17:40:07 @multigpu.py:323] [p0576] step: count(429), step_time 1435.3, mean_step_time 1429.52, it/s 0.7
  2419. [0130 17:40:08 @multigpu.py:323] [p0115] step: count(407), step_time 1403.32, mean_step_time 1449.23, it/s 0.69
  2420. [0130 17:40:08 @multigpu.py:323] [p0574] step: count(423), step_time 1403.98, mean_step_time 1442.92, it/s 0.69
  2421. 42%|####2 |423/1000[1[0130 17:40:08 @multigpu.py:323] [p0576] step: count(430), step_time 1396.29, mean_step_time 1427.46, it/s 0.7
  2422. sending to address tcp://p0112:61216
  2423. ##### Sending to neptune: online_score : 0.168805354701 , 1.6 #####
  2424. [u'online', 1.6]
  2425. receiving
  2426. sending to address tcp://p0112:61216
  2427. ##### Sending to neptune: online_score : 0.168823744721 , 1.2 #####
  2428. [u'online', 1.2]
  2429. receiving
  2430. [0130 17:40:10 @multigpu.py:323] [p0576] step: count(431), step_time 1448.01, mean_step_time 1429.87, it/s 0.7
  2431. 43%|####3 |431/1000[1[0130 17:40:10 @multigpu.py:323] [p0574] step: count(424), step_time 1774.42, mean_step_time 1429.95, it/s 0.7
  2432. [0130 17:40:10 @multigpu.py:323] [p0115] step: count(408), step_time 1866.32, mean_step_time 1429.96, it/s 0.7
  2433. sending to address tcp://p0112:61216
  2434. ##### Sending to neptune: online_score : 0.169260831343 , 1.0 #####
  2435. [u'online', 1.0]
  2436. receiving
  2437. [0130 17:40:11 @multigpu.py:323] [p0576] step: count(432), step_time 1369.59, mean_step_time 1428.26, it/s 0.7
  2438. [0130 17:40:11 @multigpu.py:323] [p0574] step: count(425), step_time 1395.36, mean_step_time 1424.81, it/s 0.7
  2439. [0130 17:40:11 @multigpu.py:323] [p0115] step: count(409), step_time 1451.41, mean_step_time 1430.19, it/s 0.7
  2440. [0130 17:40:13 @multigpu.py:323] [p0574] step: count(426), step_time 1415.79, mean_step_time 1424.25, it/s 0.7
  2441. [0130 17:40:13 @multigpu.py:323] [p0115] step: count(410), step_time 1391.93, mean_step_time 1428.01, it/s 0.7
  2442. [0130 17:40:13 @multigpu.py:323] [p0576] step: count(433), step_time 1563.09, mean_step_time 1435.84, it/s 0.7
  2443. [0130 17:40:14 @multigpu.py:323] [p0115] step: count(411), step_time 1372.63, mean_step_time 1428.21, it/s 0.7
  2444. [0130 17:40:14 @multigpu.py:323] [p0574] step: count(427), step_time 1437.07, mean_step_time 1425.46, it/s 0.7
  2445. [0130 17:40:14 @multigpu.py:323] [p0576] step: count(434), step_time 1453.46, mean_step_time 1437.36, it/s 0.7
  2446. [0130 17:40:15 @multigpu.py:323] [p0115] step: count(412), step_time 1360.42, mean_step_time 1424.96, it/s 0.7
  2447. [0130 17:40:15 @multigpu.py:323] [p0574] step: count(428), step_time 1463.36, mean_step_time 1428.86, it/s 0.7
  2448. [0130 17:40:16 @multigpu.py:323] [p0576] step: count(435), step_time 1442.92, mean_step_time 1437.97, it/s 0.7
  2449. [0130 17:40:17 @multigpu.py:323] [p0115] step: count(413), step_time 1390.98, mean_step_time 1426.58, it/s 0.7
  2450. [0130 17:40:17 @multigpu.py:323] [p0574] step: count(429), step_time 1381.28, mean_step_time 1426.57, it/s 0.7
  2451. [0130 17:40:17 @multigpu.py:323] [p0576] step: count(436), step_time 1397.1, mean_step_time 1436.04, it/s 0.7
  2452. [0130 17:40:18 @multigpu.py:323] [p0115] step: count(414), step_time 1367.71, mean_step_time 1425.04, it/s 0.7
  2453. [0130 17:40:18 @multigpu.py:323] [p0574] step: count(430), step_time 1453.28, mean_step_time 1428.14, it/s 0.7
  2454. [0130 17:40:18 @multigpu.py:323] [p0576] step: count(437), step_time 1508.3, mean_step_time 1440.91, it/s 0.69
  2455. [0130 17:40:20 @multigpu.py:323] [p0115] step: count(415), step_time 1425.78, mean_step_time 1426.99, it/s 0.7
  2456. [0130 17:40:20 @multigpu.py:323] [p0574] step: count(431), step_time 1398.99, mean_step_time 1429.52, it/s 0.7
  2457. [0130 17:40:20 @multigpu.py:323] [p0576] step: count(438), step_time 1407.49, mean_step_time 1440.27, it/s 0.69
  2458. [0130 17:40:21 @multigpu.py:323] [p0115] step: count(416), step_time 1336.99, mean_step_time 1425.25, it/s 0.7
  2459. [0130 17:40:21 @multigpu.py:323] [p0574] step: count(432), step_time 1432.96, mean_step_time 1430.0, it/s 0.7
  2460. [0130 17:40:21 @multigpu.py:323] [p0576] step: count(439), step_time 1407.52, mean_step_time 1435.24, it/s 0.7
  2461. [0130 17:40:22 @multigpu.py:323] [p0115] step: count(417), step_time 1408.77, mean_step_time 1425.25, it/s 0.7
  2462. [0130 17:40:23 @multigpu.py:323] [p0574] step: count(433), step_time 1437.57, mean_step_time 1431.12, it/s 0.7
  2463. [0130 17:40:23 @multigpu.py:323] [p0576] step: count(440), step_time 1469.58, mean_step_time 1438.75, it/s 0.7
  2464. [0130 17:40:24 @multigpu.py:323] [p0115] step: count(418), step_time 1355.93, mean_step_time 1422.38, it/s 0.7
  2465. [0130 17:40:24 @multigpu.py:323] [p0574] step: count(434), step_time 1321.66, mean_step_time 1426.67, it/s 0.7
  2466. [0130 17:40:24 @multigpu.py:323] [p0576] step: count(441), step_time 1463.11, mean_step_time 1435.09, it/s 0.7
  2467. [0130 17:40:25 @multigpu.py:323] [p0115] step: count(419), step_time 1403.54, mean_step_time 1420.35, it/s 0.7
  2468. [0130 17:40:25 @multigpu.py:323] [p0574] step: count(435), step_time 1510.72, mean_step_time 1432.69, it/s 0.7
  2469. [0130 17:40:26 @multigpu.py:323] [p0576] step: count(442), step_time 1466.47, mean_step_time 1437.79, it/s 0.7
  2470. [0130 17:40:26 @multigpu.py:323] [p0115] step: count(420), step_time 1437.86, mean_step_time 1420.88, it/s 0.7
  2471. [0130 17:40:27 @multigpu.py:323] [p0574] step: count(436), step_time 1422.17, mean_step_time 1434.52, it/s 0.7
  2472. [0130 17:40:27 @multigpu.py:323] [p0576] step: count(443), step_time 1403.78, mean_step_time 1436.92, it/s 0.7
  2473. [0130 17:40:28 @multigpu.py:323] [p0115] step: count(421), step_time 1481.53, mean_step_time 1422.9, it/s 0.7
  2474. [0130 17:40:28 @multigpu.py:323] [p0574] step: count(437), step_time 1404.15, mean_step_time 1435.84, it/s 0.7
  2475. [0130 17:40:29 @multigpu.py:323] [p0576] step: count(444), step_time 1448.02, mean_step_time 1437.3, it/s 0.7
  2476. [0130 17:40:29 @multigpu.py:323] [p0115] step: count(422), step_time 1408.9, mean_step_time 1425.41, it/s 0.7
  2477. [0130 17:40:30 @multigpu.py:323] [p0574] step: count(438), step_time 1427.92, mean_step_time 1435.25, it/s 0.7
  2478. [0130 17:40:30 @multigpu.py:323] [p0576] step: count(445), step_time 1463.08, mean_step_time 1440.13, it/s 0.69
  2479. [0130 17:40:31 @multigpu.py:323] [p0115] step: count(423), step_time 1399.13, mean_step_time 1424.2, it/s 0.7
  2480. [0130 17:40:31 @multigpu.py:323] [p0574] step: count(439), step_time 1418.53, mean_step_time 1435.87, it/s 0.7
  2481. [0130 17:40:31 @multigpu.py:323] [p0576] step: count(446), step_time 1432.68, mean_step_time 1441.84, it/s 0.69
  2482. [0130 17:40:32 @multigpu.py:323] [p0115] step: count(424), step_time 1430.48, mean_step_time 1425.31, it/s 0.7
  2483. [0130 17:40:32 @multigpu.py:323] [p0574] step: count(440), step_time 1376.25, mean_step_time 1434.45, it/s 0.7
  2484. [0130 17:40:33 @multigpu.py:323] [p0576] step: count(447), step_time 1395.59, mean_step_time 1439.26, it/s 0.69
  2485. sending to address tcp://p0112:61216
  2486. ##### Sending to neptune: online_score : 0.175806029704 , 2.3 #####
  2487. [u'online', 2.3]
  2488. receiving
  2489. [0130 17:40:34 @multigpu.py:323] [p0115] step: count(425), step_time 1367.57, mean_step_time 1422.21, it/s 0.7
  2490. [0130 17:40:34 @multigpu.py:323] [p0574] step: count(441), step_time 1522.2, mean_step_time 1441.49, it/s 0.69
  2491. [0130 17:40:34 @multigpu.py:323] [p0576] step: count(448), step_time 1427.95, mean_step_time 1439.97, it/s 0.69
  2492. [0130 17:40:35 @multigpu.py:323] [p0115] step: count(426), step_time 1404.57, mean_step_time 1423.29, it/s 0.7
  2493. [0130 17:40:35 @multigpu.py:323] [p0574] step: count(442), step_time 1472.0, mean_step_time 1443.48, it/s 0.69
  2494. [0130 17:40:36 @multigpu.py:323] [p0576] step: count(449), step_time 1448.3, mean_step_time 1440.62, it/s 0.69
  2495. [0130 17:40:36 @multigpu.py:323] [p0115] step: count(427), step_time 1345.11, mean_step_time 1420.38, it/s 0.7
  2496. [0130 17:40:37 @multigpu.py:323] [p0574] step: count(443), step_time 1446.05, mean_step_time 1445.59, it/s 0.69
  2497. [0130 17:40:37 @multigpu.py:323] [p0576] step: count(450), step_time 1419.29, mean_step_time 1441.77, it/s 0.69
  2498. sending to address tcp://p0112:61216
  2499. ##### Sending to neptune: online_score : 0.176930492719 , 1.4 #####
  2500. [u'online', 1.4]
  2501. ##### Sending to neptune: active_workers : 0.176930545833 , 3 #####
  2502. receiving
  2503. [0130 17:40:39 @multigpu.py:323] [p0576] step: count(451), step_time 1455.66, mean_step_time 1442.15, it/s 0.69
  2504. [0130 17:40:39 @multigpu.py:323] [p0115] step: count(428), step_time 2300.21, mean_step_time 1442.07, it/s 0.69
  2505. [0130 17:40:39 @multigpu.py:323] [p0574] step: count(444), step_time 1705.61, mean_step_time 1442.15, it/s 0.69
  2506. 43%|####2 |428/1000[1[0130 17:40:40 @multigpu.py:323] [p0115] step: count(429), step_time 1387.77, mean_step_time 1438.89, it/s 0.69
  2507. [0130 17:40:40 @multigpu.py:323] [p0574] step: count(445), step_time 1435.97, mean_step_time 1444.18, it/s 0.69
  2508. [0130 17:40:40 @multigpu.py:323] [p0576] step: count(452), step_time 1460.36, mean_step_time 1446.69, it/s 0.69
  2509. sending to address tcp://p0112:61216
  2510. ##### Sending to neptune: online_score : 0.177810673846 , 1.2 #####
  2511. [u'online', 1.2]
  2512. receiving
  2513. [0130 17:40:41 @multigpu.py:323] [p0574] step: count(446), step_time 1355.41, mean_step_time 1441.16, it/s 0.69
  2514. [0130 17:40:41 @multigpu.py:323] [p0115] step: count(430), step_time 1422.15, mean_step_time 1440.4, it/s 0.69
  2515. [0130 17:40:42 @multigpu.py:323] [p0576] step: count(453), step_time 1466.27, mean_step_time 1441.85, it/s 0.69
  2516. [0130 17:40:43 @multigpu.py:323] [p0574] step: count(447), step_time 1443.97, mean_step_time 1441.5, it/s 0.69
  2517. [0130 17:40:43 @multigpu.py:323] [p0115] step: count(431), step_time 1449.53, mean_step_time 1444.25, it/s 0.69
  2518. [0130 17:40:43 @multigpu.py:323] [p0576] step: count(454), step_time 1419.22, mean_step_time 1440.13, it/s 0.69
  2519. [0130 17:40:44 @multigpu.py:323] [p0574] step: count(448), step_time 1394.73, mean_step_time 1438.07, it/s 0.7
  2520. [0130 17:40:44 @multigpu.py:323] [p0115] step: count(432), step_time 1402.93, mean_step_time 1446.37, it/s 0.69
  2521. [0130 17:40:44 @multigpu.py:323] [p0576] step: count(455), step_time 1422.21, mean_step_time 1439.1, it/s 0.69
  2522. [0130 17:40:46 @multigpu.py:323] [p0115] step: count(433), step_time 1391.4, mean_step_time 1446.39, it/s 0.69
  2523. [0130 17:40:46 @multigpu.py:323] [p0574] step: count(449), step_time 1448.14, mean_step_time 1441.42, it/s 0.69
  2524. [0130 17:40:46 @multigpu.py:323] [p0576] step: count(456), step_time 1384.99, mean_step_time 1438.49, it/s 0.7
  2525. [0130 17:40:47 @multigpu.py:323] [p0115] step: count(434), step_time 1393.72, mean_step_time 1447.69, it/s 0.69
  2526. [0130 17:40:47 @multigpu.py:323] [p0574] step: count(450), step_time 1379.57, mean_step_time 1437.73, it/s 0.7
  2527. [0130 17:40:47 @multigpu.py:323] [p0576] step: count(457), step_time 1415.17, mean_step_time 1433.84, it/s 0.7
  2528. [0130 17:40:48 @multigpu.py:323] [p0574] step: count(451), step_time 1423.27, mean_step_time 1438.94, it/s 0.69
  2529. [0130 17:40:48 @multigpu.py:323] [p0115] step: count(435), step_time 1451.37, mean_step_time 1448.97, it/s 0.69
  2530. [0130 17:40:49 @multigpu.py:323] [p0576] step: count(458), step_time 1438.82, mean_step_time 1435.4, it/s 0.7
  2531. [0130 17:40:50 @multigpu.py:323] [p0574] step: count(452), step_time 1390.49, mean_step_time 1436.82, it/s 0.7
  2532. [0130 17:40:50 @multigpu.py:323] [p0115] step: count(436), step_time 1378.98, mean_step_time 1451.07, it/s 0.69
  2533. [0130 17:40:50 @multigpu.py:323] [p0576] step: count(459), step_time 1444.31, mean_step_time 1437.24, it/s 0.7
  2534. [0130 17:40:51 @multigpu.py:323] [p0574] step: count(453), step_time 1400.46, mean_step_time 1434.96, it/s 0.7
  2535. [0130 17:40:51 @multigpu.py:323] [p0115] step: count(437), step_time 1464.35, mean_step_time 1453.85, it/s 0.69
  2536. [0130 17:40:51 @multigpu.py:323] [p0576] step: count(460), step_time 1397.47, mean_step_time 1433.64, it/s 0.7
  2537. [0130 17:40:53 @multigpu.py:323] [p0574] step: count(454), step_time 1389.83, mean_step_time 1438.37, it/s 0.7
  2538. [0130 17:40:53 @multigpu.py:323] [p0115] step: count(438), step_time 1365.71, mean_step_time 1454.34, it/s 0.69
  2539. [0130 17:40:53 @multigpu.py:323] [p0576] step: count(461), step_time 1436.6, mean_step_time 1432.31, it/s 0.7
  2540. [0130 17:40:54 @multigpu.py:323] [p0574] step: count(455), step_time 1378.02, mean_step_time 1431.74, it/s 0.7
  2541. [0130 17:40:54 @multigpu.py:323] [p0115] step: count(439), step_time 1415.1, mean_step_time 1454.92, it/s 0.69
  2542. [0130 17:40:54 @multigpu.py:323] [p0576] step: count(462), step_time 1447.98, mean_step_time 1431.39, it/s 0.7
  2543. [0130 17:40:55 @multigpu.py:323] [p0574] step: count(456), step_time 1394.3, mean_step_time 1430.34, it/s 0.7
  2544. [0130 17:40:56 @multigpu.py:323] [p0115] step: count(440), step_time 1380.95, mean_step_time 1452.07, it/s 0.69
  2545. [0130 17:40:56 @multigpu.py:323] [p0576] step: count(463), step_time 1442.55, mean_step_time 1433.33, it/s 0.7
  2546. [0130 17:40:57 @multigpu.py:323] [p0574] step: count(457), step_time 1398.26, mean_step_time 1430.05, it/s 0.7
  2547. [0130 17:40:57 @multigpu.py:323] [p0115] step: count(441), step_time 1458.1, mean_step_time 1450.9, it/s 0.69
  2548. [0130 17:40:57 @multigpu.py:323] [p0576] step: count(464), step_time 1360.36, mean_step_time 1428.94, it/s 0.7
  2549. [0130 17:40:58 @multigpu.py:323] [p0574] step: count(458), step_time 1472.43, mean_step_time 1432.27, it/s 0.7
  2550. [0130 17:40:58 @multigpu.py:323] [p0115] step: count(442), step_time 1427.88, mean_step_time 1451.85, it/s 0.69
  2551. [0130 17:40:59 @multigpu.py:323] [p0576] step: count(465), step_time 1417.51, mean_step_time 1426.66, it/s 0.7
  2552. [0130 17:41:00 @multigpu.py:323] [p0574] step: count(459), step_time 1415.56, mean_step_time 1432.13, it/s 0.7
  2553. [0130 17:41:00 @multigpu.py:323] [p0115] step: count(443), step_time 1447.21, mean_step_time 1454.26, it/s 0.69
  2554. [0130 17:41:00 @multigpu.py:323] [p0576] step: count(466), step_time 1473.78, mean_step_time 1428.72, it/s 0.7
  2555. [0130 17:41:01 @multigpu.py:323] [p0574] step: count(460), step_time 1463.47, mean_step_time 1436.49, it/s 0.7
  2556. [0130 17:41:01 @multigpu.py:323] [p0115] step: count(444), step_time 1474.85, mean_step_time 1456.47, it/s 0.69
  2557. [0130 17:41:01 @multigpu.py:323] [p0576] step: count(467), step_time 1465.99, mean_step_time 1432.24, it/s 0.7
  2558. [0130 17:41:03 @multigpu.py:323] [p0574] step: count(461), step_time 1431.21, mean_step_time 1431.94, it/s 0.7
  2559. [0130 17:41:03 @multigpu.py:323] [p0115] step: count(445), step_time 1452.57, mean_step_time 1460.72, it/s 0.68
  2560. [0130 17:41:03 @multigpu.py:323] [p0576] step: count(468), step_time 1414.81, mean_step_time 1431.58, it/s 0.7
  2561. sending to address tcp://p0112:61216
  2562. ##### Sending to neptune: online_score : 0.184176382754 , 1.5 #####
  2563. [u'online', 1.5]
  2564. receiving
  2565. [0130 17:41:04 @multigpu.py:323] [p0574] step: count(462), step_time 1385.28, mean_step_time 1427.6, it/s 0.7
  2566. [0130 17:41:04 @multigpu.py:323] [p0115] step: count(446), step_time 1353.37, mean_step_time 1458.16, it/s 0.69
  2567. [0130 17:41:04 @multigpu.py:323] [p0576] step: count(469), step_time 1445.93, mean_step_time 1431.46, it/s 0.7
  2568. [0130 17:41:05 @multigpu.py:323] [p0574] step: count(463), step_time 1400.82, mean_step_time 1425.34, it/s 0.7
  2569. [0130 17:41:06 @multigpu.py:323] [p0115] step: count(447), step_time 1450.28, mean_step_time 1463.42, it/s 0.68
  2570. [0130 17:41:06 @multigpu.py:323] [p0576] step: count(470), step_time 1445.53, mean_step_time 1432.78, it/s 0.7
  2571. [0130 17:41:07 @multigpu.py:323] [p0576] step: count(471), step_time 1439.44, mean_step_time 1431.97, it/s 0.7
  2572. [0130 17:41:07 @multigpu.py:323] [p0115] step: count(448), step_time 1671.13, mean_step_time 1431.97, it/s 0.7
  2573. [0130 17:41:07 @multigpu.py:323] [p0574] step: count(464), step_time 1838.23, mean_step_time 1431.97, it/s 0.7
  2574. [0130 17:41:09 @multigpu.py:323] [p0576] step: count(472), step_time 1414.98, mean_step_time 1429.7, it/s 0.7
  2575. [0130 17:41:09 @multigpu.py:323] [p0574] step: count(465), step_time 1447.22, mean_step_time 1432.53, it/s 0.7
  2576. [0130 17:41:09 @multigpu.py:323] [p0115] step: count(449), step_time 1450.3, mean_step_time 1435.09, it/s 0.7
  2577. sending to address tcp://p0112:61216
  2578. ##### Sending to neptune: online_score : 0.185665526655 , 1.7 #####
  2579. [u'online', 1.7]
  2580. receiving
  2581. sending to address tcp://p0112:61216
  2582. ##### Sending to neptune: online_score : 0.185782292485 , 1.8 #####
  2583. [u'online', 1.8]
  2584. receiving
  2585. [0130 17:41:10 @multigpu.py:323] [p0115] step: count(450), step_time 1355.29, mean_step_time 1431.75, it/s 0.7
  2586. [0130 17:41:10 @multigpu.py:323] [p0574] step: count(466), step_time 1452.54, mean_step_time 1437.39, it/s 0.7
  2587. [0130 17:41:10 @multigpu.py:323] [p0576] step: count(473), step_time 1488.28, mean_step_time 1430.8, it/s 0.7
  2588. [0130 17:41:12 @multigpu.py:323] [p0115] step: count(451), step_time 1474.42, mean_step_time 1433.0, it/s 0.7
  2589. [0130 17:41:12 @multigpu.py:323] [p0574] step: count(467), step_time 1381.35, mean_step_time 1434.26, it/s 0.7
  2590. [0130 17:41:12 @multigpu.py:323] [p0576] step: count(474), step_time 1435.59, mean_step_time 1431.62, it/s 0.7
  2591. [0130 17:41:13 @multigpu.py:323] [p0574] step: count(468), step_time 1443.75, mean_step_time 1436.71, it/s 0.7
  2592. [0130 17:41:13 @multigpu.py:323] [p0115] step: count(452), step_time 1461.72, mean_step_time 1435.93, it/s 0.7
  2593. [0130 17:41:13 @multigpu.py:323] [p0576] step: count(475), step_time 1461.87, mean_step_time 1433.6, it/s 0.7
  2594. [0130 17:41:14 @multigpu.py:323] [p0574] step: count(469), step_time 1376.0, mean_step_time 1433.1, it/s 0.7
  2595. [0130 17:41:14 @multigpu.py:323] [p0115] step: count(453), step_time 1415.55, mean_step_time 1437.14, it/s 0.7
  2596. [0130 17:41:14 @multigpu.py:323] [p0576] step: count(476), step_time 1399.07, mean_step_time 1434.3, it/s 0.7
  2597. [0130 17:41:16 @multigpu.py:323] [p0574] step: count(470), step_time 1423.37, mean_step_time 1435.29, it/s 0.7
  2598. [0130 17:41:16 @multigpu.py:323] [p0115] step: count(454), step_time 1441.31, mean_step_time 1439.52, it/s 0.69
  2599. [0130 17:41:16 @multigpu.py:323] [p0576] step: count(477), step_time 1432.48, mean_step_time 1435.17, it/s 0.7
  2600. [0130 17:41:17 @multigpu.py:323] [p0574] step: count(471), step_time 1408.02, mean_step_time 1434.53, it/s 0.7
  2601. [0130 17:41:17 @multigpu.py:323] [p0115] step: count(455), step_time 1413.35, mean_step_time 1437.62, it/s 0.7
  2602. [0130 17:41:17 @multigpu.py:323] [p0576] step: count(478), step_time 1450.45, mean_step_time 1435.75, it/s 0.7
  2603. [0130 17:41:19 @multigpu.py:323] [p0574] step: count(472), step_time 1383.97, mean_step_time 1434.2, it/s 0.7
  2604. [0130 17:41:19 @multigpu.py:323] [p0115] step: count(456), step_time 1441.25, mean_step_time 1440.73, it/s 0.69
  2605. [0130 17:41:19 @multigpu.py:323] [p0576] step: count(479), step_time 1452.88, mean_step_time 1436.18, it/s 0.7
  2606. [0130 17:41:20 @multigpu.py:323] [p0574] step: count(473), step_time 1496.55, mean_step_time 1439.01, it/s 0.69
  2607. [0130 17:41:20 @multigpu.py:323] [p0115] step: count(457), step_time 1428.56, mean_step_time 1438.94, it/s 0.69
  2608. [0130 17:41:20 @multigpu.py:323] [p0576] step: count(480), step_time 1485.43, mean_step_time 1440.58, it/s 0.69
  2609. [0130 17:41:22 @multigpu.py:323] [p0574] step: count(474), step_time 1446.31, mean_step_time 1441.83, it/s 0.69
  2610. [0130 17:41:22 @multigpu.py:323] [p0115] step: count(458), step_time 1387.74, mean_step_time 1440.05, it/s 0.69
  2611. [0130 17:41:22 @multigpu.py:323] [p0576] step: count(481), step_time 1456.43, mean_step_time 1441.57, it/s 0.69
  2612. [0130 17:41:23 @multigpu.py:323] [p0574] step: count(475), step_time 1402.51, mean_step_time 1443.06, it/s 0.69
  2613. [0130 17:41:23 @multigpu.py:323] [p0115] step: count(459), step_time 1490.29, mean_step_time 1443.81, it/s 0.69
  2614. [0130 17:41:23 @multigpu.py:323] [p0576] step: count(482), step_time 1437.77, mean_step_time 1441.06, it/s 0.69
  2615. [0130 17:41:24 @multigpu.py:323] [p0574] step: count(476), step_time 1461.35, mean_step_time 1446.41, it/s 0.69
  2616. [0130 17:41:24 @multigpu.py:323] [p0115] step: count(460), step_time 1413.76, mean_step_time 1445.45, it/s 0.69
  2617. [0130 17:41:25 @multigpu.py:323] [p0576] step: count(483), step_time 1371.29, mean_step_time 1437.49, it/s 0.7
  2618. [0130 17:41:26 @multigpu.py:323] [p0574] step: count(477), step_time 1460.21, mean_step_time 1449.51, it/s 0.69
  2619. [0130 17:41:26 @multigpu.py:323] [p0115] step: count(461), step_time 1452.77, mean_step_time 1445.18, it/s 0.69
  2620. [0130 17:41:26 @multigpu.py:323] [p0576] step: count(484), step_time 1469.12, mean_step_time 1442.93, it/s 0.69
  2621. [0130 17:41:27 @multigpu.py:323] [p0574] step: count(478), step_time 1411.56, mean_step_time 1446.46, it/s 0.69
  2622. [0130 17:41:27 @multigpu.py:323] [p0115] step: count(462), step_time 1413.91, mean_step_time 1444.48, it/s 0.69
  2623. [0130 17:41:27 @multigpu.py:323] [p0576] step: count(485), step_time 1386.82, mean_step_time 1441.4, it/s 0.69
  2624. [0130 17:41:29 @multigpu.py:323] [p0574] step: count(479), step_time 1431.48, mean_step_time 1447.26, it/s 0.69
  2625. [0130 17:41:29 @multigpu.py:323] [p0115] step: count(463), step_time 1387.01, mean_step_time 1441.47, it/s 0.69
  2626. [0130 17:41:29 @multigpu.py:323] [p0576] step: count(486), step_time 1379.58, mean_step_time 1436.69, it/s 0.7
  2627. [0130 17:41:30 @multigpu.py:323] [p0115] step: count(464), step_time 1401.1, mean_step_time 1437.78, it/s 0.7
  2628. [0130 17:41:30 @multigpu.py:323] [p0574] step: count(480), step_time 1439.94, mean_step_time 1446.08, it/s 0.69
  2629. [0130 17:41:30 @multigpu.py:323] [p0576] step: count(487), step_time 1441.15, mean_step_time 1435.45, it/s 0.7
  2630. [0130 17:41:31 @multigpu.py:323] [p0115] step: count(465), step_time 1415.79, mean_step_time 1435.95, it/s 0.7
  2631. [0130 17:41:32 @multigpu.py:323] [p0574] step: count(481), step_time 1418.57, mean_step_time 1445.45, it/s 0.69
  2632. [0130 17:41:32 @multigpu.py:323] [p0576] step: count(488), step_time 1418.84, mean_step_time 1435.65, it/s 0.7
  2633. [0130 17:41:33 @multigpu.py:323] [p0115] step: count(466), step_time 1443.08, mean_step_time 1440.43, it/s 0.69
  2634. [0130 17:41:33 @multigpu.py:323] [p0574] step: count(482), step_time 1420.62, mean_step_time 1447.22, it/s 0.69
  2635. [0130 17:41:33 @multigpu.py:323] [p0576] step: count(489), step_time 1418.78, mean_step_time 1434.29, it/s 0.7
  2636. sending to address tcp://p0112:61216
  2637. ##### Sending to neptune: online_score : 0.192464135223 , 1.2 #####
  2638. [u'online', 1.2]
  2639. receiving
  2640. [0130 17:41:34 @multigpu.py:323] [p0574] step: count(483), step_time 1385.07, mean_step_time 1446.43, it/s 0.69
  2641. sending to address tcp://p0112:61216
  2642. ##### Sending to neptune: online_score : 0.192708853881 , 1.7 #####
  2643. [u'online', 1.7]
  2644. receiving
  2645. [0130 17:41:34 @multigpu.py:323] [p0576] step: count(490), step_time 1362.98, mean_step_time 1430.16, it/s 0.7
  2646. [0130 17:41:34 @multigpu.py:323] [p0115] step: count(467), step_time 1487.79, mean_step_time 1442.31, it/s 0.69
  2647. [0130 17:41:36 @multigpu.py:323] [p0115] step: count(468), step_time 1426.91, mean_step_time 1430.1, it/s 0.7
  2648. [0130 17:41:36 @multigpu.py:323] [p0574] step: count(484), step_time 1515.46, mean_step_time 1430.29, it/s 0.7
  2649. [0130 17:41:36 @multigpu.py:323] [p0576] step: count(491), step_time 1442.33, mean_step_time 1430.31, it/s 0.7
  2650. sending to address tcp://p0112:61216
  2651. ##### Sending to neptune: online_score : 0.193346670535 , 1.6 #####
  2652. [u'online', 1.6]
  2653. receiving
  2654. [0130 17:41:37 @multigpu.py:323] [p0115] step: count(469), step_time 1406.09, mean_step_time 1427.89, it/s 0.7
  2655. [0130 17:41:37 @multigpu.py:323] [p0574] step: count(485), step_time 1420.3, mean_step_time 1428.95, it/s 0.7
  2656. [0130 17:41:37 @multigpu.py:323] [p0576] step: count(492), step_time 1433.82, mean_step_time 1431.25, it/s 0.7
  2657. [0130 17:41:39 @multigpu.py:323] [p0115] step: count(470), step_time 1469.25, mean_step_time 1433.58, it/s 0.7
  2658. [0130 17:41:39 @multigpu.py:323] [p0574] step: count(486), step_time 1462.15, mean_step_time 1429.43, it/s 0.7
  2659. [0130 17:41:39 @multigpu.py:323] [p0576] step: count(493), step_time 1465.1, mean_step_time 1430.09, it/s 0.7
  2660. [0130 17:41:40 @multigpu.py:323] [p0574] step: count(487), step_time 1401.71, mean_step_time 1430.44, it/s 0.7
  2661. [0130 17:41:40 @multigpu.py:323] [p0115] step: count(471), step_time 1413.02, mean_step_time 1430.51, it/s 0.7
  2662. [0130 17:41:40 @multigpu.py:323] [p0576] step: count(494), step_time 1428.58, mean_step_time 1429.74, it/s 0.7
  2663. [0130 17:41:42 @multigpu.py:323] [p0574] step: count(488), step_time 1397.83, mean_step_time 1428.15, it/s 0.7
  2664. [0130 17:41:42 @multigpu.py:323] [p0115] step: count(472), step_time 1402.81, mean_step_time 1427.57, it/s 0.7
  2665. [0130 17:41:42 @multigpu.py:323] [p0576] step: count(495), step_time 1393.27, mean_step_time 1426.31, it/s 0.7
  2666. [0130 17:41:43 @multigpu.py:323] [p0574] step: count(489), step_time 1419.44, mean_step_time 1430.32, it/s 0.7
  2667. [0130 17:41:43 @multigpu.py:323] [p0576] step: count(496), step_time 1402.95, mean_step_time 1426.5, it/s 0.7
  2668. [0130 17:41:43 @multigpu.py:323] [p0115] step: count(473), step_time 1439.46, mean_step_time 1428.76, it/s 0.7
  2669. [0130 17:41:44 @multigpu.py:323] [p0574] step: count(490), step_time 1411.47, mean_step_time 1429.73, it/s 0.7
  2670. [0130 17:41:44 @multigpu.py:323] [p0576] step: count(497), step_time 1391.16, mean_step_time 1424.44, it/s 0.7
  2671. [0130 17:41:44 @multigpu.py:323] [p0115] step: count(474), step_time 1432.71, mean_step_time 1428.33, it/s 0.7
  2672. [0130 17:41:46 @multigpu.py:323] [p0576] step: count(498), step_time 1395.99, mean_step_time 1421.71, it/s 0.7
  2673. [0130 17:41:46 @multigpu.py:323] [p0574] step: count(491), step_time 1425.6, mean_step_time 1430.6, it/s 0.7
  2674. [0130 17:41:46 @multigpu.py:323] [p0115] step: count(475), step_time 1449.3, mean_step_time 1430.13, it/s 0.7
  2675. [0130 17:41:47 @multigpu.py:323] [p0574] step: count(492), step_time 1396.91, mean_step_time 1431.25, it/s 0.7
  2676. [0130 17:41:47 @multigpu.py:323] [p0576] step: count(499), step_time 1446.81, mean_step_time 1421.41, it/s 0.7
  2677. [0130 17:41:47 @multigpu.py:323] [p0115] step: count(476), step_time 1454.84, mean_step_time 1430.81, it/s 0.7
  2678. [0130 17:41:49 @multigpu.py:323] [p0574] step: count(493), step_time 1403.47, mean_step_time 1426.6, it/s 0.7
  2679. [0130 17:41:49 @multigpu.py:323] [p0576] step: count(500), step_time 1426.58, mean_step_time 1418.47, it/s 0.7
  2680. sending debugging info...
  2681. sending to address tcp://p0112:61216
  2682. ##### Sending to neptune: mean_delay : 0.19667889244 , 0.0 #####
  2683. ##### Sending to neptune: max_delay : 0.19667889244 , -0.0 #####
  2684. sending to address tcp://p0112:61216
  2685. ##### Sending to neptune: min_delay : 0.19667889244 , -0.0 #####
  2686. [u'delays', [0.0, -0.0, -0.0]]
  2687. ##### Sending to neptune: active_workers : 0.196679146356 , 3 #####
  2688. receiving
  2689. ##### Sending to neptune: cost : 0.196679431332 , -0.00439015356824 #####
  2690. ##### Sending to neptune: policy_loss : 0.196679431332 , 0.453959643841 #####
  2691. sending to address tcp://p0112:61216
  2692. ##### Sending to neptune: xentropy_loss : 0.196679431332 , -2.29136276245 #####
  2693. ##### Sending to neptune: value_loss : 0.196679431332 , 1.27546346188 #####
  2694. ##### Sending to neptune: advantage : 0.196679431332 , -0.00191708060447 #####
  2695. ##### Sending to neptune: pred_reward : 0.196679431332 , 0.342677921057 #####
  2696. ##### Sending to neptune: max_logit : 0.196679431332 , 0.181843400002 #####
  2697. [u'loss', -0.00439015356823802, 0.4539596438407898, -2.291362762451172, 1.2754634618759155, -0.0019170806044712663, 0.34267792105674744, 0.18184340000152588]
  2698. receiving
  2699. ##### Sending to neptune: active_relus : 0.19667988552 , 8799167.4 #####
  2700. ##### Sending to neptune: dp_per_s : 0.19667988552 , 89.3428788891 #####
  2701. [u'other', 8799167.4, 89.34287888906876]
  2702. receiving
  2703. [0130 17:41:49 @multigpu.py:323] [p0115] step: count(477), step_time 1390.74, mean_step_time 1428.92, it/s 0.7
  2704. [0130 17:41:50 @multigpu.py:323] [p0574] step: count(494), step_time 1432.89, mean_step_time 1425.93, it/s 0.7
  2705. [0130 17:41:50 @multigpu.py:323] [p0115] step: count(478), step_time 1369.78, mean_step_time 1428.02, it/s 0.7
  2706. [0130 17:41:50 @multigpu.py:323] [p0576] step: count(501), step_time 1493.99, mean_step_time 1420.34, it/s 0.7
  2707. [0130 17:41:51 @multigpu.py:323] [p0574] step: count(495), step_time 1441.23, mean_step_time 1427.86, it/s 0.7
  2708. [0130 17:41:51 @multigpu.py:323] [p0115] step: count(479), step_time 1400.71, mean_step_time 1423.54, it/s 0.7
  2709. [0130 17:41:52 @multigpu.py:323] [p0576] step: count(502), step_time 1473.53, mean_step_time 1422.13, it/s 0.7
  2710. [0130 17:41:53 @multigpu.py:323] [p0574] step: count(496), step_time 1426.57, mean_step_time 1426.12, it/s 0.7
  2711. [0130 17:41:53 @multigpu.py:323] [p0115] step: count(480), step_time 1464.67, mean_step_time 1426.09, it/s 0.7
  2712. [0130 17:41:53 @multigpu.py:323] [p0576] step: count(503), step_time 1417.0, mean_step_time 1424.42, it/s 0.7
  2713. [0130 17:41:54 @multigpu.py:323] [p0574] step: count(497), step_time 1475.96, mean_step_time 1426.91, it/s 0.7
  2714. [0130 17:41:54 @multigpu.py:323] [p0115] step: count(481), step_time 1456.84, mean_step_time 1426.29, it/s 0.7
  2715. [0130 17:41:54 @multigpu.py:323] [p0576] step: count(504), step_time 1443.88, mean_step_time 1423.16, it/s 0.7
  2716. [0130 17:41:56 @multigpu.py:323] [p0574] step: count(498), step_time 1423.72, mean_step_time 1427.52, it/s 0.7
  2717. [0130 17:41:56 @multigpu.py:323] [p0115] step: count(482), step_time 1441.86, mean_step_time 1427.69, it/s 0.7
  2718. [0130 17:41:56 @multigpu.py:323] [p0576] step: count(505), step_time 1450.04, mean_step_time 1426.32, it/s 0.7
  2719. [0130 17:41:57 @multigpu.py:323] [p0574] step: count(499), step_time 1483.14, mean_step_time 1430.1, it/s 0.7
  2720. [0130 17:41:57 @multigpu.py:323] [p0115] step: count(483), step_time 1466.56, mean_step_time 1431.67, it/s 0.7
  2721. [0130 17:41:57 @multigpu.py:323] [p0576] step: count(506), step_time 1446.31, mean_step_time 1429.65, it/s 0.7
  2722. [0130 17:41:59 @multigpu.py:323] [p0115] step: count(484), step_time 1380.52, mean_step_time 1430.64, it/s 0.7
  2723. [0130 17:41:59 @multigpu.py:323] [p0574] step: count(500), step_time 1416.52, mean_step_time 1428.93, it/s 0.7
  2724. sending debugging info...
  2725. sending to address tcp://p0112:61216
  2726. ##### Sending to neptune: mean_delay : 0.199472724663 , 0.0 #####
  2727. ##### Sending to neptune: max_delay : 0.199472724663 , -0.0 #####
  2728. sending to address tcp://p0112:61216
  2729. ##### Sending to neptune: min_delay : 0.199472724663 , -0.0 #####
  2730. [u'delays', [0.0, -0.0, -0.0]]
  2731. receiving
  2732. ##### Sending to neptune: cost : 0.199473265807 , -0.00781606975943 #####
  2733. sending to address tcp://p0112:61216
  2734. ##### Sending to neptune: policy_loss : 0.199473265807 , 0.0337311848998 #####
  2735. ##### Sending to neptune: xentropy_loss : 0.199473265807 , -2.29143595695 #####
  2736. ##### Sending to neptune: value_loss : 0.199473265807 , 1.2572479248 #####
  2737. ##### Sending to neptune: advantage : 0.199473265807 , -3.6175846617e-05 #####
  2738. ##### Sending to neptune: pred_reward : 0.199473265807 , 0.347458809614 #####
  2739. ##### Sending to neptune: max_logit : 0.199473265807 , 0.181484788656 #####
  2740. [u'loss', -0.007816069759428501, 0.033731184899806976, -2.291435956954956, 1.2572479248046875, -3.6175846616970375e-05, 0.3474588096141815, 0.18148478865623474]
  2741. receiving
  2742. ##### Sending to neptune: active_relus : 0.199473697742 , 8790208.01 #####
  2743. ##### Sending to neptune: dp_per_s : 0.199473697742 , 89.2401421731 #####
  2744. [u'other', 8790208.01, 89.24014217307855]
  2745. receiving
  2746. [0130 17:41:59 @multigpu.py:323] [p0576] step: count(507), step_time 1433.45, mean_step_time 1429.27, it/s 0.7
  2747. sending to address tcp://p0112:61216
  2748. ##### Sending to neptune: online_score : 0.199656606052 , 1.1 #####
  2749. [u'online', 1.1]
  2750. receiving
  2751. [0130 17:42:00 @multigpu.py:323] [p0574] step: count(501), step_time 1400.88, mean_step_time 1428.05, it/s 0.7
  2752. [0130 17:42:00 @multigpu.py:323] [p0115] step: count(485), step_time 1468.38, mean_step_time 1433.27, it/s 0.7
  2753. [0130 17:42:00 @multigpu.py:323] [p0576] step: count(508), step_time 1441.14, mean_step_time 1430.38, it/s 0.7
  2754. [0130 17:42:01 @multigpu.py:323] [p0574] step: count(502), step_time 1372.72, mean_step_time 1425.65, it/s 0.7
  2755. [0130 17:42:02 @multigpu.py:323] [p0115] step: count(486), step_time 1390.88, mean_step_time 1430.66, it/s 0.7
  2756. [0130 17:42:02 @multigpu.py:323] [p0576] step: count(509), step_time 1419.81, mean_step_time 1430.44, it/s 0.7
  2757. [0130 17:42:03 @multigpu.py:323] [p0574] step: count(503), step_time 1453.52, mean_step_time 1429.07, it/s 0.7
  2758. [0130 17:42:03 @multigpu.py:323] [p0115] step: count(487), step_time 1401.45, mean_step_time 1426.34, it/s 0.7
  2759. [0130 17:42:03 @multigpu.py:323] [p0576] step: count(510), step_time 1404.39, mean_step_time 1432.51, it/s 0.7
  2760. [0130 17:42:05 @multigpu.py:323] [p0115] step: count(488), step_time 1536.76, mean_step_time 1431.83, it/s 0.7
  2761. [0130 17:42:05 @multigpu.py:323] [p0574] step: count(504), step_time 1569.74, mean_step_time 1431.79, it/s 0.7
  2762. [0130 17:42:05 @multigpu.py:323] [p0576] step: count(511), step_time 1427.58, mean_step_time 1431.77, it/s 0.7
  2763. sending to address tcp://p0112:61216
  2764. ##### Sending to neptune: online_score : 0.201240341332 , 1.7 #####
  2765. [u'online', 1.7]
  2766. receiving
  2767. [0130 17:42:06 @multigpu.py:323] [p0574] step: count(505), step_time 1407.0, mean_step_time 1431.12, it/s 0.7
  2768. [0130 17:42:06 @multigpu.py:323] [p0115] step: count(489), step_time 1416.23, mean_step_time 1432.34, it/s 0.7
  2769. [0130 17:42:06 @multigpu.py:323] [p0576] step: count(512), step_time 1428.95, mean_step_time 1431.52, it/s 0.7
  2770. sending to address tcp://p0112:61216
  2771. ##### Sending to neptune: online_score : 0.201575732165 , 1.6 #####
  2772. [u'online', 1.6]
  2773. receiving
  2774. [0130 17:42:07 @multigpu.py:323] [p0115] step: count(490), step_time 1366.42, mean_step_time 1427.2, it/s 0.7
  2775. [0130 17:42:07 @multigpu.py:323] [p0576] step: count(513), step_time 1370.6, mean_step_time 1426.8, it/s 0.7
  2776. [0130 17:42:07 @multigpu.py:323] [p0574] step: count(506), step_time 1396.16, mean_step_time 1427.82, it/s 0.7
  2777. [0130 17:42:09 @multigpu.py:323] [p0115] step: count(491), step_time 1405.04, mean_step_time 1426.8, it/s 0.7
  2778. [0130 17:42:09 @multigpu.py:323] [p0574] step: count(507), step_time 1438.12, mean_step_time 1429.64, it/s 0.7
  2779. [0130 17:42:09 @multigpu.py:323] [p0576] step: count(514), step_time 1453.14, mean_step_time 1428.03, it/s 0.7
  2780. [0130 17:42:10 @multigpu.py:323] [p0115] step: count(492), step_time 1436.25, mean_step_time 1428.47, it/s 0.7
  2781. [0130 17:42:10 @multigpu.py:323] [p0576] step: count(515), step_time 1375.54, mean_step_time 1427.14, it/s 0.7
  2782. [0130 17:42:10 @multigpu.py:323] [p0574] step: count(508), step_time 1403.88, mean_step_time 1429.95, it/s 0.7
  2783. [0130 17:42:12 @multigpu.py:323] [p0115] step: count(493), step_time 1390.78, mean_step_time 1426.04, it/s 0.7
  2784. [0130 17:42:12 @multigpu.py:323] [p0576] step: count(516), step_time 1406.27, mean_step_time 1427.31, it/s 0.7
  2785. [0130 17:42:12 @multigpu.py:323] [p0574] step: count(509), step_time 1413.02, mean_step_time 1429.63, it/s 0.7
  2786. [0130 17:42:13 @multigpu.py:323] [p0576] step: count(517), step_time 1391.98, mean_step_time 1427.35, it/s 0.7
  2787. [0130 17:42:13 @multigpu.py:323] [p0115] step: count(494), step_time 1422.91, mean_step_time 1425.55, it/s 0.7
  2788. [0130 17:42:13 @multigpu.py:323] [p0574] step: count(510), step_time 1383.5, mean_step_time 1428.23, it/s 0.7
  2789. [0130 17:42:14 @multigpu.py:323] [p0576] step: count(518), step_time 1410.97, mean_step_time 1428.1, it/s 0.7
  2790. [0130 17:42:14 @multigpu.py:323] [p0574] step: count(511), step_time 1423.99, mean_step_time 1428.15, it/s 0.7
  2791. [0130 17:42:14 @multigpu.py:323] [p0115] step: count(495), step_time 1461.77, mean_step_time 1426.17, it/s 0.7
  2792. [0130 17:42:16 @multigpu.py:323] [p0574] step: count(512), step_time 1455.42, mean_step_time 1431.07, it/s 0.7
  2793. [0130 17:42:16 @multigpu.py:323] [p0576] step: count(519), step_time 1486.84, mean_step_time 1430.1, it/s 0.7
  2794. [0130 17:42:16 @multigpu.py:323] [p0115] step: count(496), step_time 1454.06, mean_step_time 1426.13, it/s 0.7
  2795. [0130 17:42:17 @multigpu.py:323] [p0574] step: count(513), step_time 1404.82, mean_step_time 1431.14, it/s 0.7
  2796. [0130 17:42:17 @multigpu.py:323] [p0576] step: count(520), step_time 1438.83, mean_step_time 1430.71, it/s 0.7
  2797. [0130 17:42:17 @multigpu.py:323] [p0115] step: count(497), step_time 1474.65, mean_step_time 1430.33, it/s 0.7
  2798. [0130 17:42:19 @multigpu.py:323] [p0574] step: count(514), step_time 1431.24, mean_step_time 1431.06, it/s 0.7
  2799. [0130 17:42:19 @multigpu.py:323] [p0576] step: count(521), step_time 1396.57, mean_step_time 1425.84, it/s 0.7
  2800. [0130 17:42:19 @multigpu.py:323] [p0115] step: count(498), step_time 1381.51, mean_step_time 1430.91, it/s 0.7
  2801. [0130 17:42:20 @multigpu.py:323] [p0576] step: count(522), step_time 1417.83, mean_step_time 1423.06, it/s 0.7
  2802. [0130 17:42:20 @multigpu.py:323] [p0574] step: count(515), step_time 1437.48, mean_step_time 1430.87, it/s 0.7
  2803. [0130 17:42:20 @multigpu.py:323] [p0115] step: count(499), step_time 1391.41, mean_step_time 1430.45, it/s 0.7
  2804. [0130 17:42:22 @multigpu.py:323] [p0574] step: count(516), step_time 1395.2, mean_step_time 1429.3, it/s 0.7
  2805. [0130 17:42:22 @multigpu.py:323] [p0576] step: count(523), step_time 1412.15, mean_step_time 1422.81, it/s 0.7
  2806. [0130 17:42:22 @multigpu.py:323] [p0115] step: count(500), step_time 1470.12, mean_step_time 1430.72, it/s 0.7
  2807. sending debugging info...
  2808. sending to address tcp://p0112:61216
  2809. ##### Sending to neptune: mean_delay : 0.205827626652 , 0.0 #####
  2810. sending to address tcp://p0112:61216
  2811. ##### Sending to neptune: max_delay : 0.205827626652 , -0.0 #####
  2812. ##### Sending to neptune: min_delay : 0.205827626652 , -0.0 #####
  2813. [u'delays', [0.0, -0.0, -0.0]]
  2814. receiving
  2815. ##### Sending to neptune: cost : 0.205828117198 , -0.0121910655871 #####
  2816. sending to address tcp://p0112:61216
  2817. ##### Sending to neptune: policy_loss : 0.205828117198 , -0.40860247612 #####
  2818. ##### Sending to neptune: xentropy_loss : 0.205828117198 , -2.29158711433 #####
  2819. ##### Sending to neptune: value_loss : 0.205828117198 , 1.1397330761 #####
  2820. ##### Sending to neptune: advantage : 0.205828117198 , 0.00180070789065 #####
  2821. ##### Sending to neptune: pred_reward : 0.205828117198 , 0.358477056026 #####
  2822. ##### Sending to neptune: max_logit : 0.205828117198 , 0.181056678295 #####
  2823. [u'loss', -0.012191065587103367, -0.4086024761199951, -2.2915871143341064, 1.139733076095581, 0.0018007078906521201, 0.35847705602645874, 0.1810566782951355]
  2824. receiving
  2825. ##### Sending to neptune: active_relus : 0.205828589996 , 8779679.08 #####
  2826. ##### Sending to neptune: dp_per_s : 0.205828589996 , 89.1530482324 #####
  2827. [u'other', 8779679.08, 89.15304823242096]
  2828. receiving
  2829. [0130 17:42:23 @multigpu.py:323] [p0574] step: count(517), step_time 1393.95, mean_step_time 1425.2, it/s 0.7
  2830. [0130 17:42:23 @multigpu.py:323] [p0576] step: count(524), step_time 1418.22, mean_step_time 1421.53, it/s 0.7
  2831. [0130 17:42:23 @multigpu.py:323] [p0115] step: count(501), step_time 1392.3, mean_step_time 1427.49, it/s 0.7
  2832. [0130 17:42:24 @multigpu.py:323] [p0574] step: count(518), step_time 1433.31, mean_step_time 1425.68, it/s 0.7
  2833. [0130 17:42:24 @multigpu.py:323] [p0576] step: count(525), step_time 1478.57, mean_step_time 1422.96, it/s 0.7
  2834. [0130 17:42:24 @multigpu.py:323] [p0115] step: count(502), step_time 1453.34, mean_step_time 1428.07, it/s 0.7
  2835. sending to address tcp://p0112:61216
  2836. ##### Sending to neptune: online_score : 0.206725753281 , 0.9 #####
  2837. [u'online', 0.9]
  2838. receiving
  2839. [0130 17:42:26 @multigpu.py:323] [p0574] step: count(519), step_time 1437.16, mean_step_time 1423.38, it/s 0.7
  2840. [0130 17:42:26 @multigpu.py:323] [p0576] step: count(526), step_time 1381.81, mean_step_time 1419.73, it/s 0.7
  2841. [0130 17:42:26 @multigpu.py:323] [p0115] step: count(503), step_time 1398.28, mean_step_time 1424.65, it/s 0.7
  2842. [0130 17:42:27 @multigpu.py:323] [p0576] step: count(527), step_time 1381.54, mean_step_time 1417.14, it/s 0.71
  2843. [0130 17:42:27 @multigpu.py:323] [p0574] step: count(520), step_time 1449.36, mean_step_time 1425.02, it/s 0.7
  2844. [0130 17:42:27 @multigpu.py:323] [p0115] step: count(504), step_time 1423.18, mean_step_time 1426.79, it/s 0.7
  2845. [0130 17:42:29 @multigpu.py:323] [p0576] step: count(528), step_time 1415.51, mean_step_time 1415.86, it/s 0.71
  2846. [0130 17:42:29 @multigpu.py:323] [p0115] step: count(505), step_time 1373.35, mean_step_time 1422.04, it/s 0.7
  2847. [0130 17:42:29 @multigpu.py:323] [p0574] step: count(521), step_time 1408.41, mean_step_time 1425.4, it/s 0.7
  2848. [0130 17:42:30 @multigpu.py:323] [p0576] step: count(529), step_time 1428.52, mean_step_time 1416.29, it/s 0.71
  2849. [0130 17:42:30 @multigpu.py:323] [p0574] step: count(522), step_time 1431.4, mean_step_time 1428.33, it/s 0.7
  2850. sending to address tcp://p0112:61216
  2851. [0130 17:42:30 @multigpu.py:323] [p0115] step: count(506), step_time 1448.29, mean_step_time 1424.91, it/s 0.7
  2852. ##### Sending to neptune: online_score : 0.208189289702 , 1.6 #####
  2853. [u'online', 1.6]
  2854. receiving
  2855. [0130 17:42:31 @multigpu.py:323] [p0576] step: count(530), step_time 1397.03, mean_step_time 1415.92, it/s 0.71
  2856. [0130 17:42:32 @multigpu.py:323] [p0115] step: count(507), step_time 1437.19, mean_step_time 1426.69, it/s 0.7
  2857. [0130 17:42:32 @multigpu.py:323] [p0574] step: count(523), step_time 1454.59, mean_step_time 1428.39, it/s 0.7
  2858. [0130 17:42:33 @multigpu.py:323] [p0574] step: count(524), step_time 1493.74, mean_step_time 1424.59, it/s 0.7
  2859. [0130 17:42:33 @multigpu.py:323] [p0576] step: count(531), step_time 1604.96, mean_step_time 1424.79, it/s 0.7
  2860. [0130 17:42:33 @multigpu.py:323] [p0115] step: count(508), step_time 1498.7, mean_step_time 1424.79, it/s 0.7
  2861. sending to address tcp://p0112:61216
  2862. ##### Sending to neptune: online_score : 0.209143604438 , 1.8 #####
  2863. [u'online', 1.8]
  2864. receiving
  2865. [0130 17:42:34 @multigpu.py:323] [p0574] step: count(525), step_time 1396.72, mean_step_time 1424.07, it/s 0.7
  2866. [0130 17:42:34 @multigpu.py:323] [p0115] step: count(509), step_time 1457.94, mean_step_time 1426.88, it/s 0.7
  2867. [0130 17:42:34 @multigpu.py:323] [p0576] step: count(532), step_time 1484.47, mean_step_time 1427.57, it/s 0.7
  2868. [0130 17:42:36 @multigpu.py:323] [p0574] step: count(526), step_time 1379.54, mean_step_time 1423.24, it/s 0.7
  2869. [0130 17:42:36 @multigpu.py:323] [p0115] step: count(510), step_time 1384.93, mean_step_time 1427.8, it/s 0.7
  2870. [0130 17:42:36 @multigpu.py:323] [p0576] step: count(533), step_time 1391.0, mean_step_time 1428.59, it/s 0.7
  2871. [0130 17:42:37 @multigpu.py:323] [p0574] step: count(527), step_time 1455.82, mean_step_time 1424.13, it/s 0.7
  2872. [0130 17:42:37 @multigpu.py:323] [p0576] step: count(534), step_time 1399.56, mean_step_time 1425.91, it/s 0.7
  2873. [0130 17:42:37 @multigpu.py:323] [p0115] step: count(511), step_time 1458.11, mean_step_time 1430.45, it/s 0.7
  2874. [0130 17:42:39 @multigpu.py:323] [p0115] step: count(512), step_time 1391.15, mean_step_time 1428.2, it/s 0.7
  2875. [0130 17:42:39 @multigpu.py:323] [p0574] step: count(528), step_time 1511.41, mean_step_time 1429.5, it/s 0.7
  2876. [0130 17:42:39 @multigpu.py:323] [p0576] step: count(535), step_time 1479.66, mean_step_time 1431.11, it/s 0.7
  2877. [0130 17:42:40 @multigpu.py:323] [p0576] step: count(536), step_time 1398.88, mean_step_time 1430.74, it/s 0.7
  2878. [0130 17:42:40 @multigpu.py:323] [p0115] step: count(513), step_time 1464.78, mean_step_time 1431.9, it/s 0.7
  2879. [0130 17:42:40 @multigpu.py:323] [p0574] step: count(529), step_time 1445.78, mean_step_time 1431.14, it/s 0.7
  2880. [0130 17:42:42 @multigpu.py:323] [p0115] step: count(514), step_time 1391.88, mean_step_time 1430.35, it/s 0.7
  2881. [0130 17:42:42 @multigpu.py:323] [p0574] step: count(530), step_time 1430.96, mean_step_time 1433.52, it/s 0.7
  2882. [0130 17:42:42 @multigpu.py:323] [p0576] step: count(537), step_time 1485.03, mean_step_time 1435.4, it/s 0.7
  2883. [0130 17:42:43 @multigpu.py:323] [p0115] step: count(515), step_time 1505.01, mean_step_time 1432.51, it/s 0.7
  2884. [0130 17:42:43 @multigpu.py:323] [p0574] step: count(531), step_time 1441.49, mean_step_time 1434.39, it/s 0.7
  2885. [0130 17:42:43 @multigpu.py:323] [p0576] step: count(538), step_time 1469.73, mean_step_time 1438.34, it/s 0.7
  2886. [0130 17:42:44 @multigpu.py:323] [p0115] step: count(516), step_time 1386.08, mean_step_time 1429.11, it/s 0.7
  2887. [0130 17:42:44 @multigpu.py:323] [p0574] step: count(532), step_time 1381.01, mean_step_time 1430.67, it/s 0.7
  2888. [0130 17:42:44 @multigpu.py:323] [p0576] step: count(539), step_time 1360.79, mean_step_time 1432.03, it/s 0.7
  2889. [0130 17:42:46 @multigpu.py:323] [p0574] step: count(533), step_time 1399.77, mean_step_time 1430.42, it/s 0.7
  2890. [0130 17:42:46 @multigpu.py:323] [p0576] step: count(540), step_time 1413.46, mean_step_time 1430.76, it/s 0.7
  2891. [0130 17:42:46 @multigpu.py:323] [p0115] step: count(517), step_time 1497.86, mean_step_time 1430.27, it/s 0.7
  2892. [0130 17:42:47 @multigpu.py:323] [p0574] step: count(534), step_time 1418.43, mean_step_time 1429.78, it/s 0.7
  2893. [0130 17:42:47 @multigpu.py:323] [p0576] step: count(541), step_time 1464.92, mean_step_time 1434.18, it/s 0.7
  2894. [0130 17:42:47 @multigpu.py:323] [p0115] step: count(518), step_time 1473.97, mean_step_time 1434.89, it/s 0.7
  2895. [0130 17:42:49 @multigpu.py:323] [p0574] step: count(535), step_time 1418.27, mean_step_time 1428.82, it/s 0.7
  2896. [0130 17:42:49 @multigpu.py:323] [p0576] step: count(542), step_time 1427.37, mean_step_time 1434.66, it/s 0.7
  2897. [0130 17:42:49 @multigpu.py:323] [p0115] step: count(519), step_time 1454.96, mean_step_time 1438.07, it/s 0.7
  2898. [0130 17:42:50 @multigpu.py:323] [p0574] step: count(536), step_time 1423.84, mean_step_time 1430.25, it/s 0.7
  2899. [0130 17:42:50 @multigpu.py:323] [p0576] step: count(543), step_time 1444.12, mean_step_time 1436.26, it/s 0.7
  2900. [0130 17:42:50 @multigpu.py:323] [p0115] step: count(520), step_time 1469.49, mean_step_time 1438.04, it/s 0.7
  2901. sending to address tcp://p0112:61216
  2902. ##### Sending to neptune: online_score : 0.213829125497 , 1.3 #####
  2903. [u'online', 1.3]
  2904. ##### Sending to neptune: active_workers : 0.213829183843 , 3 #####
  2905. receiving
  2906. [0130 17:42:51 @multigpu.py:323] [p0574] step: count(537), step_time 1378.7, mean_step_time 1429.49, it/s 0.7
  2907. [0130 17:42:52 @multigpu.py:323] [p0576] step: count(544), step_time 1363.11, mean_step_time 1433.5, it/s 0.7
  2908. [0130 17:42:52 @multigpu.py:323] [p0115] step: count(521), step_time 1388.45, mean_step_time 1437.85, it/s 0.7
  2909. [0130 17:42:53 @multigpu.py:323] [p0574] step: count(538), step_time 1451.19, mean_step_time 1430.38, it/s 0.7
  2910. [0130 17:42:53 @multigpu.py:323] [p0576] step: count(545), step_time 1422.98, mean_step_time 1430.72, it/s 0.7
  2911. [0130 17:42:53 @multigpu.py:323] [p0115] step: count(522), step_time 1421.59, mean_step_time 1436.26, it/s 0.7
  2912. [0130 17:42:54 @multigpu.py:323] [p0574] step: count(539), step_time 1439.49, mean_step_time 1430.5, it/s 0.7
  2913. [0130 17:42:54 @multigpu.py:323] [p0576] step: count(546), step_time 1419.38, mean_step_time 1432.6, it/s 0.7
  2914. [0130 17:42:55 @multigpu.py:323] [p0115] step: count(523), step_time 1437.96, mean_step_time 1438.24, it/s 0.7
  2915. [0130 17:42:56 @multigpu.py:323] [p0574] step: count(540), step_time 1451.98, mean_step_time 1430.63, it/s 0.7
  2916. [0130 17:42:56 @multigpu.py:323] [p0576] step: count(547), step_time 1419.85, mean_step_time 1434.52, it/s 0.7
  2917. [0130 17:42:56 @multigpu.py:323] [p0115] step: count(524), step_time 1419.43, mean_step_time 1438.06, it/s 0.7
  2918. sending to address tcp://p0112:61216
  2919. ##### Sending to neptune: online_score : 0.21545850469 , 2.3 #####
  2920. [u'online', 2.3]
  2921. receiving
  2922. [0130 17:42:57 @multigpu.py:323] [p0574] step: count(541), step_time 1421.9, mean_step_time 1431.3, it/s 0.7
  2923. [0130 17:42:57 @multigpu.py:323] [p0576] step: count(548), step_time 1434.48, mean_step_time 1435.46, it/s 0.7
  2924. [0130 17:42:57 @multigpu.py:323] [p0115] step: count(525), step_time 1418.32, mean_step_time 1440.3, it/s 0.69
  2925. [0130 17:42:59 @multigpu.py:323] [p0574] step: count(542), step_time 1432.74, mean_step_time 1431.37, it/s 0.7
  2926. [0130 17:42:59 @multigpu.py:323] [p0576] step: count(549), step_time 1452.17, mean_step_time 1436.65, it/s 0.7
  2927. [0130 17:42:59 @multigpu.py:323] [p0115] step: count(526), step_time 1431.79, mean_step_time 1439.48, it/s 0.69
  2928. [0130 17:43:00 @multigpu.py:323] [p0574] step: count(543), step_time 1390.72, mean_step_time 1428.18, it/s 0.7
  2929. [0130 17:43:00 @multigpu.py:323] [p0576] step: count(550), step_time 1431.72, mean_step_time 1438.38, it/s 0.7
  2930. [0130 17:43:00 @multigpu.py:323] [p0115] step: count(527), step_time 1469.17, mean_step_time 1441.08, it/s 0.69
  2931. [0130 17:43:02 @multigpu.py:323] [p0115] step: count(528), step_time 1439.59, mean_step_time 1438.12, it/s 0.7
  2932. [0130 17:43:02 @multigpu.py:323] [p0576] step: count(551), step_time 1598.78, mean_step_time 1438.07, it/s 0.7
  2933. [0130 17:43:02 @multigpu.py:323] [p0574] step: count(544), step_time 1693.96, mean_step_time 1438.19, it/s 0.7
  2934. [0130 17:43:03 @multigpu.py:323] [p0576] step: count(552), step_time 1420.36, mean_step_time 1434.87, it/s 0.7
  2935. [0130 17:43:03 @multigpu.py:323] [p0574] step: count(545), step_time 1434.88, mean_step_time 1440.09, it/s 0.69
  2936. [0130 17:43:03 @multigpu.py:323] [p0115] step: count(529), step_time 1502.04, mean_step_time 1440.33, it/s 0.69
  2937. [0130 17:43:05 @multigpu.py:323] [p0576] step: count(553), step_time 1381.06, mean_step_time 1434.37, it/s 0.7
  2938. [0130 17:43:05 @multigpu.py:323] [p0574] step: count(546), step_time 1432.99, mean_step_time 1442.77, it/s 0.69
  2939. [0130 17:43:05 @multigpu.py:323] [p0115] step: count(530), step_time 1425.52, mean_step_time 1442.36, it/s 0.69
  2940. sending to address tcp://p0112:61216
  2941. ##### Sending to neptune: online_score : 0.217906641629 , 1.8 #####
  2942. [u'online', 1.8]
  2943. receiving
  2944. [0130 17:43:06 @multigpu.py:323] [p0576] step: count(554), step_time 1442.1, mean_step_time 1436.5, it/s 0.7
  2945. [0130 17:43:06 @multigpu.py:323] [p0574] step: count(547), step_time 1473.52, mean_step_time 1443.65, it/s 0.69
  2946. [0130 17:43:06 @multigpu.py:323] [p0115] step: count(531), step_time 1427.41, mean_step_time 1440.82, it/s 0.69
  2947. [0130 17:43:07 @multigpu.py:323] [p0576] step: count(555), step_time 1424.39, mean_step_time 1433.73, it/s 0.7
  2948. [0130 17:43:08 @multigpu.py:323] [p0574] step: count(548), step_time 1439.58, mean_step_time 1440.06, it/s 0.69
  2949. [0130 17:43:08 @multigpu.py:323] [p0115] step: count(532), step_time 1430.38, mean_step_time 1442.78, it/s 0.69
  2950. [0130 17:43:09 @multigpu.py:323] [p0576] step: count(556), step_time 1518.46, mean_step_time 1439.71, it/s 0.69
  2951. [0130 17:43:09 @multigpu.py:323] [p0574] step: count(549), step_time 1427.17, mean_step_time 1439.13, it/s 0.69
  2952. [0130 17:43:09 @multigpu.py:323] [p0115] step: count(533), step_time 1443.05, mean_step_time 1441.7, it/s 0.69
  2953. [0130 17:43:10 @multigpu.py:323] [p0576] step: count(557), step_time 1374.29, mean_step_time 1434.18, it/s 0.7
  2954. [0130 17:43:10 @multigpu.py:323] [p0115] step: count(534), step_time 1428.15, mean_step_time 1443.51, it/s 0.69
  2955. [0130 17:43:10 @multigpu.py:323] [p0574] step: count(550), step_time 1473.98, mean_step_time 1441.28, it/s 0.69
  2956. [0130 17:43:12 @multigpu.py:323] [p0576] step: count(558), step_time 1458.34, mean_step_time 1433.61, it/s 0.7
  2957. [0130 17:43:12 @multigpu.py:323] [p0574] step: count(551), step_time 1400.64, mean_step_time 1439.24, it/s 0.69
  2958. [0130 17:43:12 @multigpu.py:323] [p0115] step: count(535), step_time 1427.15, mean_step_time 1439.62, it/s 0.69
  2959. [0130 17:43:13 @multigpu.py:323] [p0576] step: count(559), step_time 1451.05, mean_step_time 1438.12, it/s 0.7
  2960. [0130 17:43:13 @multigpu.py:323] [p0115] step: count(536), step_time 1428.3, mean_step_time 1441.73, it/s 0.69
  2961. [0130 17:43:13 @multigpu.py:323] [p0574] step: count(552), step_time 1464.41, mean_step_time 1443.41, it/s 0.69
  2962. [0130 17:43:15 @multigpu.py:323] [p0576] step: count(560), step_time 1439.48, mean_step_time 1439.42, it/s 0.69
  2963. [0130 17:43:15 @multigpu.py:323] [p0115] step: count(537), step_time 1441.73, mean_step_time 1438.92, it/s 0.69
  2964. [0130 17:43:15 @multigpu.py:323] [p0574] step: count(553), step_time 1434.15, mean_step_time 1445.13, it/s 0.69
  2965. [0130 17:43:16 @multigpu.py:323] [p0576] step: count(561), step_time 1380.15, mean_step_time 1435.18, it/s 0.7
  2966. [0130 17:43:16 @multigpu.py:323] [p0574] step: count(554), step_time 1402.8, mean_step_time 1444.35, it/s 0.69
  2967. [0130 17:43:16 @multigpu.py:323] [p0115] step: count(538), step_time 1504.04, mean_step_time 1440.43, it/s 0.69
  2968. [0130 17:43:17 @multigpu.py:323] [p0576] step: count(562), step_time 1414.47, mean_step_time 1434.54, it/s 0.7
  2969. [0130 17:43:18 @multigpu.py:323] [p0574] step: count(555), step_time 1400.29, mean_step_time 1443.45, it/s 0.69
  2970. [0130 17:43:18 @multigpu.py:323] [p0115] step: count(539), step_time 1493.64, mean_step_time 1442.36, it/s 0.69
  2971. [0130 17:43:19 @multigpu.py:323] [p0576] step: count(563), step_time 1459.15, mean_step_time 1435.29, it/s 0.7
  2972. [0130 17:43:19 @multigpu.py:323] [p0574] step: count(556), step_time 1398.45, mean_step_time 1442.18, it/s 0.69
  2973. [0130 17:43:19 @multigpu.py:323] [p0115] step: count(540), step_time 1440.72, mean_step_time 1440.92, it/s 0.69
  2974. sending to address tcp://p0112:61216
  2975. ##### Sending to neptune: online_score : 0.221927292479 , 1.7 #####
  2976. [u'online', 1.7]
  2977. receiving
  2978. [0130 17:43:20 @multigpu.py:323] [p0574] step: count(557), step_time 1431.14, mean_step_time 1444.8, it/s 0.69
  2979. [0130 17:43:20 @multigpu.py:323] [p0576] step: count(564), step_time 1454.93, mean_step_time 1439.88, it/s 0.69
  2980. [0130 17:43:21 @multigpu.py:323] [p0115] step: count(541), step_time 1474.27, mean_step_time 1445.21, it/s 0.69
  2981. [0130 17:43:22 @multigpu.py:323] [p0574] step: count(558), step_time 1400.69, mean_step_time 1442.27, it/s 0.69
  2982. [0130 17:43:22 @multigpu.py:323] [p0576] step: count(565), step_time 1448.58, mean_step_time 1441.16, it/s 0.69
  2983. [0130 17:43:22 @multigpu.py:323] [p0115] step: count(542), step_time 1401.76, mean_step_time 1444.22, it/s 0.69
  2984. sending to address tcp://p0112:61216
  2985. ##### Sending to neptune: online_score : 0.222741466893 , 1.1 #####
  2986. [u'online', 1.1]
  2987. receiving
  2988. [0130 17:43:23 @multigpu.py:323] [p0576] step: count(566), step_time 1423.63, mean_step_time 1441.37, it/s 0.69
  2989. [0130 17:43:23 @multigpu.py:323] [p0574] step: count(559), step_time 1486.25, mean_step_time 1444.61, it/s 0.69
  2990. [0130 17:43:23 @multigpu.py:323] [p0115] step: count(543), step_time 1438.65, mean_step_time 1444.26, it/s 0.69
  2991. [0130 17:43:25 @multigpu.py:323] [p0576] step: count(567), step_time 1395.37, mean_step_time 1440.15, it/s 0.69
  2992. [0130 17:43:25 @multigpu.py:323] [p0574] step: count(560), step_time 1489.61, mean_step_time 1446.49, it/s 0.69
  2993. [0130 17:43:25 @multigpu.py:323] [p0115] step: count(544), step_time 1469.23, mean_step_time 1446.75, it/s 0.69
  2994. [0130 17:43:26 @multigpu.py:323] [p0576] step: count(568), step_time 1395.78, mean_step_time 1438.21, it/s 0.7
  2995. [0130 17:43:26 @multigpu.py:323] [p0574] step: count(561), step_time 1387.96, mean_step_time 1444.8, it/s 0.69
  2996. [0130 17:43:26 @multigpu.py:323] [p0115] step: count(545), step_time 1483.14, mean_step_time 1449.99, it/s 0.69
  2997. [0130 17:43:28 @multigpu.py:323] [p0576] step: count(569), step_time 1442.61, mean_step_time 1437.74, it/s 0.7
  2998. [0130 17:43:28 @multigpu.py:323] [p0574] step: count(562), step_time 1438.72, mean_step_time 1445.1, it/s 0.69
  2999. [0130 17:43:28 @multigpu.py:323] [p0115] step: count(546), step_time 1477.18, mean_step_time 1452.26, it/s 0.69
  3000. sending to address tcp://p0112:61216
  3001. ##### Sending to neptune: online_score : 0.224450801346 , 1.6 #####
  3002. [u'online', 1.6]
  3003. receiving
  3004. [0130 17:43:29 @multigpu.py:323] [p0576] step: count(570), step_time 1441.42, mean_step_time 1438.22, it/s 0.7
  3005. [0130 17:43:29 @multigpu.py:323] [p0574] step: count(563), step_time 1451.3, mean_step_time 1448.12, it/s 0.69
  3006. [0130 17:43:29 @multigpu.py:323] [p0115] step: count(547), step_time 1422.33, mean_step_time 1449.91, it/s 0.69
  3007. [0130 17:43:31 @multigpu.py:323] [p0115] step: count(548), step_time 1458.19, mean_step_time 1450.84, it/s 0.69
  3008. [0130 17:43:31 @multigpu.py:323] [p0574] step: count(564), step_time 1747.66, mean_step_time 1450.81, it/s 0.69
  3009. [0130 17:43:31 @multigpu.py:323] [p0576] step: count(571), step_time 1850.98, mean_step_time 1450.83, it/s 0.69
  3010. [0130 17:43:32 @multigpu.py:323] [p0576] step: count(572), step_time 1377.4, mean_step_time 1448.68, it/s 0.69
  3011. [0130 17:43:32 @multigpu.py:323] [p0115] step: count(549), step_time 1430.69, mean_step_time 1447.28, it/s 0.69
  3012. [0130 17:43:32 @multigpu.py:323] [p0574] step: count(565), step_time 1487.1, mean_step_time 1453.42, it/s 0.69
  3013. [0130 17:43:34 @multigpu.py:323] [p0576] step: count(573), step_time 1371.86, mean_step_time 1448.22, it/s 0.69
  3014. [0130 17:43:34 @multigpu.py:323] [p0115] step: count(550), step_time 1427.34, mean_step_time 1447.37, it/s 0.69
  3015. [0130 17:43:34 @multigpu.py:323] [p0574] step: count(566), step_time 1374.94, mean_step_time 1450.52, it/s 0.69
  3016. [0130 17:43:35 @multigpu.py:323] [p0576] step: count(574), step_time 1433.33, mean_step_time 1447.78, it/s 0.69
  3017. [0130 17:43:35 @multigpu.py:323] [p0115] step: count(551), step_time 1411.06, mean_step_time 1446.55, it/s 0.69
  3018. [0130 17:43:35 @multigpu.py:323] [p0574] step: count(567), step_time 1443.21, mean_step_time 1449.0, it/s 0.69
  3019. [0130 17:43:36 @multigpu.py:323] [p0576] step: count(575), step_time 1403.55, mean_step_time 1446.74, it/s 0.69
  3020. [0130 17:43:37 @multigpu.py:323] [p0115] step: count(552), step_time 1421.4, mean_step_time 1446.1, it/s 0.69
  3021. [0130 17:43:37 @multigpu.py:323] [p0574] step: count(568), step_time 1440.96, mean_step_time 1449.07, it/s 0.69
  3022. [0130 17:43:38 @multigpu.py:323] [p0576] step: count(576), step_time 1403.86, mean_step_time 1441.01, it/s 0.69
  3023. [0130 17:43:38 @multigpu.py:323] [p0115] step: count(553), step_time 1453.49, mean_step_time 1446.62, it/s 0.69
  3024. [0130 17:43:38 @multigpu.py:323] [p0574] step: count(569), step_time 1424.56, mean_step_time 1448.94, it/s 0.69
  3025. [0130 17:43:39 @multigpu.py:323] [p0576] step: count(577), step_time 1452.25, mean_step_time 1444.91, it/s 0.69
  3026. [0130 17:43:39 @multigpu.py:323] [p0115] step: count(554), step_time 1380.55, mean_step_time 1444.24, it/s 0.69
  3027. [0130 17:43:39 @multigpu.py:323] [p0574] step: count(570), step_time 1430.86, mean_step_time 1446.78, it/s 0.69
  3028. [0130 17:43:41 @multigpu.py:323] [p0576] step: count(578), step_time 1408.74, mean_step_time 1442.43, it/s 0.69
  3029. [0130 17:43:41 @multigpu.py:323] [p0115] step: count(555), step_time 1435.03, mean_step_time 1444.64, it/s 0.69
  3030. [0130 17:43:41 @multigpu.py:323] [p0574] step: count(571), step_time 1406.29, mean_step_time 1447.07, it/s 0.69
  3031. [0130 17:43:42 @multigpu.py:323] [p0576] step: count(579), step_time 1439.34, mean_step_time 1441.84, it/s 0.69
  3032. [0130 17:43:42 @multigpu.py:323] [p0115] step: count(556), step_time 1421.63, mean_step_time 1444.3, it/s 0.69
  3033. [0130 17:43:42 @multigpu.py:323] [p0574] step: count(572), step_time 1442.9, mean_step_time 1445.99, it/s 0.69
  3034. [0130 17:43:43 @multigpu.py:323] [p0576] step: count(580), step_time 1380.83, mean_step_time 1438.91, it/s 0.69
  3035. [0130 17:43:44 @multigpu.py:323] [p0115] step: count(557), step_time 1395.92, mean_step_time 1442.01, it/s 0.69
  3036. [0130 17:43:44 @multigpu.py:323] [p0574] step: count(573), step_time 1402.75, mean_step_time 1444.42, it/s 0.69
  3037. [0130 17:43:45 @multigpu.py:323] [p0576] step: count(581), step_time 1417.78, mean_step_time 1440.79, it/s 0.69
  3038. [0130 17:43:45 @multigpu.py:323] [p0574] step: count(574), step_time 1430.28, mean_step_time 1445.8, it/s 0.69
  3039. [0130 17:43:45 @multigpu.py:323] [p0115] step: count(558), step_time 1527.13, mean_step_time 1443.17, it/s 0.69
  3040. [0130 17:43:46 @multigpu.py:323] [p0576] step: count(582), step_time 1476.98, mean_step_time 1443.92, it/s 0.69
  3041. [0130 17:43:47 @multigpu.py:323] [p0115] step: count(559), step_time 1452.16, mean_step_time 1441.09, it/s 0.69
  3042. [0130 17:43:47 @multigpu.py:323] [p0574] step: count(575), step_time 1483.65, mean_step_time 1449.96, it/s 0.69
  3043. [0130 17:43:48 @multigpu.py:323] [p0576] step: count(583), step_time 1433.33, mean_step_time 1442.63, it/s 0.69
  3044. [0130 17:43:48 @multigpu.py:323] [p0115] step: count(560), step_time 1500.53, mean_step_time 1444.08, it/s 0.69
  3045. [0130 17:43:48 @multigpu.py:323] [p0574] step: count(576), step_time 1490.74, mean_step_time 1454.58, it/s 0.69
  3046. [0130 17:43:49 @multigpu.py:323] [p0576] step: count(584), step_time 1371.12, mean_step_time 1438.44, it/s 0.7
  3047. [0130 17:43:50 @multigpu.py:323] [p0115] step: count(561), step_time 1460.94, mean_step_time 1443.42, it/s 0.69
  3048. [0130 17:43:50 @multigpu.py:323] [p0574] step: count(577), step_time 1482.66, mean_step_time 1457.15, it/s 0.69
  3049. [0130 17:43:51 @multigpu.py:323] [p0576] step: count(585), step_time 1448.1, mean_step_time 1438.41, it/s 0.7
  3050. [0130 17:43:51 @multigpu.py:323] [p0115] step: count(562), step_time 1387.06, mean_step_time 1442.68, it/s 0.69
  3051. [0130 17:43:51 @multigpu.py:323] [p0574] step: count(578), step_time 1464.13, mean_step_time 1460.33, it/s 0.68
  3052. sending to address tcp://p0112:61216
  3053. ##### Sending to neptune: online_score : 0.230837176641 , 1.8 #####
  3054. [u'online', 1.8]
  3055. ##### Sending to neptune: active_workers : 0.230837286645 , 3 #####
  3056. receiving
  3057. [0130 17:43:52 @multigpu.py:323] [p0576] step: count(586), step_time 1444.38, mean_step_time 1439.45, it/s 0.69
  3058. [0130 17:43:52 @multigpu.py:323] [p0115] step: count(563), step_time 1462.85, mean_step_time 1443.89, it/s 0.69
  3059. [0130 17:43:52 @multigpu.py:323] [p0574] step: count(579), step_time 1430.18, mean_step_time 1457.52, it/s 0.69
  3060. [0130 17:43:54 @multigpu.py:323] [p0576] step: count(587), step_time 1440.66, mean_step_time 1441.71, it/s 0.69
  3061. [0130 17:43:54 @multigpu.py:323] [p0115] step: count(564), step_time 1395.56, mean_step_time 1440.21, it/s 0.69
  3062. [0130 17:43:54 @multigpu.py:323] [p0574] step: count(580), step_time 1406.88, mean_step_time 1453.39, it/s 0.69
  3063. sending to address tcp://p0112:61216
  3064. ##### Sending to neptune: online_score : 0.23157861299 , 1.7 #####
  3065. [u'online', 1.7]
  3066. receiving
  3067. [0130 17:43:55 @multigpu.py:323] [p0576] step: count(588), step_time 1370.65, mean_step_time 1440.46, it/s 0.69
  3068. [0130 17:43:55 @multigpu.py:323] [p0115] step: count(565), step_time 1395.37, mean_step_time 1435.82, it/s 0.7
  3069. [0130 17:43:55 @multigpu.py:323] [p0574] step: count(581), step_time 1436.96, mean_step_time 1455.84, it/s 0.69
  3070. [0130 17:43:56 @multigpu.py:323] [p0576] step: count(589), step_time 1421.98, mean_step_time 1439.43, it/s 0.69
  3071. [0130 17:43:57 @multigpu.py:323] [p0115] step: count(566), step_time 1388.97, mean_step_time 1431.41, it/s 0.7
  3072. [0130 17:43:57 @multigpu.py:323] [p0574] step: count(582), step_time 1424.21, mean_step_time 1455.11, it/s 0.69
  3073. [0130 17:43:58 @multigpu.py:323] [p0576] step: count(590), step_time 1406.28, mean_step_time 1437.67, it/s 0.7
  3074. [0130 17:43:58 @multigpu.py:323] [p0115] step: count(567), step_time 1483.43, mean_step_time 1434.47, it/s 0.7
  3075. [0130 17:43:58 @multigpu.py:323] [p0574] step: count(583), step_time 1436.96, mean_step_time 1454.39, it/s 0.69
  3076. [0130 17:44:00 @multigpu.py:323] [p0115] step: count(568), step_time 1544.23, mean_step_time 1438.77, it/s 0.7
  3077. [0130 17:44:00 @multigpu.py:323] [p0574] step: count(584), step_time 1433.22, mean_step_time 1438.67, it/s 0.7
  3078. [0130 17:44:00 @multigpu.py:323] [p0576] step: count(591), step_time 1869.94, mean_step_time 1438.62, it/s 0.7
  3079. [0130 17:44:01 @multigpu.py:323] [p0115] step: count(569), step_time 1396.0, mean_step_time 1437.03, it/s 0.7
  3080. [0130 17:44:01 @multigpu.py:323] [p0576] step: count(592), step_time 1410.35, mean_step_time 1440.27, it/s 0.69
  3081. [0130 17:44:01 @multigpu.py:323] [p0574] step: count(585), step_time 1446.87, mean_step_time 1436.66, it/s 0.7
  3082. sending to address tcp://p0112:61216
  3083. ##### Sending to neptune: online_score : 0.233688505226 , 1.5 #####
  3084. [u'online', 1.5]
  3085. receiving
  3086. [0130 17:44:02 @multigpu.py:323] [p0115] step: count(570), step_time 1421.33, mean_step_time 1436.73, it/s 0.7
  3087. [0130 17:44:02 @multigpu.py:323] [p0576] step: count(593), step_time 1417.78, mean_step_time 1442.56, it/s 0.69
  3088. [0130 17:44:02 @multigpu.py:323] [p0574] step: count(586), step_time 1389.72, mean_step_time 1437.4, it/s 0.7
  3089. [0130 17:44:04 @multigpu.py:323] [p0115] step: count(571), step_time 1425.36, mean_step_time 1437.45, it/s 0.7
  3090. [0130 17:44:04 @multigpu.py:323] [p0574] step: count(587), step_time 1438.96, mean_step_time 1437.19, it/s 0.7
  3091. [0130 17:44:04 @multigpu.py:323] [p0576] step: count(594), step_time 1469.7, mean_step_time 1444.38, it/s 0.69
  3092. [0130 17:44:05 @multigpu.py:323] [p0115] step: count(572), step_time 1433.1, mean_step_time 1438.03, it/s 0.7
  3093. [0130 17:44:05 @multigpu.py:323] [p0576] step: count(595), step_time 1457.12, mean_step_time 1447.06, it/s 0.69
  3094. [0130 17:44:05 @multigpu.py:323] [p0574] step: count(588), step_time 1488.52, mean_step_time 1439.57, it/s 0.69
  3095. [0130 17:44:07 @multigpu.py:323] [p0115] step: count(573), step_time 1404.14, mean_step_time 1435.56, it/s 0.7
  3096. [0130 17:44:07 @multigpu.py:323] [p0576] step: count(596), step_time 1434.74, mean_step_time 1448.6, it/s 0.69
  3097. [0130 17:44:07 @multigpu.py:323] [p0574] step: count(589), step_time 1433.57, mean_step_time 1440.02, it/s 0.69
  3098. [0130 17:44:08 @multigpu.py:323] [p0115] step: count(574), step_time 1462.73, mean_step_time 1439.67, it/s 0.69
  3099. [0130 17:44:08 @multigpu.py:323] [p0576] step: count(597), step_time 1438.55, mean_step_time 1447.92, it/s 0.69
  3100. [0130 17:44:08 @multigpu.py:323] [p0574] step: count(590), step_time 1460.88, mean_step_time 1441.52, it/s 0.69
  3101. [0130 17:44:10 @multigpu.py:323] [p0115] step: count(575), step_time 1428.03, mean_step_time 1439.32, it/s 0.69
  3102. [0130 17:44:10 @multigpu.py:323] [p0576] step: count(598), step_time 1411.28, mean_step_time 1448.04, it/s 0.69
  3103. [0130 17:44:10 @multigpu.py:323] [p0574] step: count(591), step_time 1442.06, mean_step_time 1443.3, it/s 0.69
  3104. [0130 17:44:11 @multigpu.py:323] [p0115] step: count(576), step_time 1430.0, mean_step_time 1439.74, it/s 0.69
  3105. [0130 17:44:11 @multigpu.py:323] [p0576] step: count(599), step_time 1431.67, mean_step_time 1447.66, it/s 0.69
  3106. [0130 17:44:11 @multigpu.py:323] [p0574] step: count(592), step_time 1423.58, mean_step_time 1442.34, it/s 0.69
  3107. [0130 17:44:12 @multigpu.py:323] [p0115] step: count(577), step_time 1407.09, mean_step_time 1440.3, it/s 0.69
  3108. [0130 17:44:13 @multigpu.py:323] [p0574] step: count(593), step_time 1393.41, mean_step_time 1441.87, it/s 0.69
  3109. [0130 17:44:13 @multigpu.py:323] [p0576] step: count(600), step_time 1467.88, mean_step_time 1452.01, it/s 0.69
  3110. sending debugging info...
  3111. sending to address tcp://p0112:61216
  3112. ##### Sending to neptune: mean_delay : 0.236649152173 , 0.0 #####
  3113. sending to address tcp://p0112:61216
  3114. ##### Sending to neptune: max_delay : 0.236649152173 , -0.0 #####
  3115. ##### Sending to neptune: min_delay : 0.236649152173 , -0.0 #####
  3116. [u'delays', [0.0, -0.0, -0.0]]
  3117. receiving
  3118. ##### Sending to neptune: cost : 0.236649644706 , -0.0012414510129 #####
  3119. sending to address tcp://p0112:61216
  3120. ##### Sending to neptune: policy_loss : 0.236649644706 , 0.698001503944 #####
  3121. ##### Sending to neptune: xentropy_loss : 0.236649644706 , -2.29192209244 #####
  3122. ##### Sending to neptune: value_loss : 0.236649644706 , 1.43501460552 #####
  3123. ##### Sending to neptune: advantage : 0.236649644706 , -0.00304705230519 #####
  3124. ##### Sending to neptune: pred_reward : 0.236649644706 , 0.39679774642 #####
  3125. ##### Sending to neptune: max_logit : 0.236649644706 , 0.181564688683 #####
  3126. [u'loss', -0.0012414510129019618, 0.698001503944397, -2.291922092437744, 1.4350146055221558, -0.0030470523051917553, 0.3967977464199066, 0.18156468868255615]
  3127. receiving
  3128. ##### Sending to neptune: active_relus : 0.236650113596 , 8806054.65 #####
  3129. ##### Sending to neptune: dp_per_s : 0.236650113596 , 89.2224737498 #####
  3130. [u'other', 8806054.65, 89.22247374981353]
  3131. receiving
  3132. [0130 17:44:14 @multigpu.py:323] [p0115] step: count(578), step_time 1487.14, mean_step_time 1438.3, it/s 0.7
  3133. [0130 17:44:14 @multigpu.py:323] [p0574] step: count(594), step_time 1423.42, mean_step_time 1441.53, it/s 0.69
  3134. [0130 17:44:14 @multigpu.py:323] [p0576] step: count(601), step_time 1441.36, mean_step_time 1453.19, it/s 0.69
  3135. [0130 17:44:15 @multigpu.py:323] [p0115] step: count(579), step_time 1395.8, mean_step_time 1435.48, it/s 0.7
  3136. [0130 17:44:15 @multigpu.py:323] [p0574] step: count(595), step_time 1435.73, mean_step_time 1439.13, it/s 0.69
  3137. [0130 17:44:15 @multigpu.py:323] [p0576] step: count(602), step_time 1481.95, mean_step_time 1453.44, it/s 0.69
  3138. sending to address tcp://p0112:61216
  3139. ##### Sending to neptune: online_score : 0.237544338836 , 2.1 #####
  3140. [u'online', 2.1]
  3141. receiving
  3142. [0130 17:44:17 @multigpu.py:323] [p0115] step: count(580), step_time 1488.18, mean_step_time 1434.87, it/s 0.7
  3143. [0130 17:44:17 @multigpu.py:323] [p0574] step: count(596), step_time 1435.03, mean_step_time 1436.35, it/s 0.7
  3144. [0130 17:44:17 @multigpu.py:323] [p0576] step: count(603), step_time 1463.81, mean_step_time 1454.97, it/s 0.69
  3145. [0130 17:44:18 @multigpu.py:323] [p0115] step: count(581), step_time 1418.59, mean_step_time 1432.75, it/s 0.7
  3146. [0130 17:44:18 @multigpu.py:323] [p0574] step: count(597), step_time 1469.03, mean_step_time 1435.67, it/s 0.7
  3147. [0130 17:44:18 @multigpu.py:323] [p0576] step: count(604), step_time 1420.29, mean_step_time 1457.42, it/s 0.69
  3148. [0130 17:44:20 @multigpu.py:323] [p0115] step: count(582), step_time 1458.28, mean_step_time 1436.31, it/s 0.7
  3149. [0130 17:44:20 @multigpu.py:323] [p0574] step: count(598), step_time 1442.36, mean_step_time 1434.58, it/s 0.7
  3150. [0130 17:44:20 @multigpu.py:323] [p0576] step: count(605), step_time 1400.04, mean_step_time 1455.02, it/s 0.69
  3151. sending to address tcp://p0112:61216
  3152. ##### Sending to neptune: online_score : 0.238823784126 , 1.4 #####
  3153. [u'online', 1.4]
  3154. receiving
  3155. [0130 17:44:21 @multigpu.py:323] [p0574] step: count(599), step_time 1400.31, mean_step_time 1433.08, it/s 0.7
  3156. [0130 17:44:21 @multigpu.py:323] [p0115] step: count(583), step_time 1498.62, mean_step_time 1438.1, it/s 0.7
  3157. [0130 17:44:21 @multigpu.py:323] [p0576] step: count(606), step_time 1421.09, mean_step_time 1453.86, it/s 0.69
  3158. [0130 17:44:23 @multigpu.py:323] [p0115] step: count(584), step_time 1412.91, mean_step_time 1438.97, it/s 0.69
  3159. [0130 17:44:23 @multigpu.py:323] [p0576] step: count(607), step_time 1404.31, mean_step_time 1452.04, it/s 0.69
  3160. [0130 17:44:23 @multigpu.py:323] [p0574] step: count(600), step_time 1464.53, mean_step_time 1435.97, it/s 0.7
  3161. sending debugging info...
  3162. sending to address tcp://p0112:61216
  3163. ##### Sending to neptune: mean_delay : 0.239441586071 , 0.0 #####
  3164. ##### Sending to neptune: max_delay : 0.239441586071 , -0.0 #####
  3165. sending to address tcp://p0112:61216
  3166. ##### Sending to neptune: min_delay : 0.239441586071 , -0.0 #####
  3167. [u'delays', [0.0, -0.0, -0.0]]
  3168. receiving
  3169. ##### Sending to neptune: cost : 0.239442152182 , -0.0125148463994 #####
  3170. ##### Sending to neptune: policy_loss : 0.239442152182 , -0.49882593751 #####
  3171. sending to address tcp://p0112:61216
  3172. ##### Sending to neptune: xentropy_loss : 0.239442152182 , -2.29190254211 #####
  3173. ##### Sending to neptune: value_loss : 0.239442152182 , 1.18882787228 #####
  3174. ##### Sending to neptune: advantage : 0.239442152182 , 0.00211322447285 #####
  3175. ##### Sending to neptune: pred_reward : 0.239442152182 , 0.398575603962 #####
  3176. ##### Sending to neptune: max_logit : 0.239442152182 , 0.181839182973 #####
  3177. [u'loss', -0.01251484639942646, -0.49882593750953674, -2.291902542114258, 1.1888278722763062, 0.002113224472850561, 0.3985756039619446, 0.18183918297290802]
  3178. receiving
  3179. ##### Sending to neptune: active_relus : 0.239442588554 , 8800469.51 #####
  3180. ##### Sending to neptune: dp_per_s : 0.239442588554 , 89.0259561405 #####
  3181. [u'other', 8800469.51, 89.02595614054479]
  3182. receiving
  3183. [0130 17:44:24 @multigpu.py:323] [p0115] step: count(585), step_time 1430.45, mean_step_time 1440.72, it/s 0.69
  3184. [0130 17:44:24 @multigpu.py:323] [p0576] step: count(608), step_time 1455.01, mean_step_time 1456.26, it/s 0.69
  3185. [0130 17:44:24 @multigpu.py:323] [p0574] step: count(601), step_time 1447.28, mean_step_time 1436.48, it/s 0.7
  3186. [0130 17:44:25 @multigpu.py:323] [p0115] step: count(586), step_time 1403.09, mean_step_time 1441.43, it/s 0.69
  3187. [0130 17:44:25 @multigpu.py:323] [p0574] step: count(602), step_time 1389.81, mean_step_time 1434.76, it/s 0.7
  3188. [0130 17:44:25 @multigpu.py:323] [p0576] step: count(609), step_time 1459.53, mean_step_time 1458.13, it/s 0.69
  3189. sending to address tcp://p0112:61216
  3190. ##### Sending to neptune: online_score : 0.240421581931 , 1.6 #####
  3191. [u'online', 1.6]
  3192. receiving
  3193. [0130 17:44:27 @multigpu.py:323] [p0574] step: count(603), step_time 1412.17, mean_step_time 1433.52, it/s 0.7
  3194. [0130 17:44:27 @multigpu.py:323] [p0115] step: count(587), step_time 1454.29, mean_step_time 1439.97, it/s 0.69
  3195. [0130 17:44:27 @multigpu.py:323] [p0576] step: count(610), step_time 1377.06, mean_step_time 1456.67, it/s 0.69
  3196. [0130 17:44:28 @multigpu.py:323] [p0115] step: count(588), step_time 1469.67, mean_step_time 1436.24, it/s 0.7
  3197. [0130 17:44:28 @multigpu.py:323] [p0574] step: count(604), step_time 1487.58, mean_step_time 1436.24, it/s 0.7
  3198. [0130 17:44:28 @multigpu.py:323] [p0576] step: count(611), step_time 1462.63, mean_step_time 1436.31, it/s 0.7
  3199. [0130 17:44:30 @multigpu.py:323] [p0574] step: count(605), step_time 1389.52, mean_step_time 1433.37, it/s 0.7
  3200. [0130 17:44:30 @multigpu.py:323] [p0115] step: count(589), step_time 1390.94, mean_step_time 1435.99, it/s 0.7
  3201. [0130 17:44:30 @multigpu.py:323] [p0576] step: count(612), step_time 1450.4, mean_step_time 1438.31, it/s 0.7
  3202. [0130 17:44:31 @multigpu.py:323] [p0576] step: count(613), step_time 1381.31, mean_step_time 1436.49, it/s 0.7
  3203. [0130 17:44:31 @multigpu.py:323] [p0574] step: count(606), step_time 1463.87, mean_step_time 1437.08, it/s 0.7
  3204. [0130 17:44:31 @multigpu.py:323] [p0115] step: count(590), step_time 1475.51, mean_step_time 1438.7, it/s 0.7
  3205. [0130 17:44:33 @multigpu.py:323] [p0576] step: count(614), step_time 1400.85, mean_step_time 1433.04, it/s 0.7
  3206. [0130 17:44:33 @multigpu.py:323] [p0115] step: count(591), step_time 1427.56, mean_step_time 1438.81, it/s 0.7
  3207. [0130 17:44:33 @multigpu.py:323] [p0574] step: count(607), step_time 1496.17, mean_step_time 1439.94, it/s 0.69
  3208. [0130 17:44:34 @multigpu.py:323] [p0576] step: count(615), step_time 1415.65, mean_step_time 1430.97, it/s 0.7
  3209. [0130 17:44:34 @multigpu.py:323] [p0115] step: count(592), step_time 1362.75, mean_step_time 1435.29, it/s 0.7
  3210. [0130 17:44:34 @multigpu.py:323] [p0574] step: count(608), step_time 1415.51, mean_step_time 1436.29, it/s 0.7
  3211. [0130 17:44:35 @multigpu.py:323] [p0576] step: count(616), step_time 1422.25, mean_step_time 1430.35, it/s 0.7
  3212. [0130 17:44:35 @multigpu.py:323] [p0115] step: count(593), step_time 1428.81, mean_step_time 1436.52, it/s 0.7
  3213. [0130 17:44:36 @multigpu.py:323] [p0574] step: count(609), step_time 1479.88, mean_step_time 1438.61, it/s 0.7
  3214. [0130 17:44:37 @multigpu.py:323] [p0576] step: count(617), step_time 1405.63, mean_step_time 1428.7, it/s 0.7
  3215. [0130 17:44:37 @multigpu.py:323] [p0115] step: count(594), step_time 1480.72, mean_step_time 1437.42, it/s 0.7
  3216. [0130 17:44:37 @multigpu.py:323] [p0574] step: count(610), step_time 1439.04, mean_step_time 1437.52, it/s 0.7
  3217. [0130 17:44:38 @multigpu.py:323] [p0576] step: count(618), step_time 1453.8, mean_step_time 1430.83, it/s 0.7
  3218. [0130 17:44:38 @multigpu.py:323] [p0115] step: count(595), step_time 1421.83, mean_step_time 1437.11, it/s 0.7
  3219. [0130 17:44:38 @multigpu.py:323] [p0574] step: count(611), step_time 1399.35, mean_step_time 1435.38, it/s 0.7
  3220. [0130 17:44:40 @multigpu.py:323] [p0576] step: count(619), step_time 1407.74, mean_step_time 1429.63, it/s 0.7
  3221. [0130 17:44:40 @multigpu.py:323] [p0115] step: count(596), step_time 1425.51, mean_step_time 1436.89, it/s 0.7
  3222. [0130 17:44:40 @multigpu.py:323] [p0574] step: count(612), step_time 1478.83, mean_step_time 1438.14, it/s 0.7
  3223. [0130 17:44:41 @multigpu.py:323] [p0576] step: count(620), step_time 1445.59, mean_step_time 1428.51, it/s 0.7
  3224. [0130 17:44:41 @multigpu.py:323] [p0115] step: count(597), step_time 1423.78, mean_step_time 1437.72, it/s 0.7
  3225. [0130 17:44:41 @multigpu.py:323] [p0574] step: count(613), step_time 1408.85, mean_step_time 1438.91, it/s 0.69
  3226. [0130 17:44:43 @multigpu.py:323] [p0576] step: count(621), step_time 1422.57, mean_step_time 1427.57, it/s 0.7
  3227. [0130 17:44:43 @multigpu.py:323] [p0115] step: count(598), step_time 1433.0, mean_step_time 1435.02, it/s 0.7
  3228. [0130 17:44:43 @multigpu.py:323] [p0574] step: count(614), step_time 1403.31, mean_step_time 1437.91, it/s 0.7
  3229. [0130 17:44:44 @multigpu.py:323] [p0576] step: count(622), step_time 1411.85, mean_step_time 1424.07, it/s 0.7
  3230. [0130 17:44:44 @multigpu.py:323] [p0115] step: count(599), step_time 1452.46, mean_step_time 1437.85, it/s 0.7
  3231. [0130 17:44:44 @multigpu.py:323] [p0574] step: count(615), step_time 1476.66, mean_step_time 1439.95, it/s 0.69
  3232. [0130 17:44:45 @multigpu.py:323] [p0576] step: count(623), step_time 1407.37, mean_step_time 1421.25, it/s 0.7
  3233. [0130 17:44:45 @multigpu.py:323] [p0115] step: count(600), step_time 1399.95, mean_step_time 1433.44, it/s 0.7
  3234. sending debugging info...
  3235. sending to address tcp://p0112:61216
  3236. ##### Sending to neptune: mean_delay : 0.24579519444 , 0.0 #####
  3237. sending to address tcp://p0112:61216
  3238. ##### Sending to neptune: max_delay : 0.24579519444 , -0.0 #####
  3239. ##### Sending to neptune: min_delay : 0.24579519444 , -0.0 #####
  3240. [u'delays', [0.0, -0.0, -0.0]]
  3241. receiving
  3242. ##### Sending to neptune: cost : 0.245795726644 , -0.0124126961455 #####
  3243. ##### Sending to neptune: policy_loss : 0.245795726644 , -0.532102823257 #####
  3244. sending to address tcp://p0112:61216
  3245. ##### Sending to neptune: xentropy_loss : 0.245795726644 , -2.29180145264 #####
  3246. ##### Sending to neptune: value_loss : 0.245795726644 , 1.23507893085 #####
  3247. ##### Sending to neptune: advantage : 0.245795726644 , 0.00228149211034 #####
  3248. ##### Sending to neptune: pred_reward : 0.245795726644 , 0.404466867447 #####
  3249. ##### Sending to neptune: max_logit : 0.245795726644 , 0.183123975992 #####
  3250. [u'loss', -0.01241269614547491, -0.5321028232574463, -2.2918014526367188, 1.2350789308547974, 0.0022814921103417873, 0.4044668674468994, 0.18312397599220276]
  3251. receiving
  3252. ##### Sending to neptune: active_relus : 0.245796224409 , 8822338.12 #####
  3253. ##### Sending to neptune: dp_per_s : 0.245796224409 , 89.0029782201 #####
  3254. [u'other', 8822338.12, 89.00297822013982]
  3255. receiving
  3256. [0130 17:44:46 @multigpu.py:323] [p0574] step: count(616), step_time 1425.14, mean_step_time 1439.46, it/s 0.69
  3257. [0130 17:44:47 @multigpu.py:323] [p0576] step: count(624), step_time 1462.41, mean_step_time 1423.35, it/s 0.7
  3258. [0130 17:44:47 @multigpu.py:323] [p0115] step: count(601), step_time 1463.34, mean_step_time 1435.68, it/s 0.7
  3259. [0130 17:44:47 @multigpu.py:323] [p0574] step: count(617), step_time 1418.75, mean_step_time 1436.95, it/s 0.7
  3260. [0130 17:44:48 @multigpu.py:323] [p0576] step: count(625), step_time 1426.03, mean_step_time 1424.65, it/s 0.7
  3261. [0130 17:44:48 @multigpu.py:323] [p0115] step: count(602), step_time 1403.93, mean_step_time 1432.96, it/s 0.7
  3262. [0130 17:44:48 @multigpu.py:323] [p0574] step: count(618), step_time 1393.44, mean_step_time 1434.5, it/s 0.7
  3263. sending to address tcp://p0112:61216
  3264. ##### Sending to neptune: online_score : 0.246657715506 , 1.4 #####
  3265. [u'online', 1.4]
  3266. receiving
  3267. sending to address tcp://p0112:61216
  3268. ##### Sending to neptune: online_score : 0.246721496052 , 1.6 #####
  3269. [u'online', 1.6]
  3270. receiving
  3271. [0130 17:44:50 @multigpu.py:323] [p0576] step: count(626), step_time 1402.71, mean_step_time 1423.73, it/s 0.7
  3272. [0130 17:44:50 @multigpu.py:323] [p0115] step: count(603), step_time 1429.02, mean_step_time 1429.48, it/s 0.7
  3273. [0130 17:44:50 @multigpu.py:323] [p0574] step: count(619), step_time 1492.01, mean_step_time 1439.09, it/s 0.69
  3274. sending to address tcp://p0112:61216
  3275. ##### Sending to neptune: online_score : 0.247186529703 , 0.6 #####
  3276. [u'online', 0.6]
  3277. receiving
  3278. [0130 17:44:51 @multigpu.py:323] [p0576] step: count(627), step_time 1407.93, mean_step_time 1423.92, it/s 0.7
  3279. [0130 17:44:51 @multigpu.py:323] [p0115] step: count(604), step_time 1412.04, mean_step_time 1429.43, it/s 0.7
  3280. [0130 17:44:51 @multigpu.py:323] [p0574] step: count(620), step_time 1405.33, mean_step_time 1436.13, it/s 0.7
  3281. [0130 17:44:52 @multigpu.py:323] [p0576] step: count(628), step_time 1390.03, mean_step_time 1420.67, it/s 0.7
  3282. [0130 17:44:53 @multigpu.py:323] [p0115] step: count(605), step_time 1426.99, mean_step_time 1429.26, it/s 0.7
  3283. [0130 17:44:53 @multigpu.py:323] [p0574] step: count(621), step_time 1503.47, mean_step_time 1438.93, it/s 0.69
  3284. [0130 17:44:54 @multigpu.py:323] [p0576] step: count(629), step_time 1378.84, mean_step_time 1416.63, it/s 0.71
  3285. [0130 17:44:54 @multigpu.py:323] [p0115] step: count(606), step_time 1481.85, mean_step_time 1433.2, it/s 0.7
  3286. [0130 17:44:54 @multigpu.py:323] [p0574] step: count(622), step_time 1415.95, mean_step_time 1440.24, it/s 0.69
  3287. [0130 17:44:55 @multigpu.py:323] [p0576] step: count(630), step_time 1494.85, mean_step_time 1422.52, it/s 0.7
  3288. [0130 17:44:55 @multigpu.py:323] [p0115] step: count(607), step_time 1399.97, mean_step_time 1430.48, it/s 0.7
  3289. [0130 17:44:56 @multigpu.py:323] [p0574] step: count(623), step_time 1419.95, mean_step_time 1440.63, it/s 0.69
  3290. [0130 17:44:57 @multigpu.py:323] [p0115] step: count(608), step_time 1606.73, mean_step_time 1437.34, it/s 0.7
  3291. [0130 17:44:57 @multigpu.py:323] [p0574] step: count(624), step_time 1424.07, mean_step_time 1437.46, it/s 0.7
  3292. [0130 17:44:57 @multigpu.py:323] [p0576] step: count(631), step_time 1761.94, mean_step_time 1437.49, it/s 0.7
  3293. [0130 17:44:59 @multigpu.py:323] [p0576] step: count(632), step_time 1410.12, mean_step_time 1435.47, it/s 0.7
  3294. [0130 17:44:59 @multigpu.py:323] [p0115] step: count(609), step_time 1436.63, mean_step_time 1439.62, it/s 0.69
  3295. [0130 17:44:59 @multigpu.py:323] [p0574] step: count(625), step_time 1486.65, mean_step_time 1442.31, it/s 0.69
  3296. [0130 17:45:00 @multigpu.py:323] [p0576] step: count(633), step_time 1481.32, mean_step_time 1440.47, it/s 0.69
  3297. [0130 17:45:00 @multigpu.py:323] [p0115] step: count(610), step_time 1477.83, mean_step_time 1439.74, it/s 0.69
  3298. [0130 17:45:00 @multigpu.py:323] [p0574] step: count(626), step_time 1486.27, mean_step_time 1443.43, it/s 0.69
  3299. [0130 17:45:01 @multigpu.py:323] [p0115] step: count(611), step_time 1405.17, mean_step_time 1438.62, it/s 0.7
  3300. [0130 17:45:01 @multigpu.py:323] [p0576] step: count(634), step_time 1428.84, mean_step_time 1441.87, it/s 0.69
  3301. [0130 17:45:01 @multigpu.py:323] [p0574] step: count(627), step_time 1387.93, mean_step_time 1438.02, it/s 0.7
  3302. [0130 17:45:03 @multigpu.py:323] [p0115] step: count(612), step_time 1414.92, mean_step_time 1441.22, it/s 0.69
  3303. [0130 17:45:03 @multigpu.py:323] [p0574] step: count(628), step_time 1438.6, mean_step_time 1439.17, it/s 0.69
  3304. [0130 17:45:03 @multigpu.py:323] [p0576] step: count(635), step_time 1515.42, mean_step_time 1446.86, it/s 0.69
  3305. [0130 17:45:04 @multigpu.py:323] [p0115] step: count(613), step_time 1465.02, mean_step_time 1443.03, it/s 0.69
  3306. [0130 17:45:04 @multigpu.py:323] [p0576] step: count(636), step_time 1393.03, mean_step_time 1445.4, it/s 0.69
  3307. [0130 17:45:04 @multigpu.py:323] [p0574] step: count(629), step_time 1468.57, mean_step_time 1438.61, it/s 0.7
  3308. [0130 17:45:06 @multigpu.py:323] [p0115] step: count(614), step_time 1453.47, mean_step_time 1441.67, it/s 0.69
  3309. [0130 17:45:06 @multigpu.py:323] [p0576] step: count(637), step_time 1430.3, mean_step_time 1446.63, it/s 0.69
  3310. [0130 17:45:06 @multigpu.py:323] [p0574] step: count(630), step_time 1414.37, mean_step_time 1437.38, it/s 0.7
  3311. [0130 17:45:07 @multigpu.py:323] [p0574] step: count(631), step_time 1374.32, mean_step_time 1436.12, it/s 0.7
  3312. [0130 17:45:07 @multigpu.py:323] [p0576] step: count(638), step_time 1400.2, mean_step_time 1443.95, it/s 0.69
  3313. [0130 17:45:07 @multigpu.py:323] [p0115] step: count(615), step_time 1432.96, mean_step_time 1442.23, it/s 0.69
  3314. [0130 17:45:09 @multigpu.py:323] [p0576] step: count(639), step_time 1464.58, mean_step_time 1446.8, it/s 0.69
  3315. [0130 17:45:09 @multigpu.py:323] [p0574] step: count(632), step_time 1482.66, mean_step_time 1436.32, it/s 0.7
  3316. [0130 17:45:09 @multigpu.py:323] [p0115] step: count(616), step_time 1473.6, mean_step_time 1444.63, it/s 0.69
  3317. [0130 17:45:10 @multigpu.py:323] [p0574] step: count(633), step_time 1401.24, mean_step_time 1435.94, it/s 0.7
  3318. [0130 17:45:10 @multigpu.py:323] [p0115] step: count(617), step_time 1406.4, mean_step_time 1443.76, it/s 0.69
  3319. [0130 17:45:10 @multigpu.py:323] [p0576] step: count(640), step_time 1458.48, mean_step_time 1447.44, it/s 0.69
  3320. [0130 17:45:11 @multigpu.py:323] [p0115] step: count(618), step_time 1410.03, mean_step_time 1442.61, it/s 0.69
  3321. [0130 17:45:11 @multigpu.py:323] [p0574] step: count(634), step_time 1454.65, mean_step_time 1438.5, it/s 0.7
  3322. [0130 17:45:12 @multigpu.py:323] [p0576] step: count(641), step_time 1431.78, mean_step_time 1447.9, it/s 0.69
  3323. [0130 17:45:13 @multigpu.py:323] [p0115] step: count(619), step_time 1410.05, mean_step_time 1440.49, it/s 0.69
  3324. [0130 17:45:13 @multigpu.py:323] [p0576] step: count(642), step_time 1456.59, mean_step_time 1450.14, it/s 0.69
  3325. [0130 17:45:13 @multigpu.py:323] [p0574] step: count(635), step_time 1477.97, mean_step_time 1438.57, it/s 0.7
  3326. [0130 17:45:14 @multigpu.py:323] [p0115] step: count(620), step_time 1455.96, mean_step_time 1443.29, it/s 0.69
  3327. [0130 17:45:14 @multigpu.py:323] [p0576] step: count(643), step_time 1389.1, mean_step_time 1449.22, it/s 0.69
  3328. [0130 17:45:14 @multigpu.py:323] [p0574] step: count(636), step_time 1431.19, mean_step_time 1438.87, it/s 0.69
  3329. sending to address tcp://p0112:61216
  3330. ##### Sending to neptune: online_score : 0.25393662552 , 1.5 #####
  3331. [u'online', 1.5]
  3332. ##### Sending to neptune: active_workers : 0.253936661614 , 3 #####
  3333. receiving
  3334. sending to address tcp://p0112:61216
  3335. ##### Sending to neptune: online_score : 0.253976651099 , 1.5 #####
  3336. [u'online', 1.5]
  3337. receiving
  3338. [0130 17:45:16 @multigpu.py:323] [p0115] step: count(621), step_time 1374.5, mean_step_time 1438.85, it/s 0.69
  3339. [0130 17:45:16 @multigpu.py:323] [p0576] step: count(644), step_time 1412.07, mean_step_time 1446.71, it/s 0.69
  3340. [0130 17:45:16 @multigpu.py:323] [p0574] step: count(637), step_time 1401.14, mean_step_time 1437.99, it/s 0.7
  3341. [0130 17:45:17 @multigpu.py:323] [p0115] step: count(622), step_time 1455.7, mean_step_time 1441.44, it/s 0.69
  3342. [0130 17:45:17 @multigpu.py:323] [p0574] step: count(638), step_time 1388.99, mean_step_time 1437.77, it/s 0.7
  3343. [0130 17:45:17 @multigpu.py:323] [p0576] step: count(645), step_time 1450.71, mean_step_time 1447.94, it/s 0.69
  3344. [0130 17:45:19 @multigpu.py:323] [p0576] step: count(646), step_time 1429.79, mean_step_time 1449.3, it/s 0.69
  3345. [0130 17:45:19 @multigpu.py:323] [p0115] step: count(623), step_time 1483.14, mean_step_time 1444.15, it/s 0.69
  3346. [0130 17:45:19 @multigpu.py:323] [p0574] step: count(639), step_time 1500.31, mean_step_time 1438.18, it/s 0.7
  3347. sending to address tcp://p0112:61216
  3348. ##### Sending to neptune: online_score : 0.255228105783 , 1.5 #####
  3349. [u'online', 1.5]
  3350. receiving
  3351. [0130 17:45:20 @multigpu.py:323] [p0115] step: count(624), step_time 1400.48, mean_step_time 1443.57, it/s 0.69
  3352. [0130 17:45:20 @multigpu.py:323] [p0576] step: count(647), step_time 1429.17, mean_step_time 1450.36, it/s 0.69
  3353. [0130 17:45:20 @multigpu.py:323] [p0574] step: count(640), step_time 1431.48, mean_step_time 1439.49, it/s 0.69
  3354. [0130 17:45:21 @multigpu.py:323] [p0115] step: count(625), step_time 1438.56, mean_step_time 1444.15, it/s 0.69
  3355. [0130 17:45:22 @multigpu.py:323] [p0574] step: count(641), step_time 1377.16, mean_step_time 1433.17, it/s 0.7
  3356. [0130 17:45:22 @multigpu.py:323] [p0576] step: count(648), step_time 1458.03, mean_step_time 1453.76, it/s 0.69
  3357. [0130 17:45:23 @multigpu.py:323] [p0574] step: count(642), step_time 1409.73, mean_step_time 1432.86, it/s 0.7
  3358. [0130 17:45:23 @multigpu.py:323] [p0115] step: count(626), step_time 1456.32, mean_step_time 1442.87, it/s 0.69
  3359. [0130 17:45:23 @multigpu.py:323] [p0576] step: count(649), step_time 1492.76, mean_step_time 1459.45, it/s 0.69
  3360. [0130 17:45:24 @multigpu.py:323] [p0574] step: count(643), step_time 1467.67, mean_step_time 1435.25, it/s 0.7
  3361. [0130 17:45:24 @multigpu.py:323] [p0115] step: count(627), step_time 1441.65, mean_step_time 1444.96, it/s 0.69
  3362. [0130 17:45:24 @multigpu.py:323] [p0576] step: count(650), step_time 1411.33, mean_step_time 1455.28, it/s 0.69
  3363. [0130 17:45:26 @multigpu.py:323] [p0115] step: count(628), step_time 1539.64, mean_step_time 1441.6, it/s 0.69
  3364. [0130 17:45:26 @multigpu.py:323] [p0576] step: count(651), step_time 1488.15, mean_step_time 1441.59, it/s 0.69
  3365. [0130 17:45:26 @multigpu.py:323] [p0574] step: count(644), step_time 1551.55, mean_step_time 1441.62, it/s 0.69
  3366. [0130 17:45:27 @multigpu.py:323] [p0576] step: count(652), step_time 1392.0, mean_step_time 1440.68, it/s 0.69
  3367. [0130 17:45:27 @multigpu.py:323] [p0115] step: count(629), step_time 1405.88, mean_step_time 1440.06, it/s 0.69
  3368. [0130 17:45:27 @multigpu.py:323] [p0574] step: count(645), step_time 1471.01, mean_step_time 1440.84, it/s 0.69
  3369. [0130 17:45:29 @multigpu.py:323] [p0576] step: count(653), step_time 1416.31, mean_step_time 1437.43, it/s 0.7
  3370. [0130 17:45:29 @multigpu.py:323] [p0115] step: count(630), step_time 1429.28, mean_step_time 1437.64, it/s 0.7
  3371. [0130 17:45:29 @multigpu.py:323] [p0574] step: count(646), step_time 1418.72, mean_step_time 1437.46, it/s 0.7
  3372. [0130 17:45:30 @multigpu.py:323] [p0576] step: count(654), step_time 1443.74, mean_step_time 1438.18, it/s 0.7
  3373. [0130 17:45:30 @multigpu.py:323] [p0115] step: count(631), step_time 1452.88, mean_step_time 1440.02, it/s 0.69
  3374. [0130 17:45:30 @multigpu.py:323] [p0574] step: count(647), step_time 1431.32, mean_step_time 1439.63, it/s 0.69
  3375. [0130 17:45:32 @multigpu.py:323] [p0576] step: count(655), step_time 1400.57, mean_step_time 1432.44, it/s 0.7
  3376. [0130 17:45:32 @multigpu.py:323] [p0115] step: count(632), step_time 1430.96, mean_step_time 1440.82, it/s 0.69
  3377. [0130 17:45:32 @multigpu.py:323] [p0574] step: count(648), step_time 1474.59, mean_step_time 1441.43, it/s 0.69
  3378. [0130 17:45:33 @multigpu.py:323] [p0576] step: count(656), step_time 1432.96, mean_step_time 1434.43, it/s 0.7
  3379. [0130 17:45:33 @multigpu.py:323] [p0115] step: count(633), step_time 1431.99, mean_step_time 1439.17, it/s 0.69
  3380. [0130 17:45:33 @multigpu.py:323] [p0574] step: count(649), step_time 1383.33, mean_step_time 1437.17, it/s 0.7
  3381. [0130 17:45:34 @multigpu.py:323] [p0576] step: count(657), step_time 1400.38, mean_step_time 1432.94, it/s 0.7
  3382. [0130 17:45:35 @multigpu.py:323] [p0574] step: count(650), step_time 1413.86, mean_step_time 1437.14, it/s 0.7
  3383. [0130 17:45:35 @multigpu.py:323] [p0115] step: count(634), step_time 1454.74, mean_step_time 1439.24, it/s 0.69
  3384. [0130 17:45:36 @multigpu.py:323] [p0576] step: count(658), step_time 1378.2, mean_step_time 1431.84, it/s 0.7
  3385. [0130 17:45:36 @multigpu.py:323] [p0574] step: count(651), step_time 1426.33, mean_step_time 1439.74, it/s 0.69
  3386. [0130 17:45:36 @multigpu.py:323] [p0115] step: count(635), step_time 1506.58, mean_step_time 1442.92, it/s 0.69
  3387. [0130 17:45:37 @multigpu.py:323] [p0576] step: count(659), step_time 1411.7, mean_step_time 1429.19, it/s 0.7
  3388. [0130 17:45:37 @multigpu.py:323] [p0574] step: count(652), step_time 1434.51, mean_step_time 1437.34, it/s 0.7
  3389. [0130 17:45:38 @multigpu.py:323] [p0115] step: count(636), step_time 1487.17, mean_step_time 1443.59, it/s 0.69
  3390. [0130 17:45:39 @multigpu.py:323] [p0576] step: count(660), step_time 1500.11, mean_step_time 1431.27, it/s 0.7
  3391. [0130 17:45:39 @multigpu.py:323] [p0574] step: count(653), step_time 1401.87, mean_step_time 1437.37, it/s 0.7
  3392. [0130 17:45:39 @multigpu.py:323] [p0115] step: count(637), step_time 1403.42, mean_step_time 1443.45, it/s 0.69
  3393. sending to address tcp://p0112:61216
  3394. ##### Sending to neptune: online_score : 0.260792402493 , 1.3 #####
  3395. [u'online', 1.3]
  3396. receiving
  3397. [0130 17:45:40 @multigpu.py:323] [p0576] step: count(661), step_time 1437.46, mean_step_time 1431.56, it/s 0.7
  3398. [0130 17:45:40 @multigpu.py:323] [p0574] step: count(654), step_time 1417.83, mean_step_time 1435.53, it/s 0.7
  3399. [0130 17:45:40 @multigpu.py:323] [p0115] step: count(638), step_time 1405.06, mean_step_time 1443.2, it/s 0.69
  3400. [0130 17:45:42 @multigpu.py:323] [p0576] step: count(662), step_time 1407.57, mean_step_time 1429.11, it/s 0.7
  3401. [0130 17:45:42 @multigpu.py:323] [p0574] step: count(655), step_time 1442.15, mean_step_time 1433.74, it/s 0.7
  3402. [0130 17:45:42 @multigpu.py:323] [p0115] step: count(639), step_time 1439.66, mean_step_time 1444.68, it/s 0.69
  3403. sending to address tcp://p0112:61216
  3404. ##### Sending to neptune: online_score : 0.261552384694 , 1.6 #####
  3405. [u'online', 1.6]
  3406. receiving
  3407. [0130 17:45:43 @multigpu.py:323] [p0576] step: count(663), step_time 1436.11, mean_step_time 1431.46, it/s 0.7
  3408. [0130 17:45:43 @multigpu.py:323] [p0574] step: count(656), step_time 1442.82, mean_step_time 1434.32, it/s 0.7
  3409. [0130 17:45:43 @multigpu.py:323] [p0115] step: count(640), step_time 1440.04, mean_step_time 1443.88, it/s 0.69
  3410. [0130 17:45:44 @multigpu.py:323] [p0576] step: count(664), step_time 1388.96, mean_step_time 1430.3, it/s 0.7
  3411. [0130 17:45:45 @multigpu.py:323] [p0574] step: count(657), step_time 1426.29, mean_step_time 1435.58, it/s 0.7
  3412. [0130 17:45:45 @multigpu.py:323] [p0115] step: count(641), step_time 1479.51, mean_step_time 1449.13, it/s 0.69
  3413. [0130 17:45:46 @multigpu.py:323] [p0576] step: count(665), step_time 1508.77, mean_step_time 1433.2, it/s 0.7
  3414. [0130 17:45:46 @multigpu.py:323] [p0574] step: count(658), step_time 1395.82, mean_step_time 1435.92, it/s 0.7
  3415. [0130 17:45:46 @multigpu.py:323] [p0115] step: count(642), step_time 1406.01, mean_step_time 1446.65, it/s 0.69
  3416. [0130 17:45:47 @multigpu.py:323] [p0576] step: count(666), step_time 1405.08, mean_step_time 1431.97, it/s 0.7
  3417. [0130 17:45:47 @multigpu.py:323] [p0574] step: count(659), step_time 1418.44, mean_step_time 1431.82, it/s 0.7
  3418. [0130 17:45:48 @multigpu.py:323] [p0115] step: count(643), step_time 1422.24, mean_step_time 1443.6, it/s 0.69
  3419. sending to address tcp://p0112:61216
  3420. ##### Sending to neptune: online_score : 0.263147509164 , 1.8 #####
  3421. [u'online', 1.8]
  3422. receiving
  3423. [0130 17:45:49 @multigpu.py:323] [p0576] step: count(667), step_time 1399.86, mean_step_time 1430.5, it/s 0.7
  3424. [0130 17:45:49 @multigpu.py:323] [p0574] step: count(660), step_time 1423.0, mean_step_time 1431.4, it/s 0.7
  3425. [0130 17:45:49 @multigpu.py:323] [p0115] step: count(644), step_time 1418.9, mean_step_time 1444.52, it/s 0.69
  3426. [0130 17:45:50 @multigpu.py:323] [p0576] step: count(668), step_time 1365.71, mean_step_time 1425.89, it/s 0.7
  3427. [0130 17:45:50 @multigpu.py:323] [p0574] step: count(661), step_time 1455.74, mean_step_time 1435.33, it/s 0.7
  3428. [0130 17:45:50 @multigpu.py:323] [p0115] step: count(645), step_time 1434.67, mean_step_time 1444.33, it/s 0.69
  3429. [0130 17:45:52 @multigpu.py:323] [p0576] step: count(669), step_time 1467.99, mean_step_time 1424.65, it/s 0.7
  3430. [0130 17:45:52 @multigpu.py:323] [p0574] step: count(662), step_time 1431.18, mean_step_time 1436.4, it/s 0.7
  3431. [0130 17:45:52 @multigpu.py:323] [p0115] step: count(646), step_time 1457.55, mean_step_time 1444.39, it/s 0.69
  3432. [0130 17:45:53 @multigpu.py:323] [p0576] step: count(670), step_time 1433.2, mean_step_time 1425.74, it/s 0.7
  3433. [0130 17:45:53 @multigpu.py:323] [p0574] step: count(663), step_time 1384.25, mean_step_time 1432.23, it/s 0.7
  3434. [0130 17:45:53 @multigpu.py:323] [p0115] step: count(647), step_time 1400.16, mean_step_time 1442.32, it/s 0.69
  3435. [0130 17:45:55 @multigpu.py:323] [p0115] step: count(648), step_time 1497.96, mean_step_time 1440.23, it/s 0.69
  3436. [0130 17:45:55 @multigpu.py:323] [p0574] step: count(664), step_time 1712.16, mean_step_time 1440.26, it/s 0.69
  3437. [0130 17:45:55 @multigpu.py:323] [p0576] step: count(671), step_time 1778.23, mean_step_time 1440.25, it/s 0.69
  3438. [0130 17:45:56 @multigpu.py:323] [p0574] step: count(665), step_time 1379.79, mean_step_time 1435.7, it/s 0.7
  3439. [0130 17:45:56 @multigpu.py:323] [p0576] step: count(672), step_time 1433.5, mean_step_time 1442.32, it/s 0.69
  3440. [0130 17:45:56 @multigpu.py:323] [p0115] step: count(649), step_time 1457.17, mean_step_time 1442.8, it/s 0.69
  3441. [0130 17:45:58 @multigpu.py:323] [p0576] step: count(673), step_time 1417.96, mean_step_time 1442.4, it/s 0.69
  3442. [0130 17:45:58 @multigpu.py:323] [p0574] step: count(666), step_time 1481.13, mean_step_time 1438.82, it/s 0.7
  3443. [0130 17:45:58 @multigpu.py:323] [p0115] step: count(650), step_time 1437.6, mean_step_time 1443.21, it/s 0.69
  3444. [0130 17:45:59 @multigpu.py:323] [p0574] step: count(667), step_time 1362.51, mean_step_time 1435.38, it/s 0.7
  3445. [0130 17:45:59 @multigpu.py:323] [p0576] step: count(674), step_time 1432.19, mean_step_time 1441.83, it/s 0.69
  3446. [0130 17:45:59 @multigpu.py:323] [p0115] step: count(651), step_time 1409.93, mean_step_time 1441.07, it/s 0.69
  3447. [0130 17:46:00 @multigpu.py:323] [p0574] step: count(668), step_time 1451.17, mean_step_time 1434.21, it/s 0.7
  3448. [0130 17:46:00 @multigpu.py:323] [p0115] step: count(652), step_time 1423.18, mean_step_time 1440.68, it/s 0.69
  3449. [0130 17:46:01 @multigpu.py:323] [p0576] step: count(675), step_time 1480.5, mean_step_time 1445.82, it/s 0.69
  3450. [0130 17:46:02 @multigpu.py:323] [p0574] step: count(669), step_time 1377.37, mean_step_time 1433.91, it/s 0.7
  3451. [0130 17:46:02 @multigpu.py:323] [p0576] step: count(676), step_time 1375.98, mean_step_time 1442.97, it/s 0.69
  3452. [0130 17:46:02 @multigpu.py:323] [p0115] step: count(653), step_time 1451.27, mean_step_time 1441.64, it/s 0.69
  3453. [0130 17:46:03 @multigpu.py:323] [p0574] step: count(670), step_time 1407.84, mean_step_time 1433.61, it/s 0.7
  3454. [0130 17:46:03 @multigpu.py:323] [p0576] step: count(677), step_time 1418.6, mean_step_time 1443.88, it/s 0.69
  3455. [0130 17:46:03 @multigpu.py:323] [p0115] step: count(654), step_time 1465.6, mean_step_time 1442.18, it/s 0.69
  3456. [0130 17:46:05 @multigpu.py:323] [p0574] step: count(671), step_time 1388.14, mean_step_time 1431.7, it/s 0.7
  3457. [0130 17:46:05 @multigpu.py:323] [p0576] step: count(678), step_time 1384.74, mean_step_time 1444.21, it/s 0.69
  3458. [0130 17:46:05 @multigpu.py:323] [p0115] step: count(655), step_time 1431.25, mean_step_time 1438.42, it/s 0.7
  3459. [0130 17:46:06 @multigpu.py:323] [p0574] step: count(672), step_time 1407.39, mean_step_time 1430.34, it/s 0.7
  3460. [0130 17:46:06 @multigpu.py:323] [p0576] step: count(679), step_time 1490.79, mean_step_time 1448.17, it/s 0.69
  3461. [0130 17:46:06 @multigpu.py:323] [p0115] step: count(656), step_time 1417.72, mean_step_time 1434.94, it/s 0.7
  3462. [0130 17:46:07 @multigpu.py:323] [p0574] step: count(673), step_time 1434.03, mean_step_time 1431.95, it/s 0.7
  3463. [0130 17:46:08 @multigpu.py:323] [p0576] step: count(680), step_time 1383.0, mean_step_time 1442.31, it/s 0.69
  3464. [0130 17:46:08 @multigpu.py:323] [p0115] step: count(657), step_time 1430.33, mean_step_time 1436.29, it/s 0.7
  3465. [0130 17:46:09 @multigpu.py:323] [p0574] step: count(674), step_time 1434.18, mean_step_time 1432.77, it/s 0.7
  3466. [0130 17:46:09 @multigpu.py:323] [p0576] step: count(681), step_time 1426.53, mean_step_time 1441.76, it/s 0.69
  3467. [0130 17:46:09 @multigpu.py:323] [p0115] step: count(658), step_time 1511.28, mean_step_time 1441.6, it/s 0.69
  3468. [0130 17:46:10 @multigpu.py:323] [p0574] step: count(675), step_time 1409.21, mean_step_time 1431.12, it/s 0.7
  3469. [0130 17:46:10 @multigpu.py:323] [p0576] step: count(682), step_time 1378.39, mean_step_time 1440.31, it/s 0.69
  3470. [0130 17:46:11 @multigpu.py:323] [p0115] step: count(659), step_time 1437.91, mean_step_time 1441.51, it/s 0.69
  3471. sending to address tcp://p0112:61216
  3472. ##### Sending to neptune: online_score : 0.269616084165 , 1.4 #####
  3473. [u'online', 1.4]
  3474. receiving
  3475. [0130 17:46:12 @multigpu.py:323] [p0574] step: count(676), step_time 1388.14, mean_step_time 1428.39, it/s 0.7
  3476. [0130 17:46:12 @multigpu.py:323] [p0576] step: count(683), step_time 1416.55, mean_step_time 1439.33, it/s 0.69
  3477. sending to address tcp://p0112:61216
  3478. ##### Sending to neptune: online_score : 0.269859721661 , 1.8 #####
  3479. [u'online', 1.8]
  3480. receiving
  3481. [0130 17:46:12 @multigpu.py:323] [p0115] step: count(660), step_time 1474.67, mean_step_time 1443.25, it/s 0.69
  3482. [0130 17:46:13 @multigpu.py:323] [p0574] step: count(677), step_time 1442.56, mean_step_time 1429.2, it/s 0.7
  3483. [0130 17:46:13 @multigpu.py:323] [p0576] step: count(684), step_time 1408.24, mean_step_time 1440.29, it/s 0.69
  3484. [0130 17:46:14 @multigpu.py:323] [p0115] step: count(661), step_time 1416.46, mean_step_time 1440.09, it/s 0.69
  3485. [0130 17:46:15 @multigpu.py:323] [p0574] step: count(678), step_time 1410.63, mean_step_time 1429.94, it/s 0.7
  3486. [0130 17:46:15 @multigpu.py:323] [p0576] step: count(685), step_time 1397.51, mean_step_time 1434.73, it/s 0.7
  3487. [0130 17:46:15 @multigpu.py:323] [p0115] step: count(662), step_time 1455.37, mean_step_time 1442.56, it/s 0.69
  3488. sending to address tcp://p0112:61216
  3489. ##### Sending to neptune: online_score : 0.270699454414 , 1.3 #####
  3490. [u'online', 1.3]
  3491. ##### Sending to neptune: active_workers : 0.270699488322 , 3 #####
  3492. receiving
  3493. [0130 17:46:16 @multigpu.py:323] [p0574] step: count(679), step_time 1416.51, mean_step_time 1429.85, it/s 0.7
  3494. [0130 17:46:16 @multigpu.py:323] [p0576] step: count(686), step_time 1431.12, mean_step_time 1436.03, it/s 0.7
  3495. [0130 17:46:16 @multigpu.py:323] [p0115] step: count(663), step_time 1399.97, mean_step_time 1441.45, it/s 0.69
  3496. [0130 17:46:17 @multigpu.py:323] [p0574] step: count(680), step_time 1418.6, mean_step_time 1429.63, it/s 0.7
  3497. [0130 17:46:18 @multigpu.py:323] [p0576] step: count(687), step_time 1467.33, mean_step_time 1439.4, it/s 0.69
  3498. [0130 17:46:18 @multigpu.py:323] [p0115] step: count(664), step_time 1441.72, mean_step_time 1442.59, it/s 0.69
  3499. [0130 17:46:19 @multigpu.py:323] [p0574] step: count(681), step_time 1434.2, mean_step_time 1428.55, it/s 0.7
  3500. [0130 17:46:19 @multigpu.py:323] [p0576] step: count(688), step_time 1412.21, mean_step_time 1441.73, it/s 0.69
  3501. [0130 17:46:19 @multigpu.py:323] [p0115] step: count(665), step_time 1450.61, mean_step_time 1443.39, it/s 0.69
  3502. [0130 17:46:20 @multigpu.py:323] [p0574] step: count(682), step_time 1371.35, mean_step_time 1425.56, it/s 0.7
  3503. [0130 17:46:20 @multigpu.py:323] [p0576] step: count(689), step_time 1424.95, mean_step_time 1439.58, it/s 0.69
  3504. [0130 17:46:21 @multigpu.py:323] [p0115] step: count(666), step_time 1402.55, mean_step_time 1440.64, it/s 0.69
  3505. [0130 17:46:22 @multigpu.py:323] [p0574] step: count(683), step_time 1467.57, mean_step_time 1429.72, it/s 0.7
  3506. [0130 17:46:22 @multigpu.py:323] [p0576] step: count(690), step_time 1417.75, mean_step_time 1438.8, it/s 0.7
  3507. [0130 17:46:22 @multigpu.py:323] [p0115] step: count(667), step_time 1430.56, mean_step_time 1442.16, it/s 0.69
  3508. [0130 17:46:24 @multigpu.py:323] [p0576] step: count(691), step_time 1760.88, mean_step_time 1437.94, it/s 0.7
  3509. [0130 17:46:24 @multigpu.py:323] [p0574] step: count(684), step_time 1876.98, mean_step_time 1437.97, it/s 0.7
  3510. [0130 17:46:24 @multigpu.py:323] [p0115] step: count(668), step_time 1414.87, mean_step_time 1438.0, it/s 0.7
  3511. [0130 17:46:25 @multigpu.py:323] [p0576] step: count(692), step_time 1412.65, mean_step_time 1436.89, it/s 0.7
  3512. [0130 17:46:25 @multigpu.py:323] [p0574] step: count(685), step_time 1433.75, mean_step_time 1440.66, it/s 0.69
  3513. [0130 17:46:25 @multigpu.py:323] [p0115] step: count(669), step_time 1442.29, mean_step_time 1437.26, it/s 0.7
  3514. [0130 17:46:26 @multigpu.py:323] [p0576] step: count(693), step_time 1410.89, mean_step_time 1436.54, it/s 0.7
  3515. [0130 17:46:26 @multigpu.py:323] [p0115] step: count(670), step_time 1392.77, mean_step_time 1435.02, it/s 0.7
  3516. [0130 17:46:26 @multigpu.py:323] [p0574] step: count(686), step_time 1417.46, mean_step_time 1437.48, it/s 0.7
  3517. [0130 17:46:28 @multigpu.py:323] [p0115] step: count(671), step_time 1422.11, mean_step_time 1435.62, it/s 0.7
  3518. [0130 17:46:28 @multigpu.py:323] [p0574] step: count(687), step_time 1428.69, mean_step_time 1440.79, it/s 0.69
  3519. [0130 17:46:28 @multigpu.py:323] [p0576] step: count(694), step_time 1488.4, mean_step_time 1439.35, it/s 0.69
  3520. [0130 17:46:29 @multigpu.py:323] [p0576] step: count(695), step_time 1400.05, mean_step_time 1435.33, it/s 0.7
  3521. [0130 17:46:29 @multigpu.py:323] [p0115] step: count(672), step_time 1488.86, mean_step_time 1438.91, it/s 0.69
  3522. [0130 17:46:29 @multigpu.py:323] [p0574] step: count(688), step_time 1470.6, mean_step_time 1441.76, it/s 0.69
  3523. [0130 17:46:31 @multigpu.py:323] [p0574] step: count(689), step_time 1413.16, mean_step_time 1443.55, it/s 0.69
  3524. [0130 17:46:31 @multigpu.py:323] [p0115] step: count(673), step_time 1431.14, mean_step_time 1437.9, it/s 0.7
  3525. [0130 17:46:31 @multigpu.py:323] [p0576] step: count(696), step_time 1501.96, mean_step_time 1441.63, it/s 0.69
  3526. [0130 17:46:32 @multigpu.py:323] [p0576] step: count(697), step_time 1399.25, mean_step_time 1440.66, it/s 0.69
  3527. [0130 17:46:32 @multigpu.py:323] [p0115] step: count(674), step_time 1447.63, mean_step_time 1437.0, it/s 0.7
  3528. [0130 17:46:32 @multigpu.py:323] [p0574] step: count(690), step_time 1477.92, mean_step_time 1447.05, it/s 0.69
  3529. [0130 17:46:34 @multigpu.py:323] [p0574] step: count(691), step_time 1414.94, mean_step_time 1448.39, it/s 0.69
  3530. [0130 17:46:34 @multigpu.py:323] [p0115] step: count(675), step_time 1448.86, mean_step_time 1437.88, it/s 0.7
  3531. [0130 17:46:34 @multigpu.py:323] [p0576] step: count(698), step_time 1515.76, mean_step_time 1447.21, it/s 0.69
  3532. [0130 17:46:35 @multigpu.py:323] [p0115] step: count(676), step_time 1418.69, mean_step_time 1437.93, it/s 0.7
  3533. [0130 17:46:35 @multigpu.py:323] [p0574] step: count(692), step_time 1453.11, mean_step_time 1450.68, it/s 0.69
  3534. [0130 17:46:35 @multigpu.py:323] [p0576] step: count(699), step_time 1455.22, mean_step_time 1445.43, it/s 0.69
  3535. [0130 17:46:36 @multigpu.py:323] [p0115] step: count(677), step_time 1402.6, mean_step_time 1436.55, it/s 0.7
  3536. [0130 17:46:36 @multigpu.py:323] [p0574] step: count(693), step_time 1420.45, mean_step_time 1450.0, it/s 0.69
  3537. [0130 17:46:37 @multigpu.py:323] [p0576] step: count(700), step_time 1405.56, mean_step_time 1446.56, it/s 0.69
  3538. sending debugging info...
  3539. sending to address tcp://p0112:61216
  3540. ##### Sending to neptune: mean_delay : 0.276642559436 , 0.0 #####
  3541. sending to address tcp://p0112:61216
  3542. ##### Sending to neptune: max_delay : 0.276642559436 , -0.0 #####
  3543. ##### Sending to neptune: min_delay : 0.276642559436 , -0.0 #####
  3544. [u'delays', [0.0, -0.0, -0.0]]
  3545. receiving
  3546. ##### Sending to neptune: cost : 0.276643062168 , -0.00782114826143 #####
  3547. sending to address tcp://p0112:61216
  3548. ##### Sending to neptune: policy_loss : 0.276643062168 , -0.00080709339818 #####
  3549. ##### Sending to neptune: xentropy_loss : 0.276643062168 , -2.29106926918 #####
  3550. ##### Sending to neptune: value_loss : 0.276643062168 , 1.2907692194 #####
  3551. ##### Sending to neptune: advantage : 0.276643062168 , -9.11761017051e-05 #####
  3552. ##### Sending to neptune: pred_reward : 0.276643062168 , 0.41138869524 #####
  3553. ##### Sending to neptune: max_logit : 0.276643062168 , 0.188500359654 #####
  3554. [u'loss', -0.00782114826142788, -0.0008070933981798589, -2.291069269180298, 1.2907692193984985, -9.117610170505941e-05, 0.41138869524002075, 0.18850035965442657]
  3555. receiving
  3556. ##### Sending to neptune: active_relus : 0.276643491056 , 8843301.94 #####
  3557. ##### Sending to neptune: dp_per_s : 0.276643491056 , 88.940578809 #####
  3558. [u'other', 8843301.94, 88.94057880896713]
  3559. receiving
  3560. [0130 17:46:38 @multigpu.py:323] [p0574] step: count(694), step_time 1396.48, mean_step_time 1448.12, it/s 0.69
  3561. [0130 17:46:38 @multigpu.py:323] [p0576] step: count(701), step_time 1402.01, mean_step_time 1445.33, it/s 0.69
  3562. [0130 17:46:38 @multigpu.py:323] [p0115] step: count(678), step_time 1503.46, mean_step_time 1436.15, it/s 0.7
  3563. sending to address tcp://p0112:61216
  3564. ##### Sending to neptune: online_score : 0.277044685218 , 1.8 #####
  3565. [u'online', 1.8]
  3566. receiving
  3567. [0130 17:46:39 @multigpu.py:323] [p0574] step: count(695), step_time 1431.64, mean_step_time 1449.24, it/s 0.69
  3568. [0130 17:46:39 @multigpu.py:323] [p0115] step: count(679), step_time 1387.38, mean_step_time 1433.63, it/s 0.7
  3569. [0130 17:46:39 @multigpu.py:323] [p0576] step: count(702), step_time 1469.67, mean_step_time 1449.9, it/s 0.69
  3570. sending to address tcp://p0112:61216
  3571. ##### Sending to neptune: online_score : 0.277471321887 , 1.1 #####
  3572. [u'online', 1.1]
  3573. receiving
  3574. [0130 17:46:41 @multigpu.py:323] [p0574] step: count(696), step_time 1467.27, mean_step_time 1453.2, it/s 0.69
  3575. [0130 17:46:41 @multigpu.py:323] [p0115] step: count(680), step_time 1459.1, mean_step_time 1432.85, it/s 0.7
  3576. [0130 17:46:41 @multigpu.py:323] [p0576] step: count(703), step_time 1390.85, mean_step_time 1448.61, it/s 0.69
  3577. sending to address tcp://p0112:61216
  3578. ##### Sending to neptune: online_score : 0.277992236084 , 1.5 #####
  3579. [u'online', 1.5]
  3580. receiving
  3581. [0130 17:46:42 @multigpu.py:323] [p0574] step: count(697), step_time 1407.87, mean_step_time 1451.46, it/s 0.69
  3582. [0130 17:46:42 @multigpu.py:323] [p0115] step: count(681), step_time 1395.23, mean_step_time 1431.79, it/s 0.7
  3583. [0130 17:46:42 @multigpu.py:323] [p0576] step: count(704), step_time 1424.77, mean_step_time 1449.44, it/s 0.69
  3584. [0130 17:46:44 @multigpu.py:323] [p0115] step: count(682), step_time 1418.87, mean_step_time 1429.96, it/s 0.7
  3585. [0130 17:46:44 @multigpu.py:323] [p0576] step: count(705), step_time 1390.71, mean_step_time 1449.1, it/s 0.69
  3586. [0130 17:46:44 @multigpu.py:323] [p0574] step: count(698), step_time 1461.76, mean_step_time 1454.02, it/s 0.69
  3587. [0130 17:46:45 @multigpu.py:323] [p0115] step: count(683), step_time 1398.62, mean_step_time 1429.9, it/s 0.7
  3588. [0130 17:46:45 @multigpu.py:323] [p0576] step: count(706), step_time 1449.88, mean_step_time 1450.04, it/s 0.69
  3589. [0130 17:46:45 @multigpu.py:323] [p0574] step: count(699), step_time 1457.84, mean_step_time 1456.08, it/s 0.69
  3590. [0130 17:46:46 @multigpu.py:323] [p0576] step: count(707), step_time 1370.72, mean_step_time 1445.21, it/s 0.69
  3591. [0130 17:46:46 @multigpu.py:323] [p0115] step: count(684), step_time 1500.55, mean_step_time 1432.84, it/s 0.7
  3592. [0130 17:46:47 @multigpu.py:323] [p0574] step: count(700), step_time 1450.23, mean_step_time 1457.66, it/s 0.69
  3593. sending debugging info...
  3594. sending to address tcp://p0112:61216
  3595. ##### Sending to neptune: mean_delay : 0.279424770806 , 0.0 #####
  3596. ##### Sending to neptune: max_delay : 0.279424770806 , -0.0 #####
  3597. sending to address tcp://p0112:61216
  3598. ##### Sending to neptune: min_delay : 0.279424770806 , -0.0 #####
  3599. [u'delays', [0.0, -0.0, -0.0]]
  3600. receiving
  3601. ##### Sending to neptune: cost : 0.279425232477 , -0.00309272995219 #####
  3602. ##### Sending to neptune: policy_loss : 0.279425232477 , 0.459123522043 #####
  3603. sending to address tcp://p0112:61216
  3604. ##### Sending to neptune: xentropy_loss : 0.279425232477 , -2.29101920128 #####
  3605. ##### Sending to neptune: value_loss : 0.279425232477 , 1.43602633476 #####
  3606. ##### Sending to neptune: advantage : 0.279425232477 , -0.00201497459784 #####
  3607. ##### Sending to neptune: pred_reward : 0.279425232477 , 0.410510092974 #####
  3608. ##### Sending to neptune: max_logit : 0.279425232477 , 0.18886230886 #####
  3609. [u'loss', -0.003092729952186346, 0.45912352204322815, -2.2910192012786865, 1.4360263347625732, -0.0020149745978415012, 0.4105100929737091, 0.18886230885982513]
  3610. receiving
  3611. ##### Sending to neptune: active_relus : 0.279425686664 , 8863210.75 #####
  3612. ##### Sending to neptune: dp_per_s : 0.279425686664 , 89.0163982094 #####
  3613. [u'other', 8863210.75, 89.01639820936614]
  3614. receiving
  3615. [0130 17:46:48 @multigpu.py:323] [p0576] step: count(708), step_time 1449.44, mean_step_time 1447.07, it/s 0.69
  3616. [0130 17:46:48 @multigpu.py:323] [p0115] step: count(685), step_time 1449.69, mean_step_time 1432.79, it/s 0.7
  3617. [0130 17:46:48 @multigpu.py:323] [p0574] step: count(701), step_time 1451.4, mean_step_time 1458.52, it/s 0.69
  3618. [0130 17:46:49 @multigpu.py:323] [p0115] step: count(686), step_time 1420.24, mean_step_time 1433.68, it/s 0.7
  3619. [0130 17:46:49 @multigpu.py:323] [p0576] step: count(709), step_time 1505.57, mean_step_time 1451.1, it/s 0.69
  3620. [0130 17:46:49 @multigpu.py:323] [p0574] step: count(702), step_time 1430.73, mean_step_time 1461.49, it/s 0.68
  3621. [0130 17:46:51 @multigpu.py:323] [p0576] step: count(710), step_time 1397.3, mean_step_time 1450.08, it/s 0.69
  3622. [0130 17:46:51 @multigpu.py:323] [p0115] step: count(687), step_time 1450.65, mean_step_time 1434.68, it/s 0.7
  3623. [0130 17:46:51 @multigpu.py:323] [p0574] step: count(703), step_time 1462.6, mean_step_time 1461.24, it/s 0.68
  3624. [0130 17:46:52 @multigpu.py:323] [p0115] step: count(688), step_time 1520.92, mean_step_time 1439.98, it/s 0.69
  3625. [0130 17:46:52 @multigpu.py:323] [p0576] step: count(711), step_time 1559.96, mean_step_time 1440.03, it/s 0.69
  3626. [0130 17:46:52 @multigpu.py:323] [p0574] step: count(704), step_time 1453.72, mean_step_time 1440.08, it/s 0.69
  3627. [0130 17:46:54 @multigpu.py:323] [p0574] step: count(705), step_time 1416.9, mean_step_time 1439.24, it/s 0.69
  3628. [0130 17:46:54 @multigpu.py:323] [p0576] step: count(712), step_time 1449.48, mean_step_time 1441.87, it/s 0.69
  3629. [0130 17:46:54 @multigpu.py:323] [p0115] step: count(689), step_time 1481.8, mean_step_time 1441.96, it/s 0.69
  3630. [0130 17:46:55 @multigpu.py:323] [p0574] step: count(706), step_time 1416.82, mean_step_time 1439.21, it/s 0.69
  3631. [0130 17:46:55 @multigpu.py:323] [p0576] step: count(713), step_time 1436.16, mean_step_time 1443.14, it/s 0.69
  3632. [0130 17:46:55 @multigpu.py:323] [p0115] step: count(690), step_time 1442.39, mean_step_time 1444.44, it/s 0.69
  3633. [0130 17:46:57 @multigpu.py:323] [p0574] step: count(707), step_time 1404.84, mean_step_time 1438.01, it/s 0.7
  3634. [0130 17:46:57 @multigpu.py:323] [p0576] step: count(714), step_time 1425.54, mean_step_time 1439.99, it/s 0.69
  3635. [0130 17:46:57 @multigpu.py:323] [p0115] step: count(691), step_time 1429.53, mean_step_time 1444.81, it/s 0.69
  3636. [0130 17:46:58 @multigpu.py:323] [p0576] step: count(715), step_time 1397.21, mean_step_time 1439.85, it/s 0.69
  3637. [0130 17:46:58 @multigpu.py:323] [p0574] step: count(708), step_time 1485.55, mean_step_time 1438.76, it/s 0.7
  3638. [0130 17:46:58 @multigpu.py:323] [p0115] step: count(692), step_time 1433.3, mean_step_time 1442.03, it/s 0.69
  3639. [0130 17:46:59 @multigpu.py:323] [p0576] step: count(716), step_time 1443.1, mean_step_time 1436.91, it/s 0.7
  3640. [0130 17:47:00 @multigpu.py:323] [p0574] step: count(709), step_time 1439.46, mean_step_time 1440.08, it/s 0.69
  3641. [0130 17:47:00 @multigpu.py:323] [p0115] step: count(693), step_time 1417.74, mean_step_time 1441.36, it/s 0.69
  3642. [0130 17:47:01 @multigpu.py:323] [p0574] step: count(710), step_time 1380.25, mean_step_time 1435.19, it/s 0.7
  3643. [0130 17:47:01 @multigpu.py:323] [p0576] step: count(717), step_time 1399.54, mean_step_time 1436.92, it/s 0.7
  3644. [0130 17:47:01 @multigpu.py:323] [p0115] step: count(694), step_time 1413.25, mean_step_time 1439.64, it/s 0.69
  3645. [0130 17:47:02 @multigpu.py:323] [p0576] step: count(718), step_time 1416.18, mean_step_time 1431.94, it/s 0.7
  3646. [0130 17:47:02 @multigpu.py:323] [p0115] step: count(695), step_time 1381.65, mean_step_time 1436.28, it/s 0.7
  3647. [0130 17:47:02 @multigpu.py:323] [p0574] step: count(711), step_time 1458.54, mean_step_time 1437.37, it/s 0.7
  3648. [0130 17:47:04 @multigpu.py:323] [p0576] step: count(719), step_time 1403.66, mean_step_time 1429.37, it/s 0.7
  3649. [0130 17:47:04 @multigpu.py:323] [p0115] step: count(696), step_time 1425.07, mean_step_time 1436.6, it/s 0.7
  3650. [0130 17:47:04 @multigpu.py:323] [p0574] step: count(712), step_time 1508.01, mean_step_time 1440.12, it/s 0.69
  3651. sending to address tcp://p0112:61216
  3652. ##### Sending to neptune: online_score : 0.284432896111 , 2.0 #####
  3653. [u'online', 2.0]
  3654. receiving
  3655. [0130 17:47:05 @multigpu.py:323] [p0576] step: count(720), step_time 1396.07, mean_step_time 1428.89, it/s 0.7
  3656. [0130 17:47:05 @multigpu.py:323] [p0115] step: count(697), step_time 1415.77, mean_step_time 1437.26, it/s 0.7
  3657. [0130 17:47:05 @multigpu.py:323] [p0574] step: count(713), step_time 1400.91, mean_step_time 1439.14, it/s 0.69
  3658. sending to address tcp://p0112:61216
  3659. ##### Sending to neptune: online_score : 0.284738871919 , 1.2 #####
  3660. [u'online', 1.2]
  3661. receiving
  3662. [0130 17:47:07 @multigpu.py:323] [p0576] step: count(721), step_time 1397.28, mean_step_time 1428.65, it/s 0.7
  3663. [0130 17:47:07 @multigpu.py:323] [p0115] step: count(698), step_time 1417.27, mean_step_time 1432.95, it/s 0.7
  3664. [0130 17:47:07 @multigpu.py:323] [p0574] step: count(714), step_time 1433.64, mean_step_time 1441.0, it/s 0.69
  3665. [0130 17:47:08 @multigpu.py:323] [p0576] step: count(722), step_time 1401.82, mean_step_time 1425.26, it/s 0.7
  3666. [0130 17:47:08 @multigpu.py:323] [p0115] step: count(699), step_time 1443.38, mean_step_time 1435.75, it/s 0.7
  3667. [0130 17:47:08 @multigpu.py:323] [p0574] step: count(715), step_time 1549.43, mean_step_time 1446.89, it/s 0.69
  3668. sending to address tcp://p0112:61216
  3669. ##### Sending to neptune: online_score : 0.285529997481 , 1.3 #####
  3670. [u'online', 1.3]
  3671. receiving
  3672. [0130 17:47:09 @multigpu.py:323] [p0576] step: count(723), step_time 1446.5, mean_step_time 1428.04, it/s 0.7
  3673. [0130 17:47:09 @multigpu.py:323] [p0115] step: count(700), step_time 1448.74, mean_step_time 1435.23, it/s 0.7
  3674. sending debugging info...
  3675. sending to address tcp://p0112:61216
  3676. ##### Sending to neptune: mean_delay : 0.285801506374 , 0.0 #####
  3677. sending to address tcp://p0112:61216
  3678. ##### Sending to neptune: max_delay : 0.285801506374 , -0.0 #####
  3679. ##### Sending to neptune: min_delay : 0.285801506374 , -0.0 #####
  3680. [u'delays', [0.0, -0.0, -0.0]]
  3681. receiving
  3682. ##### Sending to neptune: cost : 0.285802074141 , -0.0108789335936 #####
  3683. sending to address tcp://p0112:61216
  3684. ##### Sending to neptune: policy_loss : 0.285802074141 , -0.324730724096 #####
  3685. ##### Sending to neptune: xentropy_loss : 0.285802074141 , -2.29079914093 #####
  3686. ##### Sending to neptune: value_loss : 0.285802074141 , 1.22302639484 #####
  3687. ##### Sending to neptune: advantage : 0.285802074141 , 0.0014331714483 #####
  3688. ##### Sending to neptune: pred_reward : 0.285802074141 , 0.409147292376 #####
  3689. ##### Sending to neptune: max_logit : 0.285802074141 , 0.190351292491 #####
  3690. [u'loss', -0.01087893359363079, -0.3247307240962982, -2.290799140930176, 1.2230263948440552, 0.0014331714482977986, 0.4091472923755646, 0.19035129249095917]
  3691. receiving
  3692. ##### Sending to neptune: active_relus : 0.285802551111 , 8855540.19 #####
  3693. ##### Sending to neptune: dp_per_s : 0.285802551111 , 88.9247276744 #####
  3694. [u'other', 8855540.19, 88.92472767440891]
  3695. receiving
  3696. [0130 17:47:10 @multigpu.py:323] [p0574] step: count(716), step_time 1499.9, mean_step_time 1448.52, it/s 0.69
  3697. [0130 17:47:11 @multigpu.py:323] [p0576] step: count(724), step_time 1457.24, mean_step_time 1429.67, it/s 0.7
  3698. [0130 17:47:11 @multigpu.py:323] [p0115] step: count(701), step_time 1416.31, mean_step_time 1436.29, it/s 0.7
  3699. [0130 17:47:11 @multigpu.py:323] [p0574] step: count(717), step_time 1476.77, mean_step_time 1451.97, it/s 0.69
  3700. [0130 17:47:12 @multigpu.py:323] [p0576] step: count(725), step_time 1449.38, mean_step_time 1432.6, it/s 0.7
  3701. [0130 17:47:12 @multigpu.py:323] [p0115] step: count(702), step_time 1464.9, mean_step_time 1438.59, it/s 0.7
  3702. [0130 17:47:13 @multigpu.py:323] [p0574] step: count(718), step_time 1487.39, mean_step_time 1453.25, it/s 0.69
  3703. [0130 17:47:14 @multigpu.py:323] [p0576] step: count(726), step_time 1405.97, mean_step_time 1430.41, it/s 0.7
  3704. [0130 17:47:14 @multigpu.py:323] [p0115] step: count(703), step_time 1397.31, mean_step_time 1438.52, it/s 0.7
  3705. [0130 17:47:14 @multigpu.py:323] [p0574] step: count(719), step_time 1456.28, mean_step_time 1453.17, it/s 0.69
  3706. [0130 17:47:15 @multigpu.py:323] [p0576] step: count(727), step_time 1413.06, mean_step_time 1432.52, it/s 0.7
  3707. [0130 17:47:15 @multigpu.py:323] [p0115] step: count(704), step_time 1476.83, mean_step_time 1437.34, it/s 0.7
  3708. [0130 17:47:16 @multigpu.py:323] [p0574] step: count(720), step_time 1470.73, mean_step_time 1454.19, it/s 0.69
  3709. [0130 17:47:17 @multigpu.py:323] [p0576] step: count(728), step_time 1434.16, mean_step_time 1431.76, it/s 0.7
  3710. [0130 17:47:17 @multigpu.py:323] [p0115] step: count(705), step_time 1408.52, mean_step_time 1435.28, it/s 0.7
  3711. [0130 17:47:17 @multigpu.py:323] [p0574] step: count(721), step_time 1504.53, mean_step_time 1456.85, it/s 0.69
  3712. [0130 17:47:18 @multigpu.py:323] [p0576] step: count(729), step_time 1437.14, mean_step_time 1428.34, it/s 0.7
  3713. [0130 17:47:18 @multigpu.py:323] [p0115] step: count(706), step_time 1473.58, mean_step_time 1437.95, it/s 0.7
  3714. [0130 17:47:19 @multigpu.py:323] [p0574] step: count(722), step_time 1579.61, mean_step_time 1464.29, it/s 0.68
  3715. [0130 17:47:19 @multigpu.py:323] [p0576] step: count(730), step_time 1430.95, mean_step_time 1430.02, it/s 0.7
  3716. [0130 17:47:20 @multigpu.py:323] [p0115] step: count(707), step_time 1453.48, mean_step_time 1438.09, it/s 0.7
  3717. [0130 17:47:20 @multigpu.py:323] [p0574] step: count(723), step_time 1468.18, mean_step_time 1464.57, it/s 0.68
  3718. [0130 17:47:22 @multigpu.py:323] [p0574] step: count(724), step_time 1484.95, mean_step_time 1466.14, it/s 0.68
  3719. [0130 17:47:22 @multigpu.py:323] [p0115] step: count(708), step_time 2084.26, mean_step_time 1466.25, it/s 0.68
  3720. [0130 17:47:22 @multigpu.py:323] [p0576] step: count(731), step_time 2284.87, mean_step_time 1466.27, it/s 0.68
  3721. [0130 17:47:23 @multigpu.py:323] [p0576] step: count(732), step_time 1417.49, mean_step_time 1464.67, it/s 0.68
  3722. [0130 17:47:23 @multigpu.py:323] [p0115] step: count(709), step_time 1423.44, mean_step_time 1463.34, it/s 0.68
  3723. [0130 17:47:23 @multigpu.py:323] [p0574] step: count(725), step_time 1466.45, mean_step_time 1468.61, it/s 0.68
  3724. [0130 17:47:25 @multigpu.py:323] [p0576] step: count(733), step_time 1450.55, mean_step_time 1465.39, it/s 0.68
  3725. [0130 17:47:25 @multigpu.py:323] [p0115] step: count(710), step_time 1444.98, mean_step_time 1463.47, it/s 0.68
  3726. [0130 17:47:25 @multigpu.py:323] [p0574] step: count(726), step_time 1466.99, mean_step_time 1471.12, it/s 0.68
  3727. [0130 17:47:26 @multigpu.py:323] [p0576] step: count(734), step_time 1418.3, mean_step_time 1465.02, it/s 0.68
  3728. [0130 17:47:26 @multigpu.py:323] [p0115] step: count(711), step_time 1459.8, mean_step_time 1464.98, it/s 0.68
  3729. [0130 17:47:26 @multigpu.py:323] [p0574] step: count(727), step_time 1479.33, mean_step_time 1474.85, it/s 0.68
  3730. [0130 17:47:27 @multigpu.py:323] [p0576] step: count(735), step_time 1407.67, mean_step_time 1465.55, it/s 0.68
  3731. [0130 17:47:27 @multigpu.py:323] [p0115] step: count(712), step_time 1419.85, mean_step_time 1464.31, it/s 0.68
  3732. [0130 17:47:28 @multigpu.py:323] [p0574] step: count(728), step_time 1647.55, mean_step_time 1482.95, it/s 0.67
  3733. [0130 17:47:29 @multigpu.py:323] [p0576] step: count(736), step_time 1399.97, mean_step_time 1463.39, it/s 0.68
  3734. [0130 17:47:29 @multigpu.py:323] [p0115] step: count(713), step_time 1468.94, mean_step_time 1466.87, it/s 0.68
  3735. [0130 17:47:29 @multigpu.py:323] [p0574] step: count(729), step_time 1496.2, mean_step_time 1485.78, it/s 0.67
  3736. [0130 17:47:30 @multigpu.py:323] [p0576] step: count(737), step_time 1421.17, mean_step_time 1464.47, it/s 0.68
  3737. [0130 17:47:30 @multigpu.py:323] [p0115] step: count(714), step_time 1443.28, mean_step_time 1468.37, it/s 0.68
  3738. [0130 17:47:31 @multigpu.py:323] [p0574] step: count(730), step_time 1426.75, mean_step_time 1488.11, it/s 0.67
  3739. [0130 17:47:32 @multigpu.py:323] [p0576] step: count(738), step_time 1426.87, mean_step_time 1465.01, it/s 0.68
  3740. [0130 17:47:32 @multigpu.py:323] [p0115] step: count(715), step_time 1442.87, mean_step_time 1471.43, it/s 0.68
  3741. [0130 17:47:32 @multigpu.py:323] [p0574] step: count(731), step_time 1566.06, mean_step_time 1493.48, it/s 0.67
  3742. [0130 17:47:33 @multigpu.py:323] [p0576] step: count(739), step_time 1378.06, mean_step_time 1463.73, it/s 0.68
  3743. [0130 17:47:33 @multigpu.py:323] [p0115] step: count(716), step_time 1397.81, mean_step_time 1470.07, it/s 0.68
  3744. [0130 17:47:34 @multigpu.py:323] [p0574] step: count(732), step_time 1510.79, mean_step_time 1493.62, it/s 0.67
  3745. [0130 17:47:34 @multigpu.py:323] [p0576] step: count(740), step_time 1453.38, mean_step_time 1466.59, it/s 0.68
  3746. [0130 17:47:35 @multigpu.py:323] [p0115] step: count(717), step_time 1452.81, mean_step_time 1471.92, it/s 0.68
  3747. [0130 17:47:35 @multigpu.py:323] [p0574] step: count(733), step_time 1517.2, mean_step_time 1499.44, it/s 0.67
  3748. [0130 17:47:36 @multigpu.py:323] [p0576] step: count(741), step_time 1380.13, mean_step_time 1465.73, it/s 0.68
  3749. [0130 17:47:36 @multigpu.py:323] [p0115] step: count(718), step_time 1455.88, mean_step_time 1473.85, it/s 0.68
  3750. [0130 17:47:37 @multigpu.py:323] [p0574] step: count(734), step_time 1566.86, mean_step_time 1506.1, it/s 0.66
  3751. [0130 17:47:37 @multigpu.py:323] [p0576] step: count(742), step_time 1463.09, mean_step_time 1468.8, it/s 0.68
  3752. [0130 17:47:38 @multigpu.py:323] [p0115] step: count(719), step_time 1447.73, mean_step_time 1474.07, it/s 0.68
  3753. [0130 17:47:38 @multigpu.py:323] [p0574] step: count(735), step_time 1595.75, mean_step_time 1508.41, it/s 0.66
  3754. [0130 17:47:39 @multigpu.py:323] [p0576] step: count(743), step_time 1368.8, mean_step_time 1464.91, it/s 0.68
  3755. [0130 17:47:39 @multigpu.py:323] [p0115] step: count(720), step_time 1455.55, mean_step_time 1474.41, it/s 0.68
  3756. [0130 17:47:40 @multigpu.py:323] [p0574] step: count(736), step_time 1489.2, mean_step_time 1507.88, it/s 0.66
  3757. [0130 17:47:40 @multigpu.py:323] [p0576] step: count(744), step_time 1383.44, mean_step_time 1461.22, it/s 0.68
  3758. [0130 17:47:40 @multigpu.py:323] [p0115] step: count(721), step_time 1451.48, mean_step_time 1476.16, it/s 0.68
  3759. sending to address tcp://p0112:61216
  3760. ##### Sending to neptune: online_score : 0.294545846913 , 1.8 #####
  3761. [u'online', 1.8]
  3762. ##### Sending to neptune: active_workers : 0.294546027448 , 3 #####
  3763. receiving
  3764. [0130 17:47:41 @multigpu.py:323] [p0576] step: count(745), step_time 1408.92, mean_step_time 1459.2, it/s 0.69
  3765. [0130 17:47:42 @multigpu.py:323] [p0574] step: count(737), step_time 1684.3, mean_step_time 1518.26, it/s 0.66
  3766. [0130 17:47:42 @multigpu.py:323] [p0115] step: count(722), step_time 1436.37, mean_step_time 1474.74, it/s 0.68
  3767. sending to address tcp://p0112:61216
  3768. ##### Sending to neptune: online_score : 0.294824038612 , 1.2 #####
  3769. [u'online', 1.2]
  3770. receiving
  3771. sending to address tcp://p0112:61216
  3772. ##### Sending to neptune: online_score : 0.294907082717 , 2.2 #####
  3773. [u'online', 2.2]
  3774. receiving
  3775. [0130 17:47:43 @multigpu.py:323] [p0576] step: count(746), step_time 1430.48, mean_step_time 1460.42, it/s 0.68
  3776. [0130 17:47:43 @multigpu.py:323] [p0574] step: count(738), step_time 1636.61, mean_step_time 1525.72, it/s 0.66
  3777. [0130 17:47:43 @multigpu.py:323] [p0115] step: count(723), step_time 1376.08, mean_step_time 1473.68, it/s 0.68
  3778. [0130 17:47:44 @multigpu.py:323] [p0576] step: count(747), step_time 1489.13, mean_step_time 1464.23, it/s 0.68
  3779. [0130 17:47:45 @multigpu.py:323] [p0115] step: count(724), step_time 1441.64, mean_step_time 1471.92, it/s 0.68
  3780. [0130 17:47:45 @multigpu.py:323] [p0574] step: count(739), step_time 1543.1, mean_step_time 1530.06, it/s 0.65
  3781. [0130 17:47:46 @multigpu.py:323] [p0576] step: count(748), step_time 1405.09, mean_step_time 1462.77, it/s 0.68
  3782. [0130 17:47:46 @multigpu.py:323] [p0115] step: count(725), step_time 1436.45, mean_step_time 1473.31, it/s 0.68
  3783. [0130 17:47:46 @multigpu.py:323] [p0574] step: count(740), step_time 1569.09, mean_step_time 1534.98, it/s 0.65
  3784. [0130 17:47:47 @multigpu.py:323] [p0576] step: count(749), step_time 1407.46, mean_step_time 1461.29, it/s 0.68
  3785. [0130 17:47:48 @multigpu.py:323] [p0115] step: count(726), step_time 1458.22, mean_step_time 1472.55, it/s 0.68
  3786. [0130 17:47:48 @multigpu.py:323] [p0574] step: count(741), step_time 1437.3, mean_step_time 1531.61, it/s 0.65
  3787. [0130 17:47:49 @multigpu.py:323] [p0576] step: count(750), step_time 1427.61, mean_step_time 1461.12, it/s 0.68
  3788. [0130 17:47:49 @multigpu.py:323] [p0115] step: count(727), step_time 1435.61, mean_step_time 1471.65, it/s 0.68
  3789. [0130 17:47:49 @multigpu.py:323] [p0574] step: count(742), step_time 1468.07, mean_step_time 1526.04, it/s 0.66
  3790. [0130 17:47:50 @multigpu.py:323] [p0576] step: count(751), step_time 1362.66, mean_step_time 1415.01, it/s 0.71
  3791. [0130 17:47:51 @multigpu.py:323] [p0115] step: count(728), step_time 2405.93, mean_step_time 1487.74, it/s 0.67
  3792. [0130 17:47:51 @multigpu.py:323] [p0574] step: count(743), step_time 2188.64, mean_step_time 1562.06, it/s 0.64
  3793. [0130 17:47:51 @multigpu.py:323] [p0576] step: count(752), step_time 1456.0, mean_step_time 1416.94, it/s 0.71
  3794. [0130 17:47:53 @multigpu.py:323] [p0576] step: count(753), step_time 1413.45, mean_step_time 1415.08, it/s 0.71
  3795. [0130 17:47:53 @multigpu.py:323] [p0115] step: count(729), step_time 1422.24, mean_step_time 1487.68, it/s 0.67
  3796. [0130 17:47:53 @multigpu.py:323] [p0574] step: count(744), step_time 1479.45, mean_step_time 1561.78, it/s 0.64
  3797. [0130 17:47:54 @multigpu.py:323] [p0576] step: count(754), step_time 1401.71, mean_step_time 1414.25, it/s 0.71
  3798. [0130 17:47:54 @multigpu.py:323] [p0115] step: count(730), step_time 1433.0, mean_step_time 1487.08, it/s 0.67
  3799. [0130 17:47:54 @multigpu.py:323] [p0574] step: count(745), step_time 1522.8, mean_step_time 1564.6, it/s 0.64
  3800. [0130 17:47:56 @multigpu.py:323] [p0576] step: count(755), step_time 1378.85, mean_step_time 1412.81, it/s 0.71
  3801. [0130 17:47:56 @multigpu.py:323] [p0115] step: count(731), step_time 1391.26, mean_step_time 1483.65, it/s 0.67
  3802. [0130 17:47:56 @multigpu.py:323] [p0574] step: count(746), step_time 1454.22, mean_step_time 1563.96, it/s 0.64
  3803. [0130 17:47:57 @multigpu.py:323] [p0576] step: count(756), step_time 1429.84, mean_step_time 1414.31, it/s 0.71
  3804. [0130 17:47:57 @multigpu.py:323] [p0115] step: count(732), step_time 1433.66, mean_step_time 1484.34, it/s 0.67
  3805. [0130 17:47:57 @multigpu.py:323] [p0574] step: count(747), step_time 1442.47, mean_step_time 1562.12, it/s 0.64
  3806. [0130 17:47:58 @multigpu.py:323] [p0576] step: count(757), step_time 1381.52, mean_step_time 1412.32, it/s 0.71
  3807. [0130 17:47:59 @multigpu.py:323] [p0115] step: count(733), step_time 1428.08, mean_step_time 1482.3, it/s 0.67
  3808. [0130 17:47:59 @multigpu.py:323] [p0574] step: count(748), step_time 1438.44, mean_step_time 1551.67, it/s 0.64
  3809. [0130 17:48:00 @multigpu.py:323] [p0576] step: count(758), step_time 1397.49, mean_step_time 1410.85, it/s 0.71
  3810. [0130 17:48:00 @multigpu.py:323] [p0115] step: count(734), step_time 1458.07, mean_step_time 1483.04, it/s 0.67
  3811. [0130 17:48:00 @multigpu.py:323] [p0574] step: count(749), step_time 1499.59, mean_step_time 1551.84, it/s 0.64
  3812. [0130 17:48:01 @multigpu.py:323] [p0576] step: count(759), step_time 1388.58, mean_step_time 1411.38, it/s 0.71
  3813. [0130 17:48:01 @multigpu.py:323] [p0115] step: count(735), step_time 1409.96, mean_step_time 1481.39, it/s 0.68
  3814. [0130 17:48:02 @multigpu.py:323] [p0574] step: count(750), step_time 1420.14, mean_step_time 1551.5, it/s 0.64
  3815. [0130 17:48:03 @multigpu.py:323] [p0576] step: count(760), step_time 1410.87, mean_step_time 1409.26, it/s 0.71
  3816. sending to address tcp://p0112:61216
  3817. ##### Sending to neptune: online_score : 0.300571438273 , 1.3 #####
  3818. [u'online', 1.3]
  3819. receiving
  3820. [0130 17:48:03 @multigpu.py:323] [p0115] step: count(736), step_time 1424.39, mean_step_time 1482.72, it/s 0.67
  3821. [0130 17:48:03 @multigpu.py:323] [p0574] step: count(751), step_time 1604.72, mean_step_time 1553.44, it/s 0.64
  3822. [0130 17:48:04 @multigpu.py:323] [p0576] step: count(761), step_time 1423.25, mean_step_time 1411.41, it/s 0.71
  3823. [0130 17:48:04 @multigpu.py:323] [p0115] step: count(737), step_time 1482.03, mean_step_time 1484.18, it/s 0.67
  3824. [0130 17:48:05 @multigpu.py:323] [p0574] step: count(752), step_time 1496.24, mean_step_time 1552.71, it/s 0.64
  3825. sending to address tcp://p0112:61216
  3826. ##### Sending to neptune: online_score : 0.301201084124 , 1.1 #####
  3827. [u'online', 1.1]
  3828. receiving
  3829. [0130 17:48:06 @multigpu.py:323] [p0576] step: count(762), step_time 1436.6, mean_step_time 1410.09, it/s 0.71
  3830. [0130 17:48:06 @multigpu.py:323] [p0115] step: count(738), step_time 1408.71, mean_step_time 1481.82, it/s 0.67
  3831. [0130 17:48:06 @multigpu.py:323] [p0574] step: count(753), step_time 1493.95, mean_step_time 1551.55, it/s 0.64
  3832. [0130 17:48:07 @multigpu.py:323] [p0576] step: count(763), step_time 1437.35, mean_step_time 1413.51, it/s 0.71
  3833. [0130 17:48:07 @multigpu.py:323] [p0115] step: count(739), step_time 1434.51, mean_step_time 1481.16, it/s 0.68
  3834. [0130 17:48:08 @multigpu.py:323] [p0574] step: count(754), step_time 1429.3, mean_step_time 1544.67, it/s 0.65
  3835. [0130 17:48:08 @multigpu.py:323] [p0576] step: count(764), step_time 1373.29, mean_step_time 1413.01, it/s 0.71
  3836. [0130 17:48:09 @multigpu.py:323] [p0115] step: count(740), step_time 1423.09, mean_step_time 1479.54, it/s 0.68
  3837. [0130 17:48:09 @multigpu.py:323] [p0574] step: count(755), step_time 1581.44, mean_step_time 1543.95, it/s 0.65
  3838. [0130 17:48:10 @multigpu.py:323] [p0576] step: count(765), step_time 1404.71, mean_step_time 1412.8, it/s 0.71
  3839. [0130 17:48:10 @multigpu.py:323] [p0115] step: count(741), step_time 1420.29, mean_step_time 1477.98, it/s 0.68
  3840. [0130 17:48:11 @multigpu.py:323] [p0574] step: count(756), step_time 1490.79, mean_step_time 1544.03, it/s 0.65
  3841. [0130 17:48:11 @multigpu.py:323] [p0576] step: count(766), step_time 1401.43, mean_step_time 1411.34, it/s 0.71
  3842. [0130 17:48:12 @multigpu.py:323] [p0115] step: count(742), step_time 1500.94, mean_step_time 1481.21, it/s 0.68
  3843. [0130 17:48:12 @multigpu.py:323] [p0574] step: count(757), step_time 1599.73, mean_step_time 1539.8, it/s 0.65
  3844. [0130 17:48:13 @multigpu.py:323] [p0576] step: count(767), step_time 1473.04, mean_step_time 1410.54, it/s 0.71
  3845. [0130 17:48:13 @multigpu.py:323] [p0115] step: count(743), step_time 1456.1, mean_step_time 1485.21, it/s 0.67
  3846. [0130 17:48:14 @multigpu.py:323] [p0574] step: count(758), step_time 1563.87, mean_step_time 1536.17, it/s 0.65
  3847. [0130 17:48:14 @multigpu.py:323] [p0576] step: count(768), step_time 1401.15, mean_step_time 1410.34, it/s 0.71
  3848. [0130 17:48:14 @multigpu.py:323] [p0115] step: count(744), step_time 1457.8, mean_step_time 1486.02, it/s 0.67
  3849. [0130 17:48:15 @multigpu.py:323] [p0574] step: count(759), step_time 1448.84, mean_step_time 1531.46, it/s 0.65
  3850. [0130 17:48:15 @multigpu.py:323] [p0576] step: count(769), step_time 1433.32, mean_step_time 1411.64, it/s 0.71
  3851. [0130 17:48:16 @multigpu.py:323] [p0115] step: count(745), step_time 1430.88, mean_step_time 1485.74, it/s 0.67
  3852. sending to address tcp://p0112:61216
  3853. ##### Sending to neptune: online_score : 0.304254507754 , 1.8 #####
  3854. [u'online', 1.8]
  3855. receiving
  3856. [0130 17:48:17 @multigpu.py:323] [p0574] step: count(760), step_time 1419.68, mean_step_time 1523.98, it/s 0.66
  3857. [0130 17:48:17 @multigpu.py:323] [p0576] step: count(770), step_time 1421.96, mean_step_time 1411.35, it/s 0.71
  3858. [0130 17:48:17 @multigpu.py:323] [p0115] step: count(746), step_time 1416.76, mean_step_time 1483.67, it/s 0.67
  3859. [0130 17:48:18 @multigpu.py:323] [p0574] step: count(761), step_time 1396.02, mean_step_time 1521.92, it/s 0.66
  3860. [0130 17:48:18 @multigpu.py:323] [p0576] step: count(771), step_time 1417.25, mean_step_time 1414.08, it/s 0.71
  3861. [0130 17:48:19 @multigpu.py:323] [p0115] step: count(747), step_time 1478.87, mean_step_time 1485.83, it/s 0.67
  3862. [0130 17:48:20 @multigpu.py:323] [p0574] step: count(762), step_time 1405.99, mean_step_time 1518.82, it/s 0.66
  3863. [0130 17:48:21 @multigpu.py:323] [p0574] step: count(763), step_time 1430.9, mean_step_time 1480.93, it/s 0.68
  3864. [0130 17:48:21 @multigpu.py:323] [p0115] step: count(748), step_time 2311.05, mean_step_time 1481.08, it/s 0.68
  3865. [0130 17:48:21 @multigpu.py:323] [p0576] step: count(772), step_time 2796.65, mean_step_time 1481.11, it/s 0.68
  3866. [0130 17:48:22 @multigpu.py:323] [p0115] step: count(749), step_time 1422.93, mean_step_time 1481.12, it/s 0.68
  3867. [0130 17:48:23 @multigpu.py:323] [p0574] step: count(764), step_time 1436.66, mean_step_time 1478.79, it/s 0.68
  3868. [0130 17:48:23 @multigpu.py:323] [p0576] step: count(773), step_time 1463.31, mean_step_time 1483.61, it/s 0.67
  3869. [0130 17:48:24 @multigpu.py:323] [p0115] step: count(750), step_time 1418.68, mean_step_time 1480.4, it/s 0.68
  3870. [0130 17:48:24 @multigpu.py:323] [p0576] step: count(774), step_time 1410.31, mean_step_time 1484.04, it/s 0.67
  3871. [0130 17:48:24 @multigpu.py:323] [p0574] step: count(765), step_time 1467.2, mean_step_time 1476.01, it/s 0.68
  3872. [0130 17:48:25 @multigpu.py:323] [p0115] step: count(751), step_time 1426.06, mean_step_time 1482.14, it/s 0.67
  3873. [0130 17:48:25 @multigpu.py:323] [p0576] step: count(775), step_time 1424.35, mean_step_time 1486.31, it/s 0.67
  3874. [0130 17:48:25 @multigpu.py:323] [p0574] step: count(766), step_time 1451.39, mean_step_time 1475.87, it/s 0.68
  3875. [0130 17:48:27 @multigpu.py:323] [p0115] step: count(752), step_time 1385.07, mean_step_time 1479.71, it/s 0.68
  3876. [0130 17:48:27 @multigpu.py:323] [p0576] step: count(776), step_time 1461.88, mean_step_time 1487.92, it/s 0.67
  3877. [0130 17:48:27 @multigpu.py:323] [p0574] step: count(767), step_time 1466.48, mean_step_time 1477.07, it/s 0.68
  3878. [0130 17:48:28 @multigpu.py:323] [p0115] step: count(753), step_time 1429.92, mean_step_time 1479.81, it/s 0.68
  3879. [0130 17:48:28 @multigpu.py:323] [p0576] step: count(777), step_time 1398.9, mean_step_time 1488.78, it/s 0.67
  3880. [0130 17:48:28 @multigpu.py:323] [p0574] step: count(768), step_time 1492.78, mean_step_time 1479.79, it/s 0.68
  3881. [0130 17:48:30 @multigpu.py:323] [p0115] step: count(754), step_time 1413.36, mean_step_time 1477.57, it/s 0.68
  3882. [0130 17:48:30 @multigpu.py:323] [p0576] step: count(778), step_time 1383.07, mean_step_time 1488.06, it/s 0.67
  3883. [0130 17:48:30 @multigpu.py:323] [p0574] step: count(769), step_time 1375.72, mean_step_time 1473.59, it/s 0.68
  3884. [0130 17:48:31 @multigpu.py:323] [p0115] step: count(755), step_time 1414.28, mean_step_time 1477.79, it/s 0.68
  3885. [0130 17:48:31 @multigpu.py:323] [p0576] step: count(779), step_time 1457.36, mean_step_time 1491.5, it/s 0.67
  3886. [0130 17:48:31 @multigpu.py:323] [p0574] step: count(770), step_time 1419.43, mean_step_time 1473.56, it/s 0.68
  3887. sending to address tcp://p0112:61216
  3888. ##### Sending to neptune: online_score : 0.308666802446 , 1.2 #####
  3889. [u'online', 1.2]
  3890. receiving
  3891. [0130 17:48:32 @multigpu.py:323] [p0115] step: count(756), step_time 1425.67, mean_step_time 1477.85, it/s 0.68
  3892. [0130 17:48:32 @multigpu.py:323] [p0576] step: count(780), step_time 1396.16, mean_step_time 1490.77, it/s 0.67
  3893. [0130 17:48:33 @multigpu.py:323] [p0574] step: count(771), step_time 1385.33, mean_step_time 1462.59, it/s 0.68
  3894. [0130 17:48:34 @multigpu.py:323] [p0115] step: count(757), step_time 1374.16, mean_step_time 1472.46, it/s 0.68
  3895. [0130 17:48:34 @multigpu.py:323] [p0576] step: count(781), step_time 1418.66, mean_step_time 1490.54, it/s 0.67
  3896. [0130 17:48:34 @multigpu.py:323] [p0574] step: count(772), step_time 1427.58, mean_step_time 1459.16, it/s 0.69
  3897. [0130 17:48:35 @multigpu.py:323] [p0115] step: count(758), step_time 1386.69, mean_step_time 1471.36, it/s 0.68
  3898. [0130 17:48:35 @multigpu.py:323] [p0576] step: count(782), step_time 1421.39, mean_step_time 1489.78, it/s 0.67
  3899. [0130 17:48:35 @multigpu.py:323] [p0574] step: count(773), step_time 1451.68, mean_step_time 1457.04, it/s 0.69
  3900. [0130 17:48:37 @multigpu.py:323] [p0115] step: count(759), step_time 1447.37, mean_step_time 1472.0, it/s 0.68
  3901. [0130 17:48:37 @multigpu.py:323] [p0576] step: count(783), step_time 1463.02, mean_step_time 1491.06, it/s 0.67
  3902. [0130 17:48:37 @multigpu.py:323] [p0574] step: count(774), step_time 1477.53, mean_step_time 1459.45, it/s 0.69
  3903. [0130 17:48:38 @multigpu.py:323] [p0115] step: count(760), step_time 1403.82, mean_step_time 1471.04, it/s 0.68
  3904. [0130 17:48:38 @multigpu.py:323] [p0576] step: count(784), step_time 1419.34, mean_step_time 1493.36, it/s 0.67
  3905. sending to address tcp://p0112:61216
  3906. ##### Sending to neptune: online_score : 0.310479466054 , 1.7 #####
  3907. [u'online', 1.7]
  3908. receiving
  3909. [0130 17:48:38 @multigpu.py:323] [p0574] step: count(775), step_time 1417.12, mean_step_time 1451.24, it/s 0.69
  3910. [0130 17:48:39 @multigpu.py:323] [p0115] step: count(761), step_time 1476.73, mean_step_time 1473.86, it/s 0.68
  3911. [0130 17:48:40 @multigpu.py:323] [p0576] step: count(785), step_time 1408.38, mean_step_time 1493.55, it/s 0.67
  3912. [0130 17:48:40 @multigpu.py:323] [p0574] step: count(776), step_time 1408.75, mean_step_time 1447.14, it/s 0.69
  3913. [0130 17:48:41 @multigpu.py:323] [p0115] step: count(762), step_time 1445.59, mean_step_time 1471.09, it/s 0.68
  3914. [0130 17:48:41 @multigpu.py:323] [p0576] step: count(786), step_time 1509.02, mean_step_time 1498.93, it/s 0.67
  3915. [0130 17:48:41 @multigpu.py:323] [p0574] step: count(777), step_time 1415.0, mean_step_time 1437.9, it/s 0.7
  3916. [0130 17:48:42 @multigpu.py:323] [p0115] step: count(763), step_time 1418.96, mean_step_time 1469.23, it/s 0.68
  3917. [0130 17:48:43 @multigpu.py:323] [p0576] step: count(787), step_time 1405.37, mean_step_time 1495.54, it/s 0.67
  3918. [0130 17:48:43 @multigpu.py:323] [p0574] step: count(778), step_time 1384.63, mean_step_time 1428.94, it/s 0.7
  3919. sending to address tcp://p0112:61216
  3920. ##### Sending to neptune: online_score : 0.311744225224 , 2.4 #####
  3921. [u'online', 2.4]
  3922. ##### Sending to neptune: active_workers : 0.311744298604 , 3 #####
  3923. receiving
  3924. [0130 17:48:44 @multigpu.py:323] [p0115] step: count(764), step_time 1405.48, mean_step_time 1466.62, it/s 0.68
  3925. [0130 17:48:44 @multigpu.py:323] [p0576] step: count(788), step_time 1391.93, mean_step_time 1495.08, it/s 0.67
  3926. [0130 17:48:44 @multigpu.py:323] [p0574] step: count(779), step_time 1459.28, mean_step_time 1429.46, it/s 0.7
  3927. [0130 17:48:45 @multigpu.py:323] [p0115] step: count(765), step_time 1454.4, mean_step_time 1467.79, it/s 0.68
  3928. [0130 17:48:45 @multigpu.py:323] [p0576] step: count(789), step_time 1437.04, mean_step_time 1495.27, it/s 0.67
  3929. [0130 17:48:45 @multigpu.py:323] [p0574] step: count(780), step_time 1420.63, mean_step_time 1429.51, it/s 0.7
  3930. [0130 17:48:47 @multigpu.py:323] [p0115] step: count(766), step_time 1402.97, mean_step_time 1467.1, it/s 0.68
  3931. [0130 17:48:47 @multigpu.py:323] [p0576] step: count(790), step_time 1389.53, mean_step_time 1493.65, it/s 0.67
  3932. [0130 17:48:47 @multigpu.py:323] [p0574] step: count(781), step_time 1421.5, mean_step_time 1430.78, it/s 0.7
  3933. [0130 17:48:48 @multigpu.py:323] [p0115] step: count(767), step_time 1469.79, mean_step_time 1466.65, it/s 0.68
  3934. [0130 17:48:48 @multigpu.py:323] [p0576] step: count(791), step_time 1432.2, mean_step_time 1494.39, it/s 0.67
  3935. [0130 17:48:48 @multigpu.py:323] [p0574] step: count(782), step_time 1412.91, mean_step_time 1431.13, it/s 0.7
  3936. [0130 17:48:50 @multigpu.py:323] [p0115] step: count(768), step_time 1627.26, mean_step_time 1432.46, it/s 0.7
  3937. [0130 17:48:50 @multigpu.py:323] [p0576] step: count(792), step_time 1555.32, mean_step_time 1432.33, it/s 0.7
  3938. [0130 17:48:50 @multigpu.py:323] [p0574] step: count(783), step_time 1456.03, mean_step_time 1432.38, it/s 0.7
  3939. [0130 17:48:51 @multigpu.py:323] [p0574] step: count(784), step_time 1414.5, mean_step_time 1431.27, it/s 0.7
  3940. [0130 17:48:51 @multigpu.py:323] [p0115] step: count(769), step_time 1418.66, mean_step_time 1432.25, it/s 0.7
  3941. [0130 17:48:51 @multigpu.py:323] [p0576] step: count(793), step_time 1477.83, mean_step_time 1433.05, it/s 0.7
  3942. [0130 17:48:53 @multigpu.py:323] [p0574] step: count(785), step_time 1395.25, mean_step_time 1427.68, it/s 0.7
  3943. [0130 17:48:53 @multigpu.py:323] [p0576] step: count(794), step_time 1397.31, mean_step_time 1432.4, it/s 0.7
  3944. [0130 17:48:53 @multigpu.py:323] [p0115] step: count(770), step_time 1479.11, mean_step_time 1435.27, it/s 0.7
  3945. [0130 17:48:54 @multigpu.py:323] [p0574] step: count(786), step_time 1441.36, mean_step_time 1427.18, it/s 0.7
  3946. [0130 17:48:54 @multigpu.py:323] [p0576] step: count(795), step_time 1391.2, mean_step_time 1430.75, it/s 0.7
  3947. [0130 17:48:54 @multigpu.py:323] [p0115] step: count(771), step_time 1410.43, mean_step_time 1434.49, it/s 0.7
  3948. [0130 17:48:55 @multigpu.py:323] [p0576] step: count(796), step_time 1407.22, mean_step_time 1428.01, it/s 0.7
  3949. [0130 17:48:55 @multigpu.py:323] [p0115] step: count(772), step_time 1371.4, mean_step_time 1433.8, it/s 0.7
  3950. [0130 17:48:55 @multigpu.py:323] [p0574] step: count(787), step_time 1435.45, mean_step_time 1425.62, it/s 0.7
  3951. [0130 17:48:57 @multigpu.py:323] [p0576] step: count(797), step_time 1437.44, mean_step_time 1429.94, it/s 0.7
  3952. [0130 17:48:57 @multigpu.py:323] [p0115] step: count(773), step_time 1437.88, mean_step_time 1434.2, it/s 0.7
  3953. [0130 17:48:57 @multigpu.py:323] [p0574] step: count(788), step_time 1518.88, mean_step_time 1426.93, it/s 0.7
  3954. [0130 17:48:58 @multigpu.py:323] [p0576] step: count(798), step_time 1373.41, mean_step_time 1429.46, it/s 0.7
  3955. [0130 17:48:58 @multigpu.py:323] [p0115] step: count(774), step_time 1420.67, mean_step_time 1434.57, it/s 0.7
  3956. [0130 17:48:58 @multigpu.py:323] [p0574] step: count(789), step_time 1434.05, mean_step_time 1429.85, it/s 0.7
  3957. [0130 17:49:00 @multigpu.py:323] [p0576] step: count(799), step_time 1400.6, mean_step_time 1426.62, it/s 0.7
  3958. [0130 17:49:00 @multigpu.py:323] [p0115] step: count(775), step_time 1451.44, mean_step_time 1436.42, it/s 0.7
  3959. sending to address tcp://p0112:61216
  3960. ##### Sending to neptune: online_score : 0.316423896088 , 1.5 #####
  3961. [u'online', 1.5]
  3962. receiving
  3963. [0130 17:49:00 @multigpu.py:323] [p0574] step: count(790), step_time 1374.9, mean_step_time 1427.62, it/s 0.7
  3964. [0130 17:49:01 @multigpu.py:323] [p0576] step: count(800), step_time 1407.41, mean_step_time 1427.18, it/s 0.7
  3965. sending debugging info...
  3966. sending to address tcp://p0112:61216
  3967. ##### Sending to neptune: mean_delay : 0.316783569985 , 0.0 #####
  3968. ##### Sending to neptune: max_delay : 0.316783569985 , -0.0 #####
  3969. sending to address tcp://p0112:61216
  3970. ##### Sending to neptune: min_delay : 0.316783569985 , -0.0 #####
  3971. [u'delays', [0.0, -0.0, -0.0]]
  3972. receiving
  3973. ##### Sending to neptune: cost : 0.316784079936 , -0.00433429796249 #####
  3974. ##### Sending to neptune: policy_loss : 0.316784079936 , 0.223585680127 #####
  3975. sending to address tcp://p0112:61216
  3976. ##### Sending to neptune: xentropy_loss : 0.316784079936 , -2.28890061378 #####
  3977. ##### Sending to neptune: value_loss : 0.316784079936 , 1.51052486897 #####
  3978. ##### Sending to neptune: advantage : 0.316784079936 , -0.00105175154749 #####
  3979. ##### Sending to neptune: pred_reward : 0.316784079936 , 0.432729452848 #####
  3980. ##### Sending to neptune: max_logit : 0.316784079936 , 0.198514163494 #####
  3981. [u'loss', -0.004334297962486744, 0.22358568012714386, -2.28890061378479, 1.510524868965149, -0.0010517515474930406, 0.43272945284843445, 0.1985141634941101]
  3982. receiving
  3983. ##### Sending to neptune: active_relus : 0.316784537435 , 9006400.55 #####
  3984. ##### Sending to neptune: dp_per_s : 0.316784537435 , 88.4437987855 #####
  3985. [u'other', 9006400.55, 88.44379878546097]
  3986. receiving
  3987. [0130 17:49:01 @multigpu.py:323] [p0115] step: count(776), step_time 1429.33, mean_step_time 1436.61, it/s 0.7
  3988. [0130 17:49:01 @multigpu.py:323] [p0574] step: count(791), step_time 1407.01, mean_step_time 1428.7, it/s 0.7
  3989. [0130 17:49:02 @multigpu.py:323] [p0576] step: count(801), step_time 1415.41, mean_step_time 1427.02, it/s 0.7
  3990. [0130 17:49:03 @multigpu.py:323] [p0574] step: count(792), step_time 1369.7, mean_step_time 1425.81, it/s 0.7
  3991. [0130 17:49:03 @multigpu.py:323] [p0115] step: count(777), step_time 1390.76, mean_step_time 1437.44, it/s 0.7
  3992. sending to address tcp://p0112:61216
  3993. ##### Sending to neptune: online_score : 0.317268399994 , 1.7 #####
  3994. [u'online', 1.7]
  3995. receiving
  3996. [0130 17:49:04 @multigpu.py:323] [p0576] step: count(802), step_time 1496.01, mean_step_time 1430.75, it/s 0.7
  3997. [0130 17:49:04 @multigpu.py:323] [p0574] step: count(793), step_time 1414.25, mean_step_time 1423.94, it/s 0.7
  3998. [0130 17:49:04 @multigpu.py:323] [p0115] step: count(778), step_time 1460.37, mean_step_time 1441.12, it/s 0.69
  3999. sending to address tcp://p0112:61216
  4000. ##### Sending to neptune: online_score : 0.317744605806 , 1.5 #####
  4001. [u'online', 1.5]
  4002. receiving
  4003. [0130 17:49:05 @multigpu.py:323] [p0576] step: count(803), step_time 1389.31, mean_step_time 1427.06, it/s 0.7
  4004. [0130 17:49:05 @multigpu.py:323] [p0574] step: count(794), step_time 1399.28, mean_step_time 1420.02, it/s 0.7
  4005. [0130 17:49:05 @multigpu.py:323] [p0115] step: count(779), step_time 1458.67, mean_step_time 1441.69, it/s 0.69
  4006. [0130 17:49:07 @multigpu.py:323] [p0576] step: count(804), step_time 1407.45, mean_step_time 1426.47, it/s 0.7
  4007. [0130 17:49:07 @multigpu.py:323] [p0574] step: count(795), step_time 1521.25, mean_step_time 1425.23, it/s 0.7
  4008. [0130 17:49:07 @multigpu.py:323] [p0115] step: count(780), step_time 1413.54, mean_step_time 1442.17, it/s 0.69
  4009. [0130 17:49:08 @multigpu.py:323] [p0576] step: count(805), step_time 1423.43, mean_step_time 1427.22, it/s 0.7
  4010. [0130 17:49:08 @multigpu.py:323] [p0574] step: count(796), step_time 1395.91, mean_step_time 1424.59, it/s 0.7
  4011. [0130 17:49:08 @multigpu.py:323] [p0115] step: count(781), step_time 1415.45, mean_step_time 1439.11, it/s 0.69
  4012. [0130 17:49:10 @multigpu.py:323] [p0576] step: count(806), step_time 1426.55, mean_step_time 1423.1, it/s 0.7
  4013. [0130 17:49:10 @multigpu.py:323] [p0574] step: count(797), step_time 1403.43, mean_step_time 1424.01, it/s 0.7
  4014. [0130 17:49:10 @multigpu.py:323] [p0115] step: count(782), step_time 1402.94, mean_step_time 1436.98, it/s 0.7
  4015. [0130 17:49:11 @multigpu.py:323] [p0576] step: count(807), step_time 1405.85, mean_step_time 1423.12, it/s 0.7
  4016. [0130 17:49:11 @multigpu.py:323] [p0574] step: count(798), step_time 1417.62, mean_step_time 1425.66, it/s 0.7
  4017. [0130 17:49:11 @multigpu.py:323] [p0115] step: count(783), step_time 1446.93, mean_step_time 1438.37, it/s 0.7
  4018. [0130 17:49:12 @multigpu.py:323] [p0576] step: count(808), step_time 1450.79, mean_step_time 1426.07, it/s 0.7
  4019. [0130 17:49:12 @multigpu.py:323] [p0574] step: count(799), step_time 1408.44, mean_step_time 1423.12, it/s 0.7
  4020. [0130 17:49:13 @multigpu.py:323] [p0115] step: count(784), step_time 1434.27, mean_step_time 1439.81, it/s 0.69
  4021. [0130 17:49:14 @multigpu.py:323] [p0574] step: count(800), step_time 1419.21, mean_step_time 1423.05, it/s 0.7
  4022. sending debugging info...
  4023. sending to address tcp://p0112:61216
  4024. ##### Sending to neptune: mean_delay : 0.32036268466 , 0.0 #####
  4025. ##### Sending to neptune: max_delay : 0.32036268466 , -0.0 #####
  4026. sending to address tcp://p0112:61216
  4027. ##### Sending to neptune: min_delay : 0.32036268466 , -0.0 #####
  4028. [u'delays', [0.0, -0.0, -0.0]]
  4029. receiving
  4030. ##### Sending to neptune: cost : 0.320363213552 , -0.00875152647495 #####
  4031. ##### Sending to neptune: policy_loss : 0.320363213552 , -0.275862365961 #####
  4032. sending to address tcp://p0112:61216
  4033. ##### Sending to neptune: xentropy_loss : 0.320363213552 , -2.28863835335 #####
  4034. ##### Sending to neptune: value_loss : 0.320363213552 , 1.44430506229 #####
  4035. ##### Sending to neptune: advantage : 0.320363213552 , 0.00120054150466 #####
  4036. ##### Sending to neptune: pred_reward : 0.320363213552 , 0.438394933939 #####
  4037. ##### Sending to neptune: max_logit : 0.320363213552 , 0.199205473065 #####
  4038. [u'loss', -0.008751526474952698, -0.27586236596107483, -2.2886383533477783, 1.4443050622940063, 0.0012005415046587586, 0.4383949339389801, 0.19920547306537628]
  4039. [0130 17:49:14 @multigpu.py:323] [p0576] step: count(809), step_time 1471.75, mean_step_time 1427.8, it/s 0.7
  4040. receiving
  4041. ##### Sending to neptune: active_relus : 0.320363672442 , 9008079.44 #####
  4042. ##### Sending to neptune: dp_per_s : 0.320363672442 , 86.8237239399 #####
  4043. [u'other', 9008079.44, 86.82372393990129]
  4044. receiving
  4045. [0130 17:49:14 @multigpu.py:323] [p0115] step: count(785), step_time 1388.29, mean_step_time 1436.51, it/s 0.7
  4046. [0130 17:49:15 @multigpu.py:323] [p0576] step: count(810), step_time 1411.18, mean_step_time 1428.88, it/s 0.7
  4047. [0130 17:49:15 @multigpu.py:323] [p0574] step: count(801), step_time 1438.67, mean_step_time 1423.91, it/s 0.7
  4048. [0130 17:49:15 @multigpu.py:323] [p0115] step: count(786), step_time 1420.86, mean_step_time 1437.4, it/s 0.7
  4049. [0130 17:49:17 @multigpu.py:323] [p0576] step: count(811), step_time 1454.29, mean_step_time 1429.99, it/s 0.7
  4050. [0130 17:49:17 @multigpu.py:323] [p0115] step: count(787), step_time 1397.73, mean_step_time 1433.8, it/s 0.7
  4051. [0130 17:49:17 @multigpu.py:323] [p0574] step: count(802), step_time 1481.59, mean_step_time 1427.34, it/s 0.7
  4052. [0130 17:49:18 @multigpu.py:323] [p0574] step: count(803), step_time 1418.38, mean_step_time 1425.46, it/s 0.7
  4053. [0130 17:49:18 @multigpu.py:323] [p0576] step: count(812), step_time 1467.82, mean_step_time 1425.61, it/s 0.7
  4054. [0130 17:49:18 @multigpu.py:323] [p0115] step: count(788), step_time 1465.4, mean_step_time 1425.71, it/s 0.7
  4055. [0130 17:49:20 @multigpu.py:323] [p0574] step: count(804), step_time 1392.49, mean_step_time 1424.36, it/s 0.7
  4056. [0130 17:49:20 @multigpu.py:323] [p0115] step: count(789), step_time 1419.76, mean_step_time 1425.76, it/s 0.7
  4057. [0130 17:49:20 @multigpu.py:323] [p0576] step: count(813), step_time 1429.68, mean_step_time 1423.21, it/s 0.7
  4058. [0130 17:49:21 @multigpu.py:323] [p0576] step: count(814), step_time 1372.1, mean_step_time 1421.95, it/s 0.7
  4059. [0130 17:49:21 @multigpu.py:323] [p0574] step: count(805), step_time 1411.0, mean_step_time 1425.14, it/s 0.7
  4060. [0130 17:49:21 @multigpu.py:323] [p0115] step: count(790), step_time 1485.53, mean_step_time 1426.08, it/s 0.7
  4061. [0130 17:49:22 @multigpu.py:323] [p0576] step: count(815), step_time 1413.62, mean_step_time 1423.07, it/s 0.7
  4062. [0130 17:49:23 @multigpu.py:323] [p0574] step: count(806), step_time 1455.09, mean_step_time 1425.83, it/s 0.7
  4063. [0130 17:49:23 @multigpu.py:323] [p0115] step: count(791), step_time 1448.14, mean_step_time 1427.97, it/s 0.7
  4064. sending to address tcp://p0112:61216
  4065. ##### Sending to neptune: online_score : 0.322972231905 , 1.4 #####
  4066. [u'online', 1.4]
  4067. receiving
  4068. [0130 17:49:24 @multigpu.py:323] [p0576] step: count(816), step_time 1431.39, mean_step_time 1424.28, it/s 0.7
  4069. [0130 17:49:24 @multigpu.py:323] [p0574] step: count(807), step_time 1466.36, mean_step_time 1427.37, it/s 0.7
  4070. [0130 17:49:24 @multigpu.py:323] [p0115] step: count(792), step_time 1426.63, mean_step_time 1430.73, it/s 0.7
  4071. [0130 17:49:25 @multigpu.py:323] [p0576] step: count(817), step_time 1429.91, mean_step_time 1423.9, it/s 0.7
  4072. [0130 17:49:25 @multigpu.py:323] [p0574] step: count(808), step_time 1424.38, mean_step_time 1422.65, it/s 0.7
  4073. [0130 17:49:26 @multigpu.py:323] [p0115] step: count(793), step_time 1473.78, mean_step_time 1432.52, it/s 0.7
  4074. [0130 17:49:27 @multigpu.py:323] [p0576] step: count(818), step_time 1418.34, mean_step_time 1426.14, it/s 0.7
  4075. [0130 17:49:27 @multigpu.py:323] [p0574] step: count(809), step_time 1412.76, mean_step_time 1421.59, it/s 0.7
  4076. [0130 17:49:27 @multigpu.py:323] [p0115] step: count(794), step_time 1480.85, mean_step_time 1435.53, it/s 0.7
  4077. [0130 17:49:28 @multigpu.py:323] [p0576] step: count(819), step_time 1430.83, mean_step_time 1427.66, it/s 0.7
  4078. [0130 17:49:28 @multigpu.py:323] [p0574] step: count(810), step_time 1391.33, mean_step_time 1422.41, it/s 0.7
  4079. [0130 17:49:28 @multigpu.py:323] [p0115] step: count(795), step_time 1477.14, mean_step_time 1436.82, it/s 0.7
  4080. sending to address tcp://p0112:61216
  4081. ##### Sending to neptune: online_score : 0.324464672738 , 1.3 #####
  4082. [u'online', 1.3]
  4083. receiving
  4084. [0130 17:49:30 @multigpu.py:323] [p0576] step: count(820), step_time 1396.28, mean_step_time 1427.1, it/s 0.7
  4085. [0130 17:49:30 @multigpu.py:323] [p0574] step: count(811), step_time 1441.45, mean_step_time 1424.13, it/s 0.7
  4086. [0130 17:49:30 @multigpu.py:323] [p0115] step: count(796), step_time 1473.22, mean_step_time 1439.01, it/s 0.69
  4087. [0130 17:49:31 @multigpu.py:323] [p0576] step: count(821), step_time 1408.66, mean_step_time 1426.76, it/s 0.7
  4088. [0130 17:49:31 @multigpu.py:323] [p0574] step: count(812), step_time 1404.4, mean_step_time 1425.86, it/s 0.7
  4089. [0130 17:49:31 @multigpu.py:323] [p0115] step: count(797), step_time 1453.41, mean_step_time 1442.15, it/s 0.69
  4090. 80%|#######9 |799/1000[19:[0130 17:49:32 @multigpu.py:323] [p0576] step: count(822), step_time 1435.22, mean_step_time 1423.72, it/s 0.7
  4091. [0130 17:49:32 @multigpu.py:323] [p0574] step: count(813), step_time 1444.78, mean_step_time 1427.39, it/s 0.7
  4092. [0130 17:49:33 @multigpu.py:323] [p0115] step: count(798), step_time 1475.66, mean_step_time 1442.91, it/s 0.69
  4093. [0130 17:49:34 @multigpu.py:323] [p0576] step: count(823), step_time 1458.8, mean_step_time 1427.2, it/s 0.7
  4094. [0130 17:49:34 @multigpu.py:323] [p0574] step: count(814), step_time 1431.78, mean_step_time 1429.02, it/s 0.7
  4095. [0130 17:49:34 @multigpu.py:323] [p0115] step: count(799), step_time 1477.14, mean_step_time 1443.83, it/s 0.69
  4096. [0130 17:49:35 @multigpu.py:323] [p0576] step: count(824), step_time 1397.17, mean_step_time 1426.68, it/s 0.7
  4097. [0130 17:49:35 @multigpu.py:323] [p0574] step: count(815), step_time 1377.52, mean_step_time 1421.83, it/s 0.7
  4098. [0130 17:49:36 @multigpu.py:323] [p0115] step: count(800), step_time 1403.05, mean_step_time 1443.31, it/s 0.69
  4099. sending debugging info...
  4100. sending to address tcp://p0112:61216
  4101. ##### Sending to neptune: mean_delay : 0.326430359946 , 0.0 #####
  4102. sending to address tcp://p0112:61216
  4103. sending to address tcp://p0112:61216
  4104. [0130 17:49:37 @multigpu.py:323] [p0576] step: count(825), step_time 1424.09, mean_step_time 1426.72, it/s 0.7
  4105. [0130 17:49:37 @multigpu.py:323] [p0574] step: count(816), step_time 1485.52, mean_step_time 1426.31, it/s 0.7
  4106. sending to address tcp://p0112:61216
  4107. [0130 17:49:37 @multigpu.py:323] [p0115] step: count(801), step_time 1467.26, mean_step_time 1445.9, it/s 0.69
  4108. [0130 17:49:38 @multigpu.py:323] [p0576] step: count(826), step_time 1476.38, mean_step_time 1429.21, it/s 0.7
  4109. [0130 17:49:38 @multigpu.py:323] [p0574] step: count(817), step_time 1391.94, mean_step_time 1425.74, it/s 0.7
  4110. [0130 17:49:39 @multigpu.py:323] [p0115] step: count(802), step_time 1377.75, mean_step_time 1444.64, it/s 0.69
  4111. [0130 17:49:40 @multigpu.py:323] [p0576] step: count(827), step_time 1429.75, mean_step_time 1430.4, it/s 0.7
  4112. [0130 17:49:40 @multigpu.py:323] [p0574] step: count(818), step_time 1457.73, mean_step_time 1427.74, it/s 0.7
  4113. [0130 17:49:40 @multigpu.py:323] [p0115] step: count(803), step_time 1449.73, mean_step_time 1444.78, it/s 0.69
  4114. ##### Sending to neptune: max_delay : 0.326430359946 , -0.0 #####
  4115. ##### Sending to neptune: min_delay : 0.326430359946 , -0.0 #####
  4116. [u'delays', [0.0, -0.0, -0.0]]
  4117. receiving
  4118. ##### Sending to neptune: cost : 0.327900048296 , -0.0139869665727 #####
  4119. ##### Sending to neptune: policy_loss : 0.327900048296 , -0.741453886032 #####
  4120. ##### Sending to neptune: xentropy_loss : 0.327900048296 , -2.2882630825 #####
  4121. ##### Sending to neptune: value_loss : 0.327900048296 , 1.23938500881 #####
  4122. ##### Sending to neptune: advantage : 0.327900048296 , 0.003333655186 #####
  4123. ##### Sending to neptune: pred_reward : 0.327900048296 , 0.43941155076 #####
  4124. ##### Sending to neptune: max_logit : 0.327900048296 , 0.20052549243 #####
  4125. [u'loss', -0.013986966572701931, -0.7414538860321045, -2.2882630825042725, 1.2393850088119507, 0.003333655185997486, 0.43941155076026917, 0.20052549242973328]
  4126. receiving
  4127. ##### Sending to neptune: active_relus : 0.327902424998 , 9061856.05 #####
  4128. ##### Sending to neptune: dp_per_s : 0.327902424998 , 87.6794406125 #####
  4129. [u'other', 9061856.05, 87.6794406124564]
  4130. receiving
  4131. ##### Sending to neptune: online_score : 0.327903013031 , 1.2 #####
  4132. [u'online', 1.2]
  4133. receiving
  4134. [0130 17:49:41 @multigpu.py:323] [p0576] step: count(828), step_time 1446.81, mean_step_time 1430.2, it/s 0.7
  4135. [0130 17:49:41 @multigpu.py:323] [p0574] step: count(819), step_time 1411.12, mean_step_time 1427.88, it/s 0.7
  4136. [0130 17:49:41 @multigpu.py:323] [p0115] step: count(804), step_time 1437.33, mean_step_time 1444.93, it/s 0.69
  4137. [0130 17:49:42 @multigpu.py:323] [p0574] step: count(820), step_time 1378.63, mean_step_time 1425.85, it/s 0.7
  4138. [0130 17:49:43 @multigpu.py:323] [p0576] step: count(829), step_time 1459.28, mean_step_time 1429.58, it/s 0.7
  4139. [0130 17:49:43 @multigpu.py:323] [p0115] step: count(805), step_time 1406.53, mean_step_time 1445.84, it/s 0.69
  4140. [0130 17:49:44 @multigpu.py:323] [p0574] step: count(821), step_time 1396.77, mean_step_time 1423.75, it/s 0.7
  4141. [0130 17:49:44 @multigpu.py:323] [p0576] step: count(830), step_time 1395.62, mean_step_time 1428.8, it/s 0.7
  4142. [0130 17:49:44 @multigpu.py:323] [p0115] step: count(806), step_time 1423.04, mean_step_time 1445.95, it/s 0.69
  4143. [0130 17:49:45 @multigpu.py:323] [p0574] step: count(822), step_time 1447.25, mean_step_time 1422.03, it/s 0.7
  4144. [0130 17:49:45 @multigpu.py:323] [p0576] step: count(831), step_time 1470.52, mean_step_time 1429.61, it/s 0.7
  4145. [0130 17:49:46 @multigpu.py:323] [p0115] step: count(807), step_time 1488.77, mean_step_time 1450.51, it/s 0.69
  4146. [0130 17:49:47 @multigpu.py:323] [p0115] step: count(808), step_time 1465.87, mean_step_time 1450.53, it/s 0.69
  4147. [0130 17:49:47 @multigpu.py:323] [p0576] step: count(832), step_time 1888.2, mean_step_time 1450.63, it/s 0.69
  4148. [0130 17:49:47 @multigpu.py:323] [p0574] step: count(823), step_time 1990.13, mean_step_time 1450.62, it/s 0.69
  4149. [0130 17:49:49 @multigpu.py:323] [p0115] step: count(809), step_time 1385.91, mean_step_time 1448.84, it/s 0.69
  4150. [0130 17:49:49 @multigpu.py:323] [p0576] step: count(833), step_time 1420.86, mean_step_time 1450.19, it/s 0.69
  4151. [0130 17:49:49 @multigpu.py:323] [p0574] step: count(824), step_time 1462.15, mean_step_time 1454.1, it/s 0.69
  4152. [0130 17:49:50 @multigpu.py:323] [p0115] step: count(810), step_time 1418.66, mean_step_time 1445.49, it/s 0.69
  4153. [0130 17:49:50 @multigpu.py:323] [p0576] step: count(834), step_time 1409.05, mean_step_time 1452.04, it/s 0.69
  4154. [0130 17:49:50 @multigpu.py:323] [p0574] step: count(825), step_time 1411.84, mean_step_time 1454.15, it/s 0.69
  4155. [0130 17:49:51 @multigpu.py:323] [p0115] step: count(811), step_time 1412.64, mean_step_time 1443.72, it/s 0.69
  4156. [0130 17:49:52 @multigpu.py:323] [p0576] step: count(835), step_time 1400.96, mean_step_time 1451.41, it/s 0.69
  4157. [0130 17:49:52 @multigpu.py:323] [p0574] step: count(826), step_time 1382.74, mean_step_time 1450.53, it/s 0.69
  4158. [0130 17:49:53 @multigpu.py:323] [p0115] step: count(812), step_time 1422.38, mean_step_time 1443.5, it/s 0.69
  4159. [0130 17:49:53 @multigpu.py:323] [p0574] step: count(827), step_time 1434.28, mean_step_time 1448.92, it/s 0.69
  4160. [0130 17:49:53 @multigpu.py:323] [p0576] step: count(836), step_time 1509.3, mean_step_time 1455.3, it/s 0.69
  4161. sending to address tcp://p0112:61216
  4162. ##### Sending to neptune: online_score : 0.33134215388 , 0.8 #####
  4163. [u'online', 0.8]
  4164. ##### Sending to neptune: active_workers : 0.331342185802 , 3 #####
  4165. receiving
  4166. [0130 17:49:54 @multigpu.py:323] [p0115] step: count(813), step_time 1456.38, mean_step_time 1442.63, it/s 0.69
  4167. [0130 17:49:54 @multigpu.py:323] [p0574] step: count(828), step_time 1431.32, mean_step_time 1449.27, it/s 0.69
  4168. [0130 17:49:54 @multigpu.py:323] [p0576] step: count(837), step_time 1418.01, mean_step_time 1454.71, it/s 0.69
  4169. [0130 17:49:56 @multigpu.py:323] [p0115] step: count(814), step_time 1406.15, mean_step_time 1438.9, it/s 0.69
  4170. [0130 17:49:56 @multigpu.py:323] [p0574] step: count(829), step_time 1429.44, mean_step_time 1450.11, it/s 0.69
  4171. [0130 17:49:56 @multigpu.py:323] [p0576] step: count(838), step_time 1479.45, mean_step_time 1457.76, it/s 0.69
  4172. sending to address tcp://p0112:61216
  4173. ##### Sending to neptune: online_score : 0.332217746642 , 0.9 #####
  4174. [u'online', 0.9]
  4175. receiving
  4176. [0130 17:49:57 @multigpu.py:323] [p0115] step: count(815), step_time 1472.49, mean_step_time 1438.67, it/s 0.7
  4177. [0130 17:49:57 @multigpu.py:323] [p0574] step: count(830), step_time 1459.66, mean_step_time 1453.52, it/s 0.69
  4178. [0130 17:49:57 @multigpu.py:323] [p0576] step: count(839), step_time 1454.89, mean_step_time 1458.96, it/s 0.69
  4179. [0130 17:49:59 @multigpu.py:323] [p0115] step: count(816), step_time 1392.18, mean_step_time 1434.61, it/s 0.7
  4180. [0130 17:49:59 @multigpu.py:323] [p0574] step: count(831), step_time 1366.21, mean_step_time 1449.76, it/s 0.69
  4181. [0130 17:49:59 @multigpu.py:323] [p0576] step: count(840), step_time 1417.42, mean_step_time 1460.02, it/s 0.68
  4182. sending to address tcp://p0112:61216
  4183. ##### Sending to neptune: online_score : 0.332876266903 , 1.4 #####
  4184. [u'online', 1.4]
  4185. receiving
  4186. [0130 17:50:00 @multigpu.py:323] [p0574] step: count(832), step_time 1406.13, mean_step_time 1449.85, it/s 0.69
  4187. [0130 17:50:00 @multigpu.py:323] [p0115] step: count(817), step_time 1455.63, mean_step_time 1434.73, it/s 0.7
  4188. [0130 17:50:00 @multigpu.py:323] [p0576] step: count(841), step_time 1411.56, mean_step_time 1460.17, it/s 0.68
  4189. 81%|######## |806/1000[19:[0130 17:50:02 @multigpu.py:323] [p0574] step: count(833), step_time 1442.25, mean_step_time 1449.72, it/s 0.69
  4190. [0130 17:50:02 @multigpu.py:323] [p0115] step: count(818), step_time 1425.83, mean_step_time 1432.23, it/s 0.7
  4191. [0130 17:50:02 @multigpu.py:323] [p0576] step: count(842), step_time 1418.03, mean_step_time 1459.31, it/s 0.69
  4192. [0130 17:50:03 @multigpu.py:323] [p0574] step: count(834), step_time 1427.28, mean_step_time 1449.5, it/s 0.69
  4193. [0130 17:50:03 @multigpu.py:323] [p0115] step: count(819), step_time 1511.54, mean_step_time 1433.95, it/s 0.7
  4194. [0130 17:50:03 @multigpu.py:323] [p0576] step: count(843), step_time 1472.28, mean_step_time 1459.98, it/s 0.68
  4195. [0130 17:50:04 @multigpu.py:323] [p0574] step: count(835), step_time 1462.29, mean_step_time 1453.73, it/s 0.69
  4196. [0130 17:50:04 @multigpu.py:323] [p0115] step: count(820), step_time 1412.58, mean_step_time 1434.43, it/s 0.7
  4197. [0130 17:50:04 @multigpu.py:323] [p0576] step: count(844), step_time 1389.04, mean_step_time 1459.57, it/s 0.69
  4198. [0130 17:50:06 @multigpu.py:323] [p0574] step: count(836), step_time 1450.97, mean_step_time 1452.01, it/s 0.69
  4199. [0130 17:50:06 @multigpu.py:323] [p0115] step: count(821), step_time 1431.96, mean_step_time 1432.67, it/s 0.7
  4200. [0130 17:50:06 @multigpu.py:323] [p0576] step: count(845), step_time 1426.12, mean_step_time 1459.68, it/s 0.69
  4201. [0130 17:50:07 @multigpu.py:323] [p0574] step: count(837), step_time 1483.71, mean_step_time 1456.59, it/s 0.69
  4202. [0130 17:50:07 @multigpu.py:323] [p0576] step: count(846), step_time 1441.06, mean_step_time 1457.91, it/s 0.69
  4203. [0130 17:50:07 @multigpu.py:323] [p0115] step: count(822), step_time 1465.82, mean_step_time 1437.07, it/s 0.7
  4204. [0130 17:50:09 @multigpu.py:323] [p0574] step: count(838), step_time 1427.26, mean_step_time 1455.07, it/s 0.69
  4205. [0130 17:50:09 @multigpu.py:323] [p0115] step: count(823), step_time 1444.49, mean_step_time 1436.81, it/s 0.7
  4206. [0130 17:50:09 @multigpu.py:323] [p0576] step: count(847), step_time 1471.8, mean_step_time 1460.01, it/s 0.68
  4207. [0130 17:50:10 @multigpu.py:323] [p0574] step: count(839), step_time 1411.78, mean_step_time 1455.1, it/s 0.69
  4208. [0130 17:50:10 @multigpu.py:323] [p0576] step: count(848), step_time 1398.02, mean_step_time 1457.57, it/s 0.69
  4209. [0130 17:50:10 @multigpu.py:323] [p0115] step: count(824), step_time 1426.87, mean_step_time 1436.29, it/s 0.7
  4210. [0130 17:50:12 @multigpu.py:323] [p0574] step: count(840), step_time 1421.72, mean_step_time 1457.26, it/s 0.69
  4211. [0130 17:50:12 @multigpu.py:323] [p0576] step: count(849), step_time 1433.66, mean_step_time 1456.29, it/s 0.69
  4212. [0130 17:50:12 @multigpu.py:323] [p0115] step: count(825), step_time 1437.46, mean_step_time 1437.83, it/s 0.7
  4213. [0130 17:50:13 @multigpu.py:323] [p0574] step: count(841), step_time 1459.84, mean_step_time 1460.41, it/s 0.68
  4214. [0130 17:50:13 @multigpu.py:323] [p0115] step: count(826), step_time 1400.72, mean_step_time 1436.72, it/s 0.7
  4215. [0130 17:50:13 @multigpu.py:323] [p0576] step: count(850), step_time 1425.38, mean_step_time 1457.78, it/s 0.69
  4216. [0130 17:50:14 @multigpu.py:323] [p0115] step: count(827), step_time 1374.98, mean_step_time 1431.03, it/s 0.7
  4217. [0130 17:50:14 @multigpu.py:323] [p0574] step: count(842), step_time 1388.76, mean_step_time 1457.49, it/s 0.69
  4218. [0130 17:50:15 @multigpu.py:323] [p0576] step: count(851), step_time 1446.51, mean_step_time 1456.58, it/s 0.69
  4219. [0130 17:50:16 @multigpu.py:323] [p0115] step: count(828), step_time 1508.09, mean_step_time 1433.14, it/s 0.7
  4220. [0130 17:50:16 @multigpu.py:323] [p0576] step: count(852), step_time 1418.63, mean_step_time 1433.1, it/s 0.7
  4221. [0130 17:50:16 @multigpu.py:323] [p0574] step: count(843), step_time 1501.38, mean_step_time 1433.05, it/s 0.7
  4222. [0130 17:50:17 @multigpu.py:323] [p0576] step: count(853), step_time 1382.78, mean_step_time 1431.2, it/s 0.7
  4223. [0130 17:50:17 @multigpu.py:323] [p0115] step: count(829), step_time 1417.21, mean_step_time 1434.7, it/s 0.7
  4224. [0130 17:50:17 @multigpu.py:323] [p0574] step: count(844), step_time 1492.5, mean_step_time 1434.57, it/s 0.7
  4225. [0130 17:50:19 @multigpu.py:323] [p0576] step: count(854), step_time 1431.19, mean_step_time 1432.3, it/s 0.7
  4226. [0130 17:50:19 @multigpu.py:323] [p0115] step: count(830), step_time 1462.53, mean_step_time 1436.9, it/s 0.7
  4227. [0130 17:50:19 @multigpu.py:323] [p0574] step: count(845), step_time 1454.74, mean_step_time 1436.71, it/s 0.7
  4228. [0130 17:50:20 @multigpu.py:323] [p0576] step: count(855), step_time 1445.91, mean_step_time 1434.55, it/s 0.7
  4229. [0130 17:50:20 @multigpu.py:323] [p0115] step: count(831), step_time 1409.05, mean_step_time 1436.72, it/s 0.7
  4230. [0130 17:50:20 @multigpu.py:323] [p0574] step: count(846), step_time 1417.08, mean_step_time 1438.43, it/s 0.7
  4231. [0130 17:50:22 @multigpu.py:323] [p0576] step: count(856), step_time 1438.58, mean_step_time 1431.02, it/s 0.7
  4232. [0130 17:50:22 @multigpu.py:323] [p0115] step: count(832), step_time 1501.15, mean_step_time 1440.66, it/s 0.69
  4233. [0130 17:50:22 @multigpu.py:323] [p0574] step: count(847), step_time 1478.96, mean_step_time 1440.66, it/s 0.69
  4234. [0130 17:50:23 @multigpu.py:323] [p0576] step: count(857), step_time 1419.0, mean_step_time 1431.07, it/s 0.7
  4235. [0130 17:50:23 @multigpu.py:323] [p0115] step: count(833), step_time 1378.58, mean_step_time 1436.77, it/s 0.7
  4236. [0130 17:50:23 @multigpu.py:323] [p0574] step: count(848), step_time 1425.69, mean_step_time 1440.38, it/s 0.69
  4237. sending to address tcp://p0112:61216
  4238. ##### Sending to neptune: online_score : 0.339696066909 , 1.6 #####
  4239. [u'online', 1.6]
  4240. receiving
  4241. sending to address tcp://p0112:61216
  4242. ##### Sending to neptune: online_score : 0.339726577732 , 2.1 #####
  4243. [u'online', 2.1]
  4244. receiving
  4245. sending to address tcp://p0112:61216
  4246. ##### Sending to neptune: online_score : 0.339784356621 , 1.0 #####
  4247. [u'online', 1.0]
  4248. receiving
  4249. [0130 17:50:24 @multigpu.py:323] [p0576] step: count(858), step_time 1418.17, mean_step_time 1428.0, it/s 0.7
  4250. [0130 17:50:25 @multigpu.py:323] [p0115] step: count(834), step_time 1484.33, mean_step_time 1440.68, it/s 0.69
  4251. [0130 17:50:25 @multigpu.py:323] [p0574] step: count(849), step_time 1422.01, mean_step_time 1440.01, it/s 0.69
  4252. [0130 17:50:26 @multigpu.py:323] [p0576] step: count(859), step_time 1401.0, mean_step_time 1425.31, it/s 0.7
  4253. [0130 17:50:26 @multigpu.py:323] [p0115] step: count(835), step_time 1418.18, mean_step_time 1437.96, it/s 0.7
  4254. [0130 17:50:26 @multigpu.py:323] [p0574] step: count(850), step_time 1462.0, mean_step_time 1440.13, it/s 0.69
  4255. [0130 17:50:27 @multigpu.py:323] [p0576] step: count(860), step_time 1458.91, mean_step_time 1427.38, it/s 0.7
  4256. [0130 17:50:27 @multigpu.py:323] [p0115] step: count(836), step_time 1469.8, mean_step_time 1441.84, it/s 0.69
  4257. [0130 17:50:28 @multigpu.py:323] [p0574] step: count(851), step_time 1412.3, mean_step_time 1442.43, it/s 0.69
  4258. [0130 17:50:29 @multigpu.py:323] [p0576] step: count(861), step_time 1416.72, mean_step_time 1427.64, it/s 0.7
  4259. [0130 17:50:29 @multigpu.py:323] [p0115] step: count(837), step_time 1418.69, mean_step_time 1439.99, it/s 0.69
  4260. [0130 17:50:29 @multigpu.py:323] [p0574] step: count(852), step_time 1398.8, mean_step_time 1442.07, it/s 0.69
  4261. [0130 17:50:30 @multigpu.py:323] [p0576] step: count(862), step_time 1441.13, mean_step_time 1428.79, it/s 0.7
  4262. [0130 17:50:30 @multigpu.py:323] [p0115] step: count(838), step_time 1361.38, mean_step_time 1436.77, it/s 0.7
  4263. [0130 17:50:30 @multigpu.py:323] [p0574] step: count(853), step_time 1438.35, mean_step_time 1441.87, it/s 0.69
  4264. [0130 17:50:32 @multigpu.py:323] [p0576] step: count(863), step_time 1437.12, mean_step_time 1427.04, it/s 0.7
  4265. [0130 17:50:32 @multigpu.py:323] [p0115] step: count(839), step_time 1439.47, mean_step_time 1433.17, it/s 0.7
  4266. [0130 17:50:32 @multigpu.py:323] [p0574] step: count(854), step_time 1416.04, mean_step_time 1441.31, it/s 0.69
  4267. [0130 17:50:33 @multigpu.py:323] [p0576] step: count(864), step_time 1436.49, mean_step_time 1429.41, it/s 0.7
  4268. [0130 17:50:33 @multigpu.py:323] [p0115] step: count(840), step_time 1375.77, mean_step_time 1431.33, it/s 0.7
  4269. [0130 17:50:33 @multigpu.py:323] [p0574] step: count(855), step_time 1452.58, mean_step_time 1440.82, it/s 0.69
  4270. [0130 17:50:35 @multigpu.py:323] [p0115] step: count(841), step_time 1487.83, mean_step_time 1434.12, it/s 0.7
  4271. [0130 17:50:35 @multigpu.py:323] [p0576] step: count(865), step_time 1508.24, mean_step_time 1433.52, it/s 0.7
  4272. [0130 17:50:35 @multigpu.py:323] [p0574] step: count(856), step_time 1409.41, mean_step_time 1438.75, it/s 0.7
  4273. [0130 17:50:36 @multigpu.py:323] [p0576] step: count(866), step_time 1412.2, mean_step_time 1432.07, it/s 0.7
  4274. [0130 17:50:36 @multigpu.py:323] [p0115] step: count(842), step_time 1436.41, mean_step_time 1432.65, it/s 0.7
  4275. [0130 17:50:36 @multigpu.py:323] [p0574] step: count(857), step_time 1446.84, mean_step_time 1436.9, it/s 0.7
  4276. [0130 17:50:37 @multigpu.py:323] [p0576] step: count(867), step_time 1435.4, mean_step_time 1430.25, it/s 0.7
  4277. [0130 17:50:37 @multigpu.py:323] [p0574] step: count(858), step_time 1403.72, mean_step_time 1435.72, it/s 0.7
  4278. [0130 17:50:38 @multigpu.py:323] [p0115] step: count(843), step_time 1490.01, mean_step_time 1434.93, it/s 0.7
  4279. [0130 17:50:39 @multigpu.py:323] [p0115] step: count(844), step_time 1376.99, mean_step_time 1432.43, it/s 0.7
  4280. [0130 17:50:39 @multigpu.py:323] [p0576] step: count(868), step_time 1444.38, mean_step_time 1432.57, it/s 0.7
  4281. [0130 17:50:39 @multigpu.py:323] [p0574] step: count(859), step_time 1435.49, mean_step_time 1436.91, it/s 0.7
  4282. [0130 17:50:40 @multigpu.py:323] [p0115] step: count(845), step_time 1388.07, mean_step_time 1429.96, it/s 0.7
  4283. [0130 17:50:40 @multigpu.py:323] [p0576] step: count(869), step_time 1400.91, mean_step_time 1430.93, it/s 0.7
  4284. [0130 17:50:40 @multigpu.py:323] [p0574] step: count(860), step_time 1422.81, mean_step_time 1436.96, it/s 0.7
  4285. [0130 17:50:42 @multigpu.py:323] [p0115] step: count(846), step_time 1405.59, mean_step_time 1430.21, it/s 0.7
  4286. [0130 17:50:42 @multigpu.py:323] [p0576] step: count(870), step_time 1414.07, mean_step_time 1430.37, it/s 0.7
  4287. [0130 17:50:42 @multigpu.py:323] [p0574] step: count(861), step_time 1389.45, mean_step_time 1433.45, it/s 0.7
  4288. [0130 17:50:43 @multigpu.py:323] [p0576] step: count(871), step_time 1439.59, mean_step_time 1430.02, it/s 0.7
  4289. [0130 17:50:43 @multigpu.py:323] [p0115] step: count(847), step_time 1472.43, mean_step_time 1435.08, it/s 0.7
  4290. [0130 17:50:43 @multigpu.py:323] [p0574] step: count(862), step_time 1486.02, mean_step_time 1438.31, it/s 0.7
  4291. [0130 17:50:45 @multigpu.py:323] [p0574] step: count(863), step_time 1380.9, mean_step_time 1432.28, it/s 0.7
  4292. [0130 17:50:45 @multigpu.py:323] [p0115] step: count(848), step_time 1453.38, mean_step_time 1432.34, it/s 0.7
  4293. [0130 17:50:45 @multigpu.py:323] [p0576] step: count(872), step_time 1464.71, mean_step_time 1432.33, it/s 0.7
  4294. [0130 17:50:46 @multigpu.py:323] [p0115] step: count(849), step_time 1356.6, mean_step_time 1429.31, it/s 0.7
  4295. [0130 17:50:46 @multigpu.py:323] [p0576] step: count(873), step_time 1428.41, mean_step_time 1434.61, it/s 0.7
  4296. [0130 17:50:46 @multigpu.py:323] [p0574] step: count(864), step_time 1449.39, mean_step_time 1430.13, it/s 0.7
  4297. sending to address tcp://p0112:61216
  4298. ##### Sending to neptune: online_score : 0.346096348829 , 1.0 #####
  4299. [u'online', 1.0]
  4300. receiving
  4301. [0130 17:50:47 @multigpu.py:323] [p0115] step: count(850), step_time 1441.86, mean_step_time 1428.28, it/s 0.7
  4302. [0130 17:50:47 @multigpu.py:323] [p0576] step: count(874), step_time 1386.22, mean_step_time 1432.36, it/s 0.7
  4303. [0130 17:50:48 @multigpu.py:323] [p0574] step: count(865), step_time 1461.56, mean_step_time 1430.47, it/s 0.7
  4304. [0130 17:50:49 @multigpu.py:323] [p0115] step: count(851), step_time 1407.82, mean_step_time 1428.22, it/s 0.7
  4305. [0130 17:50:49 @multigpu.py:323] [p0576] step: count(875), step_time 1401.26, mean_step_time 1430.13, it/s 0.7
  4306. [0130 17:50:49 @multigpu.py:323] [p0574] step: count(866), step_time 1420.17, mean_step_time 1430.62, it/s 0.7
  4307. [0130 17:50:50 @multigpu.py:323] [p0115] step: count(852), step_time 1406.47, mean_step_time 1423.48, it/s 0.7
  4308. [0130 17:50:50 @multigpu.py:323] [p0576] step: count(876), step_time 1468.13, mean_step_time 1431.6, it/s 0.7
  4309. [0130 17:50:50 @multigpu.py:323] [p0574] step: count(867), step_time 1370.78, mean_step_time 1425.22, it/s 0.7
  4310. [0130 17:50:52 @multigpu.py:323] [p0115] step: count(853), step_time 1394.32, mean_step_time 1424.27, it/s 0.7
  4311. sending to address tcp://p0112:61216
  4312. ##### Sending to neptune: online_score : 0.347506924139 , 1.4 #####
  4313. [u'online', 1.4]
  4314. receiving
  4315. [0130 17:50:52 @multigpu.py:323] [p0576] step: count(877), step_time 1458.1, mean_step_time 1433.56, it/s 0.7
  4316. [0130 17:50:52 @multigpu.py:323] [p0574] step: count(868), step_time 1443.72, mean_step_time 1426.12, it/s 0.7
  4317. [0130 17:50:53 @multigpu.py:323] [p0115] step: count(854), step_time 1444.81, mean_step_time 1422.29, it/s 0.7
  4318. [0130 17:50:53 @multigpu.py:323] [p0576] step: count(878), step_time 1429.86, mean_step_time 1434.14, it/s 0.7
  4319. [0130 17:50:53 @multigpu.py:323] [p0574] step: count(869), step_time 1440.8, mean_step_time 1427.06, it/s 0.7
  4320. sending to address tcp://p0112:61216
  4321. ##### Sending to neptune: online_score : 0.348158127997 , 1.8 #####
  4322. [u'online', 1.8]
  4323. ##### Sending to neptune: active_workers : 0.348158168859 , 3 #####
  4324. receiving
  4325. [0130 17:50:55 @multigpu.py:323] [p0115] step: count(855), step_time 1461.99, mean_step_time 1424.48, it/s 0.7
  4326. [0130 17:50:55 @multigpu.py:323] [p0576] step: count(879), step_time 1429.79, mean_step_time 1435.58, it/s 0.7
  4327. [0130 17:50:55 @multigpu.py:323] [p0574] step: count(870), step_time 1415.5, mean_step_time 1424.73, it/s 0.7
  4328. [0130 17:50:56 @multigpu.py:323] [p0115] step: count(856), step_time 1440.17, mean_step_time 1423.0, it/s 0.7
  4329. [0130 17:50:56 @multigpu.py:323] [p0574] step: count(871), step_time 1401.46, mean_step_time 1424.19, it/s 0.7
  4330. [0130 17:50:56 @multigpu.py:323] [p0576] step: count(880), step_time 1490.08, mean_step_time 1437.14, it/s 0.7
  4331. [0130 17:50:57 @multigpu.py:323] [p0115] step: count(857), step_time 1438.35, mean_step_time 1423.98, it/s 0.7
  4332. [0130 17:50:57 @multigpu.py:323] [p0574] step: count(872), step_time 1437.87, mean_step_time 1426.14, it/s 0.7
  4333. [0130 17:50:58 @multigpu.py:323] [p0576] step: count(881), step_time 1417.36, mean_step_time 1437.17, it/s 0.7
  4334. [0130 17:50:59 @multigpu.py:323] [p0115] step: count(858), step_time 1433.35, mean_step_time 1427.58, it/s 0.7
  4335. [0130 17:50:59 @multigpu.py:323] [p0574] step: count(873), step_time 1393.58, mean_step_time 1423.9, it/s 0.7
  4336. [0130 17:50:59 @multigpu.py:323] [p0576] step: count(882), step_time 1393.8, mean_step_time 1434.81, it/s 0.7
  4337. [0130 17:51:00 @multigpu.py:323] [p0115] step: count(859), step_time 1424.23, mean_step_time 1426.82, it/s 0.7
  4338. [0130 17:51:00 @multigpu.py:323] [p0576] step: count(883), step_time 1381.47, mean_step_time 1432.02, it/s 0.7
  4339. [0130 17:51:00 @multigpu.py:323] [p0574] step: count(874), step_time 1502.83, mean_step_time 1428.24, it/s 0.7
  4340. [0130 17:51:02 @multigpu.py:323] [p0115] step: count(860), step_time 1398.0, mean_step_time 1427.93, it/s 0.7
  4341. [0130 17:51:02 @multigpu.py:323] [p0576] step: count(884), step_time 1422.88, mean_step_time 1431.34, it/s 0.7
  4342. [0130 17:51:02 @multigpu.py:323] [p0574] step: count(875), step_time 1428.62, mean_step_time 1427.05, it/s 0.7
  4343. [0130 17:51:03 @multigpu.py:323] [p0115] step: count(861), step_time 1420.77, mean_step_time 1424.58, it/s 0.7
  4344. [0130 17:51:03 @multigpu.py:323] [p0576] step: count(885), step_time 1435.11, mean_step_time 1427.69, it/s 0.7
  4345. [0130 17:51:03 @multigpu.py:323] [p0574] step: count(876), step_time 1461.93, mean_step_time 1429.67, it/s 0.7
  4346. [0130 17:51:05 @multigpu.py:323] [p0115] step: count(862), step_time 1455.64, mean_step_time 1425.54, it/s 0.7
  4347. [0130 17:51:05 @multigpu.py:323] [p0576] step: count(886), step_time 1386.82, mean_step_time 1426.42, it/s 0.7
  4348. [0130 17:51:05 @multigpu.py:323] [p0574] step: count(877), step_time 1405.52, mean_step_time 1427.61, it/s 0.7
  4349. [0130 17:51:06 @multigpu.py:323] [p0115] step: count(863), step_time 1414.31, mean_step_time 1421.76, it/s 0.7
  4350. [0130 17:51:06 @multigpu.py:323] [p0576] step: count(887), step_time 1438.14, mean_step_time 1426.56, it/s 0.7
  4351. [0130 17:51:06 @multigpu.py:323] [p0574] step: count(878), step_time 1390.07, mean_step_time 1426.92, it/s 0.7
  4352. [0130 17:51:07 @multigpu.py:323] [p0115] step: count(864), step_time 1426.81, mean_step_time 1424.25, it/s 0.7
  4353. [0130 17:51:07 @multigpu.py:323] [p0576] step: count(888), step_time 1423.35, mean_step_time 1425.5, it/s 0.7
  4354. [0130 17:51:08 @multigpu.py:323] [p0574] step: count(879), step_time 1486.03, mean_step_time 1429.45, it/s 0.7
  4355. [0130 17:51:09 @multigpu.py:323] [p0115] step: count(865), step_time 1462.49, mean_step_time 1427.97, it/s 0.7
  4356. [0130 17:51:09 @multigpu.py:323] [p0576] step: count(889), step_time 1489.7, mean_step_time 1429.94, it/s 0.7
  4357. [0130 17:51:09 @multigpu.py:323] [p0574] step: count(880), step_time 1528.76, mean_step_time 1434.75, it/s 0.7
  4358. [0130 17:51:10 @multigpu.py:323] [p0115] step: count(866), step_time 1439.06, mean_step_time 1429.64, it/s 0.7
  4359. [0130 17:51:10 @multigpu.py:323] [p0576] step: count(890), step_time 1421.23, mean_step_time 1430.3, it/s 0.7
  4360. [0130 17:51:11 @multigpu.py:323] [p0574] step: count(881), step_time 1589.16, mean_step_time 1444.73, it/s 0.69
  4361. [0130 17:51:12 @multigpu.py:323] [p0115] step: count(867), step_time 1414.91, mean_step_time 1426.77, it/s 0.7
  4362. [0130 17:51:12 @multigpu.py:323] [p0576] step: count(891), step_time 1479.83, mean_step_time 1432.31, it/s 0.7
  4363. [0130 17:51:12 @multigpu.py:323] [p0574] step: count(882), step_time 1398.45, mean_step_time 1440.35, it/s 0.69
  4364. [0130 17:51:14 @multigpu.py:323] [p0574] step: count(883), step_time 1463.84, mean_step_time 1444.5, it/s 0.69
  4365. [0130 17:51:14 @multigpu.py:323] [p0115] step: count(868), step_time 1808.96, mean_step_time 1444.55, it/s 0.69
  4366. [0130 17:51:14 @multigpu.py:323] [p0576] step: count(892), step_time 1699.42, mean_step_time 1444.05, it/s 0.69
  4367. sending to address tcp://p0112:61216
  4368. ##### Sending to neptune: online_score : 0.353776509696 , 1.7 #####
  4369. [u'online', 1.7]
  4370. receiving
  4371. [0130 17:51:15 @multigpu.py:323] [p0115] step: count(869), step_time 1415.04, mean_step_time 1447.47, it/s 0.69
  4372. [0130 17:51:15 @multigpu.py:323] [p0576] step: count(893), step_time 1428.94, mean_step_time 1444.08, it/s 0.69
  4373. [0130 17:51:15 @multigpu.py:323] [p0574] step: count(884), step_time 1437.96, mean_step_time 1443.93, it/s 0.69
  4374. sending to address tcp://p0112:61216
  4375. ##### Sending to neptune: online_score : 0.354189967182 , 1.6 #####
  4376. [u'online', 1.6]
  4377. receiving
  4378. [0130 17:51:16 @multigpu.py:323] [p0576] step: count(894), step_time 1421.53, mean_step_time 1445.84, it/s 0.69
  4379. [0130 17:51:16 @multigpu.py:323] [p0115] step: count(870), step_time 1473.59, mean_step_time 1449.05, it/s 0.69
  4380. [0130 17:51:16 @multigpu.py:323] [p0574] step: count(885), step_time 1468.89, mean_step_time 1444.3, it/s 0.69
  4381. [0130 17:51:18 @multigpu.py:323] [p0576] step: count(895), step_time 1458.08, mean_step_time 1448.68, it/s 0.69
  4382. [0130 17:51:18 @multigpu.py:323] [p0574] step: count(886), step_time 1462.1, mean_step_time 1446.39, it/s 0.69
  4383. [0130 17:51:18 @multigpu.py:323] [p0115] step: count(871), step_time 1488.1, mean_step_time 1453.07, it/s 0.69
  4384. [0130 17:51:19 @multigpu.py:323] [p0576] step: count(896), step_time 1421.15, mean_step_time 1446.33, it/s 0.69
  4385. [0130 17:51:19 @multigpu.py:323] [p0574] step: count(887), step_time 1424.94, mean_step_time 1449.1, it/s 0.69
  4386. [0130 17:51:19 @multigpu.py:323] [p0115] step: count(872), step_time 1470.23, mean_step_time 1456.26, it/s 0.69
  4387. sending to address tcp://p0112:61216
  4388. ##### Sending to neptune: online_score : 0.35528920273 , 1.4 #####
  4389. [u'online', 1.4]
  4390. receiving
  4391. [0130 17:51:21 @multigpu.py:323] [p0576] step: count(897), step_time 1400.25, mean_step_time 1443.44, it/s 0.69
  4392. [0130 17:51:21 @multigpu.py:323] [p0574] step: count(888), step_time 1429.18, mean_step_time 1448.37, it/s 0.69
  4393. [0130 17:51:21 @multigpu.py:323] [p0115] step: count(873), step_time 1425.93, mean_step_time 1457.84, it/s 0.69
  4394. [0130 17:51:22 @multigpu.py:323] [p0576] step: count(898), step_time 1511.99, mean_step_time 1447.55, it/s 0.69
  4395. [0130 17:51:22 @multigpu.py:323] [p0574] step: count(889), step_time 1438.55, mean_step_time 1448.26, it/s 0.69
  4396. [0130 17:51:22 @multigpu.py:323] [p0115] step: count(874), step_time 1399.34, mean_step_time 1455.56, it/s 0.69
  4397. [0130 17:51:24 @multigpu.py:323] [p0115] step: count(875), step_time 1390.43, mean_step_time 1451.99, it/s 0.69
  4398. [0130 17:51:24 @multigpu.py:323] [p0574] step: count(890), step_time 1411.99, mean_step_time 1448.09, it/s 0.69
  4399. [0130 17:51:24 @multigpu.py:323] [p0576] step: count(899), step_time 1448.49, mean_step_time 1448.48, it/s 0.69
  4400. [0130 17:51:25 @multigpu.py:323] [p0576] step: count(900), step_time 1443.06, mean_step_time 1446.13, it/s 0.69
  4401. sending debugging info...
  4402. sending to address tcp://p0112:61216
  4403. [0130 17:51:25 @multigpu.py:323] [p0115] step: count(876), step_time 1472.49, mean_step_time 1453.6, it/s 0.69
  4404. ##### Sending to neptune: mean_delay : 0.356788568298 , 0.0 #####
  4405. ##### Sending to neptune: max_delay : 0.356788568298 , -0.0 #####
  4406. sending to address tcp://p0112:61216
  4407. ##### Sending to neptune: min_delay : 0.356788568298 , -0.0 #####
  4408. [u'delays', [0.0, -0.0, -0.0]]
  4409. receiving
  4410. ##### Sending to neptune: cost : 0.35678909911 , -0.0122179845348 #####
  4411. ##### Sending to neptune: policy_loss : 0.35678909911 , -0.376462846994 #####
  4412. sending to address tcp://p0112:61216
  4413. ##### Sending to neptune: xentropy_loss : 0.35678909911 , -2.28860378265 #####
  4414. ##### Sending to neptune: value_loss : 0.35678909911 , 1.10116481781 #####
  4415. ##### Sending to neptune: advantage : 0.35678909911 , 0.00160939083435 #####
  4416. ##### Sending to neptune: pred_reward : 0.35678909911 , 0.382754832506 #####
  4417. ##### Sending to neptune: max_logit : 0.35678909911 , 0.197250828147 #####
  4418. [u'loss', -0.012217984534800053, -0.3764628469944, -2.2886037826538086, 1.1011648178100586, 0.0016093908343464136, 0.3827548325061798, 0.1972508281469345]
  4419. receiving
  4420. ##### Sending to neptune: active_relus : 0.356789570782 , 9117830.05 #####
  4421. ##### Sending to neptune: dp_per_s : 0.356789570782 , 89.1289708126 #####
  4422. [u'other', 9117830.05, 89.128970812551]
  4423. receiving
  4424. [0130 17:51:25 @multigpu.py:323] [p0574] step: count(891), step_time 1477.12, mean_step_time 1451.87, it/s 0.69
  4425. [0130 17:51:26 @multigpu.py:323] [p0576] step: count(901), step_time 1402.28, mean_step_time 1445.38, it/s 0.69
  4426. [0130 17:51:26 @multigpu.py:323] [p0574] step: count(892), step_time 1390.05, mean_step_time 1449.48, it/s 0.69
  4427. [0130 17:51:26 @multigpu.py:323] [p0115] step: count(877), step_time 1425.55, mean_step_time 1452.96, it/s 0.69
  4428. [0130 17:51:28 @multigpu.py:323] [p0576] step: count(902), step_time 1435.78, mean_step_time 1447.48, it/s 0.69
  4429. [0130 17:51:28 @multigpu.py:323] [p0115] step: count(878), step_time 1411.88, mean_step_time 1451.89, it/s 0.69
  4430. [0130 17:51:28 @multigpu.py:323] [p0574] step: count(893), step_time 1516.56, mean_step_time 1455.63, it/s 0.69
  4431. [0130 17:51:29 @multigpu.py:323] [p0576] step: count(903), step_time 1406.34, mean_step_time 1448.72, it/s 0.69
  4432. [0130 17:51:29 @multigpu.py:323] [p0115] step: count(879), step_time 1462.48, mean_step_time 1453.8, it/s 0.69
  4433. [0130 17:51:29 @multigpu.py:323] [p0574] step: count(894), step_time 1401.91, mean_step_time 1450.58, it/s 0.69
  4434. [0130 17:51:31 @multigpu.py:323] [p0576] step: count(904), step_time 1378.58, mean_step_time 1446.5, it/s 0.69
  4435. [0130 17:51:31 @multigpu.py:323] [p0115] step: count(880), step_time 1418.22, mean_step_time 1454.81, it/s 0.69
  4436. [0130 17:51:31 @multigpu.py:323] [p0574] step: count(895), step_time 1464.02, mean_step_time 1452.35, it/s 0.69
  4437. [0130 17:51:32 @multigpu.py:323] [p0576] step: count(905), step_time 1438.57, mean_step_time 1446.68, it/s 0.69
  4438. [0130 17:51:32 @multigpu.py:323] [p0115] step: count(881), step_time 1422.05, mean_step_time 1454.88, it/s 0.69
  4439. [0130 17:51:32 @multigpu.py:323] [p0574] step: count(896), step_time 1423.22, mean_step_time 1450.42, it/s 0.69
  4440. [0130 17:51:34 @multigpu.py:323] [p0576] step: count(906), step_time 1412.81, mean_step_time 1447.98, it/s 0.69
  4441. [0130 17:51:34 @multigpu.py:323] [p0115] step: count(882), step_time 1423.91, mean_step_time 1453.29, it/s 0.69
  4442. [0130 17:51:34 @multigpu.py:323] [p0574] step: count(897), step_time 1433.91, mean_step_time 1451.84, it/s 0.69
  4443. [0130 17:51:35 @multigpu.py:323] [p0576] step: count(907), step_time 1400.85, mean_step_time 1446.11, it/s 0.69
  4444. [0130 17:51:35 @multigpu.py:323] [p0115] step: count(883), step_time 1509.1, mean_step_time 1458.03, it/s 0.69
  4445. [0130 17:51:35 @multigpu.py:323] [p0574] step: count(898), step_time 1437.74, mean_step_time 1454.22, it/s 0.69
  4446. [0130 17:51:36 @multigpu.py:323] [p0576] step: count(908), step_time 1459.62, mean_step_time 1447.92, it/s 0.69
  4447. [0130 17:51:37 @multigpu.py:323] [p0574] step: count(899), step_time 1388.74, mean_step_time 1449.35, it/s 0.69
  4448. [0130 17:51:37 @multigpu.py:323] [p0115] step: count(884), step_time 1415.69, mean_step_time 1457.47, it/s 0.69
  4449. sending to address tcp://p0112:61216
  4450. ##### Sending to neptune: online_score : 0.360088469982 , 0.7 #####
  4451. [u'online', 0.7]
  4452. receiving
  4453. [0130 17:51:38 @multigpu.py:323] [p0576] step: count(909), step_time 1351.63, mean_step_time 1441.02, it/s 0.69
  4454. [0130 17:51:38 @multigpu.py:323] [p0574] step: count(900), step_time 1442.32, mean_step_time 1445.03, it/s 0.69
  4455. sending debugging info...
  4456. sending to address tcp://p0112:61216
  4457. ##### Sending to neptune: mean_delay : 0.360377523568 , 0.0 #####
  4458. sending to address tcp://p0112:61216
  4459. ##### Sending to neptune: max_delay : 0.360377523568 , -0.0 #####
  4460. ##### Sending to neptune: min_delay : 0.360377523568 , -0.0 #####
  4461. [u'delays', [0.0, -0.0, -0.0]]
  4462. receiving
  4463. ##### Sending to neptune: cost : 0.360378028022 , -0.00242157001048 #####
  4464. sending to address tcp://p0112:61216
  4465. ##### Sending to neptune: policy_loss : 0.360378028022 , 0.664090633392 #####
  4466. ##### Sending to neptune: xentropy_loss : 0.360378028022 , -2.28888463974 #####
  4467. ##### Sending to neptune: value_loss : 0.360378028022 , 1.3148329258 #####
  4468. ##### Sending to neptune: advantage : 0.360378028022 , -0.00289527932182 #####
  4469. ##### Sending to neptune: pred_reward : 0.360378028022 , 0.36841109395 #####
  4470. ##### Sending to neptune: max_logit : 0.360378028022 , 0.195855751634 #####
  4471. [u'loss', -0.002421570010483265, 0.664090633392334, -2.2888846397399902, 1.3148329257965088, -0.002895279321819544, 0.3684110939502716, 0.1958557516336441]
  4472. receiving
  4473. ##### Sending to neptune: active_relus : 0.360378457175 , 9146761.02 #####
  4474. ##### Sending to neptune: dp_per_s : 0.360378457175 , 88.9845936388 #####
  4475. [u'other', 9146761.02, 88.98459363878979]
  4476. receiving
  4477. [0130 17:51:38 @multigpu.py:323] [p0115] step: count(885), step_time 1454.51, mean_step_time 1457.07, it/s 0.69
  4478. [0130 17:51:39 @multigpu.py:323] [p0576] step: count(910), step_time 1534.17, mean_step_time 1446.67, it/s 0.69
  4479. [0130 17:51:39 @multigpu.py:323] [p0574] step: count(901), step_time 1410.25, mean_step_time 1436.09, it/s 0.7
  4480. [0130 17:51:39 @multigpu.py:323] [p0115] step: count(886), step_time 1448.48, mean_step_time 1457.54, it/s 0.69
  4481. [0130 17:51:41 @multigpu.py:323] [p0576] step: count(911), step_time 1396.51, mean_step_time 1442.5, it/s 0.69
  4482. [0130 17:51:41 @multigpu.py:323] [p0574] step: count(902), step_time 1444.8, mean_step_time 1438.4, it/s 0.7
  4483. [0130 17:51:41 @multigpu.py:323] [p0115] step: count(887), step_time 1455.82, mean_step_time 1459.59, it/s 0.69
  4484. [0130 17:51:42 @multigpu.py:323] [p0115] step: count(888), step_time 1403.94, mean_step_time 1439.34, it/s 0.69
  4485. [0130 17:51:42 @multigpu.py:323] [p0574] step: count(903), step_time 1482.73, mean_step_time 1439.35, it/s 0.69
  4486. [0130 17:51:42 @multigpu.py:323] [p0576] step: count(912), step_time 1636.16, mean_step_time 1439.34, it/s 0.69
  4487. sending to address tcp://p0112:61216
  4488. ##### Sending to neptune: online_score : 0.361710860266 , 2.0 #####
  4489. [u'online', 2.0]
  4490. receiving
  4491. [0130 17:51:44 @multigpu.py:323] [p0574] step: count(904), step_time 1389.31, mean_step_time 1436.92, it/s 0.7
  4492. [0130 17:51:44 @multigpu.py:323] [p0576] step: count(913), step_time 1450.55, mean_step_time 1440.42, it/s 0.69
  4493. [0130 17:51:44 @multigpu.py:323] [p0115] step: count(889), step_time 1481.5, mean_step_time 1442.66, it/s 0.69
  4494. [0130 17:51:45 @multigpu.py:323] [p0574] step: count(905), step_time 1451.54, mean_step_time 1436.05, it/s 0.7
  4495. [0130 17:51:45 @multigpu.py:323] [p0576] step: count(914), step_time 1420.29, mean_step_time 1440.36, it/s 0.69
  4496. [0130 17:51:45 @multigpu.py:323] [p0115] step: count(890), step_time 1405.77, mean_step_time 1439.27, it/s 0.69
  4497. [0130 17:51:47 @multigpu.py:323] [p0574] step: count(906), step_time 1417.06, mean_step_time 1433.8, it/s 0.7
  4498. [0130 17:51:47 @multigpu.py:323] [p0115] step: count(891), step_time 1381.88, mean_step_time 1433.96, it/s 0.7
  4499. [0130 17:51:47 @multigpu.py:323] [p0576] step: count(915), step_time 1453.4, mean_step_time 1440.12, it/s 0.69
  4500. [0130 17:51:48 @multigpu.py:323] [p0115] step: count(892), step_time 1433.03, mean_step_time 1432.1, it/s 0.7
  4501. [0130 17:51:48 @multigpu.py:323] [p0574] step: count(907), step_time 1453.54, mean_step_time 1435.23, it/s 0.7
  4502. [0130 17:51:48 @multigpu.py:323] [p0576] step: count(916), step_time 1415.91, mean_step_time 1439.86, it/s 0.69
  4503. [0130 17:51:49 @multigpu.py:323] [p0574] step: count(908), step_time 1397.95, mean_step_time 1433.67, it/s 0.7
  4504. [0130 17:51:49 @multigpu.py:323] [p0115] step: count(893), step_time 1417.88, mean_step_time 1431.7, it/s 0.7
  4505. [0130 17:51:49 @multigpu.py:323] [p0576] step: count(917), step_time 1407.75, mean_step_time 1440.24, it/s 0.69
  4506. sending to address tcp://p0112:61216
  4507. ##### Sending to neptune: online_score : 0.363701251083 , 1.3 #####
  4508. [u'online', 1.3]
  4509. receiving
  4510. [0130 17:51:51 @multigpu.py:323] [p0574] step: count(909), step_time 1447.39, mean_step_time 1434.11, it/s 0.7
  4511. [0130 17:51:51 @multigpu.py:323] [p0115] step: count(894), step_time 1468.22, mean_step_time 1435.14, it/s 0.7
  4512. [0130 17:51:51 @multigpu.py:323] [p0576] step: count(918), step_time 1449.44, mean_step_time 1437.11, it/s 0.7
  4513. [0130 17:51:52 @multigpu.py:323] [p0574] step: count(910), step_time 1378.61, mean_step_time 1432.44, it/s 0.7
  4514. [0130 17:51:52 @multigpu.py:323] [p0576] step: count(919), step_time 1437.36, mean_step_time 1436.55, it/s 0.7
  4515. [0130 17:51:52 @multigpu.py:323] [p0115] step: count(895), step_time 1470.5, mean_step_time 1439.14, it/s 0.69
  4516. [0130 17:51:54 @multigpu.py:323] [p0574] step: count(911), step_time 1399.76, mean_step_time 1428.57, it/s 0.7
  4517. [0130 17:51:54 @multigpu.py:323] [p0576] step: count(920), step_time 1376.39, mean_step_time 1433.22, it/s 0.7
  4518. [0130 17:51:54 @multigpu.py:323] [p0115] step: count(896), step_time 1374.8, mean_step_time 1434.26, it/s 0.7
  4519. [0130 17:51:55 @multigpu.py:323] [p0574] step: count(912), step_time 1422.39, mean_step_time 1430.19, it/s 0.7
  4520. [0130 17:51:55 @multigpu.py:323] [p0115] step: count(897), step_time 1424.27, mean_step_time 1434.2, it/s 0.7
  4521. [0130 17:51:55 @multigpu.py:323] [p0576] step: count(921), step_time 1496.51, mean_step_time 1437.93, it/s 0.7
  4522. [0130 17:51:56 @multigpu.py:323] [p0574] step: count(913), step_time 1422.53, mean_step_time 1425.49, it/s 0.7
  4523. [0130 17:51:57 @multigpu.py:323] [p0115] step: count(898), step_time 1408.28, mean_step_time 1434.02, it/s 0.7
  4524. [0130 17:51:57 @multigpu.py:323] [p0576] step: count(922), step_time 1424.22, mean_step_time 1437.35, it/s 0.7
  4525. [0130 17:51:58 @multigpu.py:323] [p0574] step: count(914), step_time 1438.2, mean_step_time 1427.3, it/s 0.7
  4526. [0130 17:51:58 @multigpu.py:323] [p0115] step: count(899), step_time 1420.96, mean_step_time 1431.94, it/s 0.7
  4527. [0130 17:51:58 @multigpu.py:323] [p0576] step: count(923), step_time 1398.27, mean_step_time 1436.95, it/s 0.7
  4528. [0130 17:51:59 @multigpu.py:323] [p0574] step: count(915), step_time 1396.61, mean_step_time 1423.93, it/s 0.7
  4529. [0130 17:51:59 @multigpu.py:323] [p0115] step: count(900), step_time 1392.29, mean_step_time 1430.64, it/s 0.7
  4530. sending debugging info...
  4531. sending to address tcp://p0112:61216
  4532. ##### Sending to neptune: mean_delay : 0.366329143312 , 0.0 #####
  4533. sending to address tcp://p0112:61216
  4534. ##### Sending to neptune: max_delay : 0.366329143312 , -0.0 #####
  4535. ##### Sending to neptune: min_delay : 0.366329143312 , -0.0 #####
  4536. [u'delays', [0.0, -0.0, -0.0]]
  4537. ##### Sending to neptune: active_workers : 0.366329351332 , 3 #####
  4538. receiving
  4539. ##### Sending to neptune: cost : 0.36632966472 , -0.00497338781133 #####
  4540. sending to address tcp://p0112:61216
  4541. ##### Sending to neptune: policy_loss : 0.36632966472 , 0.409150511026 #####
  4542. ##### Sending to neptune: xentropy_loss : 0.36632966472 , -2.28915762901 #####
  4543. ##### Sending to neptune: value_loss : 0.36632966472 , 1.24341356754 #####
  4544. ##### Sending to neptune: advantage : 0.36632966472 , -0.00184266630094 #####
  4545. ##### Sending to neptune: pred_reward : 0.36632966472 , 0.362062007189 #####
  4546. ##### Sending to neptune: max_logit : 0.36632966472 , 0.194479733706 #####
  4547. [u'loss', -0.004973387811332941, 0.40915051102638245, -2.2891576290130615, 1.2434135675430298, -0.001842666300944984, 0.362062007188797, 0.19447973370552063]
  4548. receiving
  4549. ##### Sending to neptune: active_relus : 0.366330153611 , 9098058.43 #####
  4550. ##### Sending to neptune: dp_per_s : 0.366330153611 , 88.9767374844 #####
  4551. [u'other', 9098058.43, 88.97673748436563]
  4552. receiving
  4553. [0130 17:51:59 @multigpu.py:323] [p0576] step: count(924), step_time 1418.48, mean_step_time 1438.94, it/s 0.69
  4554. [0130 17:52:01 @multigpu.py:323] [p0574] step: count(916), step_time 1421.35, mean_step_time 1423.84, it/s 0.7
  4555. [0130 17:52:01 @multigpu.py:323] [p0115] step: count(901), step_time 1459.75, mean_step_time 1432.53, it/s 0.7
  4556. [0130 17:52:01 @multigpu.py:323] [p0576] step: count(925), step_time 1415.02, mean_step_time 1437.77, it/s 0.7
  4557. sending to address tcp://p0112:61216
  4558. ##### Sending to neptune: online_score : 0.366962843868 , 1.0 #####
  4559. [u'online', 1.0]
  4560. receiving
  4561. [0130 17:52:02 @multigpu.py:323] [p0574] step: count(917), step_time 1491.58, mean_step_time 1426.72, it/s 0.7
  4562. [0130 17:52:02 @multigpu.py:323] [p0115] step: count(902), step_time 1418.01, mean_step_time 1432.23, it/s 0.7
  4563. [0130 17:52:02 @multigpu.py:323] [p0576] step: count(926), step_time 1406.49, mean_step_time 1437.45, it/s 0.7
  4564. [0130 17:52:04 @multigpu.py:323] [p0574] step: count(918), step_time 1416.31, mean_step_time 1425.65, it/s 0.7
  4565. [0130 17:52:04 @multigpu.py:323] [p0115] step: count(903), step_time 1426.32, mean_step_time 1428.1, it/s 0.7
  4566. [0130 17:52:04 @multigpu.py:323] [p0576] step: count(927), step_time 1441.29, mean_step_time 1439.47, it/s 0.69
  4567. [0130 17:52:05 @multigpu.py:323] [p0574] step: count(919), step_time 1489.08, mean_step_time 1430.67, it/s 0.7
  4568. [0130 17:52:05 @multigpu.py:323] [p0115] step: count(904), step_time 1470.43, mean_step_time 1430.83, it/s 0.7
  4569. [0130 17:52:05 @multigpu.py:323] [p0576] step: count(928), step_time 1460.7, mean_step_time 1439.53, it/s 0.69
  4570. [0130 17:52:07 @multigpu.py:323] [p0574] step: count(920), step_time 1434.11, mean_step_time 1430.25, it/s 0.7
  4571. [0130 17:52:07 @multigpu.py:323] [p0115] step: count(905), step_time 1428.51, mean_step_time 1429.53, it/s 0.7
  4572. [0130 17:52:07 @multigpu.py:323] [p0576] step: count(929), step_time 1417.13, mean_step_time 1442.8, it/s 0.69
  4573. [0130 17:52:08 @multigpu.py:323] [p0115] step: count(906), step_time 1419.18, mean_step_time 1428.07, it/s 0.7
  4574. [0130 17:52:08 @multigpu.py:323] [p0574] step: count(921), step_time 1438.38, mean_step_time 1431.66, it/s 0.7
  4575. [0130 17:52:08 @multigpu.py:323] [p0576] step: count(930), step_time 1424.73, mean_step_time 1437.33, it/s 0.7
  4576. [0130 17:52:09 @multigpu.py:323] [p0115] step: count(907), step_time 1414.07, mean_step_time 1425.98, it/s 0.7
  4577. [0130 17:52:09 @multigpu.py:323] [p0574] step: count(922), step_time 1411.44, mean_step_time 1429.99, it/s 0.7
  4578. [0130 17:52:09 @multigpu.py:323] [p0576] step: count(931), step_time 1422.66, mean_step_time 1438.64, it/s 0.7
  4579. sending to address tcp://p0112:61216
  4580. ##### Sending to neptune: online_score : 0.369363233844 , 1.4 #####
  4581. [u'online', 1.4]
  4582. receiving
  4583. [0130 17:52:11 @multigpu.py:323] [p0115] step: count(908), step_time 1445.59, mean_step_time 1428.06, it/s 0.7
  4584. [0130 17:52:11 @multigpu.py:323] [p0574] step: count(923), step_time 1445.31, mean_step_time 1428.12, it/s 0.7
  4585. [0130 17:52:11 @multigpu.py:323] [p0576] step: count(932), step_time 1426.28, mean_step_time 1428.14, it/s 0.7
  4586. [0130 17:52:12 @multigpu.py:323] [p0115] step: count(909), step_time 1423.35, mean_step_time 1425.16, it/s 0.7
  4587. [0130 17:52:12 @multigpu.py:323] [p0576] step: count(933), step_time 1424.03, mean_step_time 1426.82, it/s 0.7
  4588. [0130 17:52:12 @multigpu.py:323] [p0574] step: count(924), step_time 1471.94, mean_step_time 1432.25, it/s 0.7
  4589. [0130 17:52:14 @multigpu.py:323] [p0115] step: count(910), step_time 1438.34, mean_step_time 1426.78, it/s 0.7
  4590. [0130 17:52:14 @multigpu.py:323] [p0576] step: count(934), step_time 1455.56, mean_step_time 1428.58, it/s 0.7
  4591. [0130 17:52:14 @multigpu.py:323] [p0574] step: count(925), step_time 1450.05, mean_step_time 1432.18, it/s 0.7
  4592. [0130 17:52:15 @multigpu.py:323] [p0115] step: count(911), step_time 1494.43, mean_step_time 1432.41, it/s 0.7
  4593. [0130 17:52:15 @multigpu.py:323] [p0574] step: count(926), step_time 1443.2, mean_step_time 1433.49, it/s 0.7
  4594. [0130 17:52:15 @multigpu.py:323] [p0576] step: count(935), step_time 1490.3, mean_step_time 1430.43, it/s 0.7
  4595. [0130 17:52:17 @multigpu.py:323] [p0576] step: count(936), step_time 1407.64, mean_step_time 1430.01, it/s 0.7
  4596. [0130 17:52:17 @multigpu.py:323] [p0115] step: count(912), step_time 1449.7, mean_step_time 1433.24, it/s 0.7
  4597. [0130 17:52:17 @multigpu.py:323] [p0574] step: count(927), step_time 1445.49, mean_step_time 1433.08, it/s 0.7
  4598. sending to address tcp://p0112:61216
  4599. ##### Sending to neptune: online_score : 0.371138255795 , 1.8 #####
  4600. [u'online', 1.8]
  4601. receiving
  4602. [0130 17:52:18 @multigpu.py:323] [p0115] step: count(913), step_time 1401.75, mean_step_time 1432.44, it/s 0.7
  4603. [0130 17:52:18 @multigpu.py:323] [p0576] step: count(937), step_time 1431.2, mean_step_time 1431.18, it/s 0.7
  4604. [0130 17:52:18 @multigpu.py:323] [p0574] step: count(928), step_time 1416.73, mean_step_time 1434.02, it/s 0.7
  4605. [0130 17:52:19 @multigpu.py:323] [p0576] step: count(938), step_time 1408.21, mean_step_time 1429.12, it/s 0.7
  4606. [0130 17:52:20 @multigpu.py:323] [p0574] step: count(929), step_time 1421.65, mean_step_time 1432.73, it/s 0.7
  4607. [0130 17:52:20 @multigpu.py:323] [p0115] step: count(914), step_time 1503.67, mean_step_time 1434.21, it/s 0.7
  4608. [0130 17:52:21 @multigpu.py:323] [p0576] step: count(939), step_time 1432.68, mean_step_time 1428.89, it/s 0.7
  4609. [0130 17:52:21 @multigpu.py:323] [p0115] step: count(915), step_time 1405.72, mean_step_time 1430.97, it/s 0.7
  4610. [0130 17:52:21 @multigpu.py:323] [p0574] step: count(930), step_time 1506.38, mean_step_time 1439.12, it/s 0.69
  4611. [0130 17:52:22 @multigpu.py:323] [p0576] step: count(940), step_time 1390.55, mean_step_time 1429.6, it/s 0.7
  4612. [0130 17:52:22 @multigpu.py:323] [p0574] step: count(931), step_time 1398.75, mean_step_time 1439.07, it/s 0.69
  4613. [0130 17:52:22 @multigpu.py:323] [p0115] step: count(916), step_time 1443.18, mean_step_time 1434.39, it/s 0.7
  4614. [0130 17:52:24 @multigpu.py:323] [p0576] step: count(941), step_time 1413.95, mean_step_time 1425.47, it/s 0.7
  4615. [0130 17:52:24 @multigpu.py:323] [p0115] step: count(917), step_time 1429.44, mean_step_time 1434.65, it/s 0.7
  4616. [0130 17:52:24 @multigpu.py:323] [p0574] step: count(932), step_time 1435.86, mean_step_time 1439.75, it/s 0.69
  4617. [0130 17:52:25 @multigpu.py:323] [p0576] step: count(942), step_time 1419.92, mean_step_time 1425.25, it/s 0.7
  4618. [0130 17:52:25 @multigpu.py:323] [p0574] step: count(933), step_time 1378.39, mean_step_time 1437.54, it/s 0.7
  4619. [0130 17:52:25 @multigpu.py:323] [p0115] step: count(918), step_time 1477.73, mean_step_time 1438.12, it/s 0.7
  4620. [0130 17:52:27 @multigpu.py:323] [p0576] step: count(943), step_time 1432.5, mean_step_time 1426.97, it/s 0.7
  4621. [0130 17:52:27 @multigpu.py:323] [p0574] step: count(934), step_time 1421.4, mean_step_time 1436.7, it/s 0.7
  4622. [0130 17:52:27 @multigpu.py:323] [p0115] step: count(919), step_time 1469.72, mean_step_time 1440.56, it/s 0.69
  4623. [0130 17:52:28 @multigpu.py:323] [p0576] step: count(944), step_time 1463.06, mean_step_time 1429.19, it/s 0.7
  4624. [0130 17:52:28 @multigpu.py:323] [p0574] step: count(935), step_time 1433.35, mean_step_time 1438.54, it/s 0.7
  4625. [0130 17:52:28 @multigpu.py:323] [p0115] step: count(920), step_time 1392.17, mean_step_time 1440.55, it/s 0.69
  4626. [0130 17:52:30 @multigpu.py:323] [p0576] step: count(945), step_time 1464.76, mean_step_time 1431.68, it/s 0.7
  4627. [0130 17:52:30 @multigpu.py:323] [p0574] step: count(936), step_time 1420.2, mean_step_time 1438.48, it/s 0.7
  4628. [0130 17:52:30 @multigpu.py:323] [p0115] step: count(921), step_time 1468.92, mean_step_time 1441.01, it/s 0.69
  4629. [0130 17:52:31 @multigpu.py:323] [p0576] step: count(946), step_time 1380.21, mean_step_time 1430.37, it/s 0.7
  4630. [0130 17:52:31 @multigpu.py:323] [p0574] step: count(937), step_time 1443.1, mean_step_time 1436.05, it/s 0.7
  4631. [0130 17:52:31 @multigpu.py:323] [p0115] step: count(922), step_time 1389.22, mean_step_time 1439.57, it/s 0.69
  4632. [0130 17:52:32 @multigpu.py:323] [p0576] step: count(947), step_time 1413.24, mean_step_time 1428.97, it/s 0.7
  4633. [0130 17:52:32 @multigpu.py:323] [p0574] step: count(938), step_time 1386.52, mean_step_time 1434.56, it/s 0.7
  4634. [0130 17:52:33 @multigpu.py:323] [p0115] step: count(923), step_time 1465.42, mean_step_time 1441.53, it/s 0.69
  4635. sending to address tcp://p0112:61216
  4636. ##### Sending to neptune: online_score : 0.375655125512 , 1.1 #####
  4637. [u'online', 1.1]
  4638. receiving
  4639. [0130 17:52:34 @multigpu.py:323] [p0574] step: count(939), step_time 1403.78, mean_step_time 1430.3, it/s 0.7
  4640. [0130 17:52:34 @multigpu.py:323] [p0576] step: count(948), step_time 1467.53, mean_step_time 1429.31, it/s 0.7
  4641. [0130 17:52:34 @multigpu.py:323] [p0115] step: count(924), step_time 1459.0, mean_step_time 1440.96, it/s 0.69
  4642. [0130 17:52:35 @multigpu.py:323] [p0576] step: count(949), step_time 1397.18, mean_step_time 1428.31, it/s 0.7
  4643. [0130 17:52:35 @multigpu.py:323] [p0574] step: count(940), step_time 1420.18, mean_step_time 1429.6, it/s 0.7
  4644. [0130 17:52:35 @multigpu.py:323] [p0115] step: count(925), step_time 1417.88, mean_step_time 1440.42, it/s 0.69
  4645. [0130 17:52:37 @multigpu.py:323] [p0574] step: count(941), step_time 1432.8, mean_step_time 1429.32, it/s 0.7
  4646. [0130 17:52:37 @multigpu.py:323] [p0576] step: count(950), step_time 1466.85, mean_step_time 1430.42, it/s 0.7
  4647. [0130 17:52:37 @multigpu.py:323] [p0115] step: count(926), step_time 1421.81, mean_step_time 1440.56, it/s 0.69
  4648. [0130 17:52:38 @multigpu.py:323] [p0574] step: count(942), step_time 1401.24, mean_step_time 1428.81, it/s 0.7
  4649. [0130 17:52:38 @multigpu.py:323] [p0576] step: count(951), step_time 1443.41, mean_step_time 1431.45, it/s 0.7
  4650. [0130 17:52:38 @multigpu.py:323] [p0115] step: count(927), step_time 1461.71, mean_step_time 1442.94, it/s 0.69
  4651. [0130 17:52:40 @multigpu.py:323] [p0115] step: count(928), step_time 1489.4, mean_step_time 1445.13, it/s 0.69
  4652. [0130 17:52:40 @multigpu.py:323] [p0574] step: count(943), step_time 1767.06, mean_step_time 1444.9, it/s 0.69
  4653. [0130 17:52:40 @multigpu.py:323] [p0576] step: count(952), step_time 1698.21, mean_step_time 1445.05, it/s 0.69
  4654. sending to address tcp://p0112:61216
  4655. ##### Sending to neptune: online_score : 0.377704124981 , 1.0 #####
  4656. [u'online', 1.0]
  4657. receiving
  4658. [0130 17:52:41 @multigpu.py:323] [p0115] step: count(929), step_time 1411.02, mean_step_time 1444.51, it/s 0.69
  4659. [0130 17:52:41 @multigpu.py:323] [p0576] step: count(953), step_time 1473.86, mean_step_time 1447.54, it/s 0.69
  4660. [0130 17:52:41 @multigpu.py:323] [p0574] step: count(944), step_time 1499.27, mean_step_time 1446.27, it/s 0.69
  4661. [0130 17:52:43 @multigpu.py:323] [p0115] step: count(930), step_time 1465.97, mean_step_time 1445.89, it/s 0.69
  4662. [0130 17:52:43 @multigpu.py:323] [p0576] step: count(954), step_time 1407.2, mean_step_time 1445.12, it/s 0.69
  4663. [0130 17:52:43 @multigpu.py:323] [p0574] step: count(945), step_time 1412.17, mean_step_time 1444.37, it/s 0.69
  4664. [0130 17:52:44 @multigpu.py:323] [p0576] step: count(955), step_time 1382.05, mean_step_time 1439.71, it/s 0.69
  4665. [0130 17:52:44 @multigpu.py:323] [p0115] step: count(931), step_time 1397.76, mean_step_time 1441.06, it/s 0.69
  4666. [0130 17:52:44 @multigpu.py:323] [p0574] step: count(946), step_time 1457.16, mean_step_time 1445.07, it/s 0.69
  4667. [0130 17:52:46 @multigpu.py:323] [p0115] step: count(932), step_time 1439.31, mean_step_time 1440.54, it/s 0.69
  4668. [0130 17:52:46 @multigpu.py:323] [p0576] step: count(956), step_time 1478.99, mean_step_time 1443.28, it/s 0.69
  4669. [0130 17:52:46 @multigpu.py:323] [p0574] step: count(947), step_time 1416.09, mean_step_time 1443.6, it/s 0.69
  4670. [0130 17:52:47 @multigpu.py:323] [p0115] step: count(933), step_time 1444.71, mean_step_time 1442.69, it/s 0.69
  4671. [0130 17:52:47 @multigpu.py:323] [p0576] step: count(957), step_time 1423.76, mean_step_time 1442.91, it/s 0.69
  4672. [0130 17:52:47 @multigpu.py:323] [p0574] step: count(948), step_time 1437.24, mean_step_time 1444.63, it/s 0.69
  4673. sending to address tcp://p0112:61216
  4674. ##### Sending to neptune: online_score : 0.379729216364 , 2.1 #####
  4675. [u'online', 2.1]
  4676. receiving
  4677. [0130 17:52:48 @multigpu.py:323] [p0576] step: count(958), step_time 1396.68, mean_step_time 1442.33, it/s 0.69
  4678. [0130 17:52:48 @multigpu.py:323] [p0115] step: count(934), step_time 1444.85, mean_step_time 1439.75, it/s 0.69
  4679. [0130 17:52:48 @multigpu.py:323] [p0574] step: count(949), step_time 1407.62, mean_step_time 1443.93, it/s 0.69
  4680. [0130 17:52:50 @multigpu.py:323] [p0576] step: count(959), step_time 1417.47, mean_step_time 1441.57, it/s 0.69
  4681. [0130 17:52:50 @multigpu.py:323] [p0574] step: count(950), step_time 1404.91, mean_step_time 1438.85, it/s 0.69
  4682. [0130 17:52:50 @multigpu.py:323] [p0115] step: count(935), step_time 1462.43, mean_step_time 1442.58, it/s 0.69
  4683. [0130 17:52:51 @multigpu.py:323] [p0576] step: count(960), step_time 1401.59, mean_step_time 1442.12, it/s 0.69
  4684. [0130 17:52:51 @multigpu.py:323] [p0574] step: count(951), step_time 1445.19, mean_step_time 1441.18, it/s 0.69
  4685. [0130 17:52:51 @multigpu.py:323] [p0115] step: count(936), step_time 1465.44, mean_step_time 1443.7, it/s 0.69
  4686. [0130 17:52:53 @multigpu.py:323] [p0576] step: count(961), step_time 1413.59, mean_step_time 1442.1, it/s 0.69
  4687. [0130 17:52:53 @multigpu.py:323] [p0574] step: count(952), step_time 1432.08, mean_step_time 1440.99, it/s 0.69
  4688. [0130 17:52:53 @multigpu.py:323] [p0115] step: count(937), step_time 1403.75, mean_step_time 1442.41, it/s 0.69
  4689. [0130 17:52:54 @multigpu.py:323] [p0576] step: count(962), step_time 1436.9, mean_step_time 1442.95, it/s 0.69
  4690. [0130 17:52:54 @multigpu.py:323] [p0574] step: count(953), step_time 1420.77, mean_step_time 1443.11, it/s 0.69
  4691. [0130 17:52:54 @multigpu.py:323] [p0115] step: count(938), step_time 1450.15, mean_step_time 1441.03, it/s 0.69
  4692. [0130 17:52:55 @multigpu.py:323] [p0576] step: count(963), step_time 1391.48, mean_step_time 1440.9, it/s 0.69
  4693. [0130 17:52:56 @multigpu.py:323] [p0115] step: count(939), step_time 1399.14, mean_step_time 1437.5, it/s 0.7
  4694. [0130 17:52:56 @multigpu.py:323] [p0574] step: count(954), step_time 1484.84, mean_step_time 1446.28, it/s 0.69
  4695. [0130 17:52:57 @multigpu.py:323] [p0576] step: count(964), step_time 1456.94, mean_step_time 1440.59, it/s 0.69
  4696. [0130 17:52:57 @multigpu.py:323] [p0115] step: count(940), step_time 1430.27, mean_step_time 1439.41, it/s 0.69
  4697. [0130 17:52:57 @multigpu.py:323] [p0574] step: count(955), step_time 1496.64, mean_step_time 1449.44, it/s 0.69
  4698. [0130 17:52:58 @multigpu.py:323] [p0576] step: count(965), step_time 1422.67, mean_step_time 1438.49, it/s 0.7
  4699. [0130 17:52:58 @multigpu.py:323] [p0115] step: count(941), step_time 1436.0, mean_step_time 1437.76, it/s 0.7
  4700. [0130 17:52:59 @multigpu.py:323] [p0574] step: count(956), step_time 1402.66, mean_step_time 1448.57, it/s 0.69
  4701. [0130 17:53:00 @multigpu.py:323] [p0576] step: count(966), step_time 1394.0, mean_step_time 1439.18, it/s 0.69
  4702. [0130 17:53:00 @multigpu.py:323] [p0115] step: count(942), step_time 1460.6, mean_step_time 1441.33, it/s 0.69
  4703. [0130 17:53:00 @multigpu.py:323] [p0574] step: count(957), step_time 1448.67, mean_step_time 1448.84, it/s 0.69
  4704. [0130 17:53:01 @multigpu.py:323] [p0576] step: count(967), step_time 1405.76, mean_step_time 1438.81, it/s 0.7
  4705. [0130 17:53:01 @multigpu.py:323] [p0115] step: count(943), step_time 1406.58, mean_step_time 1438.39, it/s 0.7
  4706. [0130 17:53:01 @multigpu.py:323] [p0574] step: count(958), step_time 1448.4, mean_step_time 1451.94, it/s 0.69
  4707. sending to address tcp://p0112:61216
  4708. ##### Sending to neptune: online_score : 0.383631636103 , 2.6 #####
  4709. [u'online', 2.6]
  4710. ##### Sending to neptune: active_workers : 0.383631717165 , 3 #####
  4711. receiving
  4712. [0130 17:53:02 @multigpu.py:323] [p0576] step: count(968), step_time 1398.77, mean_step_time 1435.37, it/s 0.7
  4713. [0130 17:53:03 @multigpu.py:323] [p0115] step: count(944), step_time 1453.51, mean_step_time 1438.12, it/s 0.7
  4714. [0130 17:53:03 @multigpu.py:323] [p0574] step: count(959), step_time 1431.84, mean_step_time 1453.34, it/s 0.69
  4715. [0130 17:53:04 @multigpu.py:323] [p0576] step: count(969), step_time 1474.77, mean_step_time 1439.25, it/s 0.69
  4716. [0130 17:53:04 @multigpu.py:323] [p0115] step: count(945), step_time 1394.46, mean_step_time 1436.94, it/s 0.7
  4717. [0130 17:53:04 @multigpu.py:323] [p0574] step: count(960), step_time 1483.19, mean_step_time 1456.49, it/s 0.69
  4718. [0130 17:53:05 @multigpu.py:323] [p0576] step: count(970), step_time 1470.58, mean_step_time 1439.43, it/s 0.69
  4719. [0130 17:53:06 @multigpu.py:323] [p0115] step: count(946), step_time 1519.65, mean_step_time 1441.84, it/s 0.69
  4720. [0130 17:53:06 @multigpu.py:323] [p0574] step: count(961), step_time 1434.17, mean_step_time 1456.56, it/s 0.69
  4721. [0130 17:53:07 @multigpu.py:323] [p0576] step: count(971), step_time 1422.48, mean_step_time 1438.39, it/s 0.7
  4722. [0130 17:53:07 @multigpu.py:323] [p0115] step: count(947), step_time 1403.57, mean_step_time 1438.93, it/s 0.69
  4723. [0130 17:53:07 @multigpu.py:323] [p0574] step: count(962), step_time 1431.35, mean_step_time 1458.07, it/s 0.69
  4724. sending to address tcp://p0112:61216
  4725. ##### Sending to neptune: online_score : 0.385261432992 , 2.5 #####
  4726. [u'online', 2.5]
  4727. receiving
  4728. [0130 17:53:09 @multigpu.py:323] [p0574] step: count(963), step_time 1488.1, mean_step_time 1444.12, it/s 0.69
  4729. [0130 17:53:09 @multigpu.py:323] [p0576] step: count(972), step_time 1815.85, mean_step_time 1444.27, it/s 0.69
  4730. [0130 17:53:09 @multigpu.py:323] [p0115] step: count(948), step_time 1599.35, mean_step_time 1444.43, it/s 0.69
  4731. [0130 17:53:10 @multigpu.py:323] [p0115] step: count(949), step_time 1377.1, mean_step_time 1442.73, it/s 0.69
  4732. [0130 17:53:10 @multigpu.py:323] [p0574] step: count(964), step_time 1450.41, mean_step_time 1441.67, it/s 0.69
  4733. [0130 17:53:10 @multigpu.py:323] [p0576] step: count(973), step_time 1457.84, mean_step_time 1443.47, it/s 0.69
  4734. [0130 17:53:11 @multigpu.py:323] [p0115] step: count(950), step_time 1417.53, mean_step_time 1440.31, it/s 0.69
  4735. [0130 17:53:12 @multigpu.py:323] [p0574] step: count(965), step_time 1431.62, mean_step_time 1442.65, it/s 0.69
  4736. [0130 17:53:12 @multigpu.py:323] [p0576] step: count(974), step_time 1460.11, mean_step_time 1446.11, it/s 0.69
  4737. [0130 17:53:13 @multigpu.py:323] [p0115] step: count(951), step_time 1498.04, mean_step_time 1445.32, it/s 0.69
  4738. [0130 17:53:13 @multigpu.py:323] [p0576] step: count(975), step_time 1427.4, mean_step_time 1448.38, it/s 0.69
  4739. [0130 17:53:13 @multigpu.py:323] [p0574] step: count(966), step_time 1468.06, mean_step_time 1443.19, it/s 0.69
  4740. [0130 17:53:14 @multigpu.py:323] [p0115] step: count(952), step_time 1413.3, mean_step_time 1444.02, it/s 0.69
  4741. [0130 17:53:14 @multigpu.py:323] [p0574] step: count(967), step_time 1408.96, mean_step_time 1442.84, it/s 0.69
  4742. [0130 17:53:14 @multigpu.py:323] [p0576] step: count(976), step_time 1427.37, mean_step_time 1445.8, it/s 0.69
  4743. [0130 17:53:16 @multigpu.py:323] [p0115] step: count(953), step_time 1412.05, mean_step_time 1442.39, it/s 0.69
  4744. [0130 17:53:16 @multigpu.py:323] [p0576] step: count(977), step_time 1417.17, mean_step_time 1445.47, it/s 0.69
  4745. [0130 17:53:16 @multigpu.py:323] [p0574] step: count(968), step_time 1450.61, mean_step_time 1443.5, it/s 0.69
  4746. [0130 17:53:17 @multigpu.py:323] [p0574] step: count(969), step_time 1392.42, mean_step_time 1442.74, it/s 0.69
  4747. [0130 17:53:17 @multigpu.py:323] [p0115] step: count(954), step_time 1479.82, mean_step_time 1444.14, it/s 0.69
  4748. [0130 17:53:17 @multigpu.py:323] [p0576] step: count(978), step_time 1437.37, mean_step_time 1447.51, it/s 0.69
  4749. sending to address tcp://p0112:61216
  4750. ##### Sending to neptune: online_score : 0.388035714693 , 1.1 #####
  4751. [u'online', 1.1]
  4752. receiving
  4753. [0130 17:53:19 @multigpu.py:323] [p0115] step: count(955), step_time 1388.68, mean_step_time 1440.45, it/s 0.69
  4754. [0130 17:53:19 @multigpu.py:323] [p0574] step: count(970), step_time 1410.08, mean_step_time 1443.0, it/s 0.69
  4755. [0130 17:53:19 @multigpu.py:323] [p0576] step: count(979), step_time 1452.04, mean_step_time 1449.23, it/s 0.69
  4756. [0130 17:53:20 @multigpu.py:323] [p0115] step: count(956), step_time 1440.68, mean_step_time 1439.21, it/s 0.69
  4757. [0130 17:53:20 @multigpu.py:323] [p0574] step: count(971), step_time 1435.06, mean_step_time 1442.5, it/s 0.69
  4758. [0130 17:53:20 @multigpu.py:323] [p0576] step: count(980), step_time 1443.09, mean_step_time 1451.31, it/s 0.69
  4759. [0130 17:53:22 @multigpu.py:323] [p0574] step: count(972), step_time 1401.32, mean_step_time 1440.96, it/s 0.69
  4760. [0130 17:53:22 @multigpu.py:323] [p0115] step: count(957), step_time 1474.39, mean_step_time 1442.74, it/s 0.69
  4761. [0130 17:53:22 @multigpu.py:323] [p0576] step: count(981), step_time 1491.56, mean_step_time 1455.21, it/s 0.69
  4762. [0130 17:53:23 @multigpu.py:323] [p0574] step: count(973), step_time 1395.79, mean_step_time 1439.71, it/s 0.69
  4763. [0130 17:53:23 @multigpu.py:323] [p0115] step: count(958), step_time 1392.72, mean_step_time 1439.87, it/s 0.69
  4764. [0130 17:53:23 @multigpu.py:323] [p0576] step: count(982), step_time 1429.28, mean_step_time 1454.83, it/s 0.69
  4765. [0130 17:53:24 @multigpu.py:323] [p0574] step: count(974), step_time 1448.35, mean_step_time 1437.89, it/s 0.7
  4766. [0130 17:53:24 @multigpu.py:323] [p0115] step: count(959), step_time 1500.38, mean_step_time 1444.93, it/s 0.69
  4767. [0130 17:53:25 @multigpu.py:323] [p0576] step: count(983), step_time 1436.56, mean_step_time 1457.08, it/s 0.69
  4768. [0130 17:53:26 @multigpu.py:323] [p0574] step: count(975), step_time 1406.09, mean_step_time 1433.36, it/s 0.7
  4769. [0130 17:53:26 @multigpu.py:323] [p0115] step: count(960), step_time 1421.83, mean_step_time 1444.51, it/s 0.69
  4770. [0130 17:53:26 @multigpu.py:323] [p0576] step: count(984), step_time 1435.02, mean_step_time 1455.99, it/s 0.69
  4771. [0130 17:53:27 @multigpu.py:323] [p0574] step: count(976), step_time 1450.68, mean_step_time 1435.76, it/s 0.7
  4772. [0130 17:53:27 @multigpu.py:323] [p0115] step: count(961), step_time 1472.84, mean_step_time 1446.35, it/s 0.69
  4773. [0130 17:53:27 @multigpu.py:323] [p0576] step: count(985), step_time 1397.26, mean_step_time 1454.72, it/s 0.69
  4774. [0130 17:53:29 @multigpu.py:323] [p0574] step: count(977), step_time 1470.44, mean_step_time 1436.85, it/s 0.7
  4775. [0130 17:53:29 @multigpu.py:323] [p0576] step: count(986), step_time 1413.1, mean_step_time 1455.67, it/s 0.69
  4776. [0130 17:53:29 @multigpu.py:323] [p0115] step: count(962), step_time 1433.59, mean_step_time 1445.0, it/s 0.69
  4777. [0130 17:53:30 @multigpu.py:323] [p0574] step: count(978), step_time 1425.64, mean_step_time 1435.71, it/s 0.7
  4778. [0130 17:53:30 @multigpu.py:323] [p0576] step: count(987), step_time 1473.67, mean_step_time 1459.07, it/s 0.69
  4779. [0130 17:53:30 @multigpu.py:323] [p0115] step: count(963), step_time 1473.48, mean_step_time 1448.35, it/s 0.69
  4780. sending to address tcp://p0112:61216
  4781. ##### Sending to neptune: online_score : 0.391809822453 , 2.2 #####
  4782. [u'online', 2.2]
  4783. receiving
  4784. [0130 17:53:32 @multigpu.py:323] [p0574] step: count(979), step_time 1390.61, mean_step_time 1433.65, it/s 0.7
  4785. [0130 17:53:32 @multigpu.py:323] [p0115] step: count(964), step_time 1384.83, mean_step_time 1444.91, it/s 0.69
  4786. [0130 17:53:32 @multigpu.py:323] [p0576] step: count(988), step_time 1442.05, mean_step_time 1461.23, it/s 0.68
  4787. [0130 17:53:33 @multigpu.py:323] [p0574] step: count(980), step_time 1450.56, mean_step_time 1432.02, it/s 0.7
  4788. [0130 17:53:33 @multigpu.py:323] [p0576] step: count(989), step_time 1396.6, mean_step_time 1457.32, it/s 0.69
  4789. [0130 17:53:33 @multigpu.py:323] [p0115] step: count(965), step_time 1480.84, mean_step_time 1449.23, it/s 0.69
  4790. [0130 17:53:34 @multigpu.py:323] [p0574] step: count(981), step_time 1427.71, mean_step_time 1431.69, it/s 0.7
  4791. [0130 17:53:35 @multigpu.py:323] [p0576] step: count(990), step_time 1409.66, mean_step_time 1454.27, it/s 0.69
  4792. [0130 17:53:35 @multigpu.py:323] [p0115] step: count(966), step_time 1397.85, mean_step_time 1443.14, it/s 0.69
  4793. [0130 17:53:36 @multigpu.py:323] [p0574] step: count(982), step_time 1412.82, mean_step_time 1430.77, it/s 0.7
  4794. [0130 17:53:36 @multigpu.py:323] [p0576] step: count(991), step_time 1453.47, mean_step_time 1455.82, it/s 0.69
  4795. [0130 17:53:36 @multigpu.py:323] [p0115] step: count(967), step_time 1452.68, mean_step_time 1445.6, it/s 0.69
  4796. [0130 17:53:37 @multigpu.py:323] [p0115] step: count(968), step_time 1397.95, mean_step_time 1435.53, it/s 0.7
  4797. [0130 17:53:37 @multigpu.py:323] [p0574] step: count(983), step_time 1587.46, mean_step_time 1435.74, it/s 0.7
  4798. [0130 17:53:37 @multigpu.py:323] [p0576] step: count(992), step_time 1414.94, mean_step_time 1435.78, it/s 0.7
  4799. [0130 17:53:39 @multigpu.py:323] [p0115] step: count(969), step_time 1432.59, mean_step_time 1438.3, it/s 0.7
  4800. [0130 17:53:39 @multigpu.py:323] [p0574] step: count(984), step_time 1438.65, mean_step_time 1435.15, it/s 0.7
  4801. [0130 17:53:39 @multigpu.py:323] [p0576] step: count(993), step_time 1525.21, mean_step_time 1439.15, it/s 0.69
  4802. sending to address tcp://p0112:61216
  4803. ##### Sending to neptune: online_score : 0.394126037492 , 1.4 #####
  4804. [u'online', 1.4]
  4805. receiving
  4806. [0130 17:53:40 @multigpu.py:323] [p0115] step: count(970), step_time 1408.3, mean_step_time 1437.84, it/s 0.7
  4807. [0130 17:53:40 @multigpu.py:323] [p0574] step: count(985), step_time 1457.66, mean_step_time 1436.45, it/s 0.7
  4808. [0130 17:53:40 @multigpu.py:323] [p0576] step: count(994), step_time 1441.68, mean_step_time 1438.23, it/s 0.7
  4809. [0130 17:53:42 @multigpu.py:323] [p0115] step: count(971), step_time 1414.53, mean_step_time 1433.67, it/s 0.7
  4810. [0130 17:53:42 @multigpu.py:323] [p0574] step: count(986), step_time 1401.51, mean_step_time 1433.12, it/s 0.7
  4811. [0130 17:53:42 @multigpu.py:323] [p0576] step: count(995), step_time 1420.92, mean_step_time 1437.9, it/s 0.7
  4812. [0130 17:53:43 @multigpu.py:323] [p0115] step: count(972), step_time 1420.67, mean_step_time 1434.03, it/s 0.7
  4813. [0130 17:53:43 @multigpu.py:323] [p0574] step: count(987), step_time 1406.8, mean_step_time 1433.01, it/s 0.7
  4814. [0130 17:53:43 @multigpu.py:323] [p0576] step: count(996), step_time 1412.42, mean_step_time 1437.15, it/s 0.7
  4815. [0130 17:53:45 @multigpu.py:323] [p0115] step: count(973), step_time 1475.81, mean_step_time 1437.22, it/s 0.7
  4816. [0130 17:53:45 @multigpu.py:323] [p0574] step: count(988), step_time 1505.52, mean_step_time 1435.76, it/s 0.7
  4817. [0130 17:53:45 @multigpu.py:323] [p0576] step: count(997), step_time 1430.64, mean_step_time 1437.83, it/s 0.7
  4818. sending to address tcp://p0112:61216
  4819. ##### Sending to neptune: online_score : 0.395659864677 , 1.1 #####
  4820. [u'online', 1.1]
  4821. receiving
  4822. [0130 17:53:46 @multigpu.py:323] [p0115] step: count(974), step_time 1447.97, mean_step_time 1435.63, it/s 0.7
  4823. [0130 17:53:46 @multigpu.py:323] [p0574] step: count(989), step_time 1450.31, mean_step_time 1438.65, it/s 0.7
  4824. [0130 17:53:46 @multigpu.py:323] [p0576] step: count(998), step_time 1459.58, mean_step_time 1438.94, it/s 0.69
  4825. [0130 17:53:47 @multigpu.py:323] [p0115] step: count(975), step_time 1414.3, mean_step_time 1436.91, it/s 0.7
  4826. [0130 17:53:47 @multigpu.py:323] [p0574] step: count(990), step_time 1403.42, mean_step_time 1438.32, it/s 0.7
  4827. [0130 17:53:48 @multigpu.py:323] [p0576] step: count(999), step_time 1423.1, mean_step_time 1437.49, it/s 0.7
  4828. [0130 17:53:49 @multigpu.py:323] [p0115] step: count(976), step_time 1433.69, mean_step_time 1436.56, it/s 0.7
  4829. [0130 17:53:49 @multigpu.py:323] [p0576] step: count(1000), step_time 1402.87, mean_step_time 1435.48, it/s 0.7
  4830. sending debugging info...
  4831. sending to address tcp://p0112:61216
  4832. ##### Sending to neptune: mean_delay : 0.396757584148 , 0.0 #####
  4833. sending to address tcp://p0112:61216
  4834. ##### Sending to neptune: max_delay : 0.396757584148 , -0.0 #####
  4835. ##### Sending to neptune: min_delay : 0.396757584148 , -0.0 #####
  4836. [u'delays', [0.0, -0.0, -0.0]]
  4837. receiving
  4838. ##### Sending to neptune: cost : 0.396758102775 , -0.00669238669798 #####
  4839. sending to address tcp://p0112:61216
  4840. ##### Sending to neptune: policy_loss : 0.396758102775 , 0.0405781567097 #####
  4841. ##### Sending to neptune: xentropy_loss : 0.396758102775 , -2.28896999359 #####
  4842. ##### Sending to neptune: value_loss : 0.396758102775 , 1.39176607132 #####
  4843. ##### Sending to neptune: advantage : 0.396758102775 , -0.000199115689611 #####
  4844. ##### Sending to neptune: pred_reward : 0.396758102775 , 0.394293904305 #####
  4845. ##### Sending to neptune: max_logit : 0.396758102775 , 0.192680105567 #####
  4846. [u'loss', -0.006692386697977781, 0.04057815670967102, -2.2889699935913086, 1.39176607131958, -0.0001991156896110624, 0.3942939043045044, 0.19268010556697845]
  4847. receiving
  4848. ##### Sending to neptune: active_relus : 0.396758571664 , 9202785.36 #####
  4849. ##### Sending to neptune: dp_per_s : 0.396758571664 , 88.8521250407 #####
  4850. [u'other', 9202785.36, 88.8521250406519]
  4851. receiving
  4852. 100%|##########|1000/1000[23:48<00:00, 0.70it/s]
  4853. [0130 17:53:49 @multigpu.py:323] [p0574] step: count(991), step_time 1493.14, mean_step_time 1441.22, it/s 0.69
  4854. [0130 17:53:49 @stat.py:81] async_global_step: 1000
  4855. [0130 17:53:49 @stat.py:81] learning_rate_1: 0.00015
  4856. EPOCH ENDS HERE
  4857. [0130 17:53:49 @timer.py:52] Epoch 1, global_step=1000 finished, time=1428.18sec.
  4858. [0130 17:53:50 @multigpu.py:323] [p0115] step: count(977), step_time 1437.93, mean_step_time 1434.74, it/s 0.7
  4859. [0130 17:53:50 @multigpu.py:323] [p0574] step: count(992), step_time 1422.59, mean_step_time 1442.29, it/s 0.69
  4860. [0130 17:53:50 @multigpu.py:323] [p0576] step: count(1001), step_time 1473.44, mean_step_time 1434.58, it/s 0.7
  4861. [0130 17:53:52 @multigpu.py:323] [p0115] step: count(978), step_time 1459.16, mean_step_time 1438.06, it/s 0.7
  4862. [0130 17:53:52 @multigpu.py:323] [p0576] step: count(1002), step_time 1379.25, mean_step_time 1432.07, it/s 0.7
  4863. [0130 17:53:52 @multigpu.py:323] [p0574] step: count(993), step_time 1417.62, mean_step_time 1443.38, it/s 0.69
  4864. [0130 17:53:53 @multigpu.py:323] [p0115] step: count(979), step_time 1421.27, mean_step_time 1434.11, it/s 0.7
  4865. [0130 17:53:53 @multigpu.py:323] [p0576] step: count(1003), step_time 1409.38, mean_step_time 1430.71, it/s 0.7
  4866. [0130 17:53:53 @multigpu.py:323] [p0574] step: count(994), step_time 1445.48, mean_step_time 1443.24, it/s 0.69
  4867. [0130 17:53:55 @multigpu.py:323] [p0576] step: count(1004), step_time 1420.49, mean_step_time 1429.99, it/s 0.7
  4868. [0130 17:53:55 @multigpu.py:323] [p0115] step: count(980), step_time 1438.41, mean_step_time 1434.93, it/s 0.7
  4869. [0130 17:53:55 @multigpu.py:323] [p0574] step: count(995), step_time 1407.12, mean_step_time 1443.29, it/s 0.69
  4870. [0130 17:53:56 @multigpu.py:323] [p0576] step: count(1005), step_time 1421.42, mean_step_time 1431.2, it/s 0.7
  4871. [0130 17:53:56 @multigpu.py:323] [p0115] step: count(981), step_time 1457.56, mean_step_time 1434.17, it/s 0.7
  4872. [0130 17:53:56 @multigpu.py:323] [p0574] step: count(996), step_time 1433.5, mean_step_time 1442.43, it/s 0.69
  4873. [0130 17:53:58 @multigpu.py:323] [p0576] step: count(1006), step_time 1467.35, mean_step_time 1433.91, it/s 0.7
  4874. [0130 17:53:58 @multigpu.py:323] [p0574] step: count(997), step_time 1414.08, mean_step_time 1439.61, it/s 0.69
  4875. [0130 17:53:58 @multigpu.py:323] [p0115] step: count(982), step_time 1437.83, mean_step_time 1434.38, it/s 0.7
  4876. [0130 17:53:59 @multigpu.py:323] [p0576] step: count(1007), step_time 1384.39, mean_step_time 1429.44, it/s 0.7
  4877. [0130 17:53:59 @multigpu.py:323] [p0115] step: count(983), step_time 1396.47, mean_step_time 1430.53, it/s 0.7
  4878. [0130 17:53:59 @multigpu.py:323] [p0574] step: count(998), step_time 1408.16, mean_step_time 1438.74, it/s 0.7
  4879. [0130 17:54:00 @multigpu.py:323] [p0576] step: count(1008), step_time 1379.76, mean_step_time 1426.33, it/s 0.7
  4880. [0130 17:54:00 @multigpu.py:323] [p0115] step: count(984), step_time 1402.52, mean_step_time 1431.42, it/s 0.7
  4881. [0130 17:54:00 @multigpu.py:323] [p0574] step: count(999), step_time 1406.89, mean_step_time 1439.55, it/s 0.69
  4882. sending to address tcp://p0112:61216
  4883. ##### Sending to neptune: online_score : 0.400034431352 , 0.8 #####
  4884. [u'online', 0.8]
  4885. receiving
  4886. [0130 17:54:02 @multigpu.py:323] [p0576] step: count(1009), step_time 1431.42, mean_step_time 1428.07, it/s 0.7
  4887. [0130 17:54:02 @multigpu.py:323] [p0574] step: count(1000), step_time 1397.69, mean_step_time 1436.91, it/s 0.7
  4888. sending debugging info...
  4889. sending to address tcp://p0112:61216
  4890. ##### Sending to neptune: mean_delay : 0.400312784976 , 0.0 #####
  4891. ##### Sending to neptune: max_delay : 0.400312784976 , -0.0 #####
  4892. sending to address tcp://p0112:61216
  4893. ##### Sending to neptune: min_delay : 0.400312784976 , -0.0 #####
  4894. [u'delays', [0.0, -0.0, -0.0]]
  4895. ##### Sending to neptune: active_workers : 0.400312963592 , 3 #####
  4896. receiving
  4897. ##### Sending to neptune: cost : 0.400313278304 , -0.014850821346 #####
  4898. ##### Sending to neptune: policy_loss : 0.400313278304 , -0.792145967484 #####
  4899. sending to address tcp://p0112:61216
  4900. ##### Sending to neptune: xentropy_loss : 0.400313278304 , -2.28885960579 #####
  4901. ##### Sending to neptune: value_loss : 0.400313278304 , 1.18010020256 #####
  4902. ##### Sending to neptune: advantage : 0.400313278304 , 0.00343013252132 #####
  4903. ##### Sending to neptune: pred_reward : 0.400313278304 , 0.402852445841 #####
  4904. ##### Sending to neptune: max_logit : 0.400313278304 , 0.192205965519 #####
  4905. [u'loss', -0.01485082134604454, -0.7921459674835205, -2.2888596057891846, 1.1801002025604248, 0.003430132521316409, 0.40285244584083557, 0.19220596551895142]
  4906. receiving
  4907. ##### Sending to neptune: active_relus : 0.400313773884 , 9198748.82 #####
  4908. ##### Sending to neptune: dp_per_s : 0.400313773884 , 89.0196620523 #####
  4909. [u'other', 9198748.82, 89.0196620522503]
  4910. receiving
  4911. 100%|##########|1000/1000[24:00<00:00, 0.69it/s]
  4912. [0130 17:54:02 @stat.py:81] async_global_step: 1000
  4913. [0130 17:54:02 @stat.py:81] learning_rate_1: 0.00015
  4914. EPOCH ENDS HERE
  4915. [0130 17:54:02 @timer.py:52] Epoch 1, global_step=1000 finished, time=1440.88sec.
  4916. [0130 17:54:02 @multigpu.py:323] [p0115] step: count(985), step_time 1469.01, mean_step_time 1430.82, it/s 0.7
  4917. [0130 17:54:03 @multigpu.py:323] [p0576] step: count(1010), step_time 1390.46, mean_step_time 1427.11, it/s 0.7
  4918. [0130 17:54:03 @multigpu.py:323] [p0574] step: count(1001), step_time 1500.74, mean_step_time 1440.56, it/s 0.69
  4919. [0130 17:54:03 @multigpu.py:323] [p0115] step: count(986), step_time 1467.34, mean_step_time 1434.3, it/s 0.7
  4920. [0130 17:54:04 @multigpu.py:323] [p0576] step: count(1011), step_time 1380.93, mean_step_time 1423.48, it/s 0.7
  4921. [0130 17:54:05 @multigpu.py:323] [p0574] step: count(1002), step_time 1482.45, mean_step_time 1444.04, it/s 0.69
  4922. [0130 17:54:05 @multigpu.py:323] [p0115] step: count(987), step_time 1484.52, mean_step_time 1435.89, it/s 0.7
  4923. sending to address tcp://p0112:61216
  4924. ##### Sending to neptune: online_score : 0.40136244052 , 1.6 #####
  4925. [u'online', 1.6]
  4926. receiving
  4927. [0130 17:54:06 @multigpu.py:323] [p0115] step: count(988), step_time 1423.16, mean_step_time 1437.15, it/s 0.7
  4928. [0130 17:54:06 @multigpu.py:323] [p0574] step: count(1003), step_time 1449.53, mean_step_time 1437.14, it/s 0.7
  4929. [0130 17:54:06 @multigpu.py:323] [p0576] step: count(1012), step_time 1688.2, mean_step_time 1437.15, it/s 0.7
  4930. sending to address tcp://p0112:61216
  4931. ##### Sending to neptune: online_score : 0.40154918472 , 0.9 #####
  4932. [u'online', 0.9]
  4933. receiving
  4934. [0130 17:54:08 @multigpu.py:323] [p0574] step: count(1004), step_time 1386.11, mean_step_time 1434.52, it/s 0.7
  4935. [0130 17:54:08 @multigpu.py:323] [p0115] step: count(989), step_time 1424.58, mean_step_time 1436.75, it/s 0.7
  4936. [0130 17:54:08 @multigpu.py:323] [p0576] step: count(1013), step_time 1437.21, mean_step_time 1432.75, it/s 0.7
  4937. [0130 17:54:09 @multigpu.py:323] [p0576] step: count(1014), step_time 1386.41, mean_step_time 1429.98, it/s 0.7
  4938. [0130 17:54:09 @multigpu.py:323] [p0574] step: count(1005), step_time 1472.24, mean_step_time 1435.25, it/s 0.7
  4939. [0130 17:54:09 @multigpu.py:323] [p0115] step: count(990), step_time 1449.78, mean_step_time 1438.83, it/s 0.7
  4940. [0130 17:54:10 @multigpu.py:323] [p0574] step: count(1006), step_time 1416.46, mean_step_time 1435.99, it/s 0.7
  4941. [0130 17:54:10 @multigpu.py:323] [p0576] step: count(1015), step_time 1453.32, mean_step_time 1431.6, it/s 0.7
  4942. [0130 17:54:10 @multigpu.py:323] [p0115] step: count(991), step_time 1406.55, mean_step_time 1438.43, it/s 0.7
  4943. [0130 17:54:12 @multigpu.py:323] [p0576] step: count(1016), step_time 1376.92, mean_step_time 1429.83, it/s 0.7
  4944. [0130 17:54:12 @multigpu.py:323] [p0574] step: count(1007), step_time 1404.55, mean_step_time 1435.88, it/s 0.7
  4945. [0130 17:54:12 @multigpu.py:323] [p0115] step: count(992), step_time 1434.62, mean_step_time 1439.12, it/s 0.69
  4946. [0130 17:54:13 @multigpu.py:323] [p0574] step: count(1008), step_time 1392.2, mean_step_time 1430.21, it/s 0.7
  4947. [0130 17:54:13 @multigpu.py:323] [p0115] step: count(993), step_time 1443.97, mean_step_time 1437.53, it/s 0.7
  4948. [0130 17:54:13 @multigpu.py:323] [p0576] step: count(1017), step_time 1511.07, mean_step_time 1433.85, it/s 0.7
  4949. [0130 17:54:15 @multigpu.py:323] [p0574] step: count(1009), step_time 1434.25, mean_step_time 1429.41, it/s 0.7
  4950. [0130 17:54:15 @multigpu.py:323] [p0115] step: count(994), step_time 1373.23, mean_step_time 1433.8, it/s 0.7
  4951. [0130 17:54:15 @multigpu.py:323] [p0576] step: count(1018), step_time 1411.65, mean_step_time 1431.45, it/s 0.7
  4952. [0130 17:54:16 @multigpu.py:323] [p0574] step: count(1010), step_time 1381.34, mean_step_time 1428.31, it/s 0.7
  4953. [0130 17:54:16 @multigpu.py:323] [p0115] step: count(995), step_time 1433.91, mean_step_time 1434.78, it/s 0.7
  4954. [0130 17:54:16 @multigpu.py:323] [p0576] step: count(1019), step_time 1424.85, mean_step_time 1431.54, it/s 0.7
  4955. [0130 17:54:17 @multigpu.py:323] [p0574] step: count(1011), step_time 1408.96, mean_step_time 1424.1, it/s 0.7
  4956. [0130 17:54:18 @multigpu.py:323] [p0115] step: count(996), step_time 1402.86, mean_step_time 1433.23, it/s 0.7
  4957. [0130 17:54:18 @multigpu.py:323] [p0576] step: count(1020), step_time 1480.01, mean_step_time 1435.4, it/s 0.7
  4958. [0130 17:54:19 @multigpu.py:323] [p0574] step: count(1012), step_time 1432.06, mean_step_time 1424.57, it/s 0.7
  4959. [0130 17:54:19 @multigpu.py:323] [p0115] step: count(997), step_time 1412.77, mean_step_time 1431.98, it/s 0.7
  4960. [0130 17:54:19 @multigpu.py:323] [p0576] step: count(1021), step_time 1444.97, mean_step_time 1433.97, it/s 0.7
  4961. [0130 17:54:20 @multigpu.py:323] [p0574] step: count(1013), step_time 1448.9, mean_step_time 1426.13, it/s 0.7
  4962. [0130 17:54:20 @multigpu.py:323] [p0115] step: count(998), step_time 1395.76, mean_step_time 1428.81, it/s 0.7
  4963. [0130 17:54:21 @multigpu.py:323] [p0576] step: count(1022), step_time 1410.01, mean_step_time 1435.51, it/s 0.7
  4964. [0130 17:54:22 @multigpu.py:323] [p0574] step: count(1014), step_time 1413.5, mean_step_time 1424.54, it/s 0.7
  4965. [0130 17:54:22 @multigpu.py:323] [p0115] step: count(999), step_time 1438.66, mean_step_time 1429.68, it/s 0.7
  4966. [0130 17:54:22 @multigpu.py:323] [p0576] step: count(1023), step_time 1382.26, mean_step_time 1434.16, it/s 0.7
  4967. [0130 17:54:23 @multigpu.py:323] [p0574] step: count(1015), step_time 1425.44, mean_step_time 1425.45, it/s 0.7
  4968. [0130 17:54:23 @multigpu.py:323] [p0115] step: count(1000), step_time 1496.72, mean_step_time 1432.59, it/s 0.7
  4969. sending debugging info...
  4970. sending to address tcp://p0112:61216
  4971. ##### Sending to neptune: mean_delay : 0.406299769415 , 0.0 #####
  4972. sending to address tcp://p0112:61216
  4973. ##### Sending to neptune: max_delay : 0.406299769415 , -0.0 #####
  4974. ##### Sending to neptune: min_delay : 0.406299769415 , -0.0 #####
  4975. [u'delays', [0.0, -0.0, -0.0]]
  4976. receiving
  4977. ##### Sending to neptune: cost : 0.406300278306 , -0.00499816797674 #####
  4978. sending to address tcp://p0112:61216
  4979. ##### Sending to neptune: policy_loss : 0.406300278306 , 0.282140910625 #####
  4980. ##### Sending to neptune: xentropy_loss : 0.406300278306 , -2.2886068821 #####
  4981. ##### Sending to neptune: value_loss : 0.406300278306 , 1.36670041084 #####
  4982. ##### Sending to neptune: advantage : 0.406300278306 , -0.00127228489146 #####
  4983. ##### Sending to neptune: pred_reward : 0.406300278306 , 0.407911151648 #####
  4984. ##### Sending to neptune: max_logit : 0.406300278306 , 0.192866250873 #####
  4985. [u'loss', -0.004998167976737022, 0.28214091062545776, -2.288606882095337, 1.3667004108428955, -0.0012722848914563656, 0.40791115164756775, 0.192866250872612]
  4986. 100%|##########|1000/1000[23:53<00:00, 0.70it/s]
  4987. receiving
  4988. ##### Sending to neptune: active_relus : 0.406300744679 , 9248289.22 #####
  4989. ##### Sending to neptune: dp_per_s : 0.406300744679 , 89.0294158633 #####
  4990. [u'other', 9248289.22, 89.02941586332658]
  4991. receiving
  4992. [0130 17:54:23 @stat.py:81] async_global_step: 1000
  4993. [0130 17:54:23 @stat.py:81] learning_rate_1: 0.00015
  4994. EPOCH ENDS HERE
  4995. [0130 17:54:23 @timer.py:52] Epoch 1, global_step=1000 finished, time=1433.06sec.
  4996. [0130 17:54:23 @multigpu.py:323] [p0576] step: count(1024), step_time 1482.16, mean_step_time 1437.24, it/s 0.7
  4997. sending to address tcp://p0112:61216
  4998. ##### Sending to neptune: online_score : 0.406547241608 , 1.9 #####
  4999. [u'online', 1.9]
  5000. receiving
  5001. [0130 17:54:25 @multigpu.py:323] [p0574] step: count(1016), step_time 1396.65, mean_step_time 1423.61, it/s 0.7
  5002. [0130 17:54:25 @multigpu.py:323] [p0115] step: count(1001), step_time 1433.23, mean_step_time 1431.37, it/s 0.7
  5003. [0130 17:54:25 @multigpu.py:323] [p0576] step: count(1025), step_time 1436.77, mean_step_time 1438.01, it/s 0.7
  5004. [0130 17:54:26 @multigpu.py:323] [p0574] step: count(1017), step_time 1386.37, mean_step_time 1422.22, it/s 0.7
  5005. [0130 17:54:26 @multigpu.py:323] [p0115] step: count(1002), step_time 1394.99, mean_step_time 1429.23, it/s 0.7
  5006. [0130 17:54:26 @multigpu.py:323] [p0576] step: count(1026), step_time 1390.23, mean_step_time 1434.15, it/s 0.7
  5007. [0130 17:54:27 @multigpu.py:323] [p0574] step: count(1018), step_time 1411.35, mean_step_time 1422.38, it/s 0.7
  5008. [0130 17:54:28 @multigpu.py:323] [p0115] step: count(1003), step_time 1436.11, mean_step_time 1431.21, it/s 0.7
  5009. [0130 17:54:28 @multigpu.py:323] [p0576] step: count(1027), step_time 1417.47, mean_step_time 1435.8, it/s 0.7
  5010. [0130 17:54:29 @multigpu.py:323] [p0574] step: count(1019), step_time 1413.31, mean_step_time 1422.7, it/s 0.7
  5011. [0130 17:54:29 @multigpu.py:323] [p0115] step: count(1004), step_time 1418.69, mean_step_time 1432.02, it/s 0.7
  5012. [0130 17:54:29 @multigpu.py:323] [p0576] step: count(1028), step_time 1393.45, mean_step_time 1436.49, it/s 0.7
  5013. [0130 17:54:30 @multigpu.py:323] [p0574] step: count(1020), step_time 1430.03, mean_step_time 1424.32, it/s 0.7
  5014. [0130 17:54:30 @multigpu.py:323] [p0115] step: count(1005), step_time 1461.79, mean_step_time 1431.66, it/s 0.7
  5015. [0130 17:54:30 @multigpu.py:323] [p0576] step: count(1029), step_time 1430.71, mean_step_time 1436.45, it/s 0.7
  5016. [0130 17:54:32 @multigpu.py:323] [p0574] step: count(1021), step_time 1470.66, mean_step_time 1422.82, it/s 0.7
  5017. [0130 17:54:32 @multigpu.py:323] [p0115] step: count(1006), step_time 1423.65, mean_step_time 1429.48, it/s 0.7
  5018. [0130 17:54:32 @multigpu.py:323] [p0576] step: count(1030), step_time 1439.9, mean_step_time 1438.93, it/s 0.69
  5019. [0130 17:54:33 @multigpu.py:323] [p0574] step: count(1022), step_time 1408.78, mean_step_time 1419.13, it/s 0.7
  5020. [0130 17:54:33 @multigpu.py:323] [p0115] step: count(1007), step_time 1421.95, mean_step_time 1426.35, it/s 0.7
  5021. [0130 17:54:33 @multigpu.py:323] [p0576] step: count(1031), step_time 1445.41, mean_step_time 1442.15, it/s 0.69
  5022. [0130 17:54:35 @multigpu.py:323] [p0576] step: count(1032), step_time 1395.33, mean_step_time 1427.51, it/s 0.7
  5023. [0130 17:54:35 @multigpu.py:323] [p0574] step: count(1023), step_time 1614.89, mean_step_time 1427.4, it/s 0.7
  5024. [0130 17:54:35 @multigpu.py:323] [p0115] step: count(1008), step_time 1446.89, mean_step_time 1427.54, it/s 0.7
  5025. [0130 17:54:36 @multigpu.py:323] [p0574] step: count(1024), step_time 1390.41, mean_step_time 1427.62, it/s 0.7
  5026. sending to address tcp://p0112:61216
  5027. ##### Sending to neptune: online_score : 0.409867774116 , 2.2 #####
  5028. [u'online', 2.2]
  5029. receiving
  5030. [0130 17:54:36 @multigpu.py:323] [p0576] step: count(1033), step_time 1416.02, mean_step_time 1426.45, it/s 0.7
  5031. [0130 17:54:36 @multigpu.py:323] [p0115] step: count(1009), step_time 1435.11, mean_step_time 1428.06, it/s 0.7
  5032. [0130 17:54:38 @multigpu.py:323] [p0115] step: count(1010), step_time 1396.53, mean_step_time 1425.4, it/s 0.7
  5033. [0130 17:54:38 @multigpu.py:323] [p0574] step: count(1025), step_time 1454.39, mean_step_time 1426.72, it/s 0.7
  5034. [0130 17:54:38 @multigpu.py:323] [p0576] step: count(1034), step_time 1454.06, mean_step_time 1429.83, it/s 0.7
  5035. sending to address tcp://p0112:61216
  5036. ##### Sending to neptune: online_score : 0.410494857497 , 1.8 #####
  5037. [u'online', 1.8]
  5038. receiving
  5039. [0130 17:54:39 @multigpu.py:323] [p0115] step: count(1011), step_time 1457.56, mean_step_time 1427.95, it/s 0.7
  5040. [0130 17:54:39 @multigpu.py:323] [p0574] step: count(1026), step_time 1464.19, mean_step_time 1429.11, it/s 0.7
  5041. [0130 17:54:39 @multigpu.py:323] [p0576] step: count(1035), step_time 1483.78, mean_step_time 1431.35, it/s 0.7
  5042. [0130 17:54:40 @multigpu.py:323] [p0115] step: count(1012), step_time 1456.61, mean_step_time 1429.05, it/s 0.7
  5043. [0130 17:54:40 @multigpu.py:323] [p0574] step: count(1027), step_time 1441.26, mean_step_time 1430.95, it/s 0.7
  5044. [0130 17:54:41 @multigpu.py:323] [p0576] step: count(1036), step_time 1423.45, mean_step_time 1433.68, it/s 0.7
  5045. [0130 17:54:42 @multigpu.py:323] [p0574] step: count(1028), step_time 1432.02, mean_step_time 1432.94, it/s 0.7
  5046. [0130 17:54:42 @multigpu.py:323] [p0115] step: count(1013), step_time 1439.95, mean_step_time 1428.85, it/s 0.7
  5047. [0130 17:54:42 @multigpu.py:323] [p0576] step: count(1037), step_time 1412.17, mean_step_time 1428.73, it/s 0.7
  5048. [0130 17:54:43 @multigpu.py:323] [p0115] step: count(1014), step_time 1422.85, mean_step_time 1431.33, it/s 0.7
  5049. [0130 17:54:43 @multigpu.py:323] [p0576] step: count(1038), step_time 1440.15, mean_step_time 1430.16, it/s 0.7
  5050. [0130 17:54:43 @multigpu.py:323] [p0574] step: count(1029), step_time 1458.63, mean_step_time 1434.16, it/s 0.7
  5051. [0130 17:54:45 @multigpu.py:323] [p0576] step: count(1039), step_time 1383.62, mean_step_time 1428.1, it/s 0.7
  5052. [0130 17:54:45 @multigpu.py:323] [p0115] step: count(1015), step_time 1440.15, mean_step_time 1431.64, it/s 0.7
  5053. [0130 17:54:45 @multigpu.py:323] [p0574] step: count(1030), step_time 1436.54, mean_step_time 1436.92, it/s 0.7
  5054. [0130 17:54:46 @multigpu.py:323] [p0576] step: count(1040), step_time 1419.95, mean_step_time 1425.09, it/s 0.7
  5055. [0130 17:54:46 @multigpu.py:323] [p0574] step: count(1031), step_time 1453.02, mean_step_time 1439.12, it/s 0.69
  5056. [0130 17:54:46 @multigpu.py:323] [p0115] step: count(1016), step_time 1500.55, mean_step_time 1436.53, it/s 0.7
  5057. [0130 17:54:48 @multigpu.py:323] [p0576] step: count(1041), step_time 1415.08, mean_step_time 1423.6, it/s 0.7
  5058. [0130 17:54:48 @multigpu.py:323] [p0574] step: count(1032), step_time 1400.81, mean_step_time 1437.56, it/s 0.7
  5059. [0130 17:54:48 @multigpu.py:323] [p0115] step: count(1017), step_time 1412.98, mean_step_time 1436.54, it/s 0.7
  5060. [0130 17:54:49 @multigpu.py:323] [p0576] step: count(1042), step_time 1366.53, mean_step_time 1421.42, it/s 0.7
  5061. [0130 17:54:49 @multigpu.py:323] [p0574] step: count(1033), step_time 1453.49, mean_step_time 1437.79, it/s 0.7
  5062. [0130 17:54:49 @multigpu.py:323] [p0115] step: count(1018), step_time 1435.46, mean_step_time 1438.52, it/s 0.7
  5063. [0130 17:54:50 @multigpu.py:323] [p0576] step: count(1043), step_time 1474.24, mean_step_time 1426.02, it/s 0.7
  5064. [0130 17:54:51 @multigpu.py:323] [p0574] step: count(1034), step_time 1398.87, mean_step_time 1437.06, it/s 0.7
  5065. [0130 17:54:51 @multigpu.py:323] [p0115] step: count(1019), step_time 1426.7, mean_step_time 1437.92, it/s 0.7
  5066. [0130 17:54:52 @multigpu.py:323] [p0576] step: count(1044), step_time 1480.67, mean_step_time 1425.95, it/s 0.7
  5067. [0130 17:54:52 @multigpu.py:323] [p0574] step: count(1035), step_time 1418.28, mean_step_time 1436.7, it/s 0.7
  5068. [0130 17:54:52 @multigpu.py:323] [p0115] step: count(1020), step_time 1414.59, mean_step_time 1433.82, it/s 0.7
  5069. [0130 17:54:53 @multigpu.py:323] [p0574] step: count(1036), step_time 1373.41, mean_step_time 1435.54, it/s 0.7
  5070. [0130 17:54:53 @multigpu.py:323] [p0576] step: count(1045), step_time 1500.63, mean_step_time 1429.14, it/s 0.7
  5071. [0130 17:54:53 @multigpu.py:323] [p0115] step: count(1021), step_time 1447.12, mean_step_time 1434.51, it/s 0.7
  5072. sending to address tcp://p0112:61216
  5073. ##### Sending to neptune: online_score : 0.414848094715 , 1.4 #####
  5074. [u'online', 1.4]
  5075. receiving
  5076. [0130 17:54:55 @multigpu.py:323] [p0574] step: count(1037), step_time 1446.01, mean_step_time 1438.52, it/s 0.7
  5077. [0130 17:54:55 @multigpu.py:323] [p0576] step: count(1046), step_time 1383.97, mean_step_time 1428.83, it/s 0.7
  5078. [0130 17:54:55 @multigpu.py:323] [p0115] step: count(1022), step_time 1451.57, mean_step_time 1437.34, it/s 0.7
  5079. [0130 17:54:56 @multigpu.py:323] [p0574] step: count(1038), step_time 1386.57, mean_step_time 1437.28, it/s 0.7
  5080. [0130 17:54:56 @multigpu.py:323] [p0576] step: count(1047), step_time 1478.74, mean_step_time 1431.89, it/s 0.7
  5081. [0130 17:54:56 @multigpu.py:323] [p0115] step: count(1023), step_time 1435.33, mean_step_time 1437.3, it/s 0.7
  5082. [0130 17:54:58 @multigpu.py:323] [p0574] step: count(1039), step_time 1427.69, mean_step_time 1438.0, it/s 0.7
  5083. [0130 17:54:58 @multigpu.py:323] [p0576] step: count(1048), step_time 1418.64, mean_step_time 1433.15, it/s 0.7
  5084. [0130 17:54:58 @multigpu.py:323] [p0115] step: count(1024), step_time 1436.89, mean_step_time 1438.21, it/s 0.7
  5085. [0130 17:54:59 @multigpu.py:323] [p0574] step: count(1040), step_time 1395.55, mean_step_time 1436.27, it/s 0.7
  5086. [0130 17:54:59 @multigpu.py:323] [p0115] step: count(1025), step_time 1415.06, mean_step_time 1435.87, it/s 0.7
  5087. [0130 17:54:59 @multigpu.py:323] [p0576] step: count(1049), step_time 1498.11, mean_step_time 1436.52, it/s 0.7
  5088. [0130 17:55:00 @multigpu.py:323] [p0574] step: count(1041), step_time 1429.55, mean_step_time 1434.22, it/s 0.7
  5089. [0130 17:55:01 @multigpu.py:323] [p0576] step: count(1050), step_time 1387.85, mean_step_time 1433.92, it/s 0.7
  5090. [0130 17:55:01 @multigpu.py:323] [p0115] step: count(1026), step_time 1455.92, mean_step_time 1437.49, it/s 0.7
  5091. [0130 17:55:02 @multigpu.py:323] [p0574] step: count(1042), step_time 1404.44, mean_step_time 1434.0, it/s 0.7
  5092. [0130 17:55:02 @multigpu.py:323] [p0576] step: count(1051), step_time 1454.33, mean_step_time 1434.37, it/s 0.7
  5093. [0130 17:55:02 @multigpu.py:323] [p0115] step: count(1027), step_time 1447.54, mean_step_time 1438.77, it/s 0.7
  5094. sending to address tcp://p0112:61216
  5095. ##### Sending to neptune: online_score : 0.417075758311 , 1.6 #####
  5096. [u'online', 1.6]
  5097. ##### Sending to neptune: active_workers : 0.41707579467 , 3 #####
  5098. receiving
  5099. [0130 17:55:04 @multigpu.py:323] [p0115] step: count(1028), step_time 1484.34, mean_step_time 1440.64, it/s 0.69
  5100. [0130 17:55:04 @multigpu.py:323] [p0576] step: count(1052), step_time 1519.52, mean_step_time 1440.58, it/s 0.69
  5101. [0130 17:55:04 @multigpu.py:323] [p0574] step: count(1043), step_time 1747.43, mean_step_time 1440.63, it/s 0.69
  5102. [0130 17:55:05 @multigpu.py:323] [p0576] step: count(1053), step_time 1418.04, mean_step_time 1440.68, it/s 0.69
  5103. [0130 17:55:05 @multigpu.py:323] [p0574] step: count(1044), step_time 1417.8, mean_step_time 1442.0, it/s 0.69
  5104. [0130 17:55:05 @multigpu.py:323] [p0115] step: count(1029), step_time 1472.79, mean_step_time 1442.52, it/s 0.69
  5105. [0130 17:55:06 @multigpu.py:323] [p0574] step: count(1045), step_time 1394.07, mean_step_time 1438.98, it/s 0.69
  5106. [0130 17:55:06 @multigpu.py:323] [p0576] step: count(1054), step_time 1410.82, mean_step_time 1438.52, it/s 0.7
  5107. [0130 17:55:06 @multigpu.py:323] [p0115] step: count(1030), step_time 1420.81, mean_step_time 1443.74, it/s 0.69
  5108. [0130 17:55:08 @multigpu.py:323] [p0574] step: count(1046), step_time 1393.73, mean_step_time 1435.46, it/s 0.7
  5109. [0130 17:55:08 @multigpu.py:323] [p0576] step: count(1055), step_time 1458.42, mean_step_time 1437.25, it/s 0.7
  5110. [0130 17:55:08 @multigpu.py:323] [p0115] step: count(1031), step_time 1485.93, mean_step_time 1445.16, it/s 0.69
  5111. [0130 17:55:09 @multigpu.py:323] [p0574] step: count(1047), step_time 1478.91, mean_step_time 1437.34, it/s 0.7
  5112. [0130 17:55:09 @multigpu.py:323] [p0576] step: count(1056), step_time 1488.08, mean_step_time 1440.48, it/s 0.69
  5113. [0130 17:55:09 @multigpu.py:323] [p0115] step: count(1032), step_time 1414.59, mean_step_time 1443.06, it/s 0.69
  5114. sending to address tcp://p0112:61216
  5115. ##### Sending to neptune: online_score : 0.419183745517 , 1.7 #####
  5116. [u'online', 1.7]
  5117. receiving
  5118. [0130 17:55:11 @multigpu.py:323] [p0574] step: count(1048), step_time 1408.46, mean_step_time 1436.16, it/s 0.7
  5119. [0130 17:55:11 @multigpu.py:323] [p0576] step: count(1057), step_time 1376.61, mean_step_time 1438.7, it/s 0.7
  5120. [0130 17:55:11 @multigpu.py:323] [p0115] step: count(1033), step_time 1422.66, mean_step_time 1442.19, it/s 0.69
  5121. [0130 17:55:12 @multigpu.py:323] [p0574] step: count(1049), step_time 1398.56, mean_step_time 1433.16, it/s 0.7
  5122. [0130 17:55:12 @multigpu.py:323] [p0576] step: count(1058), step_time 1436.46, mean_step_time 1438.52, it/s 0.7
  5123. [0130 17:55:12 @multigpu.py:323] [p0115] step: count(1034), step_time 1444.23, mean_step_time 1443.26, it/s 0.69
  5124. [0130 17:55:13 @multigpu.py:323] [p0574] step: count(1050), step_time 1419.26, mean_step_time 1432.29, it/s 0.7
  5125. [0130 17:55:14 @multigpu.py:323] [p0576] step: count(1059), step_time 1464.07, mean_step_time 1442.54, it/s 0.69
  5126. [0130 17:55:14 @multigpu.py:323] [p0115] step: count(1035), step_time 1432.57, mean_step_time 1442.88, it/s 0.69
  5127. [0130 17:55:15 @multigpu.py:323] [p0574] step: count(1051), step_time 1429.73, mean_step_time 1431.13, it/s 0.7
  5128. [0130 17:55:15 @multigpu.py:323] [p0576] step: count(1060), step_time 1391.39, mean_step_time 1441.11, it/s 0.69
  5129. [0130 17:55:15 @multigpu.py:323] [p0115] step: count(1036), step_time 1421.45, mean_step_time 1438.93, it/s 0.69
  5130. [0130 17:55:16 @multigpu.py:323] [p0574] step: count(1052), step_time 1391.14, mean_step_time 1430.65, it/s 0.7
  5131. [0130 17:55:16 @multigpu.py:323] [p0576] step: count(1061), step_time 1390.34, mean_step_time 1439.87, it/s 0.69
  5132. [0130 17:55:16 @multigpu.py:323] [p0115] step: count(1037), step_time 1424.11, mean_step_time 1439.48, it/s 0.69
  5133. sending to address tcp://p0112:61216
  5134. ##### Sending to neptune: online_score : 0.421175178554 , 0.9 #####
  5135. [u'online', 0.9]
  5136. receiving
  5137. [0130 17:55:18 @multigpu.py:323] [p0574] step: count(1053), step_time 1493.1, mean_step_time 1432.63, it/s 0.7
  5138. [0130 17:55:18 @multigpu.py:323] [p0576] step: count(1062), step_time 1423.26, mean_step_time 1442.71, it/s 0.69
  5139. [0130 17:55:18 @multigpu.py:323] [p0115] step: count(1038), step_time 1529.8, mean_step_time 1444.2, it/s 0.69
  5140. [0130 17:55:19 @multigpu.py:323] [p0574] step: count(1054), step_time 1428.93, mean_step_time 1434.13, it/s 0.7
  5141. [0130 17:55:19 @multigpu.py:323] [p0576] step: count(1063), step_time 1395.72, mean_step_time 1438.78, it/s 0.7
  5142. [0130 17:55:19 @multigpu.py:323] [p0115] step: count(1039), step_time 1404.79, mean_step_time 1443.11, it/s 0.69
  5143. [0130 17:55:21 @multigpu.py:323] [p0576] step: count(1064), step_time 1376.76, mean_step_time 1433.59, it/s 0.7
  5144. [0130 17:55:21 @multigpu.py:323] [p0574] step: count(1055), step_time 1439.54, mean_step_time 1435.19, it/s 0.7
  5145. [0130 17:55:21 @multigpu.py:323] [p0115] step: count(1040), step_time 1448.15, mean_step_time 1444.78, it/s 0.69
  5146. [0130 17:55:22 @multigpu.py:323] [p0576] step: count(1065), step_time 1459.01, mean_step_time 1431.51, it/s 0.7
  5147. [0130 17:55:22 @multigpu.py:323] [p0574] step: count(1056), step_time 1463.82, mean_step_time 1439.71, it/s 0.69
  5148. [0130 17:55:22 @multigpu.py:323] [p0115] step: count(1041), step_time 1461.35, mean_step_time 1445.5, it/s 0.69
  5149. [0130 17:55:23 @multigpu.py:323] [p0576] step: count(1066), step_time 1372.05, mean_step_time 1430.91, it/s 0.7
  5150. [0130 17:55:24 @multigpu.py:323] [p0574] step: count(1057), step_time 1457.14, mean_step_time 1440.27, it/s 0.69
  5151. [0130 17:55:24 @multigpu.py:323] [p0115] step: count(1042), step_time 1423.25, mean_step_time 1444.08, it/s 0.69
  5152. [0130 17:55:25 @multigpu.py:323] [p0576] step: count(1067), step_time 1508.45, mean_step_time 1432.4, it/s 0.7
  5153. [0130 17:55:25 @multigpu.py:323] [p0574] step: count(1058), step_time 1400.48, mean_step_time 1440.97, it/s 0.69
  5154. [0130 17:55:25 @multigpu.py:323] [p0115] step: count(1043), step_time 1475.45, mean_step_time 1446.09, it/s 0.69
  5155. [0130 17:55:26 @multigpu.py:323] [p0576] step: count(1068), step_time 1395.59, mean_step_time 1431.24, it/s 0.7
  5156. [0130 17:55:26 @multigpu.py:323] [p0574] step: count(1059), step_time 1448.04, mean_step_time 1441.98, it/s 0.69
  5157. [0130 17:55:27 @multigpu.py:323] [p0115] step: count(1044), step_time 1390.38, mean_step_time 1443.76, it/s 0.69
  5158. [0130 17:55:28 @multigpu.py:323] [p0576] step: count(1069), step_time 1399.62, mean_step_time 1426.32, it/s 0.7
  5159. [0130 17:55:28 @multigpu.py:323] [p0574] step: count(1060), step_time 1422.05, mean_step_time 1443.31, it/s 0.69
  5160. [0130 17:55:28 @multigpu.py:323] [p0115] step: count(1045), step_time 1472.17, mean_step_time 1446.62, it/s 0.69
  5161. [0130 17:55:29 @multigpu.py:323] [p0576] step: count(1070), step_time 1414.84, mean_step_time 1427.67, it/s 0.7
  5162. [0130 17:55:29 @multigpu.py:323] [p0574] step: count(1061), step_time 1457.26, mean_step_time 1444.69, it/s 0.69
  5163. [0130 17:55:30 @multigpu.py:323] [p0115] step: count(1046), step_time 1464.66, mean_step_time 1447.05, it/s 0.69
  5164. [0130 17:55:31 @multigpu.py:323] [p0576] step: count(1071), step_time 1431.19, mean_step_time 1426.51, it/s 0.7
  5165. [0130 17:55:31 @multigpu.py:323] [p0574] step: count(1062), step_time 1417.34, mean_step_time 1445.34, it/s 0.69
  5166. [0130 17:55:31 @multigpu.py:323] [p0115] step: count(1047), step_time 1442.89, mean_step_time 1446.82, it/s 0.69
  5167. sending to address tcp://p0112:61216
  5168. ##### Sending to neptune: online_score : 0.425126151906 , 1.7 #####
  5169. [u'online', 1.7]
  5170. receiving
  5171. sending to address tcp://p0112:61216
  5172. ##### Sending to neptune: online_score : 0.42519397027 , 1.3 #####
  5173. [u'online', 1.3]
  5174. receiving
  5175. [0130 17:55:32 @multigpu.py:323] [p0574] step: count(1063), step_time 1743.95, mean_step_time 1445.16, it/s 0.69
  5176. [0130 17:55:32 @multigpu.py:323] [p0576] step: count(1072), step_time 1891.85, mean_step_time 1445.13, it/s 0.69
  5177. [0130 17:55:32 @multigpu.py:323] [p0115] step: count(1048), step_time 1451.52, mean_step_time 1445.18, it/s 0.69
  5178. [0130 17:55:34 @multigpu.py:323] [p0576] step: count(1073), step_time 1409.99, mean_step_time 1444.73, it/s 0.69
  5179. [0130 17:55:34 @multigpu.py:323] [p0574] step: count(1064), step_time 1415.11, mean_step_time 1445.03, it/s 0.69
  5180. [0130 17:55:34 @multigpu.py:323] [p0115] step: count(1049), step_time 1455.97, mean_step_time 1444.34, it/s 0.69
  5181. [0130 17:55:35 @multigpu.py:323] [p0576] step: count(1074), step_time 1400.35, mean_step_time 1444.2, it/s 0.69
  5182. [0130 17:55:35 @multigpu.py:323] [p0574] step: count(1065), step_time 1401.07, mean_step_time 1445.38, it/s 0.69
  5183. [0130 17:55:35 @multigpu.py:323] [p0115] step: count(1050), step_time 1371.89, mean_step_time 1441.89, it/s 0.69
  5184. [0130 17:55:37 @multigpu.py:323] [p0574] step: count(1066), step_time 1459.04, mean_step_time 1448.65, it/s 0.69
  5185. [0130 17:55:37 @multigpu.py:323] [p0115] step: count(1051), step_time 1458.72, mean_step_time 1440.53, it/s 0.69
  5186. [0130 17:55:37 @multigpu.py:323] [p0576] step: count(1075), step_time 1482.75, mean_step_time 1445.42, it/s 0.69
  5187. [0130 17:55:38 @multigpu.py:323] [p0574] step: count(1067), step_time 1401.78, mean_step_time 1444.79, it/s 0.69
  5188. [0130 17:55:38 @multigpu.py:323] [p0115] step: count(1052), step_time 1408.94, mean_step_time 1440.25, it/s 0.69
  5189. [0130 17:55:38 @multigpu.py:323] [p0576] step: count(1076), step_time 1422.88, mean_step_time 1442.16, it/s 0.69
  5190. [0130 17:55:40 @multigpu.py:323] [p0576] step: count(1077), step_time 1375.78, mean_step_time 1442.12, it/s 0.69
  5191. [0130 17:55:40 @multigpu.py:323] [p0574] step: count(1068), step_time 1455.3, mean_step_time 1447.13, it/s 0.69
  5192. [0130 17:55:40 @multigpu.py:323] [p0115] step: count(1053), step_time 1468.53, mean_step_time 1442.54, it/s 0.69
  5193. [0130 17:55:41 @multigpu.py:323] [p0574] step: count(1069), step_time 1394.51, mean_step_time 1446.93, it/s 0.69
  5194. [0130 17:55:41 @multigpu.py:323] [p0576] step: count(1078), step_time 1477.76, mean_step_time 1444.18, it/s 0.69
  5195. [0130 17:55:41 @multigpu.py:323] [p0115] step: count(1054), step_time 1483.98, mean_step_time 1444.53, it/s 0.69
  5196. [0130 17:55:42 @multigpu.py:323] [p0574] step: count(1070), step_time 1435.69, mean_step_time 1447.75, it/s 0.69
  5197. [0130 17:55:42 @multigpu.py:323] [p0576] step: count(1079), step_time 1457.34, mean_step_time 1443.85, it/s 0.69
  5198. [0130 17:55:43 @multigpu.py:323] [p0115] step: count(1055), step_time 1441.54, mean_step_time 1444.98, it/s 0.69
  5199. [0130 17:55:44 @multigpu.py:323] [p0574] step: count(1071), step_time 1401.82, mean_step_time 1446.35, it/s 0.69
  5200. [0130 17:55:44 @multigpu.py:323] [p0576] step: count(1080), step_time 1395.95, mean_step_time 1444.07, it/s 0.69
  5201. [0130 17:55:44 @multigpu.py:323] [p0115] step: count(1056), step_time 1444.04, mean_step_time 1446.11, it/s 0.69
  5202. [0130 17:55:45 @multigpu.py:323] [p0574] step: count(1072), step_time 1388.22, mean_step_time 1446.21, it/s 0.69
  5203. [0130 17:55:45 @multigpu.py:323] [p0576] step: count(1081), step_time 1439.41, mean_step_time 1446.53, it/s 0.69
  5204. [0130 17:55:45 @multigpu.py:323] [p0115] step: count(1057), step_time 1424.31, mean_step_time 1446.12, it/s 0.69
  5205. [0130 17:55:47 @multigpu.py:323] [p0574] step: count(1073), step_time 1442.95, mean_step_time 1443.7, it/s 0.69
  5206. [0130 17:55:47 @multigpu.py:323] [p0576] step: count(1082), step_time 1515.43, mean_step_time 1451.14, it/s 0.69
  5207. [0130 17:55:47 @multigpu.py:323] [p0115] step: count(1058), step_time 1443.79, mean_step_time 1441.82, it/s 0.69
  5208. [0130 17:55:48 @multigpu.py:323] [p0574] step: count(1074), step_time 1441.68, mean_step_time 1444.34, it/s 0.69
  5209. [0130 17:55:48 @multigpu.py:323] [p0576] step: count(1083), step_time 1382.39, mean_step_time 1450.47, it/s 0.69
  5210. [0130 17:55:48 @multigpu.py:323] [p0115] step: count(1059), step_time 1439.56, mean_step_time 1443.56, it/s 0.69
  5211. [0130 17:55:49 @multigpu.py:323] [p0574] step: count(1075), step_time 1374.67, mean_step_time 1441.1, it/s 0.69
  5212. [0130 17:55:50 @multigpu.py:323] [p0576] step: count(1084), step_time 1449.83, mean_step_time 1454.12, it/s 0.69
  5213. [0130 17:55:50 @multigpu.py:323] [p0115] step: count(1060), step_time 1461.11, mean_step_time 1444.2, it/s 0.69
  5214. [0130 17:55:51 @multigpu.py:323] [p0574] step: count(1076), step_time 1471.64, mean_step_time 1441.49, it/s 0.69
  5215. [0130 17:55:51 @multigpu.py:323] [p0576] step: count(1085), step_time 1367.67, mean_step_time 1449.56, it/s 0.69
  5216. [0130 17:55:51 @multigpu.py:323] [p0115] step: count(1061), step_time 1464.32, mean_step_time 1444.35, it/s 0.69
  5217. [0130 17:55:52 @multigpu.py:323] [p0574] step: count(1077), step_time 1408.18, mean_step_time 1439.04, it/s 0.69
  5218. [0130 17:55:53 @multigpu.py:323] [p0576] step: count(1086), step_time 1499.73, mean_step_time 1455.94, it/s 0.69
  5219. [0130 17:55:53 @multigpu.py:323] [p0115] step: count(1062), step_time 1418.45, mean_step_time 1444.11, it/s 0.69
  5220. sending to address tcp://p0112:61216
  5221. ##### Sending to neptune: online_score : 0.431302230491 , 1.3 #####
  5222. [u'online', 1.3]
  5223. receiving
  5224. [0130 17:55:54 @multigpu.py:323] [p0574] step: count(1078), step_time 1357.14, mean_step_time 1436.87, it/s 0.7
  5225. [0130 17:55:54 @multigpu.py:323] [p0576] step: count(1087), step_time 1445.39, mean_step_time 1452.79, it/s 0.69
  5226. [0130 17:55:54 @multigpu.py:323] [p0115] step: count(1063), step_time 1366.45, mean_step_time 1438.66, it/s 0.7
  5227. [0130 17:55:55 @multigpu.py:323] [p0574] step: count(1079), step_time 1467.04, mean_step_time 1437.82, it/s 0.7
  5228. [0130 17:55:55 @multigpu.py:323] [p0576] step: count(1088), step_time 1401.09, mean_step_time 1453.06, it/s 0.69
  5229. [0130 17:55:55 @multigpu.py:323] [p0115] step: count(1064), step_time 1437.42, mean_step_time 1441.01, it/s 0.69
  5230. [0130 17:55:57 @multigpu.py:323] [p0574] step: count(1080), step_time 1421.0, mean_step_time 1437.77, it/s 0.7
  5231. [0130 17:55:57 @multigpu.py:323] [p0576] step: count(1089), step_time 1454.31, mean_step_time 1455.8, it/s 0.69
  5232. [0130 17:55:57 @multigpu.py:323] [p0115] step: count(1065), step_time 1415.52, mean_step_time 1438.18, it/s 0.7
  5233. [0130 17:55:58 @multigpu.py:323] [p0574] step: count(1081), step_time 1413.1, mean_step_time 1435.56, it/s 0.7
  5234. [0130 17:55:58 @multigpu.py:323] [p0576] step: count(1090), step_time 1415.9, mean_step_time 1455.85, it/s 0.69
  5235. [0130 17:55:58 @multigpu.py:323] [p0115] step: count(1066), step_time 1426.7, mean_step_time 1436.28, it/s 0.7
  5236. [0130 17:55:59 @multigpu.py:323] [p0574] step: count(1082), step_time 1393.46, mean_step_time 1434.37, it/s 0.7
  5237. [0130 17:56:00 @multigpu.py:323] [p0576] step: count(1091), step_time 1428.45, mean_step_time 1455.71, it/s 0.69
  5238. [0130 17:56:00 @multigpu.py:323] [p0115] step: count(1067), step_time 1429.34, mean_step_time 1435.61, it/s 0.7
  5239. [0130 17:56:01 @multigpu.py:323] [p0115] step: count(1068), step_time 1448.33, mean_step_time 1435.45, it/s 0.7
  5240. [0130 17:56:01 @multigpu.py:323] [p0576] step: count(1092), step_time 1486.72, mean_step_time 1435.46, it/s 0.7
  5241. [0130 17:56:01 @multigpu.py:323] [p0574] step: count(1083), step_time 1768.75, mean_step_time 1435.61, it/s 0.7
  5242. [0130 17:56:03 @multigpu.py:323] [p0576] step: count(1093), step_time 1380.93, mean_step_time 1434.0, it/s 0.7
  5243. [0130 17:56:03 @multigpu.py:323] [p0115] step: count(1069), step_time 1439.15, mean_step_time 1434.6, it/s 0.7
  5244. [0130 17:56:03 @multigpu.py:323] [p0574] step: count(1084), step_time 1507.57, mean_step_time 1440.23, it/s 0.69
  5245. sending to address tcp://p0112:61216
  5246. ##### Sending to neptune: online_score : 0.433990958863 , 1.7 #####
  5247. [u'online', 1.7]
  5248. ##### Sending to neptune: active_workers : 0.433991316888 , 3 #####
  5249. receiving
  5250. [0130 17:56:04 @multigpu.py:323] [p0576] step: count(1094), step_time 1471.82, mean_step_time 1437.58, it/s 0.7
  5251. [0130 17:56:04 @multigpu.py:323] [p0115] step: count(1070), step_time 1424.13, mean_step_time 1437.22, it/s 0.7
  5252. [0130 17:56:04 @multigpu.py:323] [p0574] step: count(1085), step_time 1368.88, mean_step_time 1438.62, it/s 0.7
  5253. sending to address tcp://p0112:61216
  5254. ##### Sending to neptune: online_score : 0.434331390262 , 1.6 #####
  5255. [u'online', 1.6]
  5256. receiving
  5257. [0130 17:56:05 @multigpu.py:323] [p0576] step: count(1095), step_time 1405.61, mean_step_time 1433.72, it/s 0.7
  5258. [0130 17:56:05 @multigpu.py:323] [p0115] step: count(1071), step_time 1422.46, mean_step_time 1435.4, it/s 0.7
  5259. [0130 17:56:05 @multigpu.py:323] [p0574] step: count(1086), step_time 1438.94, mean_step_time 1437.62, it/s 0.7
  5260. [0130 17:56:07 @multigpu.py:323] [p0115] step: count(1072), step_time 1412.86, mean_step_time 1435.6, it/s 0.7
  5261. [0130 17:56:07 @multigpu.py:323] [p0576] step: count(1096), step_time 1456.46, mean_step_time 1435.4, it/s 0.7
  5262. [0130 17:56:07 @multigpu.py:323] [p0574] step: count(1087), step_time 1446.37, mean_step_time 1439.85, it/s 0.69
  5263. [0130 17:56:08 @multigpu.py:323] [p0576] step: count(1097), step_time 1388.13, mean_step_time 1436.02, it/s 0.7
  5264. [0130 17:56:08 @multigpu.py:323] [p0115] step: count(1073), step_time 1441.22, mean_step_time 1434.23, it/s 0.7
  5265. [0130 17:56:08 @multigpu.py:323] [p0574] step: count(1088), step_time 1420.67, mean_step_time 1438.11, it/s 0.7
  5266. [0130 17:56:10 @multigpu.py:323] [p0576] step: count(1098), step_time 1390.21, mean_step_time 1431.64, it/s 0.7
  5267. [0130 17:56:10 @multigpu.py:323] [p0115] step: count(1074), step_time 1403.55, mean_step_time 1430.21, it/s 0.7
  5268. [0130 17:56:10 @multigpu.py:323] [p0574] step: count(1089), step_time 1454.05, mean_step_time 1441.09, it/s 0.69
  5269. [0130 17:56:11 @multigpu.py:323] [p0576] step: count(1099), step_time 1401.55, mean_step_time 1428.85, it/s 0.7
  5270. [0130 17:56:11 @multigpu.py:323] [p0115] step: count(1075), step_time 1449.39, mean_step_time 1430.6, it/s 0.7
  5271. [0130 17:56:11 @multigpu.py:323] [p0574] step: count(1090), step_time 1442.23, mean_step_time 1441.42, it/s 0.69
  5272. [0130 17:56:13 @multigpu.py:323] [p0576] step: count(1100), step_time 1444.12, mean_step_time 1431.26, it/s 0.7
  5273. sending debugging info...
  5274. sending to address tcp://p0112:61216
  5275. ##### Sending to neptune: mean_delay : 0.436645072765 , 0.0 #####
  5276. sending to address tcp://p0112:61216
  5277. ##### Sending to neptune: max_delay : 0.436645072765 , -0.0 #####
  5278. sending to address tcp://p0112:61216
  5279. ##### Sending to neptune: min_delay : 0.436645072765 , -0.0 #####
  5280. [u'delays', [0.0, -0.0, -0.0]]
  5281. receiving
  5282. ##### Sending to neptune: cost : 0.436646228035 , -0.0118187069893 #####
  5283. ##### Sending to neptune: policy_loss : 0.436646228035 , -0.450571477413 #####
  5284. ##### Sending to neptune: xentropy_loss : 0.436646228035 , -2.28830933571 #####
  5285. ##### Sending to neptune: value_loss : 0.436646228035 , 1.22608649731 #####
  5286. ##### Sending to neptune: advantage : 0.436646228035 , 0.00196923920885 #####
  5287. ##### Sending to neptune: pred_reward : 0.436646228035 , 0.406696081161 #####
  5288. ##### Sending to neptune: max_logit : 0.436646228035 , 0.194596156478 #####
  5289. [u'loss', -0.01181870698928833, -0.4505714774131775, -2.288309335708618, 1.2260864973068237, 0.0019692392088472843, 0.406696081161499, 0.19459615647792816]
  5290. receiving
  5291. ##### Sending to neptune: active_relus : 0.436649463574 , 9426596.31 #####
  5292. ##### Sending to neptune: dp_per_s : 0.436649463574 , 89.1332720899 #####
  5293. [u'other', 9426596.31, 89.13327208992234]
  5294. receiving
  5295. [0130 17:56:13 @multigpu.py:323] [p0115] step: count(1076), step_time 1434.04, mean_step_time 1430.1, it/s 0.7
  5296. [0130 17:56:13 @multigpu.py:323] [p0574] step: count(1091), step_time 1448.36, mean_step_time 1443.75, it/s 0.69
  5297. sending to address tcp://p0112:61216
  5298. ##### Sending to neptune: online_score : 0.436772664388 , 1.7 #####
  5299. [u'online', 1.7]
  5300. receiving
  5301. [0130 17:56:14 @multigpu.py:323] [p0576] step: count(1101), step_time 1425.92, mean_step_time 1430.58, it/s 0.7
  5302. [0130 17:56:14 @multigpu.py:323] [p0115] step: count(1077), step_time 1443.31, mean_step_time 1431.05, it/s 0.7
  5303. [0130 17:56:14 @multigpu.py:323] [p0574] step: count(1092), step_time 1399.08, mean_step_time 1444.29, it/s 0.69
  5304. [0130 17:56:15 @multigpu.py:323] [p0576] step: count(1102), step_time 1399.16, mean_step_time 1424.77, it/s 0.7
  5305. [0130 17:56:15 @multigpu.py:323] [p0115] step: count(1078), step_time 1415.94, mean_step_time 1429.66, it/s 0.7
  5306. [0130 17:56:16 @multigpu.py:323] [p0574] step: count(1093), step_time 1449.26, mean_step_time 1444.6, it/s 0.69
  5307. [0130 17:56:17 @multigpu.py:323] [p0576] step: count(1103), step_time 1461.29, mean_step_time 1428.71, it/s 0.7
  5308. [0130 17:56:17 @multigpu.py:323] [p0115] step: count(1079), step_time 1432.04, mean_step_time 1429.29, it/s 0.7
  5309. [0130 17:56:17 @multigpu.py:323] [p0574] step: count(1094), step_time 1396.09, mean_step_time 1442.32, it/s 0.69
  5310. [0130 17:56:18 @multigpu.py:323] [p0576] step: count(1104), step_time 1415.22, mean_step_time 1426.98, it/s 0.7
  5311. [0130 17:56:18 @multigpu.py:323] [p0115] step: count(1080), step_time 1432.88, mean_step_time 1427.87, it/s 0.7
  5312. [0130 17:56:18 @multigpu.py:323] [p0574] step: count(1095), step_time 1397.45, mean_step_time 1443.46, it/s 0.69
  5313. [0130 17:56:20 @multigpu.py:323] [p0576] step: count(1105), step_time 1417.93, mean_step_time 1429.5, it/s 0.7
  5314. [0130 17:56:20 @multigpu.py:323] [p0115] step: count(1081), step_time 1441.1, mean_step_time 1426.71, it/s 0.7
  5315. [0130 17:56:20 @multigpu.py:323] [p0574] step: count(1096), step_time 1459.25, mean_step_time 1442.84, it/s 0.69
  5316. [0130 17:56:21 @multigpu.py:323] [p0576] step: count(1106), step_time 1440.26, mean_step_time 1426.52, it/s 0.7
  5317. [0130 17:56:21 @multigpu.py:323] [p0115] step: count(1082), step_time 1419.84, mean_step_time 1426.78, it/s 0.7
  5318. [0130 17:56:21 @multigpu.py:323] [p0574] step: count(1097), step_time 1449.18, mean_step_time 1444.89, it/s 0.69
  5319. [0130 17:56:23 @multigpu.py:323] [p0576] step: count(1107), step_time 1465.27, mean_step_time 1427.52, it/s 0.7
  5320. [0130 17:56:23 @multigpu.py:323] [p0115] step: count(1083), step_time 1423.24, mean_step_time 1429.62, it/s 0.7
  5321. [0130 17:56:23 @multigpu.py:323] [p0574] step: count(1098), step_time 1523.52, mean_step_time 1453.21, it/s 0.69
  5322. [0130 17:56:24 @multigpu.py:323] [p0576] step: count(1108), step_time 1403.19, mean_step_time 1427.62, it/s 0.7
  5323. [0130 17:56:24 @multigpu.py:323] [p0115] step: count(1084), step_time 1477.21, mean_step_time 1431.61, it/s 0.7
  5324. [0130 17:56:24 @multigpu.py:323] [p0574] step: count(1099), step_time 1472.36, mean_step_time 1453.48, it/s 0.69
  5325. [0130 17:56:25 @multigpu.py:323] [p0576] step: count(1109), step_time 1415.13, mean_step_time 1425.66, it/s 0.7
  5326. [0130 17:56:25 @multigpu.py:323] [p0115] step: count(1085), step_time 1391.89, mean_step_time 1430.43, it/s 0.7
  5327. [0130 17:56:26 @multigpu.py:323] [p0574] step: count(1100), step_time 1408.9, mean_step_time 1452.87, it/s 0.69
  5328. sending debugging info...
  5329. sending to address tcp://p0112:61216
  5330. ##### Sending to neptune: mean_delay : 0.440297738579 , 0.0 #####
  5331. ##### Sending to neptune: max_delay : 0.440297738579 , -0.0 #####
  5332. sending to address tcp://p0112:61216
  5333. ##### Sending to neptune: min_delay : 0.440297738579 , -0.0 #####
  5334. [u'delays', [0.0, -0.0, -0.0]]
  5335. receiving
  5336. ##### Sending to neptune: cost : 0.440298264954 , -0.00412045605481 #####
  5337. ##### Sending to neptune: policy_loss : 0.440298264954 , 0.338887602091 #####
  5338. sending to address tcp://p0112:61216
  5339. ##### Sending to neptune: xentropy_loss : 0.440298264954 , -2.28836631775 #####
  5340. ##### Sending to neptune: value_loss : 0.440298264954 , 1.42206013203 #####
  5341. ##### Sending to neptune: advantage : 0.440298264954 , -0.00142946292181 #####
  5342. ##### Sending to neptune: pred_reward : 0.440298264954 , 0.410076230764 #####
  5343. ##### Sending to neptune: max_logit : 0.440298264954 , 0.194619998336 #####
  5344. [u'loss', -0.004120456054806709, 0.33888760209083557, -2.2883663177490234, 1.4220601320266724, -0.0014294629218056798, 0.41007623076438904, 0.19461999833583832]
  5345. receiving
  5346. ##### Sending to neptune: active_relus : 0.440298698876 , 9435936.15 #####
  5347. ##### Sending to neptune: dp_per_s : 0.440298698876 , 89.0701042424 #####
  5348. [u'other', 9435936.15, 89.07010424237725]
  5349. receiving
  5350. [0130 17:56:27 @multigpu.py:323] [p0576] step: count(1110), step_time 1438.01, mean_step_time 1426.77, it/s 0.7
  5351. [0130 17:56:27 @multigpu.py:323] [p0115] step: count(1086), step_time 1432.05, mean_step_time 1430.7, it/s 0.7
  5352. [0130 17:56:27 @multigpu.py:323] [p0574] step: count(1101), step_time 1443.19, mean_step_time 1454.38, it/s 0.69
  5353. [0130 17:56:28 @multigpu.py:323] [p0576] step: count(1111), step_time 1431.44, mean_step_time 1426.92, it/s 0.7
  5354. [0130 17:56:28 @multigpu.py:323] [p0115] step: count(1087), step_time 1435.4, mean_step_time 1431.0, it/s 0.7
  5355. [0130 17:56:29 @multigpu.py:323] [p0574] step: count(1102), step_time 1433.72, mean_step_time 1456.39, it/s 0.69
  5356. sending to address tcp://p0112:61216
  5357. ##### Sending to neptune: online_score : 0.441290416651 , 1.3 #####
  5358. [u'online', 1.3]
  5359. receiving
  5360. [0130 17:56:30 @multigpu.py:323] [p0115] step: count(1088), step_time 1701.67, mean_step_time 1443.67, it/s 0.69
  5361. [0130 17:56:30 @multigpu.py:323] [p0574] step: count(1103), step_time 1512.89, mean_step_time 1443.6, it/s 0.69
  5362. [0130 17:56:30 @multigpu.py:323] [p0576] step: count(1112), step_time 1822.69, mean_step_time 1443.72, it/s 0.69
  5363. [0130 17:56:32 @multigpu.py:323] [p0115] step: count(1089), step_time 1452.35, mean_step_time 1444.33, it/s 0.69
  5364. [0130 17:56:32 @multigpu.py:323] [p0576] step: count(1113), step_time 1466.74, mean_step_time 1448.01, it/s 0.69
  5365. [0130 17:56:32 @multigpu.py:323] [p0574] step: count(1104), step_time 1532.74, mean_step_time 1444.86, it/s 0.69
  5366. [0130 17:56:33 @multigpu.py:323] [p0115] step: count(1090), step_time 1384.42, mean_step_time 1442.34, it/s 0.69
  5367. [0130 17:56:33 @multigpu.py:323] [p0576] step: count(1114), step_time 1431.12, mean_step_time 1445.97, it/s 0.69
  5368. [0130 17:56:33 @multigpu.py:323] [p0574] step: count(1105), step_time 1461.45, mean_step_time 1449.49, it/s 0.69
  5369. sending to address tcp://p0112:61216
  5370. ##### Sending to neptune: online_score : 0.442541402446 , 1.8 #####
  5371. [u'online', 1.8]
  5372. receiving
  5373. [0130 17:56:34 @multigpu.py:323] [p0115] step: count(1091), step_time 1429.84, mean_step_time 1442.71, it/s 0.69
  5374. [0130 17:56:34 @multigpu.py:323] [p0576] step: count(1115), step_time 1403.25, mean_step_time 1445.86, it/s 0.69
  5375. [0130 17:56:35 @multigpu.py:323] [p0574] step: count(1106), step_time 1487.84, mean_step_time 1451.93, it/s 0.69
  5376. [0130 17:56:36 @multigpu.py:323] [p0115] step: count(1092), step_time 1448.2, mean_step_time 1444.48, it/s 0.69
  5377. [0130 17:56:36 @multigpu.py:323] [p0576] step: count(1116), step_time 1437.0, mean_step_time 1444.88, it/s 0.69
  5378. [0130 17:56:36 @multigpu.py:323] [p0574] step: count(1107), step_time 1386.29, mean_step_time 1448.93, it/s 0.69
  5379. [0130 17:56:37 @multigpu.py:323] [p0115] step: count(1093), step_time 1401.62, mean_step_time 1442.5, it/s 0.69
  5380. [0130 17:56:37 @multigpu.py:323] [p0576] step: count(1117), step_time 1425.33, mean_step_time 1446.74, it/s 0.69
  5381. [0130 17:56:37 @multigpu.py:323] [p0574] step: count(1108), step_time 1515.78, mean_step_time 1453.68, it/s 0.69
  5382. [0130 17:56:39 @multigpu.py:323] [p0115] step: count(1094), step_time 1457.24, mean_step_time 1445.18, it/s 0.69
  5383. [0130 17:56:39 @multigpu.py:323] [p0576] step: count(1118), step_time 1448.3, mean_step_time 1449.65, it/s 0.69
  5384. [0130 17:56:39 @multigpu.py:323] [p0574] step: count(1109), step_time 1421.13, mean_step_time 1452.04, it/s 0.69
  5385. [0130 17:56:40 @multigpu.py:323] [p0115] step: count(1095), step_time 1438.18, mean_step_time 1444.62, it/s 0.69
  5386. [0130 17:56:40 @multigpu.py:323] [p0576] step: count(1119), step_time 1476.4, mean_step_time 1453.39, it/s 0.69
  5387. sending to address tcp://p0112:61216
  5388. ##### Sending to neptune: online_score : 0.444324848586 , 0.9 #####
  5389. [u'online', 0.9]
  5390. receiving
  5391. [0130 17:56:40 @multigpu.py:323] [p0574] step: count(1110), step_time 1461.83, mean_step_time 1453.02, it/s 0.69
  5392. [0130 17:56:42 @multigpu.py:323] [p0115] step: count(1096), step_time 1436.98, mean_step_time 1444.77, it/s 0.69
  5393. [0130 17:56:42 @multigpu.py:323] [p0576] step: count(1120), step_time 1495.99, mean_step_time 1455.98, it/s 0.69
  5394. [0130 17:56:42 @multigpu.py:323] [p0574] step: count(1111), step_time 1451.65, mean_step_time 1453.18, it/s 0.69
  5395. [0130 17:56:43 @multigpu.py:323] [p0115] step: count(1097), step_time 1431.01, mean_step_time 1444.16, it/s 0.69
  5396. [0130 17:56:43 @multigpu.py:323] [p0576] step: count(1121), step_time 1425.27, mean_step_time 1455.95, it/s 0.69
  5397. [0130 17:56:43 @multigpu.py:323] [p0574] step: count(1112), step_time 1480.7, mean_step_time 1457.26, it/s 0.69
  5398. [0130 17:56:44 @multigpu.py:323] [p0115] step: count(1098), step_time 1418.93, mean_step_time 1444.31, it/s 0.69
  5399. [0130 17:56:45 @multigpu.py:323] [p0576] step: count(1122), step_time 1429.95, mean_step_time 1457.49, it/s 0.69
  5400. [0130 17:56:45 @multigpu.py:323] [p0574] step: count(1113), step_time 1467.23, mean_step_time 1458.16, it/s 0.69
  5401. [0130 17:56:46 @multigpu.py:323] [p0115] step: count(1099), step_time 1460.5, mean_step_time 1445.73, it/s 0.69
  5402. [0130 17:56:46 @multigpu.py:323] [p0576] step: count(1123), step_time 1399.13, mean_step_time 1454.38, it/s 0.69
  5403. [0130 17:56:46 @multigpu.py:323] [p0574] step: count(1114), step_time 1568.11, mean_step_time 1466.76, it/s 0.68
  5404. [0130 17:56:47 @multigpu.py:323] [p0115] step: count(1100), step_time 1442.36, mean_step_time 1446.2, it/s 0.69
  5405. sending debugging info...
  5406. sending to address tcp://p0112:61216
  5407. ##### Sending to neptune: mean_delay : 0.446297637489 , 0.0 #####
  5408. sending to address tcp://p0112:61216
  5409. ##### Sending to neptune: max_delay : 0.446297637489 , -0.0 #####
  5410. ##### Sending to neptune: min_delay : 0.446297637489 , -0.0 #####
  5411. [u'delays', [0.0, -0.0, -0.0]]
  5412. receiving
  5413. ##### Sending to neptune: cost : 0.446298163599 , -0.00521783763543 #####
  5414. sending to address tcp://p0112:61216
  5415. ##### Sending to neptune: policy_loss : 0.446298163599 , 0.202429369092 #####
  5416. ##### Sending to neptune: xentropy_loss : 0.446298163599 , -2.28854560852 #####
  5417. ##### Sending to neptune: value_loss : 0.446298163599 , 1.4182331562 #####
  5418. ##### Sending to neptune: advantage : 0.446298163599 , -0.000883870001417 #####
  5419. ##### Sending to neptune: pred_reward : 0.446298163599 , 0.415664792061 #####
  5420. ##### Sending to neptune: max_logit : 0.446298163599 , 0.193621426821 #####
  5421. [u'loss', -0.005217837635427713, 0.2024293690919876, -2.288545608520508, 1.4182331562042236, -0.0008838700014166534, 0.41566479206085205, 0.193621426820755]
  5422. receiving
  5423. ##### Sending to neptune: active_relus : 0.446298656927 , 9437430.47 #####
  5424. ##### Sending to neptune: dp_per_s : 0.446298656927 , 89.0110297975 #####
  5425. [u'other', 9437430.47, 89.01102979748308]
  5426. receiving
  5427. [0130 17:56:47 @multigpu.py:323] [p0576] step: count(1124), step_time 1413.07, mean_step_time 1454.27, it/s 0.69
  5428. [0130 17:56:48 @multigpu.py:323] [p0574] step: count(1115), step_time 1509.03, mean_step_time 1472.34, it/s 0.68
  5429. [0130 17:56:49 @multigpu.py:323] [p0115] step: count(1101), step_time 1461.1, mean_step_time 1447.2, it/s 0.69
  5430. [0130 17:56:49 @multigpu.py:323] [p0576] step: count(1125), step_time 1438.87, mean_step_time 1455.32, it/s 0.69
  5431. [0130 17:56:49 @multigpu.py:323] [p0574] step: count(1116), step_time 1484.95, mean_step_time 1473.62, it/s 0.68
  5432. [0130 17:56:50 @multigpu.py:323] [p0576] step: count(1126), step_time 1414.05, mean_step_time 1454.01, it/s 0.69
  5433. [0130 17:56:50 @multigpu.py:323] [p0115] step: count(1102), step_time 1447.27, mean_step_time 1448.57, it/s 0.69
  5434. [0130 17:56:51 @multigpu.py:323] [p0574] step: count(1117), step_time 1576.75, mean_step_time 1480.0, it/s 0.68
  5435. [0130 17:56:52 @multigpu.py:323] [p0115] step: count(1103), step_time 1403.75, mean_step_time 1447.6, it/s 0.69
  5436. [0130 17:56:52 @multigpu.py:323] [p0576] step: count(1127), step_time 1415.42, mean_step_time 1451.52, it/s 0.69
  5437. [0130 17:56:52 @multigpu.py:323] [p0574] step: count(1118), step_time 1448.47, mean_step_time 1476.25, it/s 0.68
  5438. [0130 17:56:53 @multigpu.py:323] [p0115] step: count(1104), step_time 1416.13, mean_step_time 1444.54, it/s 0.69
  5439. [0130 17:56:53 @multigpu.py:323] [p0576] step: count(1128), step_time 1455.43, mean_step_time 1454.13, it/s 0.69
  5440. [0130 17:56:54 @multigpu.py:323] [p0574] step: count(1119), step_time 1405.29, mean_step_time 1472.9, it/s 0.68
  5441. [0130 17:56:54 @multigpu.py:323] [p0115] step: count(1105), step_time 1434.33, mean_step_time 1446.67, it/s 0.69
  5442. [0130 17:56:54 @multigpu.py:323] [p0576] step: count(1129), step_time 1427.13, mean_step_time 1454.73, it/s 0.69
  5443. [0130 17:56:55 @multigpu.py:323] [p0574] step: count(1120), step_time 1456.45, mean_step_time 1475.27, it/s 0.68
  5444. [0130 17:56:56 @multigpu.py:323] [p0115] step: count(1106), step_time 1438.43, mean_step_time 1446.99, it/s 0.69
  5445. [0130 17:56:56 @multigpu.py:323] [p0576] step: count(1130), step_time 1414.84, mean_step_time 1453.57, it/s 0.69
  5446. [0130 17:56:57 @multigpu.py:323] [p0574] step: count(1121), step_time 1412.17, mean_step_time 1473.72, it/s 0.68
  5447. [0130 17:56:57 @multigpu.py:323] [p0115] step: count(1107), step_time 1409.18, mean_step_time 1445.67, it/s 0.69
  5448. [0130 17:56:57 @multigpu.py:323] [p0576] step: count(1131), step_time 1397.04, mean_step_time 1451.85, it/s 0.69
  5449. [0130 17:56:58 @multigpu.py:323] [p0574] step: count(1122), step_time 1509.22, mean_step_time 1477.5, it/s 0.68
  5450. [0130 17:57:00 @multigpu.py:323] [p0574] step: count(1123), step_time 1393.09, mean_step_time 1471.51, it/s 0.68
  5451. [0130 17:57:00 @multigpu.py:323] [p0115] step: count(1108), step_time 2220.94, mean_step_time 1471.64, it/s 0.68
  5452. [0130 17:57:00 @multigpu.py:323] [p0576] step: count(1132), step_time 2216.0, mean_step_time 1471.52, it/s 0.68
  5453. [0130 17:57:01 @multigpu.py:323] [p0115] step: count(1109), step_time 1432.01, mean_step_time 1470.62, it/s 0.68
  5454. [0130 17:57:01 @multigpu.py:323] [p0576] step: count(1133), step_time 1447.93, mean_step_time 1470.58, it/s 0.68
  5455. [0130 17:57:01 @multigpu.py:323] [p0574] step: count(1124), step_time 1464.41, mean_step_time 1468.09, it/s 0.68
  5456. [0130 17:57:02 @multigpu.py:323] [p0574] step: count(1125), step_time 1372.86, mean_step_time 1463.66, it/s 0.68
  5457. [0130 17:57:02 @multigpu.py:323] [p0115] step: count(1110), step_time 1438.77, mean_step_time 1473.34, it/s 0.68
  5458. [0130 17:57:02 @multigpu.py:323] [p0576] step: count(1134), step_time 1478.64, mean_step_time 1472.95, it/s 0.68
  5459. [0130 17:57:04 @multigpu.py:323] [p0115] step: count(1111), step_time 1376.28, mean_step_time 1470.66, it/s 0.68
  5460. [0130 17:57:04 @multigpu.py:323] [p0574] step: count(1126), step_time 1469.52, mean_step_time 1462.75, it/s 0.68
  5461. [0130 17:57:04 @multigpu.py:323] [p0576] step: count(1135), step_time 1400.41, mean_step_time 1472.81, it/s 0.68
  5462. sending to address tcp://p0112:61216
  5463. ##### Sending to neptune: online_score : 0.451079662442 , 2.5 #####
  5464. [u'online', 2.5]
  5465. ##### Sending to neptune: active_workers : 0.451079749399 , 3 #####
  5466. receiving
  5467. [0130 17:57:05 @multigpu.py:323] [p0574] step: count(1127), step_time 1377.8, mean_step_time 1462.32, it/s 0.68
  5468. [0130 17:57:05 @multigpu.py:323] [p0115] step: count(1112), step_time 1473.36, mean_step_time 1471.92, it/s 0.68
  5469. [0130 17:57:05 @multigpu.py:323] [p0576] step: count(1136), step_time 1422.99, mean_step_time 1472.11, it/s 0.68
  5470. sending to address tcp://p0112:61216
  5471. ##### Sending to neptune: online_score : 0.451481641663 , 1.4 #####
  5472. [u'online', 1.4]
  5473. receiving
  5474. [0130 17:57:07 @multigpu.py:323] [p0574] step: count(1128), step_time 1433.09, mean_step_time 1458.19, it/s 0.69
  5475. [0130 17:57:07 @multigpu.py:323] [p0115] step: count(1113), step_time 1415.25, mean_step_time 1472.6, it/s 0.68
  5476. [0130 17:57:07 @multigpu.py:323] [p0576] step: count(1137), step_time 1427.53, mean_step_time 1472.22, it/s 0.68
  5477. [0130 17:57:08 @multigpu.py:323] [p0574] step: count(1129), step_time 1434.12, mean_step_time 1458.84, it/s 0.69
  5478. [0130 17:57:08 @multigpu.py:323] [p0576] step: count(1138), step_time 1416.3, mean_step_time 1470.62, it/s 0.68
  5479. [0130 17:57:08 @multigpu.py:323] [p0115] step: count(1114), step_time 1460.82, mean_step_time 1472.78, it/s 0.68
  5480. [0130 17:57:09 @multigpu.py:323] [p0574] step: count(1130), step_time 1433.63, mean_step_time 1457.43, it/s 0.69
  5481. [0130 17:57:10 @multigpu.py:323] [p0576] step: count(1139), step_time 1407.76, mean_step_time 1467.19, it/s 0.68
  5482. [0130 17:57:10 @multigpu.py:323] [p0115] step: count(1115), step_time 1432.65, mean_step_time 1472.5, it/s 0.68
  5483. [0130 17:57:11 @multigpu.py:323] [p0574] step: count(1131), step_time 1405.0, mean_step_time 1455.09, it/s 0.69
  5484. [0130 17:57:11 @multigpu.py:323] [p0576] step: count(1140), step_time 1421.67, mean_step_time 1463.47, it/s 0.68
  5485. [0130 17:57:11 @multigpu.py:323] [p0115] step: count(1116), step_time 1431.55, mean_step_time 1472.23, it/s 0.68
  5486. [0130 17:57:12 @multigpu.py:323] [p0574] step: count(1132), step_time 1475.83, mean_step_time 1454.85, it/s 0.69
  5487. [0130 17:57:12 @multigpu.py:323] [p0576] step: count(1141), step_time 1445.96, mean_step_time 1464.51, it/s 0.68
  5488. [0130 17:57:12 @multigpu.py:323] [p0115] step: count(1117), step_time 1491.65, mean_step_time 1475.26, it/s 0.68
  5489. [0130 17:57:14 @multigpu.py:323] [p0574] step: count(1133), step_time 1395.66, mean_step_time 1451.27, it/s 0.69
  5490. [0130 17:57:14 @multigpu.py:323] [p0576] step: count(1142), step_time 1430.69, mean_step_time 1464.54, it/s 0.68
  5491. [0130 17:57:14 @multigpu.py:323] [p0115] step: count(1118), step_time 1422.91, mean_step_time 1475.46, it/s 0.68
  5492. [0130 17:57:15 @multigpu.py:323] [p0576] step: count(1143), step_time 1401.77, mean_step_time 1464.68, it/s 0.68
  5493. [0130 17:57:15 @multigpu.py:323] [p0574] step: count(1134), step_time 1496.56, mean_step_time 1447.69, it/s 0.69
  5494. [0130 17:57:15 @multigpu.py:323] [p0115] step: count(1119), step_time 1423.35, mean_step_time 1473.6, it/s 0.68
  5495. [0130 17:57:17 @multigpu.py:323] [p0576] step: count(1144), step_time 1392.55, mean_step_time 1463.65, it/s 0.68
  5496. [0130 17:57:17 @multigpu.py:323] [p0574] step: count(1135), step_time 1422.67, mean_step_time 1443.38, it/s 0.69
  5497. [0130 17:57:17 @multigpu.py:323] [p0115] step: count(1120), step_time 1431.39, mean_step_time 1473.06, it/s 0.68
  5498. sending to address tcp://p0112:61216
  5499. ##### Sending to neptune: online_score : 0.454599559704 , 2.2 #####
  5500. [u'online', 2.2]
  5501. receiving
  5502. [0130 17:57:18 @multigpu.py:323] [p0576] step: count(1145), step_time 1391.93, mean_step_time 1461.3, it/s 0.68
  5503. [0130 17:57:18 @multigpu.py:323] [p0115] step: count(1121), step_time 1430.98, mean_step_time 1471.55, it/s 0.68
  5504. [0130 17:57:18 @multigpu.py:323] [p0574] step: count(1136), step_time 1487.91, mean_step_time 1443.52, it/s 0.69
  5505. [0130 17:57:20 @multigpu.py:323] [p0576] step: count(1146), step_time 1514.51, mean_step_time 1466.33, it/s 0.68
  5506. [0130 17:57:20 @multigpu.py:323] [p0574] step: count(1137), step_time 1420.15, mean_step_time 1435.69, it/s 0.7
  5507. [0130 17:57:20 @multigpu.py:323] [p0115] step: count(1122), step_time 1458.52, mean_step_time 1472.11, it/s 0.68
  5508. [0130 17:57:21 @multigpu.py:323] [p0576] step: count(1147), step_time 1380.34, mean_step_time 1464.57, it/s 0.68
  5509. [0130 17:57:21 @multigpu.py:323] [p0574] step: count(1138), step_time 1355.25, mean_step_time 1431.03, it/s 0.7
  5510. [0130 17:57:21 @multigpu.py:323] [p0115] step: count(1123), step_time 1428.35, mean_step_time 1473.34, it/s 0.68
  5511. [0130 17:57:22 @multigpu.py:323] [p0576] step: count(1148), step_time 1437.57, mean_step_time 1463.68, it/s 0.68
  5512. [0130 17:57:22 @multigpu.py:323] [p0574] step: count(1139), step_time 1437.97, mean_step_time 1432.67, it/s 0.7
  5513. [0130 17:57:22 @multigpu.py:323] [p0115] step: count(1124), step_time 1431.56, mean_step_time 1474.11, it/s 0.68
  5514. [0130 17:57:24 @multigpu.py:323] [p0576] step: count(1149), step_time 1432.13, mean_step_time 1463.93, it/s 0.68
  5515. [0130 17:57:24 @multigpu.py:323] [p0574] step: count(1140), step_time 1433.31, mean_step_time 1431.51, it/s 0.7
  5516. [0130 17:57:24 @multigpu.py:323] [p0115] step: count(1125), step_time 1474.25, mean_step_time 1476.11, it/s 0.68
  5517. [0130 17:57:25 @multigpu.py:323] [p0576] step: count(1150), step_time 1435.1, mean_step_time 1464.94, it/s 0.68
  5518. [0130 17:57:25 @multigpu.py:323] [p0574] step: count(1141), step_time 1379.84, mean_step_time 1429.89, it/s 0.7
  5519. [0130 17:57:25 @multigpu.py:323] [p0115] step: count(1126), step_time 1453.67, mean_step_time 1476.87, it/s 0.68
  5520. [0130 17:57:27 @multigpu.py:323] [p0576] step: count(1151), step_time 1389.31, mean_step_time 1464.55, it/s 0.68
  5521. [0130 17:57:27 @multigpu.py:323] [p0574] step: count(1142), step_time 1416.3, mean_step_time 1425.25, it/s 0.7
  5522. [0130 17:57:27 @multigpu.py:323] [p0115] step: count(1127), step_time 1405.31, mean_step_time 1476.68, it/s 0.68
  5523. [0130 17:57:28 @multigpu.py:323] [p0115] step: count(1128), step_time 1483.4, mean_step_time 1439.8, it/s 0.69
  5524. [0130 17:57:28 @multigpu.py:323] [p0574] step: count(1143), step_time 1683.21, mean_step_time 1439.75, it/s 0.69
  5525. [0130 17:57:28 @multigpu.py:323] [p0576] step: count(1152), step_time 1719.88, mean_step_time 1439.75, it/s 0.69
  5526. [0130 17:57:30 @multigpu.py:323] [p0115] step: count(1129), step_time 1378.47, mean_step_time 1437.12, it/s 0.7
  5527. [0130 17:57:30 @multigpu.py:323] [p0576] step: count(1153), step_time 1401.52, mean_step_time 1437.43, it/s 0.7
  5528. [0130 17:57:30 @multigpu.py:323] [p0574] step: count(1144), step_time 1404.0, mean_step_time 1436.73, it/s 0.7
  5529. [0130 17:57:31 @multigpu.py:323] [p0115] step: count(1130), step_time 1440.34, mean_step_time 1437.2, it/s 0.7
  5530. [0130 17:57:31 @multigpu.py:323] [p0574] step: count(1145), step_time 1415.34, mean_step_time 1438.86, it/s 0.69
  5531. [0130 17:57:31 @multigpu.py:323] [p0576] step: count(1154), step_time 1429.49, mean_step_time 1434.97, it/s 0.7
  5532. [0130 17:57:33 @multigpu.py:323] [p0576] step: count(1155), step_time 1406.06, mean_step_time 1435.25, it/s 0.7
  5533. [0130 17:57:33 @multigpu.py:323] [p0574] step: count(1146), step_time 1434.59, mean_step_time 1437.11, it/s 0.7
  5534. [0130 17:57:33 @multigpu.py:323] [p0115] step: count(1131), step_time 1466.16, mean_step_time 1441.7, it/s 0.69
  5535. sending to address tcp://p0112:61216
  5536. ##### Sending to neptune: online_score : 0.459099557201 , 2.4 #####
  5537. [u'online', 2.4]
  5538. receiving
  5539. [0130 17:57:34 @multigpu.py:323] [p0115] step: count(1132), step_time 1373.58, mean_step_time 1436.71, it/s 0.7
  5540. [0130 17:57:34 @multigpu.py:323] [p0576] step: count(1156), step_time 1426.12, mean_step_time 1435.41, it/s 0.7
  5541. [0130 17:57:34 @multigpu.py:323] [p0574] step: count(1147), step_time 1435.5, mean_step_time 1440.0, it/s 0.69
  5542. sending to address tcp://p0112:61216
  5543. ##### Sending to neptune: online_score : 0.459456433588 , 1.6 #####
  5544. [u'online', 1.6]
  5545. receiving
  5546. [0130 17:57:35 @multigpu.py:323] [p0574] step: count(1148), step_time 1371.35, mean_step_time 1436.91, it/s 0.7
  5547. [0130 17:57:35 @multigpu.py:323] [p0576] step: count(1157), step_time 1404.18, mean_step_time 1434.24, it/s 0.7
  5548. [0130 17:57:35 @multigpu.py:323] [p0115] step: count(1133), step_time 1432.68, mean_step_time 1437.58, it/s 0.7
  5549. [0130 17:57:37 @multigpu.py:323] [p0574] step: count(1149), step_time 1419.67, mean_step_time 1436.19, it/s 0.7
  5550. [0130 17:57:37 @multigpu.py:323] [p0115] step: count(1134), step_time 1433.33, mean_step_time 1436.21, it/s 0.7
  5551. [0130 17:57:37 @multigpu.py:323] [p0576] step: count(1158), step_time 1456.34, mean_step_time 1436.24, it/s 0.7
  5552. [0130 17:57:38 @multigpu.py:323] [p0574] step: count(1150), step_time 1417.75, mean_step_time 1435.39, it/s 0.7
  5553. [0130 17:57:38 @multigpu.py:323] [p0115] step: count(1135), step_time 1424.22, mean_step_time 1435.78, it/s 0.7
  5554. [0130 17:57:38 @multigpu.py:323] [p0576] step: count(1159), step_time 1440.73, mean_step_time 1437.89, it/s 0.7
  5555. [0130 17:57:40 @multigpu.py:323] [p0574] step: count(1151), step_time 1422.46, mean_step_time 1436.27, it/s 0.7
  5556. [0130 17:57:40 @multigpu.py:323] [p0115] step: count(1136), step_time 1428.24, mean_step_time 1435.62, it/s 0.7
  5557. [0130 17:57:40 @multigpu.py:323] [p0576] step: count(1160), step_time 1450.56, mean_step_time 1439.34, it/s 0.69
  5558. [0130 17:57:41 @multigpu.py:323] [p0576] step: count(1161), step_time 1392.22, mean_step_time 1436.65, it/s 0.7
  5559. [0130 17:57:41 @multigpu.py:323] [p0574] step: count(1152), step_time 1491.29, mean_step_time 1437.04, it/s 0.7
  5560. [0130 17:57:41 @multigpu.py:323] [p0115] step: count(1137), step_time 1469.56, mean_step_time 1434.51, it/s 0.7
  5561. [0130 17:57:43 @multigpu.py:323] [p0574] step: count(1153), step_time 1377.57, mean_step_time 1436.13, it/s 0.7
  5562. [0130 17:57:43 @multigpu.py:323] [p0576] step: count(1162), step_time 1424.6, mean_step_time 1436.35, it/s 0.7
  5563. [0130 17:57:43 @multigpu.py:323] [p0115] step: count(1138), step_time 1468.64, mean_step_time 1436.8, it/s 0.7
  5564. [0130 17:57:44 @multigpu.py:323] [p0574] step: count(1154), step_time 1434.21, mean_step_time 1433.02, it/s 0.7
  5565. [0130 17:57:44 @multigpu.py:323] [p0576] step: count(1163), step_time 1407.8, mean_step_time 1436.65, it/s 0.7
  5566. [0130 17:57:44 @multigpu.py:323] [p0115] step: count(1139), step_time 1388.54, mean_step_time 1435.06, it/s 0.7
  5567. [0130 17:57:45 @multigpu.py:323] [p0574] step: count(1155), step_time 1405.23, mean_step_time 1432.14, it/s 0.7
  5568. [0130 17:57:45 @multigpu.py:323] [p0576] step: count(1164), step_time 1430.82, mean_step_time 1438.56, it/s 0.7
  5569. [0130 17:57:46 @multigpu.py:323] [p0115] step: count(1140), step_time 1482.05, mean_step_time 1437.59, it/s 0.7
  5570. [0130 17:57:47 @multigpu.py:323] [p0574] step: count(1156), step_time 1441.58, mean_step_time 1429.83, it/s 0.7
  5571. [0130 17:57:47 @multigpu.py:323] [p0576] step: count(1165), step_time 1415.85, mean_step_time 1439.76, it/s 0.69
  5572. [0130 17:57:47 @multigpu.py:323] [p0115] step: count(1141), step_time 1376.95, mean_step_time 1434.89, it/s 0.7
  5573. sending to address tcp://p0112:61216
  5574. ##### Sending to neptune: online_score : 0.462921938035 , 1.9 #####
  5575. [u'online', 1.9]
  5576. receiving
  5577. [0130 17:57:48 @multigpu.py:323] [p0576] step: count(1166), step_time 1386.7, mean_step_time 1433.36, it/s 0.7
  5578. [0130 17:57:48 @multigpu.py:323] [p0574] step: count(1157), step_time 1514.69, mean_step_time 1434.56, it/s 0.7
  5579. [0130 17:57:48 @multigpu.py:323] [p0115] step: count(1142), step_time 1424.27, mean_step_time 1433.18, it/s 0.7
  5580. [0130 17:57:50 @multigpu.py:323] [p0576] step: count(1167), step_time 1497.67, mean_step_time 1439.23, it/s 0.69
  5581. [0130 17:57:50 @multigpu.py:323] [p0574] step: count(1158), step_time 1395.61, mean_step_time 1436.57, it/s 0.7
  5582. [0130 17:57:50 @multigpu.py:323] [p0115] step: count(1143), step_time 1457.88, mean_step_time 1434.66, it/s 0.7
  5583. [0130 17:57:51 @multigpu.py:323] [p0576] step: count(1168), step_time 1398.13, mean_step_time 1437.26, it/s 0.7
  5584. [0130 17:57:51 @multigpu.py:323] [p0574] step: count(1159), step_time 1416.04, mean_step_time 1435.48, it/s 0.7
  5585. [0130 17:57:51 @multigpu.py:323] [p0115] step: count(1144), step_time 1414.34, mean_step_time 1433.79, it/s 0.7
  5586. [0130 17:57:53 @multigpu.py:323] [p0576] step: count(1169), step_time 1466.74, mean_step_time 1438.99, it/s 0.69
  5587. [0130 17:57:53 @multigpu.py:323] [p0574] step: count(1160), step_time 1441.19, mean_step_time 1435.87, it/s 0.7
  5588. [0130 17:57:53 @multigpu.py:323] [p0115] step: count(1145), step_time 1470.76, mean_step_time 1433.62, it/s 0.7
  5589. [0130 17:57:54 @multigpu.py:323] [p0574] step: count(1161), step_time 1391.38, mean_step_time 1436.45, it/s 0.7
  5590. [0130 17:57:54 @multigpu.py:323] [p0576] step: count(1170), step_time 1394.21, mean_step_time 1436.95, it/s 0.7
  5591. [0130 17:57:54 @multigpu.py:323] [p0115] step: count(1146), step_time 1397.14, mean_step_time 1430.79, it/s 0.7
  5592. [0130 17:57:55 @multigpu.py:323] [p0574] step: count(1162), step_time 1442.57, mean_step_time 1437.76, it/s 0.7
  5593. [0130 17:57:55 @multigpu.py:323] [p0576] step: count(1171), step_time 1501.38, mean_step_time 1442.55, it/s 0.69
  5594. [0130 17:57:55 @multigpu.py:323] [p0115] step: count(1147), step_time 1421.85, mean_step_time 1431.62, it/s 0.7
  5595. [0130 17:57:57 @multigpu.py:323] [p0115] step: count(1148), step_time 1466.54, mean_step_time 1430.78, it/s 0.7
  5596. [0130 17:57:57 @multigpu.py:323] [p0576] step: count(1172), step_time 1483.5, mean_step_time 1430.73, it/s 0.7
  5597. [0130 17:57:57 @multigpu.py:323] [p0574] step: count(1163), step_time 1542.55, mean_step_time 1430.73, it/s 0.7
  5598. [0130 17:57:58 @multigpu.py:323] [p0115] step: count(1149), step_time 1364.25, mean_step_time 1430.07, it/s 0.7
  5599. [0130 17:57:58 @multigpu.py:323] [p0574] step: count(1164), step_time 1429.55, mean_step_time 1432.01, it/s 0.7
  5600. [0130 17:57:58 @multigpu.py:323] [p0576] step: count(1173), step_time 1446.76, mean_step_time 1432.99, it/s 0.7
  5601. sending to address tcp://p0112:61216
  5602. ##### Sending to neptune: online_score : 0.466300260226 , 1.6 #####
  5603. [u'online', 1.6]
  5604. receiving
  5605. [0130 17:58:00 @multigpu.py:323] [p0115] step: count(1150), step_time 1485.05, mean_step_time 1432.3, it/s 0.7
  5606. [0130 17:58:00 @multigpu.py:323] [p0576] step: count(1174), step_time 1423.04, mean_step_time 1432.67, it/s 0.7
  5607. [0130 17:58:00 @multigpu.py:323] [p0574] step: count(1165), step_time 1455.67, mean_step_time 1434.02, it/s 0.7
  5608. [0130 17:58:01 @multigpu.py:323] [p0574] step: count(1166), step_time 1392.81, mean_step_time 1431.93, it/s 0.7
  5609. [0130 17:58:01 @multigpu.py:323] [p0115] step: count(1151), step_time 1451.4, mean_step_time 1431.56, it/s 0.7
  5610. [0130 17:58:01 @multigpu.py:323] [p0576] step: count(1175), step_time 1432.38, mean_step_time 1433.99, it/s 0.7
  5611. [0130 17:58:03 @multigpu.py:323] [p0574] step: count(1167), step_time 1418.52, mean_step_time 1431.09, it/s 0.7
  5612. [0130 17:58:03 @multigpu.py:323] [p0115] step: count(1152), step_time 1432.02, mean_step_time 1434.49, it/s 0.7
  5613. [0130 17:58:03 @multigpu.py:323] [p0576] step: count(1176), step_time 1450.0, mean_step_time 1435.18, it/s 0.7
  5614. [0130 17:58:04 @multigpu.py:323] [p0574] step: count(1168), step_time 1430.0, mean_step_time 1434.02, it/s 0.7
  5615. [0130 17:58:04 @multigpu.py:323] [p0115] step: count(1153), step_time 1400.37, mean_step_time 1432.87, it/s 0.7
  5616. [0130 17:58:04 @multigpu.py:323] [p0576] step: count(1177), step_time 1410.86, mean_step_time 1435.51, it/s 0.7
  5617. [0130 17:58:06 @multigpu.py:323] [p0574] step: count(1169), step_time 1430.97, mean_step_time 1434.58, it/s 0.7
  5618. [0130 17:58:06 @multigpu.py:323] [p0576] step: count(1178), step_time 1409.64, mean_step_time 1433.18, it/s 0.7
  5619. [0130 17:58:06 @multigpu.py:323] [p0115] step: count(1154), step_time 1457.51, mean_step_time 1434.08, it/s 0.7
  5620. [0130 17:58:07 @multigpu.py:323] [p0574] step: count(1170), step_time 1406.46, mean_step_time 1434.02, it/s 0.7
  5621. [0130 17:58:07 @multigpu.py:323] [p0576] step: count(1179), step_time 1408.16, mean_step_time 1431.55, it/s 0.7
  5622. [0130 17:58:07 @multigpu.py:323] [p0115] step: count(1155), step_time 1438.62, mean_step_time 1434.8, it/s 0.7
  5623. [0130 17:58:08 @multigpu.py:323] [p0576] step: count(1180), step_time 1363.05, mean_step_time 1427.18, it/s 0.7
  5624. [0130 17:58:08 @multigpu.py:323] [p0574] step: count(1171), step_time 1409.52, mean_step_time 1433.37, it/s 0.7
  5625. [0130 17:58:08 @multigpu.py:323] [p0115] step: count(1156), step_time 1448.35, mean_step_time 1435.81, it/s 0.7
  5626. sending to address tcp://p0112:61216
  5627. ##### Sending to neptune: online_score : 0.468956205514 , 1.7 #####
  5628. [u'online', 1.7]
  5629. ##### Sending to neptune: active_workers : 0.468956336644 , 3 #####
  5630. receiving
  5631. [0130 17:58:10 @multigpu.py:323] [p0574] step: count(1172), step_time 1419.6, mean_step_time 1429.79, it/s 0.7
  5632. [0130 17:58:10 @multigpu.py:323] [p0576] step: count(1181), step_time 1521.66, mean_step_time 1433.65, it/s 0.7
  5633. [0130 17:58:10 @multigpu.py:323] [p0115] step: count(1157), step_time 1429.55, mean_step_time 1433.81, it/s 0.7
  5634. [0130 17:58:11 @multigpu.py:323] [p0574] step: count(1173), step_time 1444.48, mean_step_time 1433.13, it/s 0.7
  5635. [0130 17:58:11 @multigpu.py:323] [p0115] step: count(1158), step_time 1404.46, mean_step_time 1430.6, it/s 0.7
  5636. [0130 17:58:11 @multigpu.py:323] [p0576] step: count(1182), step_time 1444.9, mean_step_time 1434.66, it/s 0.7
  5637. [0130 17:58:13 @multigpu.py:323] [p0574] step: count(1174), step_time 1458.27, mean_step_time 1434.33, it/s 0.7
  5638. [0130 17:58:13 @multigpu.py:323] [p0576] step: count(1183), step_time 1409.56, mean_step_time 1434.75, it/s 0.7
  5639. [0130 17:58:13 @multigpu.py:323] [p0115] step: count(1159), step_time 1438.83, mean_step_time 1433.11, it/s 0.7
  5640. [0130 17:58:14 @multigpu.py:323] [p0574] step: count(1175), step_time 1425.11, mean_step_time 1435.33, it/s 0.7
  5641. [0130 17:58:14 @multigpu.py:323] [p0576] step: count(1184), step_time 1405.42, mean_step_time 1433.48, it/s 0.7
  5642. [0130 17:58:14 @multigpu.py:323] [p0115] step: count(1160), step_time 1442.89, mean_step_time 1431.15, it/s 0.7
  5643. [0130 17:58:15 @multigpu.py:323] [p0576] step: count(1185), step_time 1369.21, mean_step_time 1431.15, it/s 0.7
  5644. [0130 17:58:16 @multigpu.py:323] [p0574] step: count(1176), step_time 1445.12, mean_step_time 1435.51, it/s 0.7
  5645. [0130 17:58:16 @multigpu.py:323] [p0115] step: count(1161), step_time 1429.14, mean_step_time 1433.76, it/s 0.7
  5646. [0130 17:58:17 @multigpu.py:323] [p0576] step: count(1186), step_time 1430.49, mean_step_time 1433.34, it/s 0.7
  5647. [0130 17:58:17 @multigpu.py:323] [p0574] step: count(1177), step_time 1413.46, mean_step_time 1430.44, it/s 0.7
  5648. [0130 17:58:17 @multigpu.py:323] [p0115] step: count(1162), step_time 1433.96, mean_step_time 1434.25, it/s 0.7
  5649. [0130 17:58:18 @multigpu.py:323] [p0576] step: count(1187), step_time 1365.08, mean_step_time 1426.71, it/s 0.7
  5650. [0130 17:58:18 @multigpu.py:323] [p0574] step: count(1178), step_time 1520.99, mean_step_time 1436.71, it/s 0.7
  5651. [0130 17:58:18 @multigpu.py:323] [p0115] step: count(1163), step_time 1451.15, mean_step_time 1433.91, it/s 0.7
  5652. [0130 17:58:20 @multigpu.py:323] [p0576] step: count(1188), step_time 1456.22, mean_step_time 1429.61, it/s 0.7
  5653. [0130 17:58:20 @multigpu.py:323] [p0574] step: count(1179), step_time 1405.58, mean_step_time 1436.19, it/s 0.7
  5654. [0130 17:58:20 @multigpu.py:323] [p0115] step: count(1164), step_time 1411.96, mean_step_time 1433.79, it/s 0.7
  5655. [0130 17:58:21 @multigpu.py:323] [p0576] step: count(1189), step_time 1377.94, mean_step_time 1425.17, it/s 0.7
  5656. [0130 17:58:21 @multigpu.py:323] [p0115] step: count(1165), step_time 1390.93, mean_step_time 1429.8, it/s 0.7
  5657. [0130 17:58:21 @multigpu.py:323] [p0574] step: count(1180), step_time 1425.96, mean_step_time 1435.43, it/s 0.7
  5658. [0130 17:58:22 @multigpu.py:323] [p0576] step: count(1190), step_time 1423.47, mean_step_time 1426.64, it/s 0.7
  5659. [0130 17:58:23 @multigpu.py:323] [p0115] step: count(1166), step_time 1432.86, mean_step_time 1431.59, it/s 0.7
  5660. [0130 17:58:23 @multigpu.py:323] [p0574] step: count(1181), step_time 1417.27, mean_step_time 1436.72, it/s 0.7
  5661. [0130 17:58:24 @multigpu.py:323] [p0576] step: count(1191), step_time 1413.47, mean_step_time 1422.24, it/s 0.7
  5662. sending to address tcp://p0112:61216
  5663. ##### Sending to neptune: online_score : 0.473143153853 , 2.2 #####
  5664. [u'online', 2.2]
  5665. receiving
  5666. [0130 17:58:24 @multigpu.py:323] [p0115] step: count(1167), step_time 1428.48, mean_step_time 1431.92, it/s 0.7
  5667. [0130 17:58:24 @multigpu.py:323] [p0574] step: count(1182), step_time 1495.99, mean_step_time 1439.39, it/s 0.69
  5668. [0130 17:58:26 @multigpu.py:323] [p0115] step: count(1168), step_time 1556.94, mean_step_time 1436.44, it/s 0.7
  5669. [0130 17:58:26 @multigpu.py:323] [p0574] step: count(1183), step_time 1481.8, mean_step_time 1436.36, it/s 0.7
  5670. [0130 17:58:26 @multigpu.py:323] [p0576] step: count(1192), step_time 1767.82, mean_step_time 1436.46, it/s 0.7
  5671. [0130 17:58:27 @multigpu.py:323] [p0574] step: count(1184), step_time 1398.17, mean_step_time 1434.79, it/s 0.7
  5672. [0130 17:58:27 @multigpu.py:323] [p0115] step: count(1169), step_time 1428.32, mean_step_time 1439.64, it/s 0.69
  5673. [0130 17:58:27 @multigpu.py:323] [p0576] step: count(1193), step_time 1467.1, mean_step_time 1437.47, it/s 0.7
  5674. [0130 17:58:29 @multigpu.py:323] [p0574] step: count(1185), step_time 1440.13, mean_step_time 1434.01, it/s 0.7
  5675. [0130 17:58:29 @multigpu.py:323] [p0576] step: count(1194), step_time 1377.48, mean_step_time 1435.2, it/s 0.7
  5676. [0130 17:58:29 @multigpu.py:323] [p0115] step: count(1170), step_time 1434.16, mean_step_time 1437.1, it/s 0.7
  5677. sending to address tcp://p0112:61216
  5678. ##### Sending to neptune: online_score : 0.474509788288 , 2.0 #####
  5679. [u'online', 2.0]
  5680. receiving
  5681. [0130 17:58:30 @multigpu.py:323] [p0574] step: count(1186), step_time 1390.93, mean_step_time 1433.92, it/s 0.7
  5682. [0130 17:58:30 @multigpu.py:323] [p0576] step: count(1195), step_time 1464.69, mean_step_time 1436.81, it/s 0.7
  5683. [0130 17:58:30 @multigpu.py:323] [p0115] step: count(1171), step_time 1453.03, mean_step_time 1437.18, it/s 0.7
  5684. [0130 17:58:31 @multigpu.py:323] [p0574] step: count(1187), step_time 1418.75, mean_step_time 1433.93, it/s 0.7
  5685. [0130 17:58:31 @multigpu.py:323] [p0115] step: count(1172), step_time 1395.88, mean_step_time 1435.37, it/s 0.7
  5686. [0130 17:58:31 @multigpu.py:323] [p0576] step: count(1196), step_time 1400.67, mean_step_time 1434.35, it/s 0.7
  5687. [0130 17:58:33 @multigpu.py:323] [p0574] step: count(1188), step_time 1443.55, mean_step_time 1434.61, it/s 0.7
  5688. [0130 17:58:33 @multigpu.py:323] [p0576] step: count(1197), step_time 1418.83, mean_step_time 1434.74, it/s 0.7
  5689. [0130 17:58:33 @multigpu.py:323] [p0115] step: count(1173), step_time 1432.91, mean_step_time 1437.0, it/s 0.7
  5690. sending to address tcp://p0112:61216
  5691. ##### Sending to neptune: online_score : 0.475631449156 , 1.7 #####
  5692. [u'online', 1.7]
  5693. receiving
  5694. [0130 17:58:34 @multigpu.py:323] [p0574] step: count(1189), step_time 1408.25, mean_step_time 1433.47, it/s 0.7
  5695. [0130 17:58:34 @multigpu.py:323] [p0115] step: count(1174), step_time 1381.5, mean_step_time 1433.2, it/s 0.7
  5696. [0130 17:58:34 @multigpu.py:323] [p0576] step: count(1198), step_time 1453.78, mean_step_time 1436.95, it/s 0.7
  5697. [0130 17:58:36 @multigpu.py:323] [p0574] step: count(1190), step_time 1429.62, mean_step_time 1434.63, it/s 0.7
  5698. [0130 17:58:36 @multigpu.py:323] [p0115] step: count(1175), step_time 1427.4, mean_step_time 1432.64, it/s 0.7
  5699. [0130 17:58:36 @multigpu.py:323] [p0576] step: count(1199), step_time 1442.31, mean_step_time 1438.66, it/s 0.7
  5700. [0130 17:58:37 @multigpu.py:323] [p0115] step: count(1176), step_time 1407.48, mean_step_time 1430.59, it/s 0.7
  5701. [0130 17:58:37 @multigpu.py:323] [p0576] step: count(1200), step_time 1383.31, mean_step_time 1439.67, it/s 0.69
  5702. sending debugging info...
  5703. sending to address tcp://p0112:61216
  5704. ##### Sending to neptune: mean_delay : 0.476802181337 , 0.0 #####
  5705. sending to address tcp://p0112:61216
  5706. ##### Sending to neptune: max_delay : 0.476802181337 , -0.0 #####
  5707. ##### Sending to neptune: min_delay : 0.476802181337 , -0.0 #####
  5708. [u'delays', [0.0, -0.0, -0.0]]
  5709. receiving
  5710. ##### Sending to neptune: cost : 0.476802654399 , -0.00253472337499 #####
  5711. ##### Sending to neptune: policy_loss : 0.476802654399 , 0.375536680222 #####
  5712. sending to address tcp://p0112:61216
  5713. ##### Sending to neptune: xentropy_loss : 0.476802654399 , -2.28917598724 #####
  5714. ##### Sending to neptune: value_loss : 0.476802654399 , 1.58919465542 #####
  5715. ##### Sending to neptune: advantage : 0.476802654399 , -0.00181525712833 #####
  5716. ##### Sending to neptune: pred_reward : 0.476802654399 , 0.453048050404 #####
  5717. ##### Sending to neptune: max_logit : 0.476802654399 , 0.191985800862 #####
  5718. [u'loss', -0.002534723374992609, 0.3755366802215576, -2.2891759872436523, 1.589194655418396, -0.001815257128328085, 0.45304805040359497, 0.19198580086231232]
  5719. receiving
  5720. ##### Sending to neptune: active_relus : 0.476803139382 , 9613262.34 #####
  5721. ##### Sending to neptune: dp_per_s : 0.476803139382 , 88.6668697671 #####
  5722. [u'other', 9613262.34, 88.66686976706502]
  5723. receiving
  5724. [0130 17:58:37 @multigpu.py:323] [p0574] step: count(1191), step_time 1487.49, mean_step_time 1438.53, it/s 0.7
  5725. [0130 17:58:38 @multigpu.py:323] [p0115] step: count(1177), step_time 1448.55, mean_step_time 1431.54, it/s 0.7
  5726. [0130 17:58:39 @multigpu.py:323] [p0576] step: count(1201), step_time 1473.19, mean_step_time 1437.25, it/s 0.7
  5727. [0130 17:58:39 @multigpu.py:323] [p0574] step: count(1192), step_time 1479.83, mean_step_time 1441.54, it/s 0.69
  5728. [0130 17:58:40 @multigpu.py:323] [p0115] step: count(1178), step_time 1453.8, mean_step_time 1434.01, it/s 0.7
  5729. [0130 17:58:40 @multigpu.py:323] [p0574] step: count(1193), step_time 1379.85, mean_step_time 1438.31, it/s 0.7
  5730. [0130 17:58:40 @multigpu.py:323] [p0576] step: count(1202), step_time 1401.42, mean_step_time 1435.07, it/s 0.7
  5731. [0130 17:58:41 @multigpu.py:323] [p0115] step: count(1179), step_time 1398.57, mean_step_time 1432.0, it/s 0.7
  5732. [0130 17:58:41 @multigpu.py:323] [p0574] step: count(1194), step_time 1400.66, mean_step_time 1435.43, it/s 0.7
  5733. [0130 17:58:41 @multigpu.py:323] [p0576] step: count(1203), step_time 1400.03, mean_step_time 1434.6, it/s 0.7
  5734. [0130 17:58:43 @multigpu.py:323] [p0576] step: count(1204), step_time 1419.88, mean_step_time 1435.32, it/s 0.7
  5735. [0130 17:58:43 @multigpu.py:323] [p0574] step: count(1195), step_time 1470.39, mean_step_time 1437.69, it/s 0.7
  5736. [0130 17:58:43 @multigpu.py:323] [p0115] step: count(1180), step_time 1514.03, mean_step_time 1435.55, it/s 0.7
  5737. [0130 17:58:44 @multigpu.py:323] [p0574] step: count(1196), step_time 1378.91, mean_step_time 1434.38, it/s 0.7
  5738. [0130 17:58:44 @multigpu.py:323] [p0576] step: count(1205), step_time 1442.53, mean_step_time 1438.99, it/s 0.69
  5739. [0130 17:58:44 @multigpu.py:323] [p0115] step: count(1181), step_time 1418.57, mean_step_time 1435.02, it/s 0.7
  5740. [0130 17:58:46 @multigpu.py:323] [p0574] step: count(1197), step_time 1424.19, mean_step_time 1434.92, it/s 0.7
  5741. [0130 17:58:46 @multigpu.py:323] [p0576] step: count(1206), step_time 1445.85, mean_step_time 1439.75, it/s 0.69
  5742. [0130 17:58:46 @multigpu.py:323] [p0115] step: count(1182), step_time 1449.86, mean_step_time 1435.82, it/s 0.7
  5743. [0130 17:58:47 @multigpu.py:323] [p0576] step: count(1207), step_time 1386.01, mean_step_time 1440.8, it/s 0.69
  5744. [0130 17:58:47 @multigpu.py:323] [p0574] step: count(1198), step_time 1435.24, mean_step_time 1430.63, it/s 0.7
  5745. [0130 17:58:47 @multigpu.py:323] [p0115] step: count(1183), step_time 1491.59, mean_step_time 1437.84, it/s 0.7
  5746. [0130 17:58:49 @multigpu.py:323] [p0576] step: count(1208), step_time 1456.2, mean_step_time 1440.8, it/s 0.69
  5747. [0130 17:58:49 @multigpu.py:323] [p0574] step: count(1199), step_time 1449.49, mean_step_time 1432.82, it/s 0.7
  5748. [0130 17:58:49 @multigpu.py:323] [p0115] step: count(1184), step_time 1419.96, mean_step_time 1438.24, it/s 0.7
  5749. [0130 17:58:50 @multigpu.py:323] [p0576] step: count(1209), step_time 1403.29, mean_step_time 1442.07, it/s 0.69
  5750. [0130 17:58:50 @multigpu.py:323] [p0574] step: count(1200), step_time 1416.98, mean_step_time 1432.38, it/s 0.7
  5751. sending debugging info...
  5752. sending to address tcp://p0112:61216
  5753. ##### Sending to neptune: mean_delay : 0.480370954143 , 0.0 #####
  5754. ##### Sending to neptune: max_delay : 0.480370954143 , -0.0 #####
  5755. sending to address tcp://p0112:61216
  5756. ##### Sending to neptune: min_delay : 0.480370954143 , -0.0 #####
  5757. [u'delays', [0.0, -0.0, -0.0]]
  5758. receiving
  5759. ##### Sending to neptune: cost : 0.480371489988 , -0.00687376549467 #####
  5760. sending to address tcp://p0112:61216
  5761. ##### Sending to neptune: policy_loss : 0.480371489988 , -0.0515709109604 #####
  5762. ##### Sending to neptune: xentropy_loss : 0.480371489988 , -2.28922891617 #####
  5763. ##### Sending to neptune: value_loss : 0.480371489988 , 1.46095776558 #####
  5764. ##### Sending to neptune: advantage : 0.480371489988 , 0.000297629856504 #####
  5765. ##### Sending to neptune: pred_reward : 0.480371489988 , 0.45008713007 #####
  5766. ##### Sending to neptune: max_logit : 0.480371489988 , 0.190380632877 #####
  5767. [u'loss', -0.006873765494674444, -0.05157091096043587, -2.289228916168213, 1.4609577655792236, 0.0002976298565044999, 0.45008713006973267, 0.19038063287734985]
  5768. receiving
  5769. ##### Sending to neptune: active_relus : 0.480371931063 , 9601806.97 #####
  5770. ##### Sending to neptune: dp_per_s : 0.480371931063 , 88.6732990532 #####
  5771. [u'other', 9601806.97, 88.6732990532418]
  5772. receiving
  5773. [0130 17:58:50 @multigpu.py:323] [p0115] step: count(1185), step_time 1475.1, mean_step_time 1442.45, it/s 0.69
  5774. [0130 17:58:51 @multigpu.py:323] [p0574] step: count(1201), step_time 1401.01, mean_step_time 1431.56, it/s 0.7
  5775. [0130 17:58:51 @multigpu.py:323] [p0576] step: count(1210), step_time 1431.24, mean_step_time 1442.46, it/s 0.69
  5776. [0130 17:58:52 @multigpu.py:323] [p0115] step: count(1186), step_time 1473.77, mean_step_time 1444.5, it/s 0.69
  5777. [0130 17:58:53 @multigpu.py:323] [p0574] step: count(1202), step_time 1393.14, mean_step_time 1426.42, it/s 0.7
  5778. [0130 17:58:53 @multigpu.py:323] [p0576] step: count(1211), step_time 1452.95, mean_step_time 1444.43, it/s 0.69
  5779. [0130 17:58:53 @multigpu.py:323] [p0115] step: count(1187), step_time 1401.6, mean_step_time 1443.15, it/s 0.69
  5780. [0130 17:58:54 @multigpu.py:323] [p0115] step: count(1188), step_time 1399.15, mean_step_time 1435.26, it/s 0.7
  5781. [0130 17:58:54 @multigpu.py:323] [p0574] step: count(1203), step_time 1659.28, mean_step_time 1435.29, it/s 0.7
  5782. [0130 17:58:54 @multigpu.py:323] [p0576] step: count(1212), step_time 1584.36, mean_step_time 1435.26, it/s 0.7
  5783. [0130 17:58:56 @multigpu.py:323] [p0576] step: count(1213), step_time 1403.86, mean_step_time 1432.09, it/s 0.7
  5784. sending to address tcp://p0112:61216
  5785. ##### Sending to neptune: online_score : 0.482001303567 , 2.1 #####
  5786. [u'online', 2.1]
  5787. receiving
  5788. [0130 17:58:56 @multigpu.py:323] [p0115] step: count(1189), step_time 1445.36, mean_step_time 1436.11, it/s 0.7
  5789. [0130 17:58:56 @multigpu.py:323] [p0574] step: count(1204), step_time 1458.93, mean_step_time 1438.33, it/s 0.7
  5790. [0130 17:58:57 @multigpu.py:323] [p0574] step: count(1205), step_time 1395.93, mean_step_time 1436.12, it/s 0.7
  5791. [0130 17:58:57 @multigpu.py:323] [p0576] step: count(1214), step_time 1477.0, mean_step_time 1437.07, it/s 0.7
  5792. [0130 17:58:57 @multigpu.py:323] [p0115] step: count(1190), step_time 1476.45, mean_step_time 1438.23, it/s 0.7
  5793. [0130 17:58:59 @multigpu.py:323] [p0574] step: count(1206), step_time 1423.39, mean_step_time 1437.74, it/s 0.7
  5794. [0130 17:58:59 @multigpu.py:323] [p0576] step: count(1215), step_time 1412.4, mean_step_time 1434.46, it/s 0.7
  5795. [0130 17:58:59 @multigpu.py:323] [p0115] step: count(1191), step_time 1499.11, mean_step_time 1440.53, it/s 0.69
  5796. sending to address tcp://p0112:61216
  5797. ##### Sending to neptune: online_score : 0.482869036661 , 2.0 #####
  5798. [u'online', 2.0]
  5799. receiving
  5800. [0130 17:59:00 @multigpu.py:323] [p0576] step: count(1216), step_time 1419.01, mean_step_time 1435.37, it/s 0.7
  5801. [0130 17:59:00 @multigpu.py:323] [p0574] step: count(1207), step_time 1436.29, mean_step_time 1438.62, it/s 0.7
  5802. [0130 17:59:00 @multigpu.py:323] [p0115] step: count(1192), step_time 1446.43, mean_step_time 1443.06, it/s 0.69
  5803. [0130 17:59:02 @multigpu.py:323] [p0576] step: count(1217), step_time 1414.54, mean_step_time 1435.16, it/s 0.7
  5804. [0130 17:59:02 @multigpu.py:323] [p0574] step: count(1208), step_time 1452.89, mean_step_time 1439.09, it/s 0.69
  5805. [0130 17:59:02 @multigpu.py:323] [p0115] step: count(1193), step_time 1457.53, mean_step_time 1444.29, it/s 0.69
  5806. [0130 17:59:03 @multigpu.py:323] [p0576] step: count(1218), step_time 1407.46, mean_step_time 1432.84, it/s 0.7
  5807. [0130 17:59:03 @multigpu.py:323] [p0574] step: count(1209), step_time 1390.8, mean_step_time 1438.22, it/s 0.7
  5808. [0130 17:59:03 @multigpu.py:323] [p0115] step: count(1194), step_time 1429.94, mean_step_time 1446.71, it/s 0.69
  5809. [0130 17:59:04 @multigpu.py:323] [p0576] step: count(1219), step_time 1411.38, mean_step_time 1431.3, it/s 0.7
  5810. [0130 17:59:04 @multigpu.py:323] [p0574] step: count(1210), step_time 1437.77, mean_step_time 1438.62, it/s 0.7
  5811. [0130 17:59:05 @multigpu.py:323] [p0115] step: count(1195), step_time 1420.12, mean_step_time 1446.35, it/s 0.69
  5812. [0130 17:59:06 @multigpu.py:323] [p0576] step: count(1220), step_time 1431.94, mean_step_time 1433.73, it/s 0.7
  5813. [0130 17:59:06 @multigpu.py:323] [p0574] step: count(1211), step_time 1405.34, mean_step_time 1434.52, it/s 0.7
  5814. [0130 17:59:06 @multigpu.py:323] [p0115] step: count(1196), step_time 1424.1, mean_step_time 1447.18, it/s 0.69
  5815. sending to address tcp://p0112:61216
  5816. ##### Sending to neptune: online_score : 0.484892646074 , 2.2 #####
  5817. [u'online', 2.2]
  5818. receiving
  5819. [0130 17:59:07 @multigpu.py:323] [p0576] step: count(1221), step_time 1406.25, mean_step_time 1430.38, it/s 0.7
  5820. [0130 17:59:07 @multigpu.py:323] [p0574] step: count(1212), step_time 1464.63, mean_step_time 1433.76, it/s 0.7
  5821. [0130 17:59:07 @multigpu.py:323] [p0115] step: count(1197), step_time 1451.57, mean_step_time 1447.33, it/s 0.69
  5822. [0130 17:59:09 @multigpu.py:323] [p0576] step: count(1222), step_time 1469.92, mean_step_time 1433.8, it/s 0.7
  5823. [0130 17:59:09 @multigpu.py:323] [p0574] step: count(1213), step_time 1403.7, mean_step_time 1434.95, it/s 0.7
  5824. [0130 17:59:09 @multigpu.py:323] [p0115] step: count(1198), step_time 1456.32, mean_step_time 1447.46, it/s 0.69
  5825. [0130 17:59:10 @multigpu.py:323] [p0576] step: count(1223), step_time 1411.45, mean_step_time 1434.38, it/s 0.7
  5826. [0130 17:59:10 @multigpu.py:323] [p0574] step: count(1214), step_time 1427.55, mean_step_time 1436.29, it/s 0.7
  5827. [0130 17:59:10 @multigpu.py:323] [p0115] step: count(1199), step_time 1462.95, mean_step_time 1450.67, it/s 0.69
  5828. [0130 17:59:11 @multigpu.py:323] [p0576] step: count(1224), step_time 1425.05, mean_step_time 1434.63, it/s 0.7
  5829. [0130 17:59:12 @multigpu.py:323] [p0574] step: count(1215), step_time 1438.4, mean_step_time 1434.69, it/s 0.7
  5830. [0130 17:59:12 @multigpu.py:323] [p0115] step: count(1200), step_time 1445.15, mean_step_time 1447.23, it/s 0.69
  5831. sending debugging info...
  5832. sending to address tcp://p0112:61216
  5833. ##### Sending to neptune: mean_delay : 0.486447773841 , 0.0 #####
  5834. sending to address tcp://p0112:61216
  5835. ##### Sending to neptune: max_delay : 0.486447773841 , -0.0 #####
  5836. ##### Sending to neptune: min_delay : 0.486447773841 , -0.0 #####
  5837. [u'delays', [0.0, -0.0, -0.0]]
  5838. ##### Sending to neptune: active_workers : 0.486448003054 , 3 #####
  5839. receiving
  5840. ##### Sending to neptune: cost : 0.486448276374 , -0.00475755473599 #####
  5841. sending to address tcp://p0112:61216
  5842. ##### Sending to neptune: policy_loss : 0.486448276374 , 0.0145185422152 #####
  5843. ##### Sending to neptune: xentropy_loss : 0.486448276374 , -2.28904032707 #####
  5844. ##### Sending to neptune: value_loss : 0.486448276374 , 1.66555476189 #####
  5845. ##### Sending to neptune: advantage : 0.486448276374 , -9.42744300119e-05 #####
  5846. ##### Sending to neptune: pred_reward : 0.486448276374 , 0.464911580086 #####
  5847. ##### Sending to neptune: max_logit : 0.486448276374 , 0.191080719233 #####
  5848. [u'loss', -0.0047575547359883785, 0.01451854221522808, -2.2890403270721436, 1.6655547618865967, -9.427443001186475e-05, 0.4649115800857544, 0.1910807192325592]
  5849. receiving
  5850. ##### Sending to neptune: active_relus : 0.486448774669 , 9607864.04 #####
  5851. ##### Sending to neptune: dp_per_s : 0.486448774669 , 88.614272994 #####
  5852. [u'other', 9607864.04, 88.61427299401984]
  5853. receiving
  5854. [0130 17:59:13 @multigpu.py:323] [p0576] step: count(1225), step_time 1419.94, mean_step_time 1433.5, it/s 0.7
  5855. [0130 17:59:13 @multigpu.py:323] [p0574] step: count(1216), step_time 1427.25, mean_step_time 1437.11, it/s 0.7
  5856. [0130 17:59:13 @multigpu.py:323] [p0115] step: count(1201), step_time 1423.0, mean_step_time 1447.45, it/s 0.69
  5857. [0130 17:59:14 @multigpu.py:323] [p0574] step: count(1217), step_time 1435.25, mean_step_time 1437.66, it/s 0.7
  5858. [0130 17:59:14 @multigpu.py:323] [p0576] step: count(1226), step_time 1497.93, mean_step_time 1436.11, it/s 0.7
  5859. [0130 17:59:15 @multigpu.py:323] [p0115] step: count(1202), step_time 1460.86, mean_step_time 1448.0, it/s 0.69
  5860. [0130 17:59:16 @multigpu.py:323] [p0576] step: count(1227), step_time 1387.57, mean_step_time 1436.19, it/s 0.7
  5861. [0130 17:59:16 @multigpu.py:323] [p0574] step: count(1218), step_time 1428.02, mean_step_time 1437.3, it/s 0.7
  5862. [0130 17:59:16 @multigpu.py:323] [p0115] step: count(1203), step_time 1384.71, mean_step_time 1442.66, it/s 0.69
  5863. [0130 17:59:17 @multigpu.py:323] [p0574] step: count(1219), step_time 1419.66, mean_step_time 1435.81, it/s 0.7
  5864. [0130 17:59:17 @multigpu.py:323] [p0576] step: count(1228), step_time 1456.31, mean_step_time 1436.19, it/s 0.7
  5865. [0130 17:59:18 @multigpu.py:323] [p0115] step: count(1204), step_time 1445.15, mean_step_time 1443.92, it/s 0.69
  5866. [0130 17:59:19 @multigpu.py:323] [p0576] step: count(1229), step_time 1376.86, mean_step_time 1434.87, it/s 0.7
  5867. [0130 17:59:19 @multigpu.py:323] [p0574] step: count(1220), step_time 1418.2, mean_step_time 1435.87, it/s 0.7
  5868. [0130 17:59:19 @multigpu.py:323] [p0115] step: count(1205), step_time 1404.72, mean_step_time 1440.4, it/s 0.69
  5869. [0130 17:59:20 @multigpu.py:323] [p0576] step: count(1230), step_time 1425.41, mean_step_time 1434.58, it/s 0.7
  5870. sending to address tcp://p0112:61216
  5871. ##### Sending to neptune: online_score : 0.488742454118 , 1.9 #####
  5872. [u'online', 1.9]
  5873. receiving
  5874. [0130 17:59:20 @multigpu.py:323] [p0574] step: count(1221), step_time 1416.16, mean_step_time 1436.63, it/s 0.7
  5875. [0130 17:59:20 @multigpu.py:323] [p0115] step: count(1206), step_time 1460.48, mean_step_time 1439.73, it/s 0.69
  5876. [0130 17:59:22 @multigpu.py:323] [p0576] step: count(1231), step_time 1449.42, mean_step_time 1434.4, it/s 0.7
  5877. [0130 17:59:22 @multigpu.py:323] [p0574] step: count(1222), step_time 1442.99, mean_step_time 1439.12, it/s 0.69
  5878. [0130 17:59:22 @multigpu.py:323] [p0115] step: count(1207), step_time 1462.84, mean_step_time 1442.8, it/s 0.69
  5879. [0130 17:59:23 @multigpu.py:323] [p0115] step: count(1208), step_time 1410.18, mean_step_time 1443.35, it/s 0.69
  5880. [0130 17:59:23 @multigpu.py:323] [p0576] step: count(1232), step_time 1763.18, mean_step_time 1443.34, it/s 0.69
  5881. [0130 17:59:23 @multigpu.py:323] [p0574] step: count(1223), step_time 1745.65, mean_step_time 1443.44, it/s 0.69
  5882. [0130 17:59:25 @multigpu.py:323] [p0576] step: count(1233), step_time 1403.5, mean_step_time 1443.32, it/s 0.69
  5883. [0130 17:59:25 @multigpu.py:323] [p0115] step: count(1209), step_time 1411.91, mean_step_time 1441.68, it/s 0.69
  5884. [0130 17:59:25 @multigpu.py:323] [p0574] step: count(1224), step_time 1449.48, mean_step_time 1442.97, it/s 0.69
  5885. [0130 17:59:26 @multigpu.py:323] [p0576] step: count(1234), step_time 1413.02, mean_step_time 1440.13, it/s 0.69
  5886. [0130 17:59:26 @multigpu.py:323] [p0115] step: count(1210), step_time 1415.36, mean_step_time 1438.62, it/s 0.7
  5887. [0130 17:59:26 @multigpu.py:323] [p0574] step: count(1225), step_time 1408.73, mean_step_time 1443.61, it/s 0.69
  5888. [0130 17:59:28 @multigpu.py:323] [p0576] step: count(1235), step_time 1410.4, mean_step_time 1440.03, it/s 0.69
  5889. [0130 17:59:28 @multigpu.py:323] [p0574] step: count(1226), step_time 1409.94, mean_step_time 1442.94, it/s 0.69
  5890. [0130 17:59:28 @multigpu.py:323] [p0115] step: count(1211), step_time 1486.26, mean_step_time 1437.98, it/s 0.7
  5891. [0130 17:59:29 @multigpu.py:323] [p0576] step: count(1236), step_time 1377.55, mean_step_time 1437.95, it/s 0.7
  5892. [0130 17:59:29 @multigpu.py:323] [p0574] step: count(1227), step_time 1451.71, mean_step_time 1443.71, it/s 0.69
  5893. [0130 17:59:29 @multigpu.py:323] [p0115] step: count(1212), step_time 1450.74, mean_step_time 1438.19, it/s 0.7
  5894. [0130 17:59:30 @multigpu.py:323] [p0576] step: count(1237), step_time 1406.86, mean_step_time 1437.57, it/s 0.7
  5895. [0130 17:59:30 @multigpu.py:323] [p0574] step: count(1228), step_time 1483.1, mean_step_time 1445.22, it/s 0.69
  5896. [0130 17:59:30 @multigpu.py:323] [p0115] step: count(1213), step_time 1449.25, mean_step_time 1437.78, it/s 0.7
  5897. sending to address tcp://p0112:61216
  5898. ##### Sending to neptune: online_score : 0.491776681079 , 1.5 #####
  5899. [u'online', 1.5]
  5900. receiving
  5901. [0130 17:59:32 @multigpu.py:323] [p0576] step: count(1238), step_time 1422.62, mean_step_time 1438.33, it/s 0.7
  5902. [0130 17:59:32 @multigpu.py:323] [p0115] step: count(1214), step_time 1383.4, mean_step_time 1435.45, it/s 0.7
  5903. [0130 17:59:32 @multigpu.py:323] [p0574] step: count(1229), step_time 1452.75, mean_step_time 1448.31, it/s 0.69
  5904. [0130 17:59:33 @multigpu.py:323] [p0576] step: count(1239), step_time 1400.1, mean_step_time 1437.76, it/s 0.7
  5905. [0130 17:59:33 @multigpu.py:323] [p0115] step: count(1215), step_time 1418.15, mean_step_time 1435.36, it/s 0.7
  5906. [0130 17:59:33 @multigpu.py:323] [p0574] step: count(1230), step_time 1462.29, mean_step_time 1449.54, it/s 0.69
  5907. [0130 17:59:35 @multigpu.py:323] [p0576] step: count(1240), step_time 1443.06, mean_step_time 1438.32, it/s 0.7
  5908. [0130 17:59:35 @multigpu.py:323] [p0115] step: count(1216), step_time 1425.73, mean_step_time 1435.44, it/s 0.7
  5909. [0130 17:59:35 @multigpu.py:323] [p0574] step: count(1231), step_time 1409.05, mean_step_time 1449.73, it/s 0.69
  5910. [0130 17:59:36 @multigpu.py:323] [p0576] step: count(1241), step_time 1385.11, mean_step_time 1437.26, it/s 0.7
  5911. [0130 17:59:36 @multigpu.py:323] [p0115] step: count(1217), step_time 1434.85, mean_step_time 1434.6, it/s 0.7
  5912. [0130 17:59:36 @multigpu.py:323] [p0574] step: count(1232), step_time 1401.94, mean_step_time 1446.59, it/s 0.69
  5913. [0130 17:59:37 @multigpu.py:323] [p0576] step: count(1242), step_time 1437.12, mean_step_time 1435.62, it/s 0.7
  5914. [0130 17:59:38 @multigpu.py:323] [p0115] step: count(1218), step_time 1427.21, mean_step_time 1433.15, it/s 0.7
  5915. [0130 17:59:38 @multigpu.py:323] [p0574] step: count(1233), step_time 1444.78, mean_step_time 1448.65, it/s 0.69
  5916. sending to address tcp://p0112:61216
  5917. ##### Sending to neptune: online_score : 0.493770758841 , 2.2 #####
  5918. [u'online', 2.2]
  5919. receiving
  5920. [0130 17:59:39 @multigpu.py:323] [p0576] step: count(1243), step_time 1458.66, mean_step_time 1437.98, it/s 0.7
  5921. [0130 17:59:39 @multigpu.py:323] [p0115] step: count(1219), step_time 1430.12, mean_step_time 1431.5, it/s 0.7
  5922. [0130 17:59:39 @multigpu.py:323] [p0574] step: count(1234), step_time 1417.29, mean_step_time 1448.13, it/s 0.69
  5923. [0130 17:59:40 @multigpu.py:323] [p0576] step: count(1244), step_time 1432.94, mean_step_time 1438.38, it/s 0.7
  5924. [0130 17:59:40 @multigpu.py:323] [p0115] step: count(1220), step_time 1428.74, mean_step_time 1430.68, it/s 0.7
  5925. [0130 17:59:41 @multigpu.py:323] [p0574] step: count(1235), step_time 1480.59, mean_step_time 1450.24, it/s 0.69
  5926. [0130 17:59:42 @multigpu.py:323] [p0576] step: count(1245), step_time 1429.8, mean_step_time 1438.87, it/s 0.69
  5927. [0130 17:59:42 @multigpu.py:323] [p0115] step: count(1221), step_time 1496.29, mean_step_time 1434.35, it/s 0.7
  5928. [0130 17:59:42 @multigpu.py:323] [p0574] step: count(1236), step_time 1433.37, mean_step_time 1450.55, it/s 0.69
  5929. [0130 17:59:43 @multigpu.py:323] [p0576] step: count(1246), step_time 1430.66, mean_step_time 1435.51, it/s 0.7
  5930. [0130 17:59:43 @multigpu.py:323] [p0115] step: count(1222), step_time 1429.09, mean_step_time 1432.76, it/s 0.7
  5931. [0130 17:59:43 @multigpu.py:323] [p0574] step: count(1237), step_time 1417.41, mean_step_time 1449.66, it/s 0.69
  5932. [0130 17:59:45 @multigpu.py:323] [p0576] step: count(1247), step_time 1401.68, mean_step_time 1436.21, it/s 0.7
  5933. [0130 17:59:45 @multigpu.py:323] [p0115] step: count(1223), step_time 1466.01, mean_step_time 1436.82, it/s 0.7
  5934. [0130 17:59:45 @multigpu.py:323] [p0574] step: count(1238), step_time 1439.89, mean_step_time 1450.25, it/s 0.69
  5935. sending to address tcp://p0112:61216
  5936. ##### Sending to neptune: online_score : 0.495688630276 , 1.6 #####
  5937. [u'online', 1.6]
  5938. receiving
  5939. [0130 17:59:46 @multigpu.py:323] [p0576] step: count(1248), step_time 1470.4, mean_step_time 1436.92, it/s 0.7
  5940. [0130 17:59:46 @multigpu.py:323] [p0574] step: count(1239), step_time 1392.69, mean_step_time 1448.9, it/s 0.69
  5941. [0130 17:59:46 @multigpu.py:323] [p0115] step: count(1224), step_time 1429.94, mean_step_time 1436.06, it/s 0.7
  5942. [0130 17:59:47 @multigpu.py:323] [p0576] step: count(1249), step_time 1418.38, mean_step_time 1438.99, it/s 0.69
  5943. [0130 17:59:48 @multigpu.py:323] [p0574] step: count(1240), step_time 1410.31, mean_step_time 1448.51, it/s 0.69
  5944. [0130 17:59:48 @multigpu.py:323] [p0115] step: count(1225), step_time 1454.57, mean_step_time 1438.56, it/s 0.7
  5945. [0130 17:59:49 @multigpu.py:323] [p0576] step: count(1250), step_time 1469.11, mean_step_time 1441.18, it/s 0.69
  5946. [0130 17:59:49 @multigpu.py:323] [p0574] step: count(1241), step_time 1455.97, mean_step_time 1450.5, it/s 0.69
  5947. [0130 17:59:49 @multigpu.py:323] [p0115] step: count(1226), step_time 1404.38, mean_step_time 1435.75, it/s 0.7
  5948. [0130 17:59:50 @multigpu.py:323] [p0576] step: count(1251), step_time 1474.91, mean_step_time 1442.45, it/s 0.69
  5949. [0130 17:59:50 @multigpu.py:323] [p0574] step: count(1242), step_time 1392.37, mean_step_time 1447.97, it/s 0.69
  5950. [0130 17:59:51 @multigpu.py:323] [p0115] step: count(1227), step_time 1441.44, mean_step_time 1434.68, it/s 0.7
  5951. [0130 17:59:52 @multigpu.py:323] [p0115] step: count(1228), step_time 1428.7, mean_step_time 1435.61, it/s 0.7
  5952. [0130 17:59:52 @multigpu.py:323] [p0574] step: count(1243), step_time 1497.43, mean_step_time 1435.55, it/s 0.7
  5953. [0130 17:59:52 @multigpu.py:323] [p0576] step: count(1252), step_time 1625.91, mean_step_time 1435.59, it/s 0.7
  5954. [0130 17:59:53 @multigpu.py:323] [p0576] step: count(1253), step_time 1398.41, mean_step_time 1435.33, it/s 0.7
  5955. [0130 17:59:53 @multigpu.py:323] [p0115] step: count(1229), step_time 1408.68, mean_step_time 1435.45, it/s 0.7
  5956. [0130 17:59:53 @multigpu.py:323] [p0574] step: count(1244), step_time 1411.42, mean_step_time 1433.65, it/s 0.7
  5957. [0130 17:59:55 @multigpu.py:323] [p0576] step: count(1254), step_time 1420.87, mean_step_time 1435.73, it/s 0.7
  5958. [0130 17:59:55 @multigpu.py:323] [p0574] step: count(1245), step_time 1447.2, mean_step_time 1435.57, it/s 0.7
  5959. [0130 17:59:55 @multigpu.py:323] [p0115] step: count(1230), step_time 1484.63, mean_step_time 1438.91, it/s 0.69
  5960. [0130 17:59:56 @multigpu.py:323] [p0576] step: count(1255), step_time 1397.69, mean_step_time 1435.09, it/s 0.7
  5961. [0130 17:59:56 @multigpu.py:323] [p0115] step: count(1231), step_time 1391.17, mean_step_time 1434.15, it/s 0.7
  5962. [0130 17:59:56 @multigpu.py:323] [p0574] step: count(1246), step_time 1455.19, mean_step_time 1437.84, it/s 0.7
  5963. [0130 17:59:58 @multigpu.py:323] [p0576] step: count(1256), step_time 1380.83, mean_step_time 1435.26, it/s 0.7
  5964. [0130 17:59:58 @multigpu.py:323] [p0115] step: count(1232), step_time 1437.02, mean_step_time 1433.47, it/s 0.7
  5965. [0130 17:59:58 @multigpu.py:323] [p0574] step: count(1247), step_time 1427.1, mean_step_time 1436.61, it/s 0.7
  5966. [0130 17:59:59 @multigpu.py:323] [p0576] step: count(1257), step_time 1488.55, mean_step_time 1439.34, it/s 0.69
  5967. [0130 17:59:59 @multigpu.py:323] [p0574] step: count(1248), step_time 1399.27, mean_step_time 1432.42, it/s 0.7
  5968. [0130 17:59:59 @multigpu.py:323] [p0115] step: count(1233), step_time 1424.49, mean_step_time 1432.23, it/s 0.7
  5969. [0130 18:00:01 @multigpu.py:323] [p0576] step: count(1258), step_time 1418.93, mean_step_time 1439.15, it/s 0.69
  5970. [0130 18:00:01 @multigpu.py:323] [p0574] step: count(1249), step_time 1382.59, mean_step_time 1428.91, it/s 0.7
  5971. [0130 18:00:01 @multigpu.py:323] [p0115] step: count(1234), step_time 1432.11, mean_step_time 1434.67, it/s 0.7
  5972. sending to address tcp://p0112:61216
  5973. ##### Sending to neptune: online_score : 0.500219986373 , 1.8 #####
  5974. [u'online', 1.8]
  5975. receiving
  5976. [0130 18:00:02 @multigpu.py:323] [p0576] step: count(1259), step_time 1450.22, mean_step_time 1441.66, it/s 0.69
  5977. [0130 18:00:02 @multigpu.py:323] [p0115] step: count(1235), step_time 1417.64, mean_step_time 1434.64, it/s 0.7
  5978. [0130 18:00:02 @multigpu.py:323] [p0574] step: count(1250), step_time 1493.7, mean_step_time 1430.48, it/s 0.7
  5979. [0130 18:00:03 @multigpu.py:323] [p0576] step: count(1260), step_time 1436.07, mean_step_time 1441.31, it/s 0.69
  5980. [0130 18:00:03 @multigpu.py:323] [p0115] step: count(1236), step_time 1446.74, mean_step_time 1435.69, it/s 0.7
  5981. [0130 18:00:03 @multigpu.py:323] [p0574] step: count(1251), step_time 1452.27, mean_step_time 1432.64, it/s 0.7
  5982. [0130 18:00:05 @multigpu.py:323] [p0576] step: count(1261), step_time 1421.8, mean_step_time 1443.15, it/s 0.69
  5983. [0130 18:00:05 @multigpu.py:323] [p0574] step: count(1252), step_time 1412.02, mean_step_time 1433.14, it/s 0.7
  5984. [0130 18:00:05 @multigpu.py:323] [p0115] step: count(1237), step_time 1443.62, mean_step_time 1436.13, it/s 0.7
  5985. [0130 18:00:06 @multigpu.py:323] [p0115] step: count(1238), step_time 1427.58, mean_step_time 1436.15, it/s 0.7
  5986. [0130 18:00:06 @multigpu.py:323] [p0576] step: count(1262), step_time 1502.99, mean_step_time 1446.44, it/s 0.69
  5987. [0130 18:00:06 @multigpu.py:323] [p0574] step: count(1253), step_time 1450.51, mean_step_time 1433.43, it/s 0.7
  5988. sending to address tcp://p0112:61216
  5989. ##### Sending to neptune: online_score : 0.50163765053 , 1.8 #####
  5990. [u'online', 1.8]
  5991. receiving
  5992. [0130 18:00:08 @multigpu.py:323] [p0576] step: count(1263), step_time 1376.02, mean_step_time 1442.31, it/s 0.69
  5993. [0130 18:00:08 @multigpu.py:323] [p0574] step: count(1254), step_time 1377.62, mean_step_time 1431.45, it/s 0.7
  5994. [0130 18:00:08 @multigpu.py:323] [p0115] step: count(1239), step_time 1444.37, mean_step_time 1436.86, it/s 0.7
  5995. [0130 18:00:09 @multigpu.py:323] [p0115] step: count(1240), step_time 1384.45, mean_step_time 1434.65, it/s 0.7
  5996. [0130 18:00:09 @multigpu.py:323] [p0576] step: count(1264), step_time 1465.98, mean_step_time 1443.96, it/s 0.69
  5997. [0130 18:00:09 @multigpu.py:323] [p0574] step: count(1255), step_time 1491.25, mean_step_time 1431.98, it/s 0.7
  5998. [0130 18:00:11 @multigpu.py:323] [p0576] step: count(1265), step_time 1377.57, mean_step_time 1441.35, it/s 0.69
  5999. [0130 18:00:11 @multigpu.py:323] [p0115] step: count(1241), step_time 1427.5, mean_step_time 1431.21, it/s 0.7
  6000. [0130 18:00:11 @multigpu.py:323] [p0574] step: count(1256), step_time 1425.77, mean_step_time 1431.6, it/s 0.7
  6001. [0130 18:00:12 @multigpu.py:323] [p0115] step: count(1242), step_time 1413.5, mean_step_time 1430.43, it/s 0.7
  6002. [0130 18:00:12 @multigpu.py:323] [p0576] step: count(1266), step_time 1485.86, mean_step_time 1444.11, it/s 0.69
  6003. [0130 18:00:12 @multigpu.py:323] [p0574] step: count(1257), step_time 1398.03, mean_step_time 1430.63, it/s 0.7
  6004. [0130 18:00:13 @multigpu.py:323] [p0576] step: count(1267), step_time 1384.89, mean_step_time 1443.27, it/s 0.69
  6005. [0130 18:00:13 @multigpu.py:323] [p0574] step: count(1258), step_time 1404.02, mean_step_time 1428.84, it/s 0.7
  6006. [0130 18:00:13 @multigpu.py:323] [p0115] step: count(1243), step_time 1449.9, mean_step_time 1429.62, it/s 0.7
  6007. [0130 18:00:15 @multigpu.py:323] [p0115] step: count(1244), step_time 1399.48, mean_step_time 1428.1, it/s 0.7
  6008. [0130 18:00:15 @multigpu.py:323] [p0574] step: count(1259), step_time 1415.75, mean_step_time 1429.99, it/s 0.7
  6009. [0130 18:00:15 @multigpu.py:323] [p0576] step: count(1268), step_time 1455.89, mean_step_time 1442.54, it/s 0.69
  6010. [0130 18:00:16 @multigpu.py:323] [p0115] step: count(1245), step_time 1399.33, mean_step_time 1425.34, it/s 0.7
  6011. [0130 18:00:16 @multigpu.py:323] [p0574] step: count(1260), step_time 1419.75, mean_step_time 1430.46, it/s 0.7
  6012. [0130 18:00:16 @multigpu.py:323] [p0576] step: count(1269), step_time 1447.95, mean_step_time 1444.02, it/s 0.69
  6013. sending to address tcp://p0112:61216
  6014. ##### Sending to neptune: online_score : 0.504519854983 , 1.2 #####
  6015. [u'online', 1.2]
  6016. ##### Sending to neptune: active_workers : 0.504519919422 , 3 #####
  6017. receiving
  6018. [0130 18:00:18 @multigpu.py:323] [p0115] step: count(1246), step_time 1424.38, mean_step_time 1426.34, it/s 0.7
  6019. [0130 18:00:18 @multigpu.py:323] [p0574] step: count(1261), step_time 1404.1, mean_step_time 1427.87, it/s 0.7
  6020. [0130 18:00:18 @multigpu.py:323] [p0576] step: count(1270), step_time 1404.17, mean_step_time 1440.78, it/s 0.69
  6021. [0130 18:00:19 @multigpu.py:323] [p0574] step: count(1262), step_time 1389.37, mean_step_time 1427.72, it/s 0.7
  6022. [0130 18:00:19 @multigpu.py:323] [p0115] step: count(1247), step_time 1435.36, mean_step_time 1426.03, it/s 0.7
  6023. [0130 18:00:19 @multigpu.py:323] [p0576] step: count(1271), step_time 1398.44, mean_step_time 1436.95, it/s 0.7
  6024. [0130 18:00:21 @multigpu.py:323] [p0115] step: count(1248), step_time 1413.05, mean_step_time 1425.25, it/s 0.7
  6025. [0130 18:00:21 @multigpu.py:323] [p0576] step: count(1272), step_time 1389.99, mean_step_time 1425.16, it/s 0.7
  6026. [0130 18:00:21 @multigpu.py:323] [p0574] step: count(1263), step_time 1449.4, mean_step_time 1425.32, it/s 0.7
  6027. [0130 18:00:22 @multigpu.py:323] [p0576] step: count(1273), step_time 1400.21, mean_step_time 1425.25, it/s 0.7
  6028. [0130 18:00:22 @multigpu.py:323] [p0574] step: count(1264), step_time 1407.21, mean_step_time 1425.11, it/s 0.7
  6029. [0130 18:00:22 @multigpu.py:323] [p0115] step: count(1249), step_time 1461.25, mean_step_time 1427.88, it/s 0.7
  6030. [0130 18:00:23 @multigpu.py:323] [p0576] step: count(1274), step_time 1490.45, mean_step_time 1428.73, it/s 0.7
  6031. [0130 18:00:23 @multigpu.py:323] [p0115] step: count(1250), step_time 1429.73, mean_step_time 1425.13, it/s 0.7
  6032. [0130 18:00:23 @multigpu.py:323] [p0574] step: count(1265), step_time 1537.12, mean_step_time 1429.6, it/s 0.7
  6033. [0130 18:00:25 @multigpu.py:323] [p0576] step: count(1275), step_time 1365.34, mean_step_time 1427.11, it/s 0.7
  6034. [0130 18:00:25 @multigpu.py:323] [p0115] step: count(1251), step_time 1407.17, mean_step_time 1425.93, it/s 0.7
  6035. [0130 18:00:25 @multigpu.py:323] [p0574] step: count(1266), step_time 1356.73, mean_step_time 1424.68, it/s 0.7
  6036. [0130 18:00:26 @multigpu.py:323] [p0576] step: count(1276), step_time 1412.74, mean_step_time 1428.7, it/s 0.7
  6037. [0130 18:00:26 @multigpu.py:323] [p0115] step: count(1252), step_time 1420.38, mean_step_time 1425.1, it/s 0.7
  6038. [0130 18:00:26 @multigpu.py:323] [p0574] step: count(1267), step_time 1491.01, mean_step_time 1427.87, it/s 0.7
  6039. [0130 18:00:28 @multigpu.py:323] [p0576] step: count(1277), step_time 1476.89, mean_step_time 1428.12, it/s 0.7
  6040. [0130 18:00:28 @multigpu.py:323] [p0115] step: count(1253), step_time 1436.83, mean_step_time 1425.72, it/s 0.7
  6041. [0130 18:00:28 @multigpu.py:323] [p0574] step: count(1268), step_time 1375.61, mean_step_time 1426.69, it/s 0.7
  6042. [0130 18:00:29 @multigpu.py:323] [p0576] step: count(1278), step_time 1414.23, mean_step_time 1427.89, it/s 0.7
  6043. [0130 18:00:29 @multigpu.py:323] [p0115] step: count(1254), step_time 1456.16, mean_step_time 1426.92, it/s 0.7
  6044. [0130 18:00:29 @multigpu.py:323] [p0574] step: count(1269), step_time 1470.79, mean_step_time 1431.1, it/s 0.7
  6045. [0130 18:00:31 @multigpu.py:323] [p0576] step: count(1279), step_time 1439.79, mean_step_time 1427.36, it/s 0.7
  6046. [0130 18:00:31 @multigpu.py:323] [p0115] step: count(1255), step_time 1390.55, mean_step_time 1425.57, it/s 0.7
  6047. [0130 18:00:31 @multigpu.py:323] [p0574] step: count(1270), step_time 1458.49, mean_step_time 1429.34, it/s 0.7
  6048. [0130 18:00:32 @multigpu.py:323] [p0576] step: count(1280), step_time 1430.71, mean_step_time 1427.1, it/s 0.7
  6049. [0130 18:00:32 @multigpu.py:323] [p0115] step: count(1256), step_time 1428.18, mean_step_time 1424.64, it/s 0.7
  6050. [0130 18:00:32 @multigpu.py:323] [p0574] step: count(1271), step_time 1435.24, mean_step_time 1428.49, it/s 0.7
  6051. [0130 18:00:33 @multigpu.py:323] [p0115] step: count(1257), step_time 1449.35, mean_step_time 1424.92, it/s 0.7
  6052. [0130 18:00:33 @multigpu.py:323] [p0576] step: count(1281), step_time 1465.3, mean_step_time 1429.27, it/s 0.7
  6053. [0130 18:00:33 @multigpu.py:323] [p0574] step: count(1272), step_time 1448.46, mean_step_time 1430.31, it/s 0.7
  6054. sending to address tcp://p0112:61216
  6055. ##### Sending to neptune: online_score : 0.509209632476 , 1.7 #####
  6056. [u'online', 1.7]
  6057. receiving
  6058. sending to address tcp://p0112:61216
  6059. ##### Sending to neptune: online_score : 0.509221385519 , 1.8 #####
  6060. [u'online', 1.8]
  6061. receiving
  6062. [0130 18:00:35 @multigpu.py:323] [p0576] step: count(1282), step_time 1407.69, mean_step_time 1424.51, it/s 0.7
  6063. [0130 18:00:35 @multigpu.py:323] [p0115] step: count(1258), step_time 1431.65, mean_step_time 1425.13, it/s 0.7
  6064. [0130 18:00:35 @multigpu.py:323] [p0574] step: count(1273), step_time 1424.04, mean_step_time 1428.99, it/s 0.7
  6065. [0130 18:00:36 @multigpu.py:323] [p0115] step: count(1259), step_time 1441.86, mean_step_time 1425.0, it/s 0.7
  6066. [0130 18:00:36 @multigpu.py:323] [p0576] step: count(1283), step_time 1467.47, mean_step_time 1429.08, it/s 0.7
  6067. [0130 18:00:36 @multigpu.py:323] [p0574] step: count(1274), step_time 1414.32, mean_step_time 1430.82, it/s 0.7
  6068. [0130 18:00:38 @multigpu.py:323] [p0115] step: count(1260), step_time 1417.55, mean_step_time 1426.66, it/s 0.7
  6069. [0130 18:00:38 @multigpu.py:323] [p0576] step: count(1284), step_time 1448.14, mean_step_time 1428.19, it/s 0.7
  6070. [0130 18:00:38 @multigpu.py:323] [p0574] step: count(1275), step_time 1415.54, mean_step_time 1427.04, it/s 0.7
  6071. sending to address tcp://p0112:61216
  6072. ##### Sending to neptune: online_score : 0.510377633837 , 1.3 #####
  6073. [u'online', 1.3]
  6074. receiving
  6075. [0130 18:00:39 @multigpu.py:323] [p0115] step: count(1261), step_time 1374.91, mean_step_time 1424.03, it/s 0.7
  6076. [0130 18:00:39 @multigpu.py:323] [p0576] step: count(1285), step_time 1424.08, mean_step_time 1430.51, it/s 0.7
  6077. [0130 18:00:39 @multigpu.py:323] [p0574] step: count(1276), step_time 1443.27, mean_step_time 1427.91, it/s 0.7
  6078. [0130 18:00:40 @multigpu.py:323] [p0115] step: count(1262), step_time 1429.56, mean_step_time 1424.83, it/s 0.7
  6079. [0130 18:00:41 @multigpu.py:323] [p0576] step: count(1286), step_time 1391.27, mean_step_time 1425.78, it/s 0.7
  6080. [0130 18:00:41 @multigpu.py:323] [p0574] step: count(1277), step_time 1455.26, mean_step_time 1430.77, it/s 0.7
  6081. [0130 18:00:42 @multigpu.py:323] [p0115] step: count(1263), step_time 1416.86, mean_step_time 1423.18, it/s 0.7
  6082. [0130 18:00:42 @multigpu.py:323] [p0576] step: count(1287), step_time 1418.46, mean_step_time 1427.46, it/s 0.7
  6083. [0130 18:00:42 @multigpu.py:323] [p0574] step: count(1278), step_time 1389.05, mean_step_time 1430.03, it/s 0.7
  6084. [0130 18:00:43 @multigpu.py:323] [p0115] step: count(1264), step_time 1434.03, mean_step_time 1424.91, it/s 0.7
  6085. [0130 18:00:43 @multigpu.py:323] [p0576] step: count(1288), step_time 1383.01, mean_step_time 1423.82, it/s 0.7
  6086. [0130 18:00:44 @multigpu.py:323] [p0574] step: count(1279), step_time 1482.24, mean_step_time 1433.35, it/s 0.7
  6087. [0130 18:00:45 @multigpu.py:323] [p0576] step: count(1289), step_time 1412.39, mean_step_time 1422.04, it/s 0.7
  6088. [0130 18:00:45 @multigpu.py:323] [p0115] step: count(1265), step_time 1457.3, mean_step_time 1427.8, it/s 0.7
  6089. [0130 18:00:45 @multigpu.py:323] [p0574] step: count(1280), step_time 1369.54, mean_step_time 1430.84, it/s 0.7
  6090. [0130 18:00:46 @multigpu.py:323] [p0576] step: count(1290), step_time 1454.01, mean_step_time 1424.53, it/s 0.7
  6091. [0130 18:00:46 @multigpu.py:323] [p0115] step: count(1266), step_time 1466.19, mean_step_time 1429.9, it/s 0.7
  6092. [0130 18:00:46 @multigpu.py:323] [p0574] step: count(1281), step_time 1429.17, mean_step_time 1432.09, it/s 0.7
  6093. [0130 18:00:48 @multigpu.py:323] [p0576] step: count(1291), step_time 1417.98, mean_step_time 1425.51, it/s 0.7
  6094. [0130 18:00:48 @multigpu.py:323] [p0115] step: count(1267), step_time 1467.76, mean_step_time 1431.52, it/s 0.7
  6095. [0130 18:00:48 @multigpu.py:323] [p0574] step: count(1282), step_time 1453.63, mean_step_time 1435.31, it/s 0.7
  6096. [0130 18:00:49 @multigpu.py:323] [p0115] step: count(1268), step_time 1453.14, mean_step_time 1433.52, it/s 0.7
  6097. [0130 18:00:49 @multigpu.py:323] [p0574] step: count(1283), step_time 1408.98, mean_step_time 1433.28, it/s 0.7
  6098. [0130 18:00:49 @multigpu.py:323] [p0576] step: count(1292), step_time 1550.01, mean_step_time 1433.51, it/s 0.7
  6099. [0130 18:00:51 @multigpu.py:323] [p0576] step: count(1293), step_time 1398.16, mean_step_time 1433.41, it/s 0.7
  6100. [0130 18:00:51 @multigpu.py:323] [p0115] step: count(1269), step_time 1414.4, mean_step_time 1431.18, it/s 0.7
  6101. [0130 18:00:51 @multigpu.py:323] [p0574] step: count(1284), step_time 1436.26, mean_step_time 1434.74, it/s 0.7
  6102. [0130 18:00:52 @multigpu.py:323] [p0115] step: count(1270), step_time 1387.23, mean_step_time 1429.05, it/s 0.7
  6103. [0130 18:00:52 @multigpu.py:323] [p0576] step: count(1294), step_time 1448.65, mean_step_time 1431.32, it/s 0.7
  6104. [0130 18:00:52 @multigpu.py:323] [p0574] step: count(1285), step_time 1440.68, mean_step_time 1429.91, it/s 0.7
  6105. [0130 18:00:53 @multigpu.py:323] [p0115] step: count(1271), step_time 1413.52, mean_step_time 1429.37, it/s 0.7
  6106. [0130 18:00:53 @multigpu.py:323] [p0574] step: count(1286), step_time 1391.43, mean_step_time 1431.65, it/s 0.7
  6107. [0130 18:00:53 @multigpu.py:323] [p0576] step: count(1295), step_time 1450.11, mean_step_time 1435.56, it/s 0.7
  6108. [0130 18:00:55 @multigpu.py:323] [p0115] step: count(1272), step_time 1425.58, mean_step_time 1429.63, it/s 0.7
  6109. [0130 18:00:55 @multigpu.py:323] [p0574] step: count(1287), step_time 1399.37, mean_step_time 1427.07, it/s 0.7
  6110. [0130 18:00:55 @multigpu.py:323] [p0576] step: count(1296), step_time 1421.49, mean_step_time 1435.99, it/s 0.7
  6111. [0130 18:00:56 @multigpu.py:323] [p0115] step: count(1273), step_time 1414.14, mean_step_time 1428.5, it/s 0.7
  6112. [0130 18:00:56 @multigpu.py:323] [p0574] step: count(1288), step_time 1441.05, mean_step_time 1430.34, it/s 0.7
  6113. [0130 18:00:56 @multigpu.py:323] [p0576] step: count(1297), step_time 1401.6, mean_step_time 1432.23, it/s 0.7
  6114. sending to address tcp://p0112:61216
  6115. ##### Sending to neptune: online_score : 0.515617745519 , 1.5 #####
  6116. [u'online', 1.5]
  6117. receiving
  6118. [0130 18:00:58 @multigpu.py:323] [p0115] step: count(1274), step_time 1429.4, mean_step_time 1427.16, it/s 0.7
  6119. [0130 18:00:58 @multigpu.py:323] [p0576] step: count(1298), step_time 1386.47, mean_step_time 1430.84, it/s 0.7
  6120. [0130 18:00:58 @multigpu.py:323] [p0574] step: count(1289), step_time 1408.34, mean_step_time 1427.22, it/s 0.7
  6121. [0130 18:00:59 @multigpu.py:323] [p0576] step: count(1299), step_time 1403.87, mean_step_time 1429.04, it/s 0.7
  6122. [0130 18:00:59 @multigpu.py:323] [p0115] step: count(1275), step_time 1463.58, mean_step_time 1430.81, it/s 0.7
  6123. [0130 18:00:59 @multigpu.py:323] [p0574] step: count(1290), step_time 1480.56, mean_step_time 1428.32, it/s 0.7
  6124. [0130 18:01:01 @multigpu.py:323] [p0576] step: count(1300), step_time 1430.43, mean_step_time 1429.03, it/s 0.7
  6125. sending debugging info...
  6126. sending to address tcp://p0112:61216
  6127. ##### Sending to neptune: mean_delay : 0.516647246612 , 0.0 #####
  6128. sending to address tcp://p0112:61216
  6129. ##### Sending to neptune: max_delay : 0.516647246612 , -0.0 #####
  6130. ##### Sending to neptune: min_delay : 0.516647246612 , -0.0 #####
  6131. [u'delays', [0.0, -0.0, -0.0]]
  6132. receiving
  6133. ##### Sending to neptune: cost : 0.51664776716 , -0.00946542713791 #####
  6134. sending to address tcp://p0112:61216
  6135. ##### Sending to neptune: policy_loss : 0.51664776716 , -0.313773274422 #####
  6136. ##### Sending to neptune: xentropy_loss : 0.51664776716 , -2.28776264191 #####
  6137. ##### Sending to neptune: value_loss : 0.51664776716 , 1.38996088505 #####
  6138. ##### Sending to neptune: advantage : 0.51664776716 , 0.00140806438867 #####
  6139. ##### Sending to neptune: pred_reward : 0.51664776716 , 0.47912633419 #####
  6140. ##### Sending to neptune: max_logit : 0.51664776716 , 0.194705918431 #####
  6141. [u'loss', -0.00946542713791132, -0.3137732744216919, -2.2877626419067383, 1.3899608850479126, 0.0014080643886700273, 0.47912633419036865, 0.19470591843128204]
  6142. receiving
  6143. ##### Sending to neptune: active_relus : 0.516648219956 , 9676168.09 #####
  6144. ##### Sending to neptune: dp_per_s : 0.516648219956 , 89.1823527596 #####
  6145. [u'other', 9676168.09, 89.18235275962654]
  6146. receiving
  6147. [0130 18:01:01 @multigpu.py:323] [p0574] step: count(1291), step_time 1398.52, mean_step_time 1426.49, it/s 0.7
  6148. [0130 18:01:01 @multigpu.py:323] [p0115] step: count(1276), step_time 1465.75, mean_step_time 1432.69, it/s 0.7
  6149. [0130 18:01:02 @multigpu.py:323] [p0576] step: count(1301), step_time 1438.25, mean_step_time 1427.68, it/s 0.7
  6150. [0130 18:01:02 @multigpu.py:323] [p0574] step: count(1292), step_time 1438.86, mean_step_time 1426.0, it/s 0.7
  6151. [0130 18:01:02 @multigpu.py:323] [p0115] step: count(1277), step_time 1457.61, mean_step_time 1433.1, it/s 0.7
  6152. [0130 18:01:03 @multigpu.py:323] [p0576] step: count(1302), step_time 1384.81, mean_step_time 1426.53, it/s 0.7
  6153. [0130 18:01:03 @multigpu.py:323] [p0115] step: count(1278), step_time 1413.94, mean_step_time 1432.22, it/s 0.7
  6154. [0130 18:01:03 @multigpu.py:323] [p0574] step: count(1293), step_time 1452.43, mean_step_time 1427.42, it/s 0.7
  6155. [0130 18:01:05 @multigpu.py:323] [p0576] step: count(1303), step_time 1399.88, mean_step_time 1423.15, it/s 0.7
  6156. [0130 18:01:05 @multigpu.py:323] [p0574] step: count(1294), step_time 1376.51, mean_step_time 1425.53, it/s 0.7
  6157. [0130 18:01:05 @multigpu.py:323] [p0115] step: count(1279), step_time 1481.14, mean_step_time 1434.18, it/s 0.7
  6158. [0130 18:01:06 @multigpu.py:323] [p0576] step: count(1304), step_time 1466.55, mean_step_time 1424.07, it/s 0.7
  6159. [0130 18:01:06 @multigpu.py:323] [p0574] step: count(1295), step_time 1445.89, mean_step_time 1427.05, it/s 0.7
  6160. [0130 18:01:06 @multigpu.py:323] [p0115] step: count(1280), step_time 1414.5, mean_step_time 1434.03, it/s 0.7
  6161. sending to address tcp://p0112:61216
  6162. ##### Sending to neptune: online_score : 0.518410489957 , 2.2 #####
  6163. [u'online', 2.2]
  6164. receiving
  6165. [0130 18:01:08 @multigpu.py:323] [p0576] step: count(1305), step_time 1397.55, mean_step_time 1422.75, it/s 0.7
  6166. [0130 18:01:08 @multigpu.py:323] [p0574] step: count(1296), step_time 1387.43, mean_step_time 1424.26, it/s 0.7
  6167. [0130 18:01:08 @multigpu.py:323] [p0115] step: count(1281), step_time 1412.38, mean_step_time 1435.9, it/s 0.7
  6168. [0130 18:01:09 @multigpu.py:323] [p0576] step: count(1306), step_time 1436.32, mean_step_time 1425.0, it/s 0.7
  6169. [0130 18:01:09 @multigpu.py:323] [p0574] step: count(1297), step_time 1474.04, mean_step_time 1425.2, it/s 0.7
  6170. [0130 18:01:09 @multigpu.py:323] [p0115] step: count(1282), step_time 1413.31, mean_step_time 1435.09, it/s 0.7
  6171. [0130 18:01:10 @multigpu.py:323] [p0576] step: count(1307), step_time 1420.68, mean_step_time 1425.11, it/s 0.7
  6172. [0130 18:01:11 @multigpu.py:323] [p0115] step: count(1283), step_time 1401.14, mean_step_time 1434.3, it/s 0.7
  6173. [0130 18:01:11 @multigpu.py:323] [p0574] step: count(1298), step_time 1436.18, mean_step_time 1427.56, it/s 0.7
  6174. [0130 18:01:12 @multigpu.py:323] [p0576] step: count(1308), step_time 1451.23, mean_step_time 1428.52, it/s 0.7
  6175. [0130 18:01:12 @multigpu.py:323] [p0574] step: count(1299), step_time 1454.11, mean_step_time 1426.15, it/s 0.7
  6176. [0130 18:01:12 @multigpu.py:323] [p0115] step: count(1284), step_time 1456.81, mean_step_time 1435.44, it/s 0.7
  6177. [0130 18:01:13 @multigpu.py:323] [p0576] step: count(1309), step_time 1384.38, mean_step_time 1427.12, it/s 0.7
  6178. [0130 18:01:13 @multigpu.py:323] [p0115] step: count(1285), step_time 1412.14, mean_step_time 1433.18, it/s 0.7
  6179. [0130 18:01:14 @multigpu.py:323] [p0574] step: count(1300), step_time 1449.74, mean_step_time 1430.16, it/s 0.7
  6180. sending debugging info...
  6181. sending to address tcp://p0112:61216
  6182. ##### Sending to neptune: mean_delay : 0.520251732734 , 0.0 #####
  6183. sending to address tcp://p0112:61216
  6184. ##### Sending to neptune: max_delay : 0.520251732734 , -0.0 #####
  6185. ##### Sending to neptune: min_delay : 0.520251732734 , -0.0 #####
  6186. [u'delays', [0.0, -0.0, -0.0]]
  6187. receiving
  6188. ##### Sending to neptune: cost : 0.520252199107 , -0.00133042002562 #####
  6189. sending to address tcp://p0112:61216
  6190. ##### Sending to neptune: policy_loss : 0.520252199107 , 0.530481159687 #####
  6191. ##### Sending to neptune: xentropy_loss : 0.520252199107 , -2.28754925728 #####
  6192. ##### Sending to neptune: value_loss : 0.520252199107 , 1.58677446842 #####
  6193. ##### Sending to neptune: advantage : 0.520252199107 , -0.0023368424736 #####
  6194. ##### Sending to neptune: pred_reward : 0.520252199107 , 0.480544269085 #####
  6195. ##### Sending to neptune: max_logit : 0.520252199107 , 0.196291729808 #####
  6196. [u'loss', -0.0013304200256243348, 0.5304811596870422, -2.2875492572784424, 1.586774468421936, -0.0023368424735963345, 0.4805442690849304, 0.1962917298078537]
  6197. receiving
  6198. ##### Sending to neptune: active_relus : 0.52025263634 , 9689908.7 #####
  6199. ##### Sending to neptune: dp_per_s : 0.52025263634 , 89.2147275754 #####
  6200. [u'other', 9689908.7, 89.21472757536068]
  6201. receiving
  6202. sending to address tcp://p0112:61216
  6203. ##### Sending to neptune: online_score : 0.520315144393 , 1.5 #####
  6204. [u'online', 1.5]
  6205. receiving
  6206. [0130 18:01:15 @multigpu.py:323] [p0576] step: count(1310), step_time 1453.13, mean_step_time 1427.08, it/s 0.7
  6207. [0130 18:01:15 @multigpu.py:323] [p0115] step: count(1286), step_time 1374.66, mean_step_time 1428.61, it/s 0.7
  6208. [0130 18:01:15 @multigpu.py:323] [p0574] step: count(1301), step_time 1403.41, mean_step_time 1428.87, it/s 0.7
  6209. [0130 18:01:16 @multigpu.py:323] [p0576] step: count(1311), step_time 1410.4, mean_step_time 1426.7, it/s 0.7
  6210. [0130 18:01:16 @multigpu.py:323] [p0574] step: count(1302), step_time 1379.33, mean_step_time 1425.16, it/s 0.7
  6211. [0130 18:01:16 @multigpu.py:323] [p0115] step: count(1287), step_time 1459.62, mean_step_time 1428.2, it/s 0.7
  6212. [0130 18:01:18 @multigpu.py:323] [p0115] step: count(1288), step_time 1434.8, mean_step_time 1427.28, it/s 0.7
  6213. [0130 18:01:18 @multigpu.py:323] [p0574] step: count(1303), step_time 1451.91, mean_step_time 1427.3, it/s 0.7
  6214. [0130 18:01:18 @multigpu.py:323] [p0576] step: count(1312), step_time 1563.34, mean_step_time 1427.37, it/s 0.7
  6215. [0130 18:01:19 @multigpu.py:323] [p0115] step: count(1289), step_time 1395.34, mean_step_time 1426.33, it/s 0.7
  6216. [0130 18:01:19 @multigpu.py:323] [p0576] step: count(1313), step_time 1409.53, mean_step_time 1427.93, it/s 0.7
  6217. [0130 18:01:19 @multigpu.py:323] [p0574] step: count(1304), step_time 1418.34, mean_step_time 1426.41, it/s 0.7
  6218. [0130 18:01:21 @multigpu.py:323] [p0574] step: count(1305), step_time 1398.31, mean_step_time 1424.29, it/s 0.7
  6219. [0130 18:01:21 @multigpu.py:323] [p0576] step: count(1314), step_time 1409.09, mean_step_time 1425.96, it/s 0.7
  6220. [0130 18:01:21 @multigpu.py:323] [p0115] step: count(1290), step_time 1429.8, mean_step_time 1428.46, it/s 0.7
  6221. [0130 18:01:22 @multigpu.py:323] [p0576] step: count(1315), step_time 1402.83, mean_step_time 1423.59, it/s 0.7
  6222. [0130 18:01:22 @multigpu.py:323] [p0115] step: count(1291), step_time 1396.94, mean_step_time 1427.63, it/s 0.7
  6223. [0130 18:01:22 @multigpu.py:323] [p0574] step: count(1306), step_time 1445.26, mean_step_time 1426.98, it/s 0.7
  6224. [0130 18:01:23 @multigpu.py:323] [p0576] step: count(1316), step_time 1414.48, mean_step_time 1423.24, it/s 0.7
  6225. [0130 18:01:23 @multigpu.py:323] [p0115] step: count(1292), step_time 1442.41, mean_step_time 1428.47, it/s 0.7
  6226. [0130 18:01:23 @multigpu.py:323] [p0574] step: count(1307), step_time 1436.64, mean_step_time 1428.84, it/s 0.7
  6227. [0130 18:01:25 @multigpu.py:323] [p0115] step: count(1293), step_time 1459.15, mean_step_time 1430.72, it/s 0.7
  6228. [0130 18:01:25 @multigpu.py:323] [p0574] step: count(1308), step_time 1428.54, mean_step_time 1428.22, it/s 0.7
  6229. [0130 18:01:25 @multigpu.py:323] [p0576] step: count(1317), step_time 1506.03, mean_step_time 1428.46, it/s 0.7
  6230. [0130 18:01:26 @multigpu.py:323] [p0115] step: count(1294), step_time 1388.66, mean_step_time 1428.68, it/s 0.7
  6231. [0130 18:01:26 @multigpu.py:323] [p0574] step: count(1309), step_time 1408.21, mean_step_time 1428.21, it/s 0.7
  6232. [0130 18:01:26 @multigpu.py:323] [p0576] step: count(1318), step_time 1477.43, mean_step_time 1433.01, it/s 0.7
  6233. sending to address tcp://p0112:61216
  6234. ##### Sending to neptune: online_score : 0.523968138827 , 0.8 #####
  6235. [u'online', 0.8]
  6236. ##### Sending to neptune: active_workers : 0.523968183862 , 3 #####
  6237. receiving
  6238. [0130 18:01:28 @multigpu.py:323] [p0574] step: count(1310), step_time 1415.15, mean_step_time 1424.94, it/s 0.7
  6239. [0130 18:01:28 @multigpu.py:323] [p0115] step: count(1295), step_time 1442.54, mean_step_time 1427.63, it/s 0.7
  6240. [0130 18:01:28 @multigpu.py:323] [p0576] step: count(1319), step_time 1366.87, mean_step_time 1431.16, it/s 0.7
  6241. [0130 18:01:29 @multigpu.py:323] [p0115] step: count(1296), step_time 1352.47, mean_step_time 1421.97, it/s 0.7
  6242. [0130 18:01:29 @multigpu.py:323] [p0576] step: count(1320), step_time 1396.82, mean_step_time 1429.48, it/s 0.7
  6243. [0130 18:01:29 @multigpu.py:323] [p0574] step: count(1311), step_time 1455.72, mean_step_time 1427.8, it/s 0.7
  6244. [0130 18:01:31 @multigpu.py:323] [p0115] step: count(1297), step_time 1460.09, mean_step_time 1422.09, it/s 0.7
  6245. [0130 18:01:31 @multigpu.py:323] [p0576] step: count(1321), step_time 1402.61, mean_step_time 1427.7, it/s 0.7
  6246. [0130 18:01:31 @multigpu.py:323] [p0574] step: count(1312), step_time 1426.71, mean_step_time 1427.19, it/s 0.7
  6247. [0130 18:01:32 @multigpu.py:323] [p0576] step: count(1322), step_time 1393.86, mean_step_time 1428.15, it/s 0.7
  6248. [0130 18:01:32 @multigpu.py:323] [p0115] step: count(1298), step_time 1479.78, mean_step_time 1425.38, it/s 0.7
  6249. [0130 18:01:32 @multigpu.py:323] [p0574] step: count(1313), step_time 1424.25, mean_step_time 1425.79, it/s 0.7
  6250. sending to address tcp://p0112:61216
  6251. ##### Sending to neptune: online_score : 0.525484263566 , 2.1 #####
  6252. [u'online', 2.1]
  6253. receiving
  6254. [0130 18:01:33 @multigpu.py:323] [p0576] step: count(1323), step_time 1420.64, mean_step_time 1429.19, it/s 0.7
  6255. [0130 18:01:33 @multigpu.py:323] [p0115] step: count(1299), step_time 1479.2, mean_step_time 1425.29, it/s 0.7
  6256. [0130 18:01:34 @multigpu.py:323] [p0574] step: count(1314), step_time 1495.65, mean_step_time 1431.74, it/s 0.7
  6257. [0130 18:01:35 @multigpu.py:323] [p0576] step: count(1324), step_time 1431.46, mean_step_time 1427.43, it/s 0.7
  6258. [0130 18:01:35 @multigpu.py:323] [p0574] step: count(1315), step_time 1379.58, mean_step_time 1428.43, it/s 0.7
  6259. [0130 18:01:35 @multigpu.py:323] [p0115] step: count(1300), step_time 1504.77, mean_step_time 1429.8, it/s 0.7
  6260. sending debugging info...
  6261. sending to address tcp://p0112:61216
  6262. ##### Sending to neptune: mean_delay : 0.526216461658 , 0.0 #####
  6263. sending to address tcp://p0112:61216
  6264. ##### Sending to neptune: max_delay : 0.526216461658 , -0.0 #####
  6265. ##### Sending to neptune: min_delay : 0.526216461658 , -0.0 #####
  6266. [u'delays', [0.0, -0.0, -0.0]]
  6267. receiving
  6268. ##### Sending to neptune: cost : 0.526216974987 , -0.0132103003561 #####
  6269. sending to address tcp://p0112:61216
  6270. ##### Sending to neptune: policy_loss : 0.526216974987 , -0.767839074135 #####
  6271. ##### Sending to neptune: xentropy_loss : 0.526216974987 , -2.2872877121 #####
  6272. ##### Sending to neptune: value_loss : 0.526216974987 , 1.36420881748 #####
  6273. ##### Sending to neptune: advantage : 0.526216974987 , 0.00328848976642 #####
  6274. ##### Sending to neptune: pred_reward : 0.526216974987 , 0.47394412756 #####
  6275. ##### Sending to neptune: max_logit : 0.526216974987 , 0.197944834828 #####
  6276. [u'loss', -0.013210300356149673, -0.7678390741348267, -2.287287712097168, 1.3642088174819946, 0.003288489766418934, 0.47394412755966187, 0.19794483482837677]
  6277. receiving
  6278. ##### Sending to neptune: active_relus : 0.526217454142 , 9719758.83 #####
  6279. ##### Sending to neptune: dp_per_s : 0.526217454142 , 89.3935747297 #####
  6280. [u'other', 9719758.83, 89.39357472974253]
  6281. receiving
  6282. [0130 18:01:36 @multigpu.py:323] [p0576] step: count(1325), step_time 1412.64, mean_step_time 1428.19, it/s 0.7
  6283. [0130 18:01:36 @multigpu.py:323] [p0574] step: count(1316), step_time 1420.66, mean_step_time 1430.09, it/s 0.7
  6284. [0130 18:01:36 @multigpu.py:323] [p0115] step: count(1301), step_time 1411.45, mean_step_time 1429.75, it/s 0.7
  6285. [0130 18:01:38 @multigpu.py:323] [p0576] step: count(1326), step_time 1386.73, mean_step_time 1425.71, it/s 0.7
  6286. [0130 18:01:38 @multigpu.py:323] [p0574] step: count(1317), step_time 1391.52, mean_step_time 1425.96, it/s 0.7
  6287. [0130 18:01:38 @multigpu.py:323] [p0115] step: count(1302), step_time 1399.21, mean_step_time 1429.05, it/s 0.7
  6288. sending to address tcp://p0112:61216
  6289. ##### Sending to neptune: online_score : 0.527047368553 , 2.0 #####
  6290. [u'online', 2.0]
  6291. receiving
  6292. [0130 18:01:39 @multigpu.py:323] [p0576] step: count(1327), step_time 1457.68, mean_step_time 1427.56, it/s 0.7
  6293. [0130 18:01:39 @multigpu.py:323] [p0574] step: count(1318), step_time 1424.61, mean_step_time 1425.38, it/s 0.7
  6294. [0130 18:01:39 @multigpu.py:323] [p0115] step: count(1303), step_time 1447.28, mean_step_time 1431.36, it/s 0.7
  6295. [0130 18:01:41 @multigpu.py:323] [p0576] step: count(1328), step_time 1488.34, mean_step_time 1429.41, it/s 0.7
  6296. [0130 18:01:41 @multigpu.py:323] [p0574] step: count(1319), step_time 1429.17, mean_step_time 1424.14, it/s 0.7
  6297. [0130 18:01:41 @multigpu.py:323] [p0115] step: count(1304), step_time 1399.1, mean_step_time 1428.47, it/s 0.7
  6298. [0130 18:01:42 @multigpu.py:323] [p0576] step: count(1329), step_time 1403.8, mean_step_time 1430.39, it/s 0.7
  6299. [0130 18:01:42 @multigpu.py:323] [p0574] step: count(1320), step_time 1445.3, mean_step_time 1423.91, it/s 0.7
  6300. [0130 18:01:42 @multigpu.py:323] [p0115] step: count(1305), step_time 1497.48, mean_step_time 1432.74, it/s 0.7
  6301. [0130 18:01:43 @multigpu.py:323] [p0576] step: count(1330), step_time 1451.89, mean_step_time 1430.32, it/s 0.7
  6302. [0130 18:01:43 @multigpu.py:323] [p0574] step: count(1321), step_time 1398.76, mean_step_time 1423.68, it/s 0.7
  6303. [0130 18:01:44 @multigpu.py:323] [p0115] step: count(1306), step_time 1401.08, mean_step_time 1434.06, it/s 0.7
  6304. [0130 18:01:45 @multigpu.py:323] [p0576] step: count(1331), step_time 1440.01, mean_step_time 1431.8, it/s 0.7
  6305. [0130 18:01:45 @multigpu.py:323] [p0574] step: count(1322), step_time 1454.99, mean_step_time 1427.46, it/s 0.7
  6306. [0130 18:01:45 @multigpu.py:323] [p0115] step: count(1307), step_time 1458.27, mean_step_time 1433.99, it/s 0.7
  6307. [0130 18:01:46 @multigpu.py:323] [p0574] step: count(1323), step_time 1604.47, mean_step_time 1435.09, it/s 0.7
  6308. [0130 18:01:46 @multigpu.py:323] [p0576] step: count(1332), step_time 1627.95, mean_step_time 1435.03, it/s 0.7
  6309. [0130 18:01:46 @multigpu.py:323] [p0115] step: count(1308), step_time 1458.09, mean_step_time 1435.16, it/s 0.7
  6310. [0130 18:01:48 @multigpu.py:323] [p0576] step: count(1333), step_time 1431.92, mean_step_time 1436.15, it/s 0.7
  6311. [0130 18:01:48 @multigpu.py:323] [p0574] step: count(1324), step_time 1443.86, mean_step_time 1436.37, it/s 0.7
  6312. [0130 18:01:48 @multigpu.py:323] [p0115] step: count(1309), step_time 1452.55, mean_step_time 1438.02, it/s 0.7
  6313. [0130 18:01:49 @multigpu.py:323] [p0115] step: count(1310), step_time 1416.31, mean_step_time 1437.34, it/s 0.7
  6314. [0130 18:01:49 @multigpu.py:323] [p0576] step: count(1334), step_time 1437.23, mean_step_time 1437.56, it/s 0.7
  6315. [0130 18:01:49 @multigpu.py:323] [p0574] step: count(1325), step_time 1444.67, mean_step_time 1438.69, it/s 0.7
  6316. [0130 18:01:51 @multigpu.py:323] [p0574] step: count(1326), step_time 1445.52, mean_step_time 1438.7, it/s 0.7
  6317. [0130 18:01:51 @multigpu.py:323] [p0576] step: count(1335), step_time 1479.94, mean_step_time 1441.42, it/s 0.69
  6318. [0130 18:01:51 @multigpu.py:323] [p0115] step: count(1311), step_time 1513.3, mean_step_time 1443.16, it/s 0.69
  6319. sending to address tcp://p0112:61216
  6320. ##### Sending to neptune: online_score : 0.530835908585 , 1.7 #####
  6321. [u'online', 1.7]
  6322. receiving
  6323. [0130 18:01:52 @multigpu.py:323] [p0574] step: count(1327), step_time 1439.03, mean_step_time 1438.82, it/s 0.7
  6324. [0130 18:01:52 @multigpu.py:323] [p0115] step: count(1312), step_time 1434.38, mean_step_time 1442.76, it/s 0.69
  6325. [0130 18:01:52 @multigpu.py:323] [p0576] step: count(1336), step_time 1471.0, mean_step_time 1444.24, it/s 0.69
  6326. [0130 18:01:54 @multigpu.py:323] [p0574] step: count(1328), step_time 1425.5, mean_step_time 1438.67, it/s 0.7
  6327. [0130 18:01:54 @multigpu.py:323] [p0115] step: count(1313), step_time 1428.68, mean_step_time 1441.23, it/s 0.69
  6328. [0130 18:01:54 @multigpu.py:323] [p0576] step: count(1337), step_time 1450.34, mean_step_time 1441.46, it/s 0.69
  6329. [0130 18:01:55 @multigpu.py:323] [p0574] step: count(1329), step_time 1386.17, mean_step_time 1437.56, it/s 0.7
  6330. [0130 18:01:55 @multigpu.py:323] [p0115] step: count(1314), step_time 1403.13, mean_step_time 1441.96, it/s 0.69
  6331. [0130 18:01:55 @multigpu.py:323] [p0576] step: count(1338), step_time 1405.76, mean_step_time 1437.87, it/s 0.7
  6332. [0130 18:01:57 @multigpu.py:323] [p0574] step: count(1330), step_time 1465.31, mean_step_time 1440.07, it/s 0.69
  6333. [0130 18:01:57 @multigpu.py:323] [p0576] step: count(1339), step_time 1444.95, mean_step_time 1441.78, it/s 0.69
  6334. [0130 18:01:57 @multigpu.py:323] [p0115] step: count(1315), step_time 1514.84, mean_step_time 1445.57, it/s 0.69
  6335. [0130 18:01:58 @multigpu.py:323] [p0574] step: count(1331), step_time 1451.43, mean_step_time 1439.86, it/s 0.69
  6336. [0130 18:01:58 @multigpu.py:323] [p0576] step: count(1340), step_time 1451.64, mean_step_time 1444.52, it/s 0.69
  6337. [0130 18:01:58 @multigpu.py:323] [p0115] step: count(1316), step_time 1430.77, mean_step_time 1449.49, it/s 0.69
  6338. [0130 18:01:59 @multigpu.py:323] [p0574] step: count(1332), step_time 1405.66, mean_step_time 1438.8, it/s 0.7
  6339. [0130 18:01:59 @multigpu.py:323] [p0576] step: count(1341), step_time 1411.03, mean_step_time 1444.94, it/s 0.69
  6340. [0130 18:02:00 @multigpu.py:323] [p0115] step: count(1317), step_time 1455.08, mean_step_time 1449.24, it/s 0.69
  6341. [0130 18:02:01 @multigpu.py:323] [p0574] step: count(1333), step_time 1401.36, mean_step_time 1437.66, it/s 0.7
  6342. [0130 18:02:01 @multigpu.py:323] [p0576] step: count(1342), step_time 1452.52, mean_step_time 1447.87, it/s 0.69
  6343. [0130 18:02:01 @multigpu.py:323] [p0115] step: count(1318), step_time 1443.73, mean_step_time 1447.43, it/s 0.69
  6344. [0130 18:02:02 @multigpu.py:323] [p0574] step: count(1334), step_time 1450.79, mean_step_time 1435.42, it/s 0.7
  6345. [0130 18:02:02 @multigpu.py:323] [p0576] step: count(1343), step_time 1401.61, mean_step_time 1446.92, it/s 0.69
  6346. [0130 18:02:02 @multigpu.py:323] [p0115] step: count(1319), step_time 1411.88, mean_step_time 1444.07, it/s 0.69
  6347. sending to address tcp://p0112:61216
  6348. ##### Sending to neptune: online_score : 0.533943954706 , 1.5 #####
  6349. [u'online', 1.5]
  6350. receiving
  6351. sending to address tcp://p0112:61216
  6352. ##### Sending to neptune: online_score : 0.533990806341 , 1.0 #####
  6353. [u'online', 1.0]
  6354. receiving
  6355. [0130 18:02:04 @multigpu.py:323] [p0574] step: count(1335), step_time 1388.52, mean_step_time 1435.86, it/s 0.7
  6356. [0130 18:02:04 @multigpu.py:323] [p0115] step: count(1320), step_time 1406.04, mean_step_time 1439.13, it/s 0.69
  6357. [0130 18:02:04 @multigpu.py:323] [p0576] step: count(1344), step_time 1489.33, mean_step_time 1449.82, it/s 0.69
  6358. [0130 18:02:05 @multigpu.py:323] [p0574] step: count(1336), step_time 1413.78, mean_step_time 1435.52, it/s 0.7
  6359. [0130 18:02:05 @multigpu.py:323] [p0115] step: count(1321), step_time 1403.22, mean_step_time 1438.72, it/s 0.7
  6360. [0130 18:02:05 @multigpu.py:323] [p0576] step: count(1345), step_time 1422.62, mean_step_time 1450.32, it/s 0.69
  6361. [0130 18:02:06 @multigpu.py:323] [p0574] step: count(1337), step_time 1472.75, mean_step_time 1439.58, it/s 0.69
  6362. [0130 18:02:07 @multigpu.py:323] [p0115] step: count(1322), step_time 1451.03, mean_step_time 1441.31, it/s 0.69
  6363. [0130 18:02:07 @multigpu.py:323] [p0576] step: count(1346), step_time 1488.87, mean_step_time 1455.42, it/s 0.69
  6364. [0130 18:02:08 @multigpu.py:323] [p0574] step: count(1338), step_time 1503.68, mean_step_time 1443.54, it/s 0.69
  6365. [0130 18:02:08 @multigpu.py:323] [p0115] step: count(1323), step_time 1434.57, mean_step_time 1440.68, it/s 0.69
  6366. [0130 18:02:08 @multigpu.py:323] [p0576] step: count(1347), step_time 1404.22, mean_step_time 1452.75, it/s 0.69
  6367. [0130 18:02:09 @multigpu.py:323] [p0574] step: count(1339), step_time 1424.48, mean_step_time 1443.3, it/s 0.69
  6368. [0130 18:02:09 @multigpu.py:323] [p0115] step: count(1324), step_time 1374.08, mean_step_time 1439.42, it/s 0.69
  6369. [0130 18:02:10 @multigpu.py:323] [p0576] step: count(1348), step_time 1401.08, mean_step_time 1448.39, it/s 0.69
  6370. [0130 18:02:11 @multigpu.py:323] [p0574] step: count(1340), step_time 1442.77, mean_step_time 1443.18, it/s 0.69
  6371. [0130 18:02:11 @multigpu.py:323] [p0115] step: count(1325), step_time 1459.87, mean_step_time 1437.54, it/s 0.7
  6372. [0130 18:02:11 @multigpu.py:323] [p0576] step: count(1349), step_time 1440.85, mean_step_time 1450.24, it/s 0.69
  6373. [0130 18:02:12 @multigpu.py:323] [p0574] step: count(1341), step_time 1431.65, mean_step_time 1444.82, it/s 0.69
  6374. [0130 18:02:12 @multigpu.py:323] [p0115] step: count(1326), step_time 1427.27, mean_step_time 1438.85, it/s 0.69
  6375. [0130 18:02:12 @multigpu.py:323] [p0576] step: count(1350), step_time 1395.23, mean_step_time 1447.41, it/s 0.69
  6376. [0130 18:02:14 @multigpu.py:323] [p0115] step: count(1327), step_time 1408.36, mean_step_time 1436.36, it/s 0.7
  6377. [0130 18:02:14 @multigpu.py:323] [p0574] step: count(1342), step_time 1472.65, mean_step_time 1445.7, it/s 0.69
  6378. [0130 18:02:14 @multigpu.py:323] [p0576] step: count(1351), step_time 1438.57, mean_step_time 1447.33, it/s 0.69
  6379. [0130 18:02:15 @multigpu.py:323] [p0115] step: count(1328), step_time 1482.76, mean_step_time 1437.59, it/s 0.7
  6380. [0130 18:02:15 @multigpu.py:323] [p0576] step: count(1352), step_time 1433.1, mean_step_time 1437.59, it/s 0.7
  6381. [0130 18:02:15 @multigpu.py:323] [p0574] step: count(1343), step_time 1443.32, mean_step_time 1437.65, it/s 0.7
  6382. [0130 18:02:17 @multigpu.py:323] [p0574] step: count(1344), step_time 1387.03, mean_step_time 1434.8, it/s 0.7
  6383. [0130 18:02:17 @multigpu.py:323] [p0576] step: count(1353), step_time 1433.47, mean_step_time 1437.67, it/s 0.7
  6384. [0130 18:02:17 @multigpu.py:323] [p0115] step: count(1329), step_time 1459.3, mean_step_time 1437.93, it/s 0.7
  6385. [0130 18:02:18 @multigpu.py:323] [p0576] step: count(1354), step_time 1409.46, mean_step_time 1436.28, it/s 0.7
  6386. [0130 18:02:18 @multigpu.py:323] [p0574] step: count(1345), step_time 1457.71, mean_step_time 1435.46, it/s 0.7
  6387. [0130 18:02:18 @multigpu.py:323] [p0115] step: count(1330), step_time 1426.61, mean_step_time 1438.44, it/s 0.7
  6388. [0130 18:02:19 @multigpu.py:323] [p0574] step: count(1346), step_time 1415.38, mean_step_time 1433.95, it/s 0.7
  6389. [0130 18:02:20 @multigpu.py:323] [p0576] step: count(1355), step_time 1492.05, mean_step_time 1436.89, it/s 0.7
  6390. [0130 18:02:20 @multigpu.py:323] [p0115] step: count(1331), step_time 1460.89, mean_step_time 1435.82, it/s 0.7
  6391. [0130 18:02:21 @multigpu.py:323] [p0574] step: count(1347), step_time 1426.43, mean_step_time 1433.32, it/s 0.7
  6392. [0130 18:02:21 @multigpu.py:323] [p0115] step: count(1332), step_time 1375.86, mean_step_time 1432.9, it/s 0.7
  6393. [0130 18:02:21 @multigpu.py:323] [p0576] step: count(1356), step_time 1408.51, mean_step_time 1433.76, it/s 0.7
  6394. [0130 18:02:22 @multigpu.py:323] [p0574] step: count(1348), step_time 1386.07, mean_step_time 1431.35, it/s 0.7
  6395. [0130 18:02:22 @multigpu.py:323] [p0115] step: count(1333), step_time 1430.46, mean_step_time 1432.99, it/s 0.7
  6396. [0130 18:02:22 @multigpu.py:323] [p0576] step: count(1357), step_time 1415.37, mean_step_time 1432.01, it/s 0.7
  6397. [0130 18:02:24 @multigpu.py:323] [p0574] step: count(1349), step_time 1406.08, mean_step_time 1432.34, it/s 0.7
  6398. [0130 18:02:24 @multigpu.py:323] [p0576] step: count(1358), step_time 1453.69, mean_step_time 1434.41, it/s 0.7
  6399. [0130 18:02:24 @multigpu.py:323] [p0115] step: count(1334), step_time 1488.36, mean_step_time 1437.25, it/s 0.7
  6400. [0130 18:02:25 @multigpu.py:323] [p0574] step: count(1350), step_time 1441.79, mean_step_time 1431.17, it/s 0.7
  6401. [0130 18:02:25 @multigpu.py:323] [p0576] step: count(1359), step_time 1387.2, mean_step_time 1431.52, it/s 0.7
  6402. [0130 18:02:25 @multigpu.py:323] [p0115] step: count(1335), step_time 1428.06, mean_step_time 1432.91, it/s 0.7
  6403. [0130 18:02:27 @multigpu.py:323] [p0574] step: count(1351), step_time 1414.6, mean_step_time 1429.33, it/s 0.7
  6404. [0130 18:02:27 @multigpu.py:323] [p0115] step: count(1336), step_time 1423.21, mean_step_time 1432.53, it/s 0.7
  6405. [0130 18:02:27 @multigpu.py:323] [p0576] step: count(1360), step_time 1503.81, mean_step_time 1434.13, it/s 0.7
  6406. sending to address tcp://p0112:61216
  6407. ##### Sending to neptune: online_score : 0.540751250254 , 1.5 #####
  6408. [u'online', 1.5]
  6409. ##### Sending to neptune: active_workers : 0.540751353833 , 3 #####
  6410. receiving
  6411. [0130 18:02:28 @multigpu.py:323] [p0574] step: count(1352), step_time 1405.06, mean_step_time 1429.3, it/s 0.7
  6412. [0130 18:02:28 @multigpu.py:323] [p0576] step: count(1361), step_time 1404.19, mean_step_time 1433.79, it/s 0.7
  6413. [0130 18:02:28 @multigpu.py:323] [p0115] step: count(1337), step_time 1450.23, mean_step_time 1432.29, it/s 0.7
  6414. [0130 18:02:29 @multigpu.py:323] [p0574] step: count(1353), step_time 1415.19, mean_step_time 1429.99, it/s 0.7
  6415. [0130 18:02:30 @multigpu.py:323] [p0576] step: count(1362), step_time 1451.61, mean_step_time 1433.74, it/s 0.7
  6416. [0130 18:02:30 @multigpu.py:323] [p0115] step: count(1338), step_time 1441.16, mean_step_time 1432.16, it/s 0.7
  6417. sending to address tcp://p0112:61216
  6418. ##### Sending to neptune: online_score : 0.541542358332 , 1.7 #####
  6419. [u'online', 1.7]
  6420. receiving
  6421. [0130 18:02:31 @multigpu.py:323] [p0574] step: count(1354), step_time 1390.2, mean_step_time 1426.96, it/s 0.7
  6422. [0130 18:02:31 @multigpu.py:323] [p0576] step: count(1363), step_time 1408.87, mean_step_time 1434.11, it/s 0.7
  6423. [0130 18:02:31 @multigpu.py:323] [p0115] step: count(1339), step_time 1413.46, mean_step_time 1432.24, it/s 0.7
  6424. [0130 18:02:32 @multigpu.py:323] [p0574] step: count(1355), step_time 1440.46, mean_step_time 1429.55, it/s 0.7
  6425. [0130 18:02:32 @multigpu.py:323] [p0576] step: count(1364), step_time 1434.43, mean_step_time 1431.36, it/s 0.7
  6426. [0130 18:02:33 @multigpu.py:323] [p0115] step: count(1340), step_time 1494.65, mean_step_time 1436.67, it/s 0.7
  6427. sending to address tcp://p0112:61216
  6428. ##### Sending to neptune: online_score : 0.542352651358 , 1.7 #####
  6429. [u'online', 1.7]
  6430. receiving
  6431. [0130 18:02:34 @multigpu.py:323] [p0574] step: count(1356), step_time 1404.31, mean_step_time 1429.08, it/s 0.7
  6432. [0130 18:02:34 @multigpu.py:323] [p0576] step: count(1365), step_time 1415.21, mean_step_time 1430.99, it/s 0.7
  6433. [0130 18:02:34 @multigpu.py:323] [p0115] step: count(1341), step_time 1412.43, mean_step_time 1437.13, it/s 0.7
  6434. [0130 18:02:35 @multigpu.py:323] [p0574] step: count(1357), step_time 1387.35, mean_step_time 1424.81, it/s 0.7
  6435. [0130 18:02:35 @multigpu.py:323] [p0576] step: count(1366), step_time 1408.33, mean_step_time 1426.96, it/s 0.7
  6436. [0130 18:02:35 @multigpu.py:323] [p0115] step: count(1342), step_time 1432.7, mean_step_time 1436.21, it/s 0.7
  6437. [0130 18:02:36 @multigpu.py:323] [p0574] step: count(1358), step_time 1415.84, mean_step_time 1420.42, it/s 0.7
  6438. [0130 18:02:37 @multigpu.py:323] [p0576] step: count(1367), step_time 1433.29, mean_step_time 1428.42, it/s 0.7
  6439. [0130 18:02:37 @multigpu.py:323] [p0115] step: count(1343), step_time 1448.73, mean_step_time 1436.92, it/s 0.7
  6440. [0130 18:02:38 @multigpu.py:323] [p0574] step: count(1359), step_time 1483.98, mean_step_time 1423.39, it/s 0.7
  6441. [0130 18:02:38 @multigpu.py:323] [p0576] step: count(1368), step_time 1436.63, mean_step_time 1430.19, it/s 0.7
  6442. [0130 18:02:38 @multigpu.py:323] [p0115] step: count(1344), step_time 1448.18, mean_step_time 1440.63, it/s 0.69
  6443. [0130 18:02:39 @multigpu.py:323] [p0574] step: count(1360), step_time 1426.99, mean_step_time 1422.6, it/s 0.7
  6444. [0130 18:02:40 @multigpu.py:323] [p0576] step: count(1369), step_time 1404.52, mean_step_time 1428.38, it/s 0.7
  6445. [0130 18:02:40 @multigpu.py:323] [p0115] step: count(1345), step_time 1413.7, mean_step_time 1438.32, it/s 0.7
  6446. [0130 18:02:41 @multigpu.py:323] [p0574] step: count(1361), step_time 1372.54, mean_step_time 1419.65, it/s 0.7
  6447. [0130 18:02:41 @multigpu.py:323] [p0576] step: count(1370), step_time 1417.31, mean_step_time 1429.48, it/s 0.7
  6448. [0130 18:02:41 @multigpu.py:323] [p0115] step: count(1346), step_time 1437.96, mean_step_time 1438.85, it/s 0.69
  6449. [0130 18:02:42 @multigpu.py:323] [p0574] step: count(1362), step_time 1486.76, mean_step_time 1420.35, it/s 0.7
  6450. [0130 18:02:42 @multigpu.py:323] [p0576] step: count(1371), step_time 1376.86, mean_step_time 1426.4, it/s 0.7
  6451. [0130 18:02:43 @multigpu.py:323] [p0115] step: count(1347), step_time 1406.57, mean_step_time 1438.76, it/s 0.7
  6452. [0130 18:02:44 @multigpu.py:323] [p0115] step: count(1348), step_time 1442.11, mean_step_time 1436.73, it/s 0.7
  6453. [0130 18:02:44 @multigpu.py:323] [p0576] step: count(1372), step_time 1638.05, mean_step_time 1436.64, it/s 0.7
  6454. [0130 18:02:44 @multigpu.py:323] [p0574] step: count(1363), step_time 1770.49, mean_step_time 1436.71, it/s 0.7
  6455. [0130 18:02:45 @multigpu.py:323] [p0576] step: count(1373), step_time 1429.14, mean_step_time 1436.43, it/s 0.7
  6456. [0130 18:02:45 @multigpu.py:323] [p0574] step: count(1364), step_time 1429.9, mean_step_time 1438.86, it/s 0.69
  6457. [0130 18:02:45 @multigpu.py:323] [p0115] step: count(1349), step_time 1432.33, mean_step_time 1435.38, it/s 0.7
  6458. [0130 18:02:47 @multigpu.py:323] [p0576] step: count(1374), step_time 1433.5, mean_step_time 1437.63, it/s 0.7
  6459. [0130 18:02:47 @multigpu.py:323] [p0574] step: count(1365), step_time 1432.46, mean_step_time 1437.59, it/s 0.7
  6460. [0130 18:02:47 @multigpu.py:323] [p0115] step: count(1350), step_time 1484.87, mean_step_time 1438.3, it/s 0.7
  6461. [0130 18:02:48 @multigpu.py:323] [p0574] step: count(1366), step_time 1406.44, mean_step_time 1437.15, it/s 0.7
  6462. [0130 18:02:48 @multigpu.py:323] [p0576] step: count(1375), step_time 1415.47, mean_step_time 1433.8, it/s 0.7
  6463. [0130 18:02:48 @multigpu.py:323] [p0115] step: count(1351), step_time 1411.37, mean_step_time 1435.82, it/s 0.7
  6464. [0130 18:02:50 @multigpu.py:323] [p0574] step: count(1367), step_time 1408.95, mean_step_time 1436.27, it/s 0.7
  6465. [0130 18:02:50 @multigpu.py:323] [p0576] step: count(1376), step_time 1485.84, mean_step_time 1437.67, it/s 0.7
  6466. [0130 18:02:50 @multigpu.py:323] [p0115] step: count(1352), step_time 1461.74, mean_step_time 1440.11, it/s 0.69
  6467. [0130 18:02:51 @multigpu.py:323] [p0574] step: count(1368), step_time 1430.87, mean_step_time 1438.51, it/s 0.7
  6468. [0130 18:02:51 @multigpu.py:323] [p0115] step: count(1353), step_time 1385.82, mean_step_time 1437.88, it/s 0.7
  6469. [0130 18:02:51 @multigpu.py:323] [p0576] step: count(1377), step_time 1434.55, mean_step_time 1438.63, it/s 0.7
  6470. sending to address tcp://p0112:61216
  6471. ##### Sending to neptune: online_score : 0.547551592191 , 1.8 #####
  6472. [u'online', 1.8]
  6473. receiving
  6474. [0130 18:02:52 @multigpu.py:323] [p0574] step: count(1369), step_time 1421.69, mean_step_time 1439.29, it/s 0.69
  6475. [0130 18:02:53 @multigpu.py:323] [p0576] step: count(1378), step_time 1407.45, mean_step_time 1436.31, it/s 0.7
  6476. [0130 18:02:53 @multigpu.py:323] [p0115] step: count(1354), step_time 1433.65, mean_step_time 1435.15, it/s 0.7
  6477. [0130 18:02:54 @multigpu.py:323] [p0574] step: count(1370), step_time 1417.29, mean_step_time 1438.07, it/s 0.7
  6478. [0130 18:02:54 @multigpu.py:323] [p0576] step: count(1379), step_time 1367.25, mean_step_time 1435.32, it/s 0.7
  6479. [0130 18:02:54 @multigpu.py:323] [p0115] step: count(1355), step_time 1423.47, mean_step_time 1434.92, it/s 0.7
  6480. [0130 18:02:55 @multigpu.py:323] [p0574] step: count(1371), step_time 1411.08, mean_step_time 1437.89, it/s 0.7
  6481. [0130 18:02:55 @multigpu.py:323] [p0576] step: count(1380), step_time 1443.27, mean_step_time 1432.29, it/s 0.7
  6482. [0130 18:02:55 @multigpu.py:323] [p0115] step: count(1356), step_time 1474.0, mean_step_time 1437.46, it/s 0.7
  6483. [0130 18:02:57 @multigpu.py:323] [p0576] step: count(1381), step_time 1368.81, mean_step_time 1430.52, it/s 0.7
  6484. [0130 18:02:57 @multigpu.py:323] [p0574] step: count(1372), step_time 1491.64, mean_step_time 1442.22, it/s 0.69
  6485. [0130 18:02:57 @multigpu.py:323] [p0115] step: count(1357), step_time 1414.76, mean_step_time 1435.68, it/s 0.7
  6486. [0130 18:02:58 @multigpu.py:323] [p0576] step: count(1382), step_time 1484.48, mean_step_time 1432.16, it/s 0.7
  6487. [0130 18:02:58 @multigpu.py:323] [p0574] step: count(1373), step_time 1429.34, mean_step_time 1442.93, it/s 0.69
  6488. [0130 18:02:58 @multigpu.py:323] [p0115] step: count(1358), step_time 1513.46, mean_step_time 1439.3, it/s 0.69
  6489. [0130 18:03:00 @multigpu.py:323] [p0576] step: count(1383), step_time 1430.51, mean_step_time 1433.25, it/s 0.7
  6490. [0130 18:03:00 @multigpu.py:323] [p0574] step: count(1374), step_time 1436.92, mean_step_time 1445.26, it/s 0.69
  6491. [0130 18:03:00 @multigpu.py:323] [p0115] step: count(1359), step_time 1418.1, mean_step_time 1439.53, it/s 0.69
  6492. [0130 18:03:01 @multigpu.py:323] [p0576] step: count(1384), step_time 1404.53, mean_step_time 1431.75, it/s 0.7
  6493. [0130 18:03:01 @multigpu.py:323] [p0574] step: count(1375), step_time 1408.26, mean_step_time 1443.65, it/s 0.69
  6494. [0130 18:03:01 @multigpu.py:323] [p0115] step: count(1360), step_time 1415.47, mean_step_time 1435.57, it/s 0.7
  6495. [0130 18:03:03 @multigpu.py:323] [p0576] step: count(1385), step_time 1442.41, mean_step_time 1433.11, it/s 0.7
  6496. [0130 18:03:03 @multigpu.py:323] [p0574] step: count(1376), step_time 1428.69, mean_step_time 1444.87, it/s 0.69
  6497. [0130 18:03:03 @multigpu.py:323] [p0115] step: count(1361), step_time 1467.4, mean_step_time 1438.32, it/s 0.7
  6498. sending to address tcp://p0112:61216
  6499. ##### Sending to neptune: online_score : 0.550689121617 , 1.9 #####
  6500. [u'online', 1.9]
  6501. receiving
  6502. sending to address tcp://p0112:61216
  6503. ##### Sending to neptune: online_score : 0.550788611637 , 2.2 #####
  6504. [u'online', 2.2]
  6505. receiving
  6506. [0130 18:03:04 @multigpu.py:323] [p0576] step: count(1386), step_time 1435.36, mean_step_time 1434.46, it/s 0.7
  6507. [0130 18:03:04 @multigpu.py:323] [p0574] step: count(1377), step_time 1426.97, mean_step_time 1446.86, it/s 0.69
  6508. [0130 18:03:04 @multigpu.py:323] [p0115] step: count(1362), step_time 1378.13, mean_step_time 1435.59, it/s 0.7
  6509. [0130 18:03:05 @multigpu.py:323] [p0574] step: count(1378), step_time 1406.52, mean_step_time 1446.39, it/s 0.69
  6510. [0130 18:03:05 @multigpu.py:323] [p0576] step: count(1387), step_time 1427.56, mean_step_time 1434.18, it/s 0.7
  6511. [0130 18:03:05 @multigpu.py:323] [p0115] step: count(1363), step_time 1406.02, mean_step_time 1433.46, it/s 0.7
  6512. [0130 18:03:07 @multigpu.py:323] [p0576] step: count(1388), step_time 1447.5, mean_step_time 1434.72, it/s 0.7
  6513. [0130 18:03:07 @multigpu.py:323] [p0574] step: count(1379), step_time 1476.22, mean_step_time 1446.0, it/s 0.69
  6514. [0130 18:03:07 @multigpu.py:323] [p0115] step: count(1364), step_time 1421.08, mean_step_time 1432.1, it/s 0.7
  6515. [0130 18:03:08 @multigpu.py:323] [p0576] step: count(1389), step_time 1389.55, mean_step_time 1433.97, it/s 0.7
  6516. [0130 18:03:08 @multigpu.py:323] [p0574] step: count(1380), step_time 1435.9, mean_step_time 1446.45, it/s 0.69
  6517. [0130 18:03:08 @multigpu.py:323] [p0115] step: count(1365), step_time 1455.47, mean_step_time 1434.19, it/s 0.7
  6518. [0130 18:03:10 @multigpu.py:323] [p0576] step: count(1390), step_time 1399.11, mean_step_time 1433.06, it/s 0.7
  6519. [0130 18:03:10 @multigpu.py:323] [p0115] step: count(1366), step_time 1419.1, mean_step_time 1433.25, it/s 0.7
  6520. [0130 18:03:10 @multigpu.py:323] [p0574] step: count(1381), step_time 1518.95, mean_step_time 1453.77, it/s 0.69
  6521. [0130 18:03:11 @multigpu.py:323] [p0576] step: count(1391), step_time 1394.68, mean_step_time 1433.95, it/s 0.7
  6522. [0130 18:03:11 @multigpu.py:323] [p0574] step: count(1382), step_time 1389.37, mean_step_time 1448.9, it/s 0.69
  6523. [0130 18:03:11 @multigpu.py:323] [p0115] step: count(1367), step_time 1411.2, mean_step_time 1433.48, it/s 0.7
  6524. [0130 18:03:13 @multigpu.py:323] [p0115] step: count(1368), step_time 1451.68, mean_step_time 1433.96, it/s 0.7
  6525. [0130 18:03:13 @multigpu.py:323] [p0576] step: count(1392), step_time 1637.72, mean_step_time 1433.93, it/s 0.7
  6526. [0130 18:03:13 @multigpu.py:323] [p0574] step: count(1383), step_time 1469.12, mean_step_time 1433.83, it/s 0.7
  6527. [0130 18:03:14 @multigpu.py:323] [p0576] step: count(1393), step_time 1392.96, mean_step_time 1432.13, it/s 0.7
  6528. [0130 18:03:14 @multigpu.py:323] [p0574] step: count(1384), step_time 1429.08, mean_step_time 1433.79, it/s 0.7
  6529. [0130 18:03:14 @multigpu.py:323] [p0115] step: count(1369), step_time 1442.2, mean_step_time 1434.45, it/s 0.7
  6530. sending to address tcp://p0112:61216
  6531. ##### Sending to neptune: online_score : 0.553857938581 , 0.8 #####
  6532. [u'online', 0.8]
  6533. receiving
  6534. [0130 18:03:15 @multigpu.py:323] [p0576] step: count(1394), step_time 1431.74, mean_step_time 1432.04, it/s 0.7
  6535. [0130 18:03:15 @multigpu.py:323] [p0115] step: count(1370), step_time 1393.41, mean_step_time 1429.88, it/s 0.7
  6536. [0130 18:03:16 @multigpu.py:323] [p0574] step: count(1385), step_time 1439.65, mean_step_time 1434.15, it/s 0.7
  6537. [0130 18:03:17 @multigpu.py:323] [p0576] step: count(1395), step_time 1411.33, mean_step_time 1431.83, it/s 0.7
  6538. [0130 18:03:17 @multigpu.py:323] [p0115] step: count(1371), step_time 1414.81, mean_step_time 1430.05, it/s 0.7
  6539. [0130 18:03:17 @multigpu.py:323] [p0574] step: count(1386), step_time 1430.7, mean_step_time 1435.36, it/s 0.7
  6540. [0130 18:03:18 @multigpu.py:323] [p0576] step: count(1396), step_time 1421.42, mean_step_time 1428.61, it/s 0.7
  6541. [0130 18:03:18 @multigpu.py:323] [p0115] step: count(1372), step_time 1413.61, mean_step_time 1427.64, it/s 0.7
  6542. [0130 18:03:18 @multigpu.py:323] [p0574] step: count(1387), step_time 1418.14, mean_step_time 1435.82, it/s 0.7
  6543. [0130 18:03:20 @multigpu.py:323] [p0115] step: count(1373), step_time 1417.49, mean_step_time 1429.23, it/s 0.7
  6544. [0130 18:03:20 @multigpu.py:323] [p0576] step: count(1397), step_time 1470.89, mean_step_time 1430.43, it/s 0.7
  6545. [0130 18:03:20 @multigpu.py:323] [p0574] step: count(1388), step_time 1456.8, mean_step_time 1437.12, it/s 0.7
  6546. [0130 18:03:21 @multigpu.py:323] [p0115] step: count(1374), step_time 1447.07, mean_step_time 1429.9, it/s 0.7
  6547. [0130 18:03:21 @multigpu.py:323] [p0574] step: count(1389), step_time 1370.25, mean_step_time 1434.54, it/s 0.7
  6548. [0130 18:03:21 @multigpu.py:323] [p0576] step: count(1398), step_time 1519.05, mean_step_time 1436.01, it/s 0.7
  6549. [0130 18:03:23 @multigpu.py:323] [p0115] step: count(1375), step_time 1392.37, mean_step_time 1428.34, it/s 0.7
  6550. [0130 18:03:23 @multigpu.py:323] [p0574] step: count(1390), step_time 1445.91, mean_step_time 1435.98, it/s 0.7
  6551. [0130 18:03:23 @multigpu.py:323] [p0576] step: count(1399), step_time 1382.11, mean_step_time 1436.75, it/s 0.7
  6552. [0130 18:03:24 @multigpu.py:323] [p0115] step: count(1376), step_time 1428.37, mean_step_time 1426.06, it/s 0.7
  6553. [0130 18:03:24 @multigpu.py:323] [p0574] step: count(1391), step_time 1393.81, mean_step_time 1435.11, it/s 0.7
  6554. [0130 18:03:24 @multigpu.py:323] [p0576] step: count(1400), step_time 1472.93, mean_step_time 1438.23, it/s 0.7
  6555. sending debugging info...
  6556. sending to address tcp://p0112:61216
  6557. ##### Sending to neptune: mean_delay : 0.556543689966 , 0.0 #####
  6558. ##### Sending to neptune: max_delay : 0.556543689966 , -0.0 #####
  6559. sending to address tcp://p0112:61216
  6560. ##### Sending to neptune: min_delay : 0.556543689966 , -0.0 #####
  6561. [u'delays', [0.0, -0.0, -0.0]]
  6562. receiving
  6563. ##### Sending to neptune: cost : 0.556544211904 , -0.00979332625866 #####
  6564. ##### Sending to neptune: policy_loss : 0.556544211904 , -0.407843053341 #####
  6565. sending to address tcp://p0112:61216
  6566. ##### Sending to neptune: xentropy_loss : 0.556544211904 , -2.28586602211 #####
  6567. ##### Sending to neptune: value_loss : 0.556544211904 , 1.44016361237 #####
  6568. ##### Sending to neptune: advantage : 0.556544211904 , 0.00166430650279 #####
  6569. ##### Sending to neptune: pred_reward : 0.556544211904 , 0.450704455376 #####
  6570. ##### Sending to neptune: max_logit : 0.556544211904 , 0.210775285959 #####
  6571. [u'loss', -0.009793326258659363, -0.40784305334091187, -2.2858660221099854, 1.4401636123657227, 0.001664306502789259, 0.4507044553756714, 0.21077528595924377]
  6572. receiving
  6573. ##### Sending to neptune: active_relus : 0.556544706888 , 9774847.18 #####
  6574. ##### Sending to neptune: dp_per_s : 0.556544706888 , 89.2697802076 #####
  6575. [u'other', 9774847.18, 89.26978020759367]
  6576. receiving
  6577. [0130 18:03:25 @multigpu.py:323] [p0115] step: count(1377), step_time 1460.58, mean_step_time 1428.35, it/s 0.7
  6578. [0130 18:03:25 @multigpu.py:323] [p0574] step: count(1392), step_time 1452.28, mean_step_time 1433.14, it/s 0.7
  6579. [0130 18:03:26 @multigpu.py:323] [p0576] step: count(1401), step_time 1385.76, mean_step_time 1439.08, it/s 0.69
  6580. [0130 18:03:27 @multigpu.py:323] [p0574] step: count(1393), step_time 1385.9, mean_step_time 1430.97, it/s 0.7
  6581. [0130 18:03:27 @multigpu.py:323] [p0115] step: count(1378), step_time 1452.95, mean_step_time 1425.33, it/s 0.7
  6582. [0130 18:03:27 @multigpu.py:323] [p0576] step: count(1402), step_time 1407.95, mean_step_time 1435.25, it/s 0.7
  6583. sending to address tcp://p0112:61216
  6584. ##### Sending to neptune: online_score : 0.557487060494 , 1.7 #####
  6585. [u'online', 1.7]
  6586. ##### Sending to neptune: active_workers : 0.557487142483 , 3 #####
  6587. receiving
  6588. [0130 18:03:28 @multigpu.py:323] [p0574] step: count(1394), step_time 1447.19, mean_step_time 1431.49, it/s 0.7
  6589. [0130 18:03:28 @multigpu.py:323] [p0115] step: count(1379), step_time 1414.04, mean_step_time 1425.12, it/s 0.7
  6590. [0130 18:03:28 @multigpu.py:323] [p0576] step: count(1403), step_time 1396.03, mean_step_time 1433.53, it/s 0.7
  6591. [0130 18:03:30 @multigpu.py:323] [p0574] step: count(1395), step_time 1396.59, mean_step_time 1430.9, it/s 0.7
  6592. [0130 18:03:30 @multigpu.py:323] [p0576] step: count(1404), step_time 1405.56, mean_step_time 1433.58, it/s 0.7
  6593. [0130 18:03:30 @multigpu.py:323] [p0115] step: count(1380), step_time 1432.02, mean_step_time 1425.95, it/s 0.7
  6594. [0130 18:03:31 @multigpu.py:323] [p0574] step: count(1396), step_time 1405.79, mean_step_time 1429.76, it/s 0.7
  6595. [0130 18:03:31 @multigpu.py:323] [p0576] step: count(1405), step_time 1407.6, mean_step_time 1431.84, it/s 0.7
  6596. [0130 18:03:31 @multigpu.py:323] [p0115] step: count(1381), step_time 1451.89, mean_step_time 1425.17, it/s 0.7
  6597. [0130 18:03:33 @multigpu.py:323] [p0576] step: count(1406), step_time 1429.22, mean_step_time 1431.53, it/s 0.7
  6598. [0130 18:03:33 @multigpu.py:323] [p0574] step: count(1397), step_time 1462.66, mean_step_time 1431.54, it/s 0.7
  6599. [0130 18:03:33 @multigpu.py:323] [p0115] step: count(1382), step_time 1407.0, mean_step_time 1426.62, it/s 0.7
  6600. [0130 18:03:34 @multigpu.py:323] [p0574] step: count(1398), step_time 1379.74, mean_step_time 1430.2, it/s 0.7
  6601. [0130 18:03:34 @multigpu.py:323] [p0115] step: count(1383), step_time 1406.98, mean_step_time 1426.67, it/s 0.7
  6602. [0130 18:03:34 @multigpu.py:323] [p0576] step: count(1407), step_time 1441.96, mean_step_time 1432.25, it/s 0.7
  6603. [0130 18:03:35 @multigpu.py:323] [p0574] step: count(1399), step_time 1403.81, mean_step_time 1426.58, it/s 0.7
  6604. [0130 18:03:35 @multigpu.py:323] [p0576] step: count(1408), step_time 1380.08, mean_step_time 1428.88, it/s 0.7
  6605. [0130 18:03:35 @multigpu.py:323] [p0115] step: count(1384), step_time 1433.42, mean_step_time 1427.28, it/s 0.7
  6606. [0130 18:03:37 @multigpu.py:323] [p0576] step: count(1409), step_time 1406.49, mean_step_time 1429.73, it/s 0.7
  6607. [0130 18:03:37 @multigpu.py:323] [p0574] step: count(1400), step_time 1479.25, mean_step_time 1428.75, it/s 0.7
  6608. sending debugging info...
  6609. sending to address tcp://p0112:61216
  6610. ##### Sending to neptune: mean_delay : 0.560071513851 , 0.0 #####
  6611. sending to address tcp://p0112:61216
  6612. ##### Sending to neptune: max_delay : 0.560071513851 , -0.0 #####
  6613. ##### Sending to neptune: min_delay : 0.560071513851 , -0.0 #####
  6614. [u'delays', [0.0, -0.0, -0.0]]
  6615. receiving
  6616. ##### Sending to neptune: cost : 0.560071992477 , -0.00401578377932 #####
  6617. sending to address tcp://p0112:61216
  6618. ##### Sending to neptune: policy_loss : 0.560071992477 , 0.167572632432 #####
  6619. ##### Sending to neptune: xentropy_loss : 0.560071992477 , -2.28561639786 #####
  6620. ##### Sending to neptune: value_loss : 0.560071992477 , 1.60402357578 #####
  6621. ##### Sending to neptune: advantage : 0.560071992477 , -0.000748563557863 #####
  6622. ##### Sending to neptune: pred_reward : 0.560071992477 , 0.458303689957 #####
  6623. ##### Sending to neptune: max_logit : 0.560071992477 , 0.2134732306 #####
  6624. [u'loss', -0.004015783779323101, 0.16757263243198395, -2.285616397857666, 1.6040235757827759, -0.0007485635578632355, 0.45830368995666504, 0.21347323060035706]
  6625. receiving
  6626. ##### Sending to neptune: active_relus : 0.560072435273 , 9771696.53 #####
  6627. ##### Sending to neptune: dp_per_s : 0.560072435273 , 89.2691596223 #####
  6628. [u'other', 9771696.53, 89.26915962233971]
  6629. receiving
  6630. [0130 18:03:37 @multigpu.py:323] [p0115] step: count(1385), step_time 1439.52, mean_step_time 1426.49, it/s 0.7
  6631. sending to address tcp://p0112:61216
  6632. ##### Sending to neptune: online_score : 0.560282720261 , 2.1 #####
  6633. [u'online', 2.1]
  6634. receiving
  6635. [0130 18:03:38 @multigpu.py:323] [p0576] step: count(1410), step_time 1412.38, mean_step_time 1430.39, it/s 0.7
  6636. [0130 18:03:38 @multigpu.py:323] [p0574] step: count(1401), step_time 1418.35, mean_step_time 1423.72, it/s 0.7
  6637. [0130 18:03:38 @multigpu.py:323] [p0115] step: count(1386), step_time 1427.55, mean_step_time 1426.91, it/s 0.7
  6638. [0130 18:03:40 @multigpu.py:323] [p0576] step: count(1411), step_time 1410.81, mean_step_time 1431.2, it/s 0.7
  6639. [0130 18:03:40 @multigpu.py:323] [p0574] step: count(1402), step_time 1430.52, mean_step_time 1425.78, it/s 0.7
  6640. [0130 18:03:40 @multigpu.py:323] [p0115] step: count(1387), step_time 1387.21, mean_step_time 1425.71, it/s 0.7
  6641. [0130 18:03:41 @multigpu.py:323] [p0115] step: count(1388), step_time 1451.92, mean_step_time 1425.72, it/s 0.7
  6642. [0130 18:03:41 @multigpu.py:323] [p0574] step: count(1403), step_time 1468.66, mean_step_time 1425.75, it/s 0.7
  6643. [0130 18:03:41 @multigpu.py:323] [p0576] step: count(1412), step_time 1529.78, mean_step_time 1425.8, it/s 0.7
  6644. [0130 18:03:43 @multigpu.py:323] [p0574] step: count(1404), step_time 1376.73, mean_step_time 1423.14, it/s 0.7
  6645. [0130 18:03:43 @multigpu.py:323] [p0576] step: count(1413), step_time 1411.29, mean_step_time 1426.72, it/s 0.7
  6646. [0130 18:03:43 @multigpu.py:323] [p0115] step: count(1389), step_time 1436.06, mean_step_time 1425.41, it/s 0.7
  6647. [0130 18:03:44 @multigpu.py:323] [p0574] step: count(1405), step_time 1419.54, mean_step_time 1422.13, it/s 0.7
  6648. [0130 18:03:44 @multigpu.py:323] [p0115] step: count(1390), step_time 1406.9, mean_step_time 1426.09, it/s 0.7
  6649. [0130 18:03:44 @multigpu.py:323] [p0576] step: count(1414), step_time 1453.25, mean_step_time 1427.79, it/s 0.7
  6650. [0130 18:03:45 @multigpu.py:323] [p0574] step: count(1406), step_time 1442.56, mean_step_time 1422.72, it/s 0.7
  6651. [0130 18:03:45 @multigpu.py:323] [p0115] step: count(1391), step_time 1404.4, mean_step_time 1425.57, it/s 0.7
  6652. [0130 18:03:45 @multigpu.py:323] [p0576] step: count(1415), step_time 1411.3, mean_step_time 1427.79, it/s 0.7
  6653. [0130 18:03:47 @multigpu.py:323] [p0574] step: count(1407), step_time 1372.41, mean_step_time 1420.44, it/s 0.7
  6654. [0130 18:03:47 @multigpu.py:323] [p0115] step: count(1392), step_time 1419.39, mean_step_time 1425.86, it/s 0.7
  6655. [0130 18:03:47 @multigpu.py:323] [p0576] step: count(1416), step_time 1451.4, mean_step_time 1429.29, it/s 0.7
  6656. [0130 18:03:48 @multigpu.py:323] [p0574] step: count(1408), step_time 1446.35, mean_step_time 1419.91, it/s 0.7
  6657. [0130 18:03:48 @multigpu.py:323] [p0115] step: count(1393), step_time 1410.03, mean_step_time 1425.48, it/s 0.7
  6658. [0130 18:03:48 @multigpu.py:323] [p0576] step: count(1417), step_time 1358.56, mean_step_time 1423.68, it/s 0.7
  6659. [0130 18:03:50 @multigpu.py:323] [p0576] step: count(1418), step_time 1427.34, mean_step_time 1419.09, it/s 0.7
  6660. [0130 18:03:50 @multigpu.py:323] [p0115] step: count(1394), step_time 1481.06, mean_step_time 1427.18, it/s 0.7
  6661. [0130 18:03:50 @multigpu.py:323] [p0574] step: count(1409), step_time 1503.11, mean_step_time 1426.56, it/s 0.7
  6662. [0130 18:03:51 @multigpu.py:323] [p0115] step: count(1395), step_time 1411.36, mean_step_time 1428.13, it/s 0.7
  6663. [0130 18:03:51 @multigpu.py:323] [p0576] step: count(1419), step_time 1463.2, mean_step_time 1423.14, it/s 0.7
  6664. sending to address tcp://p0112:61216
  6665. ##### Sending to neptune: online_score : 0.564045781096 , 2.1 #####
  6666. [u'online', 2.1]
  6667. receiving
  6668. [0130 18:03:51 @multigpu.py:323] [p0574] step: count(1410), step_time 1455.92, mean_step_time 1427.06, it/s 0.7
  6669. [0130 18:03:53 @multigpu.py:323] [p0576] step: count(1420), step_time 1406.7, mean_step_time 1419.83, it/s 0.7
  6670. [0130 18:03:53 @multigpu.py:323] [p0115] step: count(1396), step_time 1418.38, mean_step_time 1427.63, it/s 0.7
  6671. [0130 18:03:53 @multigpu.py:323] [p0574] step: count(1411), step_time 1499.37, mean_step_time 1432.34, it/s 0.7
  6672. [0130 18:03:54 @multigpu.py:323] [p0115] step: count(1397), step_time 1440.01, mean_step_time 1426.6, it/s 0.7
  6673. [0130 18:03:54 @multigpu.py:323] [p0576] step: count(1421), step_time 1460.03, mean_step_time 1423.55, it/s 0.7
  6674. 42%|####2 |421/1000[1[0130 18:03:54 @multigpu.py:323] [p0574] step: count(1412), step_time 1433.92, mean_step_time 1431.42, it/s 0.7
  6675. [0130 18:03:55 @multigpu.py:323] [p0576] step: count(1422), step_time 1431.75, mean_step_time 1424.74, it/s 0.7
  6676. [0130 18:03:56 @multigpu.py:323] [p0115] step: count(1398), step_time 1491.19, mean_step_time 1428.52, it/s 0.7
  6677. [0130 18:03:56 @multigpu.py:323] [p0574] step: count(1413), step_time 1429.86, mean_step_time 1433.62, it/s 0.7
  6678. [0130 18:03:57 @multigpu.py:323] [p0576] step: count(1423), step_time 1400.64, mean_step_time 1424.97, it/s 0.7
  6679. [0130 18:03:57 @multigpu.py:323] [p0115] step: count(1399), step_time 1451.76, mean_step_time 1430.4, it/s 0.7
  6680. [0130 18:03:57 @multigpu.py:323] [p0574] step: count(1414), step_time 1432.31, mean_step_time 1432.87, it/s 0.7
  6681. [0130 18:03:58 @multigpu.py:323] [p0576] step: count(1424), step_time 1441.02, mean_step_time 1426.74, it/s 0.7
  6682. [0130 18:03:58 @multigpu.py:323] [p0115] step: count(1400), step_time 1432.93, mean_step_time 1430.45, it/s 0.7
  6683. sending debugging info...
  6684. sending to address tcp://p0112:61216
  6685. ##### Sending to neptune: mean_delay : 0.566052081651 , 0.0 #####
  6686. sending to address tcp://p0112:61216
  6687. ##### Sending to neptune: max_delay : 0.566052081651 , -0.0 #####
  6688. ##### Sending to neptune: min_delay : 0.566052081651 , -0.0 #####
  6689. [u'delays', [0.0, -0.0, -0.0]]
  6690. receiving
  6691. ##### Sending to neptune: cost : 0.566052584383 , -0.00693348702043 #####
  6692. sending to address tcp://p0112:61216
  6693. ##### Sending to neptune: policy_loss : 0.566052584383 , 0.0658853203058 #####
  6694. ##### Sending to neptune: xentropy_loss : 0.566052584383 , -2.28544616699 #####
  6695. ##### Sending to neptune: value_loss : 0.566052584383 , 1.33207464218 #####
  6696. ##### Sending to neptune: advantage : 0.566052584383 , -0.000360454781912 #####
  6697. ##### Sending to neptune: pred_reward : 0.566052584383 , 0.440528899431 #####
  6698. ##### Sending to neptune: max_logit : 0.566052584383 , 0.215451806784 #####
  6699. [u'loss', -0.006933487020432949, 0.06588532030582428, -2.2854461669921875, 1.3320746421813965, -0.0003604547819122672, 0.44052889943122864, 0.21545180678367615]
  6700. receiving
  6701. ##### Sending to neptune: active_relus : 0.566053047445 , 9768145.36 #####
  6702. ##### Sending to neptune: dp_per_s : 0.566053047445 , 89.2624328751 #####
  6703. [u'other', 9768145.36, 89.26243287508888]
  6704. receiving
  6705. [0130 18:03:58 @multigpu.py:323] [p0574] step: count(1415), step_time 1433.64, mean_step_time 1434.72, it/s 0.7
  6706. sending to address tcp://p0112:61216
  6707. ##### Sending to neptune: online_score : 0.566195656061 , 1.7 #####
  6708. [u'online', 1.7]
  6709. receiving
  6710. [0130 18:04:00 @multigpu.py:323] [p0576] step: count(1425), step_time 1418.71, mean_step_time 1427.29, it/s 0.7
  6711. [0130 18:04:00 @multigpu.py:323] [p0115] step: count(1401), step_time 1408.59, mean_step_time 1428.28, it/s 0.7
  6712. [0130 18:04:00 @multigpu.py:323] [p0574] step: count(1416), step_time 1428.78, mean_step_time 1435.87, it/s 0.7
  6713. [0130 18:04:01 @multigpu.py:323] [p0576] step: count(1426), step_time 1384.2, mean_step_time 1425.04, it/s 0.7
  6714. [0130 18:04:01 @multigpu.py:323] [p0115] step: count(1402), step_time 1472.46, mean_step_time 1431.56, it/s 0.7
  6715. [0130 18:04:01 @multigpu.py:323] [p0574] step: count(1417), step_time 1427.13, mean_step_time 1434.1, it/s 0.7
  6716. [0130 18:04:03 @multigpu.py:323] [p0576] step: count(1427), step_time 1432.02, mean_step_time 1424.55, it/s 0.7
  6717. [0130 18:04:03 @multigpu.py:323] [p0115] step: count(1403), step_time 1397.03, mean_step_time 1431.06, it/s 0.7
  6718. [0130 18:04:03 @multigpu.py:323] [p0574] step: count(1418), step_time 1401.49, mean_step_time 1435.18, it/s 0.7
  6719. sending to address tcp://p0112:61216
  6720. ##### Sending to neptune: online_score : 0.567442069716 , 2.2 #####
  6721. [u'online', 2.2]
  6722. receiving
  6723. [0130 18:04:04 @multigpu.py:323] [p0576] step: count(1428), step_time 1388.66, mean_step_time 1424.98, it/s 0.7
  6724. [0130 18:04:04 @multigpu.py:323] [p0115] step: count(1404), step_time 1416.48, mean_step_time 1430.21, it/s 0.7
  6725. [0130 18:04:04 @multigpu.py:323] [p0574] step: count(1419), step_time 1474.8, mean_step_time 1438.73, it/s 0.7
  6726. [0130 18:04:05 @multigpu.py:323] [p0576] step: count(1429), step_time 1433.08, mean_step_time 1426.31, it/s 0.7
  6727. [0130 18:04:06 @multigpu.py:323] [p0115] step: count(1405), step_time 1485.3, mean_step_time 1432.5, it/s 0.7
  6728. [0130 18:04:06 @multigpu.py:323] [p0574] step: count(1420), step_time 1429.36, mean_step_time 1436.24, it/s 0.7
  6729. [0130 18:04:07 @multigpu.py:323] [p0576] step: count(1430), step_time 1402.01, mean_step_time 1425.79, it/s 0.7
  6730. [0130 18:04:07 @multigpu.py:323] [p0115] step: count(1406), step_time 1430.27, mean_step_time 1432.64, it/s 0.7
  6731. [0130 18:04:07 @multigpu.py:323] [p0574] step: count(1421), step_time 1446.44, mean_step_time 1437.64, it/s 0.7
  6732. 42%|####2 |421/1000[1[0130 18:04:08 @multigpu.py:323] [p0576] step: count(1431), step_time 1437.82, mean_step_time 1427.14, it/s 0.7
  6733. [0130 18:04:08 @multigpu.py:323] [p0574] step: count(1422), step_time 1387.96, mean_step_time 1435.52, it/s 0.7
  6734. [0130 18:04:08 @multigpu.py:323] [p0115] step: count(1407), step_time 1463.99, mean_step_time 1436.48, it/s 0.7
  6735. [0130 18:04:10 @multigpu.py:323] [p0115] step: count(1408), step_time 1417.2, mean_step_time 1434.74, it/s 0.7
  6736. [0130 18:04:10 @multigpu.py:323] [p0576] step: count(1432), step_time 1683.99, mean_step_time 1434.85, it/s 0.7
  6737. [0130 18:04:10 @multigpu.py:323] [p0574] step: count(1423), step_time 1453.76, mean_step_time 1434.77, it/s 0.7
  6738. [0130 18:04:11 @multigpu.py:323] [p0574] step: count(1424), step_time 1397.23, mean_step_time 1435.8, it/s 0.7
  6739. [0130 18:04:11 @multigpu.py:323] [p0576] step: count(1433), step_time 1445.1, mean_step_time 1436.54, it/s 0.7
  6740. [0130 18:04:11 @multigpu.py:323] [p0115] step: count(1409), step_time 1479.96, mean_step_time 1436.93, it/s 0.7
  6741. [0130 18:04:13 @multigpu.py:323] [p0574] step: count(1425), step_time 1455.64, mean_step_time 1437.6, it/s 0.7
  6742. [0130 18:04:13 @multigpu.py:323] [p0576] step: count(1434), step_time 1429.56, mean_step_time 1435.35, it/s 0.7
  6743. [0130 18:04:13 @multigpu.py:323] [p0115] step: count(1410), step_time 1414.51, mean_step_time 1437.31, it/s 0.7
  6744. [0130 18:04:14 @multigpu.py:323] [p0574] step: count(1426), step_time 1404.97, mean_step_time 1435.72, it/s 0.7
  6745. [0130 18:04:14 @multigpu.py:323] [p0576] step: count(1435), step_time 1443.08, mean_step_time 1436.94, it/s 0.7
  6746. [0130 18:04:14 @multigpu.py:323] [p0115] step: count(1411), step_time 1460.86, mean_step_time 1440.14, it/s 0.69
  6747. [0130 18:04:16 @multigpu.py:323] [p0574] step: count(1427), step_time 1421.89, mean_step_time 1438.2, it/s 0.7
  6748. [0130 18:04:16 @multigpu.py:323] [p0115] step: count(1412), step_time 1443.55, mean_step_time 1441.35, it/s 0.69
  6749. [0130 18:04:16 @multigpu.py:323] [p0576] step: count(1436), step_time 1507.11, mean_step_time 1439.73, it/s 0.69
  6750. [0130 18:04:17 @multigpu.py:323] [p0574] step: count(1428), step_time 1436.85, mean_step_time 1437.72, it/s 0.7
  6751. [0130 18:04:17 @multigpu.py:323] [p0115] step: count(1413), step_time 1404.07, mean_step_time 1441.05, it/s 0.69
  6752. [0130 18:04:17 @multigpu.py:323] [p0576] step: count(1437), step_time 1402.3, mean_step_time 1441.92, it/s 0.69
  6753. [0130 18:04:18 @multigpu.py:323] [p0574] step: count(1429), step_time 1403.43, mean_step_time 1432.74, it/s 0.7
  6754. [0130 18:04:19 @multigpu.py:323] [p0115] step: count(1414), step_time 1450.1, mean_step_time 1439.5, it/s 0.69
  6755. [0130 18:04:19 @multigpu.py:323] [p0576] step: count(1438), step_time 1429.5, mean_step_time 1442.02, it/s 0.69
  6756. [0130 18:04:20 @multigpu.py:323] [p0574] step: count(1430), step_time 1454.1, mean_step_time 1432.65, it/s 0.7
  6757. [0130 18:04:20 @multigpu.py:323] [p0576] step: count(1439), step_time 1378.6, mean_step_time 1437.79, it/s 0.7
  6758. [0130 18:04:20 @multigpu.py:323] [p0115] step: count(1415), step_time 1405.54, mean_step_time 1439.21, it/s 0.69
  6759. [0130 18:04:21 @multigpu.py:323] [p0574] step: count(1431), step_time 1439.17, mean_step_time 1429.64, it/s 0.7
  6760. [0130 18:04:21 @multigpu.py:323] [p0115] step: count(1416), step_time 1441.28, mean_step_time 1440.35, it/s 0.69
  6761. [0130 18:04:21 @multigpu.py:323] [p0576] step: count(1440), step_time 1499.94, mean_step_time 1442.46, it/s 0.69
  6762. [0130 18:04:23 @multigpu.py:323] [p0574] step: count(1432), step_time 1418.43, mean_step_time 1428.86, it/s 0.7
  6763. [0130 18:04:23 @multigpu.py:323] [p0115] step: count(1417), step_time 1441.13, mean_step_time 1440.41, it/s 0.69
  6764. [0130 18:04:23 @multigpu.py:323] [p0576] step: count(1441), step_time 1418.78, mean_step_time 1440.39, it/s 0.69
  6765. sending to address tcp://p0112:61216
  6766. ##### Sending to neptune: online_score : 0.57297924108 , 2.1 #####
  6767. [u'online', 2.1]
  6768. receiving
  6769. [0130 18:04:24 @multigpu.py:323] [p0574] step: count(1433), step_time 1421.44, mean_step_time 1428.44, it/s 0.7
  6770. [0130 18:04:24 @multigpu.py:323] [p0115] step: count(1418), step_time 1364.18, mean_step_time 1434.06, it/s 0.7
  6771. [0130 18:04:24 @multigpu.py:323] [p0576] step: count(1442), step_time 1402.34, mean_step_time 1438.92, it/s 0.69
  6772. [0130 18:04:26 @multigpu.py:323] [p0574] step: count(1434), step_time 1469.75, mean_step_time 1430.31, it/s 0.7
  6773. [0130 18:04:26 @multigpu.py:323] [p0115] step: count(1419), step_time 1464.25, mean_step_time 1434.68, it/s 0.7
  6774. [0130 18:04:26 @multigpu.py:323] [p0576] step: count(1443), step_time 1439.18, mean_step_time 1440.85, it/s 0.69
  6775. [0130 18:04:27 @multigpu.py:323] [p0574] step: count(1435), step_time 1390.89, mean_step_time 1428.18, it/s 0.7
  6776. [0130 18:04:27 @multigpu.py:323] [p0576] step: count(1444), step_time 1393.86, mean_step_time 1438.49, it/s 0.7
  6777. [0130 18:04:27 @multigpu.py:323] [p0115] step: count(1420), step_time 1461.02, mean_step_time 1436.09, it/s 0.7
  6778. 42%|####2 |420/1000[1[0130 18:04:28 @multigpu.py:323] [p0574] step: count(1436), step_time 1398.75, mean_step_time 1426.67, it/s 0.7
  6779. [0130 18:04:29 @multigpu.py:323] [p0576] step: count(1445), step_time 1473.53, mean_step_time 1441.23, it/s 0.69
  6780. [0130 18:04:29 @multigpu.py:323] [p0115] step: count(1421), step_time 1498.68, mean_step_time 1440.59, it/s 0.69
  6781. sending to address tcp://p0112:61216
  6782. ##### Sending to neptune: online_score : 0.574587674141 , 1.3 #####
  6783. [u'online', 1.3]
  6784. ##### Sending to neptune: active_workers : 0.57458770944 , 3 #####
  6785. receiving
  6786. [0130 18:04:30 @multigpu.py:323] [p0574] step: count(1437), step_time 1419.64, mean_step_time 1426.3, it/s 0.7
  6787. [0130 18:04:30 @multigpu.py:323] [p0115] step: count(1422), step_time 1420.56, mean_step_time 1438.0, it/s 0.7
  6788. [0130 18:04:30 @multigpu.py:323] [p0576] step: count(1446), step_time 1496.92, mean_step_time 1446.87, it/s 0.69
  6789. sending to address tcp://p0112:61216
  6790. ##### Sending to neptune: online_score : 0.574955003858 , 1.9 #####
  6791. [u'online', 1.9]
  6792. receiving
  6793. [0130 18:04:31 @multigpu.py:323] [p0574] step: count(1438), step_time 1439.34, mean_step_time 1428.19, it/s 0.7
  6794. [0130 18:04:31 @multigpu.py:323] [p0115] step: count(1423), step_time 1413.13, mean_step_time 1438.8, it/s 0.7
  6795. [0130 18:04:31 @multigpu.py:323] [p0576] step: count(1447), step_time 1426.12, mean_step_time 1446.57, it/s 0.69
  6796. [0130 18:04:33 @multigpu.py:323] [p0574] step: count(1439), step_time 1413.37, mean_step_time 1425.12, it/s 0.7
  6797. [0130 18:04:33 @multigpu.py:323] [p0115] step: count(1424), step_time 1406.32, mean_step_time 1438.29, it/s 0.7
  6798. [0130 18:04:33 @multigpu.py:323] [p0576] step: count(1448), step_time 1443.4, mean_step_time 1449.31, it/s 0.69
  6799. [0130 18:04:34 @multigpu.py:323] [p0574] step: count(1440), step_time 1384.12, mean_step_time 1422.86, it/s 0.7
  6800. [0130 18:04:34 @multigpu.py:323] [p0115] step: count(1425), step_time 1440.45, mean_step_time 1436.05, it/s 0.7
  6801. [0130 18:04:34 @multigpu.py:323] [p0576] step: count(1449), step_time 1399.53, mean_step_time 1447.63, it/s 0.69
  6802. [0130 18:04:35 @multigpu.py:323] [p0574] step: count(1441), step_time 1419.58, mean_step_time 1421.51, it/s 0.7
  6803. [0130 18:04:36 @multigpu.py:323] [p0115] step: count(1426), step_time 1411.7, mean_step_time 1435.12, it/s 0.7
  6804. [0130 18:04:36 @multigpu.py:323] [p0576] step: count(1450), step_time 1423.99, mean_step_time 1448.73, it/s 0.69
  6805. [0130 18:04:37 @multigpu.py:323] [p0574] step: count(1442), step_time 1436.64, mean_step_time 1423.95, it/s 0.7
  6806. [0130 18:04:37 @multigpu.py:323] [p0576] step: count(1451), step_time 1416.57, mean_step_time 1447.67, it/s 0.69
  6807. [0130 18:04:37 @multigpu.py:323] [p0115] step: count(1427), step_time 1510.37, mean_step_time 1437.44, it/s 0.7
  6808. [0130 18:04:39 @multigpu.py:323] [p0115] step: count(1428), step_time 1389.38, mean_step_time 1436.05, it/s 0.7
  6809. [0130 18:04:39 @multigpu.py:323] [p0574] step: count(1443), step_time 1693.65, mean_step_time 1435.94, it/s 0.7
  6810. [0130 18:04:39 @multigpu.py:323] [p0576] step: count(1452), step_time 1449.86, mean_step_time 1435.96, it/s 0.7
  6811. [0130 18:04:40 @multigpu.py:323] [p0574] step: count(1444), step_time 1416.81, mean_step_time 1436.92, it/s 0.7
  6812. [0130 18:04:40 @multigpu.py:323] [p0115] step: count(1429), step_time 1443.53, mean_step_time 1434.23, it/s 0.7
  6813. [0130 18:04:40 @multigpu.py:323] [p0576] step: count(1453), step_time 1449.07, mean_step_time 1436.16, it/s 0.7
  6814. [0130 18:04:41 @multigpu.py:323] [p0574] step: count(1445), step_time 1428.49, mean_step_time 1435.56, it/s 0.7
  6815. [0130 18:04:42 @multigpu.py:323] [p0576] step: count(1454), step_time 1435.61, mean_step_time 1436.46, it/s 0.7
  6816. [0130 18:04:42 @multigpu.py:323] [p0115] step: count(1430), step_time 1450.77, mean_step_time 1436.04, it/s 0.7
  6817. [0130 18:04:43 @multigpu.py:323] [p0574] step: count(1446), step_time 1397.11, mean_step_time 1435.17, it/s 0.7
  6818. [0130 18:04:43 @multigpu.py:323] [p0115] step: count(1431), step_time 1429.27, mean_step_time 1434.46, it/s 0.7
  6819. [0130 18:04:43 @multigpu.py:323] [p0576] step: count(1455), step_time 1445.67, mean_step_time 1436.59, it/s 0.7
  6820. [0130 18:04:44 @multigpu.py:323] [p0574] step: count(1447), step_time 1409.48, mean_step_time 1434.55, it/s 0.7
  6821. [0130 18:04:44 @multigpu.py:323] [p0576] step: count(1456), step_time 1421.41, mean_step_time 1432.31, it/s 0.7
  6822. [0130 18:04:44 @multigpu.py:323] [p0115] step: count(1432), step_time 1447.53, mean_step_time 1434.66, it/s 0.7
  6823. [0130 18:04:46 @multigpu.py:323] [p0574] step: count(1448), step_time 1405.16, mean_step_time 1432.97, it/s 0.7
  6824. [0130 18:04:46 @multigpu.py:323] [p0576] step: count(1457), step_time 1417.23, mean_step_time 1433.06, it/s 0.7
  6825. [0130 18:04:46 @multigpu.py:323] [p0115] step: count(1433), step_time 1425.03, mean_step_time 1435.71, it/s 0.7
  6826. [0130 18:04:47 @multigpu.py:323] [p0574] step: count(1449), step_time 1415.3, mean_step_time 1433.56, it/s 0.7
  6827. [0130 18:04:47 @multigpu.py:323] [p0115] step: count(1434), step_time 1399.97, mean_step_time 1433.2, it/s 0.7
  6828. [0130 18:04:47 @multigpu.py:323] [p0576] step: count(1458), step_time 1449.31, mean_step_time 1434.05, it/s 0.7
  6829. [0130 18:04:49 @multigpu.py:323] [p0574] step: count(1450), step_time 1483.76, mean_step_time 1435.04, it/s 0.7
  6830. [0130 18:04:49 @multigpu.py:323] [p0115] step: count(1435), step_time 1417.79, mean_step_time 1433.82, it/s 0.7
  6831. [0130 18:04:49 @multigpu.py:323] [p0576] step: count(1459), step_time 1436.14, mean_step_time 1436.92, it/s 0.7
  6832. [0130 18:04:50 @multigpu.py:323] [p0574] step: count(1451), step_time 1404.69, mean_step_time 1433.32, it/s 0.7
  6833. [0130 18:04:50 @multigpu.py:323] [p0115] step: count(1436), step_time 1410.73, mean_step_time 1432.29, it/s 0.7
  6834. [0130 18:04:50 @multigpu.py:323] [p0576] step: count(1460), step_time 1416.0, mean_step_time 1432.73, it/s 0.7
  6835. [0130 18:04:51 @multigpu.py:323] [p0574] step: count(1452), step_time 1442.82, mean_step_time 1434.54, it/s 0.7
  6836. [0130 18:04:51 @multigpu.py:323] [p0576] step: count(1461), step_time 1396.68, mean_step_time 1431.62, it/s 0.7
  6837. [0130 18:04:52 @multigpu.py:323] [p0115] step: count(1437), step_time 1458.91, mean_step_time 1433.18, it/s 0.7
  6838. [0130 18:04:53 @multigpu.py:323] [p0576] step: count(1462), step_time 1413.84, mean_step_time 1432.2, it/s 0.7
  6839. [0130 18:04:53 @multigpu.py:323] [p0115] step: count(1438), step_time 1434.71, mean_step_time 1436.7, it/s 0.7
  6840. [0130 18:04:53 @multigpu.py:323] [p0574] step: count(1453), step_time 1518.88, mean_step_time 1439.41, it/s 0.69
  6841. sending to address tcp://p0112:61216
  6842. ##### Sending to neptune: online_score : 0.581396578881 , 2.4 #####
  6843. [u'online', 2.4]
  6844. receiving
  6845. [0130 18:04:54 @multigpu.py:323] [p0576] step: count(1463), step_time 1382.78, mean_step_time 1429.38, it/s 0.7
  6846. [0130 18:04:54 @multigpu.py:323] [p0574] step: count(1454), step_time 1381.92, mean_step_time 1435.02, it/s 0.7
  6847. [0130 18:04:54 @multigpu.py:323] [p0115] step: count(1439), step_time 1474.45, mean_step_time 1437.22, it/s 0.7
  6848. sending to address tcp://p0112:61216
  6849. ##### Sending to neptune: online_score : 0.58178515527 , 1.7 #####
  6850. [u'online', 1.7]
  6851. receiving
  6852. [0130 18:04:56 @multigpu.py:323] [p0576] step: count(1464), step_time 1446.41, mean_step_time 1432.0, it/s 0.7
  6853. [0130 18:04:56 @multigpu.py:323] [p0574] step: count(1455), step_time 1437.95, mean_step_time 1437.37, it/s 0.7
  6854. [0130 18:04:56 @multigpu.py:323] [p0115] step: count(1440), step_time 1431.34, mean_step_time 1435.73, it/s 0.7
  6855. [0130 18:04:57 @multigpu.py:323] [p0576] step: count(1465), step_time 1396.41, mean_step_time 1428.15, it/s 0.7
  6856. [0130 18:04:57 @multigpu.py:323] [p0574] step: count(1456), step_time 1366.36, mean_step_time 1435.75, it/s 0.7
  6857. [0130 18:04:57 @multigpu.py:323] [p0115] step: count(1441), step_time 1456.16, mean_step_time 1433.61, it/s 0.7
  6858. [0130 18:04:59 @multigpu.py:323] [p0576] step: count(1466), step_time 1406.44, mean_step_time 1423.62, it/s 0.7
  6859. [0130 18:04:59 @multigpu.py:323] [p0574] step: count(1457), step_time 1428.66, mean_step_time 1436.21, it/s 0.7
  6860. [0130 18:04:59 @multigpu.py:323] [p0115] step: count(1442), step_time 1452.52, mean_step_time 1435.2, it/s 0.7
  6861. [0130 18:05:00 @multigpu.py:323] [p0574] step: count(1458), step_time 1429.06, mean_step_time 1435.69, it/s 0.7
  6862. sending to address tcp://p0112:61216
  6863. ##### Sending to neptune: online_score : 0.583168856634 , 1.6 #####
  6864. [u'online', 1.6]
  6865. receiving
  6866. [0130 18:05:00 @multigpu.py:323] [p0576] step: count(1467), step_time 1486.09, mean_step_time 1426.62, it/s 0.7
  6867. [0130 18:05:00 @multigpu.py:323] [p0115] step: count(1443), step_time 1417.34, mean_step_time 1435.41, it/s 0.7
  6868. [0130 18:05:01 @multigpu.py:323] [p0574] step: count(1459), step_time 1460.65, mean_step_time 1438.06, it/s 0.7
  6869. [0130 18:05:01 @multigpu.py:323] [p0576] step: count(1468), step_time 1442.72, mean_step_time 1426.59, it/s 0.7
  6870. [0130 18:05:02 @multigpu.py:323] [p0115] step: count(1444), step_time 1443.77, mean_step_time 1437.29, it/s 0.7
  6871. [0130 18:05:03 @multigpu.py:323] [p0574] step: count(1460), step_time 1357.34, mean_step_time 1436.72, it/s 0.7
  6872. [0130 18:05:03 @multigpu.py:323] [p0576] step: count(1469), step_time 1421.67, mean_step_time 1427.69, it/s 0.7
  6873. [0130 18:05:03 @multigpu.py:323] [p0115] step: count(1445), step_time 1390.52, mean_step_time 1434.79, it/s 0.7
  6874. [0130 18:05:04 @multigpu.py:323] [p0574] step: count(1461), step_time 1439.14, mean_step_time 1437.69, it/s 0.7
  6875. [0130 18:05:04 @multigpu.py:323] [p0576] step: count(1470), step_time 1397.91, mean_step_time 1426.39, it/s 0.7
  6876. [0130 18:05:04 @multigpu.py:323] [p0115] step: count(1446), step_time 1459.09, mean_step_time 1437.16, it/s 0.7
  6877. [0130 18:05:06 @multigpu.py:323] [p0574] step: count(1462), step_time 1395.75, mean_step_time 1435.65, it/s 0.7
  6878. [0130 18:05:06 @multigpu.py:323] [p0576] step: count(1471), step_time 1491.13, mean_step_time 1430.12, it/s 0.7
  6879. [0130 18:05:06 @multigpu.py:323] [p0115] step: count(1447), step_time 1441.38, mean_step_time 1433.71, it/s 0.7
  6880. [0130 18:05:07 @multigpu.py:323] [p0115] step: count(1448), step_time 1416.11, mean_step_time 1435.05, it/s 0.7
  6881. [0130 18:05:07 @multigpu.py:323] [p0576] step: count(1472), step_time 1545.41, mean_step_time 1434.9, it/s 0.7
  6882. [0130 18:05:07 @multigpu.py:323] [p0574] step: count(1463), step_time 1681.88, mean_step_time 1435.06, it/s 0.7
  6883. [0130 18:05:09 @multigpu.py:323] [p0574] step: count(1464), step_time 1426.25, mean_step_time 1435.53, it/s 0.7
  6884. [0130 18:05:09 @multigpu.py:323] [p0115] step: count(1449), step_time 1460.03, mean_step_time 1435.87, it/s 0.7
  6885. [0130 18:05:09 @multigpu.py:323] [p0576] step: count(1473), step_time 1517.29, mean_step_time 1438.31, it/s 0.7
  6886. [0130 18:05:10 @multigpu.py:323] [p0574] step: count(1465), step_time 1445.47, mean_step_time 1436.38, it/s 0.7
  6887. [0130 18:05:10 @multigpu.py:323] [p0115] step: count(1450), step_time 1445.33, mean_step_time 1435.6, it/s 0.7
  6888. [0130 18:05:10 @multigpu.py:323] [p0576] step: count(1474), step_time 1437.59, mean_step_time 1438.41, it/s 0.7
  6889. [0130 18:05:12 @multigpu.py:323] [p0574] step: count(1466), step_time 1384.36, mean_step_time 1435.74, it/s 0.7
  6890. [0130 18:05:12 @multigpu.py:323] [p0115] step: count(1451), step_time 1414.79, mean_step_time 1434.87, it/s 0.7
  6891. [0130 18:05:12 @multigpu.py:323] [p0576] step: count(1475), step_time 1501.42, mean_step_time 1441.19, it/s 0.69
  6892. [0130 18:05:13 @multigpu.py:323] [p0574] step: count(1467), step_time 1430.6, mean_step_time 1436.8, it/s 0.7
  6893. [0130 18:05:13 @multigpu.py:323] [p0115] step: count(1452), step_time 1469.14, mean_step_time 1435.96, it/s 0.7
  6894. [0130 18:05:13 @multigpu.py:323] [p0576] step: count(1476), step_time 1433.88, mean_step_time 1441.82, it/s 0.69
  6895. [0130 18:05:14 @multigpu.py:323] [p0574] step: count(1468), step_time 1419.41, mean_step_time 1437.51, it/s 0.7
  6896. [0130 18:05:15 @multigpu.py:323] [p0115] step: count(1453), step_time 1404.76, mean_step_time 1434.94, it/s 0.7
  6897. [0130 18:05:15 @multigpu.py:323] [p0576] step: count(1477), step_time 1375.04, mean_step_time 1439.71, it/s 0.69
  6898. [0130 18:05:16 @multigpu.py:323] [p0574] step: count(1469), step_time 1436.21, mean_step_time 1438.56, it/s 0.7
  6899. [0130 18:05:16 @multigpu.py:323] [p0115] step: count(1454), step_time 1409.09, mean_step_time 1435.4, it/s 0.7
  6900. [0130 18:05:16 @multigpu.py:323] [p0576] step: count(1478), step_time 1437.64, mean_step_time 1439.12, it/s 0.69
  6901. [0130 18:05:17 @multigpu.py:323] [p0574] step: count(1470), step_time 1467.17, mean_step_time 1437.73, it/s 0.7
  6902. [0130 18:05:17 @multigpu.py:323] [p0115] step: count(1455), step_time 1412.25, mean_step_time 1435.12, it/s 0.7
  6903. [0130 18:05:17 @multigpu.py:323] [p0576] step: count(1479), step_time 1377.61, mean_step_time 1436.2, it/s 0.7
  6904. [0130 18:05:19 @multigpu.py:323] [p0574] step: count(1471), step_time 1400.23, mean_step_time 1437.51, it/s 0.7
  6905. [0130 18:05:19 @multigpu.py:323] [p0115] step: count(1456), step_time 1438.36, mean_step_time 1436.5, it/s 0.7
  6906. [0130 18:05:19 @multigpu.py:323] [p0576] step: count(1480), step_time 1420.01, mean_step_time 1436.4, it/s 0.7
  6907. [0130 18:05:20 @multigpu.py:323] [p0115] step: count(1457), step_time 1414.06, mean_step_time 1434.26, it/s 0.7
  6908. [0130 18:05:20 @multigpu.py:323] [p0574] step: count(1472), step_time 1526.73, mean_step_time 1441.7, it/s 0.69
  6909. [0130 18:05:20 @multigpu.py:323] [p0576] step: count(1481), step_time 1460.71, mean_step_time 1439.6, it/s 0.69
  6910. [0130 18:05:22 @multigpu.py:323] [p0115] step: count(1458), step_time 1444.06, mean_step_time 1434.73, it/s 0.7
  6911. [0130 18:05:22 @multigpu.py:323] [p0576] step: count(1482), step_time 1396.01, mean_step_time 1438.71, it/s 0.7
  6912. [0130 18:05:22 @multigpu.py:323] [p0574] step: count(1473), step_time 1476.44, mean_step_time 1439.58, it/s 0.69
  6913. [0130 18:05:23 @multigpu.py:323] [p0115] step: count(1459), step_time 1415.03, mean_step_time 1431.76, it/s 0.7
  6914. [0130 18:05:23 @multigpu.py:323] [p0576] step: count(1483), step_time 1455.79, mean_step_time 1442.36, it/s 0.69
  6915. [0130 18:05:23 @multigpu.py:323] [p0574] step: count(1474), step_time 1441.3, mean_step_time 1442.55, it/s 0.69
  6916. [0130 18:05:24 @multigpu.py:323] [p0115] step: count(1460), step_time 1423.93, mean_step_time 1431.39, it/s 0.7
  6917. [0130 18:05:25 @multigpu.py:323] [p0576] step: count(1484), step_time 1390.01, mean_step_time 1439.54, it/s 0.69
  6918. [0130 18:05:25 @multigpu.py:323] [p0574] step: count(1475), step_time 1404.97, mean_step_time 1440.9, it/s 0.69
  6919. [0130 18:05:26 @multigpu.py:323] [p0115] step: count(1461), step_time 1440.58, mean_step_time 1430.61, it/s 0.7
  6920. [0130 18:05:26 @multigpu.py:323] [p0576] step: count(1485), step_time 1408.88, mean_step_time 1440.16, it/s 0.69
  6921. [0130 18:05:26 @multigpu.py:323] [p0574] step: count(1476), step_time 1362.87, mean_step_time 1440.72, it/s 0.69
  6922. sending to address tcp://p0112:61216
  6923. ##### Sending to neptune: online_score : 0.590532777773 , 1.6 #####
  6924. [u'online', 1.6]
  6925. receiving
  6926. [0130 18:05:27 @multigpu.py:323] [p0115] step: count(1462), step_time 1486.69, mean_step_time 1432.32, it/s 0.7
  6927. [0130 18:05:27 @multigpu.py:323] [p0576] step: count(1486), step_time 1470.78, mean_step_time 1443.38, it/s 0.69
  6928. [0130 18:05:27 @multigpu.py:323] [p0574] step: count(1477), step_time 1481.01, mean_step_time 1443.34, it/s 0.69
  6929. sending to address tcp://p0112:61216
  6930. ##### Sending to neptune: online_score : 0.59100319412 , 2.0 #####
  6931. [u'online', 2.0]
  6932. receiving
  6933. [0130 18:05:29 @multigpu.py:323] [p0576] step: count(1487), step_time 1376.85, mean_step_time 1437.92, it/s 0.7
  6934. [0130 18:05:29 @multigpu.py:323] [p0115] step: count(1463), step_time 1385.96, mean_step_time 1430.75, it/s 0.7
  6935. [0130 18:05:29 @multigpu.py:323] [p0574] step: count(1478), step_time 1408.74, mean_step_time 1442.33, it/s 0.69
  6936. [0130 18:05:30 @multigpu.py:323] [p0576] step: count(1488), step_time 1440.22, mean_step_time 1437.79, it/s 0.7
  6937. [0130 18:05:30 @multigpu.py:323] [p0574] step: count(1479), step_time 1449.09, mean_step_time 1441.75, it/s 0.69
  6938. [0130 18:05:30 @multigpu.py:323] [p0115] step: count(1464), step_time 1520.66, mean_step_time 1434.59, it/s 0.7
  6939. [0130 18:05:32 @multigpu.py:323] [p0574] step: count(1480), step_time 1359.54, mean_step_time 1441.86, it/s 0.69
  6940. [0130 18:05:32 @multigpu.py:323] [p0576] step: count(1489), step_time 1426.31, mean_step_time 1438.02, it/s 0.7
  6941. [0130 18:05:32 @multigpu.py:323] [p0115] step: count(1465), step_time 1408.79, mean_step_time 1435.5, it/s 0.7
  6942. sending to address tcp://p0112:61216
  6943. ##### Sending to neptune: online_score : 0.592168379691 , 1.3 #####
  6944. [u'online', 1.3]
  6945. ##### Sending to neptune: active_workers : 0.592168413864 , 3 #####
  6946. receiving
  6947. [0130 18:05:33 @multigpu.py:323] [p0576] step: count(1490), step_time 1440.04, mean_step_time 1440.13, it/s 0.69
  6948. [0130 18:05:33 @multigpu.py:323] [p0574] step: count(1481), step_time 1445.72, mean_step_time 1442.19, it/s 0.69
  6949. [0130 18:05:33 @multigpu.py:323] [p0115] step: count(1466), step_time 1403.25, mean_step_time 1432.71, it/s 0.7
  6950. [0130 18:05:35 @multigpu.py:323] [p0574] step: count(1482), step_time 1407.35, mean_step_time 1442.77, it/s 0.69
  6951. [0130 18:05:35 @multigpu.py:323] [p0576] step: count(1491), step_time 1422.11, mean_step_time 1436.68, it/s 0.7
  6952. [0130 18:05:35 @multigpu.py:323] [p0115] step: count(1467), step_time 1445.25, mean_step_time 1432.91, it/s 0.7
  6953. [0130 18:05:36 @multigpu.py:323] [p0115] step: count(1468), step_time 1456.31, mean_step_time 1434.92, it/s 0.7
  6954. [0130 18:05:36 @multigpu.py:323] [p0574] step: count(1483), step_time 1523.84, mean_step_time 1434.87, it/s 0.7
  6955. [0130 18:05:36 @multigpu.py:323] [p0576] step: count(1492), step_time 1508.82, mean_step_time 1434.85, it/s 0.7
  6956. [0130 18:05:37 @multigpu.py:323] [p0574] step: count(1484), step_time 1402.04, mean_step_time 1433.66, it/s 0.7
  6957. [0130 18:05:37 @multigpu.py:323] [p0115] step: count(1469), step_time 1415.48, mean_step_time 1432.69, it/s 0.7
  6958. [0130 18:05:37 @multigpu.py:323] [p0576] step: count(1493), step_time 1444.5, mean_step_time 1431.21, it/s 0.7
  6959. [0130 18:05:39 @multigpu.py:323] [p0576] step: count(1494), step_time 1410.45, mean_step_time 1429.85, it/s 0.7
  6960. [0130 18:05:39 @multigpu.py:323] [p0115] step: count(1470), step_time 1446.5, mean_step_time 1432.75, it/s 0.7
  6961. [0130 18:05:39 @multigpu.py:323] [p0574] step: count(1485), step_time 1516.5, mean_step_time 1437.21, it/s 0.7
  6962. [0130 18:05:40 @multigpu.py:323] [p0115] step: count(1471), step_time 1382.08, mean_step_time 1431.11, it/s 0.7
  6963. [0130 18:05:40 @multigpu.py:323] [p0576] step: count(1495), step_time 1417.69, mean_step_time 1425.67, it/s 0.7
  6964. [0130 18:05:40 @multigpu.py:323] [p0574] step: count(1486), step_time 1456.4, mean_step_time 1440.81, it/s 0.69
  6965. [0130 18:05:42 @multigpu.py:323] [p0115] step: count(1472), step_time 1440.42, mean_step_time 1429.68, it/s 0.7
  6966. [0130 18:05:42 @multigpu.py:323] [p0576] step: count(1496), step_time 1468.62, mean_step_time 1427.4, it/s 0.7
  6967. [0130 18:05:42 @multigpu.py:323] [p0574] step: count(1487), step_time 1463.15, mean_step_time 1442.44, it/s 0.69
  6968. [0130 18:05:43 @multigpu.py:323] [p0115] step: count(1473), step_time 1427.59, mean_step_time 1430.82, it/s 0.7
  6969. [0130 18:05:43 @multigpu.py:323] [p0576] step: count(1497), step_time 1416.67, mean_step_time 1429.49, it/s 0.7
  6970. [0130 18:05:43 @multigpu.py:323] [p0574] step: count(1488), step_time 1377.04, mean_step_time 1440.32, it/s 0.69
  6971. [0130 18:05:45 @multigpu.py:323] [p0115] step: count(1474), step_time 1389.29, mean_step_time 1429.83, it/s 0.7
  6972. [0130 18:05:45 @multigpu.py:323] [p0576] step: count(1498), step_time 1364.5, mean_step_time 1425.83, it/s 0.7
  6973. [0130 18:05:45 @multigpu.py:323] [p0574] step: count(1489), step_time 1432.0, mean_step_time 1440.11, it/s 0.69
  6974. [0130 18:05:46 @multigpu.py:323] [p0115] step: count(1475), step_time 1449.54, mean_step_time 1431.69, it/s 0.7
  6975. [0130 18:05:46 @multigpu.py:323] [p0576] step: count(1499), step_time 1459.3, mean_step_time 1429.91, it/s 0.7
  6976. [0130 18:05:46 @multigpu.py:323] [p0574] step: count(1490), step_time 1426.07, mean_step_time 1438.05, it/s 0.7
  6977. [0130 18:05:47 @multigpu.py:323] [p0115] step: count(1476), step_time 1472.42, mean_step_time 1433.4, it/s 0.7
  6978. [0130 18:05:47 @multigpu.py:323] [p0576] step: count(1500), step_time 1443.76, mean_step_time 1431.1, it/s 0.7
  6979. sending debugging info...
  6980. sending to address tcp://p0112:61216
  6981. ##### Sending to neptune: mean_delay : 0.596350740261 , 0.0 #####
  6982. sending to address tcp://p0112:61216
  6983. ##### Sending to neptune: max_delay : 0.596350740261 , -0.0 #####
  6984. ##### Sending to neptune: min_delay : 0.596350740261 , -0.0 #####
  6985. [u'delays', [0.0, -0.0, -0.0]]
  6986. receiving
  6987. ##### Sending to neptune: cost : 0.596351259947 , -0.00888612959534 #####
  6988. sending to address tcp://p0112:61216
  6989. ##### Sending to neptune: policy_loss : 0.596351259947 , -0.356634289026 #####
  6990. ##### Sending to neptune: xentropy_loss : 0.596351259947 , -2.28363466263 #####
  6991. ##### Sending to neptune: value_loss : 0.596351259947 , 1.50284433365 #####
  6992. ##### Sending to neptune: advantage : 0.596351259947 , 0.00149029435124 #####
  6993. ##### Sending to neptune: pred_reward : 0.596351259947 , 0.461806833744 #####
  6994. ##### Sending to neptune: max_logit : 0.596351259947 , 0.228477463126 #####
  6995. [u'loss', -0.008886129595339298, -0.3566342890262604, -2.283634662628174, 1.5028443336486816, 0.0014902943512424827, 0.4618068337440491, 0.22847746312618256]
  6996. receiving
  6997. ##### Sending to neptune: active_relus : 0.596351694663 , 9717197.02 #####
  6998. ##### Sending to neptune: dp_per_s : 0.596351694663 , 89.2814797357 #####
  6999. [u'other', 9717197.02, 89.28147973567819]
  7000. receiving
  7001. [0130 18:05:48 @multigpu.py:323] [p0574] step: count(1491), step_time 1440.54, mean_step_time 1440.07, it/s 0.69
  7002. [0130 18:05:49 @multigpu.py:323] [p0115] step: count(1477), step_time 1435.04, mean_step_time 1434.44, it/s 0.7
  7003. [0130 18:05:49 @multigpu.py:323] [p0576] step: count(1501), step_time 1432.08, mean_step_time 1429.67, it/s 0.7
  7004. [0130 18:05:49 @multigpu.py:323] [p0574] step: count(1492), step_time 1429.17, mean_step_time 1435.19, it/s 0.7
  7005. [0130 18:05:50 @multigpu.py:323] [p0576] step: count(1502), step_time 1436.44, mean_step_time 1431.69, it/s 0.7
  7006. [0130 18:05:50 @multigpu.py:323] [p0115] step: count(1478), step_time 1457.75, mean_step_time 1435.13, it/s 0.7
  7007. [0130 18:05:50 @multigpu.py:323] [p0574] step: count(1493), step_time 1425.53, mean_step_time 1432.64, it/s 0.7
  7008. [0130 18:05:52 @multigpu.py:323] [p0115] step: count(1479), step_time 1425.16, mean_step_time 1435.64, it/s 0.7
  7009. [0130 18:05:52 @multigpu.py:323] [p0576] step: count(1503), step_time 1450.49, mean_step_time 1431.43, it/s 0.7
  7010. [0130 18:05:52 @multigpu.py:323] [p0574] step: count(1494), step_time 1463.16, mean_step_time 1433.74, it/s 0.7
  7011. [0130 18:05:53 @multigpu.py:323] [p0576] step: count(1504), step_time 1405.55, mean_step_time 1432.2, it/s 0.7
  7012. [0130 18:05:53 @multigpu.py:323] [p0115] step: count(1480), step_time 1427.44, mean_step_time 1435.81, it/s 0.7
  7013. [0130 18:05:53 @multigpu.py:323] [p0574] step: count(1495), step_time 1400.87, mean_step_time 1433.53, it/s 0.7
  7014. [0130 18:05:55 @multigpu.py:323] [p0576] step: count(1505), step_time 1428.03, mean_step_time 1433.16, it/s 0.7
  7015. [0130 18:05:55 @multigpu.py:323] [p0115] step: count(1481), step_time 1469.23, mean_step_time 1437.24, it/s 0.7
  7016. [0130 18:05:55 @multigpu.py:323] [p0574] step: count(1496), step_time 1409.19, mean_step_time 1435.85, it/s 0.7
  7017. [0130 18:05:56 @multigpu.py:323] [p0576] step: count(1506), step_time 1411.85, mean_step_time 1430.21, it/s 0.7
  7018. [0130 18:05:56 @multigpu.py:323] [p0574] step: count(1497), step_time 1441.86, mean_step_time 1433.89, it/s 0.7
  7019. [0130 18:05:56 @multigpu.py:323] [p0115] step: count(1482), step_time 1455.18, mean_step_time 1435.67, it/s 0.7
  7020. [0130 18:05:57 @multigpu.py:323] [p0576] step: count(1507), step_time 1409.49, mean_step_time 1431.85, it/s 0.7
  7021. [0130 18:05:58 @multigpu.py:323] [p0574] step: count(1498), step_time 1387.26, mean_step_time 1432.82, it/s 0.7
  7022. [0130 18:05:58 @multigpu.py:323] [p0115] step: count(1483), step_time 1448.08, mean_step_time 1438.77, it/s 0.7
  7023. sending to address tcp://p0112:61216
  7024. ##### Sending to neptune: online_score : 0.599309429129 , 1.8 #####
  7025. [u'online', 1.8]
  7026. receiving
  7027. [0130 18:05:59 @multigpu.py:323] [p0576] step: count(1508), step_time 1413.71, mean_step_time 1430.52, it/s 0.7
  7028. [0130 18:05:59 @multigpu.py:323] [p0574] step: count(1499), step_time 1474.19, mean_step_time 1434.07, it/s 0.7
  7029. [0130 18:05:59 @multigpu.py:323] [p0115] step: count(1484), step_time 1407.54, mean_step_time 1433.12, it/s 0.7
  7030. sending to address tcp://p0112:61216
  7031. ##### Sending to neptune: online_score : 0.599584539665 , 1.7 #####
  7032. [u'online', 1.7]
  7033. receiving
  7034. sending to address tcp://p0112:61216
  7035. ##### Sending to neptune: online_score : 0.599748105274 , 1.6 #####
  7036. [u'online', 1.6]
  7037. receiving
  7038. [0130 18:06:00 @multigpu.py:323] [p0576] step: count(1509), step_time 1433.24, mean_step_time 1430.87, it/s 0.7
  7039. [0130 18:06:00 @multigpu.py:323] [p0574] step: count(1500), step_time 1422.44, mean_step_time 1437.22, it/s 0.7
  7040. [0130 18:06:00 @multigpu.py:323] [p0115] step: count(1485), step_time 1418.1, mean_step_time 1433.58, it/s 0.7
  7041. sending debugging info...
  7042. sending to address tcp://p0112:61216
  7043. ##### Sending to neptune: mean_delay : 0.599946719408 , 0.0 #####
  7044. sending to address tcp://p0112:61216
  7045. ##### Sending to neptune: max_delay : 0.599946719408 , -0.0 #####
  7046. ##### Sending to neptune: min_delay : 0.599946719408 , -0.0 #####
  7047. [u'delays', [0.0, -0.0, -0.0]]
  7048. receiving
  7049. ##### Sending to neptune: cost : 0.599947186046 , -0.00632883608341 #####
  7050. sending to address tcp://p0112:61216
  7051. ##### Sending to neptune: policy_loss : 0.599947186046 , -0.0338481925428 #####
  7052. ##### Sending to neptune: xentropy_loss : 0.599947186046 , -2.28367400169 #####
  7053. ##### Sending to neptune: value_loss : 0.599947186046 , 1.50743150711 #####
  7054. ##### Sending to neptune: advantage : 0.599947186046 , 5.52561978111e-05 #####
  7055. ##### Sending to neptune: pred_reward : 0.599947186046 , 0.468498200178 #####
  7056. ##### Sending to neptune: max_logit : 0.599947186046 , 0.22827039659 #####
  7057. [u'loss', -0.00632883608341217, -0.03384819254279137, -2.2836740016937256, 1.5074315071105957, 5.525619781110436e-05, 0.46849820017814636, 0.22827039659023285]
  7058. receiving
  7059. ##### Sending to neptune: active_relus : 0.599947664407 , 9739097.15 #####
  7060. ##### Sending to neptune: dp_per_s : 0.599947664407 , 89.2479513768 #####
  7061. [u'other', 9739097.15, 89.24795137675618]
  7062. receiving
  7063. [0130 18:06:02 @multigpu.py:323] [p0576] step: count(1510), step_time 1447.74, mean_step_time 1431.25, it/s 0.7
  7064. [0130 18:06:02 @multigpu.py:323] [p0115] step: count(1486), step_time 1410.68, mean_step_time 1433.95, it/s 0.7
  7065. [0130 18:06:02 @multigpu.py:323] [p0574] step: count(1501), step_time 1438.43, mean_step_time 1436.85, it/s 0.7
  7066. [0130 18:06:03 @multigpu.py:323] [p0576] step: count(1511), step_time 1375.44, mean_step_time 1428.92, it/s 0.7
  7067. [0130 18:06:03 @multigpu.py:323] [p0115] step: count(1487), step_time 1439.88, mean_step_time 1433.69, it/s 0.7
  7068. [0130 18:06:03 @multigpu.py:323] [p0574] step: count(1502), step_time 1416.34, mean_step_time 1437.3, it/s 0.7
  7069. [0130 18:06:05 @multigpu.py:323] [p0115] step: count(1488), step_time 1430.65, mean_step_time 1432.4, it/s 0.7
  7070. [0130 18:06:05 @multigpu.py:323] [p0574] step: count(1503), step_time 1426.86, mean_step_time 1432.45, it/s 0.7
  7071. [0130 18:06:05 @multigpu.py:323] [p0576] step: count(1512), step_time 1578.32, mean_step_time 1432.39, it/s 0.7
  7072. [0130 18:06:06 @multigpu.py:323] [p0574] step: count(1504), step_time 1401.97, mean_step_time 1432.45, it/s 0.7
  7073. [0130 18:06:06 @multigpu.py:323] [p0115] step: count(1489), step_time 1416.3, mean_step_time 1432.44, it/s 0.7
  7074. [0130 18:06:06 @multigpu.py:323] [p0576] step: count(1513), step_time 1443.97, mean_step_time 1432.37, it/s 0.7
  7075. [0130 18:06:08 @multigpu.py:323] [p0576] step: count(1514), step_time 1383.8, mean_step_time 1431.03, it/s 0.7
  7076. [0130 18:06:08 @multigpu.py:323] [p0574] step: count(1505), step_time 1493.87, mean_step_time 1431.32, it/s 0.7
  7077. [0130 18:06:08 @multigpu.py:323] [p0115] step: count(1490), step_time 1512.18, mean_step_time 1435.73, it/s 0.7
  7078. [0130 18:06:09 @multigpu.py:323] [p0576] step: count(1515), step_time 1453.69, mean_step_time 1432.83, it/s 0.7
  7079. [0130 18:06:09 @multigpu.py:323] [p0115] step: count(1491), step_time 1375.9, mean_step_time 1435.42, it/s 0.7
  7080. [0130 18:06:09 @multigpu.py:323] [p0574] step: count(1506), step_time 1456.25, mean_step_time 1431.31, it/s 0.7
  7081. [0130 18:06:10 @multigpu.py:323] [p0576] step: count(1516), step_time 1394.5, mean_step_time 1429.13, it/s 0.7
  7082. [0130 18:06:10 @multigpu.py:323] [p0574] step: count(1507), step_time 1372.31, mean_step_time 1426.77, it/s 0.7
  7083. [0130 18:06:10 @multigpu.py:323] [p0115] step: count(1492), step_time 1453.23, mean_step_time 1436.06, it/s 0.7
  7084. [0130 18:06:12 @multigpu.py:323] [p0574] step: count(1508), step_time 1397.08, mean_step_time 1427.77, it/s 0.7
  7085. [0130 18:06:12 @multigpu.py:323] [p0576] step: count(1517), step_time 1456.13, mean_step_time 1431.1, it/s 0.7
  7086. [0130 18:06:12 @multigpu.py:323] [p0115] step: count(1493), step_time 1446.72, mean_step_time 1437.02, it/s 0.7
  7087. [0130 18:06:13 @multigpu.py:323] [p0576] step: count(1518), step_time 1364.42, mean_step_time 1431.1, it/s 0.7
  7088. [0130 18:06:13 @multigpu.py:323] [p0574] step: count(1509), step_time 1491.4, mean_step_time 1430.74, it/s 0.7
  7089. [0130 18:06:13 @multigpu.py:323] [p0115] step: count(1494), step_time 1462.95, mean_step_time 1440.7, it/s 0.69
  7090. [0130 18:06:15 @multigpu.py:323] [p0576] step: count(1519), step_time 1408.41, mean_step_time 1428.55, it/s 0.7
  7091. [0130 18:06:15 @multigpu.py:323] [p0574] step: count(1510), step_time 1446.65, mean_step_time 1431.77, it/s 0.7
  7092. [0130 18:06:15 @multigpu.py:323] [p0115] step: count(1495), step_time 1475.53, mean_step_time 1442.0, it/s 0.69
  7093. [0130 18:06:16 @multigpu.py:323] [p0576] step: count(1520), step_time 1400.12, mean_step_time 1426.37, it/s 0.7
  7094. [0130 18:06:16 @multigpu.py:323] [p0574] step: count(1511), step_time 1419.34, mean_step_time 1430.71, it/s 0.7
  7095. [0130 18:06:16 @multigpu.py:323] [p0115] step: count(1496), step_time 1459.53, mean_step_time 1441.35, it/s 0.69
  7096. [0130 18:06:17 @multigpu.py:323] [p0576] step: count(1521), step_time 1461.32, mean_step_time 1427.83, it/s 0.7
  7097. [0130 18:06:18 @multigpu.py:323] [p0574] step: count(1512), step_time 1409.34, mean_step_time 1429.72, it/s 0.7
  7098. [0130 18:06:18 @multigpu.py:323] [p0115] step: count(1497), step_time 1461.02, mean_step_time 1442.65, it/s 0.69
  7099. [0130 18:06:19 @multigpu.py:323] [p0576] step: count(1522), step_time 1395.16, mean_step_time 1425.77, it/s 0.7
  7100. [0130 18:06:19 @multigpu.py:323] [p0574] step: count(1513), step_time 1454.11, mean_step_time 1431.14, it/s 0.7
  7101. [0130 18:06:19 @multigpu.py:323] [p0115] step: count(1498), step_time 1485.19, mean_step_time 1444.02, it/s 0.69
  7102. [0130 18:06:20 @multigpu.py:323] [p0576] step: count(1523), step_time 1449.23, mean_step_time 1425.71, it/s 0.7
  7103. [0130 18:06:20 @multigpu.py:323] [p0574] step: count(1514), step_time 1400.58, mean_step_time 1428.02, it/s 0.7
  7104. [0130 18:06:21 @multigpu.py:323] [p0115] step: count(1499), step_time 1429.95, mean_step_time 1444.26, it/s 0.69
  7105. [0130 18:06:22 @multigpu.py:323] [p0576] step: count(1524), step_time 1418.53, mean_step_time 1426.35, it/s 0.7
  7106. [0130 18:06:22 @multigpu.py:323] [p0574] step: count(1515), step_time 1416.47, mean_step_time 1428.8, it/s 0.7
  7107. [0130 18:06:22 @multigpu.py:323] [p0115] step: count(1500), step_time 1381.96, mean_step_time 1441.99, it/s 0.69
  7108. sending debugging info...
  7109. sending to address tcp://p0112:61216
  7110. ##### Sending to neptune: mean_delay : 0.605960784952 , 0.0 #####
  7111. sending to address tcp://p0112:61216
  7112. ##### Sending to neptune: max_delay : 0.605960784952 , -0.0 #####
  7113. ##### Sending to neptune: min_delay : 0.605960784952 , -0.0 #####
  7114. [u'delays', [0.0, -0.0, -0.0]]
  7115. receiving
  7116. ##### Sending to neptune: cost : 0.605961283313 , -0.00497105671093 #####
  7117. sending to address tcp://p0112:61216
  7118. ##### Sending to neptune: policy_loss : 0.605961283313 , 0.0196081455797 #####
  7119. ##### Sending to neptune: xentropy_loss : 0.605961283313 , -2.28358197212 #####
  7120. ##### Sending to neptune: value_loss : 0.605961283313 , 1.62767851353 #####
  7121. ##### Sending to neptune: advantage : 0.605961283313 , 0.000106235434941 #####
  7122. ##### Sending to neptune: pred_reward : 0.605961283313 , 0.468377023935 #####
  7123. ##### Sending to neptune: max_logit : 0.605961283313 , 0.224669262767 #####
  7124. [u'loss', -0.0049710567109286785, 0.0196081455796957, -2.2835819721221924, 1.6276785135269165, 0.0001062354349414818, 0.468377023935318, 0.22466926276683807]
  7125. receiving
  7126. ##### Sending to neptune: active_relus : 0.605961754984 , 9702806.99 #####
  7127. ##### Sending to neptune: dp_per_s : 0.605961754984 , 89.1716589292 #####
  7128. [u'other', 9702806.99, 89.17165892922594]
  7129. receiving
  7130. [0130 18:06:23 @multigpu.py:323] [p0576] step: count(1525), step_time 1436.64, mean_step_time 1426.78, it/s 0.7
  7131. [0130 18:06:23 @multigpu.py:323] [p0574] step: count(1516), step_time 1442.65, mean_step_time 1430.47, it/s 0.7
  7132. [0130 18:06:23 @multigpu.py:323] [p0115] step: count(1501), step_time 1436.4, mean_step_time 1440.35, it/s 0.69
  7133. sending to address tcp://p0112:61216
  7134. ##### Sending to neptune: online_score : 0.606569805278 , 2.3 #####
  7135. [u'online', 2.3]
  7136. receiving
  7137. [0130 18:06:25 @multigpu.py:323] [p0576] step: count(1526), step_time 1410.51, mean_step_time 1426.72, it/s 0.7
  7138. [0130 18:06:25 @multigpu.py:323] [p0574] step: count(1517), step_time 1453.84, mean_step_time 1431.07, it/s 0.7
  7139. [0130 18:06:25 @multigpu.py:323] [p0115] step: count(1502), step_time 1457.57, mean_step_time 1440.47, it/s 0.69
  7140. sending to address tcp://p0112:61216
  7141. ##### Sending to neptune: online_score : 0.606775739657 , 1.7 #####
  7142. [u'online', 1.7]
  7143. receiving
  7144. sending to address tcp://p0112:61216
  7145. ##### Sending to neptune: online_score : 0.606916066607 , 2.2 #####
  7146. [u'online', 2.2]
  7147. receiving
  7148. [0130 18:06:26 @multigpu.py:323] [p0576] step: count(1527), step_time 1413.27, mean_step_time 1426.91, it/s 0.7
  7149. [0130 18:06:26 @multigpu.py:323] [p0574] step: count(1518), step_time 1464.12, mean_step_time 1434.91, it/s 0.7
  7150. [0130 18:06:26 @multigpu.py:323] [p0115] step: count(1503), step_time 1385.83, mean_step_time 1437.36, it/s 0.7
  7151. [0130 18:06:27 @multigpu.py:323] [p0576] step: count(1528), step_time 1397.73, mean_step_time 1426.11, it/s 0.7
  7152. [0130 18:06:28 @multigpu.py:323] [p0574] step: count(1519), step_time 1409.67, mean_step_time 1431.69, it/s 0.7
  7153. [0130 18:06:28 @multigpu.py:323] [p0115] step: count(1504), step_time 1424.17, mean_step_time 1438.19, it/s 0.7
  7154. [0130 18:06:29 @multigpu.py:323] [p0576] step: count(1529), step_time 1454.14, mean_step_time 1427.15, it/s 0.7
  7155. [0130 18:06:29 @multigpu.py:323] [p0574] step: count(1520), step_time 1500.47, mean_step_time 1435.59, it/s 0.7
  7156. [0130 18:06:29 @multigpu.py:323] [p0115] step: count(1505), step_time 1486.96, mean_step_time 1441.63, it/s 0.69
  7157. [0130 18:06:30 @multigpu.py:323] [p0576] step: count(1530), step_time 1396.72, mean_step_time 1424.6, it/s 0.7
  7158. [0130 18:06:31 @multigpu.py:323] [p0574] step: count(1521), step_time 1421.62, mean_step_time 1434.75, it/s 0.7
  7159. [0130 18:06:31 @multigpu.py:323] [p0115] step: count(1506), step_time 1443.23, mean_step_time 1443.26, it/s 0.69
  7160. [0130 18:06:32 @multigpu.py:323] [p0576] step: count(1531), step_time 1410.17, mean_step_time 1426.34, it/s 0.7
  7161. [0130 18:06:32 @multigpu.py:323] [p0574] step: count(1522), step_time 1412.52, mean_step_time 1434.56, it/s 0.7
  7162. [0130 18:06:32 @multigpu.py:323] [p0115] step: count(1507), step_time 1414.24, mean_step_time 1441.98, it/s 0.69
  7163. [0130 18:06:34 @multigpu.py:323] [p0115] step: count(1508), step_time 1425.03, mean_step_time 1441.69, it/s 0.69
  7164. [0130 18:06:34 @multigpu.py:323] [p0574] step: count(1523), step_time 1528.19, mean_step_time 1439.62, it/s 0.69
  7165. [0130 18:06:34 @multigpu.py:323] [p0576] step: count(1532), step_time 1884.68, mean_step_time 1441.66, it/s 0.69
  7166. [0130 18:06:35 @multigpu.py:323] [p0576] step: count(1533), step_time 1429.69, mean_step_time 1440.94, it/s 0.69
  7167. [0130 18:06:35 @multigpu.py:323] [p0115] step: count(1509), step_time 1459.08, mean_step_time 1443.83, it/s 0.69
  7168. [0130 18:06:35 @multigpu.py:323] [p0574] step: count(1524), step_time 1493.5, mean_step_time 1444.2, it/s 0.69
  7169. [0130 18:06:36 @multigpu.py:323] [p0576] step: count(1534), step_time 1403.82, mean_step_time 1441.94, it/s 0.69
  7170. [0130 18:06:36 @multigpu.py:323] [p0115] step: count(1510), step_time 1406.96, mean_step_time 1438.57, it/s 0.7
  7171. [0130 18:06:36 @multigpu.py:323] [p0574] step: count(1525), step_time 1449.17, mean_step_time 1441.96, it/s 0.69
  7172. [0130 18:06:38 @multigpu.py:323] [p0576] step: count(1535), step_time 1401.52, mean_step_time 1439.34, it/s 0.69
  7173. [0130 18:06:38 @multigpu.py:323] [p0115] step: count(1511), step_time 1409.91, mean_step_time 1440.27, it/s 0.69
  7174. [0130 18:06:38 @multigpu.py:323] [p0574] step: count(1526), step_time 1434.71, mean_step_time 1440.89, it/s 0.69
  7175. [0130 18:06:39 @multigpu.py:323] [p0115] step: count(1512), step_time 1424.66, mean_step_time 1438.84, it/s 0.7
  7176. [0130 18:06:39 @multigpu.py:323] [p0576] step: count(1536), step_time 1467.83, mean_step_time 1443.0, it/s 0.69
  7177. [0130 18:06:39 @multigpu.py:323] [p0574] step: count(1527), step_time 1399.48, mean_step_time 1442.25, it/s 0.69
  7178. [0130 18:06:41 @multigpu.py:323] [p0115] step: count(1513), step_time 1423.8, mean_step_time 1437.7, it/s 0.7
  7179. [0130 18:06:41 @multigpu.py:323] [p0576] step: count(1537), step_time 1449.03, mean_step_time 1442.65, it/s 0.69
  7180. [0130 18:06:41 @multigpu.py:323] [p0574] step: count(1528), step_time 1401.56, mean_step_time 1442.47, it/s 0.69
  7181. [0130 18:06:42 @multigpu.py:323] [p0115] step: count(1514), step_time 1410.1, mean_step_time 1435.06, it/s 0.7
  7182. [0130 18:06:42 @multigpu.py:323] [p0576] step: count(1538), step_time 1480.84, mean_step_time 1448.47, it/s 0.69
  7183. [0130 18:06:42 @multigpu.py:323] [p0574] step: count(1529), step_time 1472.88, mean_step_time 1441.54, it/s 0.69
  7184. [0130 18:06:43 @multigpu.py:323] [p0115] step: count(1515), step_time 1391.68, mean_step_time 1430.86, it/s 0.7
  7185. [0130 18:06:44 @multigpu.py:323] [p0574] step: count(1530), step_time 1419.66, mean_step_time 1440.19, it/s 0.69
  7186. [0130 18:06:44 @multigpu.py:323] [p0576] step: count(1539), step_time 1470.23, mean_step_time 1451.56, it/s 0.69
  7187. [0130 18:06:45 @multigpu.py:323] [p0115] step: count(1516), step_time 1472.46, mean_step_time 1431.51, it/s 0.7
  7188. [0130 18:06:45 @multigpu.py:323] [p0576] step: count(1540), step_time 1379.21, mean_step_time 1450.51, it/s 0.69
  7189. [0130 18:06:45 @multigpu.py:323] [p0574] step: count(1531), step_time 1432.77, mean_step_time 1440.87, it/s 0.69
  7190. [0130 18:06:46 @multigpu.py:323] [p0115] step: count(1517), step_time 1476.83, mean_step_time 1432.3, it/s 0.7
  7191. [0130 18:06:46 @multigpu.py:323] [p0574] step: count(1532), step_time 1408.65, mean_step_time 1440.83, it/s 0.69
  7192. [0130 18:06:46 @multigpu.py:323] [p0576] step: count(1541), step_time 1439.1, mean_step_time 1449.4, it/s 0.69
  7193. [0130 18:06:48 @multigpu.py:323] [p0115] step: count(1518), step_time 1387.91, mean_step_time 1427.44, it/s 0.7
  7194. [0130 18:06:48 @multigpu.py:323] [p0576] step: count(1542), step_time 1395.19, mean_step_time 1449.4, it/s 0.69
  7195. [0130 18:06:48 @multigpu.py:323] [p0574] step: count(1533), step_time 1452.84, mean_step_time 1440.77, it/s 0.69
  7196. [0130 18:06:49 @multigpu.py:323] [p0576] step: count(1543), step_time 1407.96, mean_step_time 1447.34, it/s 0.69
  7197. [0130 18:06:49 @multigpu.py:323] [p0115] step: count(1519), step_time 1479.98, mean_step_time 1429.94, it/s 0.7
  7198. [0130 18:06:49 @multigpu.py:323] [p0574] step: count(1534), step_time 1448.62, mean_step_time 1443.17, it/s 0.69
  7199. [0130 18:06:51 @multigpu.py:323] [p0576] step: count(1544), step_time 1377.59, mean_step_time 1445.29, it/s 0.69
  7200. [0130 18:06:51 @multigpu.py:323] [p0115] step: count(1520), step_time 1407.82, mean_step_time 1431.23, it/s 0.7
  7201. [0130 18:06:51 @multigpu.py:323] [p0574] step: count(1535), step_time 1427.36, mean_step_time 1443.72, it/s 0.69
  7202. [0130 18:06:52 @multigpu.py:323] [p0576] step: count(1545), step_time 1436.21, mean_step_time 1445.27, it/s 0.69
  7203. [0130 18:06:52 @multigpu.py:323] [p0115] step: count(1521), step_time 1458.2, mean_step_time 1432.32, it/s 0.7
  7204. [0130 18:06:52 @multigpu.py:323] [p0574] step: count(1536), step_time 1439.56, mean_step_time 1443.56, it/s 0.69
  7205. sending to address tcp://p0112:61216
  7206. ##### Sending to neptune: online_score : 0.614415989121 , 0.8 #####
  7207. [u'online', 0.8]
  7208. ##### Sending to neptune: active_workers : 0.614416100515 , 3 #####
  7209. receiving
  7210. sending to address tcp://p0112:61216
  7211. ##### Sending to neptune: online_score : 0.614469838606 , 1.7 #####
  7212. [u'online', 1.7]
  7213. receiving
  7214. [0130 18:06:53 @multigpu.py:323] [p0576] step: count(1546), step_time 1365.79, mean_step_time 1443.04, it/s 0.69
  7215. [0130 18:06:54 @multigpu.py:323] [p0115] step: count(1522), step_time 1442.97, mean_step_time 1431.59, it/s 0.7
  7216. [0130 18:06:54 @multigpu.py:323] [p0574] step: count(1537), step_time 1399.53, mean_step_time 1440.85, it/s 0.69
  7217. [0130 18:06:55 @multigpu.py:323] [p0576] step: count(1547), step_time 1463.65, mean_step_time 1445.56, it/s 0.69
  7218. [0130 18:06:55 @multigpu.py:323] [p0115] step: count(1523), step_time 1383.89, mean_step_time 1431.5, it/s 0.7
  7219. [0130 18:06:55 @multigpu.py:323] [p0574] step: count(1538), step_time 1405.73, mean_step_time 1437.93, it/s 0.7
  7220. [0130 18:06:56 @multigpu.py:323] [p0576] step: count(1548), step_time 1425.38, mean_step_time 1446.94, it/s 0.69
  7221. [0130 18:06:56 @multigpu.py:323] [p0115] step: count(1524), step_time 1458.14, mean_step_time 1433.19, it/s 0.7
  7222. [0130 18:06:57 @multigpu.py:323] [p0574] step: count(1539), step_time 1469.78, mean_step_time 1440.93, it/s 0.69
  7223. sending to address tcp://p0112:61216
  7224. ##### Sending to neptune: online_score : 0.615593735774 , 0.8 #####
  7225. [u'online', 0.8]
  7226. receiving
  7227. [0130 18:06:58 @multigpu.py:323] [p0576] step: count(1549), step_time 1398.88, mean_step_time 1444.17, it/s 0.69
  7228. [0130 18:06:58 @multigpu.py:323] [p0115] step: count(1525), step_time 1391.89, mean_step_time 1428.44, it/s 0.7
  7229. [0130 18:06:58 @multigpu.py:323] [p0574] step: count(1540), step_time 1427.82, mean_step_time 1437.3, it/s 0.7
  7230. [0130 18:06:59 @multigpu.py:323] [p0576] step: count(1550), step_time 1448.98, mean_step_time 1446.79, it/s 0.69
  7231. [0130 18:06:59 @multigpu.py:323] [p0115] step: count(1526), step_time 1389.32, mean_step_time 1425.74, it/s 0.7
  7232. [0130 18:06:59 @multigpu.py:323] [p0574] step: count(1541), step_time 1421.4, mean_step_time 1437.29, it/s 0.7
  7233. [0130 18:07:01 @multigpu.py:323] [p0115] step: count(1527), step_time 1414.52, mean_step_time 1425.76, it/s 0.7
  7234. [0130 18:07:01 @multigpu.py:323] [p0576] step: count(1551), step_time 1488.68, mean_step_time 1450.71, it/s 0.69
  7235. [0130 18:07:01 @multigpu.py:323] [p0574] step: count(1542), step_time 1428.02, mean_step_time 1438.06, it/s 0.7
  7236. [0130 18:07:02 @multigpu.py:323] [p0574] step: count(1543), step_time 1433.98, mean_step_time 1433.35, it/s 0.7
  7237. [0130 18:07:02 @multigpu.py:323] [p0115] step: count(1528), step_time 1574.99, mean_step_time 1433.26, it/s 0.7
  7238. [0130 18:07:02 @multigpu.py:323] [p0576] step: count(1552), step_time 1535.72, mean_step_time 1433.27, it/s 0.7
  7239. [0130 18:07:04 @multigpu.py:323] [p0115] step: count(1529), step_time 1408.31, mean_step_time 1430.72, it/s 0.7
  7240. [0130 18:07:04 @multigpu.py:323] [p0576] step: count(1553), step_time 1426.85, mean_step_time 1433.12, it/s 0.7
  7241. [0130 18:07:04 @multigpu.py:323] [p0574] step: count(1544), step_time 1444.66, mean_step_time 1430.91, it/s 0.7
  7242. [0130 18:07:05 @multigpu.py:323] [p0576] step: count(1554), step_time 1411.79, mean_step_time 1433.52, it/s 0.7
  7243. [0130 18:07:05 @multigpu.py:323] [p0115] step: count(1530), step_time 1433.44, mean_step_time 1432.04, it/s 0.7
  7244. [0130 18:07:05 @multigpu.py:323] [p0574] step: count(1545), step_time 1416.43, mean_step_time 1429.27, it/s 0.7
  7245. [0130 18:07:06 @multigpu.py:323] [p0576] step: count(1555), step_time 1407.51, mean_step_time 1433.82, it/s 0.7
  7246. [0130 18:07:06 @multigpu.py:323] [p0115] step: count(1531), step_time 1427.83, mean_step_time 1432.94, it/s 0.7
  7247. [0130 18:07:07 @multigpu.py:323] [p0574] step: count(1546), step_time 1441.64, mean_step_time 1429.62, it/s 0.7
  7248. [0130 18:07:08 @multigpu.py:323] [p0576] step: count(1556), step_time 1427.75, mean_step_time 1431.82, it/s 0.7
  7249. [0130 18:07:08 @multigpu.py:323] [p0115] step: count(1532), step_time 1468.38, mean_step_time 1435.12, it/s 0.7
  7250. [0130 18:07:08 @multigpu.py:323] [p0574] step: count(1547), step_time 1443.35, mean_step_time 1431.81, it/s 0.7
  7251. [0130 18:07:09 @multigpu.py:323] [p0576] step: count(1557), step_time 1440.46, mean_step_time 1431.39, it/s 0.7
  7252. [0130 18:07:09 @multigpu.py:323] [p0115] step: count(1533), step_time 1405.02, mean_step_time 1434.19, it/s 0.7
  7253. [0130 18:07:09 @multigpu.py:323] [p0574] step: count(1548), step_time 1479.32, mean_step_time 1435.7, it/s 0.7
  7254. [0130 18:07:11 @multigpu.py:323] [p0576] step: count(1558), step_time 1434.45, mean_step_time 1429.07, it/s 0.7
  7255. [0130 18:07:11 @multigpu.py:323] [p0115] step: count(1534), step_time 1436.15, mean_step_time 1435.49, it/s 0.7
  7256. [0130 18:07:11 @multigpu.py:323] [p0574] step: count(1549), step_time 1454.5, mean_step_time 1434.78, it/s 0.7
  7257. [0130 18:07:12 @multigpu.py:323] [p0576] step: count(1559), step_time 1427.64, mean_step_time 1426.94, it/s 0.7
  7258. [0130 18:07:12 @multigpu.py:323] [p0115] step: count(1535), step_time 1461.25, mean_step_time 1438.97, it/s 0.69
  7259. [0130 18:07:12 @multigpu.py:323] [p0574] step: count(1550), step_time 1408.36, mean_step_time 1434.22, it/s 0.7
  7260. [0130 18:07:14 @multigpu.py:323] [p0576] step: count(1560), step_time 1428.16, mean_step_time 1429.39, it/s 0.7
  7261. [0130 18:07:14 @multigpu.py:323] [p0115] step: count(1536), step_time 1409.0, mean_step_time 1435.79, it/s 0.7
  7262. [0130 18:07:14 @multigpu.py:323] [p0574] step: count(1551), step_time 1414.37, mean_step_time 1433.3, it/s 0.7
  7263. [0130 18:07:15 @multigpu.py:323] [p0576] step: count(1561), step_time 1409.87, mean_step_time 1427.93, it/s 0.7
  7264. [0130 18:07:15 @multigpu.py:323] [p0115] step: count(1537), step_time 1457.82, mean_step_time 1434.84, it/s 0.7
  7265. [0130 18:07:15 @multigpu.py:323] [p0574] step: count(1552), step_time 1425.49, mean_step_time 1434.14, it/s 0.7
  7266. [0130 18:07:16 @multigpu.py:323] [p0576] step: count(1562), step_time 1441.37, mean_step_time 1430.23, it/s 0.7
  7267. [0130 18:07:17 @multigpu.py:323] [p0115] step: count(1538), step_time 1399.1, mean_step_time 1435.4, it/s 0.7
  7268. [0130 18:07:17 @multigpu.py:323] [p0574] step: count(1553), step_time 1392.19, mean_step_time 1431.11, it/s 0.7
  7269. [0130 18:07:18 @multigpu.py:323] [p0576] step: count(1563), step_time 1375.9, mean_step_time 1428.63, it/s 0.7
  7270. [0130 18:07:18 @multigpu.py:323] [p0115] step: count(1539), step_time 1396.81, mean_step_time 1431.24, it/s 0.7
  7271. [0130 18:07:18 @multigpu.py:323] [p0574] step: count(1554), step_time 1443.34, mean_step_time 1430.84, it/s 0.7
  7272. [0130 18:07:19 @multigpu.py:323] [p0576] step: count(1564), step_time 1417.54, mean_step_time 1430.63, it/s 0.7
  7273. [0130 18:07:19 @multigpu.py:323] [p0115] step: count(1540), step_time 1436.86, mean_step_time 1432.7, it/s 0.7
  7274. [0130 18:07:19 @multigpu.py:323] [p0574] step: count(1555), step_time 1447.6, mean_step_time 1431.85, it/s 0.7
  7275. sending to address tcp://p0112:61216
  7276. ##### Sending to neptune: online_score : 0.622098054952 , 1.3 #####
  7277. [u'online', 1.3]
  7278. receiving
  7279. [0130 18:07:21 @multigpu.py:323] [p0576] step: count(1565), step_time 1399.06, mean_step_time 1428.77, it/s 0.7
  7280. [0130 18:07:21 @multigpu.py:323] [p0115] step: count(1541), step_time 1431.0, mean_step_time 1431.34, it/s 0.7
  7281. [0130 18:07:21 @multigpu.py:323] [p0574] step: count(1556), step_time 1404.22, mean_step_time 1430.09, it/s 0.7
  7282. [0130 18:07:22 @multigpu.py:323] [p0576] step: count(1566), step_time 1383.49, mean_step_time 1429.66, it/s 0.7
  7283. [0130 18:07:22 @multigpu.py:323] [p0574] step: count(1557), step_time 1425.37, mean_step_time 1431.38, it/s 0.7
  7284. [0130 18:07:22 @multigpu.py:323] [p0115] step: count(1542), step_time 1472.93, mean_step_time 1432.83, it/s 0.7
  7285. [0130 18:07:24 @multigpu.py:323] [p0576] step: count(1567), step_time 1476.75, mean_step_time 1430.31, it/s 0.7
  7286. [0130 18:07:24 @multigpu.py:323] [p0115] step: count(1543), step_time 1435.89, mean_step_time 1435.43, it/s 0.7
  7287. [0130 18:07:24 @multigpu.py:323] [p0574] step: count(1558), step_time 1442.92, mean_step_time 1433.24, it/s 0.7
  7288. [0130 18:07:25 @multigpu.py:323] [p0576] step: count(1568), step_time 1391.28, mean_step_time 1428.61, it/s 0.7
  7289. [0130 18:07:25 @multigpu.py:323] [p0574] step: count(1559), step_time 1423.02, mean_step_time 1430.9, it/s 0.7
  7290. [0130 18:07:25 @multigpu.py:323] [p0115] step: count(1544), step_time 1434.17, mean_step_time 1434.23, it/s 0.7
  7291. sending to address tcp://p0112:61216
  7292. ##### Sending to neptune: online_score : 0.623693410556 , 1.3 #####
  7293. [u'online', 1.3]
  7294. receiving
  7295. [0130 18:07:26 @multigpu.py:323] [p0576] step: count(1569), step_time 1386.62, mean_step_time 1427.99, it/s 0.7
  7296. [0130 18:07:27 @multigpu.py:323] [p0574] step: count(1560), step_time 1398.7, mean_step_time 1429.44, it/s 0.7
  7297. [0130 18:07:27 @multigpu.py:323] [p0115] step: count(1545), step_time 1414.45, mean_step_time 1435.36, it/s 0.7
  7298. sending to address tcp://p0112:61216
  7299. ##### Sending to neptune: online_score : 0.623916657435 , 1.0 #####
  7300. [u'online', 1.0]
  7301. receiving
  7302. [0130 18:07:28 @multigpu.py:323] [p0576] step: count(1570), step_time 1444.13, mean_step_time 1427.75, it/s 0.7
  7303. [0130 18:07:28 @multigpu.py:323] [p0574] step: count(1561), step_time 1390.77, mean_step_time 1427.91, it/s 0.7
  7304. [0130 18:07:28 @multigpu.py:323] [p0115] step: count(1546), step_time 1464.95, mean_step_time 1439.14, it/s 0.69
  7305. [0130 18:07:29 @multigpu.py:323] [p0576] step: count(1571), step_time 1393.22, mean_step_time 1422.98, it/s 0.7
  7306. [0130 18:07:29 @multigpu.py:323] [p0574] step: count(1562), step_time 1461.98, mean_step_time 1429.61, it/s 0.7
  7307. [0130 18:07:29 @multigpu.py:323] [p0115] step: count(1547), step_time 1375.92, mean_step_time 1437.21, it/s 0.7
  7308. [0130 18:07:31 @multigpu.py:323] [p0574] step: count(1563), step_time 1461.96, mean_step_time 1431.01, it/s 0.7
  7309. [0130 18:07:31 @multigpu.py:323] [p0115] step: count(1548), step_time 1447.15, mean_step_time 1430.82, it/s 0.7
  7310. [0130 18:07:31 @multigpu.py:323] [p0576] step: count(1572), step_time 1695.82, mean_step_time 1430.98, it/s 0.7
  7311. [0130 18:07:32 @multigpu.py:323] [p0574] step: count(1564), step_time 1392.05, mean_step_time 1428.38, it/s 0.7
  7312. [0130 18:07:32 @multigpu.py:323] [p0115] step: count(1549), step_time 1419.71, mean_step_time 1431.39, it/s 0.7
  7313. [0130 18:07:32 @multigpu.py:323] [p0576] step: count(1573), step_time 1422.03, mean_step_time 1430.74, it/s 0.7
  7314. [0130 18:07:34 @multigpu.py:323] [p0576] step: count(1574), step_time 1374.04, mean_step_time 1428.85, it/s 0.7
  7315. [0130 18:07:34 @multigpu.py:323] [p0574] step: count(1565), step_time 1451.87, mean_step_time 1430.15, it/s 0.7
  7316. [0130 18:07:34 @multigpu.py:323] [p0115] step: count(1550), step_time 1432.63, mean_step_time 1431.35, it/s 0.7
  7317. [0130 18:07:35 @multigpu.py:323] [p0576] step: count(1575), step_time 1461.69, mean_step_time 1431.56, it/s 0.7
  7318. [0130 18:07:35 @multigpu.py:323] [p0574] step: count(1566), step_time 1442.13, mean_step_time 1430.18, it/s 0.7
  7319. [0130 18:07:35 @multigpu.py:323] [p0115] step: count(1551), step_time 1473.36, mean_step_time 1433.63, it/s 0.7
  7320. [0130 18:07:37 @multigpu.py:323] [p0576] step: count(1576), step_time 1428.61, mean_step_time 1431.61, it/s 0.7
  7321. [0130 18:07:37 @multigpu.py:323] [p0574] step: count(1567), step_time 1442.95, mean_step_time 1430.16, it/s 0.7
  7322. [0130 18:07:37 @multigpu.py:323] [p0115] step: count(1552), step_time 1424.53, mean_step_time 1431.44, it/s 0.7
  7323. [0130 18:07:38 @multigpu.py:323] [p0576] step: count(1577), step_time 1451.4, mean_step_time 1432.15, it/s 0.7
  7324. [0130 18:07:38 @multigpu.py:323] [p0115] step: count(1553), step_time 1432.63, mean_step_time 1432.82, it/s 0.7
  7325. [0130 18:07:38 @multigpu.py:323] [p0574] step: count(1568), step_time 1483.12, mean_step_time 1430.35, it/s 0.7
  7326. [0130 18:07:39 @multigpu.py:323] [p0576] step: count(1578), step_time 1415.32, mean_step_time 1431.2, it/s 0.7
  7327. [0130 18:07:39 @multigpu.py:323] [p0115] step: count(1554), step_time 1393.06, mean_step_time 1430.66, it/s 0.7
  7328. [0130 18:07:40 @multigpu.py:323] [p0574] step: count(1569), step_time 1439.62, mean_step_time 1429.6, it/s 0.7
  7329. [0130 18:07:41 @multigpu.py:323] [p0115] step: count(1555), step_time 1408.59, mean_step_time 1428.03, it/s 0.7
  7330. [0130 18:07:41 @multigpu.py:323] [p0576] step: count(1579), step_time 1419.95, mean_step_time 1430.81, it/s 0.7
  7331. [0130 18:07:41 @multigpu.py:323] [p0574] step: count(1570), step_time 1429.09, mean_step_time 1430.64, it/s 0.7
  7332. [0130 18:07:42 @multigpu.py:323] [p0576] step: count(1580), step_time 1417.08, mean_step_time 1430.26, it/s 0.7
  7333. [0130 18:07:42 @multigpu.py:323] [p0115] step: count(1556), step_time 1468.9, mean_step_time 1431.02, it/s 0.7
  7334. [0130 18:07:42 @multigpu.py:323] [p0574] step: count(1571), step_time 1420.37, mean_step_time 1430.94, it/s 0.7
  7335. [0130 18:07:44 @multigpu.py:323] [p0576] step: count(1581), step_time 1426.62, mean_step_time 1431.1, it/s 0.7
  7336. [0130 18:07:44 @multigpu.py:323] [p0115] step: count(1557), step_time 1384.78, mean_step_time 1427.37, it/s 0.7
  7337. [0130 18:07:44 @multigpu.py:323] [p0574] step: count(1572), step_time 1442.39, mean_step_time 1431.78, it/s 0.7
  7338. [0130 18:07:45 @multigpu.py:323] [p0115] step: count(1558), step_time 1406.15, mean_step_time 1427.72, it/s 0.7
  7339. [0130 18:07:45 @multigpu.py:323] [p0576] step: count(1582), step_time 1482.99, mean_step_time 1433.18, it/s 0.7
  7340. [0130 18:07:45 @multigpu.py:323] [p0574] step: count(1573), step_time 1434.9, mean_step_time 1433.92, it/s 0.7
  7341. sending to address tcp://p0112:61216
  7342. ##### Sending to neptune: online_score : 0.629087399708 , 2.1 #####
  7343. [u'online', 2.1]
  7344. receiving
  7345. [0130 18:07:47 @multigpu.py:323] [p0115] step: count(1559), step_time 1429.28, mean_step_time 1429.35, it/s 0.7
  7346. [0130 18:07:47 @multigpu.py:323] [p0576] step: count(1583), step_time 1467.46, mean_step_time 1437.75, it/s 0.7
  7347. [0130 18:07:47 @multigpu.py:323] [p0574] step: count(1574), step_time 1448.98, mean_step_time 1434.2, it/s 0.7
  7348. [0130 18:07:48 @multigpu.py:323] [p0115] step: count(1560), step_time 1374.07, mean_step_time 1426.21, it/s 0.7
  7349. [0130 18:07:48 @multigpu.py:323] [p0574] step: count(1575), step_time 1368.49, mean_step_time 1430.24, it/s 0.7
  7350. [0130 18:07:48 @multigpu.py:323] [p0576] step: count(1584), step_time 1464.55, mean_step_time 1440.1, it/s 0.69
  7351. sending to address tcp://p0112:61216
  7352. ##### Sending to neptune: online_score : 0.630022522211 , 1.6 #####
  7353. [u'online', 1.6]
  7354. receiving
  7355. [0130 18:07:49 @multigpu.py:323] [p0115] step: count(1561), step_time 1453.9, mean_step_time 1427.35, it/s 0.7
  7356. [0130 18:07:50 @multigpu.py:323] [p0574] step: count(1576), step_time 1467.22, mean_step_time 1433.39, it/s 0.7
  7357. [0130 18:07:50 @multigpu.py:323] [p0576] step: count(1585), step_time 1512.55, mean_step_time 1445.78, it/s 0.69
  7358. [0130 18:07:51 @multigpu.py:323] [p0115] step: count(1562), step_time 1488.79, mean_step_time 1428.15, it/s 0.7
  7359. [0130 18:07:51 @multigpu.py:323] [p0574] step: count(1577), step_time 1436.96, mean_step_time 1433.97, it/s 0.7
  7360. [0130 18:07:51 @multigpu.py:323] [p0576] step: count(1586), step_time 1385.99, mean_step_time 1445.9, it/s 0.69
  7361. [0130 18:07:52 @multigpu.py:323] [p0115] step: count(1563), step_time 1464.72, mean_step_time 1429.59, it/s 0.7
  7362. [0130 18:07:52 @multigpu.py:323] [p0574] step: count(1578), step_time 1443.19, mean_step_time 1433.99, it/s 0.7
  7363. [0130 18:07:52 @multigpu.py:323] [p0576] step: count(1587), step_time 1410.31, mean_step_time 1442.58, it/s 0.69
  7364. [0130 18:07:54 @multigpu.py:323] [p0115] step: count(1564), step_time 1425.23, mean_step_time 1429.14, it/s 0.7
  7365. [0130 18:07:54 @multigpu.py:323] [p0576] step: count(1588), step_time 1406.86, mean_step_time 1443.36, it/s 0.69
  7366. [0130 18:07:54 @multigpu.py:323] [p0574] step: count(1579), step_time 1450.85, mean_step_time 1435.38, it/s 0.7
  7367. [0130 18:07:55 @multigpu.py:323] [p0576] step: count(1589), step_time 1412.81, mean_step_time 1444.67, it/s 0.69
  7368. [0130 18:07:55 @multigpu.py:323] [p0115] step: count(1565), step_time 1504.99, mean_step_time 1433.67, it/s 0.7
  7369. [0130 18:07:55 @multigpu.py:323] [p0574] step: count(1580), step_time 1418.11, mean_step_time 1436.35, it/s 0.7
  7370. [0130 18:07:57 @multigpu.py:323] [p0115] step: count(1566), step_time 1413.67, mean_step_time 1431.1, it/s 0.7
  7371. [0130 18:07:57 @multigpu.py:323] [p0576] step: count(1590), step_time 1489.01, mean_step_time 1446.91, it/s 0.69
  7372. [0130 18:07:57 @multigpu.py:323] [p0574] step: count(1581), step_time 1467.07, mean_step_time 1440.17, it/s 0.69
  7373. [0130 18:07:58 @multigpu.py:323] [p0115] step: count(1567), step_time 1436.95, mean_step_time 1434.15, it/s 0.7
  7374. [0130 18:07:58 @multigpu.py:323] [p0576] step: count(1591), step_time 1394.18, mean_step_time 1446.96, it/s 0.69
  7375. [0130 18:07:58 @multigpu.py:323] [p0574] step: count(1582), step_time 1429.64, mean_step_time 1438.55, it/s 0.7
  7376. sending to address tcp://p0112:61216
  7377. ##### Sending to neptune: online_score : 0.632678529951 , 2.6 #####
  7378. [u'online', 2.6]
  7379. ##### Sending to neptune: active_workers : 0.632678614722 , 3 #####
  7380. receiving
  7381. [0130 18:08:00 @multigpu.py:323] [p0115] step: count(1568), step_time 1510.66, mean_step_time 1437.33, it/s 0.7
  7382. [0130 18:08:00 @multigpu.py:323] [p0576] step: count(1592), step_time 1487.64, mean_step_time 1436.55, it/s 0.7
  7383. [0130 18:08:00 @multigpu.py:323] [p0574] step: count(1583), step_time 1436.04, mean_step_time 1437.25, it/s 0.7
  7384. [0130 18:08:01 @multigpu.py:323] [p0576] step: count(1593), step_time 1392.4, mean_step_time 1435.07, it/s 0.7
  7385. [0130 18:08:01 @multigpu.py:323] [p0574] step: count(1584), step_time 1434.4, mean_step_time 1439.37, it/s 0.69
  7386. [0130 18:08:01 @multigpu.py:323] [p0115] step: count(1569), step_time 1487.45, mean_step_time 1440.72, it/s 0.69
  7387. [0130 18:08:02 @multigpu.py:323] [p0576] step: count(1594), step_time 1403.35, mean_step_time 1436.54, it/s 0.7
  7388. [0130 18:08:02 @multigpu.py:323] [p0574] step: count(1585), step_time 1435.68, mean_step_time 1438.56, it/s 0.7
  7389. [0130 18:08:03 @multigpu.py:323] [p0115] step: count(1570), step_time 1410.24, mean_step_time 1439.6, it/s 0.69
  7390. [0130 18:08:04 @multigpu.py:323] [p0576] step: count(1595), step_time 1445.1, mean_step_time 1435.71, it/s 0.7
  7391. [0130 18:08:04 @multigpu.py:323] [p0574] step: count(1586), step_time 1411.55, mean_step_time 1437.03, it/s 0.7
  7392. [0130 18:08:04 @multigpu.py:323] [p0115] step: count(1571), step_time 1419.73, mean_step_time 1436.92, it/s 0.7
  7393. [0130 18:08:05 @multigpu.py:323] [p0576] step: count(1596), step_time 1388.89, mean_step_time 1433.72, it/s 0.7
  7394. [0130 18:08:05 @multigpu.py:323] [p0574] step: count(1587), step_time 1495.72, mean_step_time 1439.67, it/s 0.69
  7395. [0130 18:08:05 @multigpu.py:323] [p0115] step: count(1572), step_time 1478.4, mean_step_time 1439.61, it/s 0.69
  7396. [0130 18:08:07 @multigpu.py:323] [p0576] step: count(1597), step_time 1416.04, mean_step_time 1431.95, it/s 0.7
  7397. [0130 18:08:07 @multigpu.py:323] [p0574] step: count(1588), step_time 1388.06, mean_step_time 1434.92, it/s 0.7
  7398. [0130 18:08:07 @multigpu.py:323] [p0115] step: count(1573), step_time 1472.38, mean_step_time 1441.6, it/s 0.69
  7399. [0130 18:08:08 @multigpu.py:323] [p0576] step: count(1598), step_time 1402.14, mean_step_time 1431.3, it/s 0.7
  7400. [0130 18:08:08 @multigpu.py:323] [p0574] step: count(1589), step_time 1437.85, mean_step_time 1434.83, it/s 0.7
  7401. [0130 18:08:08 @multigpu.py:323] [p0115] step: count(1574), step_time 1455.63, mean_step_time 1444.73, it/s 0.69
  7402. [0130 18:08:10 @multigpu.py:323] [p0576] step: count(1599), step_time 1450.04, mean_step_time 1432.8, it/s 0.7
  7403. [0130 18:08:10 @multigpu.py:323] [p0574] step: count(1590), step_time 1438.73, mean_step_time 1435.31, it/s 0.7
  7404. [0130 18:08:10 @multigpu.py:323] [p0115] step: count(1575), step_time 1401.08, mean_step_time 1444.35, it/s 0.69
  7405. [0130 18:08:11 @multigpu.py:323] [p0576] step: count(1600), step_time 1404.48, mean_step_time 1432.17, it/s 0.7
  7406. sending debugging info...
  7407. sending to address tcp://p0112:61216
  7408. ##### Sending to neptune: mean_delay : 0.636195923289 , 0.0 #####
  7409. ##### Sending to neptune: max_delay : 0.636195923289 , -0.0 #####
  7410. sending to address tcp://p0112:61216
  7411. ##### Sending to neptune: min_delay : 0.636195923289 , -0.0 #####
  7412. [u'delays', [0.0, -0.0, -0.0]]
  7413. receiving
  7414. ##### Sending to neptune: cost : 0.636196420259 , -0.00467890035361 #####
  7415. sending to address tcp://p0112:61216
  7416. ##### Sending to neptune: policy_loss : 0.636196420259 , 0.0171289928257 #####
  7417. ##### Sending to neptune: xentropy_loss : 0.636196420259 , -2.28448009491 #####
  7418. ##### Sending to neptune: value_loss : 0.636196420259 , 1.66845166683 #####
  7419. ##### Sending to neptune: advantage : 0.636196420259 , -0.000201532704523 #####
  7420. ##### Sending to neptune: pred_reward : 0.636196420259 , 0.438157171011 #####
  7421. ##### Sending to neptune: max_logit : 0.636196420259 , 0.21130207181 #####
  7422. [u'loss', -0.004678900353610516, 0.017128992825746536, -2.284480094909668, 1.6684516668319702, -0.00020153270452283323, 0.43815717101097107, 0.21130207180976868]
  7423. receiving
  7424. ##### Sending to neptune: active_relus : 0.636196913587 , 9650551.52 #####
  7425. ##### Sending to neptune: dp_per_s : 0.636196913587 , 89.2264312438 #####
  7426. [u'other', 9650551.52, 89.22643124376607]
  7427. receiving
  7428. [0130 18:08:11 @multigpu.py:323] [p0574] step: count(1591), step_time 1391.27, mean_step_time 1433.85, it/s 0.7
  7429. [0130 18:08:11 @multigpu.py:323] [p0115] step: count(1576), step_time 1421.31, mean_step_time 1441.97, it/s 0.69
  7430. [0130 18:08:12 @multigpu.py:323] [p0576] step: count(1601), step_time 1404.77, mean_step_time 1431.08, it/s 0.7
  7431. [0130 18:08:12 @multigpu.py:323] [p0574] step: count(1592), step_time 1449.92, mean_step_time 1434.23, it/s 0.7
  7432. [0130 18:08:13 @multigpu.py:323] [p0115] step: count(1577), step_time 1391.85, mean_step_time 1442.32, it/s 0.69
  7433. sending to address tcp://p0112:61216
  7434. ##### Sending to neptune: online_score : 0.636856070823 , 1.2 #####
  7435. [u'online', 1.2]
  7436. receiving
  7437. [0130 18:08:14 @multigpu.py:323] [p0576] step: count(1602), step_time 1425.91, mean_step_time 1428.22, it/s 0.7
  7438. [0130 18:08:14 @multigpu.py:323] [p0574] step: count(1593), step_time 1384.73, mean_step_time 1431.72, it/s 0.7
  7439. [0130 18:08:14 @multigpu.py:323] [p0115] step: count(1578), step_time 1428.74, mean_step_time 1443.45, it/s 0.69
  7440. [0130 18:08:15 @multigpu.py:323] [p0576] step: count(1603), step_time 1414.6, mean_step_time 1425.58, it/s 0.7
  7441. [0130 18:08:15 @multigpu.py:323] [p0574] step: count(1594), step_time 1450.8, mean_step_time 1431.81, it/s 0.7
  7442. [0130 18:08:15 @multigpu.py:323] [p0115] step: count(1579), step_time 1377.69, mean_step_time 1440.87, it/s 0.69
  7443. sending to address tcp://p0112:61216
  7444. ##### Sending to neptune: online_score : 0.637642319401 , 1.5 #####
  7445. [u'online', 1.5]
  7446. receiving
  7447. [0130 18:08:17 @multigpu.py:323] [p0576] step: count(1604), step_time 1430.78, mean_step_time 1423.89, it/s 0.7
  7448. [0130 18:08:17 @multigpu.py:323] [p0574] step: count(1595), step_time 1372.67, mean_step_time 1432.02, it/s 0.7
  7449. [0130 18:08:17 @multigpu.py:323] [p0115] step: count(1580), step_time 1411.3, mean_step_time 1442.74, it/s 0.69
  7450. [0130 18:08:18 @multigpu.py:323] [p0576] step: count(1605), step_time 1378.87, mean_step_time 1417.21, it/s 0.71
  7451. [0130 18:08:18 @multigpu.py:323] [p0574] step: count(1596), step_time 1486.99, mean_step_time 1433.01, it/s 0.7
  7452. [0130 18:08:18 @multigpu.py:323] [p0115] step: count(1581), step_time 1444.54, mean_step_time 1442.27, it/s 0.69
  7453. [0130 18:08:19 @multigpu.py:323] [p0576] step: count(1606), step_time 1449.49, mean_step_time 1420.38, it/s 0.7
  7454. [0130 18:08:20 @multigpu.py:323] [p0574] step: count(1597), step_time 1386.55, mean_step_time 1430.49, it/s 0.7
  7455. [0130 18:08:20 @multigpu.py:323] [p0115] step: count(1582), step_time 1479.48, mean_step_time 1441.8, it/s 0.69
  7456. [0130 18:08:21 @multigpu.py:323] [p0576] step: count(1607), step_time 1398.84, mean_step_time 1419.81, it/s 0.7
  7457. [0130 18:08:21 @multigpu.py:323] [p0574] step: count(1598), step_time 1411.25, mean_step_time 1428.89, it/s 0.7
  7458. [0130 18:08:21 @multigpu.py:323] [p0115] step: count(1583), step_time 1442.11, mean_step_time 1440.67, it/s 0.69
  7459. sending to address tcp://p0112:61216
  7460. ##### Sending to neptune: online_score : 0.639158487452 , 1.4 #####
  7461. [u'online', 1.4]
  7462. receiving
  7463. [0130 18:08:22 @multigpu.py:323] [p0576] step: count(1608), step_time 1409.88, mean_step_time 1419.96, it/s 0.7
  7464. [0130 18:08:22 @multigpu.py:323] [p0574] step: count(1599), step_time 1403.72, mean_step_time 1426.54, it/s 0.7
  7465. [0130 18:08:23 @multigpu.py:323] [p0115] step: count(1584), step_time 1416.71, mean_step_time 1440.25, it/s 0.69
  7466. [0130 18:08:24 @multigpu.py:323] [p0576] step: count(1609), step_time 1456.66, mean_step_time 1422.15, it/s 0.7
  7467. [0130 18:08:24 @multigpu.py:323] [p0574] step: count(1600), step_time 1426.08, mean_step_time 1426.94, it/s 0.7
  7468. sending debugging info...
  7469. sending to address tcp://p0112:61216
  7470. ##### Sending to neptune: mean_delay : 0.63978229026 , 0.0 #####
  7471. sending to address tcp://p0112:61216
  7472. ##### Sending to neptune: max_delay : 0.63978229026 , -0.0 #####
  7473. ##### Sending to neptune: min_delay : 0.63978229026 , -0.0 #####
  7474. [u'delays', [0.0, -0.0, -0.0]]
  7475. receiving
  7476. ##### Sending to neptune: cost : 0.639782739414 , -0.00757606513798 #####
  7477. ##### Sending to neptune: policy_loss : 0.639782739414 , -0.300617456436 #####
  7478. sending to address tcp://p0112:61216
  7479. ##### Sending to neptune: xentropy_loss : 0.639782739414 , -2.28436470032 #####
  7480. ##### Sending to neptune: value_loss : 0.639782739414 , 1.61524569988 #####
  7481. ##### Sending to neptune: advantage : 0.639782739414 , 0.00124924443662 #####
  7482. ##### Sending to neptune: pred_reward : 0.639782739414 , 0.441886663437 #####
  7483. ##### Sending to neptune: max_logit : 0.639782739414 , 0.210331350565 #####
  7484. [u'loss', -0.0075760651379823685, -0.3006174564361572, -2.284364700317383, 1.6152456998825073, 0.001249244436621666, 0.44188666343688965, 0.21033135056495667]
  7485. receiving
  7486. ##### Sending to neptune: active_relus : 0.63978320082 , 9663250.99 #####
  7487. ##### Sending to neptune: dp_per_s : 0.63978320082 , 89.2543382974 #####
  7488. [u'other', 9663250.99, 89.25433829743697]
  7489. receiving
  7490. [0130 18:08:24 @multigpu.py:323] [p0115] step: count(1585), step_time 1464.29, mean_step_time 1438.21, it/s 0.7
  7491. [0130 18:08:25 @multigpu.py:323] [p0576] step: count(1610), step_time 1392.29, mean_step_time 1417.32, it/s 0.71
  7492. [0130 18:08:25 @multigpu.py:323] [p0574] step: count(1601), step_time 1506.01, mean_step_time 1428.88, it/s 0.7
  7493. [0130 18:08:25 @multigpu.py:323] [p0115] step: count(1586), step_time 1439.58, mean_step_time 1439.51, it/s 0.69
  7494. [0130 18:08:27 @multigpu.py:323] [p0576] step: count(1611), step_time 1439.59, mean_step_time 1419.59, it/s 0.7
  7495. [0130 18:08:27 @multigpu.py:323] [p0574] step: count(1602), step_time 1467.12, mean_step_time 1430.76, it/s 0.7
  7496. [0130 18:08:27 @multigpu.py:323] [p0115] step: count(1587), step_time 1485.33, mean_step_time 1441.92, it/s 0.69
  7497. [0130 18:08:28 @multigpu.py:323] [p0574] step: count(1603), step_time 1593.5, mean_step_time 1438.63, it/s 0.7
  7498. [0130 18:08:28 @multigpu.py:323] [p0115] step: count(1588), step_time 1445.44, mean_step_time 1438.66, it/s 0.7
  7499. [0130 18:08:28 @multigpu.py:323] [p0576] step: count(1612), step_time 1871.95, mean_step_time 1438.8, it/s 0.7
  7500. [0130 18:08:30 @multigpu.py:323] [p0576] step: count(1613), step_time 1405.8, mean_step_time 1439.47, it/s 0.69
  7501. [0130 18:08:30 @multigpu.py:323] [p0574] step: count(1604), step_time 1428.88, mean_step_time 1438.35, it/s 0.7
  7502. [0130 18:08:30 @multigpu.py:323] [p0115] step: count(1589), step_time 1458.02, mean_step_time 1437.19, it/s 0.7
  7503. [0130 18:08:31 @multigpu.py:323] [p0115] step: count(1590), step_time 1395.07, mean_step_time 1436.43, it/s 0.7
  7504. [0130 18:08:31 @multigpu.py:323] [p0576] step: count(1614), step_time 1448.84, mean_step_time 1441.75, it/s 0.69
  7505. [0130 18:08:31 @multigpu.py:323] [p0574] step: count(1605), step_time 1439.13, mean_step_time 1438.53, it/s 0.7
  7506. [0130 18:08:33 @multigpu.py:323] [p0576] step: count(1615), step_time 1392.18, mean_step_time 1439.1, it/s 0.69
  7507. [0130 18:08:33 @multigpu.py:323] [p0574] step: count(1606), step_time 1439.27, mean_step_time 1439.91, it/s 0.69
  7508. [0130 18:08:33 @multigpu.py:323] [p0115] step: count(1591), step_time 1477.38, mean_step_time 1439.32, it/s 0.69
  7509. [0130 18:08:34 @multigpu.py:323] [p0576] step: count(1616), step_time 1441.12, mean_step_time 1441.71, it/s 0.69
  7510. [0130 18:08:34 @multigpu.py:323] [p0115] step: count(1592), step_time 1406.52, mean_step_time 1435.72, it/s 0.7
  7511. [0130 18:08:34 @multigpu.py:323] [p0574] step: count(1607), step_time 1473.3, mean_step_time 1438.79, it/s 0.7
  7512. [0130 18:08:35 @multigpu.py:323] [p0576] step: count(1617), step_time 1411.52, mean_step_time 1441.49, it/s 0.69
  7513. [0130 18:08:36 @multigpu.py:323] [p0115] step: count(1593), step_time 1405.83, mean_step_time 1432.39, it/s 0.7
  7514. [0130 18:08:36 @multigpu.py:323] [p0574] step: count(1608), step_time 1442.58, mean_step_time 1441.52, it/s 0.69
  7515. [0130 18:08:37 @multigpu.py:323] [p0576] step: count(1618), step_time 1444.46, mean_step_time 1443.6, it/s 0.69
  7516. [0130 18:08:37 @multigpu.py:323] [p0115] step: count(1594), step_time 1419.51, mean_step_time 1430.59, it/s 0.7
  7517. [0130 18:08:37 @multigpu.py:323] [p0574] step: count(1609), step_time 1431.5, mean_step_time 1441.2, it/s 0.69
  7518. [0130 18:08:38 @multigpu.py:323] [p0576] step: count(1619), step_time 1389.02, mean_step_time 1440.55, it/s 0.69
  7519. [0130 18:08:38 @multigpu.py:323] [p0115] step: count(1595), step_time 1395.99, mean_step_time 1430.33, it/s 0.7
  7520. [0130 18:08:38 @multigpu.py:323] [p0574] step: count(1610), step_time 1367.22, mean_step_time 1437.62, it/s 0.7
  7521. [0130 18:08:40 @multigpu.py:323] [p0115] step: count(1596), step_time 1464.15, mean_step_time 1432.48, it/s 0.7
  7522. [0130 18:08:40 @multigpu.py:323] [p0576] step: count(1620), step_time 1519.98, mean_step_time 1446.33, it/s 0.69
  7523. [0130 18:08:40 @multigpu.py:323] [p0574] step: count(1611), step_time 1466.99, mean_step_time 1441.41, it/s 0.69
  7524. [0130 18:08:41 @multigpu.py:323] [p0115] step: count(1597), step_time 1443.96, mean_step_time 1435.08, it/s 0.7
  7525. [0130 18:08:41 @multigpu.py:323] [p0576] step: count(1621), step_time 1432.41, mean_step_time 1447.71, it/s 0.69
  7526. [0130 18:08:41 @multigpu.py:323] [p0574] step: count(1612), step_time 1401.12, mean_step_time 1438.97, it/s 0.69
  7527. [0130 18:08:43 @multigpu.py:323] [p0574] step: count(1613), step_time 1387.32, mean_step_time 1439.1, it/s 0.69
  7528. [0130 18:08:43 @multigpu.py:323] [p0576] step: count(1622), step_time 1411.45, mean_step_time 1446.99, it/s 0.69
  7529. [0130 18:08:43 @multigpu.py:323] [p0115] step: count(1598), step_time 1448.26, mean_step_time 1436.06, it/s 0.7
  7530. [0130 18:08:44 @multigpu.py:323] [p0576] step: count(1623), step_time 1371.52, mean_step_time 1444.83, it/s 0.69
  7531. [0130 18:08:44 @multigpu.py:323] [p0574] step: count(1614), step_time 1392.72, mean_step_time 1436.2, it/s 0.7
  7532. [0130 18:08:44 @multigpu.py:323] [p0115] step: count(1599), step_time 1430.72, mean_step_time 1438.71, it/s 0.7
  7533. sending to address tcp://p0112:61216
  7534. ##### Sending to neptune: online_score : 0.645635401607 , 2.3 #####
  7535. [u'online', 2.3]
  7536. receiving
  7537. [0130 18:08:45 @multigpu.py:323] [p0574] step: count(1615), step_time 1387.46, mean_step_time 1436.94, it/s 0.7
  7538. [0130 18:08:46 @multigpu.py:323] [p0576] step: count(1624), step_time 1445.4, mean_step_time 1445.56, it/s 0.69
  7539. [0130 18:08:46 @multigpu.py:323] [p0115] step: count(1600), step_time 1383.25, mean_step_time 1437.31, it/s 0.7
  7540. sending debugging info...
  7541. sending to address tcp://p0112:61216
  7542. ##### Sending to neptune: mean_delay : 0.645811170273 , 0.0 #####
  7543. sending to address tcp://p0112:61216
  7544. ##### Sending to neptune: max_delay : 0.645811170273 , -0.0 #####
  7545. ##### Sending to neptune: min_delay : 0.645811170273 , -0.0 #####
  7546. [u'delays', [0.0, -0.0, -0.0]]
  7547. receiving
  7548. ##### Sending to neptune: cost : 0.645811683271 , -0.00390666676685 #####
  7549. sending to address tcp://p0112:61216
  7550. ##### Sending to neptune: policy_loss : 0.645811683271 , 0.224651888013 #####
  7551. ##### Sending to neptune: xentropy_loss : 0.645811683271 , -2.28408169746 #####
  7552. ##### Sending to neptune: value_loss : 0.645811683271 , 1.55937635899 #####
  7553. ##### Sending to neptune: advantage : 0.645811683271 , -0.00111011322588 #####
  7554. ##### Sending to neptune: pred_reward : 0.645811683271 , 0.442633777857 #####
  7555. ##### Sending to neptune: max_logit : 0.645811683271 , 0.208173751831 #####
  7556. [u'loss', -0.00390666676685214, 0.22465188801288605, -2.2840816974639893, 1.5593763589859009, -0.001110113225877285, 0.4426337778568268, 0.2081737518310547]
  7557. receiving
  7558. ##### Sending to neptune: active_relus : 0.645812171102 , 9614416.76 #####
  7559. ##### Sending to neptune: dp_per_s : 0.645812171102 , 89.1838954286 #####
  7560. [u'other', 9614416.76, 89.18389542861547]
  7561. receiving
  7562. [0130 18:08:47 @multigpu.py:323] [p0574] step: count(1616), step_time 1464.59, mean_step_time 1435.82, it/s 0.7
  7563. [0130 18:08:47 @multigpu.py:323] [p0576] step: count(1625), step_time 1436.61, mean_step_time 1448.45, it/s 0.69
  7564. [0130 18:08:47 @multigpu.py:323] [p0115] step: count(1601), step_time 1436.9, mean_step_time 1436.92, it/s 0.7
  7565. [0130 18:08:48 @multigpu.py:323] [p0574] step: count(1617), step_time 1395.09, mean_step_time 1436.24, it/s 0.7
  7566. [0130 18:08:48 @multigpu.py:323] [p0576] step: count(1626), step_time 1392.04, mean_step_time 1445.58, it/s 0.69
  7567. [0130 18:08:48 @multigpu.py:323] [p0115] step: count(1602), step_time 1412.95, mean_step_time 1433.6, it/s 0.7
  7568. [0130 18:08:50 @multigpu.py:323] [p0574] step: count(1618), step_time 1421.31, mean_step_time 1436.75, it/s 0.7
  7569. [0130 18:08:50 @multigpu.py:323] [p0576] step: count(1627), step_time 1424.94, mean_step_time 1446.88, it/s 0.69
  7570. [0130 18:08:50 @multigpu.py:323] [p0115] step: count(1603), step_time 1481.28, mean_step_time 1435.56, it/s 0.7
  7571. [0130 18:08:51 @multigpu.py:323] [p0574] step: count(1619), step_time 1422.31, mean_step_time 1437.67, it/s 0.7
  7572. [0130 18:08:51 @multigpu.py:323] [p0576] step: count(1628), step_time 1484.54, mean_step_time 1450.62, it/s 0.69
  7573. [0130 18:08:51 @multigpu.py:323] [p0115] step: count(1604), step_time 1440.25, mean_step_time 1436.73, it/s 0.7
  7574. sending to address tcp://p0112:61216
  7575. ##### Sending to neptune: online_score : 0.647553682195 , 1.6 #####
  7576. [u'online', 1.6]
  7577. receiving
  7578. [0130 18:08:53 @multigpu.py:323] [p0574] step: count(1620), step_time 1429.35, mean_step_time 1437.84, it/s 0.7
  7579. [0130 18:08:53 @multigpu.py:323] [p0576] step: count(1629), step_time 1368.05, mean_step_time 1446.19, it/s 0.69
  7580. [0130 18:08:53 @multigpu.py:323] [p0115] step: count(1605), step_time 1469.65, mean_step_time 1437.0, it/s 0.7
  7581. [0130 18:08:54 @multigpu.py:323] [p0574] step: count(1621), step_time 1427.32, mean_step_time 1433.9, it/s 0.7
  7582. [0130 18:08:54 @multigpu.py:323] [p0576] step: count(1630), step_time 1407.52, mean_step_time 1446.95, it/s 0.69
  7583. [0130 18:08:54 @multigpu.py:323] [p0115] step: count(1606), step_time 1435.4, mean_step_time 1436.79, it/s 0.7
  7584. sending to address tcp://p0112:61216
  7585. ##### Sending to neptune: online_score : 0.648334509134 , 2.7 #####
  7586. [u'online', 2.7]
  7587. receiving
  7588. [0130 18:08:55 @multigpu.py:323] [p0574] step: count(1622), step_time 1394.61, mean_step_time 1430.28, it/s 0.7
  7589. [0130 18:08:55 @multigpu.py:323] [p0576] step: count(1631), step_time 1451.09, mean_step_time 1447.52, it/s 0.69
  7590. [0130 18:08:56 @multigpu.py:323] [p0115] step: count(1607), step_time 1380.77, mean_step_time 1431.57, it/s 0.7
  7591. [0130 18:08:57 @multigpu.py:323] [p0115] step: count(1608), step_time 1406.7, mean_step_time 1429.63, it/s 0.7
  7592. [0130 18:08:57 @multigpu.py:323] [p0574] step: count(1623), step_time 1581.6, mean_step_time 1429.68, it/s 0.7
  7593. [0130 18:08:57 @multigpu.py:323] [p0576] step: count(1632), step_time 1514.3, mean_step_time 1429.64, it/s 0.7
  7594. [0130 18:08:58 @multigpu.py:323] [p0576] step: count(1633), step_time 1404.11, mean_step_time 1429.56, it/s 0.7
  7595. [0130 18:08:58 @multigpu.py:323] [p0115] step: count(1609), step_time 1422.43, mean_step_time 1427.85, it/s 0.7
  7596. [0130 18:08:58 @multigpu.py:323] [p0574] step: count(1624), step_time 1465.75, mean_step_time 1431.53, it/s 0.7
  7597. [0130 18:09:00 @multigpu.py:323] [p0115] step: count(1610), step_time 1382.41, mean_step_time 1427.22, it/s 0.7
  7598. [0130 18:09:00 @multigpu.py:323] [p0576] step: count(1634), step_time 1471.18, mean_step_time 1430.67, it/s 0.7
  7599. [0130 18:09:00 @multigpu.py:323] [p0574] step: count(1625), step_time 1425.24, mean_step_time 1430.83, it/s 0.7
  7600. [0130 18:09:01 @multigpu.py:323] [p0115] step: count(1611), step_time 1441.52, mean_step_time 1425.42, it/s 0.7
  7601. [0130 18:09:01 @multigpu.py:323] [p0574] step: count(1626), step_time 1396.18, mean_step_time 1428.68, it/s 0.7
  7602. [0130 18:09:01 @multigpu.py:323] [p0576] step: count(1635), step_time 1431.17, mean_step_time 1432.62, it/s 0.7
  7603. [0130 18:09:03 @multigpu.py:323] [p0574] step: count(1627), step_time 1411.9, mean_step_time 1425.61, it/s 0.7
  7604. [0130 18:09:03 @multigpu.py:323] [p0115] step: count(1612), step_time 1476.86, mean_step_time 1428.94, it/s 0.7
  7605. [0130 18:09:03 @multigpu.py:323] [p0576] step: count(1636), step_time 1424.65, mean_step_time 1431.8, it/s 0.7
  7606. [0130 18:09:04 @multigpu.py:323] [p0574] step: count(1628), step_time 1422.29, mean_step_time 1424.59, it/s 0.7
  7607. [0130 18:09:04 @multigpu.py:323] [p0576] step: count(1637), step_time 1418.3, mean_step_time 1432.14, it/s 0.7
  7608. [0130 18:09:04 @multigpu.py:323] [p0115] step: count(1613), step_time 1430.73, mean_step_time 1430.18, it/s 0.7
  7609. [0130 18:09:06 @multigpu.py:323] [p0574] step: count(1629), step_time 1453.18, mean_step_time 1425.68, it/s 0.7
  7610. [0130 18:09:06 @multigpu.py:323] [p0115] step: count(1614), step_time 1443.34, mean_step_time 1431.38, it/s 0.7
  7611. [0130 18:09:06 @multigpu.py:323] [p0576] step: count(1638), step_time 1464.71, mean_step_time 1433.15, it/s 0.7
  7612. [0130 18:09:07 @multigpu.py:323] [p0574] step: count(1630), step_time 1388.58, mean_step_time 1426.75, it/s 0.7
  7613. [0130 18:09:07 @multigpu.py:323] [p0576] step: count(1639), step_time 1360.35, mean_step_time 1431.72, it/s 0.7
  7614. [0130 18:09:07 @multigpu.py:323] [p0115] step: count(1615), step_time 1453.92, mean_step_time 1434.27, it/s 0.7
  7615. [0130 18:09:08 @multigpu.py:323] [p0574] step: count(1631), step_time 1412.25, mean_step_time 1424.01, it/s 0.7
  7616. [0130 18:09:08 @multigpu.py:323] [p0576] step: count(1640), step_time 1414.76, mean_step_time 1426.45, it/s 0.7
  7617. [0130 18:09:08 @multigpu.py:323] [p0115] step: count(1616), step_time 1444.89, mean_step_time 1433.31, it/s 0.7
  7618. [0130 18:09:10 @multigpu.py:323] [p0574] step: count(1632), step_time 1370.91, mean_step_time 1422.5, it/s 0.7
  7619. [0130 18:09:10 @multigpu.py:323] [p0576] step: count(1641), step_time 1455.55, mean_step_time 1427.61, it/s 0.7
  7620. [0130 18:09:10 @multigpu.py:323] [p0115] step: count(1617), step_time 1424.07, mean_step_time 1432.32, it/s 0.7
  7621. [0130 18:09:11 @multigpu.py:323] [p0574] step: count(1633), step_time 1451.77, mean_step_time 1425.72, it/s 0.7
  7622. [0130 18:09:11 @multigpu.py:323] [p0576] step: count(1642), step_time 1383.04, mean_step_time 1426.19, it/s 0.7
  7623. [0130 18:09:11 @multigpu.py:323] [p0115] step: count(1618), step_time 1362.2, mean_step_time 1428.01, it/s 0.7
  7624. [0130 18:09:13 @multigpu.py:323] [p0574] step: count(1634), step_time 1398.64, mean_step_time 1426.02, it/s 0.7
  7625. [0130 18:09:13 @multigpu.py:323] [p0576] step: count(1643), step_time 1435.56, mean_step_time 1429.39, it/s 0.7
  7626. [0130 18:09:13 @multigpu.py:323] [p0115] step: count(1619), step_time 1441.69, mean_step_time 1428.56, it/s 0.7
  7627. [0130 18:09:14 @multigpu.py:323] [p0574] step: count(1635), step_time 1430.21, mean_step_time 1428.15, it/s 0.7
  7628. [0130 18:09:14 @multigpu.py:323] [p0115] step: count(1620), step_time 1395.04, mean_step_time 1429.15, it/s 0.7
  7629. [0130 18:09:14 @multigpu.py:323] [p0576] step: count(1644), step_time 1462.5, mean_step_time 1430.25, it/s 0.7
  7630. [0130 18:09:15 @multigpu.py:323] [p0574] step: count(1636), step_time 1445.55, mean_step_time 1427.2, it/s 0.7
  7631. [0130 18:09:16 @multigpu.py:323] [p0115] step: count(1621), step_time 1430.3, mean_step_time 1428.82, it/s 0.7
  7632. [0130 18:09:16 @multigpu.py:323] [p0576] step: count(1645), step_time 1464.91, mean_step_time 1431.66, it/s 0.7
  7633. sending to address tcp://p0112:61216
  7634. ##### Sending to neptune: online_score : 0.654332980514 , 1.8 #####
  7635. [u'online', 1.8]
  7636. ##### Sending to neptune: active_workers : 0.654333023826 , 3 #####
  7637. receiving
  7638. [0130 18:09:17 @multigpu.py:323] [p0574] step: count(1637), step_time 1399.36, mean_step_time 1427.42, it/s 0.7
  7639. [0130 18:09:17 @multigpu.py:323] [p0115] step: count(1622), step_time 1442.2, mean_step_time 1430.28, it/s 0.7
  7640. [0130 18:09:17 @multigpu.py:323] [p0576] step: count(1646), step_time 1404.78, mean_step_time 1432.3, it/s 0.7
  7641. [0130 18:09:18 @multigpu.py:323] [p0574] step: count(1638), step_time 1451.67, mean_step_time 1428.93, it/s 0.7
  7642. [0130 18:09:18 @multigpu.py:323] [p0115] step: count(1623), step_time 1412.34, mean_step_time 1426.84, it/s 0.7
  7643. [0130 18:09:18 @multigpu.py:323] [p0576] step: count(1647), step_time 1444.22, mean_step_time 1433.26, it/s 0.7
  7644. sending to address tcp://p0112:61216
  7645. ##### Sending to neptune: online_score : 0.655126712455 , 1.7 #####
  7646. [u'online', 1.7]
  7647. receiving
  7648. [0130 18:09:20 @multigpu.py:323] [p0574] step: count(1639), step_time 1438.08, mean_step_time 1429.72, it/s 0.7
  7649. [0130 18:09:20 @multigpu.py:323] [p0115] step: count(1624), step_time 1449.13, mean_step_time 1427.28, it/s 0.7
  7650. [0130 18:09:20 @multigpu.py:323] [p0576] step: count(1648), step_time 1473.0, mean_step_time 1432.69, it/s 0.7
  7651. [0130 18:09:21 @multigpu.py:323] [p0574] step: count(1640), step_time 1402.93, mean_step_time 1428.4, it/s 0.7
  7652. [0130 18:09:21 @multigpu.py:323] [p0115] step: count(1625), step_time 1437.78, mean_step_time 1425.69, it/s 0.7
  7653. [0130 18:09:21 @multigpu.py:323] [p0576] step: count(1649), step_time 1437.62, mean_step_time 1436.17, it/s 0.7
  7654. [0130 18:09:23 @multigpu.py:323] [p0574] step: count(1641), step_time 1419.89, mean_step_time 1428.03, it/s 0.7
  7655. [0130 18:09:23 @multigpu.py:323] [p0115] step: count(1626), step_time 1458.74, mean_step_time 1426.85, it/s 0.7
  7656. [0130 18:09:23 @multigpu.py:323] [p0576] step: count(1650), step_time 1415.75, mean_step_time 1436.58, it/s 0.7
  7657. [0130 18:09:24 @multigpu.py:323] [p0574] step: count(1642), step_time 1456.53, mean_step_time 1431.13, it/s 0.7
  7658. [0130 18:09:24 @multigpu.py:323] [p0576] step: count(1651), step_time 1417.01, mean_step_time 1434.87, it/s 0.7
  7659. [0130 18:09:24 @multigpu.py:323] [p0115] step: count(1627), step_time 1436.49, mean_step_time 1429.64, it/s 0.7
  7660. sending to address tcp://p0112:61216
  7661. ##### Sending to neptune: online_score : 0.656686421898 , 2.6 #####
  7662. [u'online', 2.6]
  7663. receiving
  7664. [0130 18:09:26 @multigpu.py:323] [p0115] step: count(1628), step_time 1491.6, mean_step_time 1433.88, it/s 0.7
  7665. [0130 18:09:26 @multigpu.py:323] [p0576] step: count(1652), step_time 1495.11, mean_step_time 1433.91, it/s 0.7
  7666. [0130 18:09:26 @multigpu.py:323] [p0574] step: count(1643), step_time 1638.78, mean_step_time 1433.98, it/s 0.7
  7667. [0130 18:09:27 @multigpu.py:323] [p0576] step: count(1653), step_time 1416.91, mean_step_time 1434.55, it/s 0.7
  7668. [0130 18:09:27 @multigpu.py:323] [p0115] step: count(1629), step_time 1430.94, mean_step_time 1434.31, it/s 0.7
  7669. [0130 18:09:27 @multigpu.py:323] [p0574] step: count(1644), step_time 1440.25, mean_step_time 1432.71, it/s 0.7
  7670. [0130 18:09:29 @multigpu.py:323] [p0574] step: count(1645), step_time 1410.67, mean_step_time 1431.98, it/s 0.7
  7671. [0130 18:09:29 @multigpu.py:323] [p0115] step: count(1630), step_time 1455.43, mean_step_time 1437.96, it/s 0.7
  7672. [0130 18:09:29 @multigpu.py:323] [p0576] step: count(1654), step_time 1484.02, mean_step_time 1435.19, it/s 0.7
  7673. [0130 18:09:30 @multigpu.py:323] [p0574] step: count(1646), step_time 1394.32, mean_step_time 1431.89, it/s 0.7
  7674. [0130 18:09:30 @multigpu.py:323] [p0115] step: count(1631), step_time 1378.02, mean_step_time 1434.79, it/s 0.7
  7675. [0130 18:09:30 @multigpu.py:323] [p0576] step: count(1655), step_time 1393.65, mean_step_time 1433.32, it/s 0.7
  7676. [0130 18:09:31 @multigpu.py:323] [p0574] step: count(1647), step_time 1369.83, mean_step_time 1429.78, it/s 0.7
  7677. [0130 18:09:31 @multigpu.py:323] [p0115] step: count(1632), step_time 1396.42, mean_step_time 1430.76, it/s 0.7
  7678. [0130 18:09:31 @multigpu.py:323] [p0576] step: count(1656), step_time 1439.94, mean_step_time 1434.08, it/s 0.7
  7679. [0130 18:09:33 @multigpu.py:323] [p0574] step: count(1648), step_time 1416.27, mean_step_time 1429.48, it/s 0.7
  7680. [0130 18:09:33 @multigpu.py:323] [p0115] step: count(1633), step_time 1430.82, mean_step_time 1430.77, it/s 0.7
  7681. [0130 18:09:33 @multigpu.py:323] [p0576] step: count(1657), step_time 1388.18, mean_step_time 1432.58, it/s 0.7
  7682. [0130 18:09:34 @multigpu.py:323] [p0574] step: count(1649), step_time 1413.59, mean_step_time 1427.5, it/s 0.7
  7683. [0130 18:09:34 @multigpu.py:323] [p0115] step: count(1634), step_time 1411.99, mean_step_time 1429.2, it/s 0.7
  7684. [0130 18:09:34 @multigpu.py:323] [p0576] step: count(1658), step_time 1393.79, mean_step_time 1429.03, it/s 0.7
  7685. [0130 18:09:36 @multigpu.py:323] [p0115] step: count(1635), step_time 1390.83, mean_step_time 1426.05, it/s 0.7
  7686. [0130 18:09:36 @multigpu.py:323] [p0574] step: count(1650), step_time 1473.69, mean_step_time 1431.76, it/s 0.7
  7687. [0130 18:09:36 @multigpu.py:323] [p0576] step: count(1659), step_time 1429.32, mean_step_time 1432.48, it/s 0.7
  7688. [0130 18:09:37 @multigpu.py:323] [p0115] step: count(1636), step_time 1408.14, mean_step_time 1424.21, it/s 0.7
  7689. [0130 18:09:37 @multigpu.py:323] [p0574] step: count(1651), step_time 1388.65, mean_step_time 1430.58, it/s 0.7
  7690. [0130 18:09:37 @multigpu.py:323] [p0576] step: count(1660), step_time 1427.87, mean_step_time 1433.13, it/s 0.7
  7691. [0130 18:09:38 @multigpu.py:323] [p0115] step: count(1637), step_time 1398.41, mean_step_time 1422.93, it/s 0.7
  7692. [0130 18:09:38 @multigpu.py:323] [p0574] step: count(1652), step_time 1414.64, mean_step_time 1432.77, it/s 0.7
  7693. [0130 18:09:39 @multigpu.py:323] [p0576] step: count(1661), step_time 1487.97, mean_step_time 1434.76, it/s 0.7
  7694. [0130 18:09:40 @multigpu.py:323] [p0574] step: count(1653), step_time 1415.11, mean_step_time 1430.93, it/s 0.7
  7695. [0130 18:09:40 @multigpu.py:323] [p0115] step: count(1638), step_time 1471.59, mean_step_time 1428.4, it/s 0.7
  7696. [0130 18:09:40 @multigpu.py:323] [p0576] step: count(1662), step_time 1396.64, mean_step_time 1435.44, it/s 0.7
  7697. [0130 18:09:41 @multigpu.py:323] [p0574] step: count(1654), step_time 1451.77, mean_step_time 1433.59, it/s 0.7
  7698. [0130 18:09:41 @multigpu.py:323] [p0115] step: count(1639), step_time 1495.2, mean_step_time 1431.07, it/s 0.7
  7699. [0130 18:09:41 @multigpu.py:323] [p0576] step: count(1663), step_time 1411.52, mean_step_time 1434.23, it/s 0.7
  7700. [0130 18:09:43 @multigpu.py:323] [p0574] step: count(1655), step_time 1387.5, mean_step_time 1431.45, it/s 0.7
  7701. [0130 18:09:43 @multigpu.py:323] [p0115] step: count(1640), step_time 1416.6, mean_step_time 1432.15, it/s 0.7
  7702. [0130 18:09:43 @multigpu.py:323] [p0576] step: count(1664), step_time 1421.09, mean_step_time 1432.16, it/s 0.7
  7703. [0130 18:09:44 @multigpu.py:323] [p0574] step: count(1656), step_time 1435.78, mean_step_time 1430.97, it/s 0.7
  7704. [0130 18:09:44 @multigpu.py:323] [p0576] step: count(1665), step_time 1384.19, mean_step_time 1428.13, it/s 0.7
  7705. [0130 18:09:44 @multigpu.py:323] [p0115] step: count(1641), step_time 1435.24, mean_step_time 1432.4, it/s 0.7
  7706. sending to address tcp://p0112:61216
  7707. ##### Sending to neptune: online_score : 0.662334823608 , 1.6 #####
  7708. [u'online', 1.6]
  7709. receiving
  7710. [0130 18:09:46 @multigpu.py:323] [p0574] step: count(1657), step_time 1428.13, mean_step_time 1432.4, it/s 0.7
  7711. [0130 18:09:46 @multigpu.py:323] [p0576] step: count(1666), step_time 1400.26, mean_step_time 1427.9, it/s 0.7
  7712. [0130 18:09:46 @multigpu.py:323] [p0115] step: count(1642), step_time 1459.13, mean_step_time 1433.24, it/s 0.7
  7713. [0130 18:09:47 @multigpu.py:323] [p0574] step: count(1658), step_time 1380.97, mean_step_time 1428.87, it/s 0.7
  7714. [0130 18:09:47 @multigpu.py:323] [p0576] step: count(1667), step_time 1439.95, mean_step_time 1427.69, it/s 0.7
  7715. [0130 18:09:47 @multigpu.py:323] [p0115] step: count(1643), step_time 1429.71, mean_step_time 1434.11, it/s 0.7
  7716. [0130 18:09:48 @multigpu.py:323] [p0574] step: count(1659), step_time 1468.11, mean_step_time 1430.37, it/s 0.7
  7717. [0130 18:09:48 @multigpu.py:323] [p0576] step: count(1668), step_time 1404.51, mean_step_time 1424.26, it/s 0.7
  7718. [0130 18:09:49 @multigpu.py:323] [p0115] step: count(1644), step_time 1420.53, mean_step_time 1432.68, it/s 0.7
  7719. [0130 18:09:50 @multigpu.py:323] [p0574] step: count(1660), step_time 1433.18, mean_step_time 1431.88, it/s 0.7
  7720. [0130 18:09:50 @multigpu.py:323] [p0576] step: count(1669), step_time 1487.62, mean_step_time 1426.76, it/s 0.7
  7721. [0130 18:09:50 @multigpu.py:323] [p0115] step: count(1645), step_time 1414.96, mean_step_time 1431.54, it/s 0.7
  7722. [0130 18:09:51 @multigpu.py:323] [p0574] step: count(1661), step_time 1415.25, mean_step_time 1431.65, it/s 0.7
  7723. [0130 18:09:51 @multigpu.py:323] [p0576] step: count(1670), step_time 1413.34, mean_step_time 1426.64, it/s 0.7
  7724. [0130 18:09:51 @multigpu.py:323] [p0115] step: count(1646), step_time 1505.24, mean_step_time 1433.86, it/s 0.7
  7725. sending to address tcp://p0112:61216
  7726. ##### Sending to neptune: online_score : 0.664274540544 , 1.9 #####
  7727. [u'online', 1.9]
  7728. receiving
  7729. [0130 18:09:53 @multigpu.py:323] [p0574] step: count(1662), step_time 1418.69, mean_step_time 1429.76, it/s 0.7
  7730. [0130 18:09:53 @multigpu.py:323] [p0576] step: count(1671), step_time 1434.48, mean_step_time 1427.52, it/s 0.7
  7731. [0130 18:09:53 @multigpu.py:323] [p0115] step: count(1647), step_time 1434.55, mean_step_time 1433.77, it/s 0.7
  7732. [0130 18:09:54 @multigpu.py:323] [p0115] step: count(1648), step_time 1440.15, mean_step_time 1431.19, it/s 0.7
  7733. [0130 18:09:54 @multigpu.py:323] [p0576] step: count(1672), step_time 1567.38, mean_step_time 1431.13, it/s 0.7
  7734. [0130 18:09:54 @multigpu.py:323] [p0574] step: count(1663), step_time 1667.38, mean_step_time 1431.19, it/s 0.7
  7735. [0130 18:09:56 @multigpu.py:323] [p0574] step: count(1664), step_time 1393.03, mean_step_time 1428.83, it/s 0.7
  7736. [0130 18:09:56 @multigpu.py:323] [p0115] step: count(1649), step_time 1403.24, mean_step_time 1429.81, it/s 0.7
  7737. [0130 18:09:56 @multigpu.py:323] [p0576] step: count(1673), step_time 1419.22, mean_step_time 1431.25, it/s 0.7
  7738. [0130 18:09:57 @multigpu.py:323] [p0576] step: count(1674), step_time 1414.23, mean_step_time 1427.76, it/s 0.7
  7739. [0130 18:09:57 @multigpu.py:323] [p0574] step: count(1665), step_time 1456.1, mean_step_time 1431.1, it/s 0.7
  7740. [0130 18:09:57 @multigpu.py:323] [p0115] step: count(1650), step_time 1469.75, mean_step_time 1430.53, it/s 0.7
  7741. [0130 18:09:59 @multigpu.py:323] [p0576] step: count(1675), step_time 1419.68, mean_step_time 1429.06, it/s 0.7
  7742. [0130 18:09:59 @multigpu.py:323] [p0574] step: count(1666), step_time 1481.67, mean_step_time 1435.47, it/s 0.7
  7743. [0130 18:09:59 @multigpu.py:323] [p0115] step: count(1651), step_time 1466.01, mean_step_time 1434.93, it/s 0.7
  7744. [0130 18:10:00 @multigpu.py:323] [p0576] step: count(1676), step_time 1447.57, mean_step_time 1429.44, it/s 0.7
  7745. [0130 18:10:00 @multigpu.py:323] [p0115] step: count(1652), step_time 1403.98, mean_step_time 1435.3, it/s 0.7
  7746. [0130 18:10:00 @multigpu.py:323] [p0574] step: count(1667), step_time 1482.03, mean_step_time 1441.08, it/s 0.69
  7747. sending to address tcp://p0112:61216
  7748. ##### Sending to neptune: online_score : 0.666537099679 , 1.9 #####
  7749. [u'online', 1.9]
  7750. receiving
  7751. [0130 18:10:01 @multigpu.py:323] [p0576] step: count(1677), step_time 1403.16, mean_step_time 1430.19, it/s 0.7
  7752. [0130 18:10:02 @multigpu.py:323] [p0574] step: count(1668), step_time 1409.66, mean_step_time 1440.75, it/s 0.69
  7753. [0130 18:10:02 @multigpu.py:323] [p0115] step: count(1653), step_time 1492.51, mean_step_time 1438.39, it/s 0.7
  7754. [0130 18:10:03 @multigpu.py:323] [p0576] step: count(1678), step_time 1381.72, mean_step_time 1429.59, it/s 0.7
  7755. [0130 18:10:03 @multigpu.py:323] [p0115] step: count(1654), step_time 1420.5, mean_step_time 1438.81, it/s 0.7
  7756. [0130 18:10:03 @multigpu.py:323] [p0574] step: count(1669), step_time 1436.12, mean_step_time 1441.87, it/s 0.69
  7757. [0130 18:10:04 @multigpu.py:323] [p0576] step: count(1679), step_time 1393.89, mean_step_time 1427.82, it/s 0.7
  7758. [0130 18:10:04 @multigpu.py:323] [p0574] step: count(1670), step_time 1387.79, mean_step_time 1437.58, it/s 0.7
  7759. [0130 18:10:04 @multigpu.py:323] [p0115] step: count(1655), step_time 1415.76, mean_step_time 1440.06, it/s 0.69
  7760. [0130 18:10:06 @multigpu.py:323] [p0576] step: count(1680), step_time 1497.64, mean_step_time 1431.3, it/s 0.7
  7761. [0130 18:10:06 @multigpu.py:323] [p0115] step: count(1656), step_time 1409.1, mean_step_time 1440.11, it/s 0.69
  7762. [0130 18:10:06 @multigpu.py:323] [p0574] step: count(1671), step_time 1507.97, mean_step_time 1443.54, it/s 0.69
  7763. [0130 18:10:07 @multigpu.py:323] [p0576] step: count(1681), step_time 1410.04, mean_step_time 1427.41, it/s 0.7
  7764. [0130 18:10:07 @multigpu.py:323] [p0574] step: count(1672), step_time 1387.44, mean_step_time 1442.18, it/s 0.69
  7765. [0130 18:10:07 @multigpu.py:323] [p0115] step: count(1657), step_time 1492.14, mean_step_time 1444.79, it/s 0.69
  7766. [0130 18:10:09 @multigpu.py:323] [p0576] step: count(1682), step_time 1427.56, mean_step_time 1428.95, it/s 0.7
  7767. [0130 18:10:09 @multigpu.py:323] [p0574] step: count(1673), step_time 1457.14, mean_step_time 1444.29, it/s 0.69
  7768. [0130 18:10:09 @multigpu.py:323] [p0115] step: count(1658), step_time 1438.8, mean_step_time 1443.15, it/s 0.69
  7769. [0130 18:10:10 @multigpu.py:323] [p0576] step: count(1683), step_time 1418.8, mean_step_time 1429.32, it/s 0.7
  7770. [0130 18:10:10 @multigpu.py:323] [p0574] step: count(1674), step_time 1430.46, mean_step_time 1443.22, it/s 0.69
  7771. [0130 18:10:10 @multigpu.py:323] [p0115] step: count(1659), step_time 1424.56, mean_step_time 1439.62, it/s 0.69
  7772. [0130 18:10:11 @multigpu.py:323] [p0576] step: count(1684), step_time 1437.82, mean_step_time 1430.15, it/s 0.7
  7773. [0130 18:10:12 @multigpu.py:323] [p0574] step: count(1675), step_time 1401.19, mean_step_time 1443.9, it/s 0.69
  7774. [0130 18:10:12 @multigpu.py:323] [p0115] step: count(1660), step_time 1479.0, mean_step_time 1442.74, it/s 0.69
  7775. [0130 18:10:13 @multigpu.py:323] [p0576] step: count(1685), step_time 1441.35, mean_step_time 1433.01, it/s 0.7
  7776. [0130 18:10:13 @multigpu.py:323] [p0115] step: count(1661), step_time 1384.82, mean_step_time 1440.22, it/s 0.69
  7777. [0130 18:10:13 @multigpu.py:323] [p0574] step: count(1676), step_time 1493.03, mean_step_time 1446.77, it/s 0.69
  7778. [0130 18:10:14 @multigpu.py:323] [p0576] step: count(1686), step_time 1428.42, mean_step_time 1434.42, it/s 0.7
  7779. [0130 18:10:14 @multigpu.py:323] [p0574] step: count(1677), step_time 1398.29, mean_step_time 1445.28, it/s 0.69
  7780. [0130 18:10:14 @multigpu.py:323] [p0115] step: count(1662), step_time 1425.75, mean_step_time 1438.55, it/s 0.7
  7781. sending to address tcp://p0112:61216
  7782. ##### Sending to neptune: online_score : 0.670702761345 , 2.0 #####
  7783. [u'online', 2.0]
  7784. receiving
  7785. [0130 18:10:16 @multigpu.py:323] [p0576] step: count(1687), step_time 1457.19, mean_step_time 1435.28, it/s 0.7
  7786. [0130 18:10:16 @multigpu.py:323] [p0574] step: count(1678), step_time 1404.97, mean_step_time 1446.48, it/s 0.69
  7787. [0130 18:10:16 @multigpu.py:323] [p0115] step: count(1663), step_time 1432.49, mean_step_time 1438.69, it/s 0.7
  7788. [0130 18:10:17 @multigpu.py:323] [p0576] step: count(1688), step_time 1400.57, mean_step_time 1435.08, it/s 0.7
  7789. [0130 18:10:17 @multigpu.py:323] [p0574] step: count(1679), step_time 1386.12, mean_step_time 1442.38, it/s 0.69
  7790. [0130 18:10:17 @multigpu.py:323] [p0115] step: count(1664), step_time 1449.27, mean_step_time 1440.13, it/s 0.69
  7791. [0130 18:10:19 @multigpu.py:323] [p0576] step: count(1689), step_time 1439.25, mean_step_time 1432.67, it/s 0.7
  7792. [0130 18:10:19 @multigpu.py:323] [p0574] step: count(1680), step_time 1434.19, mean_step_time 1442.43, it/s 0.69
  7793. [0130 18:10:19 @multigpu.py:323] [p0115] step: count(1665), step_time 1411.0, mean_step_time 1439.93, it/s 0.69
  7794. [0130 18:10:20 @multigpu.py:323] [p0576] step: count(1690), step_time 1420.97, mean_step_time 1433.05, it/s 0.7
  7795. [0130 18:10:20 @multigpu.py:323] [p0574] step: count(1681), step_time 1493.81, mean_step_time 1446.35, it/s 0.69
  7796. [0130 18:10:20 @multigpu.py:323] [p0115] step: count(1666), step_time 1463.45, mean_step_time 1437.84, it/s 0.7
  7797. sending to address tcp://p0112:61216
  7798. ##### Sending to neptune: online_score : 0.672266149124 , 2.1 #####
  7799. [u'online', 2.1]
  7800. ##### Sending to neptune: active_workers : 0.672266205814 , 3 #####
  7801. receiving
  7802. [0130 18:10:21 @multigpu.py:323] [p0576] step: count(1691), step_time 1387.7, mean_step_time 1430.71, it/s 0.7
  7803. [0130 18:10:22 @multigpu.py:323] [p0574] step: count(1682), step_time 1396.69, mean_step_time 1445.25, it/s 0.69
  7804. [0130 18:10:22 @multigpu.py:323] [p0115] step: count(1667), step_time 1443.52, mean_step_time 1438.29, it/s 0.7
  7805. [0130 18:10:23 @multigpu.py:323] [p0115] step: count(1668), step_time 1490.48, mean_step_time 1440.81, it/s 0.69
  7806. [0130 18:10:23 @multigpu.py:323] [p0574] step: count(1683), step_time 1577.51, mean_step_time 1440.76, it/s 0.69
  7807. [0130 18:10:23 @multigpu.py:323] [p0576] step: count(1692), step_time 1768.0, mean_step_time 1440.74, it/s 0.69
  7808. [0130 18:10:25 @multigpu.py:323] [p0115] step: count(1669), step_time 1393.01, mean_step_time 1440.29, it/s 0.69
  7809. [0130 18:10:25 @multigpu.py:323] [p0574] step: count(1684), step_time 1442.02, mean_step_time 1443.21, it/s 0.69
  7810. [0130 18:10:25 @multigpu.py:323] [p0576] step: count(1693), step_time 1461.64, mean_step_time 1442.86, it/s 0.69
  7811. [0130 18:10:26 @multigpu.py:323] [p0115] step: count(1670), step_time 1395.54, mean_step_time 1436.58, it/s 0.7
  7812. [0130 18:10:26 @multigpu.py:323] [p0576] step: count(1694), step_time 1374.54, mean_step_time 1440.88, it/s 0.69
  7813. [0130 18:10:26 @multigpu.py:323] [p0574] step: count(1685), step_time 1430.36, mean_step_time 1441.92, it/s 0.69
  7814. [0130 18:10:27 @multigpu.py:323] [p0115] step: count(1671), step_time 1398.47, mean_step_time 1433.21, it/s 0.7
  7815. [0130 18:10:27 @multigpu.py:323] [p0576] step: count(1695), step_time 1413.75, mean_step_time 1440.58, it/s 0.69
  7816. [0130 18:10:27 @multigpu.py:323] [p0574] step: count(1686), step_time 1410.91, mean_step_time 1438.38, it/s 0.7
  7817. [0130 18:10:29 @multigpu.py:323] [p0576] step: count(1696), step_time 1390.11, mean_step_time 1437.71, it/s 0.7
  7818. [0130 18:10:29 @multigpu.py:323] [p0115] step: count(1672), step_time 1461.48, mean_step_time 1436.08, it/s 0.7
  7819. [0130 18:10:29 @multigpu.py:323] [p0574] step: count(1687), step_time 1448.89, mean_step_time 1436.73, it/s 0.7
  7820. sending to address tcp://p0112:61216
  7821. ##### Sending to neptune: online_score : 0.674562685821 , 1.8 #####
  7822. [u'online', 1.8]
  7823. receiving
  7824. [0130 18:10:30 @multigpu.py:323] [p0576] step: count(1697), step_time 1453.92, mean_step_time 1440.24, it/s 0.69
  7825. [0130 18:10:30 @multigpu.py:323] [p0115] step: count(1673), step_time 1464.19, mean_step_time 1434.67, it/s 0.7
  7826. [0130 18:10:30 @multigpu.py:323] [p0574] step: count(1688), step_time 1398.54, mean_step_time 1436.17, it/s 0.7
  7827. [0130 18:10:32 @multigpu.py:323] [p0574] step: count(1689), step_time 1437.25, mean_step_time 1436.23, it/s 0.7
  7828. [0130 18:10:32 @multigpu.py:323] [p0115] step: count(1674), step_time 1462.22, mean_step_time 1436.75, it/s 0.7
  7829. [0130 18:10:32 @multigpu.py:323] [p0576] step: count(1698), step_time 1489.01, mean_step_time 1445.61, it/s 0.69
  7830. [0130 18:10:33 @multigpu.py:323] [p0115] step: count(1675), step_time 1410.27, mean_step_time 1436.48, it/s 0.7
  7831. [0130 18:10:33 @multigpu.py:323] [p0576] step: count(1699), step_time 1408.13, mean_step_time 1446.32, it/s 0.69
  7832. [0130 18:10:33 @multigpu.py:323] [p0574] step: count(1690), step_time 1451.62, mean_step_time 1439.42, it/s 0.69
  7833. [0130 18:10:35 @multigpu.py:323] [p0576] step: count(1700), step_time 1407.8, mean_step_time 1441.83, it/s 0.69
  7834. sending debugging info...
  7835. sending to address tcp://p0112:61216
  7836. ##### Sending to neptune: mean_delay : 0.676095512443 , 0.0 #####
  7837. ##### Sending to neptune: max_delay : 0.676095512443 , -0.0 #####
  7838. sending to address tcp://p0112:61216
  7839. ##### Sending to neptune: min_delay : 0.676095512443 , -0.0 #####
  7840. [u'delays', [0.0, -0.0, -0.0]]
  7841. receiving
  7842. ##### Sending to neptune: cost : 0.676096027493 , -0.0114078214392 #####
  7843. ##### Sending to neptune: policy_loss : 0.676096027493 , -0.820938706398 #####
  7844. sending to address tcp://p0112:61216
  7845. ##### Sending to neptune: xentropy_loss : 0.676096027493 , -2.28196120262 #####
  7846. ##### Sending to neptune: value_loss : 0.676096027493 , 1.64269912243 #####
  7847. ##### Sending to neptune: advantage : 0.676096027493 , 0.00343541777693 #####
  7848. ##### Sending to neptune: pred_reward : 0.676096027493 , 0.465741872787 #####
  7849. ##### Sending to neptune: max_logit : 0.676096027493 , 0.206445157528 #####
  7850. [u'loss', -0.0114078214392066, -0.8209387063980103, -2.28196120262146, 1.642699122428894, 0.003435417776927352, 0.4657418727874756, 0.20644515752792358]
  7851. receiving
  7852. ##### Sending to neptune: active_relus : 0.676096511616 , 9639749.34 #####
  7853. ##### Sending to neptune: dp_per_s : 0.676096511616 , 89.2744267437 #####
  7854. [u'other', 9639749.34, 89.27442674373448]
  7855. receiving
  7856. [0130 18:10:35 @multigpu.py:323] [p0574] step: count(1691), step_time 1400.0, mean_step_time 1434.02, it/s 0.7
  7857. [0130 18:10:35 @multigpu.py:323] [p0115] step: count(1676), step_time 1481.7, mean_step_time 1440.11, it/s 0.69
  7858. [0130 18:10:36 @multigpu.py:323] [p0576] step: count(1701), step_time 1441.7, mean_step_time 1443.41, it/s 0.69
  7859. [0130 18:10:36 @multigpu.py:323] [p0574] step: count(1692), step_time 1439.86, mean_step_time 1436.64, it/s 0.7
  7860. [0130 18:10:36 @multigpu.py:323] [p0115] step: count(1677), step_time 1406.73, mean_step_time 1435.84, it/s 0.7
  7861. [0130 18:10:37 @multigpu.py:323] [p0576] step: count(1702), step_time 1396.92, mean_step_time 1441.88, it/s 0.69
  7862. [0130 18:10:37 @multigpu.py:323] [p0574] step: count(1693), step_time 1393.53, mean_step_time 1433.46, it/s 0.7
  7863. [0130 18:10:37 @multigpu.py:323] [p0115] step: count(1678), step_time 1422.24, mean_step_time 1435.01, it/s 0.7
  7864. [0130 18:10:39 @multigpu.py:323] [p0576] step: count(1703), step_time 1442.18, mean_step_time 1443.05, it/s 0.69
  7865. [0130 18:10:39 @multigpu.py:323] [p0115] step: count(1679), step_time 1388.7, mean_step_time 1433.22, it/s 0.7
  7866. [0130 18:10:39 @multigpu.py:323] [p0574] step: count(1694), step_time 1478.99, mean_step_time 1435.89, it/s 0.7
  7867. [0130 18:10:40 @multigpu.py:323] [p0115] step: count(1680), step_time 1404.89, mean_step_time 1429.51, it/s 0.7
  7868. [0130 18:10:40 @multigpu.py:323] [p0574] step: count(1695), step_time 1387.51, mean_step_time 1435.2, it/s 0.7
  7869. [0130 18:10:40 @multigpu.py:323] [p0576] step: count(1704), step_time 1446.42, mean_step_time 1443.48, it/s 0.69
  7870. [0130 18:10:42 @multigpu.py:323] [p0115] step: count(1681), step_time 1401.27, mean_step_time 1430.33, it/s 0.7
  7871. [0130 18:10:42 @multigpu.py:323] [p0574] step: count(1696), step_time 1426.99, mean_step_time 1431.9, it/s 0.7
  7872. [0130 18:10:42 @multigpu.py:323] [p0576] step: count(1705), step_time 1466.69, mean_step_time 1444.75, it/s 0.69
  7873. sending to address tcp://p0112:61216
  7874. ##### Sending to neptune: online_score : 0.678271423843 , 1.9 #####
  7875. [u'online', 1.9]
  7876. receiving
  7877. [0130 18:10:43 @multigpu.py:323] [p0115] step: count(1682), step_time 1426.85, mean_step_time 1430.39, it/s 0.7
  7878. [0130 18:10:43 @multigpu.py:323] [p0574] step: count(1697), step_time 1398.01, mean_step_time 1431.89, it/s 0.7
  7879. [0130 18:10:43 @multigpu.py:323] [p0576] step: count(1706), step_time 1392.96, mean_step_time 1442.97, it/s 0.69
  7880. [0130 18:10:44 @multigpu.py:323] [p0115] step: count(1683), step_time 1428.49, mean_step_time 1430.19, it/s 0.7
  7881. [0130 18:10:45 @multigpu.py:323] [p0576] step: count(1707), step_time 1430.49, mean_step_time 1441.64, it/s 0.69
  7882. [0130 18:10:45 @multigpu.py:323] [p0574] step: count(1698), step_time 1502.63, mean_step_time 1436.77, it/s 0.7
  7883. [0130 18:10:46 @multigpu.py:323] [p0115] step: count(1684), step_time 1463.85, mean_step_time 1430.92, it/s 0.7
  7884. [0130 18:10:46 @multigpu.py:323] [p0574] step: count(1699), step_time 1391.85, mean_step_time 1437.06, it/s 0.7
  7885. [0130 18:10:46 @multigpu.py:323] [p0576] step: count(1708), step_time 1436.71, mean_step_time 1443.44, it/s 0.69
  7886. [0130 18:10:47 @multigpu.py:323] [p0115] step: count(1685), step_time 1387.82, mean_step_time 1429.76, it/s 0.7
  7887. [0130 18:10:47 @multigpu.py:323] [p0574] step: count(1700), step_time 1430.96, mean_step_time 1436.9, it/s 0.7
  7888. sending debugging info...
  7889. sending to address tcp://p0112:61216
  7890. ##### Sending to neptune: mean_delay : 0.679671151373 , 0.0 #####
  7891. ##### Sending to neptune: max_delay : 0.679671151373 , -0.0 #####
  7892. sending to address tcp://p0112:61216
  7893. ##### Sending to neptune: min_delay : 0.679671151373 , -0.0 #####
  7894. [u'delays', [0.0, -0.0, -0.0]]
  7895. receiving
  7896. ##### Sending to neptune: cost : 0.679671659668 , 0.00126111682039 #####
  7897. sending to address tcp://p0112:61216
  7898. ##### Sending to neptune: policy_loss : 0.679671659668 , 0.562968075275 #####
  7899. ##### Sending to neptune: xentropy_loss : 0.679671659668 , -2.2817337513 #####
  7900. ##### Sending to neptune: value_loss : 0.679671659668 , 1.88018858433 #####
  7901. ##### Sending to neptune: advantage : 0.679671659668 , -0.0025423753541 #####
  7902. ##### Sending to neptune: pred_reward : 0.679671659668 , 0.480535209179 #####
  7903. ##### Sending to neptune: max_logit : 0.679671659668 , 0.207985624671 #####
  7904. [u'loss', 0.0012611168203875422, 0.5629680752754211, -2.281733751296997, 1.8801885843276978, -0.0025423753540962934, 0.48053520917892456, 0.20798562467098236]
  7905. receiving
  7906. ##### Sending to neptune: active_relus : 0.6796725444 , 9649228.0 #####
  7907. ##### Sending to neptune: dp_per_s : 0.6796725444 , 89.2170627159 #####
  7908. [u'other', 9649228.0, 89.21706271592348]
  7909. receiving
  7910. [0130 18:10:47 @multigpu.py:323] [p0576] step: count(1709), step_time 1462.95, mean_step_time 1444.63, it/s 0.69
  7911. [0130 18:10:49 @multigpu.py:323] [p0115] step: count(1686), step_time 1456.53, mean_step_time 1429.41, it/s 0.7
  7912. [0130 18:10:49 @multigpu.py:323] [p0574] step: count(1701), step_time 1414.1, mean_step_time 1432.91, it/s 0.7
  7913. [0130 18:10:49 @multigpu.py:323] [p0576] step: count(1710), step_time 1487.65, mean_step_time 1447.96, it/s 0.69
  7914. [0130 18:10:50 @multigpu.py:323] [p0115] step: count(1687), step_time 1401.89, mean_step_time 1427.33, it/s 0.7
  7915. [0130 18:10:50 @multigpu.py:323] [p0574] step: count(1702), step_time 1409.06, mean_step_time 1433.53, it/s 0.7
  7916. [0130 18:10:50 @multigpu.py:323] [p0576] step: count(1711), step_time 1391.33, mean_step_time 1448.14, it/s 0.69
  7917. [0130 18:10:52 @multigpu.py:323] [p0115] step: count(1688), step_time 1571.78, mean_step_time 1431.4, it/s 0.7
  7918. [0130 18:10:52 @multigpu.py:323] [p0574] step: count(1703), step_time 1536.91, mean_step_time 1431.5, it/s 0.7
  7919. [0130 18:10:52 @multigpu.py:323] [p0576] step: count(1712), step_time 1432.06, mean_step_time 1431.35, it/s 0.7
  7920. sending to address tcp://p0112:61216
  7921. ##### Sending to neptune: online_score : 0.68088431855 , 1.7 #####
  7922. [u'online', 1.7]
  7923. receiving
  7924. [0130 18:10:53 @multigpu.py:323] [p0574] step: count(1704), step_time 1428.53, mean_step_time 1430.82, it/s 0.7
  7925. [0130 18:10:53 @multigpu.py:323] [p0576] step: count(1713), step_time 1434.77, mean_step_time 1430.0, it/s 0.7
  7926. [0130 18:10:53 @multigpu.py:323] [p0115] step: count(1689), step_time 1478.01, mean_step_time 1435.65, it/s 0.7
  7927. [0130 18:10:55 @multigpu.py:323] [p0576] step: count(1714), step_time 1380.62, mean_step_time 1430.31, it/s 0.7
  7928. [0130 18:10:55 @multigpu.py:323] [p0574] step: count(1705), step_time 1447.34, mean_step_time 1431.67, it/s 0.7
  7929. [0130 18:10:55 @multigpu.py:323] [p0115] step: count(1690), step_time 1433.78, mean_step_time 1437.56, it/s 0.7
  7930. [0130 18:10:56 @multigpu.py:323] [p0576] step: count(1715), step_time 1425.05, mean_step_time 1430.87, it/s 0.7
  7931. [0130 18:10:56 @multigpu.py:323] [p0574] step: count(1706), step_time 1450.91, mean_step_time 1433.67, it/s 0.7
  7932. [0130 18:10:56 @multigpu.py:323] [p0115] step: count(1691), step_time 1437.53, mean_step_time 1439.51, it/s 0.69
  7933. [0130 18:10:57 @multigpu.py:323] [p0576] step: count(1716), step_time 1372.36, mean_step_time 1429.99, it/s 0.7
  7934. [0130 18:10:58 @multigpu.py:323] [p0115] step: count(1692), step_time 1399.7, mean_step_time 1436.42, it/s 0.7
  7935. [0130 18:10:58 @multigpu.py:323] [p0574] step: count(1707), step_time 1426.04, mean_step_time 1432.53, it/s 0.7
  7936. [0130 18:10:59 @multigpu.py:323] [p0576] step: count(1717), step_time 1474.3, mean_step_time 1431.0, it/s 0.7
  7937. [0130 18:10:59 @multigpu.py:323] [p0115] step: count(1693), step_time 1419.96, mean_step_time 1434.21, it/s 0.7
  7938. [0130 18:10:59 @multigpu.py:323] [p0574] step: count(1708), step_time 1448.96, mean_step_time 1435.05, it/s 0.7
  7939. sending to address tcp://p0112:61216
  7940. ##### Sending to neptune: online_score : 0.682994077471 , 2.6 #####
  7941. [u'online', 2.6]
  7942. receiving
  7943. [0130 18:11:00 @multigpu.py:323] [p0576] step: count(1718), step_time 1428.45, mean_step_time 1427.98, it/s 0.7
  7944. [0130 18:11:00 @multigpu.py:323] [p0574] step: count(1709), step_time 1409.53, mean_step_time 1433.67, it/s 0.7
  7945. [0130 18:11:00 @multigpu.py:323] [p0115] step: count(1694), step_time 1501.43, mean_step_time 1436.17, it/s 0.7
  7946. [0130 18:11:02 @multigpu.py:323] [p0576] step: count(1719), step_time 1411.97, mean_step_time 1428.17, it/s 0.7
  7947. [0130 18:11:02 @multigpu.py:323] [p0574] step: count(1710), step_time 1445.18, mean_step_time 1433.34, it/s 0.7
  7948. [0130 18:11:02 @multigpu.py:323] [p0115] step: count(1695), step_time 1403.87, mean_step_time 1435.85, it/s 0.7
  7949. [0130 18:11:03 @multigpu.py:323] [p0576] step: count(1720), step_time 1433.42, mean_step_time 1429.45, it/s 0.7
  7950. [0130 18:11:03 @multigpu.py:323] [p0574] step: count(1711), step_time 1412.82, mean_step_time 1433.99, it/s 0.7
  7951. [0130 18:11:03 @multigpu.py:323] [p0115] step: count(1696), step_time 1435.7, mean_step_time 1433.55, it/s 0.7
  7952. [0130 18:11:05 @multigpu.py:323] [p0576] step: count(1721), step_time 1423.2, mean_step_time 1428.52, it/s 0.7
  7953. [0130 18:11:05 @multigpu.py:323] [p0574] step: count(1712), step_time 1433.86, mean_step_time 1433.69, it/s 0.7
  7954. [0130 18:11:05 @multigpu.py:323] [p0115] step: count(1697), step_time 1434.6, mean_step_time 1434.94, it/s 0.7
  7955. [0130 18:11:06 @multigpu.py:323] [p0576] step: count(1722), step_time 1474.72, mean_step_time 1432.41, it/s 0.7
  7956. [0130 18:11:06 @multigpu.py:323] [p0115] step: count(1698), step_time 1399.99, mean_step_time 1433.83, it/s 0.7
  7957. [0130 18:11:06 @multigpu.py:323] [p0574] step: count(1713), step_time 1499.86, mean_step_time 1439.0, it/s 0.69
  7958. [0130 18:11:07 @multigpu.py:323] [p0576] step: count(1723), step_time 1408.59, mean_step_time 1430.73, it/s 0.7
  7959. [0130 18:11:08 @multigpu.py:323] [p0115] step: count(1699), step_time 1455.65, mean_step_time 1437.18, it/s 0.7
  7960. [0130 18:11:08 @multigpu.py:323] [p0574] step: count(1714), step_time 1492.68, mean_step_time 1439.69, it/s 0.69
  7961. [0130 18:11:09 @multigpu.py:323] [p0576] step: count(1724), step_time 1517.96, mean_step_time 1434.31, it/s 0.7
  7962. [0130 18:11:09 @multigpu.py:323] [p0115] step: count(1700), step_time 1391.7, mean_step_time 1436.52, it/s 0.7
  7963. sending debugging info...
  7964. sending to address tcp://p0112:61216
  7965. ##### Sending to neptune: mean_delay : 0.685659803549 , 0.0 #####
  7966. sending to address tcp://p0112:61216
  7967. ##### Sending to neptune: max_delay : 0.685659803549 , -0.0 #####
  7968. ##### Sending to neptune: min_delay : 0.685659803549 , -0.0 #####
  7969. [u'delays', [0.0, -0.0, -0.0]]
  7970. receiving
  7971. ##### Sending to neptune: cost : 0.685660319659 , -0.00245934864506 #####
  7972. sending to address tcp://p0112:61216
  7973. ##### Sending to neptune: policy_loss : 0.685660319659 , 0.0409755147994 #####
  7974. ##### Sending to neptune: xentropy_loss : 0.685660319659 , -2.28138971329 #####
  7975. ##### Sending to neptune: value_loss : 0.685660319659 , 1.92561733723 #####
  7976. ##### Sending to neptune: advantage : 0.685660319659 , -0.000262442481471 #####
  7977. ##### Sending to neptune: pred_reward : 0.685660319659 , 0.488827288151 #####
  7978. ##### Sending to neptune: max_logit : 0.685660319659 , 0.209326460958 #####
  7979. [u'loss', -0.0024593486450612545, 0.04097551479935646, -2.2813897132873535, 1.9256173372268677, -0.0002624424814712256, 0.48882728815078735, 0.20932646095752716]
  7980. receiving
  7981. ##### Sending to neptune: active_relus : 0.685660764972 , 9660044.15 #####
  7982. ##### Sending to neptune: dp_per_s : 0.685660764972 , 89.2862143186 #####
  7983. [u'other', 9660044.15, 89.28621431862673]
  7984. receiving
  7985. [0130 18:11:09 @multigpu.py:323] [p0574] step: count(1715), step_time 1457.41, mean_step_time 1443.18, it/s 0.69
  7986. [0130 18:11:10 @multigpu.py:323] [p0576] step: count(1725), step_time 1414.69, mean_step_time 1431.71, it/s 0.7
  7987. [0130 18:11:10 @multigpu.py:323] [p0115] step: count(1701), step_time 1473.77, mean_step_time 1440.14, it/s 0.69
  7988. [0130 18:11:11 @multigpu.py:323] [p0574] step: count(1716), step_time 1443.67, mean_step_time 1444.02, it/s 0.69
  7989. sending to address tcp://p0112:61216
  7990. ##### Sending to neptune: online_score : 0.686278829442 , 2.0 #####
  7991. [u'online', 2.0]
  7992. receiving
  7993. [0130 18:11:12 @multigpu.py:323] [p0576] step: count(1726), step_time 1385.38, mean_step_time 1431.33, it/s 0.7
  7994. [0130 18:11:12 @multigpu.py:323] [p0115] step: count(1702), step_time 1477.13, mean_step_time 1442.66, it/s 0.69
  7995. [0130 18:11:12 @multigpu.py:323] [p0574] step: count(1717), step_time 1432.63, mean_step_time 1445.75, it/s 0.69
  7996. [0130 18:11:13 @multigpu.py:323] [p0576] step: count(1727), step_time 1440.82, mean_step_time 1431.85, it/s 0.7
  7997. [0130 18:11:13 @multigpu.py:323] [p0115] step: count(1703), step_time 1427.43, mean_step_time 1442.6, it/s 0.69
  7998. [0130 18:11:13 @multigpu.py:323] [p0574] step: count(1718), step_time 1400.08, mean_step_time 1440.62, it/s 0.69
  7999. [0130 18:11:15 @multigpu.py:323] [p0576] step: count(1728), step_time 1435.43, mean_step_time 1431.79, it/s 0.7
  8000. [0130 18:11:15 @multigpu.py:323] [p0115] step: count(1704), step_time 1427.15, mean_step_time 1440.77, it/s 0.69
  8001. [0130 18:11:15 @multigpu.py:323] [p0574] step: count(1719), step_time 1447.53, mean_step_time 1443.4, it/s 0.69
  8002. [0130 18:11:16 @multigpu.py:323] [p0576] step: count(1729), step_time 1376.1, mean_step_time 1427.44, it/s 0.7
  8003. [0130 18:11:16 @multigpu.py:323] [p0115] step: count(1705), step_time 1408.15, mean_step_time 1441.79, it/s 0.69
  8004. [0130 18:11:16 @multigpu.py:323] [p0574] step: count(1720), step_time 1473.9, mean_step_time 1445.55, it/s 0.69
  8005. sending to address tcp://p0112:61216
  8006. ##### Sending to neptune: online_score : 0.68780823138 , 1.2 #####
  8007. [u'online', 1.2]
  8008. receiving
  8009. [0130 18:11:17 @multigpu.py:323] [p0576] step: count(1730), step_time 1415.12, mean_step_time 1423.82, it/s 0.7
  8010. [0130 18:11:18 @multigpu.py:323] [p0115] step: count(1706), step_time 1437.1, mean_step_time 1440.81, it/s 0.69
  8011. [0130 18:11:18 @multigpu.py:323] [p0574] step: count(1721), step_time 1391.96, mean_step_time 1444.44, it/s 0.69
  8012. [0130 18:11:19 @multigpu.py:323] [p0576] step: count(1731), step_time 1425.59, mean_step_time 1425.53, it/s 0.7
  8013. [0130 18:11:19 @multigpu.py:323] [p0115] step: count(1707), step_time 1398.22, mean_step_time 1440.63, it/s 0.69
  8014. [0130 18:11:19 @multigpu.py:323] [p0574] step: count(1722), step_time 1453.42, mean_step_time 1446.66, it/s 0.69
  8015. [0130 18:11:21 @multigpu.py:323] [p0574] step: count(1723), step_time 1386.74, mean_step_time 1439.15, it/s 0.69
  8016. [0130 18:11:21 @multigpu.py:323] [p0115] step: count(1708), step_time 1543.25, mean_step_time 1439.2, it/s 0.69
  8017. [0130 18:11:21 @multigpu.py:323] [p0576] step: count(1732), step_time 1705.21, mean_step_time 1439.19, it/s 0.69
  8018. [0130 18:11:22 @multigpu.py:323] [p0574] step: count(1724), step_time 1415.99, mean_step_time 1438.53, it/s 0.7
  8019. [0130 18:11:22 @multigpu.py:323] [p0576] step: count(1733), step_time 1415.76, mean_step_time 1438.24, it/s 0.7
  8020. [0130 18:11:22 @multigpu.py:323] [p0115] step: count(1709), step_time 1435.06, mean_step_time 1437.06, it/s 0.7
  8021. [0130 18:11:23 @multigpu.py:323] [p0574] step: count(1725), step_time 1413.88, mean_step_time 1436.85, it/s 0.7
  8022. [0130 18:11:23 @multigpu.py:323] [p0115] step: count(1710), step_time 1420.78, mean_step_time 1436.41, it/s 0.7
  8023. [0130 18:11:23 @multigpu.py:323] [p0576] step: count(1734), step_time 1449.01, mean_step_time 1441.66, it/s 0.69
  8024. [0130 18:11:25 @multigpu.py:323] [p0574] step: count(1726), step_time 1437.26, mean_step_time 1436.17, it/s 0.7
  8025. [0130 18:11:25 @multigpu.py:323] [p0576] step: count(1735), step_time 1410.08, mean_step_time 1440.91, it/s 0.69
  8026. [0130 18:11:25 @multigpu.py:323] [p0115] step: count(1711), step_time 1478.62, mean_step_time 1438.46, it/s 0.7
  8027. [0130 18:11:26 @multigpu.py:323] [p0576] step: count(1736), step_time 1427.03, mean_step_time 1443.64, it/s 0.69
  8028. [0130 18:11:26 @multigpu.py:323] [p0574] step: count(1727), step_time 1438.08, mean_step_time 1436.77, it/s 0.7
  8029. [0130 18:11:26 @multigpu.py:323] [p0115] step: count(1712), step_time 1406.9, mean_step_time 1438.82, it/s 0.7
  8030. [0130 18:11:28 @multigpu.py:323] [p0576] step: count(1737), step_time 1439.18, mean_step_time 1441.88, it/s 0.69
  8031. [0130 18:11:28 @multigpu.py:323] [p0574] step: count(1728), step_time 1446.6, mean_step_time 1436.65, it/s 0.7
  8032. [0130 18:11:28 @multigpu.py:323] [p0115] step: count(1713), step_time 1424.78, mean_step_time 1439.06, it/s 0.69
  8033. sending to address tcp://p0112:61216
  8034. ##### Sending to neptune: online_score : 0.691006313827 , 1.3 #####
  8035. [u'online', 1.3]
  8036. ##### Sending to neptune: active_workers : 0.691006366611 , 3 #####
  8037. receiving
  8038. [0130 18:11:29 @multigpu.py:323] [p0574] step: count(1729), step_time 1424.96, mean_step_time 1437.43, it/s 0.7
  8039. [0130 18:11:29 @multigpu.py:323] [p0115] step: count(1714), step_time 1415.67, mean_step_time 1434.77, it/s 0.7
  8040. [0130 18:11:29 @multigpu.py:323] [p0576] step: count(1738), step_time 1476.65, mean_step_time 1444.29, it/s 0.69
  8041. [0130 18:11:31 @multigpu.py:323] [p0115] step: count(1715), step_time 1406.08, mean_step_time 1434.89, it/s 0.7
  8042. [0130 18:11:31 @multigpu.py:323] [p0576] step: count(1739), step_time 1372.57, mean_step_time 1442.33, it/s 0.69
  8043. [0130 18:11:31 @multigpu.py:323] [p0574] step: count(1730), step_time 1460.12, mean_step_time 1438.17, it/s 0.7
  8044. [0130 18:11:32 @multigpu.py:323] [p0115] step: count(1716), step_time 1424.92, mean_step_time 1434.35, it/s 0.7
  8045. [0130 18:11:32 @multigpu.py:323] [p0574] step: count(1731), step_time 1438.66, mean_step_time 1439.46, it/s 0.69
  8046. [0130 18:11:32 @multigpu.py:323] [p0576] step: count(1740), step_time 1510.8, mean_step_time 1446.19, it/s 0.69
  8047. [0130 18:11:33 @multigpu.py:323] [p0574] step: count(1732), step_time 1382.75, mean_step_time 1436.91, it/s 0.7
  8048. [0130 18:11:33 @multigpu.py:323] [p0115] step: count(1717), step_time 1447.9, mean_step_time 1435.01, it/s 0.7
  8049. [0130 18:11:33 @multigpu.py:323] [p0576] step: count(1741), step_time 1415.93, mean_step_time 1445.83, it/s 0.69
  8050. [0130 18:11:35 @multigpu.py:323] [p0574] step: count(1733), step_time 1436.02, mean_step_time 1433.72, it/s 0.7
  8051. [0130 18:11:35 @multigpu.py:323] [p0115] step: count(1718), step_time 1439.82, mean_step_time 1437.0, it/s 0.7
  8052. [0130 18:11:35 @multigpu.py:323] [p0576] step: count(1742), step_time 1452.77, mean_step_time 1444.73, it/s 0.69
  8053. [0130 18:11:36 @multigpu.py:323] [p0115] step: count(1719), step_time 1403.97, mean_step_time 1434.42, it/s 0.7
  8054. [0130 18:11:36 @multigpu.py:323] [p0574] step: count(1734), step_time 1438.25, mean_step_time 1431.0, it/s 0.7
  8055. [0130 18:11:36 @multigpu.py:323] [p0576] step: count(1743), step_time 1377.33, mean_step_time 1443.17, it/s 0.69
  8056. [0130 18:11:38 @multigpu.py:323] [p0115] step: count(1720), step_time 1465.36, mean_step_time 1438.1, it/s 0.7
  8057. [0130 18:11:38 @multigpu.py:323] [p0574] step: count(1735), step_time 1441.22, mean_step_time 1430.19, it/s 0.7
  8058. [0130 18:11:38 @multigpu.py:323] [p0576] step: count(1744), step_time 1451.9, mean_step_time 1439.87, it/s 0.69
  8059. [0130 18:11:39 @multigpu.py:323] [p0574] step: count(1736), step_time 1405.73, mean_step_time 1428.29, it/s 0.7
  8060. [0130 18:11:39 @multigpu.py:323] [p0115] step: count(1721), step_time 1438.44, mean_step_time 1436.34, it/s 0.7
  8061. [0130 18:11:39 @multigpu.py:323] [p0576] step: count(1745), step_time 1456.66, mean_step_time 1441.97, it/s 0.69
  8062. [0130 18:11:41 @multigpu.py:323] [p0115] step: count(1722), step_time 1376.54, mean_step_time 1431.31, it/s 0.7
  8063. [0130 18:11:41 @multigpu.py:323] [p0574] step: count(1737), step_time 1420.84, mean_step_time 1427.7, it/s 0.7
  8064. [0130 18:11:41 @multigpu.py:323] [p0576] step: count(1746), step_time 1445.38, mean_step_time 1444.97, it/s 0.69
  8065. [0130 18:11:42 @multigpu.py:323] [p0115] step: count(1723), step_time 1423.86, mean_step_time 1431.13, it/s 0.7
  8066. [0130 18:11:42 @multigpu.py:323] [p0574] step: count(1738), step_time 1433.22, mean_step_time 1429.36, it/s 0.7
  8067. [0130 18:11:42 @multigpu.py:323] [p0576] step: count(1747), step_time 1447.7, mean_step_time 1445.31, it/s 0.69
  8068. sending to address tcp://p0112:61216
  8069. ##### Sending to neptune: online_score : 0.694909020795 , 1.3 #####
  8070. [u'online', 1.3]
  8071. receiving
  8072. [0130 18:11:43 @multigpu.py:323] [p0115] step: count(1724), step_time 1421.89, mean_step_time 1430.86, it/s 0.7
  8073. [0130 18:11:43 @multigpu.py:323] [p0574] step: count(1739), step_time 1427.83, mean_step_time 1428.37, it/s 0.7
  8074. [0130 18:11:44 @multigpu.py:323] [p0576] step: count(1748), step_time 1419.06, mean_step_time 1444.49, it/s 0.69
  8075. [0130 18:11:45 @multigpu.py:323] [p0115] step: count(1725), step_time 1423.5, mean_step_time 1431.63, it/s 0.7
  8076. [0130 18:11:45 @multigpu.py:323] [p0574] step: count(1740), step_time 1404.35, mean_step_time 1424.89, it/s 0.7
  8077. [0130 18:11:45 @multigpu.py:323] [p0576] step: count(1749), step_time 1449.14, mean_step_time 1448.14, it/s 0.69
  8078. sending to address tcp://p0112:61216
  8079. ##### Sending to neptune: online_score : 0.695835398568 , 1.7 #####
  8080. [u'online', 1.7]
  8081. receiving
  8082. [0130 18:11:46 @multigpu.py:323] [p0574] step: count(1741), step_time 1394.8, mean_step_time 1425.04, it/s 0.7
  8083. [0130 18:11:46 @multigpu.py:323] [p0115] step: count(1726), step_time 1435.46, mean_step_time 1431.55, it/s 0.7
  8084. [0130 18:11:46 @multigpu.py:323] [p0576] step: count(1750), step_time 1435.3, mean_step_time 1449.15, it/s 0.69
  8085. [0130 18:11:48 @multigpu.py:323] [p0115] step: count(1727), step_time 1379.68, mean_step_time 1430.62, it/s 0.7
  8086. [0130 18:11:48 @multigpu.py:323] [p0574] step: count(1742), step_time 1454.37, mean_step_time 1425.08, it/s 0.7
  8087. [0130 18:11:48 @multigpu.py:323] [p0576] step: count(1751), step_time 1403.57, mean_step_time 1448.05, it/s 0.69
  8088. [0130 18:11:49 @multigpu.py:323] [p0115] step: count(1728), step_time 1607.47, mean_step_time 1433.83, it/s 0.7
  8089. [0130 18:11:49 @multigpu.py:323] [p0574] step: count(1743), step_time 1562.31, mean_step_time 1433.86, it/s 0.7
  8090. [0130 18:11:49 @multigpu.py:323] [p0576] step: count(1752), step_time 1419.29, mean_step_time 1433.76, it/s 0.7
  8091. [0130 18:11:51 @multigpu.py:323] [p0574] step: count(1744), step_time 1386.85, mean_step_time 1432.4, it/s 0.7
  8092. [0130 18:11:51 @multigpu.py:323] [p0576] step: count(1753), step_time 1400.35, mean_step_time 1432.99, it/s 0.7
  8093. [0130 18:11:51 @multigpu.py:323] [p0115] step: count(1729), step_time 1486.65, mean_step_time 1436.41, it/s 0.7
  8094. [0130 18:11:52 @multigpu.py:323] [p0574] step: count(1745), step_time 1458.63, mean_step_time 1434.64, it/s 0.7
  8095. [0130 18:11:52 @multigpu.py:323] [p0576] step: count(1754), step_time 1454.21, mean_step_time 1433.25, it/s 0.7
  8096. [0130 18:11:52 @multigpu.py:323] [p0115] step: count(1730), step_time 1413.23, mean_step_time 1436.04, it/s 0.7
  8097. [0130 18:11:53 @multigpu.py:323] [p0574] step: count(1746), step_time 1386.86, mean_step_time 1432.12, it/s 0.7
  8098. [0130 18:11:54 @multigpu.py:323] [p0576] step: count(1755), step_time 1419.0, mean_step_time 1433.69, it/s 0.7
  8099. [0130 18:11:54 @multigpu.py:323] [p0115] step: count(1731), step_time 1405.57, mean_step_time 1432.38, it/s 0.7
  8100. sending to address tcp://p0112:61216
  8101. ##### Sending to neptune: online_score : 0.698198745781 , 1.9 #####
  8102. [u'online', 1.9]
  8103. receiving
  8104. [0130 18:11:55 @multigpu.py:323] [p0574] step: count(1747), step_time 1453.75, mean_step_time 1432.91, it/s 0.7
  8105. [0130 18:11:55 @multigpu.py:323] [p0115] step: count(1732), step_time 1397.21, mean_step_time 1431.9, it/s 0.7
  8106. [0130 18:11:55 @multigpu.py:323] [p0576] step: count(1756), step_time 1431.6, mean_step_time 1433.92, it/s 0.7
  8107. [0130 18:11:56 @multigpu.py:323] [p0574] step: count(1748), step_time 1367.6, mean_step_time 1428.96, it/s 0.7
  8108. [0130 18:11:56 @multigpu.py:323] [p0576] step: count(1757), step_time 1407.07, mean_step_time 1432.32, it/s 0.7
  8109. [0130 18:11:56 @multigpu.py:323] [p0115] step: count(1733), step_time 1444.28, mean_step_time 1432.87, it/s 0.7
  8110. [0130 18:11:58 @multigpu.py:323] [p0574] step: count(1749), step_time 1424.37, mean_step_time 1428.93, it/s 0.7
  8111. [0130 18:11:58 @multigpu.py:323] [p0576] step: count(1758), step_time 1467.88, mean_step_time 1431.88, it/s 0.7
  8112. [0130 18:11:58 @multigpu.py:323] [p0115] step: count(1734), step_time 1444.77, mean_step_time 1434.33, it/s 0.7
  8113. [0130 18:11:59 @multigpu.py:323] [p0574] step: count(1750), step_time 1439.9, mean_step_time 1427.92, it/s 0.7
  8114. [0130 18:11:59 @multigpu.py:323] [p0576] step: count(1759), step_time 1413.75, mean_step_time 1433.94, it/s 0.7
  8115. [0130 18:11:59 @multigpu.py:323] [p0115] step: count(1735), step_time 1427.89, mean_step_time 1435.42, it/s 0.7
  8116. [0130 18:12:01 @multigpu.py:323] [p0574] step: count(1751), step_time 1468.3, mean_step_time 1429.4, it/s 0.7
  8117. [0130 18:12:01 @multigpu.py:323] [p0576] step: count(1760), step_time 1429.69, mean_step_time 1429.88, it/s 0.7
  8118. [0130 18:12:01 @multigpu.py:323] [p0115] step: count(1736), step_time 1459.78, mean_step_time 1437.16, it/s 0.7
  8119. [0130 18:12:02 @multigpu.py:323] [p0574] step: count(1752), step_time 1401.17, mean_step_time 1430.32, it/s 0.7
  8120. [0130 18:12:02 @multigpu.py:323] [p0576] step: count(1761), step_time 1453.08, mean_step_time 1431.74, it/s 0.7
  8121. [0130 18:12:02 @multigpu.py:323] [p0115] step: count(1737), step_time 1427.12, mean_step_time 1436.12, it/s 0.7
  8122. [0130 18:12:03 @multigpu.py:323] [p0574] step: count(1753), step_time 1443.76, mean_step_time 1430.71, it/s 0.7
  8123. [0130 18:12:04 @multigpu.py:323] [p0576] step: count(1762), step_time 1381.0, mean_step_time 1428.15, it/s 0.7
  8124. [0130 18:12:04 @multigpu.py:323] [p0115] step: count(1738), step_time 1401.29, mean_step_time 1434.2, it/s 0.7
  8125. [0130 18:12:05 @multigpu.py:323] [p0574] step: count(1754), step_time 1450.24, mean_step_time 1431.31, it/s 0.7
  8126. [0130 18:12:05 @multigpu.py:323] [p0576] step: count(1763), step_time 1446.88, mean_step_time 1431.63, it/s 0.7
  8127. [0130 18:12:05 @multigpu.py:323] [p0115] step: count(1739), step_time 1466.91, mean_step_time 1437.34, it/s 0.7
  8128. [0130 18:12:06 @multigpu.py:323] [p0576] step: count(1764), step_time 1357.56, mean_step_time 1426.91, it/s 0.7
  8129. [0130 18:12:06 @multigpu.py:323] [p0574] step: count(1755), step_time 1422.55, mean_step_time 1430.37, it/s 0.7
  8130. [0130 18:12:06 @multigpu.py:323] [p0115] step: count(1740), step_time 1379.79, mean_step_time 1433.07, it/s 0.7
  8131. [0130 18:12:08 @multigpu.py:323] [p0576] step: count(1765), step_time 1472.51, mean_step_time 1427.7, it/s 0.7
  8132. [0130 18:12:08 @multigpu.py:323] [p0574] step: count(1756), step_time 1487.65, mean_step_time 1434.47, it/s 0.7
  8133. [0130 18:12:08 @multigpu.py:323] [p0115] step: count(1741), step_time 1445.5, mean_step_time 1433.42, it/s 0.7
  8134. sending to address tcp://p0112:61216
  8135. ##### Sending to neptune: online_score : 0.702142093049 , 1.7 #####
  8136. [u'online', 1.7]
  8137. receiving
  8138. [0130 18:12:09 @multigpu.py:323] [p0576] step: count(1766), step_time 1390.08, mean_step_time 1424.94, it/s 0.7
  8139. [0130 18:12:09 @multigpu.py:323] [p0115] step: count(1742), step_time 1414.49, mean_step_time 1435.32, it/s 0.7
  8140. [0130 18:12:09 @multigpu.py:323] [p0574] step: count(1757), step_time 1442.0, mean_step_time 1435.53, it/s 0.7
  8141. [0130 18:12:11 @multigpu.py:323] [p0576] step: count(1767), step_time 1466.38, mean_step_time 1425.87, it/s 0.7
  8142. [0130 18:12:11 @multigpu.py:323] [p0574] step: count(1758), step_time 1425.23, mean_step_time 1435.13, it/s 0.7
  8143. [0130 18:12:11 @multigpu.py:323] [p0115] step: count(1743), step_time 1448.84, mean_step_time 1436.57, it/s 0.7
  8144. [0130 18:12:12 @multigpu.py:323] [p0576] step: count(1768), step_time 1411.88, mean_step_time 1425.51, it/s 0.7
  8145. [0130 18:12:12 @multigpu.py:323] [p0115] step: count(1744), step_time 1406.64, mean_step_time 1435.8, it/s 0.7
  8146. [0130 18:12:12 @multigpu.py:323] [p0574] step: count(1759), step_time 1462.57, mean_step_time 1436.86, it/s 0.7
  8147. [0130 18:12:14 @multigpu.py:323] [p0576] step: count(1769), step_time 1486.35, mean_step_time 1427.37, it/s 0.7
  8148. [0130 18:12:14 @multigpu.py:323] [p0574] step: count(1760), step_time 1393.44, mean_step_time 1436.32, it/s 0.7
  8149. [0130 18:12:14 @multigpu.py:323] [p0115] step: count(1745), step_time 1463.1, mean_step_time 1437.78, it/s 0.7
  8150. [0130 18:12:15 @multigpu.py:323] [p0576] step: count(1770), step_time 1415.81, mean_step_time 1426.4, it/s 0.7
  8151. [0130 18:12:15 @multigpu.py:323] [p0574] step: count(1761), step_time 1446.49, mean_step_time 1438.9, it/s 0.69
  8152. [0130 18:12:15 @multigpu.py:323] [p0115] step: count(1746), step_time 1441.33, mean_step_time 1438.08, it/s 0.7
  8153. [0130 18:12:16 @multigpu.py:323] [p0574] step: count(1762), step_time 1413.98, mean_step_time 1436.88, it/s 0.7
  8154. [0130 18:12:16 @multigpu.py:323] [p0115] step: count(1747), step_time 1410.81, mean_step_time 1439.63, it/s 0.69
  8155. [0130 18:12:16 @multigpu.py:323] [p0576] step: count(1771), step_time 1504.78, mean_step_time 1431.46, it/s 0.7
  8156. sending to address tcp://p0112:61216
  8157. ##### Sending to neptune: online_score : 0.704557807446 , 2.1 #####
  8158. [u'online', 2.1]
  8159. receiving
  8160. [0130 18:12:18 @multigpu.py:323] [p0115] step: count(1748), step_time 1460.03, mean_step_time 1432.26, it/s 0.7
  8161. [0130 18:12:18 @multigpu.py:323] [p0576] step: count(1772), step_time 1433.75, mean_step_time 1432.18, it/s 0.7
  8162. [0130 18:12:18 @multigpu.py:323] [p0574] step: count(1763), step_time 1467.53, mean_step_time 1432.14, it/s 0.7
  8163. [0130 18:12:19 @multigpu.py:323] [p0574] step: count(1764), step_time 1422.07, mean_step_time 1433.9, it/s 0.7
  8164. [0130 18:12:19 @multigpu.py:323] [p0576] step: count(1773), step_time 1446.41, mean_step_time 1434.48, it/s 0.7
  8165. [0130 18:12:19 @multigpu.py:323] [p0115] step: count(1749), step_time 1464.81, mean_step_time 1431.17, it/s 0.7
  8166. [0130 18:12:21 @multigpu.py:323] [p0574] step: count(1765), step_time 1389.37, mean_step_time 1430.44, it/s 0.7
  8167. [0130 18:12:21 @multigpu.py:323] [p0576] step: count(1774), step_time 1403.14, mean_step_time 1431.93, it/s 0.7
  8168. [0130 18:12:21 @multigpu.py:323] [p0115] step: count(1750), step_time 1480.55, mean_step_time 1434.53, it/s 0.7
  8169. [0130 18:12:22 @multigpu.py:323] [p0576] step: count(1775), step_time 1423.07, mean_step_time 1432.13, it/s 0.7
  8170. [0130 18:12:22 @multigpu.py:323] [p0574] step: count(1766), step_time 1475.22, mean_step_time 1434.86, it/s 0.7
  8171. [0130 18:12:22 @multigpu.py:323] [p0115] step: count(1751), step_time 1412.36, mean_step_time 1434.87, it/s 0.7
  8172. [0130 18:12:24 @multigpu.py:323] [p0576] step: count(1776), step_time 1395.53, mean_step_time 1430.33, it/s 0.7
  8173. [0130 18:12:24 @multigpu.py:323] [p0574] step: count(1767), step_time 1428.71, mean_step_time 1433.61, it/s 0.7
  8174. [0130 18:12:24 @multigpu.py:323] [p0115] step: count(1752), step_time 1423.21, mean_step_time 1436.17, it/s 0.7
  8175. sending to address tcp://p0112:61216
  8176. ##### Sending to neptune: online_score : 0.706616413593 , 1.7 #####
  8177. [u'online', 1.7]
  8178. receiving
  8179. [0130 18:12:25 @multigpu.py:323] [p0576] step: count(1777), step_time 1471.75, mean_step_time 1433.56, it/s 0.7
  8180. [0130 18:12:25 @multigpu.py:323] [p0574] step: count(1768), step_time 1438.86, mean_step_time 1437.17, it/s 0.7
  8181. [0130 18:12:25 @multigpu.py:323] [p0115] step: count(1753), step_time 1391.3, mean_step_time 1433.53, it/s 0.7
  8182. [0130 18:12:27 @multigpu.py:323] [p0576] step: count(1778), step_time 1478.33, mean_step_time 1434.09, it/s 0.7
  8183. [0130 18:12:27 @multigpu.py:323] [p0574] step: count(1769), step_time 1474.31, mean_step_time 1439.67, it/s 0.69
  8184. [0130 18:12:27 @multigpu.py:323] [p0115] step: count(1754), step_time 1505.91, mean_step_time 1436.58, it/s 0.7
  8185. [0130 18:12:28 @multigpu.py:323] [p0576] step: count(1779), step_time 1450.3, mean_step_time 1435.91, it/s 0.7
  8186. [0130 18:12:28 @multigpu.py:323] [p0574] step: count(1770), step_time 1442.26, mean_step_time 1439.79, it/s 0.69
  8187. [0130 18:12:28 @multigpu.py:323] [p0115] step: count(1755), step_time 1436.9, mean_step_time 1437.03, it/s 0.7
  8188. [0130 18:12:29 @multigpu.py:323] [p0574] step: count(1771), step_time 1409.93, mean_step_time 1436.87, it/s 0.7
  8189. [0130 18:12:29 @multigpu.py:323] [p0576] step: count(1780), step_time 1448.5, mean_step_time 1436.85, it/s 0.7
  8190. [0130 18:12:29 @multigpu.py:323] [p0115] step: count(1756), step_time 1444.44, mean_step_time 1436.27, it/s 0.7
  8191. [0130 18:12:31 @multigpu.py:323] [p0574] step: count(1772), step_time 1430.18, mean_step_time 1438.32, it/s 0.7
  8192. [0130 18:12:31 @multigpu.py:323] [p0576] step: count(1781), step_time 1446.05, mean_step_time 1436.5, it/s 0.7
  8193. [0130 18:12:31 @multigpu.py:323] [p0115] step: count(1757), step_time 1406.18, mean_step_time 1435.22, it/s 0.7
  8194. [0130 18:12:32 @multigpu.py:323] [p0576] step: count(1782), step_time 1418.56, mean_step_time 1438.38, it/s 0.7
  8195. [0130 18:12:32 @multigpu.py:323] [p0574] step: count(1773), step_time 1475.5, mean_step_time 1439.9, it/s 0.69
  8196. [0130 18:12:32 @multigpu.py:323] [p0115] step: count(1758), step_time 1439.71, mean_step_time 1437.14, it/s 0.7
  8197. [0130 18:12:34 @multigpu.py:323] [p0574] step: count(1774), step_time 1418.35, mean_step_time 1438.31, it/s 0.7
  8198. [0130 18:12:34 @multigpu.py:323] [p0576] step: count(1783), step_time 1465.9, mean_step_time 1439.33, it/s 0.69
  8199. [0130 18:12:34 @multigpu.py:323] [p0115] step: count(1759), step_time 1447.71, mean_step_time 1436.18, it/s 0.7
  8200. [0130 18:12:35 @multigpu.py:323] [p0574] step: count(1775), step_time 1432.75, mean_step_time 1438.82, it/s 0.7
  8201. [0130 18:12:35 @multigpu.py:323] [p0576] step: count(1784), step_time 1417.86, mean_step_time 1442.35, it/s 0.69
  8202. [0130 18:12:35 @multigpu.py:323] [p0115] step: count(1760), step_time 1483.61, mean_step_time 1441.37, it/s 0.69
  8203. [0130 18:12:37 @multigpu.py:323] [p0574] step: count(1776), step_time 1427.38, mean_step_time 1435.81, it/s 0.7
  8204. [0130 18:12:37 @multigpu.py:323] [p0576] step: count(1785), step_time 1414.84, mean_step_time 1439.46, it/s 0.69
  8205. [0130 18:12:37 @multigpu.py:323] [p0115] step: count(1761), step_time 1407.42, mean_step_time 1439.47, it/s 0.69
  8206. [0130 18:12:38 @multigpu.py:323] [p0574] step: count(1777), step_time 1370.77, mean_step_time 1432.24, it/s 0.7
  8207. [0130 18:12:38 @multigpu.py:323] [p0576] step: count(1786), step_time 1478.67, mean_step_time 1443.89, it/s 0.69
  8208. [0130 18:12:38 @multigpu.py:323] [p0115] step: count(1762), step_time 1440.08, mean_step_time 1440.75, it/s 0.69
  8209. sending to address tcp://p0112:61216
  8210. ##### Sending to neptune: online_score : 0.710599316094 , 0.9 #####
  8211. [u'online', 0.9]
  8212. ##### Sending to neptune: active_workers : 0.710599354108 , 3 #####
  8213. receiving
  8214. [0130 18:12:39 @multigpu.py:323] [p0574] step: count(1778), step_time 1393.35, mean_step_time 1430.65, it/s 0.7
  8215. [0130 18:12:40 @multigpu.py:323] [p0115] step: count(1763), step_time 1404.38, mean_step_time 1438.52, it/s 0.7
  8216. [0130 18:12:40 @multigpu.py:323] [p0576] step: count(1787), step_time 1472.56, mean_step_time 1444.2, it/s 0.69
  8217. [0130 18:12:41 @multigpu.py:323] [p0574] step: count(1779), step_time 1518.58, mean_step_time 1433.45, it/s 0.7
  8218. [0130 18:12:41 @multigpu.py:323] [p0115] step: count(1764), step_time 1478.67, mean_step_time 1442.13, it/s 0.69
  8219. [0130 18:12:41 @multigpu.py:323] [p0576] step: count(1788), step_time 1448.46, mean_step_time 1446.03, it/s 0.69
  8220. [0130 18:12:42 @multigpu.py:323] [p0574] step: count(1780), step_time 1417.89, mean_step_time 1434.67, it/s 0.7
  8221. [0130 18:12:42 @multigpu.py:323] [p0115] step: count(1765), step_time 1425.04, mean_step_time 1440.22, it/s 0.69
  8222. [0130 18:12:42 @multigpu.py:323] [p0576] step: count(1789), step_time 1459.16, mean_step_time 1444.67, it/s 0.69
  8223. [0130 18:12:44 @multigpu.py:323] [p0574] step: count(1781), step_time 1475.84, mean_step_time 1436.14, it/s 0.7
  8224. [0130 18:12:44 @multigpu.py:323] [p0115] step: count(1766), step_time 1432.78, mean_step_time 1439.79, it/s 0.69
  8225. [0130 18:12:44 @multigpu.py:323] [p0576] step: count(1790), step_time 1401.72, mean_step_time 1443.97, it/s 0.69
  8226. [0130 18:12:45 @multigpu.py:323] [p0574] step: count(1782), step_time 1502.9, mean_step_time 1440.59, it/s 0.69
  8227. [0130 18:12:45 @multigpu.py:323] [p0115] step: count(1767), step_time 1451.68, mean_step_time 1441.84, it/s 0.69
  8228. [0130 18:12:45 @multigpu.py:323] [p0576] step: count(1791), step_time 1448.4, mean_step_time 1441.15, it/s 0.69
  8229. [0130 18:12:47 @multigpu.py:323] [p0115] step: count(1768), step_time 1447.9, mean_step_time 1441.23, it/s 0.69
  8230. [0130 18:12:47 @multigpu.py:323] [p0574] step: count(1783), step_time 1478.6, mean_step_time 1441.14, it/s 0.69
  8231. [0130 18:12:47 @multigpu.py:323] [p0576] step: count(1792), step_time 1434.01, mean_step_time 1441.16, it/s 0.69
  8232. [0130 18:12:48 @multigpu.py:323] [p0576] step: count(1793), step_time 1426.57, mean_step_time 1440.17, it/s 0.69
  8233. [0130 18:12:48 @multigpu.py:323] [p0115] step: count(1769), step_time 1436.7, mean_step_time 1439.83, it/s 0.69
  8234. [0130 18:12:48 @multigpu.py:323] [p0574] step: count(1784), step_time 1524.06, mean_step_time 1446.24, it/s 0.69
  8235. [0130 18:12:50 @multigpu.py:323] [p0576] step: count(1794), step_time 1413.62, mean_step_time 1440.69, it/s 0.69
  8236. [0130 18:12:50 @multigpu.py:323] [p0115] step: count(1770), step_time 1430.53, mean_step_time 1437.33, it/s 0.7
  8237. [0130 18:12:50 @multigpu.py:323] [p0574] step: count(1785), step_time 1421.68, mean_step_time 1447.86, it/s 0.69
  8238. sending to address tcp://p0112:61216
  8239. ##### Sending to neptune: online_score : 0.713830539385 , 2.3 #####
  8240. [u'online', 2.3]
  8241. receiving
  8242. [0130 18:12:51 @multigpu.py:323] [p0576] step: count(1795), step_time 1453.62, mean_step_time 1442.22, it/s 0.69
  8243. [0130 18:12:51 @multigpu.py:323] [p0115] step: count(1771), step_time 1456.81, mean_step_time 1439.55, it/s 0.69
  8244. [0130 18:12:51 @multigpu.py:323] [p0574] step: count(1786), step_time 1498.47, mean_step_time 1449.02, it/s 0.69
  8245. [0130 18:12:52 @multigpu.py:323] [p0576] step: count(1796), step_time 1380.06, mean_step_time 1441.45, it/s 0.69
  8246. [0130 18:12:53 @multigpu.py:323] [p0115] step: count(1772), step_time 1439.45, mean_step_time 1440.36, it/s 0.69
  8247. [0130 18:12:53 @multigpu.py:323] [p0574] step: count(1787), step_time 1438.11, mean_step_time 1449.49, it/s 0.69
  8248. [0130 18:12:54 @multigpu.py:323] [p0115] step: count(1773), step_time 1373.46, mean_step_time 1439.47, it/s 0.69
  8249. [0130 18:12:54 @multigpu.py:323] [p0576] step: count(1797), step_time 1508.12, mean_step_time 1443.27, it/s 0.69
  8250. [0130 18:12:54 @multigpu.py:323] [p0574] step: count(1788), step_time 1391.79, mean_step_time 1447.13, it/s 0.69
  8251. sending to address tcp://p0112:61216
  8252. ##### Sending to neptune: online_score : 0.715037188 , 2.0 #####
  8253. [u'online', 2.0]
  8254. receiving
  8255. [0130 18:12:55 @multigpu.py:323] [p0576] step: count(1798), step_time 1371.25, mean_step_time 1437.91, it/s 0.7
  8256. [0130 18:12:55 @multigpu.py:323] [p0115] step: count(1774), step_time 1477.79, mean_step_time 1438.06, it/s 0.7
  8257. [0130 18:12:55 @multigpu.py:323] [p0574] step: count(1789), step_time 1416.09, mean_step_time 1444.22, it/s 0.69
  8258. [0130 18:12:57 @multigpu.py:323] [p0115] step: count(1775), step_time 1423.69, mean_step_time 1437.4, it/s 0.7
  8259. [0130 18:12:57 @multigpu.py:323] [p0576] step: count(1799), step_time 1497.91, mean_step_time 1440.29, it/s 0.69
  8260. [0130 18:12:57 @multigpu.py:323] [p0574] step: count(1790), step_time 1439.2, mean_step_time 1444.07, it/s 0.69
  8261. [0130 18:12:58 @multigpu.py:323] [p0115] step: count(1776), step_time 1466.31, mean_step_time 1438.5, it/s 0.7
  8262. [0130 18:12:58 @multigpu.py:323] [p0576] step: count(1800), step_time 1453.53, mean_step_time 1440.54, it/s 0.69
  8263. sending debugging info...
  8264. sending to address tcp://p0112:61216
  8265. ##### Sending to neptune: mean_delay : 0.716016146938 , 0.0 #####
  8266. ##### Sending to neptune: max_delay : 0.716016146938 , -0.0 #####
  8267. sending to address tcp://p0112:61216
  8268. ##### Sending to neptune: min_delay : 0.716016146938 , -0.0 #####
  8269. [u'delays', [0.0, -0.0, -0.0]]
  8270. receiving
  8271. ##### Sending to neptune: cost : 0.716016632716 , -0.0104687893763 #####
  8272. ##### Sending to neptune: policy_loss : 0.716016632716 , -0.755453646183 #####
  8273. sending to address tcp://p0112:61216
  8274. ##### Sending to neptune: xentropy_loss : 0.716016632716 , -2.27917194366 #####
  8275. ##### Sending to neptune: value_loss : 0.716016632716 , 1.69462025166 #####
  8276. ##### Sending to neptune: advantage : 0.716016632716 , 0.00311736739241 #####
  8277. ##### Sending to neptune: pred_reward : 0.716016632716 , 0.461956590414 #####
  8278. ##### Sending to neptune: max_logit : 0.716016632716 , 0.216024249792 #####
  8279. [u'loss', -0.010468789376318455, -0.7554536461830139, -2.279171943664551, 1.6946202516555786, 0.0031173673924058676, 0.46195659041404724, 0.216024249792099]
  8280. receiving
  8281. ##### Sending to neptune: active_relus : 0.71601706273 , 9774144.07 #####
  8282. ##### Sending to neptune: dp_per_s : 0.71601706273 , 89.0824345897 #####
  8283. [u'other', 9774144.07, 89.08243458972638]
  8284. receiving
  8285. [0130 18:12:58 @multigpu.py:323] [p0574] step: count(1791), step_time 1447.3, mean_step_time 1445.94, it/s 0.69
  8286. [0130 18:13:00 @multigpu.py:323] [p0576] step: count(1801), step_time 1443.6, mean_step_time 1440.42, it/s 0.69
  8287. [0130 18:13:00 @multigpu.py:323] [p0574] step: count(1792), step_time 1389.92, mean_step_time 1443.93, it/s 0.69
  8288. [0130 18:13:00 @multigpu.py:323] [p0115] step: count(1777), step_time 1474.88, mean_step_time 1441.93, it/s 0.69
  8289. [0130 18:13:01 @multigpu.py:323] [p0576] step: count(1802), step_time 1441.11, mean_step_time 1441.55, it/s 0.69
  8290. [0130 18:13:01 @multigpu.py:323] [p0115] step: count(1778), step_time 1417.01, mean_step_time 1440.79, it/s 0.69
  8291. [0130 18:13:01 @multigpu.py:323] [p0574] step: count(1793), step_time 1536.17, mean_step_time 1446.96, it/s 0.69
  8292. [0130 18:13:03 @multigpu.py:323] [p0115] step: count(1779), step_time 1430.11, mean_step_time 1439.91, it/s 0.69
  8293. [0130 18:13:03 @multigpu.py:323] [p0576] step: count(1803), step_time 1462.05, mean_step_time 1441.36, it/s 0.69
  8294. [0130 18:13:03 @multigpu.py:323] [p0574] step: count(1794), step_time 1425.58, mean_step_time 1447.32, it/s 0.69
  8295. sending to address tcp://p0112:61216
  8296. ##### Sending to neptune: online_score : 0.717419414388 , 1.5 #####
  8297. [u'online', 1.5]
  8298. receiving
  8299. [0130 18:13:04 @multigpu.py:323] [p0115] step: count(1780), step_time 1381.04, mean_step_time 1434.79, it/s 0.7
  8300. [0130 18:13:04 @multigpu.py:323] [p0576] step: count(1804), step_time 1468.83, mean_step_time 1443.9, it/s 0.69
  8301. [0130 18:13:04 @multigpu.py:323] [p0574] step: count(1795), step_time 1551.57, mean_step_time 1453.26, it/s 0.69
  8302. [0130 18:13:05 @multigpu.py:323] [p0115] step: count(1781), step_time 1463.72, mean_step_time 1437.6, it/s 0.7
  8303. [0130 18:13:06 @multigpu.py:323] [p0576] step: count(1805), step_time 1420.94, mean_step_time 1444.21, it/s 0.69
  8304. [0130 18:13:06 @multigpu.py:323] [p0574] step: count(1796), step_time 1475.27, mean_step_time 1455.66, it/s 0.69
  8305. [0130 18:13:07 @multigpu.py:323] [p0115] step: count(1782), step_time 1433.38, mean_step_time 1437.27, it/s 0.7
  8306. [0130 18:13:07 @multigpu.py:323] [p0576] step: count(1806), step_time 1375.32, mean_step_time 1439.04, it/s 0.69
  8307. [0130 18:13:07 @multigpu.py:323] [p0574] step: count(1797), step_time 1493.17, mean_step_time 1461.78, it/s 0.68
  8308. [0130 18:13:08 @multigpu.py:323] [p0115] step: count(1783), step_time 1401.67, mean_step_time 1437.13, it/s 0.7
  8309. [0130 18:13:08 @multigpu.py:323] [p0576] step: count(1807), step_time 1430.79, mean_step_time 1436.95, it/s 0.7
  8310. [0130 18:13:09 @multigpu.py:323] [p0574] step: count(1798), step_time 1530.55, mean_step_time 1468.64, it/s 0.68
  8311. [0130 18:13:10 @multigpu.py:323] [p0115] step: count(1784), step_time 1447.27, mean_step_time 1435.56, it/s 0.7
  8312. [0130 18:13:10 @multigpu.py:323] [p0576] step: count(1808), step_time 1412.44, mean_step_time 1435.15, it/s 0.7
  8313. 80%|#######9 |799/1000[19:[0130 18:13:10 @multigpu.py:323] [p0574] step: count(1799), step_time 1397.36, mean_step_time 1462.58, it/s 0.68
  8314. [0130 18:13:11 @multigpu.py:323] [p0115] step: count(1785), step_time 1426.48, mean_step_time 1435.63, it/s 0.7
  8315. [0130 18:13:11 @multigpu.py:323] [p0576] step: count(1809), step_time 1425.22, mean_step_time 1433.46, it/s 0.7
  8316. [0130 18:13:12 @multigpu.py:323] [p0574] step: count(1800), step_time 1476.52, mean_step_time 1465.51, it/s 0.68
  8317. sending debugging info...
  8318. sending to address tcp://p0112:61216
  8319. ##### Sending to neptune: mean_delay : 0.719725394713 , 0.0 #####
  8320. sending to address tcp://p0112:61216
  8321. ##### Sending to neptune: max_delay : 0.719725394713 , -0.0 #####
  8322. ##### Sending to neptune: min_delay : 0.719725394713 , -0.0 #####
  8323. [u'delays', [0.0, -0.0, -0.0]]
  8324. receiving
  8325. ##### Sending to neptune: cost : 0.719725849695 , -0.00274460855871 #####
  8326. sending to address tcp://p0112:61216
  8327. ##### Sending to neptune: policy_loss : 0.719725849695 , 0.0771632865071 #####
  8328. ##### Sending to neptune: xentropy_loss : 0.719725849695 , -2.27880167961 #####
  8329. ##### Sending to neptune: value_loss : 0.719725849695 , 1.85032880306 #####
  8330. ##### Sending to neptune: advantage : 0.719725849695 , -0.000355047988705 #####
  8331. ##### Sending to neptune: pred_reward : 0.719725849695 , 0.449805110693 #####
  8332. ##### Sending to neptune: max_logit : 0.719725849695 , 0.217000186443 #####
  8333. [u'loss', -0.00274460855871439, 0.07716328650712967, -2.278801679611206, 1.850328803062439, -0.00035504798870533705, 0.4498051106929779, 0.21700018644332886]
  8334. receiving
  8335. ##### Sending to neptune: active_relus : 0.719726285802 , 9780323.06 #####
  8336. ##### Sending to neptune: dp_per_s : 0.719726285802 , 89.0251181659 #####
  8337. [u'other', 9780323.06, 89.02511816592134]
  8338. receiving
  8339. [0130 18:13:13 @multigpu.py:323] [p0576] step: count(1810), step_time 1408.54, mean_step_time 1433.8, it/s 0.7
  8340. [0130 18:13:13 @multigpu.py:323] [p0115] step: count(1786), step_time 1458.67, mean_step_time 1436.93, it/s 0.7
  8341. [0130 18:13:13 @multigpu.py:323] [p0574] step: count(1801), step_time 1422.7, mean_step_time 1462.85, it/s 0.68
  8342. [0130 18:13:14 @multigpu.py:323] [p0576] step: count(1811), step_time 1420.61, mean_step_time 1432.41, it/s 0.7
  8343. [0130 18:13:14 @multigpu.py:323] [p0115] step: count(1787), step_time 1511.66, mean_step_time 1439.93, it/s 0.69
  8344. [0130 18:13:14 @multigpu.py:323] [p0574] step: count(1802), step_time 1416.28, mean_step_time 1458.52, it/s 0.69
  8345. [0130 18:13:16 @multigpu.py:323] [p0576] step: count(1812), step_time 1975.63, mean_step_time 1459.49, it/s 0.69
  8346. [0130 18:13:16 @multigpu.py:323] [p0574] step: count(1803), step_time 1497.58, mean_step_time 1459.47, it/s 0.69
  8347. [0130 18:13:16 @multigpu.py:323] [p0115] step: count(1788), step_time 1839.76, mean_step_time 1459.52, it/s 0.69
  8348. [0130 18:13:17 @multigpu.py:323] [p0576] step: count(1813), step_time 1410.21, mean_step_time 1458.67, it/s 0.69
  8349. [0130 18:13:17 @multigpu.py:323] [p0115] step: count(1789), step_time 1418.83, mean_step_time 1458.63, it/s 0.69
  8350. [0130 18:13:17 @multigpu.py:323] [p0574] step: count(1804), step_time 1427.75, mean_step_time 1454.65, it/s 0.69
  8351. [0130 18:13:19 @multigpu.py:323] [p0576] step: count(1814), step_time 1428.95, mean_step_time 1459.44, it/s 0.69
  8352. [0130 18:13:19 @multigpu.py:323] [p0115] step: count(1790), step_time 1435.65, mean_step_time 1458.88, it/s 0.69
  8353. [0130 18:13:19 @multigpu.py:323] [p0574] step: count(1805), step_time 1583.53, mean_step_time 1462.75, it/s 0.68
  8354. sending to address tcp://p0112:61216
  8355. ##### Sending to neptune: online_score : 0.721848295 , 2.2 #####
  8356. [u'online', 2.2]
  8357. receiving
  8358. [0130 18:13:20 @multigpu.py:323] [p0576] step: count(1815), step_time 1385.35, mean_step_time 1456.02, it/s 0.69
  8359. [0130 18:13:20 @multigpu.py:323] [p0115] step: count(1791), step_time 1441.33, mean_step_time 1458.11, it/s 0.69
  8360. [0130 18:13:20 @multigpu.py:323] [p0574] step: count(1806), step_time 1451.14, mean_step_time 1460.38, it/s 0.68
  8361. [0130 18:13:22 @multigpu.py:323] [p0576] step: count(1816), step_time 1406.67, mean_step_time 1457.35, it/s 0.69
  8362. [0130 18:13:22 @multigpu.py:323] [p0115] step: count(1792), step_time 1455.17, mean_step_time 1458.89, it/s 0.69
  8363. [0130 18:13:22 @multigpu.py:323] [p0574] step: count(1807), step_time 1446.57, mean_step_time 1460.8, it/s 0.68
  8364. sending to address tcp://p0112:61216
  8365. ##### Sending to neptune: online_score : 0.722776274681 , 1.5 #####
  8366. [u'online', 1.5]
  8367. receiving
  8368. 80%|#######9 |799/1000[19:[0130 18:13:23 @multigpu.py:323] [p0576] step: count(1817), step_time 1429.63, mean_step_time 1453.43, it/s 0.69
  8369. [0130 18:13:23 @multigpu.py:323] [p0115] step: count(1793), step_time 1429.46, mean_step_time 1461.69, it/s 0.68
  8370. [0130 18:13:23 @multigpu.py:323] [p0574] step: count(1808), step_time 1495.21, mean_step_time 1465.97, it/s 0.68
  8371. [0130 18:13:24 @multigpu.py:323] [p0576] step: count(1818), step_time 1461.6, mean_step_time 1457.95, it/s 0.69
  8372. [0130 18:13:25 @multigpu.py:323] [p0115] step: count(1794), step_time 1423.89, mean_step_time 1459.0, it/s 0.69
  8373. [0130 18:13:25 @multigpu.py:323] [p0574] step: count(1809), step_time 1456.05, mean_step_time 1467.97, it/s 0.68
  8374. [0130 18:13:26 @multigpu.py:323] [p0576] step: count(1819), step_time 1457.85, mean_step_time 1455.94, it/s 0.69
  8375. [0130 18:13:26 @multigpu.py:323] [p0115] step: count(1795), step_time 1457.69, mean_step_time 1460.7, it/s 0.68
  8376. [0130 18:13:26 @multigpu.py:323] [p0574] step: count(1810), step_time 1477.54, mean_step_time 1469.89, it/s 0.68
  8377. [0130 18:13:27 @multigpu.py:323] [p0576] step: count(1820), step_time 1419.63, mean_step_time 1454.25, it/s 0.69
  8378. [0130 18:13:27 @multigpu.py:323] [p0115] step: count(1796), step_time 1477.3, mean_step_time 1461.25, it/s 0.68
  8379. [0130 18:13:28 @multigpu.py:323] [p0574] step: count(1811), step_time 1431.16, mean_step_time 1469.08, it/s 0.68
  8380. sending to address tcp://p0112:61216
  8381. ##### Sending to neptune: online_score : 0.724213219682 , 1.2 #####
  8382. [u'online', 1.2]
  8383. receiving
  8384. [0130 18:13:29 @multigpu.py:323] [p0576] step: count(1821), step_time 1390.79, mean_step_time 1451.61, it/s 0.69
  8385. [0130 18:13:29 @multigpu.py:323] [p0115] step: count(1797), step_time 1504.25, mean_step_time 1462.72, it/s 0.68
  8386. [0130 18:13:29 @multigpu.py:323] [p0574] step: count(1812), step_time 1439.13, mean_step_time 1471.54, it/s 0.68
  8387. [0130 18:13:30 @multigpu.py:323] [p0576] step: count(1822), step_time 1412.46, mean_step_time 1450.17, it/s 0.69
  8388. [0130 18:13:30 @multigpu.py:323] [p0115] step: count(1798), step_time 1406.23, mean_step_time 1462.18, it/s 0.68
  8389. [0130 18:13:31 @multigpu.py:323] [p0574] step: count(1813), step_time 1544.17, mean_step_time 1471.94, it/s 0.68
  8390. [0130 18:13:32 @multigpu.py:323] [p0576] step: count(1823), step_time 1487.29, mean_step_time 1451.44, it/s 0.69
  8391. [0130 18:13:32 @multigpu.py:323] [p0115] step: count(1799), step_time 1443.42, mean_step_time 1462.84, it/s 0.68
  8392. [0130 18:13:32 @multigpu.py:323] [p0574] step: count(1814), step_time 1427.82, mean_step_time 1472.05, it/s 0.68
  8393. [0130 18:13:33 @multigpu.py:323] [p0576] step: count(1824), step_time 1426.19, mean_step_time 1449.31, it/s 0.69
  8394. [0130 18:13:33 @multigpu.py:323] [p0115] step: count(1800), step_time 1415.9, mean_step_time 1464.59, it/s 0.68
  8395. sending debugging info...
  8396. sending to address tcp://p0112:61216
  8397. ##### Sending to neptune: mean_delay : 0.725740494397 , 0.0 #####
  8398. ##### Sending to neptune: max_delay : 0.725740494397 , -0.0 #####
  8399. sending to address tcp://p0112:61216
  8400. ##### Sending to neptune: min_delay : 0.725740494397 , -0.0 #####
  8401. [u'delays', [0.0, -0.0, -0.0]]
  8402. receiving
  8403. ##### Sending to neptune: cost : 0.725740992162 , 0.00403387658298 #####
  8404. sending to address tcp://p0112:61216
  8405. ##### Sending to neptune: policy_loss : 0.725740992162 , 0.927618563175 #####
  8406. ##### Sending to neptune: xentropy_loss : 0.725740992162 , -2.27812504768 #####
  8407. ##### Sending to neptune: value_loss : 0.725740992162 , 1.86684274673 #####
  8408. ##### Sending to neptune: advantage : 0.725740992162 , -0.00428262539208 #####
  8409. ##### Sending to neptune: pred_reward : 0.725740992162 , 0.439312964678 #####
  8410. ##### Sending to neptune: max_logit : 0.725740992162 , 0.219251438975 #####
  8411. [u'loss', 0.004033876582980156, 0.9276185631752014, -2.278125047683716, 1.8668427467346191, -0.004282625392079353, 0.43931296467781067, 0.21925143897533417]
  8412. receiving
  8413. ##### Sending to neptune: active_relus : 0.725741453038 , 9763375.74 #####
  8414. ##### Sending to neptune: dp_per_s : 0.725741453038 , 88.8911625156 #####
  8415. [u'other', 9763375.74, 88.89116251556536]
  8416. receiving
  8417. [0130 18:13:34 @multigpu.py:323] [p0574] step: count(1815), step_time 1463.02, mean_step_time 1467.63, it/s 0.68
  8418. [0130 18:13:34 @multigpu.py:323] [p0576] step: count(1825), step_time 1403.33, mean_step_time 1448.42, it/s 0.69
  8419. [0130 18:13:35 @multigpu.py:323] [p0115] step: count(1801), step_time 1427.97, mean_step_time 1462.8, it/s 0.68
  8420. [0130 18:13:35 @multigpu.py:323] [p0574] step: count(1816), step_time 1432.76, mean_step_time 1465.5, it/s 0.68
  8421. [0130 18:13:36 @multigpu.py:323] [p0576] step: count(1826), step_time 1400.48, mean_step_time 1449.68, it/s 0.69
  8422. [0130 18:13:36 @multigpu.py:323] [p0115] step: count(1802), step_time 1482.07, mean_step_time 1465.23, it/s 0.68
  8423. [0130 18:13:36 @multigpu.py:323] [p0574] step: count(1817), step_time 1442.84, mean_step_time 1462.98, it/s 0.68
  8424. [0130 18:13:37 @multigpu.py:323] [p0576] step: count(1827), step_time 1420.96, mean_step_time 1449.19, it/s 0.69
  8425. [0130 18:13:38 @multigpu.py:323] [p0115] step: count(1803), step_time 1445.0, mean_step_time 1467.4, it/s 0.68
  8426. [0130 18:13:38 @multigpu.py:323] [p0574] step: count(1818), step_time 1455.15, mean_step_time 1459.21, it/s 0.69
  8427. [0130 18:13:39 @multigpu.py:323] [p0576] step: count(1828), step_time 1433.15, mean_step_time 1450.23, it/s 0.69
  8428. [0130 18:13:39 @multigpu.py:323] [p0115] step: count(1804), step_time 1464.72, mean_step_time 1468.27, it/s 0.68
  8429. [0130 18:13:39 @multigpu.py:323] [p0574] step: count(1819), step_time 1462.78, mean_step_time 1462.48, it/s 0.68
  8430. [0130 18:13:40 @multigpu.py:323] [p0576] step: count(1829), step_time 1441.63, mean_step_time 1451.05, it/s 0.69
  8431. [0130 18:13:40 @multigpu.py:323] [p0115] step: count(1805), step_time 1383.17, mean_step_time 1466.11, it/s 0.68
  8432. [0130 18:13:41 @multigpu.py:323] [p0574] step: count(1820), step_time 1501.29, mean_step_time 1463.72, it/s 0.68
  8433. [0130 18:13:42 @multigpu.py:323] [p0576] step: count(1830), step_time 1455.53, mean_step_time 1453.4, it/s 0.69
  8434. [0130 18:13:42 @multigpu.py:323] [p0115] step: count(1806), step_time 1462.52, mean_step_time 1466.3, it/s 0.68
  8435. [0130 18:13:42 @multigpu.py:323] [p0574] step: count(1821), step_time 1466.15, mean_step_time 1465.9, it/s 0.68
  8436. [0130 18:13:43 @multigpu.py:323] [p0576] step: count(1831), step_time 1391.55, mean_step_time 1451.94, it/s 0.69
  8437. [0130 18:13:43 @multigpu.py:323] [p0115] step: count(1807), step_time 1404.3, mean_step_time 1460.93, it/s 0.68
  8438. [0130 18:13:44 @multigpu.py:323] [p0574] step: count(1822), step_time 1427.68, mean_step_time 1466.47, it/s 0.68
  8439. 80%|#######9 |798/1000[19:[0130 18:13:45 @multigpu.py:323] [p0574] step: count(1823), step_time 1504.06, mean_step_time 1466.79, it/s 0.68
  8440. [0130 18:13:45 @multigpu.py:323] [p0576] step: count(1832), step_time 2273.5, mean_step_time 1466.84, it/s 0.68
  8441. [0130 18:13:45 @multigpu.py:323] [p0115] step: count(1808), step_time 1955.83, mean_step_time 1466.74, it/s 0.68
  8442. [0130 18:13:47 @multigpu.py:323] [p0574] step: count(1824), step_time 1406.97, mean_step_time 1465.75, it/s 0.68
  8443. [0130 18:13:47 @multigpu.py:323] [p0576] step: count(1833), step_time 1409.99, mean_step_time 1466.83, it/s 0.68
  8444. [0130 18:13:47 @multigpu.py:323] [p0115] step: count(1809), step_time 1458.16, mean_step_time 1468.7, it/s 0.68
  8445. [0130 18:13:48 @multigpu.py:323] [p0576] step: count(1834), step_time 1391.96, mean_step_time 1464.98, it/s 0.68
  8446. [0130 18:13:48 @multigpu.py:323] [p0574] step: count(1825), step_time 1469.67, mean_step_time 1460.06, it/s 0.68
  8447. [0130 18:13:48 @multigpu.py:323] [p0115] step: count(1810), step_time 1479.4, mean_step_time 1470.89, it/s 0.68
  8448. [0130 18:13:50 @multigpu.py:323] [p0576] step: count(1835), step_time 1454.65, mean_step_time 1468.44, it/s 0.68
  8449. [0130 18:13:50 @multigpu.py:323] [p0574] step: count(1826), step_time 1450.49, mean_step_time 1460.03, it/s 0.68
  8450. [0130 18:13:50 @multigpu.py:323] [p0115] step: count(1811), step_time 1414.45, mean_step_time 1469.55, it/s 0.68
  8451. sending to address tcp://p0112:61216
  8452. ##### Sending to neptune: online_score : 0.730388570494 , 2.4 #####
  8453. [u'online', 2.4]
  8454. ##### Sending to neptune: active_workers : 0.73038869222 , 3 #####
  8455. receiving
  8456. sending to address tcp://p0112:61216
  8457. ##### Sending to neptune: online_score : 0.730456432435 , 1.1 #####
  8458. [u'online', 1.1]
  8459. receiving
  8460. [0130 18:13:51 @multigpu.py:323] [p0576] step: count(1836), step_time 1441.33, mean_step_time 1470.18, it/s 0.68
  8461. [0130 18:13:51 @multigpu.py:323] [p0574] step: count(1827), step_time 1471.22, mean_step_time 1461.26, it/s 0.68
  8462. [0130 18:13:51 @multigpu.py:323] [p0115] step: count(1812), step_time 1472.86, mean_step_time 1470.43, it/s 0.68
  8463. [0130 18:13:52 @multigpu.py:323] [p0576] step: count(1837), step_time 1422.47, mean_step_time 1469.82, it/s 0.68
  8464. [0130 18:13:52 @multigpu.py:323] [p0574] step: count(1828), step_time 1382.45, mean_step_time 1455.62, it/s 0.69
  8465. [0130 18:13:53 @multigpu.py:323] [p0115] step: count(1813), step_time 1386.05, mean_step_time 1468.26, it/s 0.68
  8466. [0130 18:13:54 @multigpu.py:323] [p0576] step: count(1838), step_time 1407.76, mean_step_time 1467.13, it/s 0.68
  8467. [0130 18:13:54 @multigpu.py:323] [p0574] step: count(1829), step_time 1432.93, mean_step_time 1454.46, it/s 0.69
  8468. [0130 18:13:54 @multigpu.py:323] [p0115] step: count(1814), step_time 1424.11, mean_step_time 1468.27, it/s 0.68
  8469. [0130 18:13:55 @multigpu.py:323] [p0576] step: count(1839), step_time 1409.52, mean_step_time 1464.71, it/s 0.68
  8470. [0130 18:13:55 @multigpu.py:323] [p0115] step: count(1815), step_time 1428.61, mean_step_time 1466.82, it/s 0.68
  8471. [0130 18:13:55 @multigpu.py:323] [p0574] step: count(1830), step_time 1454.95, mean_step_time 1453.33, it/s 0.69
  8472. [0130 18:13:57 @multigpu.py:323] [p0576] step: count(1840), step_time 1423.07, mean_step_time 1464.88, it/s 0.68
  8473. [0130 18:13:57 @multigpu.py:323] [p0115] step: count(1816), step_time 1405.78, mean_step_time 1463.24, it/s 0.68
  8474. [0130 18:13:57 @multigpu.py:323] [p0574] step: count(1831), step_time 1447.78, mean_step_time 1454.17, it/s 0.69
  8475. [0130 18:13:58 @multigpu.py:323] [p0576] step: count(1841), step_time 1445.77, mean_step_time 1467.63, it/s 0.68
  8476. [0130 18:13:58 @multigpu.py:323] [p0115] step: count(1817), step_time 1432.9, mean_step_time 1459.67, it/s 0.69
  8477. [0130 18:13:58 @multigpu.py:323] [p0574] step: count(1832), step_time 1483.64, mean_step_time 1456.39, it/s 0.69
  8478. [0130 18:14:00 @multigpu.py:323] [p0576] step: count(1842), step_time 1426.93, mean_step_time 1468.35, it/s 0.68
  8479. [0130 18:14:00 @multigpu.py:323] [p0115] step: count(1818), step_time 1473.33, mean_step_time 1463.03, it/s 0.68
  8480. [0130 18:14:00 @multigpu.py:323] [p0574] step: count(1833), step_time 1489.83, mean_step_time 1453.67, it/s 0.69
  8481. [0130 18:14:01 @multigpu.py:323] [p0576] step: count(1843), step_time 1385.74, mean_step_time 1463.28, it/s 0.68
  8482. [0130 18:14:01 @multigpu.py:323] [p0115] step: count(1819), step_time 1469.2, mean_step_time 1464.32, it/s 0.68
  8483. [0130 18:14:01 @multigpu.py:323] [p0574] step: count(1834), step_time 1398.6, mean_step_time 1452.21, it/s 0.69
  8484. [0130 18:14:02 @multigpu.py:323] [p0576] step: count(1844), step_time 1443.21, mean_step_time 1464.13, it/s 0.68
  8485. [0130 18:14:03 @multigpu.py:323] [p0115] step: count(1820), step_time 1427.86, mean_step_time 1464.92, it/s 0.68
  8486. [0130 18:14:03 @multigpu.py:323] [p0574] step: count(1835), step_time 1441.65, mean_step_time 1451.14, it/s 0.69
  8487. [0130 18:14:04 @multigpu.py:323] [p0576] step: count(1845), step_time 1500.38, mean_step_time 1468.98, it/s 0.68
  8488. [0130 18:14:04 @multigpu.py:323] [p0115] step: count(1821), step_time 1393.41, mean_step_time 1463.19, it/s 0.68
  8489. [0130 18:14:04 @multigpu.py:323] [p0574] step: count(1836), step_time 1437.75, mean_step_time 1451.39, it/s 0.69
  8490. [0130 18:14:05 @multigpu.py:323] [p0576] step: count(1846), step_time 1414.1, mean_step_time 1469.66, it/s 0.68
  8491. [0130 18:14:05 @multigpu.py:323] [p0115] step: count(1822), step_time 1463.59, mean_step_time 1462.26, it/s 0.68
  8492. [0130 18:14:06 @multigpu.py:323] [p0574] step: count(1837), step_time 1428.28, mean_step_time 1450.67, it/s 0.69
  8493. [0130 18:14:07 @multigpu.py:323] [p0576] step: count(1847), step_time 1361.17, mean_step_time 1466.67, it/s 0.68
  8494. [0130 18:14:07 @multigpu.py:323] [p0115] step: count(1823), step_time 1388.61, mean_step_time 1459.44, it/s 0.69
  8495. [0130 18:14:07 @multigpu.py:323] [p0574] step: count(1838), step_time 1460.91, mean_step_time 1450.95, it/s 0.69
  8496. [0130 18:14:08 @multigpu.py:323] [p0576] step: count(1848), step_time 1394.95, mean_step_time 1464.76, it/s 0.68
  8497. sending to address tcp://p0112:61216
  8498. ##### Sending to neptune: online_score : 0.735401380526 , 2.3 #####
  8499. [u'online', 2.3]
  8500. receiving
  8501. [0130 18:14:08 @multigpu.py:323] [p0115] step: count(1824), step_time 1443.5, mean_step_time 1458.38, it/s 0.69
  8502. [0130 18:14:08 @multigpu.py:323] [p0574] step: count(1839), step_time 1438.72, mean_step_time 1449.75, it/s 0.69
  8503. [0130 18:14:10 @multigpu.py:323] [p0576] step: count(1849), step_time 1470.8, mean_step_time 1466.22, it/s 0.68
  8504. [0130 18:14:10 @multigpu.py:323] [p0115] step: count(1825), step_time 1478.5, mean_step_time 1463.15, it/s 0.68
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!