2019-03-25 20:39:06 (225 MB/s) - ‘test_current.yaml’ saved [1952/1952]

dataset:
  input_test: ./chunks/test/
  input_train: ./chunks/
  num_chunks: 600000
  train_ratio: 0.9
gpu: 0
model:
  filters: 8
  policy_channels: 20
  residual_blocks: 1
  se_ratio: 2
name: 8x1
training:
  batch_size: 4096
  checkpoint_steps: 100
  lr_boundaries:
  - 150
  lr_values:
  - 0.1
  - 0.1
  max_grad_norm: 2
  num_batch_splits: 4
  path: ./networks
  policy_loss_weight: 0.0
  shuffle_size: 60000
  swa: false
  swa_max_n: 10
  swa_steps: 25
  test_steps: 1000
  total_steps: 10000
  train_avg_report_steps: 100
  value_loss_weight: 1.0
  warmup_steps: 100

sorting 540000 chunks...[done]
game_000012.gz - game_599995.gz
sorting 60000 chunks...[done]
game_000160.gz - game_599990.gz
Using 1 worker processes.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/data/ops/dataset_ops.py:429: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, use
    tf.py_function, which takes a python function which manipulates tf eager
    tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
    an ndarray (just call tensor.numpy()) but having access to eager tensors
    means `tf.py_function`s can use accelerators such as GPUs as well as
    being differentiable using a gradient tape.

Using 1 worker processes.
2019-03-25 20:39:18.672989: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2300000000 Hz
2019-03-25 20:39:18.674515: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x315d340 executing computations on platform Host. Devices:
2019-03-25 20:39:18.674569: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>
2019-03-25 20:39:18.775746: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:998] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-03-25 20:39:18.776309: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x315de40 executing computations on platform CUDA. Devices:
2019-03-25 20:39:18.776349: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): Tesla K80, Compute Capability 3.7
2019-03-25 20:39:18.776772: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1433] Found device 0 with properties:
name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235
pciBusID: 0000:00:04.0
totalMemory: 11.17GiB freeMemory: 11.10GiB
2019-03-25 20:39:18.776811: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1512] Adding visible gpu devices: 0
2019-03-25 20:39:19.156966: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-03-25 20:39:19.157048: I tensorflow/core/common_runtime/gpu/gpu_device.cc:990]      0
2019-03-25 20:39:19.157081: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1003] 0:   N
2019-03-25 20:39:19.157425: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10297 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /content/lczero-training/tf/tfprocess.py:681: batch_normalization (from tensorflow.python.layers.normalization) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.batch_normalization instead.
WARNING:tensorflow:From /content/lczero-training/tf/tfprocess.py:144: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.
Instructions for updating:

Future major versions of TensorFlow will allow gradients to flow
into the labels input on backprop by default.

See `tf.nn.softmax_cross_entropy_with_logits_v2`.


WARNING: The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
If you depend on functionality not listed there, please file an issue.

WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
Using 585 evaluation batches
2019-03-25 20:39:25.151418: I tensorflow/stream_executor/dso_loader.cc:152] successfully opened CUDA library libcublas.so.10.0 locally
step 1, policy=7.52252 value=1.0988 policy accuracy=0.00901442% value accuracy=34.6448% mse=0.176582
step 100, lr=0.1 policy=7.52972 value=0.297644 mse=0.0271257 reg=0.0166562 total=0.3143 (1858.84 pos/s)
/content/lczero-training/tf/tfprocess.py:513: RuntimeWarning: invalid value encountered in float_scalars
  ratios = [(tensor.name, d / w) for d, w, tensor in zip(delta_norms, weight_norms, self.weights) if not 'moving' in tensor.name]
Model saved in file: ./networks/8x1/8x1-100
saved as './networks/8x1/8x1-100' 0.53M
Weights saved in file: ./networks/8x1/8x1-100
step 200, lr=0.1 policy=7.52468 value=0.0348502 mse=0.00240767 reg=0.0170679 total=0.0519181 (4462.86 pos/s)
Model saved in file: ./networks/8x1/8x1-200
saved as './networks/8x1/8x1-200' 0.53M
Weights saved in file: ./networks/8x1/8x1-200
step 300, lr=0.1 policy=7.52107 value=0.0172885 mse=0.00119191 reg=0.016921 total=0.0342095 (4486.35 pos/s)
Model saved in file: ./networks/8x1/8x1-300
saved as './networks/8x1/8x1-300' 0.52M
Weights saved in file: ./networks/8x1/8x1-300
step 400, lr=0.1 policy=7.51661 value=0.0135307 mse=0.000948175 reg=0.0167142 total=0.0302449 (4477.95 pos/s)
Model saved in file: ./networks/8x1/8x1-400
saved as './networks/8x1/8x1-400' 0.52M
Weights saved in file: ./networks/8x1/8x1-400
step 500, lr=0.1 policy=7.51575 value=0.010787 mse=0.000742922 reg=0.0164907 total=0.0272778 (4468.2 pos/s)
Model saved in file: ./networks/8x1/8x1-500
saved as './networks/8x1/8x1-500' 0.52M
Weights saved in file: ./networks/8x1/8x1-500
step 600, lr=0.1 policy=7.51449 value=0.0102407 mse=0.000715633 reg=0.0162645 total=0.0265052 (4460.45 pos/s)
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:966: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to delete files with this prefix.
Model saved in file: ./networks/8x1/8x1-600
saved as './networks/8x1/8x1-600' 0.52M
Weights saved in file: ./networks/8x1/8x1-600
step 700, lr=0.1 policy=7.51022 value=0.00875754 mse=0.000601832 reg=0.0160357 total=0.0247932 (4475.05 pos/s)
Model saved in file: ./networks/8x1/8x1-700
saved as './networks/8x1/8x1-700' 0.52M
Weights saved in file: ./networks/8x1/8x1-700
step 800, lr=0.1 policy=7.50928 value=0.0080477 mse=0.000555158 reg=0.0158038 total=0.0238515 (4460.32 pos/s)
Model saved in file: ./networks/8x1/8x1-800
saved as './networks/8x1/8x1-800' 0.52M
Weights saved in file: ./networks/8x1/8x1-800
step 900, lr=0.1 policy=7.51171 value=0.00793566 mse=0.00055157 reg=0.0155718 total=0.0235074 (4448.89 pos/s)
Model saved in file: ./networks/8x1/8x1-900
saved as './networks/8x1/8x1-900' 0.52M
Weights saved in file: ./networks/8x1/8x1-900
step 1000, lr=0.1 policy=7.51228 value=0.00712479 mse=0.000488646 reg=0.0153411 total=0.0224659 (4375.52 pos/s)
step 1000, policy=7.51152 value=0.00665396 policy accuracy=0.052918% value accuracy=99.7541% mse=0.000463149
Model saved in file: ./networks/8x1/8x1-1000
saved as './networks/8x1/8x1-1000' 0.52M
Weights saved in file: ./networks/8x1/8x1-1000
step 1100, lr=0.1 policy=7.51024 value=0.0067119 mse=0.000463098 reg=0.0151115 total=0.0218234 (1935.3 pos/s)
Model saved in file: ./networks/8x1/8x1-1100
saved as './networks/8x1/8x1-1100' 0.52M
Weights saved in file: ./networks/8x1/8x1-1100
step 1200, lr=0.1 policy=7.50786 value=0.00593836 mse=0.000407874 reg=0.0148866 total=0.020825 (4458.03 pos/s)
Model saved in file: ./networks/8x1/8x1-1200
saved as './networks/8x1/8x1-1200' 0.52M
Weights saved in file: ./networks/8x1/8x1-1200
step 1300, lr=0.1 policy=7.50629 value=0.00569232 mse=0.000394998 reg=0.0146605 total=0.0203529 (4459.09 pos/s)
Model saved in file: ./networks/8x1/8x1-1300
saved as './networks/8x1/8x1-1300' 0.52M
Weights saved in file: ./networks/8x1/8x1-1300
step 1400, lr=0.1 policy=7.50681 value=0.00622524 mse=0.000431534 reg=0.0144379 total=0.0206632 (4425.31 pos/s)
Model saved in file: ./networks/8x1/8x1-1400
saved as './networks/8x1/8x1-1400' 0.52M
Weights saved in file: ./networks/8x1/8x1-1400
step 1500, lr=0.1 policy=7.50732 value=0.00587613 mse=0.000410718 reg=0.0142196 total=0.0200958 (4486.12 pos/s)
Model saved in file: ./networks/8x1/8x1-1500
saved as './networks/8x1/8x1-1500' 0.52M
Weights saved in file: ./networks/8x1/8x1-1500
step 1600, lr=0.1 policy=7.50584 value=0.00593065 mse=0.000416239 reg=0.0140044 total=0.019935 (4473.79 pos/s)
Model saved in file: ./networks/8x1/8x1-1600
saved as './networks/8x1/8x1-1600' 0.52M
Weights saved in file: ./networks/8x1/8x1-1600
step 1700, lr=0.1 policy=7.50391 value=0.00513716 mse=0.000353341 reg=0.0137919 total=0.0189291 (4517.04 pos/s)
Model saved in file: ./networks/8x1/8x1-1700
saved as './networks/8x1/8x1-1700' 0.52M
Weights saved in file: ./networks/8x1/8x1-1700