INFO:tensorflow:Start a new run and write summaries and checkpoints to gs://dev01-181118-181500-k8s/jobs/tensorflow-20171116094619/20171116T175526-pybullet_ant.
INFO:tensorflow:Not normalizing infinite observation range.
INFO:tensorflow:Not normalizing infinite observation range.
INFO:tensorflow:Not normalizing infinite observation range.
INFO:tensorflow:Not normalizing infinite observation range.
INFO:tensorflow:Not normalizing infinite observation range.
INFO:tensorflow:Not normalizing infinite observation range.
INFO:tensorflow:Not normalizing infinite observation range.
INFO:tensorflow:Not normalizing infinite observation range.
INFO:tensorflow:Not normalizing infinite observation range.
INFO:tensorflow:Not normalizing infinite observation range.
INFO:tensorflow:Graph contains 52717 trainable variables.
2017-11-16 17:55:54.427514: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-11-16 17:55:54.427556: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-11-16 17:55:54.427583: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
INFO:tensorflow:
--------------------------------------------------
Phase train (phase step 0, global step 0).
2017-11-16 17:56:01.453126: I tensorflow/core/kernels/logging_ops.cc:79] return and value: [12.240352][1.2100527]
2017-11-16 17:56:01.453377: I tensorflow/core/kernels/logging_ops.cc:79] normalized advantage: [-3.2393773e-06]
2017-11-16 17:56:06.279336: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:06.472229: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:06.612141: I tensorflow/core/kernels/logging_ops.cc:79] value loss: [1186.2911]
2017-11-16 17:56:06.612204: I tensorflow/core/kernels/logging_ops.cc:79] policy loss: [-0.24434008]
2017-11-16 17:56:06.612537: I tensorflow/core/kernels/logging_ops.cc:79] current penalty: [1]
2017-11-16 17:56:06.648651: I tensorflow/core/kernels/logging_ops.cc:79] kl change: [0.010649167]
2017-11-16 17:56:10.761221: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:10.764925: I tensorflow/core/kernels/logging_ops.cc:79] return and value: [19.587742][2.2418118]
2017-11-16 17:56:10.765263: I tensorflow/core/kernels/logging_ops.cc:79] normalized advantage: [6.4582191e-06]
2017-11-16 17:56:10.972183: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:11.183299: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:11.393940: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:11.617942: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:11.829309: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:12.049379: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:12.262963: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:12.469101: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:12.686628: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:12.904362: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:13.138226: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:13.422740: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:13.677663: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:13.910656: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:14.156983: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:14.402928: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [2]
2017-11-16 17:56:14.653696: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [2]
2017-11-16 17:56:14.889332: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [2]
2017-11-16 17:56:15.115552: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [2]
2017-11-16 17:56:15.327416: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [2]
2017-11-16 17:56:15.546494: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [2]
2017-11-16 17:56:15.782017: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [3]
2017-11-16 17:56:15.980239: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [3]
2017-11-16 17:56:16.184762: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [3]
2017-11-16 17:56:16.324444: I tensorflow/core/kernels/logging_ops.cc:79] value loss: [2233.5532]
2017-11-16 17:56:16.324447: I tensorflow/core/kernels/logging_ops.cc:79] policy loss: [-0.2700505]
2017-11-16 17:56:16.324878: I tensorflow/core/kernels/logging_ops.cc:79] current penalty: [1]
2017-11-16 17:56:16.358771: I tensorflow/core/kernels/logging_ops.cc:79] kl change: [0.0051352326]
2017-11-16 17:56:16.358852: I tensorflow/core/kernels/logging_ops.cc:79] decrease penalty [0]
2017-11-16 17:56:24.217513: I tensorflow/core/kernels/logging_ops.cc:79] return and value: [19.195578][3.2919481]
2017-11-16 17:56:24.217789: I tensorflow/core/kernels/logging_ops.cc:79] normalized advantage: [2.7217866e-06]
2017-11-16 17:56:29.168300: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [2]
2017-11-16 17:56:29.377434: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [2]
2017-11-16 17:56:29.515365: I tensorflow/core/kernels/logging_ops.cc:79] policy loss: [-0.26654032]
2017-11-16 17:56:29.515365: I tensorflow/core/kernels/logging_ops.cc:79] value loss: [2310.03]
2017-11-16 17:56:29.515733: I tensorflow/core/kernels/logging_ops.cc:79] current penalty: [0.66666669]
2017-11-16 17:56:29.549798: I tensorflow/core/kernels/logging_ops.cc:79] kl change: [0.010192649]
2017-11-16 17:56:42.045495: I tensorflow/core/kernels/logging_ops.cc:79] return and value: [42.0045][4.1054816]
2017-11-16 17:56:42.045626: I tensorflow/core/kernels/logging_ops.cc:79] normalized advantage: [9.2924756e-06]
2017-11-16 17:56:42.046864: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [2]
2017-11-16 17:56:42.256851: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [2]
2017-11-16 17:56:42.468273: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [2]
2017-11-16 17:56:42.682804: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [2]
2017-11-16 17:56:42.896083: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [2]
2017-11-16 17:56:43.107496: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:43.328253: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:43.550835: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:44.835364: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:45.053830: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:45.265551: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:45.473891: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:45.695920: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:45.917872: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:46.131823: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:46.343514: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:46.557672: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:46.775277: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:56:46.990251: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [2]
2017-11-16 17:56:47.192517: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [2]
2017-11-16 17:56:47.333151: I tensorflow/core/kernels/logging_ops.cc:79] policy loss: [-0.31894824]
2017-11-16 17:56:47.333151: I tensorflow/core/kernels/logging_ops.cc:79] value loss: [6625.7163]
2017-11-16 17:56:47.333423: I tensorflow/core/kernels/logging_ops.cc:79] current penalty: [0.66666669]
2017-11-16 17:56:47.367461: I tensorflow/core/kernels/logging_ops.cc:79] kl change: [0.0063542118]
2017-11-16 17:56:47.367551: I tensorflow/core/kernels/logging_ops.cc:79] decrease penalty [0]
2017-11-16 17:57:12.283395: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:57:12.286885: I tensorflow/core/kernels/logging_ops.cc:79] return and value: [76.946365][5.6052241]
2017-11-16 17:57:12.286958: I tensorflow/core/kernels/logging_ops.cc:79] normalized advantage: [3.0705769e-06]
2017-11-16 17:57:12.492056: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:57:12.706219: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:57:12.921113: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:57:13.125978: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:57:14.422103: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:57:14.630801: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:57:14.844719: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:57:15.077601: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:57:15.702902: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:57:15.909592: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:57:16.130576: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:57:16.350747: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [2]
2017-11-16 17:57:16.562118: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [2]
2017-11-16 17:57:16.770154: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [2]
2017-11-16 17:57:16.983308: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [2]
2017-11-16 17:57:17.217447: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:57:17.431475: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:57:17.544678: I tensorflow/core/kernels/logging_ops.cc:79] value loss: [14296.028]
2017-11-16 17:57:17.544678: I tensorflow/core/kernels/logging_ops.cc:79] policy loss: [-0.37481374]
2017-11-16 17:57:17.545108: I tensorflow/core/kernels/logging_ops.cc:79] current penalty: [0.44444445]
2017-11-16 17:57:17.578470: I tensorflow/core/kernels/logging_ops.cc:79] kl change: [0.0045972471]
2017-11-16 17:57:17.578596: I tensorflow/core/kernels/logging_ops.cc:79] decrease penalty [0]
INFO:tensorflow:
--------------------------------------------------
Phase eval (phase step 0, global step 30000).
INFO:tensorflow:
--------------------------------------------------
Phase train (phase step 30000, global step 55000).
2017-11-16 17:59:12.190953: I tensorflow/core/kernels/logging_ops.cc:79] return and value: [62.395721][7.3883162]
2017-11-16 17:59:12.191040: I tensorflow/core/kernels/logging_ops.cc:79] normalized advantage: [-3.1905493e-06]
2017-11-16 17:59:14.117276: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:59:14.333864: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:59:14.550089: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:59:14.761701: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:59:15.814048: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:59:16.037234: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:59:16.248372: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:59:16.458258: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [1]
2017-11-16 17:59:16.672340: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [3]
2017-11-16 17:59:16.885712: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [4]
2017-11-16 17:59:17.094937: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [3]
2017-11-16 17:59:17.310067: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [3]
2017-11-16 17:59:17.443878: I tensorflow/core/kernels/logging_ops.cc:79] value loss: [10228.255]
2017-11-16 17:59:17.443984: I tensorflow/core/kernels/logging_ops.cc:79] policy loss: [-0.366169]
2017-11-16 17:59:17.444125: I tensorflow/core/kernels/logging_ops.cc:79] current penalty: [0.2962963]
2017-11-16 17:59:17.477554: I tensorflow/core/kernels/logging_ops.cc:79] kl change: [0.0086536584]
2017-11-16 18:00:29.393816: I tensorflow/core/kernels/logging_ops.cc:79] return and value: [192.44949][8.8605843]
2017-11-16 18:00:29.393908: I tensorflow/core/kernels/logging_ops.cc:79] normalized advantage: [5.9757235e-07]
2017-11-16 18:00:29.398042: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [3]
2017-11-16 18:00:29.617240: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [3]
2017-11-16 18:00:29.840289: I tensorflow/core/kernels/logging_ops.cc:79] kl cutoff! [2]
2017-11-16 18:00:34.668360: I tensorflow/core/kernels/logging_ops.cc:79] value loss: [42459.285]
2017-11-16 18:00:34.668364: I tensorflow/core/kernels/logging_ops.cc:79] policy loss: [-0.43168113]
2017-11-16 18:00:34.668686: I tensorflow/core/kernels/logging_ops.cc:79] current penalty: [0.2962963]
2017-11-16 18:00:34.701958: I tensorflow/core/kernels/logging_ops.cc:79] kl change: [0.00825583]
INFO:tensorflow:
--------------------------------------------------
Phase eval (phase step 25000, global step 85000).
INFO:tensorflow:
--------------------------------------------------
Phase train (phase step 60000, global step 110000).