WARNING: Logging before flag parsing goes to stderr.
W0616 23:33:41.234486 140476528236416 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/gpt_2_simple/src/memory_saving_gradients.py:13: The name tf.GraphKeys is deprecated. Please use tf.compat.v1.GraphKeys instead.

Fetching checkpoint: 1.05Mit [00:00, 224Mit/s]
Fetching encoder.json: 1.05Mit [00:00, 85.9Mit/s]
Fetching hparams.json: 1.05Mit [00:00, 446Mit/s]
Fetching model.ckpt.data-00000-of-00001: 498Mit [00:03, 156Mit/s]
Fetching model.ckpt.index: 1.05Mit [00:00, 229Mit/s]
Fetching model.ckpt.meta: 1.05Mit [00:00, 136Mit/s]
Fetching vocab.bpe: 1.05Mit [00:00, 127Mit/s]
W0616 23:33:44.977926 140476528236416 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/gpt_2_simple/gpt_2.py:90: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

W0616 23:33:44.979698 140476528236416 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/gpt_2_simple/gpt_2.py:100: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

W0616 23:33:45.380354 140476528236416 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/gpt_2_simple/gpt_2.py:164: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

W0616 23:33:45.388669 140476528236416 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/gpt_2_simple/src/model.py:148: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

W0616 23:33:51.847121 140476528236416 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/gpt_2_simple/src/sample.py:71: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
W0616 23:33:51.868443 140476528236416 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/gpt_2_simple/src/sample.py:17: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
W0616 23:33:51.872190 140476528236416 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/gpt_2_simple/src/sample.py:77: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.random.categorical` instead.
W0616 23:33:51.887708 140476528236416 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/gpt_2_simple/gpt_2.py:184: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

W0616 23:33:51.888915 140476528236416 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/gpt_2_simple/src/accumulate.py:14: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
W0616 23:34:01.517626 140476528236416 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/gpt_2_simple/gpt_2.py:189: The name tf.summary.scalar is deprecated. Please use tf.compat.v1.summary.scalar instead.

W0616 23:34:01.520670 140476528236416 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/gpt_2_simple/gpt_2.py:200: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.

W0616 23:34:01.527817 140476528236416 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/gpt_2_simple/gpt_2.py:202: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

W0616 23:34:10.554982 140476528236416 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.

Loading checkpoint models/117M/model.ckpt

0it [00:00, ?it/s]

Loading dataset...
dataset has 0 tokens
Training...


---------------------------------------------------------------------------

ZeroDivisionError                         Traceback (most recent call last)

<ipython-input-4-f94d6c716ce5> in <module>()
      4
      5 sess = gpt2.start_tf_sess()
----> 6 gpt2.finetune(sess, 'shakespeare.txt', steps=1000)   # steps is max number of training steps
      7
      8 gpt2.generate(sess)

3 frames

/usr/local/lib/python3.6/dist-packages/gpt_2_simple/src/load_dataset.py in sample(self, length)
     81     def sample(self, length):
     82         assert length < self.total_size // len(
---> 83             self.chunks
     84         ), "Dataset files are too small to sample {} tokens at a time".format(
     85             length)

ZeroDivisionError: integer division or modulo by zero
_________________________________________________________________________________________________________
Fetching checkpoint: 1.05Mit [00:00, 258Mit/s]
Fetching encoder.json: 1.05Mit [00:00, 85.0Mit/s]
Fetching hparams.json: 1.05Mit [00:00, 320Mit/s]
Fetching model.ckpt.data-00000-of-00001: 498Mit [00:02, 193Mit/s]
Fetching model.ckpt.index: 1.05Mit [00:00, 173Mit/s]
Fetching model.ckpt.meta: 1.05Mit [00:00, 113Mit/s]
Fetching vocab.bpe: 1.05Mit [00:00, 126Mit/s]

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

<ipython-input-4-f3f48ea7636f> in <module>()
      8               'shakespeare.txt',
      9               model_name=model_name,
---> 10               steps=1000)   # steps is max number of training steps
     11
     12 gpt2.generate(sess)

6 frames

/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/variable_scope.py in _get_single_variable(self, name, shape, dtype, initializer, regularizer, partition_info, reuse, trainable, collections, caching_device, validate_shape, use_resource, constraint, synchronization, aggregation)
    862         tb = [x for x in tb if "tensorflow/python" not in x[0]][:5]
    863         raise ValueError("%s Originally defined at:\n\n%s" %
--> 864                          (err_msg, "".join(traceback.format_list(tb))))
    865       found_var = self._vars[name]
    866       if not shape.is_compatible_with(found_var.get_shape()):

ValueError: Variable model/wpe already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope? Originally defined at:

  File "/usr/local/lib/python3.6/dist-packages/gpt_2_simple/src/model.py", line 153, in model
    initializer=tf.random_normal_initializer(stddev=0.01))
  File "/usr/local/lib/python3.6/dist-packages/gpt_2_simple/gpt_2.py", line 165, in finetune
    output = model.model(hparams=hparams, X=context)
  File "<ipython-input-2-f3f48ea7636f>", line 10, in <module>
    steps=1000)   # steps is max number of training steps
  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2882, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2822, in run_ast_nodes
    if self.run_code(code, result):