[BUG] game crashes in several steps

[ifdog]

🐛 Bug Report

Describe the bug
One of my saved games crash on a 6 or 7 num of steps.no matter how many times I reload.
the traceback is as followings.
update: may be related to ConnorJL/GPT2#4

    Traceback(most recent call last) :
      File "/home/user/AIDungeon/venv/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call
        return fn(*args)
      File "/home/user/AIDungeon/venv/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
        options, feed_dict, fetch_list, target_list, run_metadata)
      File "/home/user/AIDungeon/venv/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
        run_metadata)
    tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[0, 0] = 1024 is not in [0, 1024)
             [[{{node sample_sequence/while/model/GatherV2_1}}]]

    During handling of the above exception, another exception occurred:

    Traceback(most recent call last) :
      File "./play.py", line 367, in <module>
        play_aidungeon_2()
      File "./play.py", line 328, in play_aidungeon_2
        result = "\n" + story_manager.act(action)
      File "/home/user/AIDungeon/story/story_manager.py", line 215, in act
        result = self.generate_result(action_choice)
      File "/home/user/AIDungeon/story/story_manager.py", line 220, in generate_result
        block = self.generator.generate(self.story_context() + action)
      File "/home/user/AIDungeon/generator/gpt2/gpt2_generator.py", line 120, in generate
        text = self.generate_raw(prompt)
      File "/home/user/AIDungeon/generator/gpt2/gpt2_generator.py", line 103, in generate_raw
        self.context: [context_tokens for _ in range(self.batch_size)]
      File "/home/user/AIDungeon/venv/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 929, in run
        run_metadata_ptr)
      File "/home/user/AIDungeon/venv/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1152, in _run
        feed_dict_tensor, options, run_metadata)
      File "/home/user/AIDungeon/venv/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1328, in _do_run
        run_metadata)
      File "/home/user/AIDungeon/venv/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1348, in _do_call
        raise type(e)(node_def, op, message)
    tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[0, 0] = 1024 is not in [0, 1024)
             [[node sample_sequence/while/model/GatherV2_1(defined at / home / user / AIDungeon / generator / gpt2 / src / model.py:186)]]

    Caused by op 'sample_sequence/while/model/GatherV2_1', defined at:
      File "./play.py", line 367, in <module>
        play_aidungeon_2()
      File "./play.py", line 160, in play_aidungeon_2
        generator = GPT2Generator()
      File "/home/user/AIDungeon/generator/gpt2/gpt2_generator.py", line 52, in __init__
        top_p = top_p,
      File "/home/user/AIDungeon/generator/gpt2/src/sample.py", line 120, in sample_sequence
        back_prop = False,
      File "/home/user/AIDungeon/venv/lib/python3.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3556, in while_loop
        return_same_structure)
      File "/home/user/AIDungeon/venv/lib/python3.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3087, in BuildLoop
        pred, body, original_loop_vars, loop_vars, shape_invariants)
      File "/home/user/AIDungeon/venv/lib/python3.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3022, in _BuildLoop
        body_result = body(*packed_vars_for_body)
      File "/home/user/AIDungeon/venv/lib/python3.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3525, in <lambda>
        body = lambda i, lv: (i + 1, orig_body(*lv))
      File "/home/user/AIDungeon/generator/gpt2/src/sample.py", line 89, in body
        next_outputs = step(hparams, prev, past = past)
      File "/home/user/AIDungeon/generator/gpt2/src/sample.py", line 75, in step
        hparams = hparams, X = tokens, past = past, reuse = tf.AUTO_REUSE
      File "/home/user/AIDungeon/generator/gpt2/src/model.py", line 186, in model
        h = tf.gather(wte, X) + tf.gather(wpe, positions_for(X, past_length))
      File "/home/user/AIDungeon/venv/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py", line 180, in wrapper
        return target(*args, ** kwargs)
      File "/home/user/AIDungeon/venv/lib/python3.7/site-packages/tensorflow/python/ops/array_ops.py", line 3273, in gather
        return gen_array_ops.gather_v2(params, indices, axis, name=name)
      File "/home/user/AIDungeon/venv/lib/python3.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 3748, in gather_v2
        "GatherV2", params=params, indices=indices, axis=axis, name=name)
      File "/home/user/AIDungeon/venv/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
        op_def = op_def)
      File "/home/user/AIDungeon/venv/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
        return func(*args, ** kwargs)
      File "/home/user/AIDungeon/venv/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3300, in create_op
        op_def = op_def)
      File "/home/user/AIDungeon/venv/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 1801, in __init__
        self._traceback = tf_stack.extract_stack()

    InvalidArgumentError (see above for traceback): indices[0, 0] = 1024 is not in [0, 1024)
             [[node sample_sequence/while/model/GatherV2_1(defined at / home / user / AIDungeon / generator / gpt2 / src / model.py:186)]]

    Saving to storage has been disabled due to abuse of the cloud bucket.Save will now be stored locally.
    Game saved.
    To load the game, type 'load' and enter the following ID: d42bc604-2a97-11ea-8844-2a346a863764
    (venv) user@debian:~/AIDungeon$

[Goodblue77]

I'm on windows and I have the same issue.

[ifdog]

limit the story.memory smaller or start a new game seems solve the issue.

[Goodblue77]

Where can I find story.memory?

[LumiWasTaken]

@Goodblue77 Hi,

the story.memory is the "self.memory" in the File story_manager.py (Line 34) i dont know if setting it to a lower Value will help. (The file itself is in the Folder story)

May i ask which Hardware you are using? i can see you are on a Linux Debian machine.

[ifdog]

I run it on cpu with two xeon X5675 and 20gb of memory.

[LumiWasTaken]

Have you tried updating to the lastest version?

[ifdog]

not yet.I will try the latest version and give a response once I have time.

[LumiWasTaken]

@ifdog did it worked?

[ifdog]

@ifdog did it worked?

No. didn't work.