GAIL tensorboard logging does not appear to work

[eufrizz]

Bug description

I'm following along with the GAIL example, and am trying to log to tensorboard. If I understand the API correctly, I should be able to log directly to tensorboard with the init_tensorboard and init_tensorboard_graph arguments (not entirely sure on the distinction, but have tried using both independently and together).
On training, it creates a *.tfevents.* file in the expected directory, but the file stays empty (i.e. 88 bytes in size) for the duration of the run, and nothing is logged to it. This is the how the GAIL is constructed:

gail_trainer = GAIL(
    demonstrations=rollouts,
    demo_batch_size=1024,
    gen_replay_buffer_capacity=512,
    n_disc_updates_per_round=8,
    venv=env,
    gen_algo=learner,
    reward_net=reward_net,
    log_dir='../runs',
    init_tensorboard=True,
    init_tensorboard_graph=True
)

If I use a custom logger, it works fine, so that's what I'm using for now.

mylogger = logger.configure('../runs', format_strs=["tensorboard"])
gail_trainer = GAIL(
    ...
    custom_logger=mylogger
)

Steps to reproduce

Follow the tutorial, and set the init_tensorboard and init_tensorboard_graph arguments true:
https://imitation.readthedocs.io/en/latest/algorithms/gail.html

Environment

• Operating system and version: macOS 13.6.6,
• Python version: 3.10.9, imitation v1.0.0 and sb3 v2.3.2 installed with conda and running in ipynb
• Output of pip freeze --all: