logs not getting written when multiprocessing_context is spawn or forkserver

[krehm]

I just opened PR #130 to fix dlio.log so that it gets reopened in spawn and forkserver child
processes so that the child log messages are not lost.

The same problem exists with dlp.log, but some of the code that needs to change is in repository
dlio-profiler. Once that is updated and its release number is bumped, then changes can
be made in dlio_benchmark to use the newer dlio-profiler version.

[krehm]

Thanks for the merge of PR #130. The dlp.log has the same problem, but is more complicated because of the ENTER and EXIT code that is use to implement the:
with Profile(name=f"{self.init.qualname}", cat=MODULE_DLIO_BENCHMARK):
code. That code makes sense in main.py where it wraps the body of the benchmark, but inside a forked/spawned child process in TorchDataset.worker_init() I don't see how to re-initialize the dlp.log without using ENTER and EXIT. Any advice would be appreciated.

[zhenghh04]

@krehm This has been fixed by @hariharan-devarajan, the corresponding PR is merged. Could you test it again whether it is working?

[krehm]

I will be out of the office until Tuesday, but will give it a try then.

[krehm]

I have not had time yet to do full testing, I am chasing another problem at the moment. But what I do notice is that when I do a MLPerf run (using 'main' branch) that it sets workflow.profiling=False, yet when I run unet3d with 'spawn' multiprocessing_context, I see messages that imply that profiling is running. So maybe child processes, at least in spawn mode, are ignoring args.do_profiling.