Jean/refacto benchmark

[jeandut]

This PR is aimed at making the benchmark logic both simpler and more robust.

[ghost]

Hi,

Thanks for starting the refactoring. Here are a few comments to go even further. I will not mark anything as compulsory given the time constraints, but it would be great to save what will not be done as an issue for future work.

1. Optimization: in fed_benchmark.py, in as much as possible, initialize dataloaders after checking if the experiment needs to be run or not
2. Cutting main in fed_benchmark.py into sub-functions to make it even more readable (it's > 370 lines long at the moment...). I give more actionable suggestions below.

• Split main into sub-functions pooled_training, local_training, strategy_training. That will make this main much more simpler (but +1 for the single_centric_training which is much better than before).
• The logic for the tests if the exps are finished or not should be hidden in sub-functions for readability.
  • replace all the len(index_of_interest) < (NUM_CLIENTS + 1) tests to check if experiment is finished by a more explicit function like if check_exp_finished(df, hyperparameters, sname, num_updates)
  • same for all the statements #dealing with edge cases that should not happen --> in sub-functions
• Lots of variables are assigned very short names, like training_dls for training_dataloaders, s for strategy, m for model... Better to use larger names
• the logic for filling the hyperparameters' param could also be delegated in sub-functions.

3. conf.py should be renamed into something more explicit, like configuration_utils.py

[jeandut]

Created an issue #154 so that this review is tracked but merging in the meantime.