[DO NOT CLOSE] Library TODOs and call for contributions

[matteobettini]

Hello people!

In this issue I will list the things I would really like to have in BenchMARL and will tick them off as they are implemented!

It is also a really good place to find something you would like to contribute.

Features

• Parameter sharing between actors and critics ([Feature] Share parameters between models #95)
• support for variable number of agents (dead agents)
• support for turn-based environments (Request for Example of AEC API Usage with Agent Masking in Petting Zoo #76)
• Parallel collection for non-vectorized environments
• Improve checkpointing (just keep last checkpoints) ([Feature] keep_checkpoints_num and checkpoint_at_end #102)
• Prioritized replay buffers

Models

• GRU model ([Model] GRU and general RNN compatibility #116)
• LSTM model (LSTM in benchmarl #103)([Model] LSTM #120)
• DeepSets model ([Model] DeepSets #96)

Algorithms

• MPO and V-MPO
• HARL algorithms (Suggestion of integrating HARL algorithms #52)

Environments

• Unity ML-Agents ([Feature] Add support for Unity MLAgents environments pytorch/rl#2161)
• MAgent2 (MAgent2 #137 )