Fantastic Work!

[slerman12]

Wow, this repository is fantastic! I am learning how to implement RL algorithms and the structure of this repository is really intuitive to me. Thank you so much for making and sharing this!

The thing that intimidates me most about implementing today's RL algorithms is that I don't know which implementations are best to learn from. The ones here have very nice structure, so I am wondering, how would you say they compare to other implementations like OpenAI Baselines? Would these perform as well? Are there any important features still needed for scaling up to harder environments like StarCraft?

I'm also wondering why certain choices were made, like using asynchronous IMPALA over synchronous? Would it be hard to modify this to be synchronous?