[Feature Request] multi-agent tutorial

[RiqiangGao]

Motivation

Multi-agent RL is a hot topic, while tutorials are lacked here.

Solution

Could you please add a tutorial for multi-agent RL? Thanks

[vmoens]

Thanks @RiqiangGao
Will do ASAP! It's an active area of development in the lib, we definitely should work on the doc

cc @matteobettini (not that you need to do it, just FYI)

[RiqiangGao]

Just a FYI, some MARL packages like https://github.com/Farama-Foundation/PettingZoo seem very good, it would be great if they can be integrated into torchrl.

[matteobettini]

I would be happy to help you in making it.

I was mainly waiting to reshape a few library features (such as multi-agent collectors #828) and other stuff that might not be super smooth yet in multi-agent torchrl. (I still haven't looked at how losses and trainers behave in multi-agent settings).

[matteobettini]

e MARL packages like https://github.com/Farama-Foundation/PettingZoo seem very good, it would be great if they can be integrated into torchrl.

Wrapping PettingZoo would be cool and useful. I know that they are in the stage of making breaking changes to the API though Farama-Foundation/PettingZoo#803

[matteobettini]

#1027 Shows some examples of multiagent training in VMAS

[pseudo-rnd-thoughts]

Wrapping PettingZoo would be cool and useful. I know that they are in the stage of making breaking changes to the API though Farama-Foundation/PettingZoo#803

Hey, I just saw this. That PR was purely an experimental idea and we haven't made any change nor have any ideas to PZ currently.
There already exist several tutorials for PettingZoo (https://pettingzoo.farama.org/tutorials/cleanrl/implementing_PPO/) for any tutorial to build upon

[matteobettini]

We have a multi-agent tutorial online, check it out at https://pytorch.org/rl/tutorials/multiagent_ppo.html