Should we use torch.compile?

[MischaPanch]

We should analyze whether using the newest pytorch acceleration tricks gives performance boosts. At least for some algos on atari, and for some on mujoco.

If it is useful, we should by default user torch.compile for all our networks. Maybe it even makes sense to do that without much testing, as I suppose that torch.compile won't have negative effects. It would be interesting to know the amount of speedup though.

Tests have to be executed on a GPU, maybe colab is sufficient.

[MischaPanch]

@arnaujc91 wanna have a look?

[arnaujc91]

Yes, can you assign it to me?