I wrote the following reinforcement learning code in PyTorch, TorchRL. It specifies an option to obtain pixel-based environment information in the env variable, where tensordict_data contains the pixel data (96x96x3) of the environment. Here is the Google Colab links.