First, that was my first project with micro controller and my experience with Teensy has been awesome. You did some amazing work with it.

So about the video player itself. There is no actual video de-compression. The video file is just a list of frame with the data I have to send to the screen. Because of the screen specs I just don't have to send much data, especially since we can refresh only specific lines of the screen. For some frames I can just update a few lines only. It's up to the video converter to decide which line to update.
Most of the optimization I had to do was mainly how to send data to the screen so I had to rewrite a lot of the ADA fruit library. The CPU is actually spending most of its time sending data to the screen. I didn't look into SPI and DMA.

The tool I have to make the video file is also very much ad-hoc. I wrote it in lua because I am very familiar with it and used Dear Imgui for the UI. The goal was mainly to quickly experiment with dithering, tweaking values. I can compare side by side two settings but I can also change values on the fly so a video can start with one dithering type and switch in the middle to a different one.
The video converter filters out lines that were identical to the previous frame. I was planning to have a threshold to "compress" even more but that was not needed so far because a lot of lines were already duplicates.

Original thread about the video player: https://twitter.com/NicMagnier/status/1195471551379976192
Thread about the dithering used for the video: https://twitter.com/NicMagnier/status/1178351976263888899