[ericfontaine]

I was wondering if maybe they preconverted the videos into a sequence of tilemaps and a set of approximate tiles. I notice they are only using the same 5 colors, which is under-utilizing the capabilites of the nes which should be able to display a 13 colors at the same time. (Each individual tile uses one palette of 4 colors, and there can be max of 4 palette used at a time, but one background color is shared by all palettes.) To more fully utilize the nes they could also use a different set of 4 palettes for every frame, which would allow for a much broader range of colors.

Hypothetically, the NES could play streaming video. For memeory limitations, a custom rom cartridge could be used which would grab a new set of video frames (consisting of tilemaps + tilepatterns) via a network controller. The 6502 core does not need to decode video, but could simply swap in chunks of these tilemaps+tilepatterns which the picture processing unit displays.

[memoryhole]

This is exactly what we did. The video frames were converted to tilesets and stored in the rom image. For playback, the memory mapper (MMC3) is used to swap between the frames without having to rely on too much CPU. Luckily Guy figured out the MMC3, otherwise we'd have only had enough space for a few frames.

Correct as well about the colors. It would have been nice to better utilize them, but we didn't have enough time to do that properly.

[ericfontaine]

great hack btw! I'm sure you had limited time to waste!

[gcirino42]

Thanks! Alex had been studying getting better compression by repacking the tiles based on a scheme I still haven't quite figured out, but we were getting CPU --> PPU bound in what we could do during vblank. The MMC3 solution let us basically dma our way around the problems and push a much larger screen update far more often. The original version ripped through playback so fast that we had to slow it down to get a decent number of frames into the available 32 CHR ROM banks (it's a mapper 4 NES rom).

The homebrew community was a fantastic resource! Shoutouts to nesdev, Shiru, and the countless articles we googled!

[memoryhole]

Thanks!

[chongli]

You can get even more than 13 simultaneous colours by writing to the palette mid-scanline. With this method it is possible to display all 410 available colours in one frame.

[ericfontaine]

Neat. I was also wondering if maybe it was possible to switch between two colors based on a certain ratio of frames, faster than the CRT phosphors to fully switch colors (or atleast faster than the eye could detect the swapping), such that it looks like a fractional blend between two colors based on that ratio. I remember doing this trick with the TI-85 calculator to give any degree of greyscale even though the LCD was only black and white.