[throwawayy2]

> could play Mario64 acceptably but pretty much no other N64 games

That's a problem with the emulator, not the hardware. I have a work-in-progress Mupen64Plus fork that can play all N64 games at full speed or better on the Pi 2 with only minor occasional slowdowns (that should all be fixable). The problem is that the emulator plugins were all designed for GL 1.x era fixed function hardware, which is very different from what you need to make software run well on tiled architectures like the VideoCore IV. It's not easy to get them running—I had to add an SSA-based optimizing shader compiler, for example—but it all ends up working out fine in the end.

I would like to try running N64 games on even cheaper hardware like the C.H.I.P. at some point. I think they have a good chance at running at full speed.

The biggest problem with the Raspberry Pi for this kind of stuff is that the software, which mostly consists of lightly optimized ports of old emulators written for Windows, holds the hardware back. If you compare instructions-per-clock of CPUs and fill rates of console GPUs from that era with the relevant hardware on even cheap modern mobile SoCs, the latter blow the former away. It's just that the games of that era pushed the hardware to its limits, while the emulators on the Pi don't even come close.