| A medium-spicy take of mine is that a bytecode VM in a GPU kernel is not as bad of an idea as one might think, and in some cases it can actually be the most reasonable solution. Some fun examples: 1. As mentioned in the post above, the Dolphin emulator famously implements the entire Gamecube/Wii GPU pipeline in a single gigantic ubershader, and this is useful because it avoids shader compilation stalls [1]. 2. Blender's Cycles renderer implements its shading graph eval system as a bytecode VM in a GPU kernel [2]. IIRC early versions of Vray GPU did something similar. There are better ways of course, but a VM gets you surprisingly far as a general approach. 3. Finally, a lot of ML frameworks (Tensorflow, PyTorch, etc) by default use the GPU relatively suboptimally (especially without kernel fusion and such). Tensor frameworks can extract a lot more perf out of GPUs using a VM-in-a-giant-kernel approach [3]. If you think abstractly about how a GPU SM actually works (using CUDA terminology here), all threads in a warp must execute in lockstep and the cost of execution divergence across threads in a warp is that you effectively run serially, losing the parallel advantage of the SM. This penalty gets magnified enormously if you are doing memory reads after wherever the execution divergence happens, since you now have multiple slow memory stalls in serial instead of one big memory read at once for all threads. If you're clever about implementing a bytecode VM, you can load as much state as you need upfront into shared memory, and then if your bytecode VM is just looping through executing a bunch of opcodes in a huge switch statement, then at least as far as the SM is concerned, there's no execution divergence! All threads look like they're doing the same thing at the same time; even if within the VM what is happening a lot is just no-ops, at the SM level you're not dealing with serialized memory stalls and serial scheduling and such. Is it the _best_ most optimal approach imaginable? Almost certainly not! But can it be a _surprisingly good_ and possibly even reasonable approach for some problem domains and specific constraints? Yeah absolutely! [1] https://dolphin-emu.org/blog/2017/07/30/ubershaders/
[2] https://www.youtube.com/watch?v=etGMk9wYwNs&t=1882s
[3] https://hazyresearch.stanford.edu/blog/2025-09-28-tp-llama-m... |