[benjreinhart]

Ahh yes, right now we're operating at a higher-level of the stack.

That said, we are investigating serving from Node and possibly on edge devices with WebGPU. For serving from Node, it would be similar to what you describe with Tensorflow compiling down to C/C++. There are various backends for frameworks like Tensorflow, Pytorch, etc. and those backends are often C/C++. We would bridge this lower-level code to Node through e.g. Node API (https://nodejs.org/api/n-api.html) or use frameworks like ONNX / ONNX Runtime.