[amelius]

No it does not assume that. Some very smart people will write that model.toVHDL() function. And keep in mind that a DL model is only a very small subset of what you can use VHDL for, and most models will have a very similar implementation in hardware from a conceptual point of view.

And don't take it too literally, VHDL could be replaced by other hardware design languages, maybe even at lower abstraction levels.

[knollimar]

Not trying to take it literally, but aren't there costs vs performance tradeoffs? Like the py.toHDL would have like (maxSize,maxCost,minThroughput) as free and that would determine energy usage?

And a GPU is already pretty optimized for inference, no? Like isn't it a bunch of FP mults? I don't think HDLs do well with that, either.