[caeril]

Any particular reason not to run this model on a single Jetson AGX Orin 64GB?

GP Core count is much lower than than the 4090 but it still does 275 int8 TOPS for only $2k

[Zemtomo]

I'm the wrong person to ask but performance wise a 4090 has over a pflop and a Google search also showed a factor of 3-4 for a 3090.

[cjbprime]

The relevant performance metric here is memory amount and bandwidth.