[woojoo666]

> This means the parameters of the trained model fit in something like 7GB (decoder only, half-precision floats) to 24GB (full model, full-precision)

> you would probably want an enterprise cloud/data-center GPU like an NVIDIA A100, especially if running batches of more than one image.

That doesn't seem so bad.

looks up price of NVIDIA A100 - $20,000

oh...ok I'll probably just pay for the service then

[nacs]

I know you're half joking here but there are more consumer-affordable versions like the Geforce RTX 3090ti ($1600 for 24GB). It may not do CUDA work as fast as the A100 but it'll be able to run the model.

For the half-precision version at 7GB there are a ton more options (the RTX 3060 has 12GB for example at ~$450).

[fennecfoxen]

p4d.24xlarge is only $33/hr! And you get 400 Gbe so it should be quick to load.