[syspec]

Wouldn't that much VRAM just be needed for training the model? Not inference?

[calum-bird]

Haha I wish! That's just loading the weights for inference :)

[syspec]

I sense a humblebrag

[calum-bird]

Haha indeed, NLP is fun!