[datsci_est_2015]

At what point will local inference catch up to today’s cloud inference? Will it ever? If it doesn’t, does that imply a certain dead-end for the LLM inference industry?

[lukaslalinsky]

I don't think at any point in foreseeable future we will have terabytes of RAM for dedicated LLM chips at home.