Hacker News new | ask | show | jobs
by jasonthorsness 462 days ago
Don't the resource requirements from even small LLMs exclude most devices/users from being able to use stuff like this?
1 comments

True, but there are likely innovations happening in multiple dimensions all at once: WebGPU improvements that better utilize a device's compute, Wasm64. And of course, LLMs over time become SLMs (smaller and smaller models), that can do a surprisingly large variety of things well.

Putting aside LLMs for a minute, even applications that do not need LLMs, but benefit from a graph database, can be unlocked to help build interactive UIs and visualizations that retain privacy on the client side without ever moving the data to a server. Loads of possibilities!