Hacker News new | ask | show | jobs
by sho_hn 1018 days ago
> Btw have you tried some locally running models instead of GPT-4 for your automation?

Not yet, but the desire is there, especially because the generic GPT-4 API is quite slow in responding to more complex prompts, not to mention the privacy concerns. I think the next version of my home NAS is likely to have a GPU in it and will run things like an appropriately tuned llama2 or similar to be the brain backbone of my smart home. Feels like an obvious direction for commercial NAS to go in as well.

Can't wait for a future where we can buy a Rasberry Pi 7 with a little analog compute ASIC running local inference with ease ... intelligent controllers everywhere!

1 comments

It seems like coral.ai is a glimpse into this future you’re waiting for. I dont know exactly what types of AI it actually speeds up on an RPi but it seems promising…