[springtimesun]

I have been doing this with opencode. Spend a fair bit of time composing and scoping a ticket. tmux on the server where the local inference lives, start a non-interactive session that starts with “read issue #123, do all your work in a feature branch” and ends with “push and open a PR”. Like not watching someone else load your dishwasher, you appreciate the end result a lot more if you don’t watch the process, even if you rearrange things before you start it.

[cube00]

Local inference can't arrive soon enough. On my inferior hardware the models still struggle to build the correct JSON to be able to call tools. Once either I get better hardware or the local models improve I'll happily leave it chugging away for a few hours to arrive at a solution because I know I'm not going to get token shock from all the looping if I'm running local inference.

[springtimesun]

I have spent way too much on hardware. I’m sure I will look back on that after the eventual price normalization with some regret. I do think we will look back at this shortage and name it as the thing that accelerated innovation at getting quality out of smaller models.

Hang tough. The other day someone posted a solution to get a model running off of SSDs. It won’t be fast, but it will be coming.