Hacker News new | ask | show | jobs
by rosenlykke 56 days ago
In what way is it getting worse — the model's reasoning, or your own setup drifting underneath you?

My observation: same model, same task, but different CLAUDE.md / hooks / skill state produces dramatically different outputs.

The hard part as a solo founder is finding the right balance between building the meta tools and making progress on the actual projects, without the meta layer drifting and quietly becoming worse over time.
2 comments
tstrimple 55 days ago
My issue was with different models. Same Claude.md, hooks skills and the rest. Same task. Almost two months ago I used CC to bootstrap a headless MacBook Air with nix-darwin for management. I configured it to be an iMessage bridge as well as secondary dns for my network. All in it took about an hour.

Last night same task for the same context but newer opus model. This time a freshly installed MBP that I wanted nix-darwin setup on to keep my tools / console config in sync across systems. To start with, it was trying to install some proprietary nix version and couldn’t fix a broken ssh terminal issue at all despite having a working example literally sitting right next to it with my working mba config.

Latest version of CC feels lobotomized.

link
rosenlykke 55 days ago
Fair point. I don't see it consistently myself, but there are def moments where it just stops parsing context on simple tasks/stops thinking. Not sure if it's model regression, context rot, or just prompts getting routed to a different/wrong expert in the MoE for a stretch — dunno.
link
tstrimple 54 days ago
I've got an example that isn't just coding. I've used CC as a book recommendation engine for a few months now. Initially I had really good results. We built a SQLite database of my reading history from audible and kindle libraries and tuned it based off of my "reviews". Since Opus 4.7 it suddenly lost the ability to check the SQLite database for things I've already read before offering recommendations. The last time I interacted with it, I told it I had completed Book 2 of a series and it recommended Book 1 of the series to me. I had never previously seen Opus be so stupid with all the information available to it. Nothing changed in my configuration between when this was a useful tool and complete garbage other than the model and harness upgrades Anthropic pushed.
link
cleverhoods 56 days ago
I second to that.

with some small caveat: reporails can help with the drifting detection a lot.

link