Yeah I have been reading a lot of posts like this lately. Technical blog post clearly written by an LLM summarizing something vibe-coded. They always start using project-specific jargon right away and they never give you enough context or backstory to understand why this thing exists. It's seems very clearly to be a symptom of someone pointing an LLM at a repo and telling it "write a github page for this project".
It really shines through in pieces like this that LLM's have a severely constrained worldview and underdeveloped theory of mind. They can't imagine that a line like "A 200-line POC that goes from 0/5 to 5/5 in four proposer steps" means nothing to me as a subtitle for the page. After all "proposer steps" and "5/5" are *right there* in it's context. Surely everyone has "proposer steps" in their context, right?
I have this problem with other people all the time. They can’t fathom why someone else wouldn’t have their exact context at any given moment. They say some non-sequitur and are immediately incredulous that I’m asking wtf they’re talking about.
I find it fascinating, all these attempts are goldmining LLMs with a harness and it's clear they're generating all the docs for AI to read and use, even the docs say "we made a MCP for this!" like some how within 2 years people no longer make choices and it's just like AIs roaming the internet trying on harnesses, etc; certainly that'd be a fascinating reality but the verbosity really is a eye-glazing experience. Who do they expect to read all of that ad copy? It's not me.
Their lines "A meta-harness is the loop that improves the harness automatically" and "the bottleneck is diagnostic context: most optimizers compress prior runs into summary statistics, while meta-harness gives the proposer up to 10M tokens of raw execution traces to grep through," seem good, no?
Have to dig into the code, but it looks like they have sound engineering around a "self-improving" agentic coding harness. Will be fun to take the code for a spin.
10M tokens of raw execution traces to grep through is slop. The tasks are fizzbuzz, palindrome, list reversal, and sum-even. The palindrome challenge is literaly this:
> Is the word "racecar" a palindrome? Answer with exactly one lowercase word: "yes" or "no". Print only the answer.
It really shines through in pieces like this that LLM's have a severely constrained worldview and underdeveloped theory of mind. They can't imagine that a line like "A 200-line POC that goes from 0/5 to 5/5 in four proposer steps" means nothing to me as a subtitle for the page. After all "proposer steps" and "5/5" are *right there* in it's context. Surely everyone has "proposer steps" in their context, right?