[skinfaxi]

How does your solution differ from OPs?

edit:

To me it looks like yours has the following negative qualities:

- closed source

- requires signup for some service

- says code doesnt leave my machine but this connects to your servers

- requires Goose AI as a dependency

[trjordan]

So, we tried feeding the logs back to the LLM, and it mostly produced slop. Lots of decisions nobody cared about. The biggest things that moved the needle were:

- Baseline it. We mine previous logs, github comments, etc. for "what you care about." That helps pull out decisions that you actually care to read.

- Anchor to code. "The code enshrines this decision" is more interesting than "the agent self-talked this." Agents don't always self-talk decisions, and the thing that ultimately matters is the behavior in code.

to your edit (and all totally fair):

- Yes, closed source and signup required. A lot of what we're driving towards is easy team sharing, so we're taking the bath early instead of building an OSS thing and rug-pulling later.

- Code doesn't leave your machine. There's an agent that runs locally. I know "trust me" isn't the strongest stance, but this comes from the multi-player future.

- Honestly, Goose AI is a remnant of a previous product. It's inert and we'll clean it up once we've gotten the last couple folks off the previous iteration.

[evansjp]

hey that's awesome! yeah the eval showed first pass was only ~65% real decisions. The fix that stuck was an entry has to name a real file it touches or it gets dropped. A code decision names code.

I agree agents don't always self-talk decisions, that's why we distill the whole transcript after the fact instead of asking them to log anything. Your baselining idea is good!