[qsort]

I am obviously paraphrasing, but the general idea is that trying to synthesize style from a codebase into e.g. a markdown guide generally doesn't work very well. What achieves style transfer is providing the model with a lot of examples of the style, conventions, patterns you want.

To put it in practice: if you point claude/codex to a repository and you ask it to implement feature X using style guide Y, the code will probably work, but you can usually get better results by saying "do it in the style of this file, it was done well there".

[brandensilva]

Right more simply put it's great at being a copy cat, exploring similar data points that match your token needs.

It is not great at decision making or judgment calls that don't have a well defined spec or plan in place yet; like unofficial or unapproved tokens if you will. A lot of this stuff simply never has had specs as it has been internal to how companies work and their secret sauce.

The closest thing we have are governance and compliance policies due to legal/business needs requiring it so it's far more well documented than operational ones in how we work. It is more about the how versus the what here I guess is what I'm saying.

But yeah this is why it does great when there are tests, design systems, evals, and other artifacts to mirror. Far more reckless and unpredictable without these things, but still great for exploration and finding the data output you seek.

[withinboredom]

Doesn't that make sense? Its text prediction. If you give it examples, it can predict. Synthesizing "put semi-colons on new lines" requires it to generate its own examples 'in its head' (so to speak) and remember that. It won't.

It's like when I see people feeding it a whole bunch of "best practices" and expect it to follow them. It won't. But you could ask it questions about the best practices all day long.

[brandensilva]

Yes, exactly. Any engineer deep on this stuff right now understands that grounded predictive engine sprinkled with RL training and are discovering what that means in terms of its strengths and weaknesses for company use.

[dnautics]

Supposing an unspecified or poorly specified function f(x), and example "f(A)=>B", "given C tell me what f(C) is" lies at the core of creativity.

Idk, calling it "just text prediction " seems unfairly dismissive of this capability

[withinboredom]

Saying that it’s dismissive is like saying writing (insert language) is dismissive that you’re just writing assembly.

at the end of the day, it presents a vector field and predicts the next vector. That’s literally the heart of intelligence just like assembly is the heart of execution. When playing table tennis, your brain is literally predicting seconds into the future to get your body into the right position.

But we aren’t discussing intelligence here. We are discussing how best to utilize that intelligence.

[dnautics]

You're making my point for me, saying table tennis is "just a proprioceptive predictor" is dismissively reductive (and not a particularly useful framework for understanding table tennis), even if it is strictly speaking accurate. It's the sort of thing someone who has no idea how hard training for table tennis is would say.

[withinboredom]

Let me put it bluntly. I’m agreeing with you but saying that isn’t what I was talking about and trying to give examples. You’re also agreeing with me.

The “idea” of table tennis and the rules. Those are things we can talk about. It’s those “best practices” I gave in my example. The actual playing of table tennis would be the examples. How to apply those best practices and what good code looks like.

[mikeyouse]

I ran into similar issues as we started to roll out LLM generated financials in our org.. I’m so used to the old SQL workflow of “grab this data from this table, that data from that table, combine it into a final result that looks like xxxx” where the tables were outputs from reports in our ERP but I was having terrible results.

Ended up pointing Claude at a few sample files from our existing reporting, gave it read-only oauth access to the ERP and said “build a new report showing the cash by project as calculated by xxxx - yyyy + zzzz in the style of the existing reports” and it basically one-shot from there.

Kind of crazy and I built a bunch of redundant check-sums because I honestly didn’t think it would be able to replace like 6 workdays of effort for the 2 FTEs who generate that kind of thing manually every month but so far so good..

[coredog64]

I was recently using Copilot to implement a small feature within a very large codebase. About 75-80% of the time, the code that was added matched the current style (warts and all). Copilot would specifically go off and research "How X is already done in the codebase" all the time.

[8note]

You basically get this for free, if the coding agent has read the relevant classes that the legacy code its touching has to match.

just dont break out a plan without also having it read the code again