[pavlov]

I think the 8% number better explains why users were so overwhelmingly happy. Assuming the suggestions in general are not distractingly wrong, then 8% of code automatically written is a decent amount of time saved researching solutions.

[layer8]

But only 22% are accepted for those 8%, which means that the 78% code suggestions that are not accepted correspond to an equivalent of over 28% of all code written. Not sure that having to spend the time evaluating an additional 28% of code in vain amounts to an overall win.

Though I guess the success rates when using Stack Overflow aren’t too dissimilar.

[shostack]

What it doesn't tell us though is how useful the rejected recommendations were.

Meaning, how many rejected solutions were sufficient to give the engineer enough context to turn a 30m task into a 5m task because they generated a recommendation, got an idea, rejected it, and rewrote it more efficiently or more correctly?

There's a lot of "devil in the details" likely buried in here.

[visarga]

Interesting that 91% find it useful but only 8% of the code is generated by LLM. This is even with a LLM tuned on the internal codebase. This will give a mild boost but not replace anyone.