[nxrabl]

> We have a blip in March / April every year; these cyclical patterns aren’t relevant to explain here.

The 'blip' post-AI is up ~30% for the months in question. There simply isn't enough data here to prove or disprove the thesis that "customer issues remain broadly stable" - you could equally argue something more along the lines of "AI engineering does not increase issues under ideal conditions but amplifies issues under external pressure."

[applfanboysbgon]

"Not relevant to explain this stat" in an article entirely about using those stats to justify their AI policy is brazen, I have to give credit.

One other point of note is that the chart only goes back to January 2025, and only dilineates "majority AI" from "not majority". If the usage rate in January 2025 was at 45%, "issue rate remained roughly stable (with a giant asterisk) when moving from 45% to 51%" is not exactly a compelling story. There's no narrative I trust less than one created by cherrypicking data to lend itself an aura of faux objectivity.

[colinhowe]

Author here. This is valid feedback, thank you. This post was originally an internal post. Internally, I have credibility as "the guy that does his homework". That doesn't carry over to the broader internet!

I think you're right that the data presented doesn't prove or disprove the thesis that "customer issues remain broadly stable". I think it lends credibility to the idea that "AI didn't lead to an obvious and immediate slopfest of bugs".

As for the bumps, we've noticed them annually and we haven't yet found a solid reason why. There’s no correlation with customer growth, PRs shipped, or features shipped. It's likely a combination of various seasonal effects in our own hiring and customer onboarding. What isn't in this article: we've more than doubled our customer base in this time period, our customers are more mature and raise bugs more readily, we've got ~60% more engineers, and our product surface area has grown a lot too. I'm yet to land on a satisfying metric that normalizes bug rates in a changing environment.

[miltonlost]

A “blip” that we have to take their word on, though the relative jump from the 1 prior year wasn’t as high as this one. Bad stats juking