Hacker News new | ask | show | jobs
by lurking_swe 28 days ago
just the other week I asked Fable 5 to diagnose the cause of some intermittent latency spikes on an API that queries an OpenSearch cluster at work. I encouraged it to look at the datadog metrics, splunk, the whole works. I let it loose to look at whatever it wanted.

End result - 2 hours later it produced a convincing theory with lots of references, and burned a bunch of tokens too of course. just for fun we tried its suggestions and deployed them to prod. Guess what? Didn’t fix the issue. Alas, a human was needed after all.

either everyone’s working on toy problems, or they’re working on very cookie-cutter code. I’m really not sure. I DO remain impressed with Fable 5 but the idea that we’ll all be unemployed in 2 years is hilarious delusion. we’re already at the point where many organizations are scaling back some of their AI spend.
1 comments
Chance-Device 28 days ago
By any chance, is the following true: you had no empirics in the loop. It couldn’t validate any of its theories by experimentation. If the solution requires being able to make actual changes in order to gather more information and it is not allowed to, and this is the only way to solve the problem, by definition it couldn’t do it, nor could you. Or was it something that could be worked out entirely on an a priori basis from the available data?

And while we’re talking about hilarious delusions, perhaps you should look at the current capability curve of AI and weigh it against the constant stream of arguments for why it couldn’t have continued at every point and yet has.

link
lurking_swe 28 days ago
i’m not an anti-AI believer. And our career is changing dramatically no doubt. i just think people overestimate short term gains and underestimate long term gains. (applicable to your comment)

In this case i had no empirics in the loop. The scenario was only reproducible under high api load. I could load test, but management isn’t eager to spend prod-like costs in staging (requires scaling opensearch a lot in stage). What can i say.

link
surgical_fire 28 days ago
> in the loop

Yes, the solution is to just burn more tokens.

link
Chance-Device 28 days ago
Would you care to say something substantive, or is that just not a thing you do?
link
surgical_fire 27 days ago
Sorry, I would need to burn more tokens for that.
link
pixel_popping 27 days ago
It actually is in most scenarios.
link