[karmasimida]

I don't see AI system too special in terms of back-end engineering, except maybe for agentic system, things are inherently stateful.

But considering how limited RPM/TPM with regards mainstream LLMs, states saving/loading is hardly the bottleneck I feel.

[pcalcado]

Same here. I've had data-intensive systems and classifiers on critical paths for non-AI apps, and the same tools I used before seem to work fine with GenAI.

The primary real difference I've found has to do with when agents make decisions; this creates arbitrary call graphs in your distributed architecture and makes it harder to provision things, optimize, and do anomaly detection.