[FpUser]

>"But still, a server should rarely ever be your bottleneck. Your application's DB I/O will be."

I have C++ server. Instead of constantly querying DB it holds all business data in RAM in appropriate structures optimized for real time usage, not how they're kept in DB. All read requests are basically limited by network IO except when some request calculates some more or less complex math. No waiting for DB. Writes are batched and a frequency of those is way less than that of reads.

It reverse proxied by Nginx. Putting Caddy instead would be a disaster if Caddy is much slower.

[mcspiff]

Not to diverge too much, but if you’re keeping everything in memory how do you handle hardware failures etc? Wouldn’t that result in data loss?

[FpUser]

Data is written to disk but requests are batched and executed as a single transaction. It is very fast. Partner systems know that the request may fail and business processes are organized accordingly. In practice it never really happens. The overall performance is insane (hundreds of times better) comparatively to some Python scripts fishing in database for every request.