[avita1]

I agree with most of this, but was surprised by your comment:

> Oh, also, the implementation of Kubernetes cron jobs is also complete garbage (spawning a new pod every job is insanely wasteful).

How often/how many cron jobs are you running that spawning a new pod per job is a problem ?

[dvt]

The first iteration (I actually wasn't around for that) was trying to run a cron for every "data ingestion job" -- at some points, we were doing about 50k+ API requests daily (FB/Instagram/Twitter/etc.) and that was absolutely not tenable using k8s cronjobs.

[snuxoll]

Why use cronjobs at all for this? This is a classic work queue problem.

[dvt]

I wasn't there for this decision, but I assume cronjobs were being treated as "cloud functions" -- and to be fair, the k8s documentation kind of makes it seem that you could technically do that, but fails miserably if you try to do so in practice.

[dilyevsky]

50k/day is less than 1 qps. This is nothing. This is either not the full story or your cluster was setup completely wrong

[regularfry]

Depends how spiky the distribution was. 20k a second at 2pm? Gonna have a bad time.