[jeffbee]

You're suggesting that preventing multiple replicas of the same job to schedule on the same machine as a good default. There's no evidence to support your conclusion, and my experience it quite the opposite. It is much better if people running batch jobs just schedule 100000 tiny replicas, and let the scheduler sort it out. This provides the cluster scheduler with plenty of liquidity. Multiple small processes are more efficient than a shared-nothing single process.

[hinkley]

Still the same question.

Do you think that batch processing is the default activity in Kubernetes, or something that people find after they are familiar with the system?

[jeffbee]

Yes, I think batch workloads are the most common workloads, in resource-weighted terms, among k8s users.

[hinkley]

You're being slippery, which comes across as dishonest.

Why does the resource weight have anything to do with the choice of defaults? Settings don't care how often they are read, they only care how often they are set. Large jobs use a disproportionate amount of total resources, sure, but they are tiny uptick in total configuration.

The stakes are higher, but so is the 'budget' for getting things right. I can deploy 5 servers and just wait to see what happens. If I'm doing an overnight job to process a billion records, I'd better be doing some due diligence beforehand, or I have nobody to blame but me. And the failure mode here is that I didn't spend money fast enough to get the job done.

With the current defaults what happens is I blow my monthly budget in one night. Which is very convenient for the vendor, but not convenient for my company.

"It is difficult to get a man to understand something when his salary depends upon his not understanding it." - Upton Sinclair