[tomasGiden]

We’re using KEDA and ScaledJob to scale tomographic reconstructions in the cloud. When a CT scanner has finished uploading a scan, we let a ScaledJob create a Job to process the data. A scan is maybe 8 hours and during that time we don’t need any compute resources. But when it’s done we need both lots of CPU and GPU power to process GBs and TBs of data rapidly to show previews to the user.

Also, when a user triggers new previews we scale up nodes to process that data. The problem there though is the scale up time of the node pool which is a few minutes for a GPU node on Azure.

We payed to have a GPU running all the time before but that got too expensive.

As a side note, would I do it again I probably wouldn’t build a data pipeline on top of KEDA ScaledJobs and possibly not use Kubernetes at all.

[7thpower]

What would you use if you were to start fresh?