A New Approach to GPU Sharing: Deterministic, SLA-Based GPU Kernel Scheduling

[medicis123]

Most GPU “sharing” solutions today (MIG, time-slicing, vGPU, etc.) still behave like partitions: you split the GPU or rotate workloads. That helps a bit, but it still leaves huge portions of the GPU idle and introduces jitter when multiple jobs compete.

We’ve come up with a different model, similar to how operating systems schedule tasks. Instead of carving up the GPU, we run multiple ML jobs inside a single shared GPU context and schedule their kernels directly. No slices, no preemption windows — just a deterministic, SLA-style kernel scheduler deciding which job’s kernels run when.

This results in the GPU behaving more like an always-on compute fabric rather than a dedicated device. SMs stay busy, memory stays warm, and high-priority jobs still get predictable latency. More details at https://woolyai.com/blog/a-new-approach-to-gpu-kernel-scheduling-for-higher-utilization/ Check out our technology at https://www.woolyai.com.