We welcome NVIDIA's decision to open-source Run:ai, and we're working to make it a first-class option on Lambda Kubernetes Service (LKS).
Lambda, the AI cloud, has supported Run:ai on our infrastructure since 2022. Our vision is to make Run:ai’s capabilities easier to adopt and operate through LKS, giving teams more control over how they allocate and share GPU capacity.
Why whole-GPU allocation leaves capacity idle
AI development puts varied demands on GPU resources. A debugging session may use a fraction of a GPU. An evaluation may launch many independent jobs. Distributed training may require many tightly connected GPUs at once. - And those needs change as a project moves from idea to experiment to production.
Allocating whole GPUs to small jobs leaves researchers waiting while usable capacity sits reserved. The same friction appears across teams, when one group's experiments queue behind another group's idle allocation. Both turn available hardware into delays between experiments.
How Run:ai handles GPU sharing and scheduling
Run:ai is a GPU scheduler for Kubernetes. It adds two things the default Kubernetes scheduler doesn’t do: GPU sharing, so more than one workload can run on a single GPU, and quota-based scheduling, so teams get a guaranteed allocation but can borrow idle capacity from each other. The result is higher GPU utilization on a shared cluster without teams having to coordinate by hand.
In practice, that comes down to two features:
- Its GPU fractions let workloads request and share portions of GPU memory.
- Its scheduling policies let projects borrow spare capacity for interruptible jobs and return it when other teams need their share.
Together they answer two practical questions: how much of a GPU does this experiment need, and how should a shared cluster respond when different teams need it at different times?
Keeping GPUs healthy with node auto-remediation
Sharing and preemption both rest on an assumption that the GPU underneath is healthy. On GPU hardware at scale, that’s not a given. Accelerators throw XID errors, NVLink and PCIe links degrade, memory rows get remapped, and InfiniBand ports drop below their expected rate. A scheduler can’t tell a healthy GPU from a failing one, so it keeps sending jobs to a node that drops them.
This is the part LKS owns. LKS is managed Kubernetes on Lambda GPUs, and auto-remediation ships in it today: Lambda monitors every node for those faults and surfaces each as a standard Kubernetes node condition. When one fires, the node is cordoned so nothing new schedules to it, running work is drained, and the fault is either corrected automatically or escalated straight to hardware repair, and the node returns only after it revalidates. A bad GPU becomes a routine event rather than an incident, which is what lets a scheduler above it keep placing work without anyone filing a ticket.
Run:ai on LKS
GPU scheduling is the layer we deliberately leave open. Next, LKS is headed to an AI software library: schedulers that run on Kubernetes, such as Slurm and Run:ai, delivered as managed components that Lambda validates against our hardware and keeps current, rather than assembled from Helm charts by hand. Run:ai is a first-class target for that library, and we're working with NVIDIA to get there. Open sourcing is what makes that possible: it lets Lambda package, test, and maintain Run:ai as part of LKS rather than give customers a set of integration instructions. We're aiming for an out-of-the-box integration, with Lambda owning the packaging, the validation, and the ongoing operations.
As a company that produces open research, Lambda values being able to inspect, adapt, and improve the tools we depend on. NVIDIA's decision to open-source Run:ai extends that freedom to orchestration. That is the direction we see for AI infrastructure, and where we are taking LKS. Teams choose how to allocate and share their GPUs, while the platform keeps them healthy and available.
To learn more, read about Kubernetes on Lambda, read our docs, or talk to one of our MLEs.