[plicense]

I would suggest also looking at https://github.com/bazelbuild/remote-apis. Its essentially a standard API for remote (any binary) execution as a service and there are several reference implementations of it (Buildgrid, BuildBarn, Google's own service etc).

And you can consider using gVisor to minimize container breakouts to a great extent.

[adamgordonbell]

I'll checkout that remote-apis link.

gVisor was considered but so far it looks like the next iteration with be using firecracker vms. Our backend is buildkit and it can't run in gvisor containers without some work.

[ithkuil]

Firecracker looks great but it requires bare metal instances or nested virtualization (which is not supported by EC2 instances IIRC).

How do you run firecracker?

[iampims]

EC2 metal instances are expensive but they let you run FC.