[wlamartin]

Booting VMs on your worker seems like exactly the kind of debris concourse is encouraging the avoidance of by enforcing the containerisation of your builds. That said, I can't think of a reason why you couldn't boot a VM in a privileged Garden container, have you tried?

[mugsie]

Yeah - I can't remember the actual failure, but it did not work out well.

We also need the VMs to survive across build steps.

We are working on an P/IaaS - so integration testing requires actually booting a full stack, then running an install, then testing the resultant servers are actually running Kubernetes

[wlamartin]

Ah that's a shame, I still suspect it would _probably_ work with the right tweaking but I appreciate there's a hurdle there that might be frustrating to overcome and that's no good.

We also test a PaaS and require VM spin up but we call out to GCP/AWS as necessary.

[jacques_chester]

> We also need the VMs to survive across build steps.

You might represent them as a pool resource, so they can be safely handed from job to job. A lot of teams do this for stuff where they want to share an expensive, long-lived, stateful resource.

Disclosure: I work at Pivotal.

[mugsie]

The VMs are per job - once the CI run tests that the orchestration has created the right system config on the VMs, and that basic integration tests work, we destroy the VMs.

I have worked in two companies now with Cloud Foundry distros, and both of them have started using Concourse, and then moved to Jenkins (for non Cloud Foundry projects - I shudder to think what would happen to a distro that attempted to not use Concourse)

[jacques_chester]

I'm still lost. You said at the top that you need the VMs to survive across build steps (I read that as jobs), but now they're per-job?

The Concourse->Jenkins thing will definitely happen without better docs, examples and widespread understanding. Concourse is non-obvious to newbies and the error messages need a lot of love. Or any love.

I don't quite understand the parenthetical, could you elaborate?

[smgoller]

I second what wlamartin said. We have pipelines that do integration tests that spin up 30 instance clusters in AWS. Trying to spin up something similar inside a concourse worker just isn't worth trying.