[vosper]

I lot of my infrastructure runs on the spot market. If you launch an individual machine there's no guarantee that at any time it won't be pre-empted (terminated). You do a little time to handle graceful shutdown of your code before the machine goes away, so it's not like the plug was pulled out at the wall.

If you're using Spot Fleets (a new feature) AWS will take a spec of the compute capacity you want and find it for you, across different instance types and availability zones, which makes it more likely you'll get the capacity you want.

In practice spot machines are pretty stable. We run hundreds across several different instance types and they're often up for weeks at a time.

So unless your batch job absolutely must not be delayed you might give spots a go. We used to run critical analytics EMR (Hadoop) jobs on spots and it saved us a a fortune - probably hundreds of thousands of dollars. Very occasionally we would switch them to on-demand (automatically) if we couldn't get spots at the price we wanted.

[vgt]

Have you looked at Google's Preemptible VMs? The big difference is that it's a flat 70% off fee, rather than a volatile market, and Google has relatively few but versatile VM types, which makes it easier to architect infrastructure (ex. no need for a high-IO VM or a high-network VM.. just attach a high-IO disk to a regular VM and networking is world-class).

[vosper]

We're heavily integrated into the rest of the AWS system, it wouldn't be worth the engineering effort. Also, the spots don't make up a large enough portion of our overall AWS spend, if we wanted to save money there are other things we could do that would let us stay on AWS (which we like).