Is there any benefit to using coreOS when you don't need a million machines? How much work is it to start with, for example, if you have no idea what your scaling needs will be in the future?
No work at all; basic coreos is just docker managed by systemd. Can you make your application docker-izable? Were you planning on using process management? If you answered yes to both of these questions... :)
Further, coreos tries to make you write applications in a 12-factor-y way, so when the time does come for you to use a million machines, you won't need to make huge adjustments to your deployments (just plug the container and init script into fleet and let it roll).
The main benefit is it's an easy to update, minimal OS. And docker's already set up for you and updated regularly.
Our cloud-config implementation is designed so that you can use the same config to set up a new machine to match an existing one, and have it automatically join the cluster. Start with 3 machines and scale up as you need.
Learning systemd is one bump you'll have to get over, but all of the major distros will be using it, so they're is no better time to try it out.
I'm interested in it because I plan to use docker to ease application management, and with CoreOS I barely have to maintain the operating system because it's so lightweight. So it greatly simplifies things :)
If you think docker is a good fit for your application, this is kind of the next step.
The only significant benefit over traditional operating systems/network services is it's designed to work on flaky hardware and networks. The only use cases i've found for decentralized distributed networks of application services (other than obscure stuff like parallel processing of large datasets) is when you have no guarantees of availability. As far as "scaling", you don't need coreOS to build a scalable network (and i've seen no performance benchmarks of coreOS running on thousands of machines at a time, so I have no idea how well it scales)
Run from root partition A and only update on partition B, essentially via chroot. Downtime required for updates is the cost of a reboot. If, however, upon an error during boot, reboot into other partition which will be your last known good config.
Seems simple and straight-forward but is difficult for unmodified Debian-based distributions, so they've addressed this as a key feature of CoreOS.
Further, coreos tries to make you write applications in a 12-factor-y way, so when the time does come for you to use a million machines, you won't need to make huge adjustments to your deployments (just plug the container and init script into fleet and let it roll).