| Can't speak for the author, but I can tell you what we do in our company, which is also completely microservice-based. Backstory: We used to have a helper tool that allowed a developer to run any app locally. It tried to set up the same stack as we were using in production: HAproxy, Nginx, Ruby, Node, PostgreSQL. It was problematic, because people had machines that differed slightly: Different versions of OS X (or Linux), Homebrew (some used MacPorts), Ruby, Postgres, etc. We could have spent a lot of time on a script that normalized everything and verified that the correct versions of everything was installed, but the problem with developing such a tool is that you won't know what's going to break until you hire a new developer who needs to bootstrap his box. Or until the next OS X release, or something like that. Syncing everything with the production environment was also difficult. The way we configure our apps, a lot of the environment (list of database servers, Memcached instances, RabbitMQ hosts, logging etc.) is injected. So with this system we'd have to duplicate the injection: Once in Puppet (for production), a second time on the developer boxes. So we decided pretty early on to mirate to Vagrant. --- We now run the whole stack on a Linux VM using Vagrant, configured with the same Puppet configuration we use for our production servers. The Vagrant box is configured from the exact same Puppet configuration that we use for our production and staging clusters. The Puppet config has a minimal set of variables/declarations that customize the environment that need be tweaked for the environment. From the point of view of the Puppet config, it's just another cluster. We periodically produce a new Vagrant box with updates whenever there are new apps or new system services. Updating the box is a matter of booting a new clean box and packaging it; Puppet takes care of all the setup. We plan on automating the box builds at some point. To make the workflow as painless as possible, we have internal "all-round monkey wrench" tool for everything a developer needs to interact with both the VM and our clusters, such as for fetching and installing a new box (we don't use Vagrant Cloud). One big benefit of using Vagrant is that this internal tool can treat it as just another cluster. The same commands we use to interact with prod/staging — to deploy a new app version, for example — are used to interact with the VM. One notable configuration change we need for Vagrant is a special DNS server. Our little tool modifies the local machine (this is super easy on OS X) and tells it to use the VM's DNS server to resolve ".dev". The VM then runs dnsmasq, which resolves "*.dev" into its own IP. We also have an external .com domain that resolves to the internal IP, for things like Google's OAuth that requires a public endpoint. All the apps that run on the VM then respond to various hosts ending with .dev. Another important configuration change is support for hot code reloading. This bit of magic has two parts: - First, we use Vagrant shared folders to allow the developer to selectively "mount" a local application on the VM; when you deploy an app this way, instead of deploying from a Git repo, it simply uses the mounted folder, allowing you to run the app with your local code that you're editing. - Secondly, when apps run on the VM, they have some extra code automatically injected by the deployment runtime that enables hot code reloading. For Node.js backends, we use a watch system that simply crashes the app on file changes; for the front end stuff, we simply swap out statically-built assets with dynamic endpoints for Browserify and SASS to build the assets every time the app asks for them (with incremental rebuilding, of course). For Ruby backends, we use Sinatra's reloader. --- Overall, we are very happy with the Vagrant solution. The only major pain point we have faced is not really technical: It's been hard for developers to understand exactly how the box works. Every aspect of the stack needs to be documented so that developers can know where the look and what levers to pull when an app won't deploy properly or a queue isn't being processed correctly. Without this information, the box seems like black magic to some developers, especially those with limited experience with administrating Linux. We also sometimes struggle with bugs in Vagrant or Virtualbox. For example, sometimes networking stops working, and DNS lookups fail. Or the VM dies when your machine resumes from sleep [1]. Or the Virtualbox file system corrupts files that it reads [2]. Or Virtualbox suddenly sits there consuming 100% CPU for no particular reason. It happens about once every week, so we're considering migrating to VMware. Another option is to actually give people the option of running their VM in the cloud, such as on Digital Ocean. I haven't investigated how much work this would be. The downside would obviously be that it requires an Internet connection. The benefit would be that you could run much larger, faster VMs, and since they'd have public IPs you could easily share your VM and your current work with other people. Another benefit: They could automatically update from Puppet. The boxes we build today are configured once from Puppet, and then Puppet is excised entirely from the VM. Migrating to a new box version can be a little painful since you lose any test data you had in your old box. As for your question about what services to run: It's a good question. Right now we only build a single box running everything, even though we have a few different front-end apps that people work on that all use the same stack. We'll probably split this into multiple boxes at some point, as memory usage is starting to get quite heavy. But since all the apps share 90% of the same backend microservices, the difference between the boxes are mostly going to be which front-end apps they run. [1] https://www.virtualbox.org/ticket/13874 [2] https://www.virtualbox.org/ticket/819 |