We just officially released the autoscaling runner in https://about.gitlab.com/2016/03/29/gitlab-runner-1-1-releas... Please let me know what we can do to improve it for your use case.
Also see my other comment in this post https://news.ycombinator.com/item?id=11387327