| KOPS is pretty awesome in that: * Actually pretty much works for what's in scope.. * It's got some nice configuration options that are discoverable and not hidden away in envars... * Some good prelim docs explaining how kubernetes is bootstrapped * Cluster management seems to function properly * Updating/upgrading What's missing IMHO(from an AWS user's standpoint including kops and k8s): * SUPER unapproachable codebase ATM for KOPS and friends * More flexible cluster dns naming so we can leverage real wildcard certs accross dev environments * Running kubernetes in private networks * Passing in existing networks created through other tools(terraform, cloudformation, custom etc) * Responsibility for stuff seems spread out across projects and is unclear which lies where(also leading to an unapproachable-ness for contributions) * AWS controllers that don't seem to fully leverage the AWS API's (traffic balanced to all nodes and then proxy'd via kube proxy; no autoscale life cycle event hooks) * Unclear situation on the status of ingress controllers; are they even in use now or is it all the old way?! * No audit trails * IAM roles for pods * Stuff I'm probably missing It's very frustrating TBH. On one hand AWS ECS has IAM roles for containers now, for the new Application Loadbalancer, and private subnet support. On the other hand they DON't have pet sets, automatic EBS volume mounting(WTF), a secrets store, configuration API, etc. Also frustrating is I feel the barrier to contribute is a too high ATM even though I have the skills necessary.. It's SO close though. If I can get private, existing subnet support I can probably start running auto provisioned clusters that are of use for some of our ancillary services in production. From there I might be able to help contribute to KOPS and AWS controllers. Right now it looks like there is just this one guy doing most of the work on AWS and KOPS; probably quite overloaded. |
Running kubernetes in private networks: You could probably get private subnet support by - Deploying manually or deploying with a script, then changing things in AWS (route tables, public IP, etc) to be private, manually afterwards (both cumbersome but possible) - Using NodePort instead of LoadBalancer on any services