[superb-owl]

> Isn't it just Kubernetes under the hood? That's not the right question...Kubernetes is just one of the building blocks for easy and secure app deployment. It is not a one-shot answer for day two operations such as reliable databases, load balancing, backups, certificates, and data security...With SetOps, you don't need to care how we run containers – you'll profit from the sensible choices and long hours our infrastructure experts spent to make sure it runs well.

I have really mixed feelings about this response.

On the one hand, I 100% agree - vanilla k8s is not prod-ready, and you need to do a _lot_ of work to figure out some things, especially around persistent storage (but load balancing and certs are a pretty solved problem).

But the line "you don't need to care how we run containers" bugs me. Maybe your two-person start up doesn't need to know, but eventually you will grow to the point that you _do_ need to care how things are running, and need control over it. This is why so many companies end up outgrowing Heroku and have to go through an expensive migration.

What I'd love to see is a "batteries-included Kubernetes", which allows me to slowly take control over more and more of the stack, until I'm a 1000 person company and ready to run my own clusters.

[latebird22]

Interesting thought. I can see your case for some growing companies. However I'm not sure if you would always need Kubernetes for this. What would you like adjust when we grow into a 1000 person company? I think you have still similar requirements like autoscaling, resource allocation, zero-downtime deployments etc. This is also possible without direct access to the container management.

And there are a lot of companies which do not become the next Unicorn and need an easy way to manage their container workloads.

SetOps currently uses ECS since it comes with no additional overhead costs for the management plane/API and does the container management job well enough. However this is not a definite decision and ECS could be replaced in the future. The main point is that there is a simple abstraction for users managing the workloads and that the "backend" is interchangeable.

[superb-owl]

Here's a few examples of policies/needs that I've seen companies run up against at scale. K8s does a great job solving them.

* Understanding which workloads share a node's memory/CPU, and isolating certain workloads for security reasons

* Running specific workloads on specific instance types (e.g. with GPU or extra CPU)

* Configuring network policy between workloads

* Airgapping certain workloads

* Setting priority levels for different workloads, so some scale more rapidly while others have to wait for a new node to be provisioned

* Customized scaling behavior (e.g. based on the depth of a queue or latency metrics)

* Multi-region support for DR

I could probably go on :)

[latebird22]

You are totally right. For these specific use cases you probably want full control. :) I guess if you have these requirements a tool like SetOps, which simplifies the management, might not be the right fit then.

Although some of these requirement, like running specific workloads on specific instance types, could easily be implemented.

[solatic]

> SetOps currently uses ECS

Not everything is a stateless HTTP microservice. Solutions like ECS start to fall apart when you try to run stateful workloads, especially when the lifecycle of the workloads needs to be coordinated to prevent loss of availability or data (i.e. cannot tolerate 2/3 of the containers being knocked offline at the same time). AWS does not offer a managed datastore (e.g. RDS) for every datastore, and many of the datastores it does offer (e.g. MSK) are "let's tick this box in the quest for covering all our customer needs" but not cost-effective for production workloads.

Maybe, as a product, you make a decision to tell your customers, when you need to run something like that, go hire DevOps and migrate off. But you'll be more credible if you're up-front with what kinds of workloads you don't intend to support, so that customers who have a strategic vision for engineering can say, hey these guys will be great for me for the next few years, now I'm more likely to buy in.

[Legogris]

They do have Batch, which IIRC was originally only for ECS, for that scenario.

It's quite horrible in many ways but not I don't think any of it is really because the ECS model is somehow inherently unsuitable for stateful long-running workloads, it's Batch being a bad product.

Both ECS and Batch are just leaky abstractions enough that some unknown unknown or forgotten nuance will come along and bite you when you least need it and make you question the value proposition.

[andrewstuart2]

It's funny you mention that. I'm currently a bit stalled (mostly for lack of time/motivation, but trying to get back on it very recently) trying to build exactly that. I'd be very curious what you think of the premise I've outlined on my landing page [0]. Basically, I have built the same thing a half dozen times and I want to do it for more people, for less money individually. And as part of the bootstrapping and DX, provide training on all the components, how to use them, and why I chose them. Ultimately you should be able to walk away eventually if I've gotten my part right. Or don't, if I can continue adding value at a rate you think is worth it.

[0] https://microcumul.us

[arthurcolle]

There is very little on this page. The bios are cut off on the Team page, the landing page is basically just a blurb. I much prefer sites that enable immediate demos of the thing in question, or at least something that gives me a better sense of what the value-add is. The site itself reminds me of consultancy sites more than a 'product' oriented startup page. Just my $0.02

[andrewstuart2]

You're not wrong, and that's the part I'm working on now. I've honestly mostly just used this page to suss out what I want to do, but apparently I could do better about communicating what it means to me. I'm guessing a video demo would be helpful, plus actually being able to take it for a test run. I'm pretty stoked about what I have, but it's just not quite ready to do much other than a video demo.

I'd hoped the intro page was enough to get the idea across in general, but it sounds like I need to improve there.

[lukeramsden]

This also makes sense as somebody who has never deployed to or operated k8s - I want the most prod-ready and batteries-included cluster config, which I deploy myself on my own infra, and can then customise as and when I need to, and learn as I go.

[_whiteCaps_]

Have you seen https://github.com/cloudfoundry/cf-for-k8s?

[latebird22]

My experience with tools like these on Kubernetes, even with something stable and widely used as e.g. certmanager, is that in the end something still breaks and you still need to get familiar with all the building blocks. In comparison if you use managed services like the AWS Certificates service you have less friction and less risk for something to break.

[chrisweekly]

Good abstractions always allow you to drop down a level as needed.