[NathanFlurry]

I hope this isn't the case here with Rivet. I genuinely believe that Kubernetes does a good job for what's on the tin (i.e. container orchestration at scale), but there's an evolution that needs to happen.

If you'll entertain my argument for a second:

The job of someone designing systems like this is to decide what are the correct primitives and invest in building a simple + flexible platform around those.

The original cloud primitives were VMs, block devices, LBs, and VPCs.

Kubernetes became popular because it standardized primitives (pods, PVCs, services, RBAC) that containerized applications needed.

Rivet's taking a different approach of investing in different three primitives based on how most organizations deploy their applications today:

- Stateless Functions (a la Fluid Compute)

- Stateful Workers (a la Cloudflare Durable Objects)

- Containers (a la Fly.io)

I fully expect to raise a few hackles claiming these are the "new primitives" for modern applications, but our experience shows it's solving real problems for real applications today.

Edit: Clarified "original cloud primitives"

[motorest]

> Rivet's taking a different approach of investing in different three primitives based on how most organizations deploy their applications today:

I think your take only reflects buzzword-driven development, and makes no sense beyond that point. A "stateless function" is at best a constrained service which supports a single event handler. What does that buy you over Kubernetes plain old vanilla deployments? Nothing.

To make matters worse, it doesn't seem that your concept was thought all the way through. I mean, two of your concepts (stateless functions and stateful workers) have no relationship with containers. Cloudflare has been for years telling everyone who listens that they based their whole operation on tweaking the V8 engine to let multiple tenants run their code in how many V8 isolates they want and need. Why do you think you need containers to run a handler? Why do you think you need a full blown cluster orchestrating containers just to run a function? Does that make sense to you? It sounds like you're desperate to shoehorn the buzzword "Kubernetes" next to "serverless" in a way it serves absolutely no purpose beyond being able to ride a buzzword.

[eddythompson80]

I don't disagree with the overall point you're trying to make. However, I'm very familiar with the type of project that is (seeing as I have implemented a similar one at work 5 years ago) so I can answer some of your questions regarding "How does one arrive at such architecture".

> Why do you think you need containers to run a handler?

You don't, but plenty of people don't care and ask for this shit. This is probably another way of saying "buzzword-driven" as people ask for "buzzwords". I've heard plenty of people say things like

       We're looking for a container native platform
       We're not using containers yet though.
       We were hoping we can start now, and slowly containerize as we go

or

       I want the option to use containers, but there is no business value in containers for me today. So I would rather have my team focus on the code now, and do containers later

These are actual real positions by actual real CTOs commanding millions of dollars in potential contracts if you just say "ummm, sure.. I guess I'll write a Dockerfile template for you??"

> Why do you think you need a full blown cluster orchestrating containers just to run a function?

To scale. You need to solve the multi-machine story. Your system can't be a single node system. So how do you solve that? You either roll up your sleeves and go learn how Kafka or Postgres does it for their clusters or you let Kubernetes most of that hardwork and deploy your "handlers" on it.

> Does that make sense to you?

Well... I don't know. These types of systems (of which I have built 2) are extremely wasteful and bullshit by design. A design that there will never be a shortage of demand for.

It's a really strange pattern too. It has so many gotchas on cost, waste, efficiency, performance, code organization, etc. You always look and whoever built these things either has a very limited system in functionality, or they have slowly reimplemented what a "Dockerfile" is, but "simpler" you know. it's "simple" because they know the ins and outs of it.

[stavros]

> To scale. You need to solve the multi-machine story. Your system can't be a single node system.

Why can't it be? How many customers do you have that you can't deploy a bunch of identical workers over a beefy database?

Companies spend so much time on this premature optimization, that they forget to actually write some features.

[motorest]

> You don't, but plenty of people don't care and ask for this shit. This is probably another way of saying "buzzword-driven" as people ask for "buzzwords".

That's a fundamental problem with the approach OP is trying to sell. It's not solving any problem. It tries to sell a concept that is disconnected from technologies and real-world practices, requires layers of tech that solve no problem nor have any purpose, and doesn't even simplify anything at all.

I recommend OP puts aside 5 minutes to go through Cloudflare's docs on Cloudflare Workers that they released around a decade ago, and get up to speed on what it actually takes to put together stateless functions and durable objects. Dragging Kubernetes to the problem makes absolutely no sense.

[kentonv]

Where did Nathan say he's using Kubernetes? I think I missed something. His comment describes a new alternative to Kubernetes. He's presenting stateless functions and stateful actors as supplementing containers. He knows all about Cloudflare Workers -- Rivet is explicitly marketed as an alternative to it.

It feels like you didn't really read his comment yet are responding with an awful lot of hostility.

[conradev]

I currently need a container if I need to handle literally anything besides HTTP

[motorest]

> I currently need a container if I need to handle literally anything besides HTTP

You don't. A container only handles concerns such as deployment and configuration. Containers don't speak HTTP either: they open ports and route traffic at a OSI layer lower than HTTP's.

[conradev]

Yes! All I was trying to say:

Containers can contain code which open arbitrary ports using the provided kernel interface whereas serverless workers cannot. Workers can only handle HTTP using the provided HTTP interface.

I don’t need a container, sure, I need a system with a network sockets API.

[mdaniel]

FWIW, Lambda takes the opposite of your assertion: there are function entrypoints and the HTTP or gRPC or Stdin is an implementation detail; one can see that in practice via the golang lambda "bootstrap" shim <https://pkg.go.dev/github.com/aws/aws-lambda-go@v1.49.0/lamb...> which is invoked by the Runtime Interface Emulator <https://github.com/aws/aws-lambda-runtime-interface-emulator...>

I don't have the links to Azure's or GCP's function emulation framework, but my recollection is that they behave similarly, for similar reasons

[conradev]

Oh yes! I was thinking about the V8 isolate flavor of stateless functions (Cloudflare, Fastly, etc). I had forgotten about the containerized Linux microVM stateless functions (Lambda, Cloud Run, etc). They have everything, and my favorite use is https://github.com/stanfordsnr/gg

Funnily, enough, the V8 isolates support stdio via WASM now

[motorest]

> Workers can only handle HTTP using the provided HTTP interface.

Not true. Cloudflare Workers support Cron triggers and RPC calls in the form of service bindings. Also, Cloudflare Queues support consumer workers.

Claiming "Workers can only handle HTTP" is also meaningless as HTTP is also used to handle events. For example, Cloudflare Queues supports consumers using HTTP short polling.

[conradev]

You forgot email, too!

But I still can't handle SSH or proxy WireGuard or anything like that (yet!)

[lovehashbrowns]

While I think it’s great that kubernetes standardized primitives and I do love that IMO its best “feature” is its declarative nature and how easy it is to figure out what other devs did for an app without digging through documentation. It’s the easiest thing to go through a cluster and “reverse engineer” what’s going on. One of the legacy apps I’m migrating to kubernetes right now has like 20 different deployment scripts that all do different things to get a bunch of drupal multi sites up and running correctly whereas the kubernetes equivalent is a simple deployment helm chart where the most complicated component is the dockerfile. How does Rivet handle this? If I give 100 devs the task of deploying an app there do they get kinda fenced into a style of development that’s then simple to “reverse engineer” by someone familiar with the platform?