[strken]

I'm not totally anti-webhook or the person you replied to, but I'd prefer at-most-once delivery via something that establishes a reusable connection (grpc or even websockets?) and backed by an events endpoint like Stripe's where the client can read everything that would have been sent. That way the client can replay all the events at leisure and retries aren't the server's responsibility.

[stephenr]

> I'd prefer at-most-once delivery via something that establishes a reusable connection (grpc or even websockets?) and backed by an events endpoint like Stripe's where the client can read everything that would have been sent. That way the client can replay all the events at leisure and retries aren't the server's responsibility.

Isn't that basically a description of SSE (https://en.m.wikipedia.org/wiki/Server-sent_events)?

[strken]

You could use SSE, long polling, or even a webhook. With the latter two you'd miss out on some of the performance gains of not needing to re-establish a connection unless the producer does http streaming, but the main stability points are not using webhooks as the sole means of delivery and not flushing events after they're delivered. So many webhook implementations don't go far enough and just fling events at the consumer with a short-term retry policy, or none at all, and then don't provide a way to see what events got missed.

[stephenr]

I mean, you can use anything if you just want a dumb pipe to put an ad-hoc data stream over.

My point is that SSE is already a standard to do exactly the thing you're asking for: publish a one-way event stream, with a built-in mechanism for reconnecting and telling the remote end the last event you received (so you can catch up on anything missed during the disconnected period).

[strken]

Right, sure, I've never used it outside a browser but I guess it would work fine. My point is that A) no matter which standard you use for sending a stream of events it's probably better than a webhook, and B) it's still important to provide both a long retention window and good tools to query events for debugging no matter what you use.

[TheCycoONE]

I also prefer events over a persistent connection for efficiency, but webhooks are far better when the client is using a function as a service model.

[simonw]

Right: in this day and age there are an almost unlimited array of inexpensive, easy ways to spin up an HTTP endpoint that can receive a POST request.

We have a lot of experience scaling those kinds of endpoints (and making them redundant) too.

Spinning up an always-on server that can maintain a persistent connection - and reconnect automatically if it drops, and if the server is rebooted with an update and suchlike - is a whole lot harder. Possible, but not nearly as easy.

[strken]

My experience is that if your server doesn't retry webhooks then you could be doing something more efficient, and if it does retry webhooks then that indicates it's important not to miss anything and you should use persistent events rather than relying on bug-prone "if there's a 200, drop the event from the DB" webhook retries from the server.

I do think there's an argument for the interoperability of webhooks for integrating different services. I'm skeptical that they're the best choice from a purely technical perspective.