[grejdi]

Webhooks are great for producers of events, and I'd argue that it's too cumbersome for them to provide an '/events' endpoint primary because of scaling. With webhooks, they can offload events at their own pace.

For consumers, I agree with most here that Kafka is certainly overkill. We've gotten away with a very simple architecture to have reliable event consumption. We point all webhooks to an (AWS) API Gateway backed by Lambdas. The Lambdas push the events to an SQS queue (FIFO-queue, if it needs some sort of sequence), and we take our time consuming the events through a very generic poll.

[masklinn]

> Webhooks are great for producers of events, and I'd argue that it's too cumbersome for them to provide an '/events' endpoint primary because of scaling. With webhooks, they can offload events at their own pace.

TBF they could do something similar with `/events`, instead of pushing events to a webhooks-sending queue just push them to the events buffer, which could even be a circular buffer just to point out that the essay is completely wrong. TFA is not asking for /events, they're asking for a very specific kind of /events with a large non-drained buffer. Something which would only ever work for low number of events: $dayjob's github integration takes in several events per second.

A proper event stream would be nice though, github's webhooks delivery system is not exactly reliable.

[jcrites]

> With webhooks, they can offload events at their own pace.

They can't offload webhooks at their own pace if the two parties want reliable delivery. The server providing the webhook might be experiencing a prolonged outage, in which case the sender needs the ability to buffer the events anyway.