[atraac]

We've built similar thing to sync certain Salesforce objects to our .NET backend but Salesforce has their streaming all built around CometD which uses something I believe an insane person came up with - Bayeux Protocol. It's essentially an HTTP streaming, you shoot a GET call to an API and it starts streaming bytes of data. If it stops or timeouts, you're supposed to shoot another GET to continue streaming bytes, if nothing comes back you timeout and retry.

It's an absolute hell, most community frameworks barely work, there's zero good solutions to error handling, we had our Azure Application Insights constantly red because of timed-out requests that are in fact good, because they just mean there were no events to transmit.

I refuse to believe that multi-billion dollar company that is Salesforce couldn't come up with a RabbitMQ sink or virtually any better solution to that problem, especially that they could gate it behind a subscription most companies would pay for since they spend millions on SF either way...

[kunalrgarg]

Wow congrats on making it through with CometD and Bayeux. I think salesforce has realized that a lot of their APIs aren’t useful when using non standard approaches. I think the move from SOAP to also including REST was a signal that they’re trying to be more useful in this realm. I definitely agree a RabbitMQ sink would've been so ideal! I’m hoping it’s somewhere on their roadmap to make our lives easier.

[wmfiv]

I don't have any first hand experience but there is an integration with AWS EventBridge.

https://aws.amazon.com/blogs/compute/building-salesforce-int...

I know even less about Azure, but it looks like Azure Data Factory (?) provides some kind of similar functionality?

https://learn.microsoft.com/en-us/azure/data-factory/connect...

[robertlagrant]

From memory, one reasonably good solution I saw was with a company called Validic that did IoT integration stuff for medical devices. They got you to set up an HTTP SSE stream, that you just connected to and consumed from as and when events came through.

[kunalrgarg]

I haven’t heard of Valadic, but I’ll definitely check it out. Thanks for flagging!