Fivetran and Hightouch do ELT and reverse ETL, respectively, which are one-way syncs. While Bracket can handle one-way syncs, two-way syncs (in situations where you need writes from both Postgres and Salesforce to sync) are our bread and butter.
We’ve worked with some folks who were cobbling together ELT and rETL to achieve a type of two-way sync, but they faced three problems with this approach: most importantly, handling for infinite loops, where an insert in one data source triggers an insert in the other, which then gets picked up anew, etc. Second, it wasn’t clear how to handle merge conflicts when the same record is edited from both sides (we give priority to the primary source). Finally, it was frustrating to debug problems when having to log into two separate tools.
For sure, definitely has not been an easy problem to solve. The two-way syncing is one tough element (for the reasons mentioned above), and configuring our requests to the Salesforce API in a way that minimizes our footprint (while still keeping syncs fast) has been another tricky task.
I'l also give a shout-out to Airbyte (https://airbyte.com/), with which I've had some limited success with integrating Salesforce to a local database. The particular pull for Airbyte is that we can self-host the open source version, rather than pay Fivetran a significant sum to do this for us.
It's an immature tool, so I don't yet know that I can claim we've spent _less_ than Fivetran on the additional engineering and ops time, but it feels like it has potential to do so once stabilized.
I don't know much about Bracket, but looking through their docs[1] it looks like they support streaming from Salesforce as an alternative to batch/scheduled jobs (which is what Fivetran does)
Yup! We offer two ways of interacting with Salesforce: polling and streaming. The benefits of streaming are faster syncs and optimized Salesforce API usage.
Right now, we're focusing on use cases that require changes to be synced within a minute. Fivetran and its ilk are generally more focused on large data batches at less frequent intervals (e.g., to unlock 1-minute syncs at Fivetran, you must be on the Enterprise plan, whereas we offer that out of the gate with the Starter plan).
Yeah exactly. One-way data syncing is a big market, and these companies have plenty of use cases to tackle, which is why I think two-way syncing has not been a priority for them.
That’s one reason why people trying to stand up two-way syncs have been stuck with dated tech (Heroku Connect) or have had to build in-house.
We’ve worked with some folks who were cobbling together ELT and rETL to achieve a type of two-way sync, but they faced three problems with this approach: most importantly, handling for infinite loops, where an insert in one data source triggers an insert in the other, which then gets picked up anew, etc. Second, it wasn’t clear how to handle merge conflicts when the same record is edited from both sides (we give priority to the primary source). Finally, it was frustrating to debug problems when having to log into two separate tools.