Hacker News new | ask | show | jobs
by Fiahil 1325 days ago
Hi and congrats !

> bidirectional data connectors which aim to solve automated ETL (eg moving data from an API to Snowflake), reverse ETL (we pipe the data straight to your CRM)

I need exactly this at work. However, my requirements are :

- it has to be "self-hosted", usually in my clients' VPC, sending data to a third party is a big no-no.

- most of them use SAP, Salesforces and many other esoteric CRMs & ERPs. It has to work for at least the most important ones.

- the target medium is usually the cloud provider's object storage (S3, ...), snowflake is quite rare. It also have to be flexible enough to support our own versioning strategy.

Would it work with your product ?

3 comments

+1 to a version which works in a VPC in a public cloud. This is starting to become table stakes for some large enterprise products.
We completely understand. We don't support it right now, but we built the product to be completely cloud native so this is straight forward for us to implement. But yeah, it's definitely in our roadmap.
The economics can be nice too, run everything on their bill :)
Launch it through Cloud Marketplaces. I believe rev-share just got much more favorable, especially at Google Cloud.
Thanks, that’s a great idea once we get the on-prem version ready!
Thanks a lot for the comment Fiahil!

All of this is in our roadmap so we are glad to hear that we are on the right track. However we would love to know more about your last point. If you can email us to andres@movinglake.com and edgar@movinglake.com so we can discuss it, we would greatly appreciate it!

Have you evaluated self-hosted Airbyte?

Can run on client's VPC infra.

Has connectors for SAP, Salesforce, some other CRMs, S3.

Not sure about own versioning strategy though.

The cost for evaluating solutions is quite high in energy and time. Plus, we will end up pushing this vendor to all our clients, so it has to be absolutely stainless and work with everyone, and everywhere.

Airbyte, looks "okay by the cover". If I want to go any further I would need to see what happens when I need to extract data from an esoteric, unsupported, undocumented, Java-based ERP that is no longer sold by its own developer. Unfortunately, I don't have the time nor the skills to do it. So, in the end, we let our clients write their own ETLs on their own stuff and match their datasets to our data model.

Unless I make the happy uncounter with a tool that focus on these forgotten ERP and CRM and don't completely break down when it encounters a weird FTP server serving "encrypted zip files", I'll just let it go.

Having a "Facebook ads" connector is not enough, and not where problems lies.