[tyingq]

ETL is a funny space. At least in the "Enterprise" world, it's dominated by Ab Initio, which is crazy expensive.

They seem to be coasting too, for quite some time. Their website is probably the most terrible site I've ever seen for an expensive piece of software. You can't even tell what it is, how to buy it, or even how to contact them. https://www.abinitio.com/en/

[thenaturalist]

What's your source for the fact that Ab Initio is dominating?

Other than that, there are a several other tools in the Enterprise Analytics space that fall into a similar pattern like Alteryx or Collibra. But from their perspective it makes perfect sense, I guess. When your sales is done by relationship building and there isn't much competition once you're in, there isn't really a need to boast a fancy website or make an effort.

If anyone has a good resource on how enterprise IT procurement is done or the dynamics around it, I'd love to read up on that.

[GordonS]

I was going to ask the same question - I've never even heard of Ab Initio.

[tyingq]

See this Quora Q&A:

https://www.quora.com/Which-companies-in-the-USA-use-Ab-Init...

My suspicion is that their customers are mostly companies that use Teradata, because it has a fair amount of Teradata specific features. Probably not good news for their future, but lucrative for now.

[kfk]

Well I have a lot of experience with IT procurement. IT is more about cost and risk mitigation than technology. The last thing IT wants is to try new software that might blow up in the hands of their unprepared outsourced helpdesk support people. So the absolute first thing you want to show is robust and cheap support or even better a way to make sure your product works with their helpdesk setup. Another thing is cost and how your product helps them run things with less money. They could not care less about the quality of the product because they will not be the ones using it. In general you sell by telling a story that fits with whatever story they are telling the business. So if IT is telling business cloud is the next thing and you sell on premise you won’t go anywhere.

[tyingq]

Just anectdotal experience. By "dominating", I mean in non-tech US Fortune 500 companies. They have very few employees, and the software is very expensive.

"When your sales is done by relationship building and there isn't much competition once you're in, there isn't really a need to boast a fancy website or make an effort."

Sure, but try and find a phone number or email address on their site. They've taken coasting to a whole new level.

[jmngomes]

More than 90% of the enterprises I've worked with use one of the enterprise tools listed in Gartner's magic quadrant, typically from the Leader quadrant, sometimes from the Niche quadrant, here are a few examples: https://duckduckgo.com/?q=gartner+magic+quadrant+data+integr...

What's actually mind boggling for me, and I wonder if it's a bit of over-engineering, is people going for complex setups (oh, it's just Airflow scripts with k8s and a little bit of SystemD services and configs plus some shell scripts) when there are COTS tools that do more for less engineering cost. Yes, these carry a price tag, but it's usually quite less than paying for engineers to babysit a tool with a ton of moving parts...

[massaman_yams]

Or, you can look at the (say) 6 month sales cycle for an enterprise platform, and the internal political wrangling that may be required to get approval for the line item, as significant hurdles to moving forward with an ETL project. There are some legitimate, and some less valid, reasons for many engineers' (and engineering-driven orgs') bias against commercial options; and yes, sometimes this results in over-engineering.

Airflow can do some things many commercial tools cannot, though, so for some it is the right option.

[dsr_]

FWIW, Ab Initio gave me the worst interview experience of my life. We ended up with 24 hours of interviews over the course of two weeks, then they dropped me as soon as we started discussing compensation. (This was almost 20 years ago; perhaps they've changed since then.)