[texasbigdata]

So what’s the end state solution? The tools change and/or mature slower than a fast company will grow. Where are you going from here?

Are you eventually running your own cluster? Or, are you paying per Gb per second how GCP bills with decoupled compute / memory / storage? Or maybe with a little bit more overhead with the orchestration cost on top of consumption with how azure bills for ADF.

Just from reading blog posts, Spotify’s move was interesting, regarding their annual “songs you liked” at the holidays, where they sort of ran a hybrid.

Sorry to digress but what is the end state. Is there just a number where it makes sense to hire people and build and/or shift vendors? And if yes, what vendor is “after” stich? Astronomer maybe. Sorry if that’s basic.

[nikitaga]

Solution seems obvious, volume discounts in one shape or another.

SaaS companies pricing is often geared towards startups that "can't afford" to control costs, i.e. developer time spent on minimizing cost has a higher monetary and opportunity cost than ignoring those costs in the short term, even if they're piling up at $50/mo for every auxiliary service.

But that strategy does not account for those same startups after they grow enough to take a breath and start optimizing costs.

[teej]

The end state is you DIY. Segment isn’t doing anything secret/magic, it’s just well executed and super convenient.

The “how” question has changed every 3 years in the last 10 that I’ve done data work. That trend is driven primarily by data scale and partially that analytics and ML access patterns have evolved a lot.

[bleonard]

We are building Grouparoo (https://www.grouparoo.com) because we think you should be able to do this yourself and own/control the data - without the incentive structure being messed up from the cost pressure.

[texasbigdata]

Do y’all have pricing info?

This post [0], and the open job requisitions suggest you’re using a bit of cash to do this. What is your business model?

0: https://mobile.twitter.com/grouparoo/status/1292844163617239...