Hacker News new | ask | show | jobs
by bognition 1197 days ago
I love seeing open source alternatives to these major vendors.

IMO the reason these vendors can and do charge so much is not because telemetry software is hard. Its because managing data at large volumes is incredibly challenging.

No matter what solution you pick, you're going to hit scaling issues eventually. Sure you can start with a single instance, but eventually you're going to need to shard, backup, replicate, etc... If you're running a saas business with uptime expectations from your customers then your telemetry systems need to be rock solid.

This means either paying through the nose for a system that is mostly going to work. Or spinning up a team of engineers to active manage and operate your telemetry systems, and then spending years making your system rock solid.

Managing these systems is a challenging problem and its not one that most software engineers want to do or will enjoy. So its difficult to find the right people.

4 comments

Yeah exactly. I rarely if ever run these kinds of platforms for exactly that reason. I've been at small, resource constrained orgs where could not afford the developer hours to keep open source products up to date so we just buy entry-level licenses. And I've worked at huge orgs where we have the people but they have more urgent tasks so handing over $250k a year to a SaaS vendor is still cheaper and less risky than hiring.

I still very much appreciate the open source stuff as it at least keeps vendors honest.

> IMO the reason these vendors can and do charge so much is not because telemetry software is hard.

I always saw it as "they are charging for their polished UI/experience"

The UI of https://zipkin.io/ versus DataDog is kind of... not really in the same ballpark?

Polished UI saves user time, and when your users are engineers it is definitely put into my calculations when we're looking at these venders versus in house.
Wish they actually sold the UI, not tried to bunch it up with data processing/storage at outrageous price even compared to usual cloud offerings
I find myself surprised because I totally would love a job like that but yep, filter bubble and all, guess I just didn't realize mine up until your comment.
We should talk, i'd love to hire you
Shame that you haven't replied. Sounded like a super interesting job.
Are you actually planning on building this? We should talk if so, this is a super interesting problem domain IMO.
And I would still participate!
My email is in my profile and I'm currently free, resting.

Ping me or tell me where to find you, would love to talk.

If you use AWS X-Ray and CloudWatch integration, you can both use open source and pay through the nose.