[readdit]

I like and use HyperDX in production and like it a lot. So kudos to the team for building and merging with Clickhouse. I found a lot of monetary value switching over to HyperDX considering it's significantly more cost efficient for our needs.

Should we be starting to prepare for the original HyperDX product to be deprecated and potentially move over to ClickStack?

[mikeshi42]

First off, always really excited to hear from our production users - glad to hear you're getting good value out of the platform!

HyperDX isn't being deprecated, you can probably see on the marketing page it's still really prominently featured as an integral part of the stack - so nothing changing there.

We do of course want to get users onto HyperDX v2 and the overall ClickStack pattern. This doesn't mean HyperDX is going away by any means - just that HyperDX is focused a lot more on the end-user experience, and we get to leverage the flexibility, learnings and performance of a more exposed ClickHouse-powered core which is the intent of ClickStack. On the engineering side, we're working on making sure it's a smooth path for both open source and cloud.

side note: weird I thought I replied to this one already but I've been dealing with spotty wifi today :)

[HatchedLake721]

Still confused where HyperDX ends and where ClickStack starts.

Is HyperDX === ClickStack?

Is ClickStack = HyperDX + something closed source?

Is ClickStack just a cloud version of HyperDX?

Is it same thing, HyperDX, rebranded as ClickStack?

[mikeshi42]

This is good feedback to make things more clear :) HyperDX is part of ClickStack, so ClickStack = { HyperDX, ClickHouse, OTel }. This is the stack we recommend that will deploy in seconds and _just work_, and can scale up to PB+ and beyond as well with some additional effort (more than a few seconds unfortunately, but one day...)

HyperDX v2, the version that is now stable and shipped in ClickStack, focuses more on the querying layer. It lets users have more customization around ClickHouse (virtually any schema, any deployment).

Optionally, users can leverage other ways of getting data into ClickHouse like Vector, S3, etc. but still use HyperDX v2 on top. Previously in HyperDX v1 you _had_ to use OTel and our ingestion pipeline and our schemas. This is no longer true in v2.

Let me know if this explanation helps

[wvh]

What's your opinion on OTel when trying to keep things small and performant? I've got some experience working with OTel the last few years, and I'm a bit afraid of the expanding scope and complexity compared to "simpler", more targeted solutions, like for instance Vector.

I'm just asking because you mention OTel and "other ways" in your post, and you must have a good overview over the options and where the market is headed.

[mikeshi42]

It's actually not clear to me that Vector is any simpler than OTel imo (VRL is way more complicated than OTTL for instance). You can also use otel collector builder (ocb) to build a slimmed binary.

My take is that OTel is overall the best investment, it's widely supported across the board by many companies and other vendors. It's also constantly being improved with interesting ideas like otel-arrow which will make it even more performant (and columnar friendly!)

We'll also continue invest in the OTel ecosystem ourselves in making it easier and easier to get started :)

That being said, I'm not saying that OTel collector is always the right choice, we want to meet users where they are. Some users have data that gets piped into S3 files and we ingest off of a S3 bucket just due to how they've collected data, some use Vector due to its flexibility with VRL, focus on logs, or specific integrations it provides out of the box. So the answer is always - it depends :) but I do like OTel and think the future is bright.

[wvh]

Thanks, it's good to hear opinions from those in the trenches at the frontline, as it were.

[3dteemu]

I'm also a bit confused. I'm using HyperDX cloud and sending telemetry directly from NextJS. What's the benefit of using ClickStack compared to HyperDX cloud?

[mikeshi42]

ClickStack is currently just open source - so there's no cloud or a fully hosted offering yet! (Of course you can always pair ClickStack with ClickHouse Cloud to have your ClickHouse hosted for you).

But in this case there's probably no reason for you :) These improvements will come to our cloud offering of course as we work on rolling out upgrades from HyperDX v1 to v2 in cloud.