Hacker News new | ask | show | jobs
by out_of_protocol 635 days ago
As I understand article, there are new API (unofficial , for now)

- Jetstream(1) (no compression) and

- Jetstream(2) (zstd compression).

And my comment means (1) not really needed, except some specific scenarios

1 comments

It's impossible to use the compressed version of the stream without using a client that has the baked-in ZSTD dictionary. This is a usability issue for folks using languages without a Jetstream client who just want to consume the websocket as JSON. It also makes things like using websocat and unix pipes to build some kind of automation a lot harder (though probably not impossible).

FWIW the default mode is uncompressed unless the client explicitly requests compression with a custom header. I tried using per-message-deflate but the support for it in the websocket libraries I was using was very poor and it has the same problem as streaming compression in terms of CPU usage on the Jetstream server.

> It also makes things like using websocat and unix pipes to build some kind of automation a lot harder

Would anybody realistically be using those tools with this volume of data, for anything but testing?

It's not that much data. Certainly nothing zstdcat can't handle.