Hacker News new | ask | show | jobs
Announcing Zstandard in Rust (trifectatech.org)
63 points by jmillikin 10 days ago
3 comments
ikawe 10 days ago
It’d be nice if they at least mentioned the existing pure rust implementation.

https://github.com/KillingSpark/zstd-rs

And better yet if they compared it explicitly in their “why” section. As it is, I’m left guessing.

link
merlincorey 10 days ago
The "Why" section exclusively opens with a reference to zstd-rs:

> Why though?

> Using zstd in Rust is already supported via the zstd crate, so why bother with a whole new implementation?

Separately, I'd say their end goal seems to be replacing the C library dependency in Rust's zstd-rs:

> We have our own fork of the zstd that uses libzstd-rs-sys instead of the C library. We'd like to upstream this at some point.

link
soulbadguy 10 days ago
They seems to be referering to https://github.com/gyscos/zstd-rs which looks like a wrapper around the C library. https://github.com/KillingSpark/zstd-rs. is a pure rust implementation
link
zamalek 10 days ago
That is an absolutely massive amount of unsafe, is the plan to reduce it over time? The benefits of the Rust build system are reason alone to use this (as a direct competitor to the C version), but claims about memory safety are suspect no matter how confident the authors are about correctness.
link
mswphd 9 days ago
they're using c2rust. It transpiles c -> rust "directly", e.g. ints map to libc::c_int, pointers map to pointers, array access map to dereferencing pointers at an offset, etc.

Typically the suggested way to use it is to obtain "bad" rust directly from it (but which still works etc), and then make that "bad" rust more idiomatic. I don't remember any publicly visible success stories for this though. The current Bun rewrite is arguably of this form, but not exactly a "success story" in producing idiomatic, safe rust. Other large rewrites that were well-publicized/docoument, e.g. the fish shell from c++ -> Rust, took a different approach (factor into modules/rewrite each module by hand).

https://fishshell.com/blog/rustport/

link
tnelsond4 9 days ago
I was using the original zstd rust crate for my web app, but I couldn't get my module any smaller than 150kb, but when I switched to C for web app I was able to do a whole lot of optimizing and culling until I got it down to 39kb.

I wanted to like Rust, but I was using unsafe code for performance and such anyway that it made more sense to switch to C.

link
elevation 9 days ago
> I got it down to 39kb.

If you're counting KBs and you have to include zstd yourself (a reverse proxy isn't handling it for you) does that mean you're targeting an embedded device? Are you supporting TLS at that size?

link
tnelsond4 4 days ago
39kb includes the zstd decoder which is statically compiled into it. I'm not targeting microcontrollers, but I am targeting smartphones in areas with bad internet connections, so every kilobyte saved is nice. I also just like to optimize things and make them minimal while still being powerful. I don't handle tls, GitHub pages handles all that, I'm just using it to host a static pwa.
link