[mcqueenjordan]

Another point of clarification that is of great importance to the results, and is a common Rust newcomer error: The benchmarks for the Rust implementation (in the original post that got all the traction) were run with a /debug/ build of rust, i.e. not an optimized binary compiled with --release.

So it was comparing something that a) didn't do meaningful parsing against b) the full parsing rust implementation in a non-optimized debug build.

[SushiHippie]

Am I missing something? In the git repository [0] it says:

> needletail_benchmark folder was compiled using the command cargo build --release and ran using the following command ./target/release/<binary> <path/to/file.fq>.

Or are you talking about something else here?

[0] https://github.com/MoSafi2/MojoFastTrim

[mcqueenjordan]

It was later edited, after it had basically made the rounds.

[SushiHippie]

Ah okay, found the commit that changed the benchmark numbers

https://github.com/MoSafi2/MojoFastTrim/commit/530bffaf21663...

[tehsauce]

How much does this particular result change when running in release mode?

[alpaca128]

Depending on the code I've seen performance increases above 100x in some cases. While that's not exactly the norm, benchmarking Rust in debug mode is absolutely pointless even as a rough estimate.

[eyegor]

Is there any compiled language that doesn't benefit heavily from release builds? That would be interesting if true.

[pornel]

This can happen in languages that use dynamic constructs that can't be optimized out. For example, there was a PHP-to-native compiler (HipHop/HPHPc) that lost to faster interpreters and JIT.

Apple's Rosetta 2 translates x86-64 to aarch64 that runs surprisingly fast, despite being mostly a straightforward translation of instructions, rather than something clever like a recompiling optimizing JIT.

And the plain old C is relatively fast without optimizations, because it doesn't rely on abstraction layers being optimized out.

[adgjlsfhk1]

Julia, for example runs by default with -O2 and debug info turned on. It's a good combo between debug-ability and performance.

[fwip]

On my machine, running the debug executable on the medium-size dataset takes ~14.5 seconds, and release mode takes ~0.8 seconds.

[adgjlsfhk1]

do you know why debug mode for rust is so slow? is it also compiling without any optimization by default? it's it checks for overflow?

[FridgeSeal]

The optimisation passes are expensive (not the largest source of compile time duration though).

Debug mode is designed to build as-fast-as-possible while still being correct, so that you can run your binary (with debug symbols) ASAP.

Overflow checks are present even in release mode, and some write-ups seem to indicate they have less overhead than you’d think.

Rust lets your configure your cargo configs to apply some optimisation passes even in debug, if you wish. There’s also a config to have your dependencies optimised (even in debug) if you want. The Bevy tutorial walks through doing this, as a concrete example.

[jakobnissen]

That's not right, Rust only checks for overflow in release mode for numbers where its value is known at compile time. In debug mode all operations are checked for overflow.

[estebank]

Integer overflows can be enabled in release mode by modifying your Cargo.toml with

    [profile.release]
    overflow-checks = true

IMO it should have been the default.

[FridgeSeal]

Aahh, my bad. TIL.

[fwip]

Yes, optimization is disabled by default in debug mode, which makes your code more debuggable. Overflow checks are also present in debug mode, but removed in release mode. Bounds checking is present in release mode as well as debug mode, but can sometimes be optimized away.

There's also some debug information that is present in the file in debug mode, which leads to a larger binary size, but shouldn't meaningfully affect performance except in very simple/short programs.

[bhansconnect]

This is not accurate. The blog post used `--release` for it's Rust numbers. The confusion comes from the 50% performance win being specific to running on an M2 mac. On an x86_64 Linux machine, the results are more or less equivalent.