[jstimpfle]

> So we’ve managed to just smash a C program that was looked at by thousands of eyes of quite hardcore low-level Unix hackers over a few decades. We did this with a handful of lines of pure, mutation-less, idiomatic Haskell, achieving about 4 to 5 times of throughput of the C version and spending less than an hour on all the optimizations.

I've done many very arrogant things in my life, because I've been a strange guy with lack of self-esteem who doesn't have a clue how many things he doesn't know. I hope I've never been this arrogant, though.

[0xd34df00d]

Those are fairly trivial and well-known optimizations that I did (and I by no means am an expert in writing high-performant code), so all the honors go to GHC authors.

[jstimpfle]

Thanks for replying. I want to tell you that I feel deep regret for my impulse to publically shame you - even though I've gotten a lot of points for this comment, and did not really receive criticism for it.

Hey - if you make performance optimizations and compare implementations, it's probably best not to jump to quick conclusions. I would advise to brush up on C to get a feel for performance. Or, in times where it's popular to shit on C, and claim that it's not really a low-level language anymore, I would advise to write some assembler (which I've barely done). In practice it's unlikely that you find yourself in a situation where you can write code in a high-level language that runs considerably faster than what you could realistically write in C. The only situation where that can happen that I can see, is when the program does something that is so complicated but seems so arbitrary that you simply can't bring yourself to invest more time than what you need to hack together a quick Python script, and what you would be willing to write in C would be not even the asymptotically best approach that you can see.

For a simple program like wc, that situation does not apply, and if you find surprising results, it's best to first check for other possible reasons than "thousands of graybeards were wrong".

[0xd34df00d]

No worries, that's a natural reaction!

> In practice it's unlikely that you find yourself in a situation where you can write code in a high-level language that runs considerably faster than what you could realistically write in C.

I do way more C++ (in fact, I don't do pure C at all), and aliasing has bitten me and my code performance more often than I'd like. While there are workarounds, I'd probably consider spending time and effort on them as rather unrealistic in a sense. So it surely doesn't contradict my world model if a language with a stricter type system (Haskell? Rust? ATS anyone?) achieves better results on at least some of the tasks with less effort and less dependence on implementation details.

Although ironically I'm going to write something low-level for the Haskell bytestrings library today evening, in C with intrinsics (so almost assembly modulo stuff like register allocation).

[hnlmorg]

The github repo description is equally distasteful too:

> wc implemented in Haskell (significantly faster than GNU coreutils version — oops I did it again

For reference, I'm referring to the "oops I did it again" part. It's really hard to take that comment as "honours go to GHC authors".

Also, I suggest you try running the GNU wc with unicode turned off because unicode is computationally expensive and you're deliberately disabling unicode support in your own code anyway. I appreciate you said you'd add in the edge cases that GNU does in your next blog post but disabling unicode in GNU for this benchmark would show good faith that you're at least trying to compare like for like. And if GHC still out performs then you can at least legitimately say:

> My code outperforms GNU for non-unicode strings

Which currently you cannot because your claim is based on incorrect benchmarks.

[0xd34df00d]

> For reference, I'm referring to the "oops I did it again" part. It's really hard to take that comment as "honours go to GHC authors".

Was overly excited when I created the repo after obtaining the first results. Childish indeed, thanks for reminding, fixed.

> Also, I suggest you try running the GNU wc with unicode turned off because unicode is computationally expensive and you're deliberately disabling unicode support in your own code anyway.

I tried running wc as `LC_ALL=C wc file.txt`, and it (surprisingly for me) resulted in worse run time for wc (my default locale is ru_RU.UTF-8 for comparison). This reproduced on two machines of mine and also on a machine of a friend of mine who also gave my code a shot.

My bad for omitting this in the post, I'll update it accordingly.

[hnlmorg]

wc running slower seems wrong. I wonder if it's doing additional sanitising because your LANG differs from your LC_ALL. I'd need to read through the code to get a handle on what it does and doesn't expect though but I definitely wouldn't expect wc to run slower.

[pinkfoot]

Perhaps - just possibly maybe perhaps - he was suggesting its is his superior tool and technology that allowed his team to do this.

Do you think you took the most generous possible interpretation possible?

[jstimpfle]

Given that I was reminded of myself, I concede it might be closer to the least generous one.

[armitron]

He is comparing apples and oranges and his conclusion can only be characterized as widely misleading and off the mark.

The most generous possible interpretation is to say that he's not being intentionally misleading but actually believes what he writes. I fail to see how that improves matters.

[AnIdiotOnTheNet]

> he was suggesting its is his superior tool and technology that allowed his team to do this

He really thinks it makes sense that a "C program that was looked at by thousands of eyes of quite hardcore low-level Unix hackers over a few decades" is that much slower than something he threw together? That's like doing some back of the napkin math in a bar and declaring relativity wrong (while also not being Edward Witten). Any reasonable person would suspect that this isn't a very likely outcome and dig deeper to figure out if that's really true or not. My guess is he just implemented a very trivial and naive version of the C code and that's why it is faster.

[dependenttypes]

It would be great if your post added value in the discussion.