Hacker News new | ask | show | jobs
by KirinDave 5797 days ago
Is it just me, or did Zed not describe his testing methodology in any detail?

I can't even find a reference to his OS configuration and version details that he's developing on, which seems to me like a critical detail.

1 comments

There's the pipetest.c file that everyone uses (since 2002) linked off that blog post, but I got tired and went to sleep.

Today I'm crafting how I ran the tests and releasing all the code and asking everyone to test my results. I am completely assuming I am wrong so looking for other people to test it.

Incidentally, if you google for "pipetest.c" you'll it's kind of the gold standard for this comparison, so if that code is wrong, then the entire assumption that epoll is better needs to be redone.

Okay. And I appreciate that, I'll look.

To make your process scientific, I'd like to suggest you add the following things to the post when you find it convenient:

1. A detailed explanation of your methodology, preferably with source code. This is so we can reproduce the tests. The ability to reproduce your work is a critical part of any process calling itself science.

2. A detailed list of the hardware you used & its deployment. (For reasons listed above).

3. Your raw data should be made available upon request so other people can work it as well.

P.S., aren't you concerned about I/O overhead with your superpoll proposal? It seems like the added resource allocation and the time spent in zeromq is going to eat up the small advantages you gain?