Hacker News new | ask | show | jobs
by no_gravity 2220 days ago
I wanted to play with variations of the code. For that it is useful to make it output a "summary" so you know the variation you tried is computationally equivalent.

For the first benchmark, I added a combined string length calcuclation:

    def main():
     r = 0
     for j in range(20):
       for i in range(1000000):
         r += len(str(i))
     print(r)
    main()
When I execute it:

    time python3 test.py
I get 8.3s execution time.

The PHP equivalent:

    <?php
    function main() {
     $r = 0;
     for ($j=0;$j<20;$j++)
       for ($i=0;$i<1000000;$i++)
         $r += strlen($i);
     print("$r\n");
    }
    main();
When I execute it:

    time php test.php
Finishes in 1.4s here. So about 6x faster.

Executing the Python version via PyPy:

    time pypy test.py
Gives me 0.49s. Wow!

For better control, I did all runs inside a Docker container. Outside the container, all runs are about 20% faster. Which I also find interesting.

Would like to see how the code performs in some more languages like Javascript, Ruby and Java.

2 comments

I've added rust using this code

    fn main() {
        let mut r = 0;

        for _x in 0..20 {
            for y in 0..1_000_000 {
                r += y.to_string().len();
            }
        }

        println!("{}", r);
    }
Surprisingly PyPy is the fastest

    % hyperfine target/release/perftest "php perftest.php" "python perftest.py" "pypy perftest.py" -w 3      
    Benchmark #1: target/release/perftest
    Time (mean ± σ):     624.8 ms ±   9.8 ms    [User: 623.0 ms, System: 0.8 ms]
    Range (min … max):   614.5 ms … 644.0 ms    10 runs
    
    Benchmark #2: php perftest.php
    Time (mean ± σ):     697.8 ms ±  18.3 ms    [User: 696.7 ms, System: 1.1 ms]
    Range (min … max):   650.1 ms … 718.0 ms    10 runs
    
    Benchmark #3: python perftest.py
    Time (mean ± σ):      3.326 s ±  0.071 s    [User: 3.313 s, System: 0.003 s]
    Range (min … max):    3.232 s …  3.419 s    10 runs
    
    Benchmark #4: pypy perftest.py
    Time (mean ± σ):     270.7 ms ±   5.7 ms    [User: 257.5 ms, System: 13.0 ms]
    Range (min … max):   257.8 ms … 277.8 ms    11 runs
    
    Summary
    'pypy perftest.py' ran
        2.31 ± 0.06 times faster than 'target/release/perftest'
        2.58 ± 0.09 times faster than 'php perftest.php'
        12.29 ± 0.37 times faster than 'python perftest.py'
Allocating a string to compute the length and then throwing it away is gonna be real slow, yeah...
Due to Rust's concern about thread-safety, the standard println! macro acquires a lock prior to writing to stdout. This means that if you're doing a lot of printing, it can be quite slow due to the extra work of acquiring and releasing that lock.

This is fairly well known and if it's a bottleneck in your program the recommendation is to acquire the lock yourself and then write directly to stdout.

My Java timing is very close to Rust. I have a feeling PyPy is eliminating the string conversion as dead code. When I alter my Java toy code to not prevent elimination of the toString() call it runs close to speed of PyPy.
> I have a feeling PyPy is eliminating the string conversion as dead code.

If I'm not mistaken, the specific variation of the benchmark shown upthread could not be eliminating the string conversion, as it's adding the length of the resulting string to a variable which is later printed.

> len(str(i))

I found Java's to be quicker than Rust:

Java:

    time java Test
    117777800
            0.60 real         0.52 user         0.12 sys
Rust:

    time ./test
    117777800
            1.94 real         1.89 user         0.00 sys
What flags did you use with Rust?
Both `-O` and `-C opt-level=3`. I'm guessing this is one of those high throughput scenarios where having a GC is quicker because it's able to defer deallocation until the end of the program.
-O is opt level 2, incidentally.

Yeah, I think you’re right. Very interesting!

The next PHP release will sport a JIT compiler [1].

I'd wager it will reach similar performance than pypy.

[1] https://stitcher.io/blog/new-in-php-8

Didn't read yet, but I imagine it'll store the JIT'd code on disk for further requests?

This will be amazing when it happens.