Hacker News new | ask | show | jobs
by pietroglyph 2757 days ago
Interestingly, my numbers are completely different relative to each other when compared with yours... I wonder how the number of runs is determined and if this affects the results.

  Benchmarking, the higher ops/sec the better.
  Firefox 63.0 on Fedora 64-bit.
  
  Test 100 needles against 16 dictionaries of hostnames
    -                 Set-based x 1,707 ops/sec ±1.93% (60 runs sampled)
    -               Regex-based x 4,078 ops/sec ±0.88% (25 runs sampled)
    -      Trie-based (1st-gen) x 10,038 ops/sec ±1.41% (64 runs sampled)
    -   Trie-based JS (2nd-gen) x 7,258 ops/sec ±1.03% (40 runs sampled)
    - Trie-based WASM (2nd-gen) x 8,033 ops/sec ±1.08% (44 runs sampled)
  Done.
4 comments

  Benchmarking, the higher ops/sec the better.
  Firefox 65.0 on Windows 10 64-bit.

  Test 100 needles against 16 dictionaries of hostnames
  -            Set-based x 1,647 ops/sec ±2.23% (13 runs sampled)
  -          Regex-based x 1,178 ops/sec ±4.95% (48 runs sampled)
  -                 Set-based x 1,854 ops/sec ±1.94% (59 runs sampled)
  - Trie-based (1st-gen) x 275 ops/sec ±4.61% (56 runs sampled)
  -               Regex-based x 2,947 ops/sec ±2.32% (61 runs sampled)
  - Trie-based (2nd-gen) x 671 ops/sec ±2.60% (47 runs sampled)
  Done.

  -      Trie-based (1st-gen) x 9,311 ops/sec ±1.61% (51 runs sampled)
  -   Trie-based JS (2nd-gen) x 7,121 ops/sec ±1.28% (40 runs sampled)
  - Trie-based WASM (2nd-gen) x 7,983 ops/sec ±2.44% (63 runs sampled)
  Done.
And mine seems to look different than yours, but similarly lopsided.
I got these results on an i7 8700K (not overclocked):

  Benchmarking, the higher ops/sec the better.
  Firefox 63.0 on Windows 10 64-bit.

  Test 100 needles against 16 dictionaries of hostnames
    -                 Set-based x 1,917 ops/sec ±1.21% (14 runs sampled)
    - Trie-based (2nd-gen) x 829 ops/sec ±0.64% (66 runs sampled)
  Done.

    -               Regex-based x 4,475 ops/sec ±0.37% (27 runs sampled)
    -      Trie-based (1st-gen) x 8,624 ops/sec ±0.29% (48 runs sampled)
    -   Trie-based JS (2nd-gen) x 8,848 ops/sec ±0.43% (49 runs sampled)
    - Trie-based WASM (2nd-gen) x 10,441 ops/sec ±0.30% (57 runs sampled)
  Done.
Interestingly, my CPU only boosts to 4.3 GHz during this benchmark and CPU utilisation of the Firefox process only goes up to 8%.
That's what I got too:

    Benchmarking, the higher ops/sec the better.
    Firefox 63.0 on Linux 64-bit.

    Test 100 needles against 16 dictionaries of hostnames
      -                 Set-based x 1,992 ops/sec ±0.71% (15 runs sampled)
      -               Regex-based x 5,148 ops/sec ±0.18% (30 runs sampled)
      -      Trie-based (1st-gen) x 11,797 ops/sec ±0.49% (63 runs sampled)
      -   Trie-based JS (2nd-gen) x 8,471 ops/sec ±0.50% (47 runs sampled)
      - Trie-based WASM (2nd-gen) x 9,543 ops/sec ±0.48% (52 runs sampled)
    Done.
I don't see why trie matching performance would depend on the host OS... Unless WASM runs out of process and requires some form of IPC, I guess?
Interesting that an iPhone XS is the fastest of all the benchmarks above.

Benchmarking, the higher ops/sec the better. Safari 12.0 on Apple iPhone (iOS 12.1).

Test 100 needles against 16 dictionaries of hostnames - Set-based x 3,763 ops/sec ±0.19% (68 runs sampled) - Regex-based x 392 ops/sec ±0.16% (66 runs sampled) - Trie-based (1st-gen) x 20,669 ops/sec ±0.15% (69 runs sampled) - Trie-based JS (2nd-gen) x 16,926 ops/sec ±0.71% (69 runs sampled) Done.

Safari is also a lot faster than Firefox/Chrome on the Trie-based versions (non-wasm)
For some reasons, I found that Firefox's results for WASM are always better after a first run -- i.e. clicking "Lookup" again without reloading the page. I don't know what is the explanation of this.
Firefox can cache the compiled WASM. So perhaps your benchmarks include compilation warm-up on the first run.