[parkovski]

To get the best performance out of HHVM, you have to let it "warm up" (JIT the functions you're using), and compile the bytecode ahead of time. Otherwise it assumes you could include new files it hasn't seen before, and has to skip out on some optimizations to accommodate that. So he can't claim he's faster when he's running it like this: @exec('hhvm ' . __FILE__ . " $test");

It's also worth noting that Facebook is actually pursuing this same approach with Hack (making the language more optimizable). And they've tried the AOT approach, and it turned out to be a huge pain for several reasons.

[ircmaxell]

Please look at the test script again. It does indeed warm up the JIT.

The timing results come from within HHVM. Basically, the test is dispatched within HHVM, isolating just the function call itself. It's then executed 5 times (timed independently) and then the bottom 2 executions are thrown away. See: https://gist.github.com/ircmaxell/8859d7f1bbee7b2f6e16#file-...

So the warm-up period happens. It's not just "benchmark `hhvm file.php`". The benchmark happens inside of HHVM.

So yes, JIT warmup is accounted for 100%. In fact, PHP, HHVM and Recki-CT all execute with the exact same pre-compile steps (parsing and any pre-compiling are ignored in the benchmark).

And their AOT approach that they tried was all-or-nothing. Meaning the entire code base had to be converted. This is a "compile what you want, and run the rest normally". In fact, this is 100% compatible with Hack and HHVM...