Nothing to see here. Author uses Octane benchmark where bigger is better, so v8 ~26 beats the "experimental engine".
Q: What do the scores mean?
A: In a nutshell: bigger is better. Octane measures the time a test takes to complete and then assigns a score that is inversely proportional to the run time (historically, Firefox 2 produced a score of 100 on an old benchmark rig the V8 team used).
Reproduced the lower node 0.11.10 performance on gentoo.
time node core_engine_benchmark.js
as author answered below comment, the performance gain/loss could be from switches hence the combination of latest v8/node
Clearly the blog posting focuses around the second item (as a biggest replacement reasoning) which is available on both new and old v8. I couldn't reproduce the same problem on i.e. spidermonkey cli but it's visible on node 0.10.26 / 0.11.13
Also, they report run times for the process - it's unclear if they are looking at the total score or the run times. If score, then it seems they are looking at it in the wrong direction, as you said. If run times, then those are meaningless in the Octane benchmark, as it estimates how many executions are possible in a fixed time frame, not how long a fixed workload takes to run.
I love what you are doing, so don't get me wrong but it's just a declaration of intent and not binding. Also, not being open source can make people question where actually the speed difference comes from. Did you trade stability with speed? A shortcut for a common operation? Who knows!
Yet this is just a declaration for now, we do not have any intention to keep it closed source. There is no need to trade stability with speed since most of the 'current' performance difference with JXcore comes from less amount of JS <-> native switches and better memory management. Very soon we will be open sourcing it! Thanks
How exactly does JXCore manage to beat years worth of optimisation engineering by Google? I'm skeptical, especially since I can imagine an engineering tradeoff between performance and stability. Maybe V8 is slower because they fixed crash bugs caused by overoptimistic code generation. What assurances does anyone have that their slower but working V8 code won't be fast and broken in a new engine?
It appears that the reported numbers for benchmark X are _the # of times that X can be run before one second elapses_, so parent comment is correct and the premise of the blogpost (that the new node.js version with V8 has worse performance than the earlier version / the private fork has better performance than the new version and slightly worse performance than the old) is contradicted by the evidence presented.
Not a good way to look competent, posting something like this. Countdown until edit or takedown...
Doesn't this line calculate the amount of time it takes to compute a single task? (elapsed time / number of runs = time per run) Doesn't this mean that JXcore is actually faster, since the number would be smaller with more runs?
// Suites of benchmarks consist of a name and the set of benchmarks in
// addition to the reference timing that the final score will be based
// on. This way, all scores are relative to a reference run and higher
// scores implies better performance.
Seems double-silly. First there's the whole "smaller is better... I mean, whoops!"-ness of the blogpost. Second is the general choice of benchmark: JXcore is supposed to be a multithreaded JS engine, so it seems like you'd want to benchmark its multithreaded performance against V8's single thread perf. on some workload that can take advantage of multiple threads.
Source code protection? I guess this is some form of security by obscurity when distributing to clients? If it's for in-house code, why would you limit it to the set of data covered by this framework rather than proper encryption for your whole project?
Q: What do the scores mean?
A: In a nutshell: bigger is better. Octane measures the time a test takes to complete and then assigns a score that is inversely proportional to the run time (historically, Firefox 2 produced a score of 100 on an old benchmark rig the V8 team used).
https://developers.google.com/octane/faq