[yxhuvud]

Last time I tried to run Truffle on the company test suite it spent an hour processing 1/8th of the suite. Then it was killed by the OOM killer. On a 32 GB machine. To be fair there was only one or two errors during that, so compatibility is at least getting there.

Meanwhile Ruby 2.6.5 chugs along and finishes the whole suite in 8 minutes. Never hitting any unreasonable amounts of memory.

[chrisseaton]

Yes TruffleRuby struggles to run test code because it works against the optimisations we add to make production code fast. For example when you add more profiling to better optimise the hot code it makes the cold code slower, and tests are almost all cold code.

Also our C extension emulation layer used to be extraordinarily slow while we made it work correctly, and it's still rather slow.

It's a challenge but we're working on it.

But TruffleRuby is the only alternative Ruby implementation to even run major applications that I've tried at Shopify.

[pvsukale3]

Unrelated: But thank you for the work you and your team are doing. I wish you all the success.

[ksec]

Agreed. I often felt people give little credit and appreciation to the Truffle Ruby teams.

[yxhuvud]

I'm not certain I buy that argument. Yes, the actual code of each test would be slower than usual, but the things around it, test harness, factories etc should get plenty of execution. And at least in our not so very good test code base, that is where most of the time is spent.

[alberth]

Amazing work and really appreciate all you're doing to better the ecosystem.

Question: (a) do you believe we'll ever get to a day when TruffleRuby/Graal will be able to 100% run Ruby on Rails and (b) how much faster over MRI do you believe it could achieve?

[chrisseaton]

Compatibility yeah I don't think there's any reason we can't be very close to 100% there.

For parts of the Ruby ecosystem running production workloads, such as the Liquid template engine, I see 6x performance today. I think that's a realistic goal in time.

[alberth]

Wow, that's amazing!

Any rough estimates on when the masses will be able to run RoR on TruffleRuby?

(Just order of magnitude timing ... e.g. late 2020, mid 2021, etc.)

[chrisseaton]

If people are motivated to help TruffleRuby then the best thing they can do is test the specs and performance of their gems.

If people have Sinatra apps or similar they could also try those now.

I'm in the GraalVM Slack - https://www.graalvm.org/community/ - ask me for help.

Don't want to speculate more than that.

[BubRoss]

What speed gains do you get? Does it ever seen like a lot more trouble to try to optimize ruby instead of just writing slow parts in a different language?

[chrisseaton]

> What speed gains do you get?

I'm aiming for around 6-10x for real production code.

> Does it ever seen like a lot more trouble to try to optimize ruby instead of just writing slow parts in a different language?

I'm pretty philosophical about this.

People want to write and run Ruby code. I trust they've got their own good reason for that. Taking that as a given, I want to let them run it as fast as they can, with the best tooling I can give them. Instead of 'you shouldn't do that' I want to have the attitude of 'ok we'll see what we can do'.

I do personally really like Ruby and I reach for it pretty much whenever I start a new project.

[BubRoss]

Telling people not to do something is very acceptable in a company. If it is users doing something with a product, that's different, but technical decisions matter.

[joelbluminator]

Are you serious - the whole Shopify test suite finishes under 8 minutes? That's pretty great no? Are there multiple processes running it?

[bretthopper]

That person is commenting about another company, not Shopify. Though at one point in time our test suite did run under 10 minutes (across ~100 agents). Ref: https://buildkite.com/case-studies/shopify