Well, Shopify is investing in optimizing Ruby, so they believe Ruby was the component with the most opportunity for improvement. And the results are showing there was indeed lots of places where improvements could be made.
That's like saying the leading F1 team is investing in optimizing side mirrors, therefore they believe side mirrors are the component with the most opportunity for improvement.
The fact is that performance oriented organizations optimize everything unless they have math telling them it isn't worth optimizing.
> The fact is that performance oriented organizations optimize everything unless they have math telling them it isn't worth optimizing.
Most companies don’t have unlimited budgets. Performance-oriented organizations profile and then spend money where profiling tells them to. Shopify isn’t hiring people to contribute to MySQL or Redis internals. They hired a full team to work on Ruby internals, not just creating YJIT, but also on CRuby’s memory layout, hiring the lead on TruffleRuby, and funding academic programming language research on Ruby.
No company has an infinite budget to “optimize everything”. It is clear where internal performance testing pointed Shopify (at Ruby) and with double digit gains being extracted year after year, their profiling didn’t lie. And other Ruby on Rails shops are seeing similar double digit performance wins, not on fake benchmarks, but on actual page load times and traffic that can be handled by a server.
Disclaimer: I'm a member of the Ruby & Rails infrastructure team at Shopify and Ruby committer.
> Shopify isn’t hiring people to contribute to MySQL or Redis internals
You are not wrong, but the main reason is that contrary to Ruby, MySQL and Redis are used by a lot of huge companies and are themselves owned by companies with full time people on it.
In comparison Ruby is still mostly a volunteer ran project that receive little funding and effort relative to its importance.
As for why trying to make Ruby faster at all in the first place. It's not because it was too slow, it's mostly just that at our scale, the engineering time spent on optimizing the runtime pretty much pays for itself.
But that's nothing specific to Ruby, in most very large software companies you will find similar efforts. e.g. I remember Twitter had a team working on the JVM, etc.
thank you for that insight! If I had to guess, early on shopify had a choice to say jump to Java or a JVM-based language and scrap Ruby altogether, but they chose that the benefits of ruby outweigh the benefits of java's performance. So now given that shopify is large enough spending $1MM to win $5MM on infrastructure yearly is easily worth it.
In any case, I am very grateful to Shopify because I think if they decided to switch over to java back in the day, ruby may actually be a mostly dead language by now.
> had a choice to say jump to Java or a JVM-based language and scrap Ruby altogether
As someone who joined Shopify 10 years ago, my perception is that it wasn't an option.
At the time competition was fierce and the priority was to get new features to grow.
If you are a free service Twitter, with a strong network effect, it makes sense to retool to reduce your costs, as it's a primary factor in profitability.
If you are a B2B paying service like Shopify with little to no network effect to keep your leading position, halting features for years while you re-tool, re-train etc is a death sentence.
My (hot) take is that if Shopify had decided to switch over to Java back in the day, Shopify may actually be a mostly dead company by now.
Most companies don’t have unlimited budgets. Performance-oriented organizations profile and then spend money where profiling tells them to.
The bizarre part here is that anyone who paint themselves into a corner with ruby, then decide that they need to implement an entire new fancy VM instead of just find targeted bottlenecks and rewriting them in C++. They writing complicated native programs to make a VM that is only 15% faster when they could rewrite specific sections that are slow and speed those up by 200x or more.
Of course, most teams do. Some teams have had 2 revisions to side mirrors this season.. and of course it didn't decide the WC, which is my point. Organizations don't merely optimize "the bottleneck" they optimize everything worth optimizing.
Well objectively, one could say both Ruby and RoR are slow. The first via The Computer Language Benchmarks Game[0], and the latter via TechEmpower Web Frameworks Benchmarks[1].
In practice though the database is doing most of the work, most of the time so is typically slower than both.
> "Also that view is largely biased by the typical assumption that Ruby == Rails."
Given that the predominate / vast majority use case for Ruby is for web development and Rails being the dominate Ruby web framework ... it's a fair assumption & characterization for people to make that the Ruby == Rails use case.
Sort of. Twice as many rails hosts means more DB connections which generally means more load/memory on the DB or more load/memory on the external connection pooler.
It's only a bit of incremental load, but it's easy to overlook how many other systems need to run to make Rails scale.
No he’s his examples he’s talking 50 stores over 10 machines vs 50 stores over 5 machines. Both would require the same DB count, but the second would save on server costs for stores.