Hacker News new | ask | show | jobs
by nnq 4968 days ago
...I never get it why scaling is seen as such a "hot" problem and everybody seems inclined to make language and technology choices based on it ...I bet Twitter could have scaled well keeping Ruby and rewriting performance critical parts in C and maybe keep tweaking and tuning the Ruby interpreter, much along the way the Youtube team did with Python and C and now some Go I hear...

I think their decision was more influenced by the experience of their team of programmers or by employable talent pool - they went the JVM|Java|Scala route because they had people with experience in high level languages and the JVM. If they happened to start with a team with "C hackers" background they would've gone the Ruby|C way and it would've worked as well.

...I think almost any language and technology can (be made to) scale, even to Twitter scale, at least if it's open-source and you have people with the required technology to hack around the internals and recode performance critical parts in lower level languages (basically C, C++ or Go nowadays)
1 comments
willvarfar 4968 days ago
Did you skip the bit in the article where they mentioned the faster Ruby VM they built, before moving to JVM?

http://engineering.twitter.com/2011/03/building-faster-ruby-...

link
nnq 4968 days ago
no, I didn't skip it, and I can only guess that either they considered they wasted to many resources for such a "small" performance gain or didn't have people with enough skill in the right areas to keep going this direction. But no, I don't know all the details and steps of Twitter's switch and my experience is in the Python ecosystem but I think lots of things are similar...
link
Xylakant 4968 days ago
Well, there are several problems with MRI-1.8.x/REE that come into play when you try to go for raw performance. The biggest problem is probably the "stop the world" GIL that occurs around the GC and AFAIK around all external C calls. The second biggest problem is that the lack of multi-core capabilities. Both can bite you pretty badly when you're aiming for raw performance of a single process. They're both not as bad when you're doing web-level stuff where it matters little whether you start one, two or 20 processes to handle your load.
link