[thinkingfish]

In general, Twitter infrastructure didn't start by building solutions in house. Like most early stage companies, it started by using whatever it can find in the OSS repertoire. But a lot of problems that are non-existing at a smaller scale or at least tolerable become more acute when you scale up.

This is when the developers face a decision: to improve the existing solution, or develop a new one? There are many factors that impact the final decision. Maturity of the technology? Community support/inclusion? How big is the necessary change? What scale can existing solution handle by design? How does in-house talent compare to those maintaining the original? What's the focus/need of the in-house use cases compared to the broader objective of the OSS solution? It should not be a surprise that the answer differs from project to project. But if the decision is to use the existing solutions, it probably won't raise much eyebrow/question.

It is very common for larger-scale operations to write up their own solutions. Because a different scale can change the nature of the problem fundamentally. Google, Facebook and whoever has production fleet above the 10K range tend to build a lot of in-house solutions, often after similar considerations as mentioned above.