[gertburger]

Being a python programmer I am a bit worried that one (In this case) needs to use java to overcome a scaling problem. From my experiences with twisted, which is on much smaller scale than the article, I have never found twisted to be anykind of bottleneck.

It may be that since twisted(python) can only use one cpu effectively per interpreter(GIT et al) that it got left behind java which can easily use multiple CPUs for threads.

A slightly different architecture might be required then where multiple python processes are used.

[amix]

So far at Plurk we have struggled to overcome scalability issues - - and none of them are due to Python, even thought we use Python a lot.

Most of the problems we have seen have been related to the database. It was and still is the biggest bottleneck. And all the people I know that drive big sites will confirm this.

This all said, one should evaluate problems and not blindly use Python for everything. Python is a great language, but some other language is a better choice for some problems. And here Java and specially Java's NIO library is a much better choice for doing a server that should handle tons of open connections.

[majek]

You don't have to use Java. That's exactly the reason I hacked together EvServer: http://code.google.com/p/evserver/wiki/Documentation

Because of GIL, there is no point of using multiple threads in Python. Scaling must be done through adding more processes. Than an IPC communication starts to be an issue - that's what messaging middleware is for. The sooner you integrate messaging with your project - the better.

Once you have a messaging platform - comet can be done in any technology you prefer, it really doesn't matter. That's because the scaling complexity is handled by the message broker.

[d0mine]

s/GIT et al/GIL et al/