[KaiserPro]

Rails is fast, if you've never dealt with anything in HPC.

Also I was more referencing that if you rebuild an app you expect it to be faster.

However:

100 Requests a second and at 20% CPU. Really!?

Considering basically all rails/node.js/php/etc apps do is add frilly bits to data held in a database, there is no real reason why it should be slow.

To put it into context, say you are serving a home page, its maybe 150k of HTML/JS of which 90% of it is the same regardless of who visits the page. (hence why proxies are so effective) Thats 15Megs a second.

Not exactly Uber fast, considering its effectively a dumbarse file server.

To put it in context, your standard linux fileserver will (over NFS) push out 1.1gigabyte a second at 30% CPU (assuming disks allow) From a ZFS file system (which is computing the checksums of each 4k block....)

so yes, rails is slow.

[enko]

You know, when we say "X is fast/slow" we usually are basing the comparison on something even remotely related. If we don't do that, the comparison is basically useless.

Of course rails, and almost any other software, is "slow" compared to HPC. I struggle to name two less similar applications. You might as well say that aircraft carriers are slow compared to F16s - well yes, yes they are.

The rest of your comment indicates to me you have no experience with web programming. Of course the cacheable parts are simply read from a disk, or even memory. It's the uncacheable parts that are the problem. And no, it's not just "adding frilly bits to data held in a database".

You seem to have a bad case of "shit's easy syndrome" - the tendency of people who have no idea what they're talking about to assume that everything is easy, and anyone who disagrees must simply be stupid or at least incompetent. Well, if you can build a web framework that's as productive to use as rails but works 100x faster, you will be a millionaire practically overnight. Have at it. Forgive me if I don't hold my breath.

[KaiserPro]

Oh but it really is adding frilly bits to data.

Have a good long hard look at the data flow of your web app. Take for example a CMS: A user comes along, the web page is generated, The data comes from a database, its encapsulated in HTML/JSON/x and then sent to the client. (either all at once or chopped up into bits and done asynchronously). If you're advanced, you might pull data from an API (still repacking data in a veneer of *ML)

I have a case of "shits already done" syndrome. For 99% of companies any old web framework will do. Those 1% that need a bit extra, spend the man hours on engineering ways round the deficiencies in the framework they chose. Most of that effort goes into expanding the namespace of the database, Because IO is a pain at scale.

I work in VFX where efficiency and scale really matter. I look after 16k cores and ~5+pb of tier 1 disk storage. 1% increase of efficiency in CPU usage is worth heafty cash. I've seen many, many fads. (Map+reduce, rails, mongodb, couchdb) All of them are re-inventions of the wheel (Map+reduce was seen as a task dispatching system, Rails was supposedly going to replace QT, Mongo/couchdb was supposedly going to replace a posix file system, and postgres as well)

We're now back at the stage where Postgres +SQL is awesome, Filesystems are good at storing unstructured data, and pythonQT really isn't that scary after all.

Although node.js and callbacks are starting to become popular now.....

[enko]

I think we are talking about different things here.

I'm well aware that 99% of companies could get by with any old web framework. Should they, though? Of course not. Since we're talking about rails, let's use that example. It's an extremely productive, flexible, capable rapid application environment. And yes, it's not a speed demon, especially when used naively, no-one denies that. So why do companies choose it?

It's not because they're stupid or inexperienced. It's because the tradeoffs of higher productivity are worth the penalty in execution speed. Slightly higher server costs are worth the massive boost in productivity compared to other options. Why do you think it's so popular? If you're going to reply "because people are cargo cult following lemmings", you are simply wrong. The tradeoffs make sense to them, that's why.

So you're a sysadmin. Linux uses python all over the place. Why is that? They could have used any old language. But they chose python, even though it's slower than C, because the tradeoffs - easy to read, write, maintain - are worth it.

VFX could not be a more different environment than rapid web application development. As you say, 1% improvement means big bucks. You know what 1% improvement gets you on the web? Diddly shit. And every few percent raw speed increase on the web, as you drop down into the less dynamic frameworks, costs you massively in dev time, maintainability, hireability. It is a balance people strike. If I could increase my team's productivity 10% at the cost of doubling our server expenses I would take that deal instantly.

I can't speak to the fads you mention, except I also generally disdain them. But progress does happen. Which is why the web sites for the films you help make are probably not just written in any old web framework, they're probably written in rails or node or what have you, because the tradeoffs are worth it.

Remember what site you're on. This is about startups, small companies trying to grow fast. It's not about hyper-optimised rendering farms. Different worlds, dude!

Anyway, I think you actually agree with me, and I with you, if we just step into each other's shoes for a second.

Although I don't know what's so hard about making a couple of animated videos. 99% of the time you can just use any old rendering framework. I mean you just ask for a frame, it generates it, and saves it to disk right? Dude, I could do that with povray in 1995 on a 386. Whyever would you need 16 thousand computers for that? And 5PB of RAID? I can store a whole movie in like 3GB and that's 1080P! Actually my imac can make pretty realistic graphics in games, maybe you should buy one of them? Shit's easy right? : P

[KaiserPro]

so long as you had a 386 with the floating point unit. Ironically for a while the iMac was the only choice until the retina for decent graphics. (and CPU, still waiting for the fucking dustbin)