[nickzoic]

> But it is expressly not a benchmark of database servers.

Oh, okay, cool. I'd misinterpreted it as being a kind of a "full stack" test. I didn't look at any of the "fortunes" examples, either, as it happens.

I can see where you're coming from, but I'd still be concerned that the concurrency and connection pooling behaviour could be quite different depending on the DB query behaviour.

What I should have said is "... predictive of how well your website will actually work under load". But nothing much is predictive of that except for trying it with synthetic data ...

[bhauer]

You're right that it's impossible to fully disentangle the performance of the database server from the frameworks' ORMs and their platforms' drivers and connection pools. We indicate which database server is being used in each test permutation and one can make very rough observations from the data--such as "MySQL appears very slightly faster in this type of use-case than Postgres"--but like I said, that's not really the purpose of the project. If one wants to compare database servers, there are many better resources for that insight.

You're also right that nothing can predict how your application will perform under load until you build it and test it.

By testing the fundamentals of web application frameworks, however, we hope to inform a preliminary selection process (along with self-selects such as comfort level with code type and community) to give you a rough idea of capacity before you build out the full application. I feel especially that the massive spread of the performance numbers--covering many orders of magnitude as it does--is illuminating to newbies and also valuable to seasoned pros.