[pg]

Curiously, it's inherently impossible to compare what I think matters most for languages: solving new problems. To compare two languages you have to have people solve the same problem, and that means it has to be a predefined one.

[gruseom]

I agree. Predefined problems are one way people compare languages; code snippet comparisons are the other. Neither touches the important thing, which is how languages differ as a medium for thought. Different languages give rise to different ideas, which give rise to different programs.

The other problem with studying this question formally is that it is so psychologically rooted that people almost invariably "find" that their favorite language turns out the best.

[trapper]

This is what statistics are for, and is exactly the same situation that many other disciplines find themselves in where treatment a influences treatment b.

[bOR_]

Not the easiests of problems though. Do you know what statistics you would apply?

IQ measurements seem to have a similar problem: doing test a, influences your score on test b, because you get more used to the problems presented.. but doing test a and b just marginally increases your general intelligence, which is what you are trying to measure, while it might have a big influence on someone's score in these tests.

With IQ tests, you can expect that doing test c till z will eventually saturate someone.. but that might only work because you generally do not get the answers. In coding, you do get the answer back (whether your code compiles/works or not), so eventually both programmers will just be limited by the speed with which they type.

[trapper]

>Not the easiests of problems though. Do you know what statistics you would apply?

There are many ways. Off the top of my head:

- to focus on speed of development you could use a practice or warm-up

> provide each with a practice in another language

> provide a sample solution in pseudocode in both functional and imperative style

- To focus on speed of problem solving you could use topcoder data + research of participants programming language backgrounds.

The important thing is that if the effect is large it's easy to spot. According to some, the difference is an order of magnitude. If that's true, it will be trivial to spot.