[pbsd]

It's not just important that they pick up things fast. If you ask to do some task in Python, that would be done much faster in (say) C#, they should be able to point this out, instead of just following the recipe you gave them. Good judgment is important.

[blacksmythe]

That is a hard judgment to make, unless you know both Python and C#. Any of the candidates are going to be able to do the task far quicker in the technology they already know.

[pedrocarvalho]

Yes, agreed. What I'm trying to see is how they behave with something they're not used to, how they take on the problem. These are some of the things I try to analyze:

Do they search and read about the problem? Are they able to implement what they just researched? Are they looking for recipes and trying to copy and paste the code from some website? Do they have critical thinking (what you mentioned) and propose better solutions with better tools? Do they communicate during the problem solving stage, or are quiet, closed and isolated? Do they ask questions? Or behave like asking questions is a sign of weakness?

[rhizome]

Keep in mind that a person who passes your test well will be more expensive.