[SilasX]

It depends on what "not code in X correctly" means. If they missed a few syntactical things, it's fine. If they're obviously still "thinking in a different language", then no. For example, if you ask them to loop over a list of items in Python, they shouldn't write:

    for i in range(len(items)):
        do_stuff_to(items[i]))

[vlasev]

Well, even though it's not idiomatic, it still works. A good question to follow up with is if the candidate is familiar with iterators.

[SilasX]

Right, the point is that such an answer would reveal a more superficial understanding of the language than they might have earlier implied.