[stygiansonic]

Talking about a project seems fine to me.

But the self-assessment on a scale of 1-10 just seems like a quick way out for the interviewer and can only serve to find fault with the candidate and to me, doesn't seem like it provides much information.

For example, what does 10 mean? Does that mean that they have mastered the language (and all of its minute details, implementation, etc.) and would find it impossible to learn more? Knowing that, the only possible individuals who would think about rating themselves a 10 are:

1) The language designers themselves. (Possibly not even - see the comment below about Bjarne Stroustrup)

2) Those that think they know the language so well that they would rate themselves a 10.

You're probably not interviewing people from group (1). (Or if you were, you wouldn't ask them this question as it would reflect more on yourself than the candidate) So, if someone rates themselves a 10, you can probably assume they are in group (2) or have otherwise misunderstood the question. I don't know for sure, but I feel there are better ways to rate a candidate other than asking them a question, that, if they answer "10" on, means they aren't suitable and are likely overrating themselves.

So where does that leave us? Most people who are comfortable with the language are likely to rate themselves a 6, 7 or 8, being conservative and not wanting to expose themselves to a tricky follow-up question should they self-rate too highly. Again - this seems to be leading toward gotchas, and seems adversarial.

If you're going to ask for the self-rating, at least limit it to something less fine-grained: Beginner, Novice, Expert/experienced/whatever, so you have an idea for the candidate's comfort level with a particular language. A 1-10 rating is way too granular (what's the difference between 3 and 4?).

But really, I think asking the candidate to rate themselves is really just a spin on the "What's your greatest weakness?" question - it's asking the interviewee to do the interviewer's job.

[web007]

You ask them to rank themselves on a scale from 1 to 10 to see if they know their limitations. I used to answer 10 for some things when I was young and dumb because I had no perspective of what expert-level knowledge really was. Now that I'm older (and still dumb) at least I know how much I don't know.

When they do give themselves a midrange answer N, the appropriate follow-up is "What do you know that a (N-1) doesn't?" and/or "What would does a (N+1) know that you don't?" Either of these will let you calibrate your internal rating system with their self-assessment. It will also let the candidate demonstrate their understanding of the topic in a non-adversarial way. The follow-up questions might convince them that they should change their initial ranking as well, which is also a useful calibration of their knowledge and meta-knowledge.

[smil]

So according to this logic, nobody should have hired you back in the day?

[ericlippert]

The solution to this problem is to follow up the question with "oh, so you're an eight, great, what is something that a seven would have difficulty with?" Make them calibrate the scale for you.

[reagency]

Why not just give them a scale and ask them where they fit? Quizzing them about in the structure of the scale is awkward game-playing.

[cpitman]

Or flip it around, and ask them what they are going to do in order to get to 9. It shows how self aware they are of their own knowledge gaps and also calibrates the scale.

On the off chance they say 10, well, they better know just about everything.

[nmrm2]

(TL;DR: Multi-purpose languages are pretty large and used in particular ways across different domains. It doesn't even make sense to ask about the difference between a 7 and an 8. An honest interviewee with become bogged down realizing they're not a competent social science researcher who can design useful fine-grained rubrics, and will refuse to give any answer to these questions except a long-winded and hopefully not too ego-bruising "this is stupid/doesn't make sense/it depends/I'm not a 1. Everyone else in the 3-8 range is just bullshitting the +/-1,2 deltas. So in other words, the entire number and reasoning is completely just BSing.)

> what is something that a seven would have difficulty with?"

My honest answer is "this is an ill-posed question and there's no way of stating for certain that feature X or library Y is something every FooLang developer who's not as good/informed as me doesn't know."

But of course that won't get me the job. So I just pick based upon my perception of your personality/background from the 5-8 range. When asked for justification, I choose the most esoteric-but-not-inconsequential thing I know and bullshit for a minute or two while being personable. Whelp, that was a useful signal as long as your goal as to find a professional bullshitter...

Concretely, I've created stand-alone implementations and hacked on compilers for a couple of languages, but would not have a non-BS answer to this question for those languages. Mostly because I'm totally unfamiliar with the most recent set of popular libraries in that language's ecosystem, and also there are certain language features that I know how to implement but don't know the canonical usage patterns/idioms. Your standard FooLang dev could probably hack together certain programs in their sleep that would take a week for me to write. But then for other programs -- or especially if you needed a static analysis or optimization -- I could whip it up in a couple of days where the standard FooLang dev would probably take weeks just getting up to speed on the language spec and some particularly thorny parts of the implementation. Hm. Apples and Oranges. And what if you want a static analysis/optimization that exploits idiomatic use of FooLang's whizz-bang BarLib? Hm. Not sure who has the upper hand.

I know you'll say "but that's the point -- I want to hear exactly that reasoning so I know what you know/how you assess language familiarity." But this is all meta -- performance on this question crucially depends upon 1) ability to bullshit (i.e., come up with plausible-sounding hypotheses and give plausible reasons for them, quickly and on-the-spot -- this is not a particularly crucial skill in software development); and 2) ability to intuit the purpose of the question.

In conclusion, I'm not really sure what the point of this exercise is, except to select for bullshitters or people who read your HN posts.

> Make them calibrate the scale for you.

You're not just asking for a calibration, you're asking for a rubric and a scale. That makes a lot of sense as an interview task if you're hiring a manager who will be doing lots of interviewing or a business processes/HR processes expert/researcher. I'm not sure why this is a relevant skill for a software engineer, though.