[xiphias]

Hi, I just went through the interview questions, they were fun (they said that I did exceptionally well, but nothing concrete, like percentage). I'm not looking for job, just though I try it out.

What I was interested in is whether the questions get harder, if I answer well, but they seemed random.

You could use logistic regression to estimate the level of an interviewer and adjust the questions to get to the same accuracy with less time (or to improve accuracy with the same number of questions/time)

[ebakan]

So you're right that the quiz does try to be harder if you're doing well, but it'll also give you easier questions if an incorrect answer lowers its confidence in your ability estimate. We have a pretty sizeable bank of potential questions to ask a candidate, but the quiz tries to strike an optimal balance between appropriate difficulty and maximum informativeness. For example, we wouldn't want to as you a particularly difficult question unless we're confident that it's a) a good fit for your estimated ability level, and b) will give us more information about your ability than any other question in the bank.

You're right that tailoring question difficulty to ability level can drastically increase a test's accuracy. But while a logistic regression model works well when you have a fixed quiz or a low number of questions, it isn't flexible enough to work with a fully adaptive system like we have at Triplebyte. Our models are loosely based on the kinds of systems that the MGAT or GRE use, but we've implemented significant extensions on top of those approaches to fit our needs.

[xiphias]

Thanks for the answer. When I was implementing a language learning program (who hasn't? :) ), using logistic regression was working quite well to quickly find my vocabulary level in about 10 questions in the top 10000 most frequently used words list adaptively (I ran a full logistic regression on the user dataset after each new data point, by mapping the position of the words to the estimated level of the user), and the questions just felt right. So I'm not talking about multiple logistic regression model, just using 1 variable, which works with lots of questions (as long as the question hardness is well calibrated).

Although I'm happy that you're trying to predict the most informative question, for me some questions near the end felt trivial, so either my feeling wasn't right about the hardness of a question, or the algorithm has lots of space to improve, or the question hardness levels weren't calibrated optimally.

Anyways congrats for the success for your startup (I just hope that you prioritize people who don't have U.S. VISA)!

[euyyn]

Yeah GRE does that. They just use your answers as some sort of dichotomic search: they start giving you harder (or easier) questions until you are answering correctly ~50% of them.