[donovanr]

Some feedback on the quiz:

- a few of the questions were very good, and either spoke to key high level concepts, or were specific while being language agnostic. (e.g which one of these layers wouldn't you need, why wouldn't this type of classifier work on this data).

- too many of the questions were hyper-focused on the minutiae of word embeddings, tensor flow syntax, SQL queries, and recommender schemes.

- many of the questions were constructed vaguely enough that "I don't know" would be the technically correct answer even though I don't think that was what you were going for.

metadata: recent PhD with serious grad courses in ML and working in DL/CV for the past year using a non-tensorflow framework (PyTorch).

[edouard-harris]

This is great feedback. Thanks!

We're constantly iterating on the quiz and it would be great to get more detailed thoughts on it.

If you'd like to do that, please get in touch! (Email in my profile)

[donovanr]

would be happy to, but I don't see your email there -- mine's in my profile (I think!) if you'd like to get in touch

[edouard-harris]

Sorry, realized it wasn't public. Just updated, should be there now!

[ellisv]

Took the quiz and completely agree. Most questions were either overly concerned with detail or too vague.

High-level I don't think a quiz is necessarily the right tool either. Reminds me too much of taking the SAT or GRE.

[edouard-harris]

Yeah we definitely aren't convinced that a quiz is the optimal format for this evaluation.

Statistically, it does an OK job at being an initial filter. My biggest concern at the moment is that it's too coarse of a tool and it might be mistakenly turning away competent people.

Definitely a work in progress. If you have ideas on alternative formats or better questions, please email me. (Email in my profile.)