[nothing0001]

Just make a loss function that gives +1 for correct answer, -1 for incorrect answers and 0 for unknown. Since this idea took me like 10 seconds, I suppose something like this must/should have been used before.

Unrelated, what happen when we train a model using a discontinuous function? Could the trained model be used to detect some pattern in the data?, for example if the input vector to such a model is a direct sum of two independent variables could such model be used to detect that the problem can be decomposed in two independent problems. Sorry of being off-topic and thinking aloud.

Edited: The following result is from (1): In summary, while softmax classifier probabilities are not directly useful as confidence estimates, simple statistics derived from softmax distributions provide a surprisingly effective way to determine whether an example is misclassified or from a different distribution from the training data, as demonstrated by our experimental results

(1) https://arxiv.org/abs/1610.02136