He can use it such that the model's predictions are still manually verified by him before actually submitting them or whatever. Seems pretty harmless to me.
In his current situation, if he zones out, he produces nothing. In the situation where he's aided by the potentially faulty described system, if he zones out, he produces erroneously transcribed data.