The answers can be recorded and reviewed. The other points are true, or is there a way to make outcomes deterministic, when compared to previous versions while allowing to add more knowledge in newer versions?
It's possible to make any model deterministic. Used to be just to save the seed, but I'm not sure it still is now that everything is distributed. Maybe a little more effort.
determinism isn’t really enough, we want “predictable”. Most of these AI wavefunctions are “chaotic” - tiny changes in state can cause wildly divergent outcomes
A part of my question that you didn't go into was, can new knowledge be added in a new version without making the answers with knowledge learned in previous versions non-deterministic?
changing the input (data) means you get a different output (model).
source data has nothing to do with model determinism.
as an end-user of AI products, your perspective might be that the models are non-deterministic, but really it’s just different models returning different results … because they are different models.
“end-user non-determinism” is only really solved by repeatedly using the same version of a trained model (like a normal software dependency), potentially needing a bunch of work to upgrade the (model) dependency version later on.