A generative AI may well be deterministic and generate repeatable output for a given inpiut, but that doesn't mean the output is correct for every input. It may merely generate the wrong answer consistently.
Roughly, they are asking whether you can take an AI system and effectively reduce it to an analytic function and tell, without actually running the AI, what the output is going to be with a particular input.