Yes, BUT that's not taken into account in these tests. It is given beforehand what you get to train on, and shall way say "there are known problems" on this front.
So the benchmarks say how well model X does on this exact transcription taks given this exact training data, and no other knowledge.
Even basic things, like female/male voices in train vs test set don't match.
So the benchmarks say how well model X does on this exact transcription taks given this exact training data, and no other knowledge.
Even basic things, like female/male voices in train vs test set don't match.