Intuitively I don't see that there's much value in asking radiologists to subjectively "rank" the images. Surely the thing that needs to be tested here is patient outcomes?
That needs to be tested eventually - there's a reason we go from petridish testing to animal testing to human testing with medicine, it stands to reason that medical tools should follow similar stages.