There are vastly more people with at best second hand information and at worst willing to make stuff up than there are people directly involved.
This is why people are allowed to confront their accusers in court and we are suspicious of hearsay.