It seems that on some level, you have to in order to not just constantly reflecting upon your thoughts and researching facts. Whether you trust a given source should surely depend upon its reputation regarding the validity of its claims.
I agree and by reputation you mean accuracy. We implicitly know not to judge anything as 100% true and implicitly apply skepticism towards sources - the skepticism is decided by our past experience with the sources.
Think of LLMs as the less accurate version of scientific journals.
Accuracy certainly does play a role, but this in itself is not sufficient for preventing an infinite regress – how does one determine the accuracy of a source if not by evaluating claims about the source, that themselves have sources that need to be checked for accuracy? Empirical inquiries are optimal but often very unpractical. Reputation is accuracy as imperfectly valued by society or specific social groups collectively.