You have a lot of people watching the trusted source to make sure that it's append-only and follows the rules for appends. It's a reputation thing; not impossible but it will be detected.
Actually I meant the source tree, and it's because Linus signed the git commit, and thus the tree and all its history. But yeah, the distro works too, which is the whole point: centralized trust does work without a solution to the "no trust" problem. And it works because you build an identity that is consistent over time. I can trust the 4.10 tree because it's signed by the same keys that have been signing kernel keys for years, which are themselves cross-signed by a bunch of trusted identities who have similarly built histories.
This is about tracking the source code using git. The specific thing being guaranteed is that they won't rewrite history; if they rebase you (and many others) will detect it.
Of course you're still trusting them in other ways.