They control the update servers. So it's possible to target a single user with a single build that no one else ever sees. What percentage of users verify every release?
In theory, Binary Transparency (https://binary.transparency.dev/) solves that among other things. To pass verification, an update has to prove that it's included in a public log of releases.
But I guess Signal doesn't implement it?