[pariahHN]

Yes but they would have to alter the clients code - this would be reflected in the source and comparing hashes of the binary (as Signal would have to keep that code change out of the public repo). So yes, the design of the system quite effectively prevents a client side backdoor from going unnoticed.

Edit: also, this is a deliberate design choice not an arbitrary one.