Because the process is well designed. No single person or single party was left alone with ballots, marked or unmarked, at any point. Any counting machines get spot-checked. Any count discrepancies from voter rolls to ballots cast trigger manual counts. Packages of blank ballots are sealed. Voting machines are locked, so no one can easily get into them to add/remove/replace ballots without the key(s). Used ballots are sealed at the end of the election. Signed chains of custody for everything. Etc.
Multiple people in both parties, more or less constantly observed by completely unrelated election observers? That is probably the single hardest scenario possible to execute a conspiracy. Conspiracies are hard enough when there are back rooms and hidden corners. Ballot counts are designed to have none.