[dragonwriter]

> However, the DM is not technically supposed to just "change the rules as you go" to decide whether or not something is successful

That's...highly variable. Page 236 of the 5th Edition Dungeon Master’s Guide addresses three valid approaches with pros and cons (nominally to the dice, but because of the role the dice play implicitly to the rules as a whole) ranging from using the rules, stats, and dice rolls for everything, to using the DM’s narrative judgment for everything, to a hybrid blended approach. (Previous editions of the rules had similar license to the same range of approaches with different framing.)

[bigstrat2003]

I would still say that in no case is the DM supposed to just change the rules as they go along. Whatever the method used by the DM, it should really be consistently applied.