Well, if you've defined valid representation (and if not, you can by just copying the specs from the Python constructor functions you are calling, whose documentation has the definition they support), match to that before calling the conversion function rather than trapping conversion errors.
Even with the exact same representation, it will speed up the no cases at the expense of slightly slowing down the yes cases (assuming that that is also what the function itself does), but since this is called on tokens with no a-priori reason to expect a match as the normal case, you’ll have more no than yes, so even if it was symmetric speed swap that'd be a win (and I think the existing way you lose more to each exception that you gain avoiding duplicating the check on matches, so it should be a bigger gain that you'd expect if it was symmetric.)
Even with the exact same representation, it will speed up the no cases at the expense of slightly slowing down the yes cases (assuming that that is also what the function itself does), but since this is called on tokens with no a-priori reason to expect a match as the normal case, you’ll have more no than yes, so even if it was symmetric speed swap that'd be a win (and I think the existing way you lose more to each exception that you gain avoiding duplicating the check on matches, so it should be a bigger gain that you'd expect if it was symmetric.)