[amluto]

> But I'm not confident that even that guarantees adversarial robustness: for instance, there may be some clever way to efficiently implement a nonzero polynomial that vanishes at every small integer (say, every integer with absolute value < 10^100) and that would break this hardened variant.

That’s a lot of zeros.

I wonder if there’s a construction with the property that, for all pairs of circuits with size below some threshold (with size appropriately defined), then, if the circuits are not algebraically equal, then, under randomization of whatever parameters are randomized, their embedded versions are, with high probability, not equal and differ in at least a constant fraction of inputs.

At the very least, I imagine that most ML kernels have polynomial-ish order far below 100 in a sense where e^x is defined to have low polynomial-ish order. sin and cos might be worse due to actually having infinite zeros, but maybe pi isn’t constructible and no one can find those zeros.