[rockdoe]

This is understood. RSA depends on prime factoring. (EC)DH depends on discrete logarithms. These problems have been studied by the kind of people you mention, from all over the world, an in some cases massive speedups and breakthroughs have been made. But even after all those years, the problems are still "hard enough". That's what gives them confidence.

I'm not even sure what you would imagine the alternative would be. An algorithm that isn't amendable to mathematical analysis?

Even for symmetrical ciphers, which look far less like pure math, analysis proceeds very much along mathematical lines (differential/linear cryptanalysis), or in some cases, via algebra as well: https://en.wikipedia.org/wiki/XSL_attack

[charrisku]

I can't really say what I imagine the alternative to be (if I could, I'd write it up), but I do know that if it had an inherent complexity in it's mathematical structure, I would image it to be more resistant to attack by a good mathematician than otherwise. At present we cannot prove that any of these algorithms is actually NP complete and the best device in existence to prove the conventional wisdom wrong sits between two ears. I think any algorithm designed to make the human brain less effective at analyzing it necessarily adds security, although I wholeheartedly cede the point that it also makes the flaws in said algorithm harder to find, so there is obviously a trade-off inherent to this approach. I also agree that any analysis will ultimately take a mathematical flavor (it is an algorithm of course, math is essentially the only trick humans have come up with here), but that isn't to say we can't craft one which makes that analysis fiendishly difficult. Thanks for the XSL link, I'm not a cryptographer, so wasn't aware of that.