Have you tried knockd https://linux.die.net/man/1/knockd ? You send a special sequence of "knocks" to the server (packets to different ports) and it executes a command such as allowing your IP for a time period. No JWTs.
My go-to is fwnop (https://www.cipherdyne.org/fwknop/) which is actually similar to the OP’s thing but battle tested. Only downside is it’s not available on iOS, so recently I setup WireGuard for my iPad.
And yes, ssh pubkey + fwnop + WireGuard + fail2ban is ridiculous overkill, but hey, it’s my homelab server. That’s how I learn this stuff.
And yes, ssh pubkey + fwnop + WireGuard + fail2ban is ridiculous overkill, but hey, it’s my homelab server. That’s how I learn this stuff.