[kjaftaedi]

You don't have to use a third party, you just have to specify an authority.

If you say the self-signed certificate is the authority, that's no problem.

In my private lab environment I have one certificate (for everything).

The certificate is installed on all servers, and even the clients use the same certificate to authenticate, and everyone trusts the cert as an authority.

You'd never run a production setup like this, but there is nothing stopping you.