The keys to the kingdom are ever more being placed in the hands of relatively few internet custodians. Figuratively here of course, since the private keys are generated locally and never transmitted to LE.
Is it really a single point of failure though? Certificates are renewed well in advance, and there are several free alternatives with ACME support to LetsEncrypt today.
Switching to a new provider in case LetsEncrypt goes down is as simple as updating your scripts.