The reference implementation is not the same as knowing which code runs on "your" hardware and having all keys from "your" hardware. If you are protected by a steel door but you don't have the keys, you are not safe, you are imprisoned.
This seems like a ridiculous argument. There is a fully source open source secure boot implementation, that if you have concerns about it you can fully audit, just as you can fully audit your current setup if you wanted to - but almost certainly have not.
So you are suggesting me to audit an implementation which I do not necessarily run and there's no way to know if I do. What's the point? How will it help?
The code running on my hardware is open, so anybody from the community can audit it and I have a possibility to verify that this is what I run at least by reflashing it. And I did reflash it. This approach is getting more reliable with more software becoming reproducible.
Coreboot supports secure boot, so why isn't that sufficient?