They are great as they have cuda cores etc., the only downside is you have to use their ubuntu image to utilize all the features as the drivers are proprietary, you can install other distributions but you will lose that functionality.
If you are looking for an -almost- as powerful SBC (with m.2 support as I see you have it in the original design), give a look at Rock5-B (1), I didn’t test it personally unlike nVidia ones, but it seems it has a great specs, as you might also struggle to secure jetson ones sometimes due to stocks being sold out.
I remember I tried Yocto, indeed it’s customizable, but when I tested it, its performance wasn’t on par with the stock one, specifically with gstreamer and vp9 encoding/decoding. Maybe it’s changed now though, worth the try OP.
Heh - I can't resist throwing a mention in for my favorite here too (1). Available, cheap, x86 based... I love them :-) I have 3-4 of the H2 models, and they have only gotten better !