I've been using Git LFS with several large Unity projects in the past several years. Never really had any problems. It was always just "enable and forget" kind of thing.
Yeah, unless you can avoid large files entirely, or are okay with a separate tool (this pisses of a lot of devs IME), then just don't use git? I don't like that option. I think this is sensational. At this point LFS is a really big deal. I don't think it's going anywhere or that LFS users will be shafted.
This is my experience as well so far. It took 15-20 minutes to learn about it, install it, and setup configs. Since then I haven't had to think about it once.