Why do you avoid DRF? It's what keeps me coming back to Django over .NET Core or other API stacks (although I have yet to tilt at RoR in earnest/anger).
I just want to start by acknowledging the demo of basic functionality is fantastic and feels very magical, and–perhaps crucially–it has been a couple years since I gave up on DRF completely. With that out of the way, I've run into issues with it when building actual applications. DRF serializers were absurdly slow. Relations were handled poorly. It steers you away from many of Django's standard features, including the forms interface, which is one of Django's best features. The documentation was lacking for rough patches in the API.