Because parasitic entities tend to misbehave. There's always been other search engines, Google is in slow decline (or accelerated, depending on your viewpoint) so blocking them because they slop users into oblivion. Sometimes you have to say: This his how far I was willing to let this go and now it's over.
heres the rub, whos to say in the future or even now that google is really respecting the robot.txt in the first place. I know for a fact that my sites are still getting hit by AI bots but I'm "blocking" them via cloudflare. Way too many IP's originating from data center ISP. Just saying respecting the robot.txt maybe just PR lip service.