Major AI players don’t want to use someone else web index as they may cut it off or jack up the prices etc. major players want to build their own web index
And this is why we see our logs overloaded with ABot BBot CBot etc, every single "AI" company makes their own bot and they all crawl the same pages over and over.