>After this change, any trusted crawler will continue to see the old copy of robots.txt.
I think this still works for crawlers like Google