Is anycast "just" (!) broadcasting different routes for different parts of the internet for the same IP address? I cannot imagine this is trivial to get right at all.
Definitely non-trivial! Deep BGP expertise is required to operate anycast at any significant scale. And RTP/WebRTC media traffic is perhaps particularly tricky, because UDP is so stateless but media servers need to maintain a relatively large amount of state for each "connection."