That's a very fair point, but on my system telnet is 211144 bytes. How big is a javascript runtime + browser + browser sandbox. I have no idea, but I'd be really surprised if it was less than 3 orders of magnitude bigger, and not at all surprised if it was 4 orders of magnitude bigger. There's just more places for things to go wrong.
And, telnet isn't installed by default on many systems. So...YMMV.
Are you being hyperbolic or do you seriously think the attack surface area of ANSI escape sequences is 'much more' than, say, Javascrpt?