Hacker News new | ask | show | jobs
by kevin_nisbet 1158 days ago
So I had this conversation with one of the traffic shaping vendors many years ago. At the time when encryption still wasn't that common, they didn't sound too worried about the shift to all encrypted connections. Their product was already starting to use a behavioral analysis to shape the traffic most of their customers were interested in.

So to detect bittorrent, they'd build a profile about how many bit torrent clients operate, the packet and connection creation patterns used, and then slap a throttle on. Looking at some independent analysis, these products might only detect 50% of the bittorent traffic, and have a false positive rate, especially for bittorent users also doing something else. And the ISPs don't care, they get what they need if they clamp 50% of the traffic.

So I'm not disputing that everything encrypted is a good thing, just pointing out that because it's encrypted doesn't necessarily mean the shaping equipment can't figure out enough to throttle bit torrent.

> If I had to guess, I'd say that they incorrectly thought that some specific IP address (range) serves predominantly one type of data. So they throttle by the only data point they have, destination ip, and the collateral damage is everything else hosted on that ip address.

This is plausible. As I recall, the way some of the equipment worked was it would sniff out DNS requests, and then mark the IP address as this destination. So if someone set's a rule for example.com, it might accidentally apply to alice.com using the same IP address.

My knowledge on the industry is out of date though.

1 comments

This is great insight. Thank you for that.

I imagine lots of people are or have spent lots of money and time trying to figure out the type of data or connection from patterns as you say.

A more nuanced and correct statement would have been to say that it's much harder to do than it used to be, when you could just look at the mime-type or similar to figure out what to throttle.

If you're interested in learning more, deep packet inspection, and specifically "encrypted traffic classification" are fairly mature in industry. Many traffic shapers are using products like Enea's Qosmos ixEngine or home-grown equivalents which can identify thousands of applications and protocols.

Most providers that use traffic shaping don't care about content, and the encrypted traffic classification is enough to make traffic policy decisions.