Hacker News new | ask | show | jobs
by edmundsauto 972 days ago
How do they lie on speed tests? I thought that measured actual data throughput.
7 comments

Presumably, an ISP might detect if a test was being run and unthrottle data speeds for the test, analogous to the Volkswagon emission testing scandal.

edit: strange to post this and find a second post with exactly the same two points submitted while writing. I guess my thoughts aren't that special

This was why Netflix setup Fast.com to use production servers and use data loads that mimic actual streaming video. Early on in streaming ISPs were throttling Netflix traffic when the household streaming demands started spiking (around 6 at night when every house in America would get home and turn on some streams). I believe there was a whole fight over peering agreements related to that... Speedtest.net was definitely getting gamed and may still be or at least Comcast et al was prioritizing burst traffic because residential customers realized the 'up to' home internet service could be pretty bad if your cable lines was over subscribed.
> Speedtest.net was definitely getting gamed and may still be or at least Comcast et al was prioritizing burst traffic

Hell, during the height of this Comcast partnered with Speedtest, and now hav their own Speedtest servers all over the country, so you don't even leave Comcast's network.

> Hell, during the height of this Comcast partnered with Speedtest, and now hav their own Speedtest servers all over the country, so you don't even leave Comcast's network.

Most ISP's do, and if you tell it to use the server of some other provider off-net they'll happily make the excuse that "we can only guarantee bandwidth to the edge of our network."

It's a crap excuse, because, duh, you have no control over the path traffic takes once it leaves your AS on hopefully the best route - but if you have poor connectivity to your peers that's still on you.

I think most larger ISPs current have their own speed test nodes these days.

It’s problematic if they don’t, because if Comcast detects a virgin running a speed test, they can throttle virgin traffic to give lies as results. So virgin need to run their own nodes.

If you kill net neutrality, that behaviour is legal.

Simplest way to do this: good speed on Speedtest, rubbish everywhere else.

Also, by default Speedtest tests the connection from you to the nearest testing node. A lot of times it's the one hosted by your ISP, so the measured speed is the speed inside the ISP's local network and not the speed from you to outer Internet.

I've been using the Google widget when you search "speedtest"

It tests using a node 200+ miles away (they tell you where) AND it actually shows slow numbers when the pipe is slow. Turns out Amazon Video was the culprit, somehow using 80% of a 100Mb connection for a single stream...

> I've been using the Google widget when you search "speedtest"

Google doesn't show me this widget, but the speed test on Google Fiber's website is just a frontend for Ookla's Speedtest I mentioned. So I suspect that the widget does the same or similar stuff. I don't know for sure though.

> It tests using a node 200+ miles away (they tell you where)

"200+ miles away" from your physical location doesn't mean "outer internet outside your ISP". It doesn't mean "far away from your ISP's network" either.

For example, some mobile carriers may operate country-wide, but route everything through Important Central Connector To Outer Internet™ in the capital city. Consequently, internet services think that user's phone is in capital city.

So while you see "City 200 miles away" as a test node, it may actually be the closest one because your ISP's network structure adds these mandatory 200 miles. ISP's internet access node and speed test node may be literally across the street or in adjacent server racks.

Not saying that it is certainly the case, just trying to explain why words on test screen may mean things different from the ones we assume.

> AND it actually shows slow numbers when the pipe is slow. Turns out Amazon Video was the culprit

From your words, it didn't show you slow pipe. It showed you regular one that was busy with streaming video.

Also, ability to watch streaming doesn't necessary indicates actual internet speed since streaming services like Netflix can provide ISPs with the hardware for caching [1] and streaming from inner ISP network. Not sure if Amazon does that though.

> Amazon Video was the culprit, somehow using 80% of a 100Mb connection for a single stream...

Amazon streams with a higher bitrate than most of the competitors. 80 Mbps isn't that surprising given that Bluray disks have 40 Mbps bitrate for 1080p video. With streaming you also preload some video ahead in a buffer, so hitting 80 Mbps is completely normal.

[1] https://openconnect.netflix.com/

The other node is in another country that does not have the same ISP. Again, it seemed accurate, so not sure why your even bringing it up. We could watch it change as we turned on and off the Amazon stream. That's how we figured out that it was this one service, not the ISP or Internet at large

Amazon says how many gigs per hour it uses for different qualities, none went over 2 digits in an hour, so there should have been sufficient bandwidth.

I assume they can prioritize speed test traffic. The Volkswagen method.
I have had my internet dragging to crawl open speed rest and suddenly all of my tabs finish loading and have decent speed for the next 30 minutes just to slow down again. I am considering making a cron job to run wget to load speedtest home page every 20minutes
Yes, exactly this.
Your ISP controls the flow of traffic to many different networks. Speedtest is always on a fast connection to give you the big number. Meanwhile they may aggressively throttle traffic to 80% of the other networks. Fast.com is a better speed test that is more (but not wholly) representative of your bandwidth to the internet at large.
Some ISPs scramble to host their own speedtest or have a special version, like xfinity.speedtest.net
You do traffic shaping and prioritize speed test traffic.
This isn’t lying, it’s measuring something accurately but not measuring what you want it to measure. There isn’t a contract or regulation that they can’t prioritize certain traffic.

It does feel a bit deceptive, I grant. But to me, lying would be if they tricked speedtest into display 10mbit when only 2mbit of traffic was transferred.

Everyone games the tests. That’s why you can’t trust just about any performance test unless you run it for your workload.