[richbhanover]

I'm surprised that no one has mentioned bufferbloat yet. In most home routers, people can see seconds of delay when there's significant other traffic on the link. (And remember, "other traffic" can be ordinary web browsing, where pages average 2 mbytes these days...)

The latency (and jitter) caused by bufferbloat has been a solved problem for almost five years: fq_codel, and the newer cake qdisc's are are in the Linux kernel, and people's home routers now could install LEDE/OpenWrt (https://lede-project.org). Two links:

What can I do about bufferbloat? https://www.bufferbloat.net/projects/bloat/wiki/What_to_do_a...

Test your network for bufferbloat: http://dslreports.com/speedtest

[minikites]

I believe Ubiquiti's routers also implement fq_codel: https://community.ubnt.com/t5/EdgeMAX-Updates-Blog/EdgeMAX-E...

> Add new "smart queue" feature providing FQ-CoDel + HTB function and this can be configured in the Web UI to provide better QoS experience for the WAN connection.

[ra1n85]

Bufferbloat is a side affect of congestion or microbursts, and not the primary cause of poor voice calls. Either packets get dropped (shallow buffer) or queued (enough buffer), and in both cases the quality of the call will suffer.

Mitigation: Don't congest. If you need to, mark VOIP traffic with a DSCP value that has queues that are serviced prior to all others (e.g., EF, DSCP46).

[colanderman]

Yes. I'm a Republic Wireless (WiFi VoIP phone) customer. They mark their RTP packets with DSCP 48 (CS6 / ToS 192) and their SIP packets with DSCP 56 (CS7 / ToS 224). I limit ingress & egress bandwidth on my home router (a Mikrotik), and prioritize flows with those DSCPs above all other traffic.

The result is that even when my pipe is completely full, VoIP packet latency is practically unaffected. Compare to ~3 s latencies I would incur by Comcast's awful traffic shaping.

[AstralStorm]

Applications do it already, but edge routers are adept at ignoring these flags or applying them only to their own service.

So that you use it and not competition.

Net neutrality would be the antidote.

[ra1n85]

You're correct, most applications do.

QoS/CoS services are best when clearly spelled out in service agreements. Providers may clear markings, but likely because either they don't want to have a free for all when it comes to their queues, or because they didn't purchase the gear that offers the rich queuing needed.

[saurik]

If those flags provide better quality of service, why am I not applying them to all of my traffic? Any attempt to "know" if my use is legitimate would undermine network neutrality.

[richbhanover]

Sure, you could make yourself crazy trying to prioritize all the traffic with varying kinds of flags.

Or you could just let an algorithm (SQM - fq_codel or cake) automatically determine which flows are sending more than their fair share of traffic into the bottleneck link, and offer backpressure to slow those applications down, so other applications work well.

re: net neutrality. That applies to whether my ISP (or any provider upstream in the Internet) should prioritize packets/data from various sources toward (or from) me. The answer should be, No. My contract with my ISP is that I get X mbps of data. The contract doesn't say the ISP will provide higher priority or data rate for service Y over service Z...

SQM has no effect on net neutrality: I'm simply configuring my own router to prioritize the data that I'm requesting (or sending). My ISP should handle them in a totally neutral way...

[richbhanover]

Yes, Don't Congest. But you can spend your life marking various kinds of traffic (many apps already do mark it, as someone down-thread mentioned), and pray that your router (and remote equipment) handles the markings properly.

Or just use SQM with Cake qdisc. It actually looks at the "sojourn time" for packets in various streams/flows of data and offers backpressure for sessions that are generating queues of data: all the other flows "go right through" without a lot of other settings.

[pier25]

I have almost no bufferbloat when downloading (about 1ms) but about 300ms when uploading.

What does this mean?

[ra1n85]

I didn't look at that tool, but just reading this suggests that you're being shaped/policed more aggressively on egress (in the direction of your provider) than you are on ingress. Shaping and policing (typically) both use the same mechanisms as interface buffers to temporarily store packets in order to prevent congestion/discards. The distinction between them is that shaping actively buffers packets, where policing will more aggressively drop them (and aim to trigger a TCP congestion event) - in most cases, policing will involve a burst bucket that is somewhat similar to a shallow queue.

This would make sense if you have a higher bit rate from your provider for down vs up.

[colanderman]

What speed connection do you have, and what speed connection did that tool report?

If your connection is fast enough that that tool is unable to max it out for whatever reason (say you're on a poor WiFi connection), you will experience no bufferbloat.

[pimeys]

Thanks for these! I have the Omnia Turris with Openwrt at home, which has everything needed to fix this. I'd also hope that problems playing music in our home network from the NAS when moving big files in the lan could be solved with these settings.

[lobster_johnson]

Anyone know if Mikrotik's RouterOS suffers from this?

[colanderman]

Bufferbloat is typically a disease of your ISP (which throttles traffic), not of your router. A Mikrotik in its default configuration, like most other routers, will do nothing to address – nor exacerbate – bufferbloat caused by ISP throttling.

However, it is fairly simple to address ISP bufferbloat using RouterOS's "simple queues". In the most basic configuration, just add a simple queue targeting your WAN interface, with bandwidth limits in each direction slightly (~5-10%) lower than what your ISP throttles you to. If you have a slower connection (<10 Mbps), select "default-small" as the queue type. That's it, bufferbloat solved. (You can get fancier with multiple queues or different queue types, but the above alone gives you 90% of the benefit.)

[richbhanover]

Not so - Bufferbloat can occur everywhere there's a bottleneck link.

Your home router's link to the ISP is likely to be one, and most don't have any mitigation, and suffer from high latency. (And various ISP's equipment has bufferbloat for data coming toward your home.)

Some sites suggest that you knock your bandwidth down to 70% of the link speed to "leave some headroom" for other packets. That's fine if you want to give away 30% of your capacity.

SQM (implemented through fq_codel or cake qdisc's) takes off a few percentage points from the rated link speed, and achieves great results with minimal setup and configuration. Check out https://www.bufferbloat.net/projects/bloat/wiki/What_to_do_a... for more info.

[colanderman]

That is… exactly what I said? Bufferbloat occurs where there's congestion. That's generally on the ISP's router, since they're the ones throttling your connection to 10 Mbps or whatever.

Yes, if you have a gigabit fiber connection and your home router can only handle 50 Mbps, you can get bufferbloat on your home router. But most people aren't in that scenario.

SQM is great, but not available on RouterOS, which is what the GP was asking about.

[richbhanover]

Yes, but it can occur on both ends of the bottleneck. Your ISP's equipment has a bottleneck link toward your home, and can have bloated buffers on that side. But if your router doesn't handle the uplink well, it'll be bloated as well.

Fortunately, SQM is available (from LEDE) for many MikroTik routers, so there may be a way out for the GP.

[colanderman]

That's not how the Internet works. If your ISP is throttling, both upstream and downstream traffic is buffered on its router.

This is a consequence of how congestion is signaled in IP networks: by dropping packets. If the ISP is the bottleneck (it almost always is, due to throttling in both directions), it signals this to upstream equipment (i.e. the source of the packets) by dropping packets. They aren't sent back to some random router to be queued up.

Bufferbloat comes in because the ISP is trying to make use experience a little less crappy by hanging on to bursts of packets which would otherwise be dropped, with the intent of forwarding them on at the throttled rate. The consumer's router does not do this because it has no way of knowing this is happening. (But see below.) This is fine, except ISPs make their buffers HUGE.

There are two ways around this: one is, as you suggested, to use fq_codel, which monitors flows to infer packet buffering or loss at the ISP. So then, it is able to avoid sending packets which would be buffered or dropped by the ISP. The other is, as I suggested, to make your router the one dropping and buffering packets, which it normally does NOT do. It has no reason to: the links to the modem and the client are typically much faster than what the ISP throttles to, and those links are all it can see. (Though it's not unlikely that the user has subscribed to a higher speed than their wireless can handle, in which case inbound traffic is queued on the router, NOT outbound traffic.)

Yes, if your router is underpowered and you have a high-speed connection through the ISP, then your router could become a bottleneck and show bufferbloat. That is almost never the case, and when it is, it can happen in both directions.

The GP has "a way out", it is exactly to perform the steps I indicated. fq-codel is nice to have but NOT necessary.

[lobster_johnson]

Awesome, thanks!

[Bombthecat]

Not in Germany, kernels and routers need! To be locked by law.

I accidentally updated without know. Now I'm locked...

[detaro]

citation needed. If that were true, quite a few router models available would be illegal, so it probably isn't.

(TP-link started locking down at least some models, in response to FCC rules, and some manufacturers don't allow you to downgrade, but there is no legal requirement to do so and often ways around it. Other manufacturers like Linksys openly advertise their routers for use with OpenWRT)

[Bombthecat]

Oh boy, direct down vote instead of looking on your own or asking...

http://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELE...

http://www.tp-link.de/download/Archer-C7_V2.html#Firmware 1.For Archer C7(EU)V2. 2.The EU firmware was specialized for CE certification and can't be downgraded to other version, please click here for choosing your region and selecting the most suitable firmware version to upgrade. 3.Old firmware's configuration file can be imported into this new firmware; 4.Your device's configuration will be lost after upgrading, which means you need to configure your device again

All routers are doing the same. EU law.

[richbhanover]

> All routers are doing the same. EU Law.

Citation still needed.

a) I reviewed your eur-lex.europa.eu link, and didn't find anything that prohibits installing other firmware. Please cite the chapter/article that's relevant.

b) re: TP-Link firmware. It is true that TP-Link has added a technical means that makes it impossible to install certain other TP-Link firmware (the "downgrade" they mention). However, there are other firmware distributions (LEDE/OpenWrt/DD-WRT/others) that have the capability to install over the TP-Link factory firmware.

[detaro]

Sorry, was a bit to quick and I misread your post as describing the current situation, where this not (yet) applies.

And last I've heard about it they extended the validity of the old rules for undefined time (due to most of the standards not being ready), but I can't find a good source for it now, so I might have gotten that wrong. My mistake for not checking better.