[richbhanover]

Yes, but it can occur on both ends of the bottleneck. Your ISP's equipment has a bottleneck link toward your home, and can have bloated buffers on that side. But if your router doesn't handle the uplink well, it'll be bloated as well.

Fortunately, SQM is available (from LEDE) for many MikroTik routers, so there may be a way out for the GP.

[colanderman]

That's not how the Internet works. If your ISP is throttling, both upstream and downstream traffic is buffered on its router.

This is a consequence of how congestion is signaled in IP networks: by dropping packets. If the ISP is the bottleneck (it almost always is, due to throttling in both directions), it signals this to upstream equipment (i.e. the source of the packets) by dropping packets. They aren't sent back to some random router to be queued up.

Bufferbloat comes in because the ISP is trying to make use experience a little less crappy by hanging on to bursts of packets which would otherwise be dropped, with the intent of forwarding them on at the throttled rate. The consumer's router does not do this because it has no way of knowing this is happening. (But see below.) This is fine, except ISPs make their buffers HUGE.

There are two ways around this: one is, as you suggested, to use fq_codel, which monitors flows to infer packet buffering or loss at the ISP. So then, it is able to avoid sending packets which would be buffered or dropped by the ISP. The other is, as I suggested, to make your router the one dropping and buffering packets, which it normally does NOT do. It has no reason to: the links to the modem and the client are typically much faster than what the ISP throttles to, and those links are all it can see. (Though it's not unlikely that the user has subscribed to a higher speed than their wireless can handle, in which case inbound traffic is queued on the router, NOT outbound traffic.)

Yes, if your router is underpowered and you have a high-speed connection through the ISP, then your router could become a bottleneck and show bufferbloat. That is almost never the case, and when it is, it can happen in both directions.

The GP has "a way out", it is exactly to perform the steps I indicated. fq-codel is nice to have but NOT necessary.

[richbhanover]

I'm sorry, but I have to disagree. The laws of physics (of the internet) dictate that the piece of equipment at a bottleneck is responsible for handling bloat/congestion on its own end. Your ISP's head end/DSLAM/etc. has an opportunity to prevent download bloat (with data being sent to the home), but it has no way to know what's happening on the consumer end of the link.

Your router at home needs to have the same logic: it needs to control the buffering of data being sent toward the ISP. fq_codel/cake actually measures the time packets dwell in the queues (their "sojourn time") and drops/marks with ECN some of the packets of flows that build up a queue.

A cool feature of fq_codel/cake qdisc in LEDE is that it can control bloat in both directions. It imposes a bottleneck that's slightly (a few percent) below the actual ISP download link speed so that traffic queues up within your home router (instead of at the DSLAM/head end). Consequently, it can do the fq_codel/cake algorithm on the download (as well as the upload) direction, keeping your link unbloated at a very small loss of link speed.

[colanderman]

You seem to be working under the fundamental misunderstanding that the link is the bottleneck. IT IS NOT. The ISP's router is the bottleneck, because it typically throttles the connection.

For example, my home network has a 100 Mbit bidirectional Ethernet connection between my router and my modem, and my modem has a 152 Mbps downlink (4 bonded DOCSIS channels) and an 81 Mbps uplink (3 bonded DOCSIS channels).

Neither of those is the bottleneck, because my ISP throttles my connection entirely on their router to 3.5 Mbps downstream, 1 Mbps upstream. I, like most in the US, cannot afford a 100+ Mbps home connection.

What you said would hold true IF, say, I had a gigabit connection (but still used a 100 Mbps Ethernet link). Then, yes, upstream traffic would build up in the upstream queue on my own router or modem. But that's not the case, and neither my modem nor router see anywhere near the 100+ Mbps of traffic needed to saturate the link.

If you do not understand this, I suggest looking at the queue depth on an actual router, with all traffic shaping turned off, while running a speedtest. You will see the interface queues in both directions be completely empty.

[richbhanover]

Yes, but... A "bottleneck" occurs wherever there is a high speed link going to a lower speed link. In your case, the 100 Mbps Ethernets in your home feed through the router to the 1 Mbps upstream your ISP provides. That's another bottleneck.

At this point in a conversation, I always recommend people measure their actual network, to see if they're happy with the situation. If it's good, then everyone's happy.

What results to you get from the DSLReports Speed Test (www.dslreports.com/speedtest)? It measures latency (lag) during the download and upload parts of the test, and will show if your router (or your ISP's router) is buffering too much data, and giving you undesired latency. Best regards.

[colanderman]

The 100 Mbps is from the router to the modem. There is no such thing as a 1 Mbps "link" to an ISP. That bottleneck is imposed by traffic shaping on the ISP's router and nowhere else. (An exception would be something like ADSL, where the uplink truly is the bottleneck. But even then, the queue builds up in the modem, NOT the router!)

Here's a picture, since you seem to be ignoring my words:

     150 Mbps WiFi
           |
      home router
           |
    100 Mbps Ethernet
           |
         modem
           |
    152/81 Mbps DOCSIS
           |
    +-ISP router-----------------+
    |      |  <-- QUEUE          |
    | 3.5/1 Mbps traffic shaping |
    |      |  <-- QUEUE          |
    +------|---------------------+
           |
     10 Gbps fiber (or whatever)

Or, in the case of ADSL:

     150 Mbps WiFi
           |
      home router
           |
    100 Mbps Ethernet
           |
         modem
           |  <-- QUEUE
     8/1 Mbps ADSL
           |
    +-ISP router-----------------+
    |      |                     |
    | 3.5/1 Mbps traffic shaping |
    |      |  <-- QUEUE          |
    +------|---------------------+
           |
     10 Gbps fiber (or whatever)

In either case, neither queue builds up in the home router -- that is exactly the problem, since we can't control the queue size in the modem or ISP! But by traffic shaping to 3.25/0.85 Mbps in the home router (either manually or automatically with fq-codel), we force the queues to build up there, thus giving us control.

[richbhanover]

> In either case, neither queue builds up in the home router -- that is exactly the problem, since we can't control the queue size in the modem or ISP! But by traffic shaping to 3.25/0.85 Mbps in the home router (either manually or automatically with fq-codel), we force the queues to build up there, thus giving us control.

YES! I understand what you're saying. (I had been treating the cable/DSL links and their terminating equipment as a black box; your description was being far more specific. Thanks for sticking with me to make yourself clear.)

And as you point out, the fix in all cases (at least, until our ISPs get serious about solving bufferbloat) is to force the home router to take control of the buffering by shaping the traffic a bit below the actual link speed. Thanks.