advanced atm qos router setup to reduce bufferbloat

[#1]

hi

I have a question related to advanced setup of my archer d5 on plusnet's adsl2+ 21cn

I was always using ubr as atm qos type but I have read that vbr-rt (or cbr) may help with proper traffic shaping at least on my side of things. when i'm configuring new wan connection and choosing vbr-rt as qos type, I have an option to enter pcr, scr and mbs values (they're on 0 now). I wonder if you know what would be plusnets default or community recommended values?

I understand they may depend on actual speeds etc, so in my circumstances my connection is (usually) like that;

up sync speed at least 976 (sometimes higher)

down sync at least 19200 (sometimes higher)

i'm on a fastpath, snr margins are around 6 and I have no issues with packets, drops etc - so connection itself is really stable.

but trouble starts when connection starts to saturate a bit, then lag spikes when gaming are massive, I have also tested connection with dsl reports and bufferbloat is like 2000ms on up and pretty much similar on downlink as well.

I have chatted with support many times and apparently there is some kind of global network issue with gateways connecting to the rest of the internet, that causes a lot of bufferbloat - but it's been investigated for weeks now with no visible resolution, so I was hoping to minimise it by properly configuring qos on a router.

I tried bandwidth control with various setting and limits, which have "some" effect, but still I get massive lag, no matter how harsh i'm cutting up and downlink

so, if anyone has any ideas about trying to properly set up qos - especially with proper initial wan vbr-rt or cbr configuration - please share!

thanks a lot

[#2]

I think any ATM QoS related settings will be completely irrelevant. The ATM layer in ADSL is basically a useless inconvenience that gets added and removed at each end of the ADSL link.

Setting bandwidth limits at the router won't really solve a bufferbloat problem. The bufferbloat issue is that the ADSL upstream bandwidth is the bottleneck, a computer can send data to the router much faster the router can send it up the ADSL link. So a large queue builds up in the router, and traffic is delayed by the time it takes to get from the end of the queue to the front. Setting a lower upstream speed limit won't really solve the problem, traffic will still queue up in the router.

The way QoS can be used to solve (or workaround) bufferbloat issues is by configuring it to prioritize things so that time-sensitive traffic jumps the queue. Even something to prioritize small packets would be fairly effective.

Managing the downstream direction at the router isn't as important as the upstream, because traffic has already passed through the bottleneck of the ADSL link by the time it has reached your router.

[#3]

Router QOS can be a bit hit and miss, especially with nat when specifying internal addresses.

Can you get cli access to the router and run normal linux commands? Would be useful to see what it did.

What settings did you try on the bandwidth section?

Maybe, depending on what games you run, just setting udp as higher prio than tcp could help.

Edit: like ejs said - you should be doing the qos on ip.

[#4]

thanks for your reply

I was hoping that there is a way to use that particular values to regulate traffic flow on the pppoa side of things, but i see your point - none of the support team was even able to give me any specific values to use. probably i will just stick to ubr like always and stop chasing unicorns

but from what you wrote, it seems like router is REALLY imprtant bit in that puzzle, when it comes to proper buffering of packets. as i wrote earlier, i use archer d5, kinda low-medium tier router i believe, would getting more "expensive" one help with things? i need to tell that i have tried asus dsl a68u - it completely destroyed my line, had constant disconnections and ended up on capped speeds for couple of months. i loved user interface though and felt like it done much better job when it comes to latency - but it was overshadowed by poor modem and disconnects. I have also tried nighthawk d7000 - but latency issues and lag spikes were even worse than my archer d5 - i didn't feel that it was giving any more value for money (extra £100), apart from cool looks

when it comes managing up and down, i have actually noticed that cutting downstream on qos to around 15000 (from 19000) kbps reduced download bufferbloat significantly - at least according to dslreports speedtest. and the weird thing with upload test is that is not constant(ish) rate, it spikes up and down massively during even short test.

as for the type of qos, i am just now using ip based, as i have 2 pc's (me and my flatmate) on wired and both of us are gaming online (plus downloading). both pc's have top priority set for all traffic, with rest of ip pool is set to lower priority (all non-wired traffic, phones etc). maybe i'm not experienced enough to set it super properly, but this type of qos setup is making very little difference and we still get massive lag and latency issues. i was trying to set qos based on specific ports for games, voip etc., but i find it difficult to source all setting used by specific games, discord, downloads over steam, origin, http etc. (that's why i went for ip based prioritising).

[#5]

thanks for your reply as well

i can connect to the router via telnet and run some basic commands - but they are really just what i can find in normal setup page

here is the screenshot from my bandwith control section

please note that current speeds are a bit lower that i stated in first post, as plusnet raised my snr margin to 9 in hope to reduce latency - it didn't work and i feel like its even worse (i've asked them to lower it back to 6)

as you also see cap for uplink and downlink of 800-15000 gives me very low bufferbloat during dslreports test, but in real life it makes little differences to my latency issues.

[#6]

If the games you play use UDP then try making UDP prio 1 with min a bit higher than you have.

If the cli command

tc qdisc show

produces some output then it should be possible to to try and work out what is going on.

dsl reports measure tcp bufferbloat, so it's possible with a working set up for gaming that it still shows bloat, but gaming should still be OK. Your current settings will still be putting game traffic with other traffic though, you need to sort that out.

[#7]

mmm, that sounds like an idea to try!

so would it look like

1st pc ip - all ports udp on priority 1

1st pc ip - all ports tcp on priority lets say 3

same with 2nd pc

all other ip's priority lets say 5

would that have a chance to work better?

as for cli commands i will try them later on, as i'm not at home at the moment

thanks for this tips, will report back

[#8]

Yea, trial and error, hopefully something will work.

GUI controls don't really give any clue about what the real commands are, which is why hopefully CLI will show what's really gets set up.

[#9]

hi again

haven't replied in a while as I was busy with work and never had a chance to test and try everything

so... cli commands don't work at all, router must be blocked by manufacturer.

and for the bandwidth control I have taken your tips into account and played with setup a bit. here it is;

with that settings there is no more issues with ping and latency and all gaming pc's at home have fair share of minimum resources. but painful thing is that I had to go hard on download limit - upping it even to 16000 (with actual sync speed of 19500) introduced a lot of latency issues and lag spikes. it looks like upload capping is not such a source of issue in my case as quite big download limiting, to keep things smooth I'm sacrificing 4-5Mbps of bandwidth

thanks everyone for help so far and (hopefully) last question - would getting better router like archer vr600 with some better specs make a difference in that case, when my connection seems to saturate half-way through?

thanks

[#10]

Great that it's working, shame about the CLI.

Depending on how it's handling overheads you may not be loosing quite as much as you think.

Sync rates are ATM rates which include a lot of overheads.  If the 15000 figure is at IP level then you are not comparing like with like. Comparing speed test results with and without QOS should show what you are really loosing TCP throughput wise. If it turns out the router does seem to be working at ATM rates then you should be able to get closer to sync rate on upload than you are.

You do need to back off more for downloads to avoid the ISP/BT buffers filling and hurting latency and some traffic, like torrents are worse than others for hitting your connection hard.

I don't think a different router would change anything, other than if it was open, you could see, and possibly tweak what's going on a bit.

[#11]

hi again

I seem to be waking that thread from the dead

couple of things changed; I managed to get my hands on archer d9 router and therefore I can run cli commands

this is screenshot from the command tc qdisc show (please note that qos is active - when I run it without the output was half the lines)

this is with qos looking like that:

after ip pools, priority 1 is given to udp packets and priority 5 to tcp

if you could please have a look at that output it would be much appreciated

cheers

edit:// I was so happy about previous settings with qos with old router, but after longer testing it was still causing occasional big spikes, that's why I want to dig further

[#12]

So it looks like the qos is on ifb0 and ifb1, which are virtual devices and there will be some filter rules on some of the other real devices that send traffic to them. I don't know which exactly as it looks like a complicated setup.

There's going to be a lot more output when looking in detail, if possible use something to log in that has a big scroll back or session logging facility so you can copy/paste a big chunk of text rather than pictures - maybe post to some pastebin.

For starters to work out the detail get the output of this list - maybe just paste it in from here. Depending on what it shows eg. if iptables is involved, more may be needed. Some may fail/throw an error - it doesn't matter.

tc -s -d qdisc show dev ifb0

tc -s -d qdisc show dev ifb1

tc -s -d class show dev ifb0

tc -s -d class show dev ifb1

tc -s filter show dev ifb0

tc -s filter show dev ifb1

tc -s filter show dev eth1

tc -s filter show dev eth2

tc -s filter show dev eth0.2

tc -s filter show dev eth0.3

tc -s filter show dev eth0.4

tc -s filter show dev eth0.5

ip link show

brctl show

uname -a

[#13]

uff, just got through them

attached as .txt

thanks for checking!

[#14]

Thanks it will take a bit of time to work out what's going on properly.

I can see that it is not allowing for atm overheads and it is using iptables to mark the traffic which means more info is needed.

Can you get the output of a couple more please - one I forgot in the first list and one to see how it's marking the packets.

tc -s filter show dev ppp0

iptables -L -vnt mangle

thanks.

[#15]

thanks a lot again for looking at that logs - I'd have no clue how to work such details

please find attached logs you need - but ipatables with mantle command didn't work but i used table named filter instead - hope output will be helpful

cheers