Odd TBB test ?

[#1]

I wouldn't expect 0, tcp relies somewhat on drops from the end of a full buffer as an indication to back off.

Saying "tcp" is a bit ambiguous though as there are a variety of congestion control variants, some of which rely less on loss. I think google run "their" bbr now - but we are not testing from google. Upstream loss is also going to depend on shaper, or if none on what the modems shaper does. My ECI has a shorter buffer than my huawei, the upstream below is linux default congestion control cubic + cake (sort of fq_codel) with rtt set at 200ms and tweaked really close to my upstream bandwidth. Downstream - well I am synced higher so not comparing like with like, but when Plusnet QOS was off  I observed their shaper would burst to 80ms then settle at 40ms.

Connecting to host bouygues.testdebit.info, port 5200
Reverse mode, remote host bouygues.testdebit.info is sending
[  4] local 192.168.0.3 port 42990 connected to 89.84.127.53 port 5200
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  7.10 MBytes  59.5 Mbits/sec                  
[  4]   1.00-2.00   sec  7.37 MBytes  61.8 Mbits/sec                  
[  4]   2.00-3.00   sec  7.39 MBytes  62.0 Mbits/sec                  
[  4]   3.00-4.00   sec  6.93 MBytes  58.2 Mbits/sec                  
[  4]   4.00-5.00   sec  7.37 MBytes  61.9 Mbits/sec                  
[  4]   5.00-6.00   sec  7.39 MBytes  62.0 Mbits/sec                  
[  4]   6.00-7.00   sec  7.39 MBytes  62.0 Mbits/sec                  
[  4]   7.00-8.00   sec  7.39 MBytes  62.0 Mbits/sec                  
[  4]   8.00-9.00   sec  7.39 MBytes  62.0 Mbits/sec                  
[  4]   9.00-10.00  sec  7.39 MBytes  62.0 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4] 0.00-10.00  sec  75.8 MBytes  63.6 Mbits/sec     36   sender
[  4] 0.00-10.00  sec  73.4 MBytes  61.5 Mbits/sec                  receiver


Connecting to host bouygues.testdebit.info, port 5200
[  4] local 192.168.0.3 port 43000 connected to 89.84.127.53 port 5200
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  2.13 MBytes  17.9 Mbits/sec    1   65.0 KBytes       
[  4]   1.00-2.00   sec  2.24 MBytes  18.8 Mbits/sec    1   65.0 KBytes       
[  4]   2.00-3.00   sec  2.21 MBytes  18.6 Mbits/sec    1   63.6 KBytes       
[  4]   3.00-4.00   sec  2.21 MBytes  18.5 Mbits/sec    1   65.0 KBytes       
[  4]   4.00-5.00   sec  2.19 MBytes  18.4 Mbits/sec    1   63.6 KBytes       
[  4]   5.00-6.00   sec  2.24 MBytes  18.8 Mbits/sec    0   86.3 KBytes       
[  4]   6.00-7.00   sec  2.21 MBytes  18.5 Mbits/sec    1   82.0 KBytes       
[  4]   7.00-8.00   sec  2.19 MBytes  18.4 Mbits/sec    1   79.2 KBytes       
[  4]   8.00-9.00   sec  2.21 MBytes  18.5 Mbits/sec    1   74.9 KBytes       
[  4]   9.00-10.00  sec  2.23 MBytes  18.7 Mbits/sec    1   73.5 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4] 0.00-10.00  sec  22.1 MBytes  18.5 Mbits/sec     9   sender
[  4] 0.00-10.00  sec  22.0 MBytes  18.4 Mbits/sec                  receiver

[#2]

The graph is interesting, I guess the > 350ms latency is during the upstream part of the test.

Being synced at only 2 meg will make things 10x worse than for me, assuming same sized buffers - lucky you can do QOS 🙂

[#3]

Well I can only assume I'm experiencing some kind of congestion then.

Best effort test as follows. By the way I have an ECI and get slightly better sync - but I prefer the HG612 for stability. And yes I have tried that as well.

iperf3 -c bouygues.testdebit.info -p 5200 -R -d
send_parameters:
{
        "tcp":  true,
        "omit": 0,
        "time": 10,
        "parallel":     1,
        "reverse":      true,
        "len":  131072,
        "no_fq_socket_pacing":  1,
        "client_version":       "3.1.4"
}
Connecting to host bouygues.testdebit.info, port 5200
Reverse mode, remote host bouygues.testdebit.info is sending
SO_SNDBUF is 65228
Congestion algorithm is newreno
[  4] local xx.xx.xx.xx port 43759 connected to 89.84.127.53 port 5200
tcpi_snd_cwnd 14437 tcpi_snd_mss 1440 tcpi_rtt 72812
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  3.33 MBytes  27.9 Mbits/sec
tcpi_snd_cwnd 14437 tcpi_snd_mss 1440 tcpi_rtt 72812
[  4]   1.00-2.00   sec  3.91 MBytes  32.8 Mbits/sec
tcpi_snd_cwnd 14437 tcpi_snd_mss 1440 tcpi_rtt 72812
[  4]   2.00-3.00   sec  4.38 MBytes  36.8 Mbits/sec
tcpi_snd_cwnd 14437 tcpi_snd_mss 1440 tcpi_rtt 72812
[  4]   3.00-4.00   sec  4.42 MBytes  37.1 Mbits/sec
tcpi_snd_cwnd 14437 tcpi_snd_mss 1440 tcpi_rtt 72812
[  4]   4.00-5.00   sec  3.81 MBytes  31.9 Mbits/sec
tcpi_snd_cwnd 14437 tcpi_snd_mss 1440 tcpi_rtt 72812
[  4]   5.00-6.00   sec  4.43 MBytes  37.2 Mbits/sec
tcpi_snd_cwnd 14437 tcpi_snd_mss 1440 tcpi_rtt 72812
[  4]   6.00-7.00   sec  4.42 MBytes  37.1 Mbits/sec
tcpi_snd_cwnd 14437 tcpi_snd_mss 1440 tcpi_rtt 72812
[  4]   7.00-8.00   sec  4.41 MBytes  37.0 Mbits/sec
tcpi_snd_cwnd 14437 tcpi_snd_mss 1440 tcpi_rtt 72812
[  4]   8.00-9.00   sec  4.31 MBytes  36.1 Mbits/sec
tcpi_snd_cwnd 14437 tcpi_snd_mss 1440 tcpi_rtt 72812
send_results
{
        "cpu_util_total":       4.226300,
        "cpu_util_user":        0.381578,
        "cpu_util_system":      3.948770,
        "sender_has_retransmits":       -1,
        "congestion_used":      "newreno",
        "streams":      [{
                        "id":   1,
                        "bytes":        46639424,
                        "retransmits":  -1,
                        "jitter":       0,
                        "errors":       0,
                        "packets":      0
                }]
}
get_results
{
        "cpu_util_total":       0.114435,
        "cpu_util_user":        0.091286,
        "cpu_util_system":      0,
        "sender_has_retransmits":       1,
        "streams":      [{
                        "id":   1,
                        "bytes":        48355680,
                        "retransmits":  5,
                        "jitter":       0,
                        "errors":       0,
                        "packets":      0
                }]
}
[  4]   9.00-10.00  sec  4.31 MBytes  36.1 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  46.1 MBytes  38.7 Mbits/sec    5             sender
[  4]   0.00-10.00  sec  44.5 MBytes  37.3 Mbits/sec                  receiver

iperf Done.

[#4]

I seem to be having a better evening!

Yeah the Up is pretty Horrid...

[#5]

Your iperf3 results look fine assuming you are synced at 40mbit - in fact 37.3 as reported is slightly over what you get if you calculate overheads - but then from testing ages ago (on 100mbit eth) I recall iperf with short runs does tend to call slightly over than expected.

On the dsl reports bufferbloat - that's quite extreme 🙂

It is possible with QOS on to get bloated results like that, as it seems to measure using tcp - so you can have perfectly working setup for icmp/udp latency and a test that looks like that at the same time.

[#6]

I did say best effort

I prefer ON.

The odd spike here and there of 40 Up is much better!

Oh well... had enough for tonight.

[#7]

nice waves http://www.thinkbroadband.com/speedtest/results.html?id=149155188314904053262

gateway pcn-bng04.

[#8]

http://www.thinkbroadband.com/speedtest/results.html?id=149176194313572717077

[#9]

http://www.thinkbroadband.com/speedtest/results.html?id=149189487627317457033

[#10]

We have movement in a better direction Today.

http://www.thinkbroadband.com/speedtest/button/149201223713037651975.png

iperf3 -c ping.online.net -p 5200 -R
Connecting to host ping.online.net, port 5200
Reverse mode, remote host ping.online.net is sending

[ 4] local 192.168.0.14 port 53572 connected to 62.210.18.40 port 5200
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 4.08 MBytes 34.2 Mbits/sec
[ 4] 1.00-2.00 sec 4.42 MBytes 37.1 Mbits/sec
[ 4] 2.00-3.00 sec 4.43 MBytes 37.1 Mbits/sec
[ 4] 3.00-4.00 sec 4.36 MBytes 36.6 Mbits/sec
[ 4] 4.00-5.00 sec 4.31 MBytes 36.1 Mbits/sec
[ 4] 5.00-6.00 sec 4.30 MBytes 36.1 Mbits/sec
[ 4] 6.00-7.00 sec 4.30 MBytes 36.1 Mbits/sec
[ 4] 7.00-8.00 sec 4.31 MBytes 36.1 Mbits/sec
[ 4] 8.00-9.00 sec 4.31 MBytes 36.2 Mbits/sec
[ 4] 9.00-10.00 sec 4.31 MBytes 36.1 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.00 sec 44.7 MBytes 37.5 Mbits/sec 3 sender
[ 4] 0.00-10.00 sec 43.9 MBytes 36.8 Mbits/sec receiver

iperf Done.

[#11]

Been silent the past couple of weeks as I've been busy with other things.

Moved back to BT as I received a decent offer over the phone, so I took them up on it.

No problems so far after the switch, line went from Fastpath to Interleaved which got applied to both down/up giving a hearty 16ms of extra delay (loved it ) and now the line has surprisingly settled with G.INP on BOTH down/up, as opposed to just being applied to the downstream which was the norm since the whole ECI/Lantiq debacle.

I'll post some TBB tests tomorrow if anyone is interested (Though they will be on IPv6), I haven't seen any sort of weird threading issues on BT as I had on PN and gaming seems to be much better, I guess that's to do with BT's routing/peering and whatnot. 

Hope these issues get fixed for you guys at some point.

[#12]

Well, my problem seems to have faded into the mists of ether' I was at the point of moving due to frustraiting problems with Iplayer etc... but we seem to be fixed for the moment.

Just for Once it would be nice of staff to turn up and explain the problem instead of bleeting on that it's complicated or some other such Drivel. Just be honest like A&A would be!

http://www.thinkbroadband.com/speedtest/button/149207348415528820757.png

DOWN

iperf3 -c bouygues.testdebit.info -p 5200 -R
Connecting to host bouygues.testdebit.info, port 5200
Reverse mode, remote host bouygues.testdebit.info is sending
[ 4] local         port 59854 connected to 89.84.127.53 port 5200
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 4.24 MBytes 35.5 Mbits/sec
[ 4] 1.00-2.00 sec 4.43 MBytes 37.2 Mbits/sec
[ 4] 2.00-3.00 sec 4.43 MBytes 37.2 Mbits/sec
[ 4] 3.00-4.00 sec 4.36 MBytes 36.6 Mbits/sec
[ 4] 4.00-5.00 sec 4.31 MBytes 36.2 Mbits/sec
[ 4] 5.00-6.00 sec 4.31 MBytes 36.2 Mbits/sec
[ 4] 6.00-7.00 sec 4.31 MBytes 36.2 Mbits/sec
[ 4] 7.00-8.00 sec 4.31 MBytes 36.1 Mbits/sec
[ 4] 8.00-9.00 sec 4.31 MBytes 36.2 Mbits/sec
[ 4] 9.00-10.00 sec 4.31 MBytes 36.2 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.00 sec 46.0 MBytes 38.6 Mbits/sec 1 sender
[ 4] 0.00-10.00 sec 44.1 MBytes 37.0 Mbits/sec receiver

iperf Done.

UP

iperf3 -c bouygues.testdebit.info -p 5200
Connecting to host bouygues.testdebit.info, port 5200
[ 4] local       port 59858 connected to 89.84.127.53 port 5200
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 269 KBytes 2.20 Mbits/sec 3 30.9 KBytes
[ 4] 1.00-2.00 sec 198 KBytes 1.62 Mbits/sec 0 39.4 KBytes
[ 4] 2.00-3.00 sec 225 KBytes 1.84 Mbits/sec 0 42.2 KBytes
[ 4] 3.00-4.00 sec 197 KBytes 1.61 Mbits/sec 0 42.2 KBytes
[ 4] 4.00-5.00 sec 211 KBytes 1.73 Mbits/sec 0 45.0 KBytes
[ 4] 5.00-6.00 sec 228 KBytes 1.87 Mbits/sec 0 49.2 KBytes
[ 4] 6.00-7.00 sec 217 KBytes 1.77 Mbits/sec 0 57.7 KBytes
[ 4] 7.00-8.00 sec 231 KBytes 1.89 Mbits/sec 0 77.3 KBytes
[ 4] 8.00-9.00 sec 248 KBytes 2.03 Mbits/sec 0 107 KBytes
[ 4] 9.00-10.00 sec 257 KBytes 2.11 Mbits/sec 0 149 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.00 sec 2.23 MBytes 1.87 Mbits/sec 3 sender
[ 4] 0.00-10.00 sec 2.14 MBytes 1.80 Mbits/sec receiver

iperf Done.

[#13]

---

@30FTTC06 wrote:

Well, my problem seems to have faded into the mists of ether' I was at the point of moving due to frustraiting problems with Iplayer etc... but we seem to be fixed for the moment.

Just for Once it would be nice of staff to turn up and explain the problem instead of bleeting on that it's complicated or some other such Drivel. Just be honest like A&A would be!

We haven't changed anything network-wise or on your account, so if it's suddenly loads better, it's could be something not directly under our control.

I'm going to try and summarise what been going on in this thread:

1 - People noticed that they would get decent 6x speeds from TBB tests, but VERY slow 1x speeds ( <1Mbps vs 50Mbps+ in some cases)

2 - Those differences disappeared a few weeks later, leaving 1x more like the 6x results, it hasn't returned since.

3 - Some people started to see "ramp up" patterns in the 1x graphs, like the 1x took time to get to full speed when compared with the 6x speeds tests on TBB.  Doesn't happen all the time.

4 - Sometimes the 1x speed test dips mid tests in ways that aren't seen on the 6x graphs.

5 - Using iPerf directly from the routers, the "ramp up"  and "dips" are replicated, ruling out the TBB tests as being the causes.

6 - We've looked deeper at some of the lines showing this and found at least one example of errors on the line itself.  @rongtw had an engineer out who found corroded joints which have since been repaired.   This doesn't actually appear to have fixed the ramp ups/speed dips

7 - The people reporting the differences are indicating that some days are better than others.  (i.e. doesn't happen at all)

Does that seem like a reasonable summary of where we've got to?  

We've have a good look at the network and tried to identify if anything is incorrect on our side which could be causing it, but can't find anything suspicious.   We tried some tweaks relating to another issue (https://community.plus.net/t5/Fibre-Broadband/Problems-with-Fastly-Content-Delivery-Network-in-South...) which we thought might be having an effect here, but it didn't seem to change anything.

Please remember that there is more than the Plusnet network involved when running speedtests.  You've got your own networks and kit, exchanges, backhaul and general peering and transit out to the hosts running the speedtesters when doing them and anything which cause re-transmissions or lost packets at any point could cause these sorts of issues.   We think we've got the bit under our direct control working right, and if there are issues between you and our networks raising faults explaining the impact should mean we can get Openreach involved providing it is taking your line outside of the agreed tolerances.   There could well be things further up the network from us which are causing the dips which we don't have control over

@NoZephy Your story is interesting.  What is different about gaming?  How are you noticing it?   There should be no perceivable difference between BT and our performance here.   It could be that the DLM reset that happens when you migrated has fixed something.   Just to be clear, it's definitely not the peering/transit because we are both using AS2856

[#14]

In terms of gaming it's just a lot more stable and consistent.

If I had tried gaming on days when the threading issues were present, it would be unplayable. Like others had reported in the past with threading issues, it's not just relatable to pure http download speed as it seems to affect everything. Some webpages won't load when you send off the first request and would be waiting for a response, only to immediately load if you refreshed, weird things like that would only happen when the threading issues were present.

Forgot to add, my setup for BT is exactly the same as I had with PN.

PC >Cat6> Billion 8800NL >Internet>

 

[#15]

Did we ever progress a fault on this on your account?  Did we get to a DLM reset?