Performance issue - intermittant poor RTT to first Plus Net hop

[#1]

Hi,
First get the formalities out of the way.  It's a DSL "Max" type service, same as were have had with a previous provider on this same line for around 4 years.   Answers to the fundamental questions ..
1.   Line status info ...
[tt]  --- DSL Information ---
 DSL Driver Version:  AnnexA version - A2pB023k.d21d
 DSL VPI/VCI:         0/38
 DSL Status:          Showtime
 DSL Mode:            G.DMT
 DSL Channel:          
 DSL Upstream Rate:   448 Kbps
 DSL Downstream Rate: 4160 Kbps
                       Down         up    
 DSL Noise Margin:     5.5 dB      18.0 dB
 DSL Attenuation:     56.0 dB      31.5 dB
 DSL Transmit Power:  19.3 dBm     12.3 dBm
[/tt]
Synch speed is pretty much as good as it gets here, our line is over 5km and clearly not top notch.
2. BTW speed tester returns an error Your speed test has completed and the results are shown above, however during the test an error occurred while trying to retrieve additional details regarding your service. As a result we are unable to determine if the speed you received during the test is acceptable for your service. Please re-run the test if you require this additional information.
3. Line speed from Plus Net account page = 3.0meg
All tests carried out with host connected by Ethernet flying lead direct to router, microfilter connected to BT test jack socket, different microfilters tried.
With that background let me explain the fault.   The fault is intermittant episodes of very high latency, 800 to 1000ms RTT rather than the normal 30-40ms.   This shows at the first hop after our premises into the Plus Net network, for example ...
[tt]  Tracing route to relay.plus.net [212.159.9.107] over a maximum of 30 hops:

 1 3 ms 3 ms 3 ms 192.168.1.1
 2 927 ms 941 ms 935 ms lo0-central10.ptw-ag03.plus.net [195.166.128.197]
 3 935 ms 915 ms 913 ms link-a-central10.ptw-gw01.plus.net [212.159.2.152]
 4 885 ms 912 ms 910 ms xe-4-3-0.ptw-cr01.plus.net [212.159.0.244]
 5 948 ms 909 ms 913 ms ae1.pcl-cr01.plus.net [195.166.129.1]
 6 971 ms 916 ms 914 ms po2.pcl-gw01.plus.net [195.166.129.41]
 7 955 ms 974 ms 973 ms gi5-8.ptp-cr01.plus.net [84.93.224.48]
 8 951 ms 990 ms 975 ms vlan2657.ptp-elb01.plus.net [84.93.224.36]
 9 949 ms 972 ms 947 ms relay.plus.net [212.159.9.107]
[/tt]
The excessive RTT is unrelated to normal use on the circuit, with one oddity.  If I run a speed test or other bulk data transfer during one of these episodes, then while that test is running the ping RTTs drop to a normal 30-40ms during the actual transfer, then go back to 800-1000ms immediately the transfer ends.
When first reported Plus Net tweaked our target noise margin, and enable interleaving, neither of which stopped the problem recurring.
Any ideas?
Thanks,  Tony S

[#2]

Tony,
Can you please set up a TBB BQM?  Not having a static IP address will make loss of PPP (new IP address) tedious, but it will enable you to visualise latency and packet loss.  This requires ping response on WAN to be enabled, so I do hope you are not using the 2704n router? 
Kevin

[#3]

Thanks Kevin, I've just set that up.   Does PN only change IP if the ppp session disconnects from our end?   I've seen some ISPs who turn over dynamic IPs seemingly at random within relatively short timescales, so I had sort of assumed that anything pointing to the current IP would be pretty useless.   I can enable my router to respond to outside ping, just verified that's working.
(Obsolete TTB removed)
I've set up some monitors with PRTG from inside the network, logging ping times to smtp.plus.net as well as to two Open DNS servers.    Can you suggest the best Plus Net host for me to use?  I want something that should always have a decent response within their network, ideally I'd use the first hop but that's subject to change with my IP address.    Annoyingly neither of the two routers I have to hand support SNMP,  ideally I'd have a plot of ping time vs utilisation to confirm that it's not our load that's dragging the response down.  

[#4]

It doesn't seem to be set up correctly, because the TBB BQM is solid red (100% packet loss).

[#5]

Yes, IP address change.    Now on static (which I didn't realise I could get on my account!)
http://www.thinkbroadband.com/ping/share/9ef826fc108476a09683ac050c479f43.html

[#6]

I suggest using ntp.plus.net as ping target. This is in London where the gateways are located so avoids the trip to Sheffield where the mail relays are.
David

[#7]

Thanks, have changed to ntp as suggested.  Also, now I'm on static can I assume the first hop will remain consistent?  That would be a good test outbound from home.  I'm going to see if we've got a spare DSL router at work I can borrow, if so I can configure and IP SLA on that and monitor from Solarwinds, taking all my home stuff out of the equation.

[#8]

The first hop will still probably change each time you re-connect, the static IP address does not affect that.

[#9]

BQM looks a bit grim!  Were you doing anything 'heavy' around midday?
In answer to an earlier (now irrelevant) question - PlusNET only changes your dynamic IP address after either end has dropped the PPP session.

[#10]

Nothing too severe going on then, and the RTT was OK.  That TTB graph only covers up to 140ms, which I think could easily be hit if we're using the link normally.  Our bad spells are 800ms or over (over 1200 sometimes).  I'm guessing these will show on TTB as blue because it will be enough to push the average off the scale.  You can see it at 10:00 but only at the very start of the trace.  At that time my PRTG traces show over 1000ms to all the different destinations I'm testing.  However all these traces, like the TTB graph were in the process of being set up and fiddled with today.  For example by default PRTG logs the average of maximums if that makes any sense, meaning that spikes get diluted in the longer term graphs.  Only during today did I check all the sensors to make sure that it using true maximums.
Nothing much more I can do now, this fault has been logged Plus Net since 3rd September, but this Wed we had a minor fault on the copper line, and now PN have the call on hold till next weekend.

[#11]

Nice example of the issue from 5:30 this morning.  Blue "Ping Time" is derived from the average of five, at 60 second intervals (60 sec between samples of 5, not between each ping).  So for example at 05:27 the five pings averaged 902ms and the slowest was 1600ms, then a minute later the average was down to 413, then 312 at 05:29 before back to normal at 05:30.  Clearly in this case only lasted a couple of minutes.