Little peak time latency humps - split off at staff request

[#1]

@Dave - can you post again in this thread on Monday afternoon to remind us to gateway hop to PCL-AG01 and PCL-AG02.
I wondered with your current theory(s) about what is going on with evening latency, if you have an explanation for the following graphs ?.
These graphs of my connection are from when I was away on holiday, and therefore the line was not being actively used.
The only things connected to my modem/router were my SamKnows monitoring box, and my VoIP phone.
There was no wireless access points active, and the ethernet switch to the rest of my network was switched off.
21st July -

The days in-between these graphs show the evening latency getting worse day by day.
25th July -

The thin blue spikes in the top graph are the SamKnows speedtests, which get drowned out on the second graph during the evening.
In this following graph, I had come home from holiday and gateway hopped around 16:00, and the evening latency hump has gone despite me using the connection for browsing, email, etc.
27th July -

You are talking about adjusting the traffic queues, but I had no traffic ! - so what is going on ?  

[#2]

Quote from: purleigh

You are talking about adjusting the traffic queues, but I had no traffic ! - so what is going on ?

Quote from: dave

In the labs we've seen that a busy network can potentially increase the latency a small amount on the titanium queue so we're trying to change that so it doesn't.

Is the implication there, as puleigh sort of implies, that pings will simply look better because they're in the titanium queue, or is it about more than that?
Obviously, I'm interested as regards the overall latency of gold traffic in the network.
Doesn't network congestion increase latency across all traffic, or are you trying to completely insulate titanium queues from those effects?
If so, doesn't that render ping useless as a true measure of network latency?

[#3]

Under network congestion you would normally see one or more of the following, increased latency, packet loss, decreased speeds. What you will actually see will vary between network, user, type of traffic and amount of congestion.
On our network we should be able to protect traffic in the titanium queue so that under congestion traffic in titanium doesn't see packet loss, the speed stays the same and any change in latency is so small no-one notices.
We want to test to see if that's working as it should and if not improve it.
purleigh - you may not have had traffic on your line (apart from the TBB tester etc.) but there would be traffic on the network, we don't think the effect is down to the per user traffic prioritisation we do, we think it's the network as a whole.
Our network is made up of lots and lots of 1Gbps and 10Gbps links. Each of those links is designed to carry a certain amount of traffic, some are active, some are backups, some are part of resilient sets etc. On the links that go out to BTW from the E320 routers we have shapers set to prioritise traffic on both a user level and network level. It's normal for some of those links to run close to the maximum and thus reach the shaper level and it's where the traffic is close we think this behaviour is happening. Not 100% hence the testing, and some of the graphs don't correlate with with traffic reaching the shaper level but you can get similar patterns from BT side congestion or by using your connection in the same way so some of the graphs may be red herrings.
Will post back on Monday when the change has been made yeah.

[#4]

Look forward to seeing it, I'll hop when it happens.

[#5]

I'm still trying to get my head around how the evening time latency hump was consistently and significantly WORSE for a week when my line was not being used, and immediately improved when I started actively using it again.
I would have expected any latency issue to be worse when more traffic was flowing, not less