cancel
Showing results for 
Search instead for 
Did you mean: 

Any capacity issues?

deathtrap
Grafter
Posts: 1,064
Thanks: 4
Registered: ‎23-04-2013

Re: Any capacity issues?

@AndyH, DNS servers what do they have to do with the TAP 3 test? And yes DNS servers are manually configured in router No where does it mention changing DNS servers or setting the router to auto assign them. If this is needed then BTW need to take their finger out of their ... and update the instructions instead of making assumptions that everyone is happy to use their isp's DNS servers
AndyH
Grafter
Posts: 6,824
Thanks: 1
Registered: ‎27-10-2012

Re: Any capacity issues?

The BT test account is restricted to using the BT assigned DNS servers and the test3.speedtest.btwholesale.com domain. DNS lookups will fail using your own configured DNS addresses, so the test will not work.
I have pointed this out before, but the instructions for the test have not been changed.
chrcoluk
Grafter
Posts: 1,990
Thanks: 5
Registered: ‎11-12-2013

Re: Any capacity issues?

I have been quiet as I have not had noticeable congestion for some weeks now, steam yesterday (or night before not sure) even started downloading a massive far cry DLC early evening which was flatlined maxed out, feel sorry for those still affected.
Of course its entirely possible I am just lucky with been on a good end point, or if my problem was BTw related maybe they fixed the chokepoint that was affecting me.
Chris
Legend
Posts: 17,724
Thanks: 600
Fixes: 169
Registered: ‎05-04-2007

Re: Any capacity issues?

I've continued passing your information on to the guys looking in to these issues and they've been doing a load of work behind the scenes to keep looking in to this.

Unfortunately there doesn’t seem to be a single cause of everyone’s problem, there are a lot of similar symptoms but not everyone is on the same network kit either within our network or BT Wholesale’s, not everyone is seeing the exact same issue at the same time. i.e. there’s no common link between everyone.
Earlier this week (as mentioned earlier in the thread) there were some issues reported by other ISPs regarding latency and peak time slowdowns too. There was also an issue on Monday night which did put our network out of balance, further compounded by an issue on Wednesday morning. These would normally have been posted to service status, but unfortunately seem to have been missed (that's something I've passed feedback on internally).
Due to the network being out of balance, the huge amount of traffic for iOS 8.2, Champions League Football etc would have caused issues for some customers on the higher populated endpoints.
There are also some issues we've reported on the Wholesale network and you can see an update of the ones identified here:

http://community.plus.net/exchange-information/
We've also continued to improve the monitoring and reporting, so the team can see common links between latency increases and help to find the causes of the most common speed issues.
We know that it must be frustrating seeing slower speeds but we're still looking, it’s just taking time because there are so many variables.

To help us continue to investigate, there's an updated set of information we'd really appreciate being filled in.

Date speed problem started: e.g. 10th March
Exchange Code (you can find this by searching for your exchange here http://usertools.plus.net/exchanges/): e.g. SLCD
Speedtest results: Be useful to see the results from multiple tests
TBB Latency graph:
Does the problem affect any of the following:
Gaming - Y/N
VoIP - Y/N
Streaming - Y/N
Downloading files - Y/N

What impact are the slow speeds having on your normal use of your connection? e.g. My file downloads are slow or iPlayer is buffering.
Former Plusnet Staff member. Posts after 31st Jan 2020 are not on behalf of Plusnet.
deathtrap
Grafter
Posts: 1,064
Thanks: 4
Registered: ‎23-04-2013

Re: Any capacity issues?

@ You say that some customers affected  share the same symptoms, but the underlying cause isnt the same,and imply that everyone has a different underlying cause ,i find that claim very hard to believe when you have carbon copy ping graphs,like those below,
All these are plusnet ping monitors from last night  All display the same latency /jitter issue at the same time , therefore they do share the same issue as for throughput  ect , we have no visibility  on that
And here is my ping monitor from last night

Also this isn't the 1st time that there have been ping monitors with identical latency /jitter oddities , that to put quite simply should not be there, the fact they are there is an indication that something is wrong at a network level usually, but plusnet in the past seem to pass this type of thing off ,imo because it wasnt service affecting, but guess what it is now, ignore the warning signs at your peril
Chris
Legend
Posts: 17,724
Thanks: 600
Fixes: 169
Registered: ‎05-04-2007

Re: Any capacity issues?

Quote from: deathtrap
@ You say that no two customers affected  share the same symptoms  i find that claim very hard to believe , and here's why

Sorry, where have I said that? What I have said is this:
Quote
Unfortunately there doesn’t seem to be a single cause of everyone’s problem, there are a lot of similar symptoms but not everyone is on the same network kit either within our network or BT Wholesale’s, not everyone is seeing the exact same issue at the same time. i.e. there’s no common link between everyone.
Former Plusnet Staff member. Posts after 31st Jan 2020 are not on behalf of Plusnet.
AndyH
Grafter
Posts: 6,824
Thanks: 1
Registered: ‎27-10-2012

Re: Any capacity issues?

@ deathtrap - I think those users were seeing the same issue that other ISPs picked up, where one link to a core node was hitting its limits.
chrcoluk
Grafter
Posts: 1,990
Thanks: 5
Registered: ‎11-12-2013

Re: Any capacity issues?

Chris have you instructed tech support to NOT close tickets related to this?
deathtrap
Grafter
Posts: 1,064
Thanks: 4
Registered: ‎23-04-2013

Re: Any capacity issues?

IMO  those ping monitors tell a very different story but you still want to disregard it, your choice, but do expect me to go along with what your saying
Chris
Legend
Posts: 17,724
Thanks: 600
Fixes: 169
Registered: ‎05-04-2007

Re: Any capacity issues?

@deathtrap  I see you've edited your post, your misreading what I'm saying.
I have not said that 'everyone has a different underlying cause' - what I have said is 'there doesn’t seem to be a single cause of everyone’s problem' - these are not the same things.
We're not disregarding anything at this point.
Former Plusnet Staff member. Posts after 31st Jan 2020 are not on behalf of Plusnet.
deathtrap
Grafter
Posts: 1,064
Thanks: 4
Registered: ‎23-04-2013

Re: Any capacity issues?

@Chris Yes i edited my post as i had forgot to add my ping monitor and realised i had worded what i wanted to say incorrectly  You say
Quote
I have not said that 'everyone has a different underlying cause' - what I have said is 'there doesn’t seem to be a single cause of everyone’s problem' - these are not the same things.
yes there maybe more than a single underying cause assuming that  they are also seeing  issues with throughput ect, they may not be ,But what is displayed in those ping graphs is because they all must share one or more thing in common, ie they must be routed with plusnet kit/network the same/connect to BTW using the same endpoint interconnects , as once it goes further into BT network the less chance that the same infrastructure will be shared  between us, Why IMO this could well lie within Plusnet's  realm, or BTW's interconnects into Plusnet But perish the through  that may be the case  is the impression i have so far  from plusnet, who have cast the blame on everything and anything  apart from that it maybe something to do with  them
When  in reality  no one actually knows what's going on for sure  this after  some 2 mths nearly 3 in my case (excluding those cases that where diagnosed possibly only because other ISP's reported and badgered bt about it)  there have been and still are almost nightly peak time issues that are unique to the plusnet customer base,
kitz
Aspiring Pro
Posts: 833
Thanks: 55
Registered: ‎08-06-2007

Re: Any capacity issues?

I dont like doing the 'My connection is ok' thing, but Im posting because I was specifically asked elsewhere if I would do so.
@Chris
As you know I was on of the original people badly affected by this issue and for many weeks my throughput at peak time would be down to circa 20Mbps.
I did all the usual tests and gave all the symptoms further back in this thread.  Basically my SVLAN was fine, tests were all performed via ethernet and I even turned wifi off for a while.  My signs were completely typical of congestion somewhere on the network.
Then one evening I was totally fed up of it and set off to do a gateway hop session.  I landed on central12.pcl-bng02 [195.166.130.153] and immediately my speeds went from 20Mb up to 76Mbps.
This was on 25th of Jan and since that date I've continually been connected to the same pipe and Ive had no problems at all.  My speeds are good 24/7 and my latency is also good.  So there was obviously something that for weeks had affected my speeds, which suddenly cured when landing on the right gateway.
However I see the problem still continues for several people who had the same issue as me and therefore its why I've been asked to comment.

So first lets look at the transit points where I understand congestion could possibly occur.
1) LOCAL - DSLAM to OLT/ Openreach handoverpoint. This area is supposed to be well/over provisioned with multiple GbE links  (Is this the CVLAN?)
2) SVLAN - OLT/ Openreach handover point -> WDM Exchange -> bRAS (BTw backhaul)
3) MSILs (Core Node Locations)
4) CORE - BT's major backbone
5) ISP HOST LINKS - ISP gateways
6) PEERING/TRANSIT - Path from ISP to the Internet.

Now lets try to eliminate some of them
1) LOCAL - For me this obviously must be OK, it must also be OK for those whose problems are cured with a gateway hop.
2) SVLAN - Mine was always green, it must also be OK for those whose problems are cured by a gateway hop
3) MSILs - Plusnet uses WBMC Shared so its BTw's responsibility to maintain the MSILs.
4) CORE - Highly unlikely to be the main core, its far too well meshed and resilient as its the UK's major backbone.
5) ISP HOST LINKS - Possible because problems can be cured with a gateway hop
6) PEERING/TRANSIT - Highly unlikely as can be cured with a gateway hop.  My speeds are perfectly fine now.
So basically that leaves us with 2 areas:-
MSILS & ISP HOST LINKS.
You (Plusnet) categorically state that there is sufficient capacity so that leaves us with the MSILs. 
These are supposed to be monitored by BTw and increased when necessary.  Information about BTw maintained MSILs capacity is sparse.    Is it possible that when an EU performs a PPP session drop that their transit is via a different MSIL.  The fact that routing will change depending on which PNGateway you connect to, the fact that the MSIL is after the RAS and BT has multiple MSILs at each core node suggests that it could be possible to get routed via different MSILs at each PPP re-connect... or at least possible depending on which location you will hop off at the core (This will be different depending on whether you connect to a PTN, PTW or PCL gateway)
How does BTw monitor the MSILs?  When does it increase capacity.  We know from WBC ISPs that the MSILs are one of the major points of contention and congestion.  Does Plusnet have access to any information about the BTw MSILs and if they can be checked for congestion. 
The only other place I can possibly thing of a bottle neck is the point where you hop off the core and before the ISP host links... again this is BTw controlled.  If you (Plusnet) are denying that its your host links/gateways then what response/feedback ( if any)  are you getting from BTw?
AndyH
Grafter
Posts: 6,824
Thanks: 1
Registered: ‎27-10-2012

Re: Any capacity issues?

@ Kitz
I think Plusnet will have been dealing with BTw/TSO over this for a while given the fact that they could not see any issues within their own network. It is tricky though because I don't think everyone sees the same symptoms. I found something unusual that Andrue was also able to see, but it doesn't look like anyone else saw this.
I suspect the issue lies with a host link, i.e. the connection from interconnect node at Faraday/Stepney Green/Colindale/Colombo as I believe these will be the only unique parts of the network specific to Plusnet in WBMC Shared. Each host link is 10Gbps, consisting of two physical fibres, between the interconnect and the Plusnet data centres (http://www.sinet.bt.com/sinet/SINs/pdf/460v1p5.pdf). Given the amount of contracted bandwidth they purchase (we can only see a limited amount on here http://www.plus.net/support/service/network_performance/broadband_bandwidth_usage.shtml), Plusnet clearly have a lot of host links/physical fibre links.
Plusnet will get data reports showing their usage for their contracted bandwidth and also the host links. They obviously need to know their usage in order to plan for future capacity requirements etc., but also to make sure they do not breach the BTw limits on the contracted bandwidth/host links (I know for the host links, that they're not allowed to go over 9Gbps on a 10Gbps host link without permission).
Seeing as the WBMC Shared network is shared with other ISPs using the same service, it's interesting that we don't appear to have seen other ISPs/users reporting quite the same issues as small number of people on here. I suspect most small-mid sized ISPs use this service because of the cost of having their own APs etc.
The interesting thing is that overall, most people aren't seeing dropped packets. If a device (either within the WBMC Shared network or within Plusnet's network) was running at capacity, then I would expect to see packet loss. Instead though, people are seeing variable throughput on both the downstream and upstream. I think this makes things tricky to diagnose, particularly as it's not common to see upstream affected.
What is clear though is that end user bandwidth requirements and usage continue to grow at an exponential rate. This growth appears to be happening at a faster rate than the BTw (and their own monitoring for the SVLANs is showing this at the moment). I think it's pretty clear that if a large number of SVLANs are breaching upgrade rules, then there will be pinch-points in other parts of the BTw network (WBMC interconnect/core).
fibreuser
Grafter
Posts: 167
Registered: ‎28-05-2014

Re: Any capacity issues?

Could it be possible that whatever does the traffic prioritisation is getting slower at peak times and causing these throughput issues?
AndyH
Grafter
Posts: 6,824
Thanks: 1
Registered: ‎27-10-2012

Re: Any capacity issues?

@ fibreuser - I would assume Plusnet network admins have looked at this and the CPU usage etc. to rule it out. There was not too dissimilar an issue a couple of years ago, but Plusnet found the cause after their investigation - http://community.plus.net/forum/index.php/topic,111170.msg959759.html#msg959759