PPP dropped unable to authenticate - 18.00 on 17th Sept 2014

[#1]

Quote from: Townman

RADIUS servers - is that PlusNet's or BT Wholesale's?

In this latest instant I suspect the latter contributed most significantly.

Quote from: Townman

Does this imply that stale L2TP sessions are a red herring and that the "back-off" (power down) duration is just to spread the reconnection load?

There are many reasons why leaving things powered off can help. It relieves the pressure on support, our phone system, the broadband platform, clears your router memory, resets your local network connectivity. The old adage of switching things off and back on really can work wonders

Quote from: Townman

The fact that a factory reset cures things in this condition somewhat implies that something going wrong in the authentication failure is screwing the router's configuration - has the issue been raised with the vendors?

I'm of the opinion that almost all issues can be solved with a manual PPP connect or disconnect and reconnect. Not everybody is comfortable doing this though. If RADIUS platforms are struggling or there are aren't any available sessions when you try connecting to a gateway then you can end up 'walled gardened'. This is where you see the green BT Wholesale screen or get assigned a 172.x.x.x IP. When this happens your router will not try reconnecting over time because it thinks it's already online. This causes issues for those that:

• Leave the router thinking it'll just spring back into life only to find that they're still having problems after everything's been confirmed as fixed.
  
• Factory reset the router, but then get 'walled gardened' which stops the TR069 platform from reconfiguring the router and requires further reboots/resets.
  

Quote from: Townman

Is this issue related to this - http://community.plus.net/forum/index.php/topic,114808.msg1020778.html#msg1020778 ?

Not directly and I don't think that will be contributing too significantly to the various problems, if at all.

[#2]

Bob,
Thank you for your time and contribution - the "walled gardened" and consequences explanation explained a few gaps in understanding / knowledge.
Finally, if the issues were thought to be frustrated by the BT RADIUS servers, is anything going to be escalated to them, so that they perform better in the recovery from an ISP's MSO?

Cheers,
Kevin

[#3]

Are you serious? You want to blame the BT radius servers if they don't cope very well with many thousands of users suddenly being dumped onto them, because plusnet dropped that many users onto them? How much extra would you like BTWholesale to charge to cover the cost of fancy new radius servers that can effortlessly handle 1 million authentications per second, just in case that many users suddenly try connecting at the same time?
Also, can't the other end of the 172.x.x.x IP just close the connection after say 15 minutes? It seems like something like that would solve some of the mentioned problems, like too many users trying to connect at the same time - some end up waiting for 15 minutes before trying again. Ended up stuck in a "walled garden"? Only for 15 minutes.
Is the failure to automatically re-establish a connection something to do with technicolor firmware using vast amounts of custom software for almost everything rather than the more typical open-source components used by a lot of other routers? Well I suppose it could be and we'll never know a definitive answer to that.

[#4]

"Blame" is a bit emotional: if one wants to improve the post MSO experience then PlusNet need to objectively identify the pinch points and consider how they might be mitigated, so that post MSO (from whatever cause) reconnection is less painful than users found things to be earlier this week.  I am sure there might be several solutions - timing out the 172.*.*.* connections sounds like a "smart idea" - however there is a need to identify what is actually broken first so that there can be a focus on fixing it.
If we want answers (definitive or otherwise) - which we do - we need to be asking the questions.  If would be interesting to know if anyone using a non TG router suffered the same reconnection difficulties?

[#5]

When a connection attempt results in a 172.*.*.* IP address that is because it has failed to connect to Plusnet. Adding a timeout is therefore outside of Plusnet's control - although they could make the case to BT for something to be done.

[#6]

All very valid points and people in networking will relate.
However.
The brand impact alone is very damaging -  Let alone how it reflects on the network team.
May as well just put on the service status page /sms /s-media:
"Sorry we can not cope at the moment.
Please try again later"

[#7]

Well I'm expecting a "simplified" explanation to be supplied to frustrated users who can't get back online, which will helpfully explain that they can't get back online due to BTWholesale's systems being overloaded. Perhaps it will cleverly not actually say that BTWholesale's systems are at fault, but might well give people that impression.
Perhaps plusnet should focus on fixing things at their end so we don't get another repeat performance (if it was a similar issue as the last time), rather than focus on improving the post MSO experience.

[#8]

All I know is that I had significant problems getting the TG582n to reconnect so gave up and substituted a ZyXEL, which connected instantly. 24 hours later I tried the Technicolor again and it worked, although I had to connect manually and the interface on it is so slooooow.

[#9]

only way i could connect was to get someone from openreach to come out and fix pn mess.took 2 hours to find that pn had some steath session going on so it couldnt even get an ip.poor service

[#10]

Quote from: ejs

Perhaps plusnet should focus on fixing things at their end so we don't get another repeat performance (if it was a similar issue as the last time), rather than focus on improving the post MSO experience.

I completely agree with you that the cause of this MSO needs to be understood and eliminated, however there are a multitude of technology stacks which could fail and bring about a MSO.  There is therefore as a mitigation strategy good merit in ensuring that the recovery from a MSO is as slick as possible.  Recovery from disaster is a key component in IT management planning - it needs to be predictable and swift.  The recovery from this event has been seen to be neither predictable nor swift.  Whilst many users reconnected swiftly many were left unable to connect for hours - some apparently are still struggling and the cause of their difficulty does not seem to be clearly understood.
Prevention and containment need to go hand-in-hand.
@ghos7em1,
You illustrate the issue well - something somewhere in the complex technology stack between BT and PlusNet had been left in a queer state!  This needs to be better understood and simpler remedy routes are required.  Are you fibre or ADSL?  I suspect that FTTC presents different and more complex issues than ADSL in the "recovery" window.
@KevinG,
I expect that in trying to get the TG582n to reconnect you did a power off sequence as requested by PN?  What befuddles me is how after powering off the TG582n and the connection cannot be re-established is that thought to be caused by the TG, unless it is the other end throwing some extreme exception event which kicks the TG in the nether region?  If this is the case, it needs to be profiled by PlusNet and fed back to the product owners for rectification.  People simply calling the TG a pile of crud does not move anyone forwards.

[#11]

speedtest on bng2 central10

since the outage every night is congestion.
just hopped back onto a ipv6 gw
central10 ag04


is central 10 on ag04 same pipe as central10 bng02?

[#12]

Quote from: Townman

I expect that in trying to get the TG582n to reconnect you did a power off sequence as requested by PN?  What befuddles me is how after powering off the TG582n and the connection cannot be re-established is that thought to be caused by the TG, unless it is the other end throwing some extreme exception event which kicks the TG in the nether region?  If this is the case, it needs to be profiled by PlusNet and fed back to the product owners for rectification.  People simply calling the TG a pile of crud does not move anyone forwards.

I'd like to disagree about that last sentence, if it might mean different routers were supplied in future. The trouble is, any "investigation" of anything Technicolor related usually seems to involve dishing out a replacement, perhaps sending a particular unit back to Technicolor for them to find nothing wrong with it, and then perhaps setting up some kind of test scenario where the Technicolor works fine, and then finding that there's no problem with it.

[#13]

I'm having no issues with my TG582 whatsoever - since I upgraded the firmware to 10.2.2.B

[#14]

@ejs,
Supplying different routers might be beneficial in the future; today and into the future there is a vast population of TG routers.  If there are verifiable issues with them, they need to be profiled, rectified and a firmware update rolled out.
@Jelv,
I did not experience significant issues before upgrading to 10.2.2.B - however afterwards it has been highly stable.  I know ejs finds the mention of it galling, but my 65+ day connection is circa 3 times longer than anything I've seen previously with the TG582n router or any other router I have used.  That said I've never spent huge sums on routers.

[#15]

@townman yes m8 FTTC.