Showing results for 
Search instead for 
Do you mean 

Easy Setup Disabled 13/08

Highlighted
Community Gaffer Community Gaffer
Posts: 12,366
Thanks: 367
Fixes: 42
Registered: ‎04-04-2007

Easy Setup Disabled 13/08

Afternoon folks,
Quick post to bring your attention to the following that I have just published to our Service Status feed:
[quote author="Service Status]Emergency TR069 Platform Maintenance (automated router setup) - Thursday 13th August
When's this work happening?
This afternoon.
What does it affect?
Our TR069 platform which is used to automatically configure customers' routers when they are first plugged in to a Plusnet broadband-enabled telephone line - http://community.plus.net/blog/2009/08/06/router-setup-the-easy-way/
How long will it take?
We will announce further details regarding restoration of the service at a later date.
What does the work involve?
We are taking the platform offline to investigate a recently identified performance problem that has the potential to negatively impact other customer-facing services.
Is it likely to cause problems?
Newly delivered routers will not automatically configure themselves when plugged in. Instead customers will need to configure the router manually by following the normal manufacturer's setup instructions. Customers changing their password via our website may find that the confirmation message contains innacuracies that refer to the automatic configuration of your router.
Is there anything else I need to know?
We will post another announcement once we're in a position to be able to bring the service back online.
Kind Regards,
Bob Pullen
Customer Support
Basically, we've identified a problem since launch that seems to be putting an increasing amount of load on certain parts of the network. As a precaution we've decided to take the service off line whilst we investigate to mitigate any potential problems this could create for other customer-facing services.
If anybody's any questions then I'll do my best to answer them, however please be mindful that none of the Easy Setup functionality will be working at the moment.

Bob Pullen
Plusnet Products Team
If I've been helpful then please give thanks ⤵

12 REPLIES
Community Veteran
Posts: 1,886
Registered: ‎05-04-2007

Re: Easy Setup Disabled 13/08

Bob,
With all due respect, this hardly instils any confidence in the PN implementation of TR069 into the infrastructure does it?
Why is it only now after "successful" trials that this has now been taken offline, and what kind of loads are we talking about here?
After all, there must have only been in the region of 100+ routers sent out on the trial, and since the rollout it's likely that only maybe another 100 or so of these routers have been dispatched from the warehouse.
What kind of data volumes are we talking when it comes to the promulgation of the information from the TR069 server via the network down to the router?
Surely we are only talking 500kb at the most to send what is I'm guessing a string of code to populate the router (after being told the serial number) with the relevant username and password information to make it work...
Community Gaffer Community Gaffer
Posts: 12,366
Thanks: 367
Fixes: 42
Registered: ‎04-04-2007

Re: Easy Setup Disabled 13/08

We've switched things back on.
@Chemical Brother just about to leave the office but I'll answer your questions when I get home...

Bob Pullen
Plusnet Products Team
If I've been helpful then please give thanks ⤵

Seasoned Pro
Posts: 1,843
Thanks: 361
Fixes: 2
Registered: ‎10-04-2007

Re: Easy Setup Disabled 13/08

That's really good news Bob.  The TR069 platform has so much potential, it would have been sad to see the implementation delayed.
If there are any specific tests that need redoing, I'm happy to give them a whirl - but when I get back home from Norfolk on Sunday evening.  Just drop me a PM.
Maurice
Community Gaffer Community Gaffer
Posts: 12,366
Thanks: 367
Fixes: 42
Registered: ‎04-04-2007

Re: Easy Setup Disabled 13/08

Quote from: Chemical
Why is it only now after "successful" trials that this has now been taken offline, and what kind of loads are we talking about here?

Well put it this way, the database was running in excess of 5,000 queries a second and the SSL connections on our load balancers were close to maxing out. It was very difficult to simulate this volume of load in the test environment. The decision to take the platform offline may have been a little extreme in retrospect however it's better to be safe than sorry.
Quote
After all, there must have only been in the region of 100+ routers sent out on the trial, and since the rollout it's likely that only maybe another 100 or so of these routers have been dispatched from the warehouse.

There were 5,500 routers registered with the ACS when we switched the platform off, you under-estimate the number of orders we get! Wink
Quote
What kind of data volumes are we talking when it comes to the promulgation of the information from the TR069 server via the network down to the router?
Surely we are only talking 500kb at the most to send what is I'm guessing a string of code to populate the router (after being told the serial number) with the relevant username and password information to make it work...

I'm not sure of the exact payload but that's not really the issue here. As I alluded to earlier, it's more the database overheads and resources required to deal with all the SSL connections. We've since learnt that the platform spec we were working to assumed that the TR069 'tree' of the routers we'd be using would contain roughly 300 parameters. Whilst this is accurate for some kit (namely D-Links), the Thomson's have in excess of 4,500 parameters.
I'll post back to this thread after the weekend to let everybody know what the next steps are.

Bob Pullen
Plusnet Products Team
If I've been helpful then please give thanks ⤵

Community Veteran
Posts: 1,886
Registered: ‎05-04-2007

Re: Easy Setup Disabled 13/08

Cheers Bob,
Thanks for the explanation - like our friend mentioned in a previous post, it would have been a shame to have this feature turned off permanently!
My post wasn't meant to be a "get at PN" style post, more expressing disappointment in something that had so much potential to see it go wrong!!
As for the numbers - 5,500 on the system is a lot, and hadn't banked on that many signups.
Thanks again Bob, have a great weekend - I'm enjoying my 40 hour weeks now  Wink
Community Gaffer Community Gaffer
Posts: 12,366
Thanks: 367
Fixes: 42
Registered: ‎04-04-2007

Re: Easy Setup Disabled 13/08

Quote from: Chemical
My post wasn't meant to be a "get at PN" style post, more expressing disappointment in something that had so much potential to see it go wrong!!

A feeling that we're not entirely devoid of. It is a shame when things don't quite go to plan, but we're doing all we can to get things in a good place as soon as possible.
Quote
Thanks again Bob, have a great weekend - I'm enjoying my 40 hour weeks now  Wink

Thanks, you too Smiley

Bob Pullen
Plusnet Products Team
If I've been helpful then please give thanks ⤵

Community Veteran
Posts: 37,502
Thanks: 517
Fixes: 43
Registered: ‎15-06-2007

Re: Easy Setup Disabled 13/08

Steps back in amazement -  Shocked
Posted 6th August
Quote from: Chris
Thanks everyone for your testing and feedback on this, it's certainly been appreciated and has allowed us to fix a number of issues before launching the service.
We've turned this live this afternoon, so anyone who orders a router from today onwards will be able to take advantage of the easier to setup functionality.

So in 1 week you had approaching 5500 customers with the new routers - unless I have totally misunderstood (happens with old age)
Community Gaffer Community Gaffer
Posts: 12,366
Thanks: 367
Fixes: 42
Registered: ‎04-04-2007

Re: Easy Setup Disabled 13/08

We've been sending routers configured with the TR069 username for a while now, so there will be some devices registered from before launch too.

Bob Pullen
Plusnet Products Team
If I've been helpful then please give thanks ⤵

Community Veteran
Posts: 37,502
Thanks: 517
Fixes: 43
Registered: ‎15-06-2007

Re: Easy Setup Disabled 13/08

Pity - in excess of 250,000 new customers per year sounded very impressive  Grin
Community Veteran
Posts: 3,486
Registered: ‎02-10-2008

Re: Easy Setup Disabled 13/08

Quote from: Bob
We've been sending routers configured with the TR069 username

Aha - maybe that's why when I tried with getstarted username again to test - it didn't work ! ?.
So if we do a factory reset - this does reset it to tr069 username then we should be reset just like new customers now ?.
And if we wanted to set up our referrals routers so that it would work like this if they needed to do a factory reset when we were not around,
could we put the latest firmware in - and then raise a ticket for you to put the routers serial number on the appropriate server ? ? ?.
Or aren't you keen on us doing that at the moment ?.
Community Gaffer Community Gaffer
Posts: 12,366
Thanks: 367
Fixes: 42
Registered: ‎04-04-2007

Re: Easy Setup Disabled 13/08

Quote from: mal0z
Aha - maybe that's why when I tried with getstarted username again to test - it didn't work ! ?.

The password on the getstarted account has been changed so you won't be able to connect to it.
Quote
So if we do a factory reset - this does reset it to tr069 username then we should be reset just like new customers now ?.

Theoretically, yes.
Quote
And if we wanted to set up our referrals routers so that it would work like this if they needed to do a factory reset when we were not around,
could we put the latest firmware in - and then raise a ticket for you to put the routers serial number on the appropriate server ? ? ?.

You shouldn't need to raise a ticket because the walled garden will handle all of that by itself. Having said that, the platform has been switched off again for the time being...

Bob Pullen
Plusnet Products Team
If I've been helpful then please give thanks ⤵

Community Gaffer Community Gaffer
Posts: 12,366
Thanks: 367
Fixes: 42
Registered: ‎04-04-2007

Re: Easy Setup Disabled 13/08

For those that didn't see it, we've turned some aspects of the Easy Setup platform back on.
The tr069@plusdsl.net username was switched on again last friday and the warehouse started scanning routers again on Monday.
The routers that start landing with customers today onwards should auto-configure themselves. If there's a problem with the scanning though then the customer is now getting taken to a landing page advising them that they'll need to configure the router manually. This will be in place until mid-September at the earliest at which point we hope to have the walled garden back up and running. In between now and then we've a number of server upgrades planned that we'll be conducting over the coming weeks.

Bob Pullen
Plusnet Products Team
If I've been helpful then please give thanks ⤵