cancel
Showing results for 
Search instead for 
Did you mean: 

Power failure causing major BT broadband shut-out

VileReynard
Seasoned Pro
Posts: 10,580
Thanks: 191
Fixes: 9
Registered: 01-09-2007

Power failure causing major BT broadband shut-out

Quote
October 3rd, 2011
Power failure causing major BT broadband shut out
BT broadband users are having a bad time of it today, and it’s all thanks to a power failure in Birmingham. Customers in far-reached areas such as Belfast, Edinburgh, Swansea and London are reporting loss of service and have got the hump that inundated BT customer service staff aren’t answering the helpline.
The phone behemoths have just issued the following statement:
    “We can confirm that, as a result of a power failure at one of our major exchanges, some customers may currently be experiencing loss of broadband service.
    Our engineers are on-site and the majority of customers’ service has already been restored. We are working to restore service to remaining customers as soon as possible this afternoon.
    Should any customers continue to experience difficulty in accessing their broadband service, they are advised to turn their Hub or modem off and on again.”
Classic – if in doubt, you should ALWAYS try turning it off and on again…

10 REPLIES
Community Veteran
Posts: 5,316
Thanks: 465
Fixes: 1
Registered: 21-03-2011

Re: Power failure causing major BT broadband shut-out

Suffering a major power failure at a major data communications hub is totally unacceptable. There are many ways to ensure a fully resilient power system to important facilities. It is not rocket science.
Someone senior in BT should be fired over this outage, especially the person who said fully resilient power was too expensive.
Now Zen, but a +Net residue.
Community Veteran
Posts: 13,921
Thanks: 514
Fixes: 7
Registered: 01-08-2007

Re: Power failure causing major BT broadband shut-out

You know how this works though don't you. We're BT. We're big. We don't have to answer to anyone. We don't have to be responsible to anyone.
Thats the way all big companies work. BT will never give a rats about people loosing connectivity. All they care about is their proifits and share values.
I need a new signature... i'm bored of the old one!
Community Veteran
Posts: 1,850
Registered: 11-08-2007

Re: Power failure causing major BT broadband shut-out

now imagine that happening when all your stuff's on the cloud. 
Ellis
Grafter
Posts: 213
Registered: 04-02-2011

Re: Power failure causing major BT broadband shut-out

I can't understand how this could happen. When I was at BT (25 years ago), our trunk exchange had three massive ships diesel engines to supply power for the whole area. The first would come in within seconds of a mains failure, if that failed the second one would start within 30 seconds, The third one was being serviced. The first and second were kept up to heat with immersion heaters in the sump and cooling system. I never knew the first one to fail, but then it wasn't called on much. so why haven't they fot a similar system now?
Community Veteran
Posts: 2,556
Thanks: 171
Fixes: 2
Registered: 27-05-2011

Re: Power failure causing major BT broadband shut-out

Ellis
at the same time (25 years ago) I worked at Dale Electric supplying the same type of Diesel standby generator. But for computer centres and electronic telephone exchanges we also supplied UPS (batteries with chargers and inverters) to provide uninterruptable power. They also had static (solid state) switches to change over in case the inverter failed. I'm sure the same is available now (at a price) 
To do is to be - Neitzsche
To be is to do - Kant
do be do be do - Sinatra
pierre_pierre
Grafter
Posts: 19,757
Registered: 30-07-2007

Re: Power failure causing major BT broadband shut-out

all the generators are here, and mighty impressive when they run them up as a demo http:// http://www.internalfire.com/

pierre_pierre
Grafter
Posts: 19,757
Registered: 30-07-2007

Re: Power failure causing major BT broadband shut-out

I have a small video of the first one being cranked up, its an Allen.  there is a small petter are compressor filling up a large tank.  The operator hand winds over the big one with  a lever on the fly wheel, to just after TDC, did something round the back out of site, then open up a valve to No 1 clinder,  chuf, chuf and it was away.
They have 7 Modern barn type buildingd full of them
Community Veteran
Posts: 13,921
Thanks: 514
Fixes: 7
Registered: 01-08-2007

Re: Power failure causing major BT broadband shut-out

Quote from: artificer
now imagine that happening when all your stuff's on the cloud. 

Aside from eating your PN bandwidth, BT failures are another reason to keep your files locally on your HDD.
I need a new signature... i'm bored of the old one!
Community Veteran
Posts: 5,316
Thanks: 465
Fixes: 1
Registered: 21-03-2011

Re: Power failure causing major BT broadband shut-out

There's a whole range of different standby power facilities now available and generally they are very reliable. They can be configured so that if one power source, such as a generator, fails another source will take over automatically. I sometimes work with a company who provides audit of facilities in trading floors and data centres. This done as part of the client company business continuity insurance assessment. I have four standard questions I invariably use to kick off the review:

  • How long will it take to close down your servers and network if the standby generator fails?

  • What is the endurance of your UPS system for essential load?

  • How frequently do you perform a live power outage test?

  • How many independent fuel feeds do you have to your generators?


If I get good answers to those questions I know it is a well managed site. I'd love to visit the BT Data Centre where the outage occurred and ask those same questions.  I'd even do it pro bono.
Now Zen, but a +Net residue.
itsme
Grafter
Posts: 5,924
Thanks: 1
Registered: 07-04-2007

Re: Power failure causing major BT broadband shut-out

Is there are more information available to what actual happen. The reports I have seen are very short on information, I have seen quoted that only 5% of the system was affect.
Quote from: PeterLoftus
But for computer centres and electronic telephone exchanges we also supplied UPS (batteries with chargers and inverters) to provide uninterruptable power. They also had static (solid state) switches to change over in case the inverter failed. I'm sure the same is available now (at a price) 

When I first worked in telephone exchanges in the 70's there were large battery rooms filled with lead acid batteries which was the first line of defence before the generators kicked in. Now all the battery  rooms have been removed, some now house the LLU equipment, and I not sure what the first line of defence is before the generators kick in. Also I'm not sure whether the generators have enough output to keep up both broadband and telephony system. As BT have a obligation to keep the telephony system up perhaps the broadband systems are sacrificed.