cancel
Showing results for 
Search instead for 
Did you mean: 

Mother-in-Law suffers Boot Storm?

Community Veteran
Posts: 5,322
Thanks: 467
Fixes: 1
Registered: 21-03-2011

Mother-in-Law suffers Boot Storm?

I was hoping for a peaceful week this week with the M.I.L flying out to Hong Kong to stay with the Far East branch of the tribe. Sadly ( Wink ) she's one of those who had her flight messed up by the Air Traffic Control computer  system failure. Now she is worried her plane might crash into another after take-off. She's 79 years old and not really confident with computers.
In order to reassure her, I've been searching the media to find some detail of the cause for the failure but at present there's only Management Speak being released by NATS.  However I did notice an earlier article that NATS spent a lot of money a couple of years ago to introduce Virtual Desktops in order to save £9 million on support charges. These VDI systems can be prone to "Boot Storms" when all of the remote devices try to reconnect over the network at the same time after an outage. Surely they wouldn't have connected the main Air Traffic Control system, which runs on an IBM 390 mainframe, via this virtualisation system?
It would be refreshing if the NATS people actually confessed once the cause of the problem is identified.
Now Zen, but a +Net residue.
14 REPLIES
Community Veteran
Posts: 16,851
Thanks: 1,138
Fixes: 13
Registered: 06-11-2007

Re: Mother-in-Law suffers Boot Storm?

Scroll down this page ( about 8 tweets )
https://twitter.com/flightradar24

and you should see a pic of the control centre,

and the cause.....
Community Veteran
Posts: 16,851
Thanks: 1,138
Fixes: 13
Registered: 06-11-2007

Re: Mother-in-Law suffers Boot Storm?

Another tweet on same site.... (posted 9 hours ago )...
Quote

Heathrow is closing late tonight. And next wave of arrivals is already en route

Presumably that was relating to the early hours of this morning.... ( as it was posted about 10.30  to 11.00 pm  ).....
Community Veteran
Posts: 5,322
Thanks: 467
Fixes: 1
Registered: 21-03-2011

Re: Mother-in-Law suffers Boot Storm?

Quote from: shutter

and the cause.....

Shutter don't remind me. I once sent one of my engineers to fix a "problem" on New Year's Eve. He had to fly out from London to Amsterdam. The cause? A cleaner needed a power socket.... 
Now Zen, but a +Net residue.
Community Veteran
Posts: 16,851
Thanks: 1,138
Fixes: 13
Registered: 06-11-2007

Re: Mother-in-Law suffers Boot Storm?

Just a thought...... ! 
why can`t they use laptops as "standby".... or even notebooks.... my Hannspree 10" has a 6 hour battery life !...  they could be on the Flight Radar site and/or ( with two notebooks) on PlaneFinder site  ( some flights only appear on one....)  ... at least they would have "some" idea where the planes were.... perhaps need to be more talkative on ATC to "keep up"... with actual location/speed/height/direction...
Community Veteran
Posts: 5,322
Thanks: 467
Fixes: 1
Registered: 21-03-2011

Re: Mother-in-Law suffers Boot Storm?

Here's a quote via the BBC:
Martin Clipp, a former senior operations manager at Nats, told BBC Radio 4's Today programme: "You can't build absolute resilience... in terms of the technology that backs up the systems.
Former air traffic controller Martin Clipp says funding for Nats systems "is being squeezed"
"The system runs pretty much at full pelt. That means as soon as the slightest thing goes wrong the ripples go out enormously."
However, Mr Clipp said Nats was being required by the Civil Aviation Authority to cut costs, which had led to redundancies this year.
"There is no risk in safety but there is risk in service continuity," he said. "You get what you pay for."

I'm afraid it shows a lack of technical skill/experience. I was building fault tolerant systems back in 1986. Those ran continuously (24 x 7) even when we did operating system upgrades for at least five years. If the system is already running at full capacity ("full pelt") there's something seriously wrong with their system sizing and planning.
Now Zen, but a +Net residue.
Community Veteran
Posts: 38,251
Thanks: 937
Fixes: 56
Registered: 15-06-2007

Re: Mother-in-Law suffers Boot Storm?

you need to read this - assuming it is accurate - http://www.telegraph.co.uk/news/aviation/11291495/UK-flights-chaos-Air-traffic-control-computers-usi...;
Quote
A consultant who has worked for Nats said it knew its software needed to be replaced a decade ago but will be relying on the 1960s programmes for another two years.
Martyn Thomas, Visiting Professor of Software Engineering at the University of Oxford, said: “The National Airspace System that performs flight data processing was originally written for American airspace in the late 1960s.
“It wasn’t designed to cope with the volume of air traffic we have today, or to interface with modern computer software.”
Prof Thomas said the NAS system was written using a now defunct computer language called Jovial, meaning Nats has to train programmers in Jovial just to maintain the antiquated software. 
Moderator
Moderator
Posts: 25,789
Thanks: 1,134
Fixes: 47
Registered: 14-04-2007

Re: Mother-in-Law suffers Boot Storm?

I'm glad I've only flown twice (Vancouver and back) and am unlikely to in the future.
Customer and Forum Moderator.
Product of the Tyrell Corporation
Community Veteran
Posts: 16,851
Thanks: 1,138
Fixes: 13
Registered: 06-11-2007

Re: Mother-in-Law suffers Boot Storm?

This is the "official" explanation of what happened....
http://www.nats.aero/news/swanwick-technical-failure-update-1220/
Community Veteran
Posts: 38,251
Thanks: 937
Fixes: 56
Registered: 15-06-2007

Re: Mother-in-Law suffers Boot Storm?

which tells you very little
Quote
In this instance a transition between the two states caused a failure in the system which has not been seen before. The failure meant that the controllers were unable to access all of the data regarding individual flight plans which significantly increases their workload.
Community Veteran
Posts: 5,322
Thanks: 467
Fixes: 1
Registered: 21-03-2011

Re: Mother-in-Law suffers Boot Storm?

Just looked up "Jovial" . Apparently it is similar to ALGOL 58.  Oh dear! Back in the 70s I used to do some minor programming in ALGOL. Very easy to get hopelessly wrong or write some Instant Legacy code.

Note:Jovial is used in the B52 bomber systems!!!
Now Zen, but a +Net residue.
Community Veteran
Posts: 4,599
Thanks: 753
Fixes: 3
Registered: 06-11-2014

Re: Mother-in-Law suffers Boot Storm?

It's funny thinking how easy it is to cock up the ATC system these days, I mean, before the advent of computers and internet connectivity, what did they do to control air traffic? Binoculars, radio, telephones, pen & paper, and smart people........
As Scotty once said, the more you overthink the plumbing, the easier it is to stop up the drains... Grin
Community Veteran
Posts: 5,345
Thanks: 609
Registered: 23-09-2010

Re: Mother-in-Law suffers Boot Storm?

Quote from: twocvbloke
before the advent of computers and internet connectivity, what did they do to control air traffic?

A man walking in front holding a red flag. Smiley
Community Veteran
Posts: 3,380
Thanks: 2
Registered: 18-01-2013

Re: Mother-in-Law suffers Boot Storm?

I vaguely remember the old ATC being run with tickets that were moved around in various places and manual scheduling of the landings.
Community Veteran
Posts: 5,322
Thanks: 467
Fixes: 1
Registered: 21-03-2011

Re: Mother-in-Law suffers Boot Storm?

A spokesman was saying the outage was one they'd not experienced before and was caused by a single line of code out of millions.  I've got news for him; most software outages are caused a single line of code! It is nice PR speak by a management drone scrabbling for excuses.
Now Zen, but a +Net residue.