So somehow I’ve managed to get the task of editing the EOD for the first time, so please bear with me as it could be quite interesting as my spelling, punctuation and grammar is legendary within the company for all the wrong reasons.

Well here goes anyhow…

Today as many of you are aware has been quite a testing time here at "PlusNet Towers" with the major power outage happening at 1am this morning causing all kinds of chaos around the business. You can see more details on this on the service status posting.

Starting us off today is Tim from Marketing:

It has been quite hectic today down in Marketing. Chris has been working on a variety of future PR opportunities, Spencer and myself have been designing and building various new pages and generally making things look pretty, finally Martyn has been performing his usual duties creating other interesting marketing material. On top of this we have been doing all our usual day to day tasks.

Down at the Company Rally (my first at PlusNet) Tero or ‘Mr Stats’ as he should be known was deservedly recognised and awarded for his services to web Statistics, well done Tero. Spencer, Chris and Martyn were also nominated for their brilliant work over the last couple of months. All in all a great result for the team.

Colin has this update from Web Development

Arriving this morning, I was greeted by one of the networks engineers telling me we’d lost a data centre and asking me to check that the Community Site was OK. Replying with a quip of "That’s a bit careless" I proceeded to my desk expecting to find nothing on the Community Site working, but thanks to the quick fixes by our colleagues in Networks, it was fine… and I think only Jonny noticed the problem anyway!

James has been been working on enhancements to the Community Site, enabling us to include content from our Flickr and YouTube accounts in to the Community Site account.
Rupert has been working on an enhancement to an email form for Marketing, so that it can be more intelligent, while Sam’s been looking at prototyping improvements to the Manage My Mail tool, to do with the recent move to Postini.
Grzegorz has been working on building some of the templates for moving house as part of Phase 2 of the Madasafish Project.
The content writers have been working on email issues relating to Madasafish Phase 2, and fixing some problems which have occurred recently.
Dan has been working on updating the Ticket Footer text and enhancements to the Portal Accessibility tool.

Like the rest of the company, we also attended the Company Rally where we reviewed performance up until the end of January, and reviewing some of the plans going forwards.

Tony is today’s lucky winner from the Development team…

Quite a busy day for the team, making sure any problems that arose on the live systems were dealt with quickly and efficiently.

The Project Management guys have been looking at how we manage the current projects when Ali disappears to New Zealand for the next month (I’m not jealous - much)!!!!!
Work has progressed well on sign-up updates for new versions and amendments to the existing ones, together with the upgrade process amendments. The benefits from this work should make it easier for new and existing customers to use.
The second phase of one of our major projects for this quarter is well underway. Like most major projects, there is a significant amount of development work to do, so most of the Development team will be involved in one way or another.
Work has also continued with developing some updates to our internal systems. These are vital to ensure we continue to operate efficiently. We are adopting a phased release strategy, so we get the benefits of incremental changes as soon as they are available.
And finally, we had our monthly rally today - a vital way to understand how the business is performing, what we’re aiming to do in the future and to recognise the magnificent contributions that some of the team have made - congratulations to all those winning Plusnetter awards!!!

From CSC we have Chris with an update on the top 5…

1. Email
   A knock on effect from the power outage resulted in the first few hours fielding calls regarding email problems and a massive flood of tickets as well.
2. Router configuration
   Customers calling up for assistance in setting up their routers, a mixture of ones supplied by ourselves and purchased themselves.
3. Fault updates/Logging New faults
   Calls from users chasing up ongoing issues new faults been raised. No specific area’s highlighted or any major MSO’s driving calls into the centre.
4. Access to rapidshare/profiles.yahoo.com
   This reared it ugly head again this morning but was quickly addressed by Ian Glennon and Denis as well..
5. Sales queries
   A slow but steady stream of calls from potential new customers looking to come over due to a number of reasons.

Networks have performed one hell of a job after the power outage this morning, well done lads from everyone here in CSC.

Catherine roars up with an update for QA

Well, today I think the company in general really do have an excuse to use the forbidden word after the power outage this morning. Fortunately it didn’t affect QA too much, although our rollout procedure was knocked out for most of the day. At least this meant that we’ve been able to use the day to get on with some of the other things we don’t normally have the time for.

The male sections of the team also enjoyed a pub lunch, although as I had a prior lunchtime appointment I partook of the buffet provided in the canteen, which was very tasty, mini spring rolls for the win! We’ve also been critiquing Rys on Team ITW’s hair and dealing with the various P1s that have come in. That’s all folks!

John from networks rolls up with the prized report of the day!

OK so everyone always says it’s been a busy day but, as I’m sure you’ll appreciate if you’ve seen the data centre outage service notice, today’s been a belter here in networks.

The power outage took out many of our critical services and several people worked through from the small hours to get us back in business again either by failing systems over to our disaster recovery site or recovering them at the primary site where possible.When the rest of us came in tasks were shared out and that’s been us for the day - Ops, Dev and the DBAs.

There’s a couple of things still to sort out overnight but otherwise things are back to where they should be. Well done to all involved but a particular big-up to Riz who was on call so was in at the start and put in more than his fair share.

..and last but in no way least we have James from Comms

For one reason or another, it’s just been me and Chris in the building today and it’s been quite a day!

We started off pegging it round the building trying to get as many updates as possible for all the areas affected by the Data centre outage and advising customers and colleagues alike as to the relevant state of play.
This afternoon, Chris has been updating a range of Service Statuses, whereas I’ve been placing orders, dealing with escalations, personal ticks, Dev change controls and keeping an eye on our forums.

We’ll sleep well tonight.

Thanks for reading my first edition of being editor of the EOD, I’d like to leave you all on this monumental day with the following thought…

"There will come a time when you believe everything is finished. That will be the beginning." - Louis L’Amou

Goodnight everybody…

Bookmark to:

This entry was posted by Jonny Wing on Thursday, February 21st, 2008 at 6:12 pm and is tagged with and is posted in the category EOD. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.