Earlier today, we made the following announcement :

Over the last few weeks we have made significant improvements to our response times and call answer rates on our 0845 numbers, and are currently progressing well through a backlog of tickets.

Due to this, we have made the decision not to reinstate the Premium Rate numbers for the time being, and instead have decided to maintain the focus on delivering essential support better and more efficiently.

We will review this situation in a couple of weeks, and will make a final decision whether to reinstate the Premium Rate numbers then.

Kind Regards
Mand Beckett
Customer Support

I personally think this is a welcomed move and makes a lot of sense in the current climate.

No Comments »

New Problems Raised: 7

P1 35607 Tickets getting stuck when been through the fault checker (Upgraded from P2)
P1 36106 TPAR not re-activating account after payment
P1 36108 V2 Contol panels for PAYH

Problems Passed Back for Closure/Signoff: 4

Problems added to Hopper:-
35970 CCGI and Frontpage Issues
35607 Tickets getting stuck when been through the fault checker – (Added weight to ensure resource available on this all day later upgraded to P1)

For Hopper consideration:-
36005 Tiscali ceases rejected – The XML Cease tool does not seem to work for Tiscali customers. This was raised as a P1 previously but I don’t think it was fixed. I tested this on a live account today. If there is no cease tommorrow then this is probably a P1 and CSC should be alerted not to use the tool for Tiscali customers.

35667 Force9 customers on PlusNet product type – There’s about 35 customers who are on the wrong product for their visp (ie have a Plusnet account but F9 components. This was due to an error when consolidating legacy products back in May. This isn’t an issue unless the customer changes their account in which case they lose all access to the net, email, webspace etc. 3 customers have done this to date.

35481 0808 fault backup number – This is becoming a bit of a nightmare and is causing uneccessary work within the CSC. We have turned the script off that removes components at the moment in order to fix a P1 so this may have a positive impact on this.

Service Status Posts:-
PAYH Control Panel (36108) – NEW
P2P Issues – UPDATE
Broadband Disconnections – (Ref 34944) – UPDATE
CCGI Issues (35970) – UPDATE

Planned Maintenance Posts:-
Emergency Maintenance – London Loadbalancers – Friday 29th September 8:00am – 9:00am
Wireless Broadband Maintenance – 1:01am to 4:01am Thursday 5th October
Telehouse Maintenance – 6:00am to 9:00am Saturday 30th September

Announcement Posts:-
None

Summary:

The intermittent issues affecting the CGI servers is ongoing. We have some maintenence scheduled on the loadbalancers tommorrow morning to try and resolve this.
Tiscali Disconnections are still the prevalent issue. We had a disconnection on the non-problematic gateway but that turned out to be Tiscali maintenence (thankfully). We are putting measures in place to make sure we are alerted to this sort of thing in future. The replacement hardware for ptn-ag1 didn’t arrive on time so the work to swap this kit out has been postponed until Saturday, we’re hoping this will fix the LLU disconnect issue that’s plagued us for the last month.

Another 2 parbin P1’s were raised today, both reoccurring issues and both having occurred twice now in the past week.

Spent a significant volume of time on P1 35607 Tickets getting stuck when been through the fault checker (Upgraded from P2). This occurs quite frequently for varying reasons but we need to make sure it’s fixed for faults phase 2. It also created significant work for the CSC and alot of customer disatisfaction when it happens. We need a failover so tickets don’t get stuck when problems are encountered. Work will continue on this one tommorrow.

Service Status Due:-
CCGI Issues (35970) – UPDATE @ 1pm Friday
Broadband Disconnections – (Ref 34944) – UPDATE @ 4pm Friday
Missing Webstats (35910) – UPDATE @ 3pm Friday
Premium Rate Support Availability – UPDATE @ 3pm Friday

Announcements Due:-
VMBU problem update and interim usage accounting update for PAYG customers

Other Stuff:-
There has been discussion of having 3 separate problem hoppers. The existing hopper will be used for Dev work with another hopper for Content rollouts and a third for Networks (pending approval). This would certainly help raise visibility of key issues across the 3 departments and would assist the tracking of issues within the CSC.

I have today collated an issues log on open VMBU/CBC/Ellacoya issues in the problem tool. I have closed a number of duplicates and we will be drafting an announcement for the CSC/Customer Base so we all know what issues there are. This should hopefully put an end to the dispcrepencies often reported by End Users.

Comms coverage is low tommorrow. See internal forum posts for further info. I will be available on my mobile should I need to be contacted and Mand and Liam are in the office. Mand is primary problem contact.

No Comments »

No Comments »

Comms Coverage Today
=================
Mand : 8 – 5 – Problem Management
Liam : 9 – 6 – Portals, PUG, 3rd Party

CSC Support / Tasks / Escalations : Shared

Summary
=======
For the next few days, Comms will have a more limited presence than normal due to staffing issues caused by pre-booked holidays prior to the new team / shift pattern being formed.

Mand is tackling Problem Management as her main role today and I will be working the forums as much as I can.  Please bear with us through the next few days – all will be back to normal on Tuesday!

No Comments »

Comms Coverage Today
=================
Luke : 11-8 – TSM
Liam : 9-6 – PUG, Portals & 3rd Party
Mand : 8-12:30 – Portals & Escalations
Bob : 9-6 – Problems and Service Status
Dave : 1:30-midnight (split shift) – Portals and Usenet / Problems

Comms EoD Summary :
================
No major changes to the Top 5 today, but we do see the introduction of speed issues being mentioned with some customers complaining about poor speeds on FTP / P2P / Usenet etc… with some traffic being completely blocked at level one peak time (where it should only be reduced).

No major complaints with regards to Tiscali disconnections, but it remains at number one whilst we get the issue fully resolved and an update out. We have had complaints over Tiscali fault handling and lead times for LLU MACs, however.

The CCGI issues have demanded full time attention for the last three days and are causing a major headache for many of our more experienced customers. Networks are continuing to investigate the cause of excess connections and packet loss routing traffic to the CCGI servers. Meanwhile, more hardware has been allocated for CCGI following upgrades to our mail platform over the next few weeks.

The availaibility of Comms over the next few days will be more scarce than usual due to pre-booked holidays prior to the new team and rota being launched. Please see announcements for staffing levels.

Top 5 Customer Affecting Issues Today
============================
1 – Tiscali
2 – CCGI Slowness / Timeouts
3 – Webstats / Platform / Speeds on P2P – FTP etc…
4 – Max Regrade Issues
5 – Support Availability / Speed

Comms Coverage Tomorrow
====================
Liam : 9-6
Mand : 8-5

No Comments »

Daily snapshot statistics for CSC contacts through ticketing system and by telephone.

Open Tickets at 10pm, Thursday 28th September :-

Customer Support Centre : 1291
CSC Account Changes : 9
CSC Callback Pool : 34
CSC DSL Faults Total : 259
CSC Tiscali Faults Total : 27
CSC Team Leader : 6 (personally assigned)
CSC Agent : 74 (personally assigned)
Total Tickets : 1941

Call Stats at 10pm, Thursday 28th September :-

Faults Line Calls Waiting : 2
Faults Line Longest Wait : 14min 57secs

Customer Support Calls Waiting : 6
Customer Support Longest Wait : 25min 08secs

Business Calls Waiting : 1
Business Calls Longest Wait : 7mins 42secs

Metronet Calls Waiting : 4
Metronet Calls Longest Wait : 24min 24secs

Total Calls Waiting : 13
Longest Caller Waiting : 25min 08secs

No Comments »

No Comments »

New Problems Raised: 10
P1 36069 EDR died
P1 36067 Parbin emails
P1 36071 Phone System fallen over
P1 36065 ITW GELIFE cannot use static IP’s
P1 36083 DNS Lookups intermittently failing (OOH Callout)

Problems Passed Back for Closure/Signoff: 3

Problems added to Hopper:-
No room at the inn!

For Hopper consideration:-
35481 0808 fault backup number – This is becoming a bit of a nightmare and is causing uneccessary work within the CSC
35971 Metronet Webmail out of memory – Getting an increasing number of reports of this which appears to be a development issue
36070 Problems sending to Hotmail from Webmail – Few reports at present but this could cause issues as there are forum threads about it.
35970 CCGI and Frontpage Issues – This is likely to be raised to a P1 as we need a fix today

Service Status Posts:-
CCGI Issues (35970) – UPDATE
Broadband Disconnections – (Ref 34944) – UPDATE
DNS Problems – NEW
CCGI Issues (35970) – UPDATE
DNS Problems – RESOLVED

Planned Maintenance Posts:-
None

Announcement Posts:-
None

Summary:
A slightly quieter day with regards to Service Status postings. 34944 Intermittent connection – LLU is still the prevalent issue. The situation remains the same with ptn-ag1 dropping sessions but not ptn-ag2. We have been in conf calls with Juniper and expect the arrival of replacement hardware for this ERX tommorrow at Telehouse. We hope to have this installed on Friday.

The Phone system fell over again at about 2pm for a few minutes. This was back up shortly but calls were not being routed to the correct skillsets. A BT engineer was called on site and this was resolved within an hour.

Another Parbin P1 was raised due to emails not being routed through to the support team. None were lost and this was swiftly rectified.

An issue was raised with static IPs for one of our ITW customers. This is still under investigation and we are awaiting an update from BT.

There was another P1 affecting the effective date reports for WLR. This had no impact on actual order completion and is now fixed.

Finally, we had an issue with DNS lookups at about 7.15pm. This was quickly identified within about 20 minutes and the OOH engineer called. BIND was restarted and the problem rectified within a couple of hours.

The intermittent issues affecting the CGI servers is causing alot of ill feedback from the Usergroup and a number of customers. The CSC are also continuing to receive reports of this and a Service Status has been open for a number of days now. It is likely this will be escalated to P1.

Service Status Due:-
CCGI Issues (35970) – UPDATE @ 1pm tommorrow
Broadband Disconnections – (Ref 34944) – UPDATE @ 5pm tommorrow
P2P Issues – UPDATE – @ 5pm tommorrow
Missing Webstats (35910) – UPDATE @ 3pm 29/09/06
Premium Rate Support Availability – UPDATE @ 3pm 29/09/06

Other Stuff:-
WLR Phase 3 rolled out successfully overnight however there were numerous problems during the rollout where the CSC’s ability to work was impacted significantly.

No Comments »

Comms Coverage Today
=================
Luke : 9-6 – TSM
Liam : 9-6 – PUG, Portals & 3rd Party
Mand : 8-5 – Portals & Escalations
Bob : 12-9 – Problems and Service Status
Dave : 1:30-midnight (split shift) – Portals and Usenet

Comms EoD Summary :
=================
Customers are generally pleased that we appear to have the Tiscali issue closer to resolution. They are, however, beginning to request further detail as to the cause of the issue and the reason for the long delay. We have asked Networks for a detailed report once the issue is confirmed as resolved.

Platform stability is, again, very much a discussion point within the various customer forums. The stability of our CCGI platform has caused a number of complaints, especially in our PUG forums, from more savvy customers. Webstats availability is causing similar issues and the outage of part of our DNS platform this evening also features in our top customer affecting issues.

We have initiated the process of populating our Knowledge Base and will be making strides into answering the CSC Questions we recently collected with a view to publishing these.

Top 5 Customer Affecting Issues Today
============================
1 – Tiscali
2 – CCGI Slowness / Timeouts
3 – Webstats / Platform / DNS Outage / E-Mail Stability
4 – Max Regrade Issues
5 – Support Availability / Speed

Comms Coverage Tomorrow
====================
Luke : 9-6
Liam : 9-6
Bob : 9-6
Mand : 8-5
Dave : 1:30 – midnight (split shift)

No Comments »

Luke wanted to stick this on my laptop – no way! Clearly Ian wasn’t too happy with that either.

No Comments »