ccgi server clock wrong (for the 5th time)

[#1]

Still playing up 
I am running on: ccgi04.plus.net
Wed May 13 08:06:05 UTC 2009
Wed May 13 09:06:06 UTC 2009
Wed May 13 08:06:06 UTC 2009
Wed May 13 08:06:06 UTC 2009

[#2]

Customer Support have finally replied to my ticket with:

Quote

Thanks for bringing this to our attention. I am passing your ticket to our Networks team who should be able to update the timezone on ccgi02. and: We currently have an open problem (ID 56494) which our networks team are currently investigating with our CGI platform. Please accept my apology for any inconvenience caused, we hope have this problem resolved shortly.

I actually brought this to their attention with a phone call at about 7:30 last night and they said that they would get it fixed.
What's going on? It was meant to be an automated NTP system. When I reported the server timeouts yesterday, I asked them to check the clock when it was re-started. Obviously they didn't do it and went home and left it.
Currently this is happening every couple of days - The server goes belly-up, they fix the server and the clock is wrong. The previous explanation for the clock error:

Quote

ptn-ccgi03 having a corrupt /etc/adjtime file

is looking odd, since it is now ptn-ccgi2 that has the incorrect clock.
Still very frustrated....
Tony

[#3]

It's being raised in our daily operational review meeting today.

[#4]

What Which time is the meeting?

[#5]

10:30am.  I'll try and get an update.

[#6]

It's already happened, I raised the time on the CGI servers in there. It's still being looked at, for some reason each time the servers are rebooted they are losing the time settings. It's an open problem and actively being worked.

[#7]

The times should now be correct.

[#8]

I hope it has been tattooed on people's foreheads that after restarting any ccgi servers they must check the time!

[#9]

Yes I believe that they want to avoid the precursor to the end of the world happening again.

[#10]

Quote from: Chris

for some reason each time the servers are rebooted they are losing the time settings.

It is always 1 hour fast and it has only happened after the clocks went from GMT to BST.
Is that a clue, or do you want me to specifically point you to daylight savings?

[#11]

Yes it's one hour difference, and yes it's started happening since the change from GMT to BST. As I said in my previous response, the reason for the time being lost on reboot is being looked at.

[#12]

Although the symptoms look fairly easy to correct, it's likely that because of the clustered setup, and centralised backend it's not quite as simple as ticking a  'daylight savings' box like under Windoze.
B.