ccgi server clock wrong

[#1]

Quote from: MauriceC

EDIT:  It was further back than I thought  http://usergroup.plus.net/pugit/view.php?id=174

Are you going to re-open that PUGIT as it is obviously not fixed?

[#2]

I've asked our Net Ops guys to take another look at the cause behind this when they next get the chance (ref:267233).

[#3]

Hi All,
I've found the problem, for those of you who are technically minded it was caused by ptn-ccgi03 having a corrupt /etc/adjtime file and a hardware clock set to UTC, when the server was booting it set the time to be 1 hour behind all of the others. I'll get a change raised this afternoon to get it sorted out so that it does not re-occur.
NTP is configured and running on all 4 ccgi servers and they all show exactly the same time now, I suspect that the slight time variations show earlier in this post were probably caused by the hardware clocks on all the servers being slightly out and so it has taken time for NTP to adjust the time so that they are all accurate.
[quote=http://ccgi.csogilvie.plus.com/dates.php]
I am running on: ccgi04.plus.net
Tue Apr 28 13:50:58 UTC 2009
Tue Apr 28 13:50:58 UTC 2009
Tue Apr 28 13:50:59 UTC 2009
Tue Apr 28 13:50:59 UTC 2009

[#4]

Clock wrong again - I phoned support to tell them. Thhey said that they will sort it out. Last time we had to wait unitl Monday!!!
Results of the clock ckeck show:
Sat May  9 20:01:17 UTC 2009
Sat May  9 21:01:18 UTC 2009
Sat May  9 20:01:19 UTC 2009
Sat May  9 20:01:19 UTC 2009
Server 2 is one hour forward.
This REALLY messes up my tracker system.
Tony
[EDIT] have also raised ticket number 28749306...

[#5]

I would like to express my thanks to the CS representative who dealt with it really professionally and set the server clock to the correct time.
I wish I could name him, but I understand from posts in other topics that naming CS representatives is not the done thing.
Anyway... Thank you JS 
Tony

[#6]

OK, the ccgi server went down and I asked in my ticket that the clock should be checked after it was rebooted.
Guess what, the server was brought back up and the clock was wrong.
I added to the ticket but nothing happened - no reply and no correction of the clock.
I phoned and told the CS operative that the clock was wrong and he said "yes, I can see it is wrong. I will get it corrected".
Nearly an hour after the phone call, it is still not corrected.
I will phone again, but I should not have to.
Tony

[#7]

The CS person said that there was a note for it to be done, but they only call out the technical people for P1 problems. I said that they did not have to call them out as they could adjust it remotely and it only takes about a minute to do. He said he would try to get permission to call them.
A note has been added to  my ticket saying that they "will ask somebody to take a look at this for you".
Tony

[#8]

They obviously looked at it - but didn't do anything.

[#9]

Quote from: Duncan

I've found the problem, for those of you who are technically minded it was caused by ptn-ccgi03 having a corrupt /etc/adjtime file and a hardware clock set to UTC, when the server was booting it set the time to be 1 hour behind all of the others. I'll get a change raised this afternoon to get it sorted out so that it does not re-occur. NTP is configured and running on all 4 ccgi servers and they all show exactly the same time now, I suspect that the slight time variations show earlier in this post were probably caused by the hardware clocks on all the servers being slightly out and so it has taken time for NTP to adjust the time so that they are all accurate.

Is NTP really running? Perhaps NTP is giving the wrong time....
Server 2 now seems to have had a "a corrupt /etc/adjtime file and a hardware clock set to UTC". If so, it is strange how the same fault occurs on a different server 2 weeks later.
Still angry - tracker statistics still being messed up 12 hours later...
Tony

[#10]

Incredible - still wrong!
See http://ccgi.csogilvie.plus.com/dates.php
Anyone CCGI users seeing different times reported by this tool for the four servers should raise a ticket.

[#11]

Duncan tells me that he will be looking at this today.

[#12]

Server 2 is not working again.
I am running on: ccgi03.plus.net
Tue May 12 12:06:14 UTC 2009
500 Can't connect to 212.159.7.150:80 (connect: timeout)
Tue May 12 12:06:25 UTC 2009
Tue May 12 12:06:25 UTC 2009
perhaps there is some maintenance going on (like the clockl).

[Edit] - no current maintenance according to CS representative on support phone line..

[#13]

The servers are  now all showing the correct time and seem to be running properly.
Now all we need is an explanation.

[#14]

I called into CS to report servers 2 and 3 giving timeouts.
The fixed them but server 2 clock is  wrong again 

[#15]

Still wrong
I am running on: ccgi04.plus.net
Wed May 13 00:13:58 UTC 2009
Wed May 13 01:13:58 UTC 2009
Wed May 13 00:13:58 UTC 2009
Wed May 13 00:13:58 UTC 2009