Linode Forum
Linode Community Forums
 FAQFAQ    SearchSearch    MembersMembers      Register Register 
 LoginLogin [ Anonymous ] 
Post new topic  Reply to topic
Author Message
 Post subject: Clock drift!
PostPosted: Sun Mar 07, 2010 11:56 am 
Offline
Senior Member

Joined: Fri Sep 21, 2007 4:12 pm
Posts: 78
I noticed last night that the clock on my linode was two minutes (okay okay, 104 seconds) slow. Thinking that the clock on the linode was synched to the clock on the host, I threw in a support ticket ... and it seems that the newer paravirt kernels (if you've selected the "Latest 2.6 Paravirt" option) will now drift.

Because this is indeed a VPS provider, the host load may drastically slow your clock down. In short, your clock is going to run slow.

If you don't want your clock to run slow, install and run ntpd.

So go install ntpd! Package name might be "ntp" or "ntpd." Be sure to start the service too.

Hey, linode people! Any chance we could get an internal ntp server to sync to? :)

edit: hey linode people! How about opening up the ntp daemon on the hosts to the VMs they support? I like that idea even better.


Last edited by kbrantley on Sun Mar 07, 2010 4:01 pm, edited 1 time in total.

Top
   
 Post subject:
PostPosted: Sun Mar 07, 2010 1:07 pm 
Offline
Senior Member

Joined: Mon Oct 27, 2008 10:24 am
Posts: 173
Website: http://www.worshiproot.com
I realize you weren't aware of it, but I thought this was fairly common knowledge?


Top
   
 Post subject:
PostPosted: Sun Mar 07, 2010 1:46 pm 
Offline
Senior Member

Joined: Fri Sep 21, 2007 4:12 pm
Posts: 78
Previously, with the UML kernerls, you couldn't adjust your clock -- it was hard synced with the host.

I had a UML based linode, that is now a paravirt-kernel linode.

This is news to me.


Top
   
 Post subject:
PostPosted: Sun Mar 07, 2010 1:54 pm 
Offline
Senior Member

Joined: Fri Jan 09, 2009 5:32 pm
Posts: 634
do a search in the forums, it's pretty well known. Just grab ntpdate.


Top
   
 Post subject:
PostPosted: Sun Mar 07, 2010 3:28 pm 
Offline
Senior Member

Joined: Mon Oct 12, 2009 1:20 am
Posts: 50
Is there any benefit to running ntpd vs just setting up a cronjob to do "ntpdate pool.ntp.org" every month or something?


Top
   
 Post subject:
PostPosted: Sun Mar 07, 2010 4:00 pm 
Offline
Senior Member

Joined: Fri Sep 21, 2007 4:12 pm
Posts: 78
glg wrote:
do a search in the forums, it's pretty well known. Just grab ntpdate.


I did, and found... very very little. Search for "clock AND drift" and you'll see why I posted this.

arjones85 wrote:
Is there any benefit to running ntpd vs just setting up a cronjob to do "ntpdate pool.ntp.org" every month or something?


The advantage of ntpd is that it continually keeps your clock accurate. Instead of jumping forward (or whatever) a large number of seconds, it continually makes minor adjustments to keep the clock in sync. ntpd also (with most distro-provided config files) syncs with several NTP sources at once, which can help negate the effects of general internet latency.

I was seeing a drift of ~2.4186 seconds per day. ntpdate could have been run twice daily and it would help, but with ntpd, you can forget about it entirely.

Note that you can't run ntpd and ntpdate at the same time, as the NTP protocol uses port 139 exclusively on both the server and the client side.


Top
   
 Post subject:
PostPosted: Sun Mar 07, 2010 4:09 pm 
Offline
Senior Member
User avatar

Joined: Sat Aug 30, 2008 1:55 pm
Posts: 1739
Location: Rochester, New York
Please don't use ntpdate via cron to keep your clock synced.

Most importantly for your system, it will change your clock abruptly, which can make log timestamps ambiguous. If your clock is fast (its time is ahead of the reference time), a sudden jump backwards can freak things out.

It also causes a sudden and heavy load on the time servers at the top of the minute, even more so at the common cron times (*, */2, */3, etc). You're much better off investing the ~1MB of RAM and running ntpd, pointing it at 0.pool.ntp.org, 1.pool.ntp.org, and 2.pool.ntp.org.

This message brought to you by 0.19% of us.pool.ntp.org :-)

(Also, the clock decoupling appears to be a side effect of the mainline Linux Xen implementation. I believe we're seeing the physical host's inherent clock drift. The virtualization is THAT GOOD.)


Top
   
 Post subject:
PostPosted: Sun Mar 07, 2010 5:37 pm 
Offline
Junior Member

Joined: Tue Feb 03, 2004 2:37 pm
Posts: 26
hoopycat wrote:
It also causes a sudden and heavy load on the time servers at the top of the minute, even more so at the common cron times (*, */2, */3, etc).

The statistics geek in me would love to see a graph of requests for the top of the minute requests. It would seem that the spike would relate to the amount of drift in clocks for those that request them. Probably more like a prolonged 15 second spike on each side of the top of the minute?


Top
   
 Post subject:
PostPosted: Sun Mar 07, 2010 6:52 pm 
Offline
Senior Member

Joined: Fri Sep 21, 2007 4:12 pm
Posts: 78
hoopycat wrote:
(Also, the clock decoupling appears to be a side effect of the mainline Linux Xen implementation. I believe we're seeing the physical host's inherent clock drift. The virtualization is THAT GOOD.)


So my host drifts more than one second per twelve hours? :/

I sure hope not...

*pats ntpd*


Top
   
 Post subject:
PostPosted: Sun Mar 07, 2010 7:20 pm 
Offline
Senior Member

Joined: Sat Mar 28, 2009 4:23 pm
Posts: 415
Website: http://jedsmith.org/
Location: Out of his depth and job-hopping without a clue about network security fundamentals
kbrantley wrote:
So my host drifts more than one second per twelve hours? :/

I sure hope not...

I have a PC that drifts a minute a day. I've heard of worse. There is a threshold where NTP will tell you to get bent and no longer sync because you drift so much, and I've heard of it being hit, too.

_________________
Disclaimer: I am no longer employed by Linode; opinions are my own alone.


Top
   
 Post subject:
PostPosted: Sun Mar 07, 2010 7:47 pm 
Offline
Senior Member

Joined: Fri Sep 21, 2007 4:12 pm
Posts: 78
jed wrote:
kbrantley wrote:
So my host drifts more than one second per twelve hours? :/

I sure hope not...

I have a PC that drifts a minute a day. I've heard of worse. There is a threshold where NTP will tell you to get bent and no longer sync because you drift so much, and I've heard of it being hit, too.


Sounds a bit like my router... (which has done that before)


Top
   
 Post subject:
PostPosted: Sun Mar 07, 2010 8:21 pm 
Offline
Senior Member
User avatar

Joined: Sat Aug 30, 2008 1:55 pm
Posts: 1739
Location: Rochester, New York
jeffml wrote:
The statistics geek in me would love to see a graph of requests for the top of the minute requests. It would seem that the spike would relate to the amount of drift in clocks for those that request them. Probably more like a prolonged 15 second spike on each side of the top of the minute?


Rest assured that if I could remember where I've seen the graphs, I would have included them. :-) I figure they're somewhere in the timekeepers mailing list archives...

Also, never underestimate how accurate your clock will be if you run ntpdate every minute.

(Don't get me started on htpdate.)

kbrantley wrote:
So my host drifts more than one second per twelve hours? :/


But of course. Your host's hardware clock oscillator deviates from an ideal reference oscillator by about 23 parts per million, or 0.0023%. If we assume a 32768Hz oscillator, it's off by less than 1Hz. That's an easy software fix.

Now, the hard thing to fix is the frequency drift: the rate of change of the frequency is a pain in the butt. ntpd is really good about handling that.

Fun reading: http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm


Top
   
 Post subject: Re: Clock drift!
PostPosted: Sun Mar 07, 2010 11:32 pm 
Offline

Joined: Sun Jan 04, 2009 11:01 pm
Posts: 1
Website: http://jtsage.com
Location: Pittsburgh, PA
kbrantley wrote:
Hey, linode people! Any chance we could get an internal ntp server to sync to? :)


why yes, that is a possibility. This is the short list, there are others.
http://www.linode.com/wiki/index.php/Internal_Services

_________________
~-~ J.T. Sage ~-~
[-=-jtsage.com-=-]


Top
   
 Post subject:
PostPosted: Mon Mar 08, 2010 11:33 am 
Offline
Senior Member
User avatar

Joined: Tue Nov 24, 2009 1:59 pm
Posts: 362
Urm, if I can ask for clarification... is NTP necessary only when using paravirt kernels, or with the "oldstyle Xen" 2.6.18.8 too?
ISTR that one doesn't let me mess with the clock at all...


Top
   
 Post subject:
PostPosted: Mon Mar 08, 2010 11:36 am 
Offline
Senior Member
User avatar

Joined: Tue May 26, 2009 3:29 pm
Posts: 1691
Location: Montreal, QC
I like syncing to ntp.canonical.com... from my Windows boxes :P


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic


Who is online

Users browsing this forum: No registered users and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
RSS

Powered by phpBB® Forum Software © phpBB Group