An Introduction To NTP - Network Time Protocol On Linux

NTP For The Impatient: Syncing Your Linux Desktop System Clock In FourFiveSeconds1

Using ntpd on your Debian based GNU/Linux desktop is as easy as apt-get install ntp and to forget about it.

End of tutorial, if you have a working internet connection and wait a few minutes your time will now be automatically synced.

Thanks for reading.

Oh, you are still here and want to know more? OK, here we go. :)

Why You Should No Longer Use ntpdate

In order to keep your computer time synched many of us are used to setting up a cronjob with ntpdate. But the developers have decided to deprecate ntpdate and recommend reverting to the ntpd (Network Time Protocol Daemon) instead. The good news: Using ntpd is even simpler.

ntpd -q
Exit the ntpd just after the first time the clock is set. This behavior mimics that of the ntpdate program, which is to be retired2.

This has been tested on: Debian GNU/Lnux 8 “Jessie Software versions: ntpd 4.2.6p5 Should work with: Any debian based distro such has Ubuntu, Linux Mint etc. Most of this tutorial is generic and valid on many more Linux distributions. Some commands, paths, file names and basic configuration procedures might vary.

You want to know more: NTP - Some background information

The three letters of NTP stand for Network Time Protocol. It dates back to a time before 1985, which makes it one of the oldest protocols still used within today’s internet3. In terms of technology it is kind of a dinosaur in a good sense. I kind of like dinosaurs, though I haven’t met one yet. But they are so cool they have a sympathy bonus as far as I am concerned. And I am a Jim Henson Fan. ;)

NTP is a protocol designed to synchronize the clocks of computers over a network4.

Benefits of keeping in sync

There is no way to overestimate the significance of solid time synchronicity. Take the time stamps in log files. If your system time is wrong you might have quite some trouble to properly analyse them, especially in retrospect. You remember that the incident happened at 10 am, but the log file …
Or if you have ever wondered about some strange sorting arrangements for example inside your email client and later discovered that your PC clock was anywhere but nowhere near your local time zone. Everything reverted back to normal once you set it back to normality. Anything from realtime network gaming up to High Availability clusters fundamentally rely upon NTP. No synchronicity, no HA. Simple as that.
This is where the NTP project kicks in. It does some really outstanding work and IMO should get quite some praise for its fundamental services5.

http://www.ntp.org.

  • A pool of information about the NTP project, find the RFCs here: http://www.ntp.org/rfc.html
  • NTP Project (R&D), which conducts research and development upon the protocol and the reference implementation with ntpd
  • NTP Public Services Project, which provide support and additional development for the official implementation
  • Community supported documentation
  • Issue / Bug reporting
  • Mailing lists
  • Public time server lists
  • Reference implementation downloads
  • Server pool project

Troubleshooting and optimizing ntpd

These steps are not really necessary, but interesting. All you need to do is verifying that the ntpd process is up and running:

root@debian:~# ps aux | grep ntpd
ntp        885  0.0  0.8  33364  4128 ?        Ss   12:50   0:00 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 108:113

Don’t be confused that you won’t see an output with netstat. As ntpd is a daemon it is not constantly up and running (and eating your system resources) but called from time to time.

You can also check, who your peers are, then you definetely know that it is communicating with the outside world. (See glossary for a more detailed explanation)

root@debian:/var/log/ntpstats# ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
+89.163.176.82   78.47.234.59     3 u   96   64  376   78.167   11.433  52.100
+ulm156.startded 8.42.238.198     2 u   49   64  377  181.376   -1.377 152.061
+tsusubs.gurkens 84.201.30.243    3 u  101   64  376   79.106   10.183  29.485
*orac.technovisi 131.188.3.221    2 u   37   64  377  129.149   -1.258 157.693

For basic use you can now really forget about it. NTP will now already be up and running, synchronizing your clock.

Common pitfalls are:

  • Name resolution is not working. Check for the servers fom /etc/ntp.conf, for example with ping pool.ntp.org
  • Check your firewall(s) for UDP port 123. It must be open.
  • Beware of the “Panic Threshold” if your time deviates more than 1000s(see below)
  • Refer to /var/log/syslog

Prove that ntpd is not only running, but actually working

This too is not necessary, but interesting and it too is quite easy to verify. Change your system’s time and wait a few minutes. NTPd should automatically connect to its default time servers and adjust your setting. Remember, the syntax to set the date manually is MMDDhhmm. Certainly we need to stop ntp because otherwise the port will be blocked. :)
If you are unpatient you can do this manually with ntpd -gq which fulfils the same purpose as the old ntpdate. Don’t forget to stop ntpd from running in daemon mode, or you will not be able to reproduce the following sequence.

root@play:/etc# service ntp stop
root@play:/etc# date 12121212
Saturday 12 December  12:12:00 CET 2015
root@play:/etc# ntpd -gq
ntpd: time set -20651213.473498s
root@play:/etc# date
Friday 17 April  12:45:30 CEST 2015
root@play:/etc# 
root@play:/etc# service ntp start

Beware of the gremlins: The Panic Threshhold

Beware of the panic threshold:
As per default ntp has got a “panic threshold”. That is, if the time on your deviates more than 1000 seconds from the time server’s, it just logs to /var/log/syslog. The official documentation states that each “TOY” chip is today able to be more stable and if the time deviates that much there must be something seriously wrong.

This was the basic test. Now that we have started ntpd in daemon mode we can change the time again and wait for the automatic synchronisation, which should happen after some time as can be seen either from looking at the output of date or from /var/log/messages:

Dec 12 12:12:09 play ntpd[23907]: Listening on routing socket on fd #22 for interface updates
Apr 17 12:45:23 play ntpd[23907]: ntpd: time set -20651213.473498 s
Apr 17 12:48:09 play ntpd[26305]: ntpd 4.2.6p5@1.2349-o Sat Feb  7 11:23:09 UTC 2015 (1)
Apr 17 12:48:09 play ntpd[26306]: proto: precision = 0.461 usec

But if you want to do some more work, the fun begins right now. Let’s dive deeper into ntpd.

A Quicker Time: How To Chose The Perfect Time Server

If you want to tune your ntpd for performance, the most cruical aspect is chosing time servers near you as your synchronisation destination. The two aspects you should keep in mind are

  • Chose time servers near you for quick network connections
  • Chose time servers with a fast and relyable network connection

You will most likely not be able to differentiate the latter, therefore you should concentrate upon selecting as local time servers as possible. Or “stratum server”, as they are called in NTP-speak. Stratum servers are upstream time servers. They are the ones you can connect to and get your local time synchronized. There is a hierarchy of stratum servers.

  1. Stratum 1: Directly at the source, for example the atom clock
  2. Stratum 2: Server that fetches its time from a stratum 1 server
  3. Stratum 3: Server that fetches its time from a stratum 2 server
  4. Stratum 15: The lowest level, you may use this to force internal synchronisation from this host in case the internet connection fails.

It is misleading to think that the higher the server you are polling is placed in the stratum hierarchy the more accurate your time will be. Load distribution is the key. If everybody and her cat are polling one single server, it will become pretty slow to answer and your time could be behind. And usually if a stratum 2 server deviates by say 50 ms (!) that will be faily great for our usual servers. Quite often stratum 1 servers are not publicly accessable, but if you ask the owner for example via email you might get access. Apple, Microsoft, T-online are for example all maintaining their own stratum 1 servers.

A Pool Near You: NTP-Server Pools for public consumption

Volunteers are running NTP-Server pools all over the world. Free of charge for the benefit of us computer owners. A shoutout of “Thank you!” to all those busy people working in the background with very few of us even realizing they are doing such a great job.

pool.ntp.org

Though in order to achieve redundancy there are several entry points to this pool, the most generic one is

0.pool.ntp.org
1.pool.ntp.org
2.pool.ntp.org
3.pool.ntp.org

But there are also pools for specific regions:

Australia:

0.au.pool.ntp.org
1.au.pool.ntp.org
2.au.pool.ntp.org
3.au.pool.ntp.org

Europe:

0.europe.pool.ntp.org
1.europe.pool.ntp.org
2.europe.pool.ntp.org
3.europe.pool.ntp.org

Germany:

0.de.pool.ntp.org
1.de.pool.ntp.org
2.de.pool.ntp.org
3.de.pool.ntp.org

As usual, avoid long distance transportation, support your locals, chose a server pool near you. You can have a look at all zones and the distribution of pool servers including some uptime statistics etc. at http://www.pool.ntp.org/zone/@.

Meet ntp.conf: Understanding The Config Files

Meet ntp.conf

But it’s always good to know what have got per default, let’s meet Debian Jessie’s initial /etc/ntp.conf (for clarity reasons I edited out the lines with comments):

driftfile /var/lib/ntp/ntp.drift

statistics loopstats peerstats clockstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
filegen clockstats file clockstats type day enable

server 0.debian.pool.ntp.org iburst
server 1.debian.pool.ntp.org iburst
server 2.debian.pool.ntp.org iburst
server 3.debian.pool.ntp.org iburst

restrict -4 default kod notrap nomodify nopeer noquery
restrict -6 default kod notrap nomodify nopeer noquery

restrict 127.0.0.1
restrict ::1

The Driftfile

This is a kind of logfile for internal use, so to speak. After about one hour determination ntpd writes the current frequency offset into the driftfile in order to compensate for differences between the system clock’s frequency and the frequency required to stay synchronized with UTC. This file must be writable to ntpd as it is not updated but rewritten. There is no need for further human intervention.

server

The server this ntpd is to poll. As already stated, it is best to change this to more local ones in order to avoid network delays.

iburst
sends a burst of eight packets only if connection fails at the first attempt.
burst
always sends a burst of eight packes. Please: Do.not.do.this. This is rude and you might feel the consequences in getting blacklisted.

Restrictions

These restrictions can get really tricky and difficult to debug. Please do a thorrough risk assassment before you through yourself into uneccessary trouble. For further information please see the security section below.

default
XXXXXXXXXXX!!!!!!!!!!!!!1
kod
Kiss Of Death” packages are sent upon unwanted queries
notrap
do not send control message protocol traps (useful for debugging but not for production)
nomodify
no changes to the configuration (with ntpq or ntpdc)
nopeer
this ntp server cannot for peer associations
noquery
no ntpq / ntpc queries are answered, i.e. no information about the server / underlying OS are submitted
-4 / -6
are simply to differentiate between IPv4 / IPv6

restrict 127.0.0.1
~ allow localhost administration, no further restrictions

statistics

Statistics are just preconfigured, not enabled. You need to uncomment the statsdir line in order to activate statistics.

statsdir /var/log/ntpstats/

Once nptd has been restarted, this directory is populated with the appropriate logfiles as configured via the statistics line.

statistics loopstats peerstats clockstats

These can be used to determin, how relyable your ntp server is working.

root@debian:~# ls /var/log/ntpstats/
loopstats  loopstats.20150417  peerstats  peerstats.20150417

Upon inspection these files contain:

loopstat values:
day, second, offset, drift compensation, estimated error, stability, polling interval
peerstat values:
day, second, address, status, offset, delay, dispersion, skew (variance)
root@debian:/var/log/ntpstats# cat loopstats
57129 64416.905 -0.006997675 31.738 0.002474052 0.016523 6
57129 64683.175 0.000556490 31.880 0.003533977 0.052426 6
root@debian:/var/log/ntpstats# cat loopstats.20150417 
57129 64416.905 -0.006997675 31.738 0.002474052 0.016523 6
57129 64683.175 0.000556490 31.880 0.003533977 0.052426 6

root@debian:/var/log/ntpstats# cat peerstats
57129 64410.900 89.163.176.82 9024 -0.012878306 0.129534255 7.937501150 0.000000238
57129 64411.851 62.75.202.83 9024 0.009276046 0.080874456 7.945313226 0.000000238
57129 64412.853 89.163.176.82 9024 0.009072799 0.082810385 3.937508875 0.021951105
57129 64412.906 89.163.224.169 9024 -0.017378055 0.135734027 7.937501256 0.000000238
57129 64413.904 62.75.202.83 9024 -0.009169461 0.133853703 3.949227736 0.018445507
57129 64413.905 46.4.77.168 9024 -0.003549880 0.134970048 7.937502085 0.000000238
57129 64414.900 89.163.176.82 9024 -0.014582144 0.129866968 1.937516840 0.016769905
57129 64414.901 89.163.224.169 9024 -0.017207784 0.130701624 3.937509347 0.000170272
57129 64415.853 46.4.77.168 9024 0.011465509 0.082971576 3.937510238 0.015015389
57129 64415.906 62.75.202.83 9024 -0.011151875 0.135691746 1.951188755 0.014512579
57129 64416.904 89.163.224.169 9024 -0.008854296 0.133994702 1.937517167 0.008439053
57129 64416.905 89.163.176.82 9024 -0.006997676 0.134883061 0.937522736 0.010806883
57129 64422.851 89.163.224.169 9024 0.007999143 0.080839377 0.187552194 0.022871727
57129 64610.491 62.75.202.83 9024 0.006150204 0.083006499 0.018368950 0.015814113
57129 64683.175 46.4.77.168 9024 -0.012447183 0.131535204 0.002378211 0.031523855
57129 64747.848 89.163.176.82 933a 0.011432541 0.078166882 0.002284470 0.019835604
57129 64749.899 46.4.77.168 963a -0.001258364 0.129148849 0.001646762 0.026058269
57129 64750.849 89.163.224.169 9424 0.010182744 0.079105566 0.002002297 0.033874898

root@debian:/var/log/ntpstats# cat peerstats.20150417 
57129 64410.900 89.163.176.82 9024 -0.012878306 0.129534255 7.937501150 0.000000238
57129 64411.851 62.75.202.83 9024 0.009276046 0.080874456 7.945313226 0.000000238
57129 64412.853 89.163.176.82 9024 0.009072799 0.082810385 3.937508875 0.021951105
57129 64412.906 89.163.224.169 9024 -0.017378055 0.135734027 7.937501256 0.000000238
57129 64413.904 62.75.202.83 9024 -0.009169461 0.133853703 3.949227736 0.018445507
57129 64413.905 46.4.77.168 9024 -0.003549880 0.134970048 7.937502085 0.000000238
57129 64414.900 89.163.176.82 9024 -0.014582144 0.129866968 1.937516840 0.016769905
57129 64414.901 89.163.224.169 9024 -0.017207784 0.130701624 3.937509347 0.000170272
57129 64415.853 46.4.77.168 9024 0.011465509 0.082971576 3.937510238 0.015015389
57129 64415.906 62.75.202.83 9024 -0.011151875 0.135691746 1.951188755 0.014512579
57129 64416.904 89.163.224.169 9024 -0.008854296 0.133994702 1.937517167 0.008439053
57129 64416.905 89.163.176.82 9024 -0.006997676 0.134883061 0.937522736 0.010806883
57129 64422.851 89.163.224.169 9024 0.007999143 0.080839377 0.187552194 0.022871727
57129 64610.491 62.75.202.83 9024 0.006150204 0.083006499 0.018368950 0.015814113
57129 64683.175 46.4.77.168 9024 -0.012447183 0.131535204 0.002378211 0.031523855
57129 64747.848 89.163.176.82 933a 0.011432541 0.078166882 0.002284470 0.019835604
57129 64749.899 46.4.77.168 963a -0.001258364 0.129148849 0.001646762 0.026058269
57129 64750.849 89.163.224.169 9424 0.010182744 0.079105566 0.002002297 0.033874898

In order to find out, who these servers are, you may use ntpq -p (with name resolution) or ntpq -pn (without name resolution).

root@debian:/var/log/ntpstats# ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
+89.163.176.82   78.47.234.59     3 u   96   64  376   78.167   11.433  52.100
+ulm156.startded 8.42.238.198     2 u   49   64  377  181.376   -1.377 152.061
+tsusubs.gurkens 84.201.30.243    3 u  101   64  376   79.106   10.183  29.485
*orac.technovisi 131.188.3.221    2 u   37   64  377  129.149   -1.258 157.693
root@debian:/var/log/ntpstats# host  62.75.202.83
83.202.75.62.in-addr.arpa domain name pointer ulm156.startdedicated.com.
root@debian:/var/log/ntpstats# ntpq -pn
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
+89.163.176.82   78.47.234.59     3 u  128   64  376   78.167   11.433  52.100
+62.75.202.83    8.42.238.198     2 u    9   64  377  181.376   -1.377 141.079
+89.163.224.169  84.201.30.243    3 u  129   64  376   79.106   10.183  29.485
*46.4.77.168     131.188.3.221    2 u   65   64  377  129.149   -1.258 157.693

filegen

As you reckon, this is the place where you can configure the way these statistics are created.

peerstats
A line is appended to the peerstats file with each NTP packet or reference clock update received
loopstats
A line is appended with each system clock update and records statistics for each quequest loop
clockstats
A line is appended to the clockstatsfile with every update received from a reference clock driver
sysstats
A line is appended to the sysstats file with system statistics. This is done hourly.

ntpd Cheatsheet

Tools and parts of the ntpd “suite”

ntpd ntpq ntpc

ntpd -q One-Shot, equivalent to ntpdate. Starts nptd manually, ignores the panic gate and queries the timeserver once. When combined with -x the timeframe around which the time may deviate upon the host is widened from 128ms to 600ms. Though this is not recommended. ntpq -p see a list of peers, good to see whom ntpd is talking to ntpq -p

Glossary

Understanding ntpq -p

root@debian:/var/log/ntpstats# ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
+89.163.176.82   78.47.234.59     3 u   96   64  376   78.167   11.433  52.100
+ulm156.startded 8.42.238.198     2 u   49   64  377  181.376   -1.377 152.061
+tsusubs.gurkens 84.201.30.243    3 u  101   64  376   79.106   10.183  29.485
*orac.technovisi 131.188.3.221    2 u   37   64  377  129.149   -1.258 157.693
remote
peers speficified in the ntp.conf file * = current time source # = source selected, distance exceeds maximum value o = source selected, Pulse Per Second (PPS) used + = source selected, included in final set x = source false ticker . = source selected from end of candidate list – = source discarded by cluster algorithm blank = source discarded high stratum, failed sanity
refid
synchronization source for the remote source
stratum
your source’s stratum level
t
types available l = local (such as a GPS, WWVB) u = unicast (most common) m = multicast b = broadcast – = netaddr
when
number of seconds passed since last response
poll
polling interval, in seconds, for source
reach
sucess/failure to reach source, i.e. for example 377 all attempts successful
delay
roundtrip time to receive a reply (in milliseconds)
offset
time difference between the client server and source (milliseconds)
disp/jitter
the difference between two samples (in milliseconds)

  1. Song by Rihanna / Kanye West / Paul McCartney. I’m a Beatles-Fan. That means a lot.

  2. from man(1) ntpdate

  3. http://en.wikipedia.org/wiki/Network_Time_Protocol

  4. quoted from http://www.ntp.org

  5. There are alternative implementations, such as crony http://chrony.tuxfamily.org (also server-client), systemd-timesyncd http://www.freedesktop.org/wiki/Software/systemd which is part of systemd (client only) and OPENntpd http://www.openntpd.org for BSD systems.