Share This:

On December 31st, 2016 there will be a (positive) leap second. If you happen to be on holiday over that period - woohoo! An extra second of holiday for free. If you're working, make sure you claim that extra second back

 

Very simply, a leap second is an adjustment of, well, a second so that atomic clocks do not differ from Earth's rotational time by more than 0.9 seconds. There are a large number of resources on the internet describing what a leap second is and why it's necessary so I won't bore you with more detail than that but some of the references below may help if you're interested.

 

What do you need to do to avoid potential issues this may cause to the Discovery appliance? Simple - keep up to date with OSUs and/or make sure you're connected to an NTP server (up to date and configured to handle the leap second).

 

Having time synced with an NTP server has 2 options. Either the server will just 'step' the time and the client updates time immediately or you can choose to 'slew' the time which means slowly apply the difference between server and client. By default, ntp configured through the Discovery UI sets the former option. Admittedly this wasn't a conscious decision, but rather it's the default setting. Without slewing, you may see an extra second (23:59:60) in any logs/output. If your system is slewed, minute (small ) changes will be introduced so that the time will stay within about 128ms of 'normal' time but you are unlikely to see the odd time output. During the June 2015 leap second, no issues were reported against ADDM/Discovery that were due to the leap second with the default setting. I don't know of any customers who configured the ntp client to slew, but I also know of no issues raised if this was set.

 

If you do not have NTP configured then either you will have to manually adjust the time after the leap second or, if your OSU is up to date, this will be accounted for by Red Hat updates. Red Hat have recently (30th September 2016) released updates to RHEL >=5 to add the December leap second to tzdata packages. These missed the September OSU but will be in the October OSU. We will release an RHEL 5 OSU in October. This will include all RHEL 5 updates since the last issued 5.x OSU. Similar to NTP configured systems, no issues were raised against ADDM/Discovery for the June 2015 leap second if the latest OSU had been applied.

 

There are numerous bugs in the early RHEL 6.x releases (6.0-6.3) related to leap seconds. The OSU updated the system to RHEL 6.4 in the March 2013 OSU, ideally you have updated since then 6.4 doesn't take into account the December 2016 leap second, but it does prevent some of the issues seen by systems after leap seconds occur (high CPU usage, system hangs, etc...). Other issues have been discovered (e.g. slew mode in ntp adjusting time immediately on detection of a leap second) since then. These have been fixed in later versions but most of these did not cause severe issues and rather issues with sorting or triggers.

 

Red hat suggest the following command to check if your ntp server is going to issue the leap second. Running this in on or after the 1st December 2016 should show a leap second is being issued:

ntpq -c "lassoc" -c "mrv &1 &999 leap,srcadr,stratum"

 

the output will look something like:

[root@host ~]$ ntpq -c "lassoc" -c "mrv &1 &999 leap,srcadr,stratum"

 

ind assid status  conf reach auth condition  last_event cnt

===========================================================

  1 37228  963a   yes   yes  none  sys.peer    sys_peer  3

srcadr=lon-switch.bmc.com, leap=00, stratum=2

If you have "leap=01" rather than "00", the leap second will be accounted for.

 

Red Hat have released a "Leap Second Issue Detector" (https://access.redhat.com/labs/leapsecond/) script. You will need a Red Hat account.

 

References (some may require Red Hat login):

IERS notification of leap second (https://datacenter.iers.org/eop/-/somos/5Rgv/latest/16)

Resolve Leap Second issues in Red Hat Enterprise Linux (https://access.redhat.com/articles/15145)

Interesting graph showing millisecond offset (http://tycho.usno.navy.mil/leapsec.html)

Previously issued Red Hat workarounds (https://access.redhat.com/articles/199563)

Simulate a ntp leap second (https://access.redhat.com/solutions/1333383)