Two nights ago, a leap second was added to system clocks running Linux, causing much-undesired havoc. On July 1st at 12:00 AM UTC, both of my Amazon EC2 instances fired an alarm indicating high CPU usage. I investigated to find that it was MySQL that was eating all of the CPU. I logged in and ran SHOW PROCESSLIST to find that no queries were running (these servers don't get hit much after business hours). I stopped MySQL, CPU utilization dropped back down to 1-3% (as normal). I restarted MySQL, and it started eating a lot of CPU again. Then, I restarted the server (shutdown -r now), and the problem went away. Both servers had the exact same problem (running Ubuntu 12.04 LTS). In my particular case, MySQL began eating CPU, even after being restarted. It was a livelock. The only relevant item I saw in the syslog was: Jun 30 23:59:59 hostname kernel: [14152976.187987] Clock: inserting leap second 23:59:60 UTC Oh yeah... leap seconds. Those are super impor...
Blog about science and technology, especially software development and space ships. ;) Blake is a software developer who specializes in building inventory management and project management solutions for small or mid-sized businesses. He also spends a fair amount of time on embedded software and database research.