Sometimes you have to face nasty service crashes, e.g. due to memory leaks (yes,
logstash, I’m talking about you).
Using the systemd timers feature is a handy and idiomatic way to restart systemd services (ofc we could use cron, but that’s not what this is about anyway).
However, everyone knows by now that
systemd can be mystical sometimes and its arcanes are somehow complex to understand.
That being said, I want to restart my logstash service 4 times after it’s been started. Therefore I need two files:
- A service unit file that restarts my logstash service
- A timer file that triggers my restart service
The restart service unit file looks straightforward:
# /etc/systemd/system/logstash-restart.service [Unit] Description=restart logstash [Service] Type=oneshot ExecStart=/bin/systemctl restart logstash
However, I struggled a bit on how to restart the service every 4 hours after activation.
I did not want to use the
OnCalendar option because that would be akin to using cron and wouldn’t leverage the granularity of systemd timer functions.
As it turns out, timers can be scheduled to be activated at a given time interval after they’ve been started with the
OnActiveSec directive but that would only kick in once.
The timer needs to be told to activate again after its initial activation with the
Eventually, the time when the timer was activated can be stored on disk and that can be done with the
Persistent option. This option is false by default, which is OK because it doesn’t need to be enabled in my case - I want the timer to activate only after the service has been running for a given number of hours.
This gives us the following timer file:
# /etc/systemd/system/logstash-restart.timer [Timer] OnActiveSec=4h OnUnitActiveSec=4h [Install] WantedBy=timer.target
With both of those files on disk, there remains the need to enable and start the timer (that creates a symbolic link at the appropriate target location so it will be started at next boot, and also starts the countdown from now on):
# systemctl enable logstash-restart.timer # systemctl start logstash-restart.timer
Checking if the timer shows up:
# systemctl list-timers --all NEXT LEFT LAST PASSED UNIT ACTIVATES Fri 2017-11-17 20:32:19 CET 3h 59min left Thu 2017-11-16 19:50:23 CET 20h ago logstash-restart.timer logstash-restart.service
systemd tracks execution times closely so it’s easy to tell whenever the timer has fired and when it will activate next.