This site is now 100% read-only, and retired.

Using runit for maintaining services

Posted by Steve on Sat 4 Jan 2014 at 10:38

There are several programs, and systems, out there for supervising the execution of child-processes. Perhaps one of the most well-known is runit, which was modelled upon the daemontools software by djb.

Once installed the runit package allows you to easily deploy and control services:

  • Finding them.
  • Starting them.
  • Stopping them.

One of the biggest reasons for using runit is that services restart automatically when they crash, or otherwise exit. For that purpose I use it to launch the vast majority of my node.js applications.

Although we've been talking about the control of services I should make it clear that we're not talking about system servers - although runit could be used in that way if you prefer, we're explicitly not talking about replacing init, we're talking about controlling user-installed services.

To get started install the package:

root@shelob:~# apt-get install runit

Once you've done that you'll find a new daemon has been launched, via an addition to your /etc/inittab file. There are a few processes involved but we mostly don't need to care about them. The important thing about runit is that it launches processes by scanning a particular directory - /etc/service - looking for subdirectories which contain executable scripts named run.

If you create a new directory, and inside that create an executable file called run it will be launched, and relaunched if it exits.

For example if you were to create the directory /etc/service/test, then save the following contents to the file /etc/service/test/run you'd have created a new service:

touch /tmp/blah

This service would run, executing the run script, and then it would exit. Then it would start again, over and over again until you stopped it.

Stopping the service is a matter of using the most useful command installed via runit: sv. The sv command can be used to check on the statutus of a service:

root@shelob:~# sv status /etc/service/test
down: /etc/service/test: 0s, normally up, want up

root@shelob:~# sv status /etc/service/test
down: /etc/service/test: 1s, normally up, want up

It can also be used to start a service, or stop it:

root@shelob:~# sv stop /etc/service/test/
ok: down: /etc/service/test/: 0s, normally up, want up

root@shelob:~# sv status /etc/service/test/
down: /etc/service/test/: 17s, normally up

This trivial service isn't terribly interesting, continuously touching a file. But for simple deamons such as node.js-based servers it can be very useful.

Simply create a directory, and inside that place an executable script named run with contents such as this:

# Run our API - v2 - restarting on failure.

#  Steve deploys node beneath /opt/node
if [ -d /opt/node ]; then
    export PATH=/opt/node/bin:$PATH

# Ensure we're current.
git pull --update --quiet

# Install any missing modules, via npm.
if [ -e "package.json" ]; then
    npm install

#  Finally launch the service, with suitable logging.
exec node server.js 2>&1 >> /var/log/node.js.log

That's all it takes to :

  • Refresh the service from a remote git repository.
  • Launch it.
    • Being confident that when the service exits it will be automatically restarted.

If ever I want to update my service I can either run sv stop ..", followed by "sv start ..", or I can run:

root@shelob:~# sv restart /etc/service/blogspam.js/
ok: run: /etc/service/blogspam.js/: (pid 14712) 0s

It is possible to run many scripts like this, for example you can launch SSH under runit by removing the standard init-script, and creating a file /etc/service/sshd/run:

exec /usr/sbin/sshd -D

You'd not see much gain, or difference, but if your system ran out of RAM and the SSH service died it would be restarted if you went down this route. Similarly you can deploy a service that a user can have permission to restart, simply execute the process under their user-ID, and they have permission to "kill -9" it, and it will then restart:

su - user -c "exec /home/user/"

I've used that technique before to ensure a particular webserver deamons are running, and still allowing the users to restart them if they wished:

exec thttpd -D -C /srv/ -M 3600

In this case the thttpd webserver changes to a particular user-id after it launches:

root@www ~# ps -ef | grep s-steve
s-steve   7792  2752  0 04:25 ?        00:00:00 thttpd -D -C /srv/ -M 3600

Now that user can kill it, to trigger a restart:

s-steve@www:~$ kill 7792

As expected the service is alive again:

root@www~# sv status /etc/service/
run: /etc/service/ (pid 16948) 32s

The runit website contains a small collection of run-scripts which can be useful to examine; they've very handy if you have a deamon, or two, which you need to launch and which doesn't come with a real initscript.