If you have a running process, one that you want to keep running, on one of your servers that needs to be supervised, then Daemontools is, in my opinion, the perfect choice. It can keep a process running, keep logs on those processes, and just simply give you one less thing to monitor on your server. Most will prefer to use upstart, which is fine, but I think it sucks compared to Daemontools.
What is Daemontools?
daemontools is a collection of tools for managing UNIX services.
supervise monitors a service. It starts the service and restarts the
service if it dies. Setting up a new service is easy: all supervise needs
is a directory with a run script that runs the service.
multilog saves error messages to one or more logs. It optionally
timestamps each line and, for each log, includes or excludes lines matching
specified patterns. It automatically rotates logs to limit the amount of
disk space used. If the disk fills up, it pauses and tries again, without
losing any data.
Why should you use Daemontools?
Observe this link.
/service better than
rc.local? Why should my package rely on
Answer: Several reasons:
|Easy service installation and removal||No||No||Yes||No||Yes|
|Easy first-time service startup||No||No||No||No||Yes|
|Easy, reliable signalling||No||No||No||No||Yes|
|Clean process state||Yes||Yes||No||No||Yes|
Easy service installation and removal.
init.d, creating your new service means linking some files into a centralized directory, and removing the service means removing those files. This is easy for you to automate.
In contrast, with
rc.local, creating a new service means adding some lines to a centralized file, and removing the service means locating and removing those lines. This is much more difficult to automate; there are no portable tools for editing the file. Most packages leave it to the user to do the editing.
Easy first-time service startup.
/service, once you've created your service, it automatically starts within five seconds.
In contrast, with
rc.local, starting the service means an extra command.
ttys, if your daemon dies, it is automatically restarted.
In contrast, with
rc.local, daemons are not monitored. Your daemon will fail to start if, for example, a previously started daemon has temporarily chewed up almost all available memory. The system administrator has to manually restart your daemon after he notices the problem.
Easy, reliable signalling.
/service, the system administrator can use
svc to control your daemon. For example:
svc -h /service/yourdaemon:sends HUP
svc -t /service/yourdaemon:sends TERM, and automatically restarts the daemon after it dies
svc -d /service/yourdaemon:sends TERM, and leaves the service down
svc -u /service/yourdaemon:brings the service back up
svc -o /service/yourdaemon:runs the service once
In contrast, with
init.d , and rc.local, you have to go to extra work to locate the daemon process ID if you want to send it signals. This is not easy to do reliably.
Clean process state.
ttys, when the system administrator restarts a service, the service receives the same fresh process state that it received at boot time.
In contrast, with init.d and rc.local, you have to go to tons of extra work to clean up environment variables, resource limits, controlling ttys, etc. Programmers screw this up all the time, even when they don't care about portability to systems with different process states; system administrators then encounter mysterious failures when they restart daemons.
/service, your program works the same way on every system: Linux, BSD, Solaris, etc.
rc.local vary from system to system. Some systems don't have
init.d, for example, and the ones that do have it don't agree on its location. This is extremely annoying for cross-platform system administrators.
NOTE: Everything needs to be ran as root
mkdir -p /package chmod 1755 /package cd /package wget http://cr.yp.to/daemontools/daemontools-0.76.tar.gz tar -xpf daemontools-0.76.tar.gz rm -f daemontools-0.76.tar.gz cd admin/daemontools-0.76 package/install
IF INSTALL FAILS
ed ./src/conf-cc 1s/$/ -include errno.h/ wq
apt-get install csh "(for Ubuntu/Debian users)" csh -cf '/command/svscanboot &'
Starting Daemontools on Boot
sed -i "1 a\csh -cf '/command/svscanboot &'" /etc/rc.local chmod +x /etc/rc.local
I personally like putting my services in the
/services directory as
daemontools will look for a new service in the
/service directory. It
makes it easier for tab completion. :-)
mkdir /services mkdir /services/somerandomservice echo -e '#!/bin/sh\nexec somerandomcommand' > /services/somerandomservice/run chmod 755 /services/somerandomservice/run ln -s /services/somerandomservice/ /service/
Linking your service's daemon directory to your
/service directory will
tell daemontools to bring your service up. You can check the uptime of your
(note: if your service does not keep a longer uptime than 1 second, then something is wrong and you need to check your run script, see debugging for more info)
Run Service With a Specific User
To do this is no different from making a service. You'll just use the
setuidgid tool that ships with daemontools.
mkdir /services/somerandomuserservice ed /services/somerandomuserservice/run a #!/bin/sh exec setuidgid somerandomuser somerandomcommand . w !chmod 755 % q ln -s /services/somerandomuserservice/ /service/
Now with some Ruby applications, you may experience problems with this method because
setuidgid is much more simple from a full login. It only sets the uid and gid as the name implies. If you need the full login functionality then try this:
mkdir /services/somerandomuserservice ed /services/somerandomuserservice/run a #!/bin/sh exec su - somerandomuser -c 'exec somerandomcommand' . w !chmod 755 % q ln -s /services/somerandomuserservice/ /service/
adduser logger mkdir /services/somerandomservice/log mkdir /services/somerandomservice/log/main chown logger /services/somerandomservice/log/main ed /services/somerandomservice/log/run a #!/bin/sh exec setuidgid logger multilog t ./main . w !chmod 755 % q
I can only help you with debugging daemontools here. Drop in #isotope11 on freenode if you have further problems, and someone will surly help.
Checking the run script
cd /service/somerandomservice svc -d . ./run
Take note of any errors that are preventing your run script from running. Do they ring any bells? Have you Googled the error?
After you've fixed your bug, bring your service back up with:
svc -u /service/somerandomservice
Checking the logs
cd /service/somerandomservice tail -n5 log/main/current
If your processes is throwing any errors in the system, they will be logged here. If
current isn't there, then it has not had an error yet and you can celebrate.
Restart a service
svc -t /service/somerandomservice
rm /service/somerandomservice svc -dx /services/somerandomservice
David Chapman is a Ruby on Rails developer at isotope|eleven with over 5 years development experience. He's currently majoring in Electrical Engineering with a minor in Computer Science at the University of Alabama at Birmingham. David is a Linux geek, a math geek, a physics geek, a serious gamer, and a reader of Reddit. His super power is eating chips and salsa.