Re^2: Proc::PID::File problem generating pid files, or: does it matter where a pid file lives?

Not in principle. Pid files have their problems (something may throw away the file, the pid may have been reused, race conditions if you don't use locking (but if you use locking, why have a pid file?)), but they are just a file.

In discussing whether or not a "pid" (or "lock") file is good or bad, it is useful to define what one should be used for. The point of the lock file is to signal other processes that access to a system resource is controlled (for whatever reason).
Take a simple, single user environment. If I start a program in a window that utilizes several specific data files, and those file should only be updated by my program, the simple way is to implement file locking. That way, if another instance of the program is started, it will fail on file access. Not very pretty (if multiple files are accessed, you have to lock all of them), but effective. A better way is to check for the instance of the lock file - a way of saying "this set of resources is in use". Cleanup is easy. It also allows for graceful recovery when the user mistakenly starts another instance of the program.
Now, extend that to long running processes (daemons or server processes) that require exclusive access to specific system resources (database, web, ftp etc). Such access may be relatively easy to handle (network ports, etc), or relatively expensive (cleanly shutting down a large database server can be time consuming, with failure extremely painful). Under these circumstances, you would need to be able to control the server from an arbitrary location (not necessarily the terminal process from which it was started).
Enter the "pid" file (a file whose existance signals that the service is - or was - running). The content of the file is the process id (hence the name) of the controlling process. The file's (non)existance can communicate the state of the process:

File exists, PID points to running process:
Controlling process is running, should be available (not always true).
File exists, PID points to non-existant process:
Process was running, but did not shut down cleanly. Some cleanup will probably be necessary.
File does not exist, but process appears in system process table:
- Programming logic error: Process did not properly create environment on startup (more information later).
- Some other process deleted the PID file.
File does not exist and process does not appear in system process table:
Startup should be safe.

When starting up such a server process (which will usually be started as the "root" user), the following sequence should normally be followed:

Check for existence and program access to:
- PID directory (typically /var/run or /usr/local/var/run)
- Log file directory
- Data file directory
- Other system resources (network ports, specific hardware)
Reset user credentials (first reset group ID, then user ID)
Create PID file and lock for exclusive access.
Open log file(s)
Allocate resource(s)

Orderly process shutdown is generally in the reverse order.

As noted above, the PID file is typically located in /var/run (or /usr/local/var/run). This allows for orderly and generic startup and shutdown procedures, as well as troubleshooting. If you do not have to go searching for pid files, it is much quicker.

The PID file also serves another purpose: it allows external resources to communicate with the service in a specified manner. Take, for instance, the "cron" process, which provides scheduled processes to run. In order to maintain the process, you can either edit the configuration files and restart the process, or you can send it a message to reread it's files. The "standard" unix method for maintaining the configuration is through the "crontab" program. When the configuration is changed (by crontab -e or crontab -r, the program sends a "SIGHUP" signal to the process whose id is contained in the PID file. The cron process does not shut down, but simply rereads it's configuration.
The "syslog" daemon works on the same basis. If you were to shut down the daemon, then restart it, you would lose messages. Instead, it simply rereads the configuration file on the fly and acts appropriately.

Yes, the use of such files adds some complexity to the program, but in many cases, the added functionality is worth it (at least in my not so humble opinion :-D ). YMMV.

Update: Added explicit reference to pid file location.

Comment on Re^2: Proc::PID::File problem generating pid files, or: does it matter where a pid file lives?

Replies are listed 'Best First'.
Re^3: Proc::PID::File problem generating pid files, or: does it matter where a pid file lives? by afoken (Chancellor) on May 22, 2010 at 15:36 UTC
Check for existence and program access to: PID directory (typically /var/run or /usr/local/var/run) Log file directory Data file directory Other system resources (network ports, specific hardware) Reset user credentials (first reset group ID, then user ID) Create PID file and lock for exclusive access. Open log file(s) Allocate resource(s) Are you aware that this still has a race condition? You run a lot of tests in step 1, most of those tests involve system calls. Step 2 has two system calls. Each and every system call may cause a task switch to a malicious program that -- with a little bit of luck and good timing -- can change what you checked for in step 1, causing the following steps to fail rather unexpectedly. And each and every system call may cause a task switch to a second instance fighting for the PID file. Daemons do not need PID files, and most daemons contain code that they don't really need, for backgrounding, logging, restarting, dropping privileges, and to prevent multiple instances. The daemontools reduce code complexity in daemons and they take care of backgrounding, logging, restarting, dropping privileges, and single instances. Even communication via signals works completely without PID files (with a patch, SIGUSR1 and SIGUSR2 can also be sent). Daemontools may look strange, and some of DJBs decisions (errno, location in filesystem, ...) may cause a little bit of confusion, but once you unterstand what happens, the daemontools are the most natural way to implement daemons on Unix and derivates. Alexander -- Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)	[reply]
Re^4: Proc::PID::File problem generating pid files, or: does it matter where a pid file lives? by proceng (Scribe) on May 22, 2010 at 17:36 UTC
Are you aware that this still has a race condition? You run a lot of tests in step 1, most of those tests involve system calls. Step 2 has two system calls. Each and every system call may cause a task switch to a malicious program that -- with a little bit of luck and good timing -- can change what you checked for in step 1, causing the following steps to fail rather unexpectedly. And each and every system call may cause a task switch to a second instance fighting for the PID file. Alexander: Yes, I am quite aware that there may be a race condition (but also that the condition may be rare). Since many of the steps, by their nature, can not be atomic, that is a chance that is taken. I would rather know before attempting to connect to a database (for example) that it could be in an inconsistent state due to an abnormal termination. While everything may work right, simply knowing about the problem can go a long way to preventing it from happening in the future. Many of the options (like the daemontools that you note), do the same things that I describe (just masking them from the programmer). While you are correct that ~~many~~some daemons do not need certain functions, that does not mean that they do not have their place. Add to that the fact that packages such as daemontools have their own built in set of limitations: System requirements daemontools works only under UNIX. and that they require an extra installation step to accomplish the same goal, and you can see that it is useful to know what is being done and why. In my opinion, when you have to adjust to another's decisions (and add complexity at the same time), it is not necessarily a good thing.	[reply]
Re^3: Proc::PID::File problem generating pid files, or: does it matter where a pid file lives? by JavaFan (Canon) on May 31, 2010 at 09:06 UTC
The PID file also serves another purpose: it allows external resources to communicate with the service in a specified manner. I would say, that's the only reason to use PID files. There's no need to use PID files to prevent simultaneous access to resources; for that, lock files are enough. And if all you care about is preventing concurrent running of the same program (which is what the OP needs), all you need to do is obtain a lock on yourself (no external files needed): `flock DATA, LOCK_EX or die "Another instance is already running";` [download]	[reply] [d/l]


No such thing as a small change
	PerlMonks