In case it's of use, I wrote a couple of scripts to run a command, killing it if it took more than a specified timeout, and reporting how long it took to run:

• Timing and timing out Unix commands (using fork, exec, kill, waitpid)
• Timing Windows commands (first attempt using Win32::Process)
• Re: Timing Windows commands (improved version using Win32::Job)

Update: Using this node nowadays for links on Threading and Concurrency.

Perl Documentation

• threads (perldoc) - The use of interpreter-based threads in perl is officially discouraged
• perlfork (perldoc) - Perl's fork() emulation
• Perl 5 issue: Why are threads discouraged? #14691 (github)
• perlipc (perldoc) (Originally by TomC, very comprehensive)
• IO::Select (perldoc) (handy when writing single-threaded event-driven servers in Perl)

CPAN

Perl monk marioroy has produced some excellent CPAN modules in this domain:

• MCE by marioroy (Many-Core Engine for Perl providing parallel processing capabilities - can be configured to use processes or threads (via use_threads parameter), by default MCE spawns child processes on UNIX platforms and threads on Windows)
• MCE::Grep by marioroy
• Mutex by marioroy
• MCE::Shared by marioroy

See also:

• Parallel::ForkManager by Gabor Szabo (father of the Perl Maven web site) and the infamous Yanick Champoux

Event-driven Programming

Event-driven programming is an attractive alternative to threads, especially when using scripting languages. For Perl, there is:

• POE (portable multitasking and networking framework for any event loop) by Chris Williams (originally developed by Rocco Caputo)
• Mojo::IOLoop (Minimalistic event loop) by Sebastian Riedel (part of Mojolicious)
• AnyEvent by Marc Lehmann

See also:

• Re: Multiple consecutive connections to a socket (probably a stupid question) - a simple single-threaded event-driven server in Perl using IO::Select
• Discussion of differences between event-driven and thread-based servers (SO)

As noted by marioroy, chunking complements event-driven programming. See also: event-driven references in the Wikipedia References section below.

Some further chunking examples from marioroy:

• Re^2: MCE segmentation fault
• Re^2: Problem in RAM usage while threading the program
• Re: What is the most efficient way to split a long string (see body for details/constraints)?

Some Perl Monks Threading and Concurrency Nodes

• Threads, Multi-Processing, and Inter-Process Communication - a list of sanctioned nodes by PM pedagogues (2006)

• Missing (i)threads Features? by liz (2003)
• Things you need to know before programming Perl ithreads by liz (2003)

• Re: Writing Solid CPAN Modules by BrowserUk (2005)
• Use more threads. by BrowserUk (2006)
• Parrot, threads & fears for the future. by BrowserUk (2006)
• The problem with "The Problem with Threads" by BrowserUk (2014)

• Re: Spawning processes using Win32::Process by me (2005)
• Comparing close-on-exec behaviour between Unix and Windows by me (2006)
• Re^4: Nobody Expects the Agile Imposition (Part VI): Architecture (Signals are a crock) by me (2011) - discusses early history of Windows NT, especially signals, threads and its POSIX subsystem

• returning from a thread ? by Foggy Bottoms (2003)
• Perl threads test intermittently hangs on Windows by me (2004)
• Perl threads test on Windows: how to close thread handles? by me (2004)
• Perl threads sort test program crashes by me (2004)
• Replacing closures (to work around threads crash) by me (2004)
• perl threads and dual core hyper-threading by JFarr (2006)
• How to write a test for backticks threads hang by me (2006)
• perl multithreading by spx2 (2007)

• UnitTesting multi-threaded apps by alain_desilets (2011)
• Threads and HTTPS Segfault by onelesd (2011)
• Threads and signals in Windows by bojinlund (2013)

• Re: what is the meaning of $| in perl? (autoflush) by me (2014) - using autoflush on log files
• The 10**21 Problem (Part I) by me (2014) - Four-part series on HPC (solving an embarrassingly parallel problem)

• Trying to Understand the Discouragement of Threads by benwills (2014)
• How to use multithreading in perl? by anonymonk (2014)
• Can I/O operations on the same IO::Socket be executed in different threads? by anonymonk (2015)
• Threads From Hell #2: How To Search A Very Huge File [SOLVED] by karlgoethebier (2015)
• Can we write multi-threaded scripts ? by toughy (2018) (especially response by choroba)
• Problem in RAM usage while threading the program by beherasan (2019)

• Calling External Commands More Safely by haukex (2017)
• Re^2: Having to manually escape quote character in args to "system"? by salva (2017)
• The problem of "the" default shell by afoken (2017)

• Detect whether a writeable filehandle has closed? by jdporter (2021)
• Signals vs. Windows by DanEllison (2012)
• trying to get timeout to work by Skeeve (2012)
• Windows run an exe print its output and cut it off after a time by anshumangoyal (2012)
• Backticks and SIGALRM by nemo (2007)
• Check for another program availability by hrcerq (2021)

• Multi threading a by dnamonk (2021)
• Perl Multi Processing. by pritesh_ugrankar (2021)
• MCE: How to access variables globally by biohisham (2021)
• Re^3: Killing subprocess in windows by me (2021)

• Perl forked processes and variable sharing by fireblood (2022) (see especially responses by dave_the_m and marioroy)
• Sharing STDIN after fork by Bod (2022)
• XS module in ithreads Perl much slower in threads::join after adding SvOBJECT_off by etj (2022)
• Can I have a Perl script, initiated from a browser, fork itself, and not wait for the child to end? by bartender1382 (2022)
• Parallel processing on Windows by BernieC (2022)

Unit Testing Concurrent Code

Test-driven development, a practice enabling developers to detect bugs early by incorporating unit testing into the development process, has become wide-spread, but it has only been effective for programs with a single thread of control. The order of operations in different threads is essentially non-deterministic, making it more complicated to reason about program properties in concurrent programs than in single-threaded programs.

-- from a recent PhD proposal to develop a concurrent testing framework

See the "Testing Concurrent Software References" section below for more references in this active area of research. Though I haven't used any of these tools yet, I'd be interested to hear from folks who have or who have general advice and tips on how to troubleshoot and fix complex concurrency-related bugs. In particular, I'm not aware of any Perl-based concurrent testing frameworks.

In practice, the most effective, if crude, method I've found for dealing with nasty concurrency bugs is good tracing code at just the right places combined with understanding and reasoning about the code, performing experiments, and "thinking like a detective".

One especially useful experiment (mentioned in Clean Code) is to add "jiggle points" at critical places in your concurrent code and have the jiggle point either do nothing, yield, or sleep for a short interval. There are more sophisticated tools available, for example IBM's ConTest, that use this approach to flush out bugs in concurrent code.

Testing Concurrent Software References

• Testing Concurrent Programs
• Chess: Tool for finding bugs in concurrent programs
• stack overflow question on testing concurrent software
• Intel Parallel Inspector
• Java Concurrency in Practice
• Clang ThreadSanitizer
• Parasoft Jtest

Wikipedia References

• Comparison of programming paradigms
• Thread (computing)
• Multithreading (computer architecture)
• Computer multitasking
• Multi-core processor
• Simultaneous multithreading
• Thread pool
• Thread safety
• Signal (IPC)
• C signal handling
• Win32 Thread Information Block

• Event-driven architecture
• Event-driven programming
• Cooperative multitasking
• Asynchronous IO
• Node.js (Javascript)
• Perl Object Environment
• Twisted (Python)
• EventMachine (Ruby)
• Reactor pattern

• Real-time computing
• Worst-case execution time

• Grid computing
• Parallel computing
• Concurrent computing
• Massively parallel
• Embarrassingly parallel
• Supercomputer
• Neural network

• Cloud computing
• Distributed computing
• Volunteer computing
• SETI@home
• Great Internet Mersenne Prime Search
• List of distributed computing projects

• GPGPU
• OpenCL
• CUDA

• Hardware acceleration
• Field-programmable gate array (FPGA)
• Verilog

See Also

• "HPC (High Performance Computing) in Perl and C++" section on my home node
• Re: "Magic tools" that take the fun away (Build/DevOps/Rex References) - mentions GNU parallel, a Unix command line tool, written in Perl, that allows the user to execute shell scripts or commands in parallel ... similar to pexec and xargs.
• Re^2: Organizational Culture (Part II): Meta Process (BioPerl/PDL and Data Science References)
• Re: Perl Memory problem ... (Memory Tools)
• Re: Check for another program availability (Running External Processes on Unix and Windows References)

Updated: Extra references added long after the original reply was made.