In case it's of use, I wrote a couple of scripts to run a command, killing it if it took more than a specified timeout, and reporting how long it took to run:

• Timing and timing out Unix commands (using fork, exec, kill, waitpid)
• Timing Windows commands (first attempt using Win32::Process)
• Re: Timing Windows commands (improved version using Win32::Job)

Update: Using this node nowadays for links on Threading and Concurrency.

Perl Documentation

• threads (perldoc) - The use of interpreter-based threads in perl is officially discouraged
• perlfork (perldoc) - Perl's fork() emulation
• Perl 5 issue: Why are threads discouraged? #14691 (github)
• perlipc (perldoc) (Originally by TomC, very comprehensive)
• IO::Select (perldoc) (handy when writing single-threaded event-driven servers in Perl)

CPAN

Perl monk marioroy has produced some excellent CPAN modules in this domain:

• MCE by marioroy (Many-Core Engine for Perl providing parallel processing capabilities - can be configured to use processes or threads (via use_threads parameter), by default MCE spawns child processes on UNIX platforms and threads on Windows)
• MCE::Grep by marioroy
• Mutex by marioroy
• MCE::Shared by marioroy

See also:

• Parallel::ForkManager by Gabor Szabo (father of the Perl Maven web site) and the infamous Yanick Champoux
• Net::Server - extensible, general Perl server engine by Paul Seamons

Event-driven Programming

Event-driven programming is an attractive alternative to threads, especially when using scripting languages.

For Perl, Mojolicious seems to be the most popular nowadays, as described at:

• Re: Daemonizing (or otherwise speeding up) a high-overhead script? (Mojolicious References)

See also:

• POE (portable multitasking and networking framework for any event loop) by Chris Williams (originally developed by Rocco Caputo)
• AnyEvent by Marc Lehmann

• Re: Multiple consecutive connections to a socket - example event-driven server using IO::Select - a simple single-threaded event-driven server in Perl using IO::Select
• Discussion of differences between event-driven and thread-based servers (SO)

Chunking

As noted by marioroy, chunking complements event-driven programming.

Some further chunking examples from marioroy:

• Re^2: MCE segmentation fault
• Re^2: Problem in RAM usage while threading the program
• Re: What is the most efficient way to split a long string (see body for details/constraints)?

See also: event-driven references in the Wikipedia References section below.

TLS/SSL Related

On CPAN:

• Net::SSLeay
• Protocol-TLS
• Protocol::TLS::Server
• Protocol::TLS::Client

• Net::LDAP

Some Perl Monks Threading and Concurrency Nodes

• Threads, Multi-Processing, and Inter-Process Communication - a list of sanctioned nodes by PM pedagogues (2006)

• Missing (i)threads Features? by liz (2003)
• Things you need to know before programming Perl ithreads by liz (2003)

• Re: Writing Solid CPAN Modules by BrowserUk (2005)
• Use more threads. by BrowserUk (2006)
• Parrot, threads & fears for the future. by BrowserUk (2006)
• The problem with "The Problem with Threads" by BrowserUk (2014)

• Multithreading, how to? by iSina (2008) - lots of responses by BrowserUk
• Re^6: Multithreading, how to? by BrowserUk

• Re: Spawning processes using Win32::Process - simple example of forking many like processes by me (2005)
• Comparing close-on-exec behaviour between Unix and Windows by me (2006)

• returning from a thread ? by Foggy Bottoms (2003)
• Perl threads test intermittently hangs on Windows by me (2004)
• Perl threads test on Windows: how to close thread handles? by me (2004)
• Perl threads sort test program crashes by me (2004)
• Replacing closures (to work around threads crash) by me (2004)
• perl threads and dual core hyper-threading by JFarr (2006)
• How to write a test for backticks threads hang by me (2006)
• perl multithreading by spx2 (2007)

• UnitTesting multi-threaded apps by alain_desilets (2011)
• Threads and HTTPS Segfault by onelesd (2011)

• Re: what is the meaning of $| in perl? (Buffering/autoflush/Unicode/UTF-8 References) by me (2014) - using autoflush on log files
• The 10**21 Problem (Part I) by me (2014) - Four-part series on HPC (solving an embarrassingly parallel problem)

• Trying to Understand the Discouragement of Threads by benwills (2014)
• Threading with a twist by scorpion7 (2017) - see responses by ikegami and Discipulus (re discouragement of threads)

• How to use multithreading in perl? by anonymonk (2014)
• Can I/O operations on the same IO::Socket be executed in different threads? by anonymonk (2015)
• Threads From Hell #2: How To Search A Very Huge File [SOLVED] by karlgoethebier (2015)
• Windows 10 and Win32::Process by Alphaphi (2018)
• Can we write multi-threaded scripts ? by toughy (2018) (especially response by choroba)
• Problem in RAM usage while threading the program by beherasan (2019)

• Calling External Commands More Safely by haukex (2017)
• Re^2: Having to manually escape quote character in args to "system"? by salva (2017)
• The problem of "the" default shell by afoken (2017)

• Detect whether a writeable filehandle has closed? by jdporter (2021)
• trying to get timeout to work by Skeeve (2012)
• Windows run an exe print its output and cut it off after a time by anshumangoyal (2012)
• Backticks and SIGALRM by nemo (2007)
• Check for another program availability by hrcerq (2021)

• Multi threading a by dnamonk (2021)
• Perl Multi Processing. by pritesh_ugrankar (2021)
• MCE: How to access variables globally by biohisham (2021)
• Re^3: Killing subprocess in windows by me (2021)

• Perl forked processes and variable sharing by fireblood (2022) (see especially responses by dave_the_m and marioroy)
• Sharing STDIN after fork by Bod (2022)
• XS module in ithreads Perl much slower in threads::join after adding SvOBJECT_off by etj (2022)
• Can I have a Perl script, initiated from a browser, fork itself, and not wait for the child to end? by bartender1382 (2022)
• Inconsistent results with keys on thread shared hashes by izomiac (2022) - response by ikegami

• Parallel processing on Windows by BernieC (2022)
• fork() on win10 by BernieC (2023)
• Parallel Processing on win10 by BernieC (2023)

• Rosetta Code: Long List is Long (2022-2023) - lots of instructive replies from marioroy
• Risque Romantic Rosetta Roman Race (2023) - lots of instructive replies from marioroy using MCE

• MCE Sandbox 2023-08 by marioroy - deliberately designed for necroposts (added in replies as he learns new things)
• [OT] The Long List is Long resurrected by marioroy (2024) (mentions Taichi Lang, a DSL embedded in Python)

• How to portably stop processes? by haj (2023)
• [Win32] differing threads behaviour by syphilis (Apr 2023) - Mingw-w64 moving from pthreads to mcf threads?
• Using AnyEvent to create a TLS server by Bodger (Aug 2023) - writing an Event Loop TLS Server
• AnyEvent server/client don't cooperate by yuri123 (Oct 2023) - writing an AnyEvent Server

• Daemonizing (or otherwise speeding up) a high-overhead script? by anonymonk (Aug 2023) - Catalyst (MVC) script with 30+ seconds startup time (Web framework, Model–view–controller)
• Re: Daemonizing (or otherwise speeding up) a high-overhead script? by Corion. Mentions PPerl - implements a prefork server for arbitrary scripts; the idea is that it launches, does the costly initialization, and then forks into the background ... then, if you ever launch the script again, it will connect to the background server and skip the costly initialization. Minion job queue also mentioned.

• IPC::Open, Parallel::ForkManager, or Sockets::IO for parallelizing? by mldvx4 (Sep 2023) - replies mention Mojolicious, Mojo::Promise, Net::Async::HTTP and Future (by Paul Evans), LWP::Parallel, MCE, Thread::Queue, Forking::Amazing, ...
• Perl and Threading by scorpion7 (Nov 2023) - panic: del_backref, *svp=0 phase=DESTRUCT refcnt=3 during global destruction; choroba reply recommends Thread::Queue

• Systemd socket activated service in perl by chrestomanci (2024) - Unix forking server running as a long running systemd daemon; wants to convert to a systemd socket activated service (see e.g. systemd.socket)
• Re^3: Why is my counter not updating in a threaded application? by hv - contrasts processes vs threads, discusses IPC mechanisms (pipes, shared memory, databases)

Windows and Strawberry Perl

• Strawberry Perl (wikipedia)
• Mingw-w64 (wikipedia) - gcc-based C/C++ compiler used by Strawberry Perl (Note: MSYS2 ("minimal system 2") is a software distribution and a development platform for Microsoft Windows, based on Mingw-w64 and Cygwin ... MSYS2 and its bash environment is used by Git)
• Cygwin (wikipedia) - a Unix-like environment and command-line interface for Microsoft Windows (uses cross-compilers from Mingw-w64 project)
• How is fork() implemented in Cygwin? - Cygwin fork() essentially works like a non-copy on write version of fork() (like old Unix versions used to do). Because of this it can be a little slow. In most cases, you are better off using the spawn family of calls if possible.
• Strawberry Perl (official site)
• Strawberry Perl (github)

• Re: Windows precompiled binaries or DIY compile by syphilis (Sep 2023) : Perl 5.38.0 builds easily on Windows, and "straight out of the box" - using either Visual Studio 2022 or mingw-w64 ports of gcc-12 or gcc-13
• WinLibs standalone build of GCC and MinGW-w64 for Windows : winlibs.com - a free C and C++ compiler for Microsoft Windows
• Microsoft Visual Studio 2022 Community Edition

• berrybrew (github) by stevieb - perlbrew for Windows Strawberry Perl

Parallel::ForkManager

• Why is my counter not updating in a threaded application? by cormanaz (Mar 2024) - threads vs processes using Parallel::ForkManager (see RETRIEVING DATASTRUCTURES from child processes)
• Re: Why is my counter not updating in a threaded application? by marioroy - examples using Parallel::ForkManager and MCE::Child, MCE::Shared and MCE

• Parallel:ForkManager how to pass back a list by cormanaz (Apr 2024) - more on retrieving data structures from child processes
• Re: Parallel:ForkManager how to pass back a list - davido replies that for version 0.7.6+, each child process may optionally send 1 data structure (a reference to a string, hash or array) back to the parent
• Parallel::ForkManager install fails by cormanaz (Apr 2024) - ikegami notes that Parallel::ForkManager doesn't make sense in Windows, which doesn't have native fork

• Parallel Processing on win10 by BernieC (2023) - marioroy, me and ikegami respond, don't use fork on Windows
• running a function for different list parallely using fork by noviceuser (2023) - replies from hippo, ikegami, marioroy using Parallel::ForkManager

• Parallel::ForkManager and utf8::all crash perl on Windows by szabgab (2012)
• Crash with ForkManager on Windows by amitsq (2017) - replies from kcott and marioroy
• Fork vs pThreads by ThelmaJay (2013) - question about Parallel::ForkManager, reply from BrowserUk

Promises and Futures

• Future by Paul Evans
• Promises by Yanick Champoux (compares to Scala Promises and Futures)
• C++ std::promise
• C++ std::future - future and promise enable transfer of a value between two tasks without explicit use of a lock (the "system" implements the transfer efficiently); when a task wants to pass a value to another, it puts the value into a promise; the system makes that value appear in the corresponding future; promise provides simple put operations to match future's get

Signals

• Re^4: Nobody Expects the Agile Imposition (Part VI): Architecture (Signals are a crock :) by me (2011) - discusses early history of Windows NT, especially signals, threads and its POSIX subsystem
• $SIG{ALRM} and windows vista? by 7stud (2010) - lots of replies from ikegami and BrowserUk
• Re: Handling killing the perl process by afoken (2010)
• Signals vs. Windows by DanEllison (2012)
• Re: Signals vs. Windows by BrowserUk (2012)
• controlling threads with Tk: while loop vs. signals by zentara (2012) -- see response by BrowserUk
• Threads and signals in Windows by bojinlund (2013)
• Re^2: Kill a child nicely by Melly (2023)

C++ Multi-Threading

• C++ Concurrency support library
• C++ std::thread
• boost::thread
• Threading Building Blocks (Intel Thread Building Blocks)
• OpenMP (OpenMP Multi-Processing)

• Rosetta Code: Long List is Long (lots of multi-processing examples in this very long thread)
• Re: Rosetta Test: Long List is Long - Abseil (ditto)

• NVIDIA HPC SDK Version 24.3 Documentation (nvc++ is a C++17 compiler for NVIDIA GPUs and AMD, Intel, OpenPOWER, and Arm CPUs. It invokes the C++ compiler, assembler, and linker for the target processors with options derived from its command line arguments. nvc++ supports ISO C++17, supports GPU and multicore CPU programming with C++17 parallel algorithms, OpenACC, and OpenMP)
• The three compilers used by marioroy in Long List is Long are g++, clang++, nvc++. I also used Google abseil library.

• Should we migrate gcc from winpthreads to mcfgthread? (mingw)
• boost::thread vs std::thread vs pthread (SO)

Unit Testing Concurrent Code

Test-driven development, a practice enabling developers to detect bugs early by incorporating unit testing into the development process, has become wide-spread, but it has only been effective for programs with a single thread of control. The order of operations in different threads is essentially non-deterministic, making it more complicated to reason about program properties in concurrent programs than in single-threaded programs.

-- from a recent PhD proposal to develop a concurrent testing framework

See the "Testing Concurrent Software References" section below for more references in this active area of research. Though I haven't used any of these tools yet, I'd be interested to hear from folks who have or who have general advice and tips on how to troubleshoot and fix complex concurrency-related bugs. In particular, I'm not aware of any Perl-based concurrent testing frameworks.

In practice, the most effective, if crude, method I've found for dealing with nasty concurrency bugs is good tracing code at just the right places combined with understanding and reasoning about the code, performing experiments, and "thinking like a detective".

One especially useful experiment (mentioned in Clean Code) is to add "jiggle points" at critical places in your concurrent code and have the jiggle point either do nothing, yield, or sleep for a short interval. There are more sophisticated tools available, for example IBM's ConTest, that use this approach to flush out bugs in concurrent code.

Testing Concurrent Software References

• Testing Concurrent Programs
• Chess: Tool for finding bugs in concurrent programs
• stack overflow question on testing concurrent software
• Intel Parallel Inspector
• Java Concurrency in Practice
• Clang ThreadSanitizer
• Parasoft Jtest

Wikipedia References

• Comparison of programming paradigms
• Thread (computing)
• Multithreading (computer architecture)
• Computer multitasking
• Multi-core processor
• Simultaneous multithreading
• Thread pool
• Thread safety
• Inter-process communication (IPC) : File, Signal, Socket, Message queue, Pipe, Shared memory, Message passing, Memory-mapped file
• Signal (IPC)
• C signal handling

• Win32 Thread Information Block
• Mingw-w64

• Event-driven architecture
• Event-driven programming
• Cooperative multitasking
• Asynchronous IO
• Node.js (Javascript)
• Perl Object Environment
• Twisted (Python)
• EventMachine (Ruby)
• Reactor pattern
• Chunking (computing)

• Real-time computing
• Worst-case execution time

• Grid computing
• Parallel computing
• Concurrency (computer science)
• Concurrent computing
• Concurrency pattern
• Massively parallel
• Embarrassingly parallel
• Supercomputer
• Neural network

• Transport Layer Security (TLS)
• Comparison of TLS implementations
• Certificate authority

• Cloud computing
• Distributed computing
• Volunteer computing
• SETI@home
• Great Internet Mersenne Prime Search
• List of distributed computing projects

• GPGPU (see here for Array Processing/HPC/GPGPU References)
• OpenCL
• CUDA

• Hardware acceleration
• Field-programmable gate array (FPGA)
• Verilog

• Race condition (wikipedia)
• Heisenbug (wikipedia)
• lsof (wikipedia)

See Also

• "HPC (High Performance Computing) in Perl and C++" section on my home node
• Re: "Magic tools" that take the fun away (Releng/DevOps/Cloud/Server References) - mentions GNU parallel, a Unix command line tool, written in Perl, that allows the user to execute shell scripts or commands in parallel ... similar to pexec and xargs.
• Re^2: Organizational Culture (Part II): Meta Process (BioPerl/PDL/AI/Embedded/Data Science References)
• Re: Perl Memory problem ... (Memory Tools References)
• Re: Check for another program availability (Running External Processes on Unix and Windows References)
• Re: Security techniques every programmer should know (Security References)
• Re: declaring lexical variables in shortest scope: performance? (on Code Optimization and Performance References)

Updated: Extra references added long after the original reply was made.