multiprocessing --- Process-based parallelism — Dokumentasi Python 3.15.2
Navigasi index modul | berikut | sebelum | Python » 3.15.2 documentation » The Python standard library » Eksekusi Serentak » multiprocessing --- Process-based parallelism | Tema | multiprocessing --- Process-based parallelism ¶ Source code: Lib/multiprocessing/ Availability : not Android, not iOS, not WASI. This module is not supported on mobile platforms or WebAssembly platforms . Pengantar ¶ multiprocessing is a package that supports spawning processes using an API similar to the threading module.
The multiprocessing package offers both local and remote concurrency, effectively side-stepping the Global Interpreter Lock by using subprocesses instead of threads. Due to this, the multiprocessing module allows the programmer to fully leverage multiple processors on a given machine. It runs on both POSIX and Windows. The multiprocessing module also introduces the Pool object which offers a convenient means of parallelizing the execution of a function across multiple input values, distributing the input data across processes (data parallelism).
The following example demonstrates the common practice of defining such functions in a module so that child processes can successfully import that module. This basic example of data parallelism using Pool , from multiprocessing import Pool def f ( x ): return x * x if __name__ == '__main__' : with Pool ( 5 ) as p : print ( p . map ( f , [ 1 , 2 , 3 ])) will print to standard output [ 1 , 4 , 9 ] The multiprocessing module also introduces APIs which do not have analogs in the threading module, like the ability to terminate , interrupt or kill a running process.
Lihat juga concurrent.futures.ProcessPoolExecutor offers a higher level interface to push tasks to a background process without blocking execution of the calling process. Compared to using the Pool interface directly, the concurrent.futures API more readily allows the submission of work to the underlying process pool to be separated from waiting for the results. The Process class ¶ In multiprocessing , processes are spawned by creating a Process object and then calling its start() method.
Process follows the API of threading.Thread . A trivial example of a multiprocess program is from multiprocessing import Process def f ( name ): print ( 'hello' , name ) if __name__ == '__main__' : p = Process ( target = f , args = ( 'bob' ,)) p . start () p . join () To show the individual process IDs involved, here is an expanded example: from multiprocessing import Process import os def info ( title ): print ( title ) print ( 'module name:' , __name__ ) print ( 'parent process:' , os . getppid ()) print ( 'process id:' , os . getpid ()) def f ( name ): info ( 'function f' ) print ( 'hello' , name ) if __name__ == '__main__' : info ( 'main line' ) p = Process ( target = f , args = ( 'bob' ,)) p . start () p . join () For an explanation of why the if __name__ == '__main__' part is necessary, see Programming guidelines .
The arguments to Process usually need to be picklable so they can be passed to the child process. If you tried typing the above example directly into a REPL it could lead to an AttributeError in the child process trying to locate the f function in the __main__ module. Contexts and start methods ¶ Depending on the platform, multiprocessing supports three ways to start a process. These start methods are spawn The parent process starts a fresh Python interpreter process.
The child process will only inherit those resources necessary to run the process object's run() method. In particular, unnecessary file descriptors and handles from the parent process will not be inherited. Starting a process using this method is rather slow compared to using fork or forkserver . Available on POSIX and Windows platforms. The default on Windows and macOS. fork The parent process uses os.fork() to fork the Python interpreter.
The child process, when it begins, is effectively identical to the parent process. All resources of the parent are inherited by the child process. Note that safely forking a multithreaded process is problematic. Available on POSIX systems. Berubah pada versi 3.14: This is no longer the default start method on any platform. Code that requires fork must explicitly specify that via get_context() or set_start_method() . Berubah pada versi 3.12: If Python is able to detect that your process has multiple threads, the os.fork() function that this start method calls internally will raise a DeprecationWarning .
Use a different start method. See the os.fork() documentation for further explanation. forkserver When the program starts and selects the forkserver start method, a server process is spawned. From then on, whenever a new process is needed, the parent process connects to the server and requests that it fork a new process. The fork server process is single threaded unless system libraries or preloaded imports spawn threads as a side-effect so it is generally safe for it to use os.fork() .
No unnecessary resources are inherited. Available on POSIX platforms which support passing file descriptors over Unix pipes such as Linux. The default on those, unless there is no writable temporary directory (see tempfile.gettempdir() ), which the fork server requires; spawn is the default in that case. Berubah pada versi 3.14: This became the default start method on POSIX platforms. Berubah pada versi 3.15.1: spawn is now the default start method when there is no writable temporary directory, for example on a read-only file system.
Previously forkserver remained the default there and starting a process failed with an OSError . The default is determined when it is first needed rather than when multiprocessing is imported. See gh-155717 . Berubah pada versi 3.4: spawn added on all POSIX platforms, and forkserver added for some POSIX platforms. Child processes no longer inherit all of the parents inheritable handles on Windows. Berubah pada versi 3.8: On macOS, the spawn start method is now the default.
The fork start method should be considered unsafe as it can lead to crashes of the subprocess as macOS system libraries may start threads. See bpo-33725 . Berubah pada versi 3.14: On POSIX platforms the default start method was changed from fork to forkserver to retain the performance but avoid common multithreaded process incompatibilities. See gh-84559 . On POSIX using the spawn or forkserver start methods will also start a resource tracker process which tracks the unlinked named system resources (such as named semaphores or SharedMemory objects) created by processes of the program.
When all processes have exited the resource tracker unlinks any remaining tracked object. Usually there should be none, but if a process was killed by a signal there may be some "leaked" resources. (Neither leaked semaphores nor shared memory segments will be automatically unlinked until the next reboot. This is problematic for both objects because the system allows only a limited number of named semaphores, and shared memory segments occupy some space in the main memory.) To select a start method you use the set_start_method() in the if __name__ == '__main__' clause of the main module.
For example: import multiprocessing as mp def foo ( q ): q . put ( 'hello' ) if __name__ == '__main__' : mp . set_start_method ( 'spawn' ) q = mp . Queue () p = mp . Process ( target = foo , args = ( q ,)) p . start () print ( q . get ()) p . join () set_start_method() should not be used more than once in the program. Alternatively, you can use get_context() to obtain a context object. Context objects have the same API as the multiprocessing module, and allow one to use multiple start methods in the same program. import multiprocessing as mp def foo ( q ): q . put ( 'hello' ) if __name__ == '__main__' : ctx = mp . get_context ( 'spawn' ) q = ctx .
Queue () p = ctx . Process ( target = foo , args = ( q ,)) p . start () print ( q . get ()) p . join () Note that objects related to one context may not be compa
The rest of this page is on the live site.