Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-125714: Doc: Explain that multiprocessing only supports unpicklable functions through the fork method #125750

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion Doc/library/concurrent.futures.rst
Original file line number Diff line number Diff line change
Expand Up @@ -412,7 +412,8 @@ to a :class:`ProcessPoolExecutor` will result in deadlock.
.. versionchanged:: 3.14
The default process start method (see
:ref:`multiprocessing-start-methods`) changed away from *fork*. If you
require the *fork* start method for :class:`ProcessPoolExecutor` you must
require the *fork* start method for :class:`ProcessPoolExecutor` or need to run
:ref:`unpicklable functions <pickle-picklable>`, you must
explicitly pass ``mp_context=multiprocessing.get_context("fork")``.

.. _processpoolexecutor-example:
Expand Down
7 changes: 7 additions & 0 deletions Doc/library/multiprocessing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,8 @@ to start a process. These *start methods* are
will not be inherited. Starting a process using this method is
rather slow compared to using *fork* or *forkserver*.

Functions run with this method must be :ref:`picklable <pickle-picklable>`.

Available on POSIX and Windows platforms. The default on Windows and macOS.

*fork*
Expand All @@ -124,6 +126,9 @@ to start a process. These *start methods* are
inherited by the child process. Note that safely forking a
multithreaded process is problematic.

Functions run with this method do *not* need to be
:ref:`picklable <pickle-picklable>`.

Available on POSIX systems.

.. versionchanged:: 3.14
Expand All @@ -146,6 +151,8 @@ to start a process. These *start methods* are
side-effect so it is generally safe for it to use :func:`os.fork`.
No unnecessary resources are inherited.

Functions run with this method must be :ref:`picklable <pickle-picklable>`.

Available on POSIX platforms which support passing file descriptors over
Unix pipes such as Linux. The default on those.

Expand Down
21 changes: 13 additions & 8 deletions Doc/whatsnew/3.14.rst
Original file line number Diff line number Diff line change
Expand Up @@ -451,14 +451,6 @@ Deprecated
as a single positional argument.
(Contributed by Serhiy Storchaka in :gh:`109218`.)

* :mod:`multiprocessing` and :mod:`concurrent.futures`:
The default start method (see :ref:`multiprocessing-start-methods`) changed
away from *fork* to *forkserver* on platforms where it was not already
*spawn* (Windows & macOS). If you require the threading incompatible *fork*
start method you must explicitly specify it when using :mod:`multiprocessing`
or :mod:`concurrent.futures` APIs.
(Contributed by Gregory P. Smith in :gh:`84559`.)

* :mod:`os`:
:term:`Soft deprecate <soft deprecated>` :func:`os.popen` and
:func:`os.spawn* <os.spawnl>` functions. They should no longer be used to
Expand Down Expand Up @@ -625,6 +617,8 @@ Others
:meth:`~object.__index__`. (Contributed by Mark Dickinson in :gh:`119743`.)


.. _whatsnew314-porting-to-python314:

Porting to Python 3.14
======================

Expand All @@ -643,6 +637,17 @@ Changes in the Python API
This temporary change affects other threads.
(Contributed by Serhiy Storchaka in :gh:`69998`.)

* :mod:`multiprocessing` and :mod:`concurrent.futures`:
The default start method (see :ref:`multiprocessing-start-methods`) changed
away from *fork* to *forkserver* on platforms where it was not already
*spawn* (Windows & macOS). If you require the threading incompatible *fork*,
progval marked this conversation as resolved.
Show resolved Hide resolved
which can run :ref:`unpicklable functions <pickle-picklable>`,
start method you must explicitly specify it when using :mod:`multiprocessing`
or :mod:`concurrent.futures` APIs.
Code running pickable functions on these platforms without explicitly opting
progval marked this conversation as resolved.
Show resolved Hide resolved
into the *fork* method will now raise :exc:`pickle.PicklingError`.
(Contributed by Gregory P. Smith in :gh:`84559`.)


Build changes
=============
Expand Down
Loading