Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cpsy/python multiprocessing #12

Open
dunland opened this issue Feb 8, 2022 · 0 comments
Open

cpsy/python multiprocessing #12

dunland opened this issue Feb 8, 2022 · 0 comments
Labels
Information just some collected info stored here not to forget.

Comments

@dunland
Copy link
Member

dunland commented Feb 8, 2022

See stackoverflow on [How do CPU cores get allocated to python processes in multiprocessing?](https://stackoverflow.com/a/49993798):

Python doesn't do anything to bind processes or threads to cores; it just leaves things up to the OS. When you spawn a bunch of independent processes (or threads, but that's harder to do in Python), the OS's scheduler will quickly and efficiently get them spread out across your cores without you, or Python, needing to do anything (barring really bad pathological cases).

I'm also assuming you aren't using explicit locks, other synchronization, Manager objects, or anything else that will serialize your code. If you do, obviously you can't get full parallelism.

And I'm also assuming you aren't using (mutable) shared memory, like a multiprocessing.Array that everyone writes to. This can cause cache and page conflicts that can be almost as bad as explicit locks.

@dunland dunland added the Information just some collected info stored here not to forget. label Feb 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Information just some collected info stored here not to forget.
Projects
None yet
Development

No branches or pull requests

1 participant