Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restricting core usage #179

Open
PaMartini opened this issue Nov 14, 2024 · 3 comments
Open

Restricting core usage #179

PaMartini opened this issue Nov 14, 2024 · 3 comments

Comments

@PaMartini
Copy link

Is there a way to specify the cores that are used during training?
I am using the Python interface to Somoclu.
I tried:

import psutil
import os

cores = [0,1,3]

p = psutil.Process(os.getpid())
p.cpu_affinity(cores)

os.environ["OMP_PROC_BIND"] = 'TRUE'
os.environ["OMP_PLACES"] = ','.join(f'{{{core}}}' for core in cores)

... script that uses Somoclu to train a SOM ...

But this does not work for me ...
(I guess the parallelization of Somoclu ignores the set parameters.)
(Also, I know that this is possible via taskset, but I would prefer to specify the cores in the python code.)

@xgdgsc
Copy link
Collaborator

xgdgsc commented Nov 15, 2024

What if setting a global env variable instead of inside python? Or start a new process with those env variables?

@PaMartini
Copy link
Author

Isn't

os.environ["OMP_PROC_BIND"] = 'TRUE'
os.environ["OMP_PLACES"] = ','.join(f'{{{core}}}' for core in cores)

setting a global env variable?

How would you do this: "Or start a new process with those env variables?"

The reason for my request was that i would like to have the cores on which SOM training with Somoclu is run as an input parameter to my python function. If this is not possible due to Somoclus parallelization beeing out of reach for Pyton, I guess the best option would be to use taskset from the commandline.

@xgdgsc
Copy link
Collaborator

xgdgsc commented Nov 20, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants