diff --git a/README_THREADED.rst b/README_THREADED.rst index 4d427f943..5db7307fc 100644 --- a/README_THREADED.rst +++ b/README_THREADED.rst @@ -31,3 +31,22 @@ whether you should set the number of threads to 1 (serial) or more your are done! Francesc Alted + +Pluggable Threading Backend +--------------------------- + +Instead of having Blosc use its *own* thread pool, you can instead call `blosc_set_threads_callback(threads_callback, callback_data)` to install your own threading backend. This gives Blosc the possibility to use the same threading mechanism as one you are using in the rest of your program (e.g. OpenMP or Intel TBB), sharing the same threads, rather than starting its own threads that compete with yours for the CPU cores. + +Here, `threads_callback` is a function of the form: + +.. code-block:: c + void threads_callback(void *callback_data, void (*dojob)(void *), int numjobs, size_t jobdata_elsize, void *jobdata) + { + int i; + for (i = 0; i < numjobs; ++i) + dojob(((char *) jobdata) + ((unsigned) i)*jobdata_elsize); + } + +that simply calls `dojob` on the given `jobdata` array for `numjobs` elements of size `jobdata_elsize`, returning when all of the `dojob` calls have completed. The key point is that your `threads_callback` routine can execute the `dojob` calls *in parallel* if it wants. For example, if you are using OpenMP your `threads_callback` function might use `#pragma omp parallel for`. + +The `blosc_set_threads_callback` function should be called before any Blosc function (before any Blosc contexts are created), to inhibit Blosc from spawning its own worker threads. In this case, `blosc_set_nthreads` and similar functions set an upper bound to the `numjobs` that is passed to your `threads_callback` rather than an actual number of threads.