Skip to content

Commit

Permalink
documentation for blosc_set_threads_callback (#86)
Browse files Browse the repository at this point in the history
* documentation for blosc_set_threads_callback

* Update README_THREADED.rst
  • Loading branch information
stevengj authored and FrancescAlted committed Sep 13, 2019
1 parent 0487b95 commit e611c6c
Showing 1 changed file with 19 additions and 0 deletions.
19 changes: 19 additions & 0 deletions README_THREADED.rst
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,22 @@ whether you should set the number of threads to 1 (serial) or more
your are done!

Francesc Alted

Pluggable Threading Backend
---------------------------

Instead of having Blosc use its *own* thread pool, you can instead call `blosc_set_threads_callback(threads_callback, callback_data)` to install your own threading backend. This gives Blosc the possibility to use the same threading mechanism as one you are using in the rest of your program (e.g. OpenMP or Intel TBB), sharing the same threads, rather than starting its own threads that compete with yours for the CPU cores.

Here, `threads_callback` is a function of the form:

.. code-block:: c
void threads_callback(void *callback_data, void (*dojob)(void *), int numjobs, size_t jobdata_elsize, void *jobdata)
{
int i;
for (i = 0; i < numjobs; ++i)
dojob(((char *) jobdata) + ((unsigned) i)*jobdata_elsize);
}
that simply calls `dojob` on the given `jobdata` array for `numjobs` elements of size `jobdata_elsize`, returning when all of the `dojob` calls have completed. The key point is that your `threads_callback` routine can execute the `dojob` calls *in parallel* if it wants. For example, if you are using OpenMP your `threads_callback` function might use `#pragma omp parallel for`.

The `blosc_set_threads_callback` function should be called before any Blosc function (before any Blosc contexts are created), to inhibit Blosc from spawning its own worker threads. In this case, `blosc_set_nthreads` and similar functions set an upper bound to the `numjobs` that is passed to your `threads_callback` rather than an actual number of threads.

0 comments on commit e611c6c

Please sign in to comment.