Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

main domain blocking with Unix.sleep only called inside tasks #114

Open
hyphenrf opened this issue Jun 3, 2023 · 1 comment
Open

main domain blocking with Unix.sleep only called inside tasks #114

hyphenrf opened this issue Jun 3, 2023 · 1 comment

Comments

@hyphenrf
Copy link

hyphenrf commented Jun 3, 2023

While attempting to port this Rust snippet, which executes in ~3 seconds + some overhead irrespective of thread count

let mut tasklist = vec![];
for _ in 1..tasks {
    let t = thread::spawn(|| thread::sleep(Duration::from_secs(3)));
    tasklist.push(t)
}
for t in tasklist { t.join().unwrap() }

to domainslib as so

let tasklist =
  Array.init tasks (fun _ -> async pool (fun _ -> Unix.sleep 10))
in
run pool (fun _ -> Array.iter (await pool) tasklist)

where tasks is user-supplied, I found that for n > num_domains + 1, the wait times start to change significantly.
e.g. at num_domains = 2, and tasks = 100, the execution time is ~33x what's expected.

I've searched the docs here for "the right way to go idle" in different wordings but found nothing on this. I'm not sure what's different with Unix.sleep or domainslib's task implementation.

@hyphenrf hyphenrf changed the title main domain blocking with calls to Unix.sleep that only exist in the task pool main domain blocking with Unix.sleep only called inside tasks Jun 4, 2023
@polytypic
Copy link
Contributor

polytypic commented Jun 7, 2023

This is unfortunately the case with pretty much all blocking functions in the OCaml Stdlib (sleep, lock, select, ...). Those functions do not (at least not yet) work in a scheduler friendly manner, so to speak. They will instead block the current thread or the entire domain if you only have a single thread running in a domain.

Using a currently experimental domain-local-timeout mechanism for setting timeout callbacks in a scheduler friendly manner and a domain-local-await mechanism for blocking in a scheduler friendly manner, it is possible to implement a sleepf function (among many other things) that works in a scheduler friendly manner:

let sleepf seconds =
  let t = Domain_local_await.prepare_for_await () in
  let _ = Domain_local_timeout.set_timeoutf seconds t.release in
  t.await ()

As explained in the documentation of domain-local-timeout, you need to explicitly tell it that it can use the Thread and Unix libraries (to avoid depending on them directly):

let () = Domain_local_timeout.set_system (module Thread) (module Unix)

But with the above setup done, one can then implement a program that behaves closer to what is desired:

let () =
  let tasks = try int_of_string Sys.argv.(1) with _ -> 1000 in
  let seconds = try float_of_string Sys.argv.(2) with _ -> 1.0 in
  let tasklist =
    Array.init tasks (fun _ -> async pool (fun _ -> sleepf seconds))
  in
  run pool (fun _ -> Array.iter (await pool) tasklist)

Running that program (all of the above snippets as a program) takes roughly the specified number of seconds.

Note that, at the time of writing this, the default implementation of the experimental set_timeoutf is not optimized as it uses a simple list resulting in quadratic behaviour and for large number of timeouts (or tasks in this case) that will cause problems.

Note that memory usage grows rapidly with very large numbers of tasks and that can cause issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants