CP013: Who should have control over bulk execution affinity?

[AerialMantis]

Whilst adding the bulk_execution_affinity properties to the paper there was some discussion about who should be responsible for specifying the properties, the execution context, the executor or both.

(taken from #48):

Ruyk:

Let me see if I understand this properly. Assuming this is a simple fixed size thread pool of size 4 underneath:

The thread pool itself is created and "maintained" by the execution context. That means the threads are created on construction of the execution context. At this points, threads are bound to wherever execution(s) resource(s) have been indicated on construction of the executionContext, if any.

Then we perform a require for a bulk executor following a scatter policy. What do you expect to happen?
A. The threads of the thread pool are re-bound following the scatter placement of threads per resource
B. The new affExec will enforce placement of execution agents on the thread pool threads following the scatter policy.
C. Neither of the above
My understanding is B from the proposal.
However, if what I want to do is place the actual execution threads following scatter policy on the given cores, I would need to pass the policy on construction of the execution context, rather than the executor (since the thread pool has been created already). I could potentially re-bind threads after they have been created, but that has a cost that could be avoided if the initial placement of resources of the thread pool is done on construction. Can we have an alternative constructor where these policies are passed to the execution context? If we are having a fine-grain selection of member_of resources, this will allow for the high-level interface to be used in that case.

Now we call bulk_execute with the callable. How is each execution agent placed onto an execution resource? If the executor property is thread_execution_mapping_t or other_execution_mapping_t , that means the executor will query the placement (presumably an id) and place agents on existing threads. Which one is a valid placement?
A. Agent 0 in Thread 0, Agent 1 in Thread 2, Agent 2 in Thread 1, Agent 3 in Thread 3, then loop over again for the remaining 4 agents
B. Agent 0 in Thread 0, Agent 1 in Thread 3, Agent 2 in Thread 1, Agent 3 in Thread 3, then loop over again for the remaining 4 agents
C. Either of the above
My understanding is A will be correct, or at least commonly expected.
However, when oversubscribing agents on threads is not clear what to expect when using these policies. Will the executor hold execution of an agent until the resource "far away" is available or will execute consecutively if possible?
Also note that the same execution context (in this case, the same thread pool) may be in use by multiple executors. What is the expectation in terms of placement of agents on threads?

AerialMantis:

The way I see this working is that if you were to construct an execution context, say the static_thread_pool and initialise it with a set number of threads of execution, such as 4. By default, the bulk_execution_affinity property would be none so the thread pool wouldn't be required to make any kind of guarantee as to how it's threads of execution are bound to the underlying resources. It could automatically bind each of its threads of execution to a certain resource if it makes sense for it to do so, but it wouldn't have to. It would only be when a different bulk_execution_affinity property was requested by an executor that it would be required to perform binding in a particular pattern. So when this happens if the thread pool had already performed binding then it may have to rebind to achieve the binding pattern requested by the executor.

The reason for having these properties on the executor rather than the execution context is to make it more usable as a high-level feature. There are some executor implementations which will not make use of an execution context, such as inline executors which are just created as concrete executor types and used without referring to an explicit execution context, an execution context still exists though it's implicit.

I think we should have a more fine-grained interface for configuring execution contexts where you can specify specific affinity binding on construction. In this case, the executor which the execution context provides would only support the bulk_execution_affinity property which the execution context was constructed with, so could not be requested to use another, essentially becoming query only. So essentially this would mean that you could specify affinity binding at the execution context level or the executor level, but specifying it at the execution context level would take precidence over the executor level as the executor would inherit the property.

In terms of how the affinity binding is implemented within bulk_execute, I would also agree that A would be the most commonly expected, though I think it would be okay to allow an implementation to decide, providing it matched the requirements of the pattern and was consistent across invocations of bulk_execute. For oversubscribing agents I would expect that you would have multiple agents mapped to the same threads of execution, so only one agent could progress at a time. As to how those agents are ordered I think that would be at the discretion of the implementation, though implementations would be expected to order agents in way which is most efficient for the requesting binding pattern.

You raise an interesting point here in how an execution context will deal with multiple executors submitting work to it. In terms of affinity binding patterns, I think I would expect that if one executor was to request one bulk_execution_affinity property and another executor was to request a different property then one task would have to be scheduled before the other with the affinity binding pattern being altered between tasks. Though we would recommend users not do this frequently as it could be inneficient to keep rebinding the threads of execution. If both tasks were to be using the same affinity binding pattern, then they could conceivably be overlayed, though I think this would be at the discretion of the implementation as to whether this could be done efficiently.

Ruyk:

So essentially this would mean that you could specify affinity binding at the execution context level or the executor level, but specifying it at the execution context level would take precedence over the executor level as the executor would inherit the property.
I guess the opposite would make more sense: The one closer to the user is the one that should make the final decision.

I guess my concern is that associating the binding with the executor may incur in extra costs, since the executor dispatch may be called in a part of the code that is performance critical, whereas the executor context can be defined anywhere.

Apart from that, what you says makes sense to me. Since this is an exploratory paper, I am happy if this is merged with the minor change above and just add a straw poll for deciding where we should do the affinity (executor context, executor or both).

AerialMantis:

Yeah, that's a good point, perhaps what we want then is for the executor to be able to override the behaviour of the execution context. Maybe then the property of the execution context could be inherited by the executor as the default behaviour, but can still be altered by the executor.

I think that's a fair concern, that putting control over affinity binding at the executor level could incur costs at the wrong time, though if we also add the ability to configure the execution context as well with the same property then this should alleviate that, as the cost will be paid at configuration time.

Okay great, I will add some notes to the paper covering some of the points we discussed here and I will add a straw poll for where the affinity binding capability should go.