Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(agent): proper label/mode based delegation #89

Merged
merged 3 commits into from
Apr 1, 2024
Merged

Conversation

Meshiest
Copy link
Contributor

@Meshiest Meshiest commented Apr 1, 2024

When locating agents for tx cannon compute or nodes within an env, the control plane will attempt to fulfill a request for agents of specific labels.

To solve the challenge of doing set difference in bulk, the fixedbitset crate is being used to convert labels and modes into a mask, and then that mask can be compared performantly using SIMD instructions (rather than iterating the sets many many times).

The delegation algorithm is as follows:

  1. get a set of all the labels that nodes from the environment may use
  2. convert the agents to a list of ids, bitmasks (of the labels + available node modes), and a weak reference to an arc that's held only when an agent is in an env
  3. partition nodes based on whether they want a specific agent or set of labels
  4. resolve agents for nodes that have a specific id (easy, parallel)
  5. resolve agents for nodes that have a set of labels by comparing the fixedbitsets (parallel)
  6. return a list of all pairings or list of all nodes that failed to find an agent (and why)

Example

topology is requesting 3 nodes with different labels:

  • node 1: validator with label alpha
  • node 2: validator with label bravo
  • node 3: client with label charlie

the available agents are:

  • agent 1: alpha, bravo, with modes validator, client
  • agent 2: alpha, bravo, with modes validator
  • agent 3: charlie, with modes client, prover

A bitmask will be created with each bit being in a different pair of brackets:
[validator][prover][client][compute][alpha][bravo][charlie]

The nodes and agents will be converted into the following masks:

  • node 1: 1000100
  • node 2: 1000010
  • node 3: 0010001
  • agent 1: 1010110
  • agent 2: 1000110
  • agent 3: 0110001

The node's masks can then be compared in parallel to check if they are a subset of the agent's masks, and if a match is detected, the agent can be atomically selected to be used in the environment.

@Meshiest Meshiest requested review from voximity and gluax April 1, 2024 03:32
@gluax gluax merged commit fe8f8bb into main Apr 1, 2024
2 checks passed
@Meshiest Meshiest deleted the feat-agent-delegation branch April 21, 2024 01:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants