Execute multiple compute graphs in parallel #9900
Unanswered
paomiannanjue
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Sometimes, there are some nodes that can be executed in parallel in llms. For example, GLU processes input data in two ways, which can be ran on two threads. But in llamacpp, nodes can only be executed in sequence.Maybe we can support node parallelism further improving computing efficiency.
Beta Was this translation helpful? Give feedback.
All reactions