How to support parallel mode in process orchestration?

[zcc0511]

How to support parallel mode in process orchestration?

[Allamss]

I am keen to ascertain the approximate timeline for the implementation of parallel branch functionality within our workflow system; In the context of multi-model systems and AI agents, there is a pressing need for concurrent processing of inputs and outputs across multiple models. The sequential execution of tasks proves excessively time-consuming for generative AI. Consequently, the implementation of parallel branching within workflow systems would be immensely beneficial, significantly enhancing overall efficiency and performance

[crazywoola]

Actually we are working on this at this moment.

[Allamss]

I greatly appreciate your prompt response and am thrilled to hear that parallel branch functionality is currently in development. This is excellent news!

Given the critical importance of this feature for enhancing the efficiency of multi-model systems and AI agents, I’m eagerly looking forward to its implementation. The ability to process inputs and outputs concurrently across multiple models will undoubtedly be a game-changer for many users, including myself.

While I understand that quality development takes time. If possible, could you share any rough estimates on when we might expect to see this feature rolled out, even if it’s just a beta version?

Thank you once again for your hard work on this crucial functionality. Your efforts to continually improve the workflow system are truly appreciated by the community. We’re excited to see how this will transform our projects and processes.

[Allamss]

Perhaps the assistance of the open-source community would be advantageous in this endeavor? I would be more than willing to dedicate my leisure hours to contribute towards the realization of this functionality. @takatost

[akramer-zibra]

+1 for this request. The ability to run tasks in parallel would be a huge performance boost for certain use cases. I can well imagine that this could work in a similar way to the "Iteration" block, except that the processes within such bounding block run in parallel. Maybe this is a good start for such an enhancement @Allamss?

How is the process chain built in dify? does it work via events?

[Allamss]

The ‘Iteration block’ proves inadequate for our specific use case. Our construct bears a closer resemblance to an ‘IF/ELSE block’, albeit executed in parallel. Our scenario predominantly involves feeding identical input to diverse Large Language Models (LLMs), followed by an LLM-driven selection of the optimal output from the array of results generated. This approach aligns more closely with ‘AI agent’ methodologies. Crucially, implementing parallel execution at the stage where identical input is distributed to different LLMs would yield a significant enhancement in processing speed.

[tobegit3hub]

Looking forward to this feature as well.

[laipz8200]

We have released a preview version of this feature in #7985 and look forward to your experience and feedback.