Enhancing Workflow with Loop and Sleep Nodes for Repeated Execution and Interval Control

[Woo0ood]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

We would like to have the capability to repeatedly request an HTTP interface or make multiple calls to an LLM within a workflow (or any other process that requires repeated execution to achieve a result). The loop should continue until the returned information meets specific formats or conditions. Additionally, it should be possible to add appropriate intervals between requests.

2. Additional context or comments

To achieve this, we developed Loop and Sleep nodes. The Loop node's logic combines iteration and if-else nodes, allowing it to use internal nodes' output variables to determine whether to exit the loop. Additionally, a maximum iteration limit can be set. The Sleep node pauses execution for a specified duration until the sleep period ends. We seek the official and community stance and suggestions on this new feature. If it receives support from the official team and the community, we will organize this part of the code and submit it to the Dify community.

3. Can you help us with this feature?

• I am interested in contributing to this feature.

[crazywoola]

Link similar feature requests here.

• Add WHILE loop or TRY block to workflow #12289
• Integrate a manual 'break' into the iterate node #12217

Please feel free to open a PR for this.

[dosubot]

Hi, @Woo0ood. I'm Dosu, and I'm helping the Dify team manage their backlog. I'm marking this issue as stale.

Issue Summary:

• You suggested adding Loop and Sleep nodes to enhance workflow functionality.
• The Loop node would allow repeated execution until conditions are met, with iteration limits.
• The Sleep node would introduce delays between executions.
• @crazywoola linked similar feature requests and encouraged a pull request for this enhancement.

Next Steps:

• Please let us know if this issue is still relevant to the latest version of the Dify repository by commenting here.
• If there is no further activity, this issue will be automatically closed in 15 days.

Thank you for your understanding and contribution!

[DavideDelbianco]

I would love this kind of nodes especially when you have to interact with APIs that have paginated results.
To interact with the knowledge datasets of Dify, to search a KB by name I had to use a Code node to execute the HTTP request in a loop until no more pages are available or the KB with desired name is found.

Simple actions executed in the Code node takes a huge amount of time due to the Sandbox and native nodes would outperform that.

[dosubot]

@crazywoola, the user @DavideDelbianco has indicated that the addition of Loop and Sleep nodes is still relevant, especially for interacting with APIs that have paginated results. Could you please assist them with this issue?

[DavideDelbianco]

@Woo0ood I guess you are not going to open a PR for this feature ?

[Woo0ood]

@Woo0ood I guess you are not going to open a PR for this feature ?

@DavideDelbianco, #13058 PR is currently being worked on by the Dify team and will be available in an upcoming release.🥸

Although the PR was closed (due to an incorrect operation), we’ve communicated with the Dify team and confirmed that the PR will be merged.

[WhiteNightMo]

@Woo0ood I subscribed to this issue, please update this issue after the merge, thanks!

[Woo0ood]

@Woo0ood I subscribed to this issue, please update this issue after the merge, thanks!

[Woo0ood]

@Woo0ood I subscribed to this issue, please update this issue after the merge, thanks!

all done😁

[WhiteNightMo]

@Woo0ood I subscribed to this issue, please update this issue after the merge, thanks!

all done😁

Hi, I tried git clone main branch code but I don't see the loop node in workflow, am I missing something in the configuration?

[Woo0ood]

@Woo0ood I subscribed to this issue, please update this issue after the merge, thanks!

all done😁

Hi, I tried git clone main branch code but I don't see the loop node in workflow, am I missing something in the configuration?

Could you please let me know how you are running the project?
Using Docker or running the code locally?

[WhiteNightMo]

Could you please let me know how you are running the project?您能告诉我您是如何运行这个项目的吗？ Using Docker or running the code locally?使用 Docker 还是在本地运行代码？

Yes, I deployed via docker and it was git clone main branch, not release v1.0.0.

[Woo0ood]

Could you please let me know how you are running the project?您能告诉我您是如何运行这个项目的吗？ Using Docker or running the code locally?使用 Docker 还是在本地运行代码？

Yes, I deployed via docker and it was git clone main branch, not release v1.0.0.

If you are using the official Docker image, the Loop Node feature is not included in the current Docker image yet. You will need to wait for the next official release. Alternatively, you can build a Docker image yourself from the web directory.

[WhiteNightMo]

If you are using the official Docker image, the Loop Node feature is not included in the current Docker image yet. You will need to wait for the next official release. Alternatively, you can build a Docker image yourself from the web directory.

Wow, thanks for the reply!

[WhiteNightMo]

@Woo0ood Hi, I tested the loop node and he's pretty awesome. But when I try to use the loop node to query the result of some asynchronous task, there are two small problems:

1. there is no interval between each loop and the 100 max executions always finish very fast;
2. the loop node will not output the result externally, causing me to query the task result again after the loop node.

I don't know how you feel about my problem, I think the first one is very critical.

[Woo0ood]

@Woo0ood Hi, I tested the loop node and he's pretty awesome. But when I try to use the loop node to query the result of some asynchronous task, there are two small problems:

1. there is no interval between each loop and the 100 max executions always finish very fast;
2. the loop node will not output the result externally, causing me to query the task result again after the loop node.

I don't know how you feel about my problem, I think the first one is very critical.

Thank you for testing and sharing your feedback.

Actually, both points you mentioned were initially considered during our early design discussions but were later removed after discussions with the Dify official team:

Interval between loops:
We originally had a dedicated Sleep/Wait node to control intervals between loops (you can see it in the images from our issue), but the Dify official team suggested removing it to maintain simplicity. They recommended achieving the same functionality using a Code node or encapsulating it as a custom tool.

Loop node output:
We also considered adding direct output variables for the Loop node, but the official team advised us to hold off on this for now. As a temporary workaround, If you are using ChatFlow, I personally recommend using Conversation Variables. If you think direct outputs from the loop node would be beneficial, please consider opening a new issue so the official team can evaluate this feature request 😄.

[WhiteNightMo]

Thank you for testing and sharing your feedback.

Actually, both points you mentioned were initially considered during our early design discussions but were later removed after discussions with the Dify official team:

Interval between loops: We originally had a dedicated Sleep/Wait node to control intervals between loops (you can see it in the images from our issue), but the Dify official team suggested removing it to maintain simplicity. They recommended achieving the same functionality using a Code node or encapsulating it as a custom tool.

Loop node output: We also considered adding direct output variables for the Loop node, but the official team advised us to hold off on this for now. As a temporary workaround, If you are using ChatFlow, I personally recommend using Conversation Variables. If you think direct outputs from the loop node would be beneficial, please consider opening a new issue so the official team can evaluate this feature request 😄.

Sorry, I just realized my reply didn't go out!

I think it can be implemented by referring to the HTTP Request node, so that you don't have to add a separate Sleep/Wait node.

[avasquezkudaw]

Thank you for testing and sharing your feedback.
Actually, both points you mentioned were initially considered during our early design discussions but were later removed after discussions with the Dify official team:
Interval between loops: We originally had a dedicated Sleep/Wait node to control intervals between loops (you can see it in the images from our issue), but the Dify official team suggested removing it to maintain simplicity. They recommended achieving the same functionality using a Code node or encapsulating it as a custom tool.
Loop node output: We also considered adding direct output variables for the Loop node, but the official team advised us to hold off on this for now. As a temporary workaround, If you are using ChatFlow, I personally recommend using Conversation Variables. If you think direct outputs from the loop node would be beneficial, please consider opening a new issue so the official team can evaluate this feature request 😄.

Sorry, I just realized my reply didn't go out!

I think it can be implemented by referring to the HTTP Request node, so that you don't have to add a separate Sleep/Wait node.

I understand that the sleep functionality can be used independently of whether the request fails or not, as some functionalities require constantly querying an endpoint until a specific result is obtained.

[WhiteNightMo]

Sorry, I just realized my reply didn't go out!

I think it can be implemented by referring to the HTTP Request node, so that you don't have to add a separate Sleep/Wait node.

I understand that the sleep functionality can be used independently of whether the request fails or not, as some functionalities require constantly querying an endpoint until a specific result is obtained.

No, no, no, I mean the Loop node can refer to the Http Request node with MAX RETRIES and RETRY INTERVAL controls, right now the Loop node only has the Maximum Loop Count and lacks the LOOP INTERVAL control.