Replies: 5 comments 7 replies
-
Thanks for opening your first issue here! Be sure to follow the issue template! |
Beta Was this translation helpful? Give feedback.
-
Could you please be more precise and explain your use case in more detail? What problem are you really trying to solve? Ideally explaining it on examples that you experienced. |
Beta Was this translation helpful? Give feedback.
-
When using an external task sensor, running DAG manually does not work normally. for example ` |
Beta Was this translation helpful? Give feedback.
-
How about params - I still do not understand what you are asking for - but that would be my guess tha it's likely what you want. In either case I am converting that to a discussion until this is clarified. |
Beta Was this translation helpful? Give feedback.
-
Hi @potiuk I can't be sure that this is exactly what @Lee2532 is talking about, but I think I'm correct (and the fix proposed in the PR would solve the problem I currently have - though I don't like how it fixes it). We use However, if you do a manual run of the DAG, the The alternatives all seem not so great from my point of view:
def wait_if_scheduled(dag_run, ti: TaskInstance):
"""
Determines whether to run the wait or the no wait task based on if the run is scheduled or not
"""
group_name = ti.task_id.split(".", 2)[0]
if dag_run.external_trigger is True:
return f"{group_name}.waited"
else:
return f"{group_name}.wait"
with TaskGroup("external_dag_waiter") as elt_wait:
branch = BranchPythonOperator(
task_id="maybe_skip", python_callable=wait_if_scheduled
)
waiter = ExternalTaskSensor(
task_id="wait",
external_dag_id="external_dag",
poke_interval=300,
mode="reschedule",
)
waited = EmptyOperator(
task_id="waited", trigger_rule="none_failed_min_one_success"
)
branch >> waiter >> waited
branch >> waited But that's quite a bit of noise in a DAG. |
Beta Was this translation helpful? Give feedback.
-
Description
when the External Task Sensor is manually executed, not work
Use case/motivation
We can add options to perform functions such as scheduling when executing manually.
Since we FAIL the DAG with External Task Sensor when executing manually, we add logic to pass when executing manually
Related issues
No response
Are you willing to submit a PR?
Code of Conduct
Beta Was this translation helpful? Give feedback.
All reactions