Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove references to ConcurrentTaskRunner from docs and ensure examples use .wait() #14144

Merged
merged 8 commits into from
Jun 20, 2024
2 changes: 1 addition & 1 deletion docs/2.19.x/concepts/task-runners.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: task Runners
title: Task runners
description: Task runners enable you to engage specific executors for Prefect tasks, such as for concurrent, parallel, or distributed execution of tasks.
---

Expand Down
9 changes: 1 addition & 8 deletions docs/3.0rc/develop/write-flows/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -409,7 +409,7 @@ Flows allow a great deal of configuration by passing arguments to the decorator.
| `retries` | An optional number of times to retry on flow run failure. |
| <span class="no-wrap">`retry_delay_seconds`</span> | An optional number of seconds to wait before retrying the flow after failure. This is only applicable if `retries` is nonzero. |
| `flow_run_name` | An optional name to distinguish runs of this flow; this name can be provided as a string template with the flow's parameters as variables; you can also provide this name as a function that returns a string. |
| `task_runner` | An optional [task runner](/3.0rc/develop/write-tasks/use-task-runners/) to use for task execution within the flow when you `.submit()` tasks. If not provided and you `.submit()` tasks, the `ConcurrentTaskRunner` is used. |
| `task_runner` | An optional [task runner](/3.0rc/develop/write-tasks/use-task-runners/) to use for task execution within the flow when you `.submit()` tasks. If not provided and you `.submit()` tasks, the `ThreadPoolTaskRunner` is used. |
| `timeout_seconds` | An optional number of seconds indicating a maximum runtime for the flow. If the flow exceeds this runtime, it is marked as failed. Flow execution may continue until the next task is called. |
| `validate_parameters` | Boolean indicating whether parameters passed to flows are validated by Pydantic. Default is `True`. |
| `version` | An optional version string for the flow. If not provided, we will attempt to create a version string as a hash of the file containing the wrapped function. If the file cannot be located, the version will be null. |
Expand Down Expand Up @@ -600,7 +600,6 @@ Running the `hello_world()` flow (in this example from the file `hello.py`) crea
```bash
$ python hello.py
15:19:21.651 | INFO | prefect.engine - Created flow run 'daft-cougar' for flow 'Hello Flow'
15:19:21.651 | INFO | Flow run 'daft-cougar' - Using task runner 'ConcurrentTaskRunner'
15:19:21.945 | INFO | Flow run 'daft-cougar' - Created task run 'Print Hello-84f0fe0e-0' for task 'Print Hello'
Hello Marvin!
15:19:22.055 | INFO | Task run 'Print Hello-84f0fe0e-0' - Finished in state Completed()
Expand Down Expand Up @@ -676,7 +675,6 @@ Running this flow produces the following result:
<div class="terminal">
```bash
22:22:36.864 | INFO | prefect.engine - Created flow run 'acrid-tuatara' for flow 'always-fails-flow'
22:22:36.864 | INFO | Flow run 'acrid-tuatara' - Starting 'ConcurrentTaskRunner'; submitted tasks will be run concurrently...
22:22:37.060 | ERROR | Flow run 'acrid-tuatara' - Encountered exception during execution:
Traceback (most recent call last):...
ValueError: This flow immediately fails
Expand Down Expand Up @@ -713,7 +711,6 @@ Running this flow produces the following result:
<div class="terminal">
```bash
18:32:05.345 | INFO | prefect.engine - Created flow run 'auburn-lionfish' for flow 'always-fails-flow'
18:32:05.346 | INFO | Flow run 'auburn-lionfish' - Starting 'ConcurrentTaskRunner'; submitted tasks will be run concurrently...
18:32:05.582 | INFO | Flow run 'auburn-lionfish' - Created task run 'always_fails_task-96e4be14-0' for task 'always_fails_task'
18:32:05.582 | INFO | Flow run 'auburn-lionfish' - Submitted task run 'always_fails_task-96e4be14-0' for execution.
18:32:05.610 | ERROR | Task run 'always_fails_task-96e4be14-0' - Encountered exception during execution:
Expand Down Expand Up @@ -763,7 +760,6 @@ Running this flow produces the following result&mdash;it succeeds because it ret
<div class="terminal">
```bash
18:35:24.965 | INFO | prefect.engine - Created flow run 'whispering-guan' for flow 'always-succeeds-flow'
18:35:24.965 | INFO | Flow run 'whispering-guan' - Starting 'ConcurrentTaskRunner'; submitted tasks will be run concurrently...
18:35:25.204 | INFO | Flow run 'whispering-guan' - Created task run 'always_fails_task-96e4be14-0' for task 'always_fails_task'
18:35:25.205 | INFO | Flow run 'whispering-guan' - Submitted task run 'always_fails_task-96e4be14-0' for execution.
18:35:25.232 | ERROR | Task run 'always_fails_task-96e4be14-0' - Encountered exception during execution:
Expand Down Expand Up @@ -814,7 +810,6 @@ Note that the final state is `Failed`, but the states of each of the returned fu
<div class="terminal">
```bash
20:57:51.547 | INFO | prefect.engine - Created flow run 'impartial-gorilla' for flow 'always-fails-flow'
20:57:51.548 | INFO | Flow run 'impartial-gorilla' - Using task runner 'ConcurrentTaskRunner'
20:57:51.645 | INFO | Flow run 'impartial-gorilla' - Created task run 'always_fails_task-58ea43a6-0' for task 'always_fails_task'
20:57:51.686 | INFO | Flow run 'impartial-gorilla' - Created task run 'always_succeeds_task-c9014725-0' for task 'always_succeeds_task'
20:57:51.727 | ERROR | Task run 'always_fails_task-58ea43a6-0' - Encountered exception during execution:
Expand Down Expand Up @@ -870,7 +865,6 @@ Running this flow produces the following result.
<div class="terminal">
```bash
18:37:42.844 | INFO | prefect.engine - Created flow run 'lavender-elk' for flow 'always-succeeds-flow'
18:37:42.845 | INFO | Flow run 'lavender-elk' - Starting 'ConcurrentTaskRunner'; submitted tasks will be run concurrently...
18:37:43.125 | INFO | Flow run 'lavender-elk' - Created task run 'always_fails_task-96e4be14-0' for task 'always_fails_task'
18:37:43.126 | INFO | Flow run 'lavender-elk' - Submitted task run 'always_fails_task-96e4be14-0' for execution.
18:37:43.162 | INFO | Flow run 'lavender-elk' - Created task run 'always_succeeds_task-9c27db32-0' for task 'always_succeeds_task'
Expand Down Expand Up @@ -911,7 +905,6 @@ Running this flow produces the following result.
<div class="terminal">
```bash
21:02:45.715 | INFO | prefect.engine - Created flow run 'sparkling-pony' for flow 'always-succeeds-flow'
21:02:45.715 | INFO | Flow run 'sparkling-pony' - Using task runner 'ConcurrentTaskRunner'
21:02:45.816 | INFO | Flow run 'sparkling-pony' - Created task run 'always_fails_task-58ea43a6-0' for task 'always_fails_task'
21:02:45.853 | ERROR | Task run 'always_fails_task-58ea43a6-0' - Encountered exception during execution:
Traceback (most recent call last):...
Expand Down
2 changes: 1 addition & 1 deletion docs/3.0rc/develop/write-tasks/dask-and-ray.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -439,7 +439,7 @@ Many workflows include a variety of tasks, and not all of them benefit from para
Because task runners are specified on flows, you can assign different task runners to tasks by using
[subflows](/3.0rc/develop/write-flows/#composing-flows) to organize those tasks.

This example uses the default `ConcurrentTaskRunner`. Then you call a `ray_greetings()` subflow that
This example uses the default `ThreadPoolTaskRunner`. Then you call a `ray_greetings()` subflow that
uses the `RayTaskRunner` to execute the same tasks in a Ray instance.

```python
Expand Down
8 changes: 3 additions & 5 deletions docs/3.0rc/develop/write-tasks/use-task-runners.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,8 @@ which runs tasks concurrently within a thread pool.
The following Prefect-developed task runners for parallel or distributed task
execution are available as [integrations](/integrations/catalog/):

- [`DaskTaskRunner`](https://prefecthq.github.io/prefect-dask/) can run tasks using
[`dask.distributed`](http://distributed.dask.org/).
- [`RayTaskRunner`](https://prefecthq.github.io/prefect-ray/) can run tasks using
[Ray](https://www.ray.io/).
- [`DaskTaskRunner`](https://github.com/PrefectHQ/prefect/tree/main/src/integrations/prefect-dask) can run tasks using [`dask.distributed`](http://distributed.dask.org/).
- [`RayTaskRunner`](https://github.com/PrefectHQ/prefect/tree/main/src/integrations/prefect-ray) can run tasks using [Ray](https://www.ray.io/).

<Note>
**Concurrency vs. parallelism**
Expand Down Expand Up @@ -208,7 +206,7 @@ def my_task():
def my_flow():
future = my_task.submit()
result = future.result(raise_on_failure=False)
if future.get_state().is_failed():
if future.state.is_failed():
# `result` is an exception! handle accordingly
...
else:
Expand Down
2 changes: 1 addition & 1 deletion src/prefect/flows.py
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ class Flow(Generic[P, R]):
be provided as a string template with the flow's parameters as variables,
or a function that returns a string.
task_runner: An optional task runner to use for task execution within the flow;
if not provided, a `ConcurrentTaskRunner` will be used.
if not provided, a `ThreadPoolTaskRunner` will be used.
description: An optional string description for the flow; if not provided, the
description will be pulled from the docstring for the decorated function.
timeout_seconds: An optional number of seconds indicating a maximum runtime for
Expand Down
Loading