Skip to content

Commit

Permalink
Update docs links for monodocs (#4834)
Browse files Browse the repository at this point in the history
* update docs links in main README

Signed-off-by: nikki everett <[email protected]>

* update links in contribution guide

Signed-off-by: nikki everett <[email protected]>

* update links in concepts docs

Signed-off-by: nikki everett <[email protected]>

* update docs links in community roadmap

Signed-off-by: nikki everett <[email protected]>

* update link in contribution guide

Signed-off-by: nikki everett <[email protected]>

* update link in concept doc

Signed-off-by: nikki everett <[email protected]>

* update links in deployment docs

Signed-off-by: nikki everett <[email protected]>

* update link in API docs landing page

Signed-off-by: nikki everett <[email protected]>

* update CHANGELOG links

Signed-off-by: nikki everett <[email protected]>

* update flyteidl links

Signed-off-by: nikki everett <[email protected]>

* update links in rfcs

Signed-off-by: nikki everett <[email protected]>

* update remaining old docs links

Signed-off-by: nikki everett <[email protected]>

* change flyteidl files back to master state

Signed-off-by: nikki everett <[email protected]>

---------

Signed-off-by: nikki everett <[email protected]>
  • Loading branch information
neverett authored and katrogan committed Feb 20, 2024
1 parent 98dd797 commit 4d98306
Show file tree
Hide file tree
Showing 27 changed files with 176 additions and 63 deletions.
2 changes: 1 addition & 1 deletion CHANGELOG/CHANGELOG-v0.11.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
* New to flyte? https://start.flyte.org takes you through first run experience. (Thanks to @jeevb)
* [Grafana templates](https://docs.flyte.org/en/latest/howto/monitoring/index.html) for monitoring Flyte System and User Workflows.
* [Extend Flyte](https://docs.flyte.org/en/latest/plugins/index.html) docs.
* [FlyteIdl Docs](https://docs.flyte.org/projects/flyteidl/en/latest/) are published! You can learn about the core language that makes it all work.
* [FlyteIdl Docs](https://docs.flyte.org/en/latest/) are published! You can learn about the core language that makes it all work.
* [Additional knob](https://github.com/flyteorg/flytepropeller/pull/219/files#diff-91657d6448dfbf87f4cecf126ad02bd668ea233edcf74e860ef4f54bdd4cb552R78) for fine tuning flyte propeller performance that speeds up executions drastically.
* OidC support for Google Idp (And other OidC compliant Idps)
* Various stabilization bugs.
Expand Down
2 changes: 1 addition & 1 deletion CHANGELOG/CHANGELOG-v0.14.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,6 @@
- More use case driven examples in flytesnacks

## flytectl
- flytectl is ready for BETA. check it out - https://docs.flyte.org/projects/flytectl/en/latest/
- flytectl is ready for BETA. check it out - https://docs.flyte.org/en/latest/flytectl/

Please see the [flytekit release](https://github.com/flyteorg/flytekit/releases/tag/v0.18.0) for the full list and more details.
8 changes: 4 additions & 4 deletions CHANGELOG/CHANGELOG-v0.17.0.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
# Flyte v0.17.0

## Platform
1. Recovery Mode: Executions that fail due to external system failures (e.g. external system being down) can now be rerun in recovery mode ([flytectl --recover docs](https://docs.flyte.org/projects/flytectl/en/latest/gen/flytectl_create_execution.html)). It's also available in the UI:
1. Recovery Mode: Executions that fail due to external system failures (e.g. external system being down) can now be rerun in recovery mode ([flytectl --recover docs](https://docs.flyte.org/en/latest/flytectl/gen/flytectl_create_execution.html)). It's also available in the UI:

<img src="https://i.imgur.com/hYYzkLK.png" alt="Recovery mode in the UI" width="800"/>


## Flytekit
1. Great Expectations Integration ([docs](https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/flytekit_plugins/greatexpectations/index.html#great-expectations)).
1. Great Expectations Integration ([docs](https://docs.flyte.org/en/latest/flytesnacks/auto/integrations/flytekit_plugins/greatexpectations/index.html#great-expectations)).
1. Access to durable blob stores (AWS/GCS/etc) are now pluggable.
1. Local task execution has been updated to also trigger the type engine.
1. Tasks that have `cache=True` should now be cached when running locally as well ([docs](https://docs.flyte.org/projects/cookbook/en/latest/auto/core/flyte_basics/task_cache.html#how-local-caching-works)).
1. Tasks that have `cache=True` should now be cached when running locally as well ([docs](https://docs.flyte.org/en/latest/flytesnacks/auto/core/flyte_basics/task_cache.html#how-local-caching-works)).

Please see the [flytekit release](https://github.com/flyteorg/flytekit/releases/tag/v0.22.0) for the full list and more details.

Expand All @@ -32,7 +32,7 @@ Please see the [flytekit release](https://github.com/flyteorg/flytekit/releases/


## FlyteCtl
1. `flytectl upgrade` to automatically upgrade itself ([docs](https://docs.flyte.org/projects/flytectl/en/latest/gen/flytectl_upgrade.html)).
1. `flytectl upgrade` to automatically upgrade itself ([docs](https://docs.flyte.org/en/latest/flytectl/gen/flytectl_upgrade.html)).
1. `--dryRun` is available in most commands with server-side-effects to simulate the operations before committing any changes.

And various stabilization [fixes](https://github.com/flyteorg/flyte/milestone/17?closed=1)!
2 changes: 1 addition & 1 deletion CHANGELOG/CHANGELOG-v1.1.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
### User Improvements
Support for [Optional types](https://github.com/flyteorg/flyte/issues/2426). With the inclusion of Union types in flytekit, we can now support optional types.

[Flyte Deck](https://github.com/flyteorg/flyte/issues/2175) is now available. Please take a look at the [documentation](https://docs.flyte.org/projects/cookbook/en/latest/auto/core/flyte_basics/deck.html#sphx-glr-auto-core-flyte-basics-deck-py) and also the [OSS presentation](https://www.youtube.com/watch?v=KqyBYIaAZ7c) that was done a few weeks back.
[Flyte Deck](https://github.com/flyteorg/flyte/issues/2175) is now available. Please take a look at the [documentation](https://docs.flyte.org/en/latest/flytesnacks/auto/core/flyte_basics/deck.html#sphx-glr-auto-core-flyte-basics-deck-py) and also the [OSS presentation](https://www.youtube.com/watch?v=KqyBYIaAZ7c) that was done a few weeks back.


### Backend Improvements
Expand Down
4 changes: 2 additions & 2 deletions CHANGELOG/CHANGELOG-v1.10.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ Programmatically consuming inputs and outputs using flyteremote became a lot eas

![Usage snippet](./images/v1.10.0-flyteconsole-programmatic-access.png)

You'll now be able to use offloaded types in [eager workflows](https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/advanced_composition/eager_workflows.html#id1).
You'll now be able to use offloaded types in [eager workflows](https://docs.flyte.org/en/latest/flytesnacks/examples/advanced_composition/eager_workflows.html).

More ergonomic improvements to [pyflyte](https://docs.flyte.org/projects/flytekit/en/latest/pyflyte.html#pyflyte-cli), including the inclusion of a progress bar, the ability to activate launchplans, and the ability to interact with gate nodes in local executions.
More ergonomic improvements to [pyflyte](https://docs.flyte.org/en/latest/api/flytekit/pyflyte.html), including the inclusion of a progress bar, the ability to activate launchplans, and the ability to interact with gate nodes in local executions.

And much more. Here's the exhaustive list of changes:

Expand Down
2 changes: 1 addition & 1 deletion CHANGELOG/CHANGELOG-v1.2.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
- dbt plugin (https://github.com/flyteorg/flyte/issues/2202)
- cache overriding behavior is now open to all types (https://github.com/flyteorg/flyte/issues/2912)
- Bug: Fallback to pickling in the case of unknown types used Unions (https://github.com/flyteorg/flyte/issues/2823)
- [pyflyte run](https://docs.flyte.org/projects/flytekit/en/latest/design/clis.html#pyflyte-run) now supports [imperative workflows](https://docs.flyte.org/projects/cookbook/en/latest/auto/core/flyte_basics/imperative_wf_style.html#sphx-glr-auto-core-flyte-basics-imperative-wf-style-py)
- [pyflyte run](https://docs.flyte.org/en/latest/api/flytekit/design/clis.html#pyflyte-run) now supports [imperative workflows](https://docs.flyte.org/en/latest/flytesnacks/auto/core/flyte_basics/imperative_wf_style.html#sphx-glr-auto-core-flyte-basics-imperative-wf-style-py)
- Newlines are now stripped from client secrets (https://github.com/flyteorg/flytekit/pull/1163)
- Ensure repeatability in the generation of cache keys in the case of dictionaries (https://github.com/flyteorg/flytekit/pull/1126)
- Support for multiple images in the yaml config file (https://github.com/flyteorg/flytekit/pull/1106)
Expand Down
2 changes: 1 addition & 1 deletion CHANGELOG/CHANGELOG-v1.3.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ Users can now configure workflow execution to overwrite the cache. Each task in


### Support for Dask
Users will be able to spawn [Dask](https://www.dask.org/) ephemeral clusters as part of their workflows, similar to the support for [Ray](https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/kubernetes/ray_example/ray_example.html#sphx-glr-auto-integrations-kubernetes-ray-example-ray-example-py) and [Spark](https://docs.flyte.org/projects/cookbook/en/stable/auto/integrations/kubernetes/k8s_spark/pyspark_pi.html).
Users will be able to spawn [Dask](https://www.dask.org/) ephemeral clusters as part of their workflows, similar to the support for [Ray](https://docs.flyte.org/en/latest/flytesnacks/auto/integrations/kubernetes/ray_example/ray_example.html#sphx-glr-auto-integrations-kubernetes-ray-example-ray-example-py) and [Spark](https://docs.flyte.org/en/latest/flytesnacks/examples/k8s_spark_plugin/index.html).


## Looking Ahead
Expand Down
4 changes: 2 additions & 2 deletions CHANGELOG/CHANGELOG-v1.5.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ def wf(a: int) -> str:

Notice how calls to `t1_fixed_b` do not need to specify the `b` parameter.

This also works for [MapTasks](https://docs.flyte.org/projects/cookbook/en/latest/auto/core/control_flow/map_task.html#sphx-glr-auto-core-control-flow-map-task-py) in a limited capacity. For example:
This also works for [MapTasks](https://docs.flyte.org/en/latest/flytesnacks/auto/core/control_flow/map_task.html#sphx-glr-auto-core-control-flow-map-task-py) in a limited capacity. For example:

```
from flytekit import task, workflow, partial, map_task
Expand Down Expand Up @@ -107,5 +107,5 @@ Map tasks do not support partial tasks with lists as inputs.

## Flyteconsole

Multiple bug fixes around [waiting for external inputs](https://docs.flyte.org/projects/cookbook/en/latest/auto/core/control_flow/waiting_for_external_inputs.html#waiting-for-external-inputs).
Multiple bug fixes around [waiting for external inputs](https://docs.flyte.org/en/latest/flytesnacks/auto/core/control_flow/waiting_for_external_inputs.html#waiting-for-external-inputs).
Better support for dataclasses in the launch form.
4 changes: 2 additions & 2 deletions CHANGELOG/CHANGELOG-v1.9.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ In this release we're announcing two experimental features, namely (1) ArrayNode

### ArrayNode map tasks

ArrayNodes are described more fully in [RFC 3346](https://github.com/flyteorg/flyte/blob/master/rfc/system/3346-array-node.md), but the summary is that ArrayNode map tasks are a drop-in replacement for [regular map tasks](https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/control_flow/map_task.html#map-tasks), the only difference being the submodule used to import the `map_task` function.
ArrayNodes are described more fully in [RFC 3346](https://github.com/flyteorg/flyte/blob/master/rfc/system/3346-array-node.md), but the summary is that ArrayNode map tasks are a drop-in replacement for [regular map tasks](https://docs.flyte.org/en/latest/flytesnacks/examples/advanced_composition/map_task.html), the only difference being the submodule used to import the `map_task` function.
More explicitly, let's say you have this code:

```python
Expand Down Expand Up @@ -42,7 +42,7 @@ def wf(xs: List[int]) -> List[int]:

Execution tags allow users to can discover their executions and other flyte entities more easily, by creating smarter groupings. The feature is described in this [RFC](https://github.com/flyteorg/flyte/blob/master/rfc/system/0001-flyte-execution-tags.md).

As mentioned before, this feature is shipped in an experimental capacity, the idea being that we're going to incorporate the feedback of the community as we iterate. More work is expected to give prominence to the feature in flyteconsole, in the meanwhile, the feature is supported via [Remote](https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/remote_access/index.html).
As mentioned before, this feature is shipped in an experimental capacity, the idea being that we're going to incorporate the feedback of the community as we iterate. More work is expected to give prominence to the feature in flyteconsole, in the meanwhile, the feature is supported via [Remote](https://docs.flyte.org/en/latest/api/flytekit/remote.html#remote-access).


## Flytekit
Expand Down
22 changes: 11 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,32 +99,32 @@ Go to the [Deployment guide](https://docs.flyte.org/en/latest/deployment/deploym

## Tutorials
- [Fine-tune Code Llama on the Flyte codebase](https://github.com/unionai-oss/llm-fine-tuning/tree/main/flyte_llama#readme)
- [Forecast sales with Horovod and Spark](https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/forecasting_sales/index.html)
- [Nucleotide Sequence Querying with BLASTX](https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/blast/index.html)
- [Forecast sales with Horovod and Spark](https://docs.flyte.org/en/latest/flytesnacks/examples/forecasting_sales/index.html)
- [Nucleotide Sequence Querying with BLASTX](https://docs.flyte.org/en/latest/flytesnacks/examples/blast/index.html)

## Features
🚀 **Strongly typed interfaces**: Validate your data at every step of the workflow by defining data guardrails using Flyte types.<br>
🌐 **Any language**: Write code in any language using raw containers, or choose [Python](https://github.com/flyteorg/flytekit), [Java](https://github.com/flyteorg/flytekit-java), [Scala](https://github.com/flyteorg/flytekit-java) or [JavaScript](https://github.com/NotMatthewGriffin/pterodactyl) SDKs to develop your Flyte workflows. <br />
🔒 **Immutability**: Immutable executions help ensure reproducibility by preventing any changes to the state of an execution. <br />
🧬 **Data lineage**: Track the movement and transformation of data throughout the lifecycle of your data and ML workflows. <br />
📊 **Map tasks**: Achieve parallel code execution with minimal configuration using [map tasks](https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/advanced_composition/map_task.html). <br />
📊 **Map tasks**: Achieve parallel code execution with minimal configuration using [map tasks](https://docs.flyte.org/en/latest/flytesnacks/examples/advanced_composition/map_task.html). <br />
🌎 **Multi-tenancy**: Multiple users can share the same platform while maintaining their own distinct data and configurations. <br />
🌟 **Dynamic workflows**: [Build flexible and adaptable workflows](https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/advanced_composition/dynamics.html) that can change and evolve as needed, making it easier to respond to changing requirements. <br />
⏯️ [Wait](https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/advanced_composition/waiting_for_external_inputs.html) for **external inputs** before proceeding with the execution. <br />
🌳 **Branching**: [Selectively execute branches](https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/advanced_composition/conditions.html) of your workflow based on static or dynamic data produced by other tasks or input data. <br />
🌟 **Dynamic workflows**: [Build flexible and adaptable workflows](https://docs.flyte.org/en/latest/flytesnacks/examples/advanced_composition/dynamic_workflow.html) that can change and evolve as needed, making it easier to respond to changing requirements. <br />
⏯️ [Wait](https://docs.flyte.org/en/latest/flytesnacks/examples/advanced_composition/waiting_for_external_inputs.html) for **external inputs** before proceeding with the execution. <br />
🌳 **Branching**: [Selectively execute branches](https://docs.flyte.org/en/latest/flytesnacks/examples/advanced_composition/conditional.html) of your workflow based on static or dynamic data produced by other tasks or input data. <br />
📈 **Data visualization**: Visualize data, monitor models and view training history through plots. <br />
📂 **FlyteFile & FlyteDirectory**: Transfer [files](https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/advanced_composition/files.html) and [directories](https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/advanced_composition/folders.html) between local and cloud storage. <br />
🗃️ **Structured dataset**: Convert dataframes between types and enforce column-level type checking using the abstract 2D representation provided by [Structured Dataset](https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/data_types_and_io/structured_dataset.html). <br />
📂 **FlyteFile & FlyteDirectory**: Transfer [files](https://docs.flyte.org/en/latest/flytesnacks/examples/data_types_and_io/file.html#file) and [directories](https://docs.flyte.org/en/latest/flytesnacks/examples/data_types_and_io/folder.html) between local and cloud storage. <br />
🗃️ **Structured dataset**: Convert dataframes between types and enforce column-level type checking using the abstract 2D representation provided by [Structured Dataset](https://docs.flyte.org/en/latest/flytesnacks/examples/data_types_and_io/structured_dataset.html). <br />
🛡️ **Recover from failures**: Recover only the failed tasks. <br />
🔁 **Rerun a single task**: Rerun workflows at the most granular level without modifying the previous state of a data/ML workflow. <br />
🔍 **Cache outputs**: Cache task outputs by passing `cache=True` to the task decorator. <br />
🚩 **Intra-task checkpointing**: [Checkpoint progress](https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/advanced_composition/checkpoint.html) within a task execution. <br />
🚩 **Intra-task checkpointing**: [Checkpoint progress](https://docs.flyte.org/en/latest/flytesnacks/examples/advanced_composition/checkpoint.html) within a task execution. <br />
**Timeout**: Define a timeout period, after which the task is marked as failure. <br />
🏭 **Dev to prod**: As simple as changing your [domain](https://docs.flyte.org/en/latest/concepts/domains.html) from development or staging to production. <br />
💸 **Spot or preemptible instances**: Schedule your workflows on spot instances by setting `interruptible` to `True` in the task decorator. <br />
☁️ **Cloud-native deployment**: Deploy Flyte on AWS, GCP, Azure and other cloud services. <br />
📅 **Scheduling**: [Schedule](https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/productionizing/lp_schedules.html) your data and ML workflows to run at a specific time. <br />
📢 **Notifications**: Stay informed about changes to your workflow's state by configuring [notifications](https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/productionizing/lp_notifications.html) through Slack, PagerDuty or email. <br />
📅 **Scheduling**: [Schedule](https://docs.flyte.org/en/latest/flytesnacks/examples/productionizing/lp_schedules.html) your data and ML workflows to run at a specific time. <br />
📢 **Notifications**: Stay informed about changes to your workflow's state by configuring [notifications](https://docs.flyte.org/en/latest/flytesnacks/examples/productionizing/lp_notifications.html) through Slack, PagerDuty or email. <br />
⌛️ **Timeline view**: Evaluate the duration of each of your Flyte tasks and identify potential bottlenecks. <br />
💨 **GPU acceleration**: Enable and control your tasks’ GPU demands by requesting resources in the task decorator. <br />
🐳 **Dependency isolation via containers**: Maintain separate sets of dependencies for your tasks so no dependency conflicts arise. <br />
Expand Down
Loading

0 comments on commit 4d98306

Please sign in to comment.