Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kedro-airflow: implement @task and @dag decorators in DAG, generated by plugin #705

Open
DimedS opened this issue May 29, 2024 · 2 comments

Comments

@DimedS
Copy link
Contributor

DimedS commented May 29, 2024

Description

Following the investigation done in #25:

We will implement the @task and @dag decorators from the TaskFlow API into the DAG generated by the kedro-airflow plugin from the Kedro project. This implementation will:

  1. Make the DAG code shorter and cleaner.
  2. Allow us to move away from using KedroOperator, simplifying the user experience and addressing Kedro plugins issue Introduce KedroOperator for Airflow #482.
  3. Demonstrate our commitment to keeping up with current best practices and open the way for potentially more extensive use of TaskFlow in the future.

Possible Implementation

Modify airflow_dag_template.j2 accordingly

@noklam
Copy link
Contributor

noklam commented May 29, 2024

This seems contradicts to the conclusion of #25?

@ankatiyar
Copy link
Contributor

@noklam We discussed a bit more and decided to adopt certain aspects of the TaskFlow Api for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: No status
Development

No branches or pull requests

3 participants