Is there a way to backfill historical data (once) for a new Flow? #3371
Unanswered
newskooler
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi,
I just started reading about Prefect (and have a little experience using Airflow).
My goal is to set a task which runs daily in Prefect and collects data to a folder (I guess that's what Prefect can help me do for sure).
Also my goal is to populate the historical data (as if I ran this job back in time).
In Airflow there is this concept of
start_date
which when set in the past, the DAG will run since that date and populate on each interval.For example, if I have a task which takes a date and returns the data for that date, such as:
Is there a native way to do this in Prefect?
I could not find this answered anywhere in here or the docs.
Any help / guidance will be super useful.
P.S. I have also asked the question here and expanded with a bit more information on what I have already tried.
Beta Was this translation helpful? Give feedback.
All reactions