Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[prototype] AssetExecutionContext partition methods on sub context #19236

Closed

Conversation

jamiedemaria
Copy link
Contributor

@jamiedemaria jamiedemaria commented Jan 16, 2024

Summary & Motivation

This PR is a prototype for a potential API for partition methods on the AssetExecutionContext. This version moves all of the partition related properties and methods to a PartitionContext object so that they are grouped and made a more distinct from other context methods

Hooli PR - dagster-io/hooli-data-eng-pipelines#61
DOP PR - https://github.com/dagster-io/internal/pull/7931
Nice things about this design:

  • methods will be grouped together in the API docs since they will all fall under the PartitionContext.
  • Similarly, users will know they aren't missing a partition method in the long list of context methods

issues with this API design:

  • The naming right now isn't great, it involves a lot of repetition of partition. But the removal of some of the partitions may sacrifice clarity

How I Tested These Changes

@jamiedemaria
Copy link
Contributor Author

jamiedemaria commented Jan 16, 2024

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

Current dependencies on/for this PR:

This stack of pull requests is managed by Graphite.

@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from 38b66f9 to 50c80ab Compare January 17, 2024 21:13
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-3 branch from 53676ac to 3db0d20 Compare January 17, 2024 21:13
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from 50c80ab to 1fee854 Compare January 18, 2024 15:55
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-3 branch from 3db0d20 to 9405338 Compare January 18, 2024 15:55
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from 1fee854 to 558590d Compare January 19, 2024 16:38
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-3 branch from 9405338 to 4cde14d Compare January 19, 2024 16:38
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from 558590d to bcc9cab Compare January 19, 2024 19:09
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-3 branch from 4cde14d to 4963964 Compare January 19, 2024 19:09
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from bcc9cab to af681dc Compare January 19, 2024 20:13
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-3 branch from 4963964 to c583b85 Compare January 19, 2024 20:13
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from af681dc to 3016287 Compare January 23, 2024 00:11
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-3 branch from c583b85 to e0845f3 Compare January 23, 2024 00:11
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from 3016287 to 04fd627 Compare January 26, 2024 19:10
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-3 branch from e0845f3 to fa03263 Compare January 26, 2024 19:10
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from 04fd627 to 0e1fc3e Compare January 26, 2024 20:05
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-3 branch from fa03263 to 5621118 Compare January 26, 2024 20:05
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from 0e1fc3e to dc6edf0 Compare January 29, 2024 15:39
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-3 branch from 5621118 to b0b01ba Compare January 29, 2024 15:39
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from dc6edf0 to ccc4260 Compare January 29, 2024 16:30
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-3 branch from b0b01ba to 3b9df09 Compare January 29, 2024 16:30
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from ccc4260 to f416c05 Compare January 29, 2024 17:26
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-3 branch from 3b9df09 to 7194e6f Compare January 29, 2024 17:26
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from f416c05 to 51d46aa Compare January 29, 2024 18:16
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-3 branch from 7194e6f to 529b017 Compare January 29, 2024 18:16
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from 51d46aa to 2670364 Compare January 29, 2024 19:44
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-3 branch from 529b017 to b526d68 Compare January 29, 2024 19:44
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from 2670364 to 2660075 Compare January 29, 2024 20:03
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-3 branch from b526d68 to 6676a12 Compare January 29, 2024 20:03
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from 2660075 to a5828e1 Compare January 30, 2024 17:23
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-3 branch from 6676a12 to fbe85cf Compare January 30, 2024 17:23
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from a5828e1 to f5be566 Compare January 30, 2024 20:19
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-3 branch from fbe85cf to 7624561 Compare January 30, 2024 20:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant