Skip to content

Commit

Permalink
Fix IdentityPartitionMapping perf optimization (#18456)
Browse files Browse the repository at this point in the history
## Summary & Motivation
#17703 removed partitions_def
from DefaultPartitionMapping, which these optimizations were relying on
existing.

We should probably remove that field from the base class now that it is
no longer reliably there and is only available on time based partition
mapping - but that is out of scope for this fix

## How I Tested These Changes
Dry run for an asset graph involving multi-partitions finishes quickly
again
  • Loading branch information
gibsondan authored Dec 2, 2023
1 parent 31d7c56 commit b5bac82
Showing 1 changed file with 2 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ def get_upstream_mapped_partitions_result_for_partitions(
if downstream_partitions_subset is None:
check.failed("downstream asset is not partitioned")

if downstream_partitions_subset.partitions_def == upstream_partitions_def:
if downstream_partitions_def == upstream_partitions_def:
return UpstreamPartitionsResult(downstream_partitions_subset, [])

upstream_partition_keys = set(
Expand Down Expand Up @@ -150,7 +150,7 @@ def get_downstream_partitions_for_partitions(
if upstream_partitions_subset is None:
check.failed("upstream asset is not partitioned")

if upstream_partitions_subset.partitions_def == downstream_partitions_def:
if upstream_partitions_def == downstream_partitions_def:
return upstream_partitions_subset

upstream_partition_keys = set(upstream_partitions_subset.get_partition_keys())
Expand Down

0 comments on commit b5bac82

Please sign in to comment.