Skip to content

Commit

Permalink
return error on excessive partition count
Browse files Browse the repository at this point in the history
  • Loading branch information
houqp committed May 28, 2024
1 parent 8f62677 commit 18b46bb
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions datafusion/core/src/datasource/listing/helpers.rs
Original file line number Diff line number Diff line change
Expand Up @@ -380,6 +380,12 @@ pub async fn pruned_partition_list<'a>(
partitions.len(),
&list_path
);
if partitions.len() > 90 {
return Err(DataFusionError::Execution(format!(
"Excessive partition count due to unconstrainted filter: {}",
partitions.len()
)));
}

let pruned =
prune_partitions(table_path, partitions, filters, partition_cols).await?;
Expand Down

0 comments on commit 18b46bb

Please sign in to comment.