Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(substrait): replace SessionContext with a trait #13343

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

notfilippo
Copy link
Contributor

Which issue does this PR close?

Closes #13342.

Rationale for this change

See original issue.

What changes are included in this PR?

See original issue.

Are these changes tested?

Are there any user-facing changes?

Yes, the arguments for the substrait consumer / producer are changed. I could ideally provide some deprecated shim methods to provide backwards compatibility.

@github-actions github-actions bot added core Core DataFusion crate substrait labels Nov 10, 2024
pub(crate) fn schema_for_ref(
/// Retrieve the [`SchemaProvider`] for a specific [`TableReference`], if it
/// esists.
pub fn schema_for_ref(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if there is a way in which we could keep this method as pub(crate), but I don't see many reasons to make this method not public.

datafusion/substrait/src/logical_plan/consumer.rs Outdated Show resolved Hide resolved
datafusion/substrait/src/logical_plan/consumer.rs Outdated Show resolved Hide resolved
@notfilippo notfilippo changed the title feat(substrait): replace SessionContext with SessionState feat(substrait): replace SessionContext with a trait Nov 15, 2024
@notfilippo
Copy link
Contributor Author

I've updated this PR to use a trait with the required method and not the SessionState

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @notfilippo -- I think this structure looks good to me and it is consistent with other trait based APIs in the system

I have a few comments that might help about naming, but I think the documentation is what is really important to do. However, I also think we could do it as a follow on PR and this PR could be merged as is.

};

#[async_trait]
pub trait Context: Sync + Send + FunctionRegistry {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we please add some documentation here explaining

  1. What this trait is used for (provide the needed API to substrait encoding/deocing)
  2. That it is implemented for SessionState?

Also as Context is fairly generic, perhaps we could rename it to something like SubstraitContext or SubstraitPlanningContext

}

#[async_trait]
impl Context for SessionState {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would you think about also impl Context for SessionContext ? That would minimize the potential API changes on downstream users (also it would likely allow you to avoid many of the changes in this PR to the tests)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core DataFusion crate substrait
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[substrait] Replace SessionContext with a trait
3 participants