Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Add ExtraParamsMixin to Databricks Autoloader #2

Open
dannymeijer opened this issue May 22, 2024 · 0 comments
Open

[FEATURE] Add ExtraParamsMixin to Databricks Autoloader #2

dannymeijer opened this issue May 22, 2024 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@dannymeijer
Copy link
Member

Is your feature request related to a problem? Please describe.

No specific problem

Describe the solution you'd like

Right now koheesio's Databricks Autoloader implementation supports adding extra parameters through the options Field. Ideally we should make it possible to add these options through ExtraParamsMixin.

This would make it possible to call the class like this:

foo = AutoLoader(
    ...
    extra_option_that_i_want_to_add=42
)

In this example extra_option_that_i_want_to_add would be recognized as an ExtraParams field, and dynamically added to the get_options() property.

Unit tests should not need to be changed for this to work.

Describe alternatives you've considered

N/A - ExtraParamsMixin is used in a lot of other places

Additional context

It might be worth looking if any other classes are not yet using ExtraParamsMixin and address them accordingly.

@dannymeijer dannymeijer added the enhancement New feature or request label May 22, 2024
@dannymeijer dannymeijer self-assigned this May 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: No status
Development

No branches or pull requests

1 participant