Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] option to disable package model schemas when model is disabled #9762

Closed
3 tasks done
pnadolny13 opened this issue Mar 14, 2024 · 3 comments
Closed
3 tasks done
Labels
enhancement New feature or request

Comments

@pnadolny13
Copy link

pnadolny13 commented Mar 14, 2024

Is this your first time submitting a feature request?

  • I have read the expectations for open source contributors
  • I have searched the existing issues, and I could not find an existing issue for this feature
  • I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature

I'd like to override a model that comes from a package in order to write custom logic. The docs explain how to disable a model in order To avoid a compilation error due to duplicate models.. but it seems that schemas and tests always persist from the base package. I understand that its useful in many cases to enforce an interface this way (i.e. so downstream package models dont fail) but in other cases it would be nice to have the flexibility to be able to make schema changes at my own risk.

I'm wondering if its possible to add an additional configuration flag under the override model configuration in dbt_project.yml:

+enabled:  false
+schema_enabled: false

Ideally +enabled: false would become or also accept a dictionary like this but I suspect that is a larger lift:

+enabled:
    model: false
    schema: false
    test: false

Referenced in this slack thread also.

Describe alternatives you've considered

No response

Who will this benefit?

Anyone trying to use packages heavily and needs more control over how to override models.

Are you interested in contributing this feature?

I'm happy to contribute but I'm unfamiliar with the codebase so getting some direction on appetite for this feature and aligning on an implementation would help before I start.

Anything else?

No response

@pnadolny13 pnadolny13 added enhancement New feature or request triage labels Mar 14, 2024
@dbeatty10
Copy link
Contributor

Thanks for opening this issue and referencing the Slack thread @pnadolny13 !

Could you share a simple (but detailed) code example so that we can more easily reproduce the situation you are seeing?

Did you try the following steps, by any chance?

  1. Disable the model from the package that you don't want
  2. Create your own version of the model with a different / unique name
    • i.e., if the package model is named segment_web_page_views.sql, then name your version overridden_segment_web_page_views.sql
  3. Optional: set a custom alias to the original name
    • so that the materialized database table has the same name as model from the package
      # models/_models.yml
      
      models:
        - name: overridden_segment_web_page_views
          config:
            alias: segment_web_page_views

mirnawong1 added a commit to dbt-labs/docs.getdbt.com that referenced this issue Mar 15, 2024
## What are you changing in this pull request and why?

While researching dbt-labs/dbt-core#9762,
there were some opportunities to improve the readability of the relevant
docs.

## Checklist
- [x] Review the [Content style
guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md)
so my content adheres to these guidelines.
@pnadolny13
Copy link
Author

pnadolny13 commented Mar 15, 2024

@dbeatty10 thanks for the quick reply! I think your suggestion actually resolved my question and I've changed my opinion on this.

My use case is to override a model from a package but if I want to make any changes in the parent project to the schema.yml then it errors out because its still reading the package schema.yml contents even though the model is disabled. On second thought this is probably a good thing because making changes to the schema in the override might cause downstream models in the package to fail, so whether it was by design or not its acting as a contract. If there was ever a case where I need to override the model and change the schema, assuming I'm well aware of the consequences, then I can use the alias approach you recommended.

I can close this issue if you'd like and re-open it later if anything changes.

@dbeatty10
Copy link
Contributor

Awesome @pnadolny13 -- thanks for collaborating on this!

I'm going to close this, but definitely reach out if anything changes and we can re-visit it.

In the meantime, I've opened this issue to update our docs: dbt-labs/docs.getdbt.com#5094

It contains an alternative idea that I didn't actually try out yet.

@dbeatty10 dbeatty10 closed this as not planned Won't fix, can't repro, duplicate, stale Mar 15, 2024
@dbeatty10 dbeatty10 removed the triage label Mar 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants