Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prerequisite for project dependencies: 1 dbt project == 1 dbt Cloud project #5447

Closed
1 task done
jtcohen6 opened this issue May 6, 2024 · 1 comment · Fixed by #5453
Closed
1 task done

Prerequisite for project dependencies: 1 dbt project == 1 dbt Cloud project #5447

jtcohen6 opened this issue May 6, 2024 · 1 comment · Fixed by #5453
Labels
content Improvements or additions to content improvement Use this when an area of the docs needs improvement as it's currently unclear

Comments

@jtcohen6
Copy link
Collaborator

jtcohen6 commented May 6, 2024

Contributions

  • I have read the contribution docs, and understand what's expected of me.

Link to the page on docs.getdbt.com requiring updates

https://docs.getdbt.com/docs/collaborate/govern/project-dependencies
https://docs.getdbt.com/docs/deploy/deploy-environments#staging-environment
https://docs.getdbt.com/best-practices/how-we-mesh/mesh-4-faqs

What part(s) of the page would you like to see updated?

  • Project dependencies (for cross-project references) require that 1 dbt project == 1 dbt Cloud project.
    • If the same project name is duplicated across multiple dbt Cloud projects, then cross-project references will resolve non-deterministically.
  • To that end, if one dbt project should have both a Staging environment and a Production environment, then the Staging + Production environments need to be defined within the same dbt Cloud project.

Additional information

We don't live in this reality yet, but we're very close.

Copying context from dbt-labs/dbt-core#10092 (comment):

To that end, we are working hard to remove the historical limitations that required customers to "duplicate" projects in dbt Cloud — Staging environments for data isolation (beta), env-level permissions and env-level data warehouse connections (coming very soon).

@jtcohen6 jtcohen6 added content Improvements or additions to content improvement Use this when an area of the docs needs improvement as it's currently unclear labels May 6, 2024
@jtcohen6
Copy link
Collaborator Author

jtcohen6 commented May 6, 2024

Also (in conversation with @will-sargent-dbtlabs):

  • 1 source == 1 source
  • Always use conditional logic within the same source definition (e.g. env_var) to add prefixes/suffixes conditionally, if different source data for Production versus Staging/Development environments
  • (Corollary: Strongly discourage overriding ref/source macros to accomplish this, or elsehow putting this as conditional logic within models)
sources:
  - name: my_source
    schema: "my_source_schema_{{ env_var('ENV_SUFFIX') }}"

jtcohen6 added a commit that referenced this issue May 14, 2024
resolves #5447

See linked issue for context

---------

Co-authored-by: Matt Shaver <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
content Improvements or additions to content improvement Use this when an area of the docs needs improvement as it's currently unclear
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant