diff --git a/website/docs/docs/deploy/advanced-ci.md b/website/docs/docs/deploy/advanced-ci.md new file mode 100644 index 00000000000..22e6f5c3485 --- /dev/null +++ b/website/docs/docs/deploy/advanced-ci.md @@ -0,0 +1,28 @@ +--- +title: "Advanced CI" +id: "advanced-ci" +sidebar_label: "Advanced CI" +description: "Advanced CI enables developers to compare changes by demonstrating the changes the code produces." +--- + +Advanced CI helps developers answer the question, “Will this PR build the correct changes in production?” By demonstrating the data changes that code changes produce, users can ensure they always ship trusted data products as they develop. + +Customers control what data to use and may implement synthetic data if pre-production or development data is heavily regulated or sensitive. The data selected by users is cached on dbt Labs' systems for up to 30 days. dbt Labs does not access Advanced CI cached data for its benefit, and the data is only used to provide services to clients as they direct. This caching optimizes compute usage so that the entire comparison is not rerun against the data warehouse each time the **Compare** tab is viewed. + +## Data caching + +When you run Advanced CI (by enabling **Compare changes**), dbt Cloud stores a cache of no more than 100 records for each modified model. By caching this data, users can view the examples of changed data without rerunning the comparison against the data warehouse every time. To display the changes, dbt Cloud uses a cached version of a sample of data records. These data records are queried from the database using the connection configuration (such as user, role, service account, and so on.) set in the CI job's environment. + + + +The cache is encrypted, stored in Amazon S3 or Azure blob storage in your account’s region, and automatically deleted after 30 days. No data is retained on dbt Labs' systems beyond this period. Users accessing a CI run that is more than 30 days old will not be able to see the comparison; instead, they will see a message indicating that the data has expired. No other third-party subcontractor(s), aside from the storage subcontractor(s), has access to the cached data. + + + +## Connection permissions + +The **Compare changes** feature uses the same credentials as your CI job, as defined in your CI job’s environment. Since all users will be able to view the comparison results and the cached data, the account administrator must ensure that client CI credentials are appropriately restricted. + +In particular, if you use dynamic data masking in your data warehouse, the cached data will no longer be dynamically masked in the Advanced CI output, depending on the permissions of the users who view it. We recommend limiting user access to unmasked data or considering using synthetic data for the Advanced CI testing functionality. + + diff --git a/website/docs/docs/deploy/continuous-integration.md b/website/docs/docs/deploy/continuous-integration.md index e033fc16fb7..a9e0239f394 100644 --- a/website/docs/docs/deploy/continuous-integration.md +++ b/website/docs/docs/deploy/continuous-integration.md @@ -2,6 +2,7 @@ title: "Continuous integration in dbt Cloud" sidebar_label: "Continuous integration" description: "You can set up continuous integration (CI) checks to test every single change prior to deploying the code to production just like in a software development workflow." +pagination_next: "docs/deploy/advanced-ci" --- To implement a continuous integration (CI) workflow in dbt Cloud, you can set up automation that tests code changes by running [CI jobs](/docs/deploy/ci-jobs) before merging to production. dbt Cloud tracks the state of what’s running in your production environment so, when you run a CI job, only the modified data assets in your pull request (PR) and their downstream dependencies are built and tested in a staging schema. You can also view the status of the CI checks (tests) directly from within the PR; this information is posted to your Git provider as soon as a CI job completes. Additionally, you can enable settings in your Git provider that allow PRs only with successful CI checks be approved for merging. diff --git a/website/sidebars.js b/website/sidebars.js index ae5e05d4aae..c533975076b 100644 --- a/website/sidebars.js +++ b/website/sidebars.js @@ -462,7 +462,16 @@ const sidebarSettings = { "docs/deploy/deployments", "docs/deploy/job-scheduler", "docs/deploy/deploy-environments", + { + type: "category", + label: "Continuous integration", + collapsed: true, + link: { type: "doc", id: "docs/deploy/continuous-integration" }, + items: [ "docs/deploy/continuous-integration", + "docs/deploy/advanced-ci", + ], + }, "docs/deploy/continuous-deployment", { type: "category", diff --git a/website/static/img/docs/deploy/compare-changes.png b/website/static/img/docs/deploy/compare-changes.png new file mode 100644 index 00000000000..4cdbf680cda Binary files /dev/null and b/website/static/img/docs/deploy/compare-changes.png differ diff --git a/website/static/img/docs/deploy/compare-credentials.png b/website/static/img/docs/deploy/compare-credentials.png new file mode 100644 index 00000000000..bcf81a52748 Binary files /dev/null and b/website/static/img/docs/deploy/compare-credentials.png differ diff --git a/website/static/img/docs/deploy/compare-expired.png b/website/static/img/docs/deploy/compare-expired.png new file mode 100644 index 00000000000..453a330231a Binary files /dev/null and b/website/static/img/docs/deploy/compare-expired.png differ