-
Notifications
You must be signed in to change notification settings - Fork 951
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'current' into mult-unique-keys
- Loading branch information
Showing
11 changed files
with
179 additions
and
18 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -60,25 +60,47 @@ Update your versions in packages.yml, then run dbt deps | |
|
||
## Predictable package installs | ||
|
||
Starting in dbt Core v1.7, dbt generates a `package-lock.yml` file in the root of your project. This contains the complete set of resolved packages based on the `packages` configuration in `dependencies.yml` or `packages.yml`. Each subsequent invocation of `dbt deps` will install from the _locked_ set of packages specified in this file. Storing the complete set of required packages (with pinned versions) in version-controlled code ensures predictable installs in production and consistency across all developers and environments. | ||
Starting in dbt v1.7, dbt generates a `package-lock.yml` file in the root of your project. This file ensures consistent and predictable package installs by storing the exact versions (including commit SHAs) of all resolved packages specified in your `packages.yml` or `dependencies.yml`. This consistency is crucial for maintaining stability in development and production environments, preventing unexpected issues from new releases with potential bugs. | ||
|
||
The `package-lock.yml` file should be committed in Git initially, and then updated and committed only when you want to change versions or uninstall a package (for example `dbt deps --upgrade` or `dbt deps --lock`). | ||
When you run `dbt deps`, dbt installs packages based on the locked versions in `package-lock.yml`. To update these locked versions, you must explicitly run `dbt deps --upgrade` and commit the updated `package-lock.yml` file. Storing this file in version control guarantees consistency across all environments and for all developers. | ||
|
||
The `package-lock.yml` file includes a `sha1_hash` of the `packages` config. This enables dbt to detect if the `packages` config has been updated, and to rerun dependency resolution. To only check for changes to the `packages` config and update the lock file accordingly without installing those packages, provide the `--lock` flag (that is, `dbt deps --lock`). | ||
### Managing `package-lock.yml` | ||
|
||
### Forcing upgrades | ||
The `package-lock.yml` file should be committed to Git initially and updated only when you intend to change versions or uninstall a package. For example, run `dbt deps --upgrade` to get updated package versions or `dbt deps --lock` to update the lock file based on changes to the packages config without installing the packages. | ||
|
||
It's possible to force package resolution to rerun, even if the `packages` config hasn't changed, by running `dbt deps --upgrade`. This enables you to get the latest commits from the `main` branch of an internally maintained `git` package while accepting the risk of unpredictable builds. | ||
To bypass using `package-lock.yml` entirely, you can add it to your project's `.gitignore`. However, this approach sacrifices the predictability of builds. If you choose this route, we strongly recommend adding version pins for third-party packages in your `packages` config. | ||
|
||
An alternative to running `dbt deps --upgrade` in production is to "ignore" the lock file by adding `package-lock.yml` to your project's `.gitignore` file. | ||
### Detecting changes in `packages` config | ||
|
||
If you pursue either approach, dbt Labs strongly recommends adding version pins for third-party packages within your `packages` config. | ||
The `package-lock.yml` file includes a `sha1_hash` of your packages config. If you update `packages.yml`, dbt will detect the change and rerun dependency resolution during the next `dbt deps` command. To update the lock file without installing the new packages, use the `--lock` flag: | ||
|
||
## Add specific packages | ||
```shell | ||
dbt deps --lock | ||
``` | ||
|
||
### Forcing package updates | ||
|
||
To update all packages, even if `packages.yml` hasn’t changed, use the `--upgrade` flag: | ||
|
||
```shell | ||
|
||
dbt deps --upgrade | ||
|
||
``` | ||
|
||
This is particularly useful for fetching the latest commits from the `main` branch of an internally maintained Git package. | ||
|
||
:::warning | ||
Forcing package upgrades may introduce build inconsistencies unless carefully managed. | ||
::: | ||
|
||
### Adding specific packages | ||
|
||
The `dbt deps` command can add or update an existing package configuration — no need to remember the exact syntax for package configurations. | ||
The `dbt deps` command can add or update package configurations directly, saving you from remembering exact syntax. | ||
|
||
For Hub packages (default), which are the easiest to install: | ||
#### Hub packages (default) | ||
|
||
Hub packages are the default package types and the easiest to install. | ||
|
||
```shell | ||
dbt deps --add-package dbt-labs/[email protected] | ||
|
@@ -87,11 +109,15 @@ dbt deps --add-package dbt-labs/[email protected] | |
dbt deps --add-package dbt-labs/snowplow@">=0.7.0,<0.8.0" | ||
``` | ||
|
||
For other package types, use the `--source` flag: | ||
#### Non-Hub packages | ||
|
||
Use the `--source` flag to specify the type of package to be installed: | ||
|
||
```shell | ||
# add package from git | ||
|
||
# Git package | ||
dbt deps --add-package https://github.com/fivetran/[email protected] --source git | ||
|
||
# add package from local | ||
# Local package | ||
dbt deps --add-package /opt/dbt/redshift --source local | ||
``` |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,114 @@ | ||
import React, { useState, useMemo } from 'react'; | ||
import Markdown from 'markdown-to-jsx'; | ||
|
||
const stripMarkdown = (text) => { | ||
let strippedText = text.replace(/\[([^\]]+)\]\([^)]+\)/g, '$1'); | ||
strippedText = strippedText.replace(/[_*`~]/g, ''); | ||
return strippedText; | ||
}; | ||
|
||
const parseMarkdownTable = (markdown) => { | ||
const rows = markdown.trim().split('\n'); | ||
const headers = rows[0].split('|').map((header) => header.trim()).filter(Boolean); | ||
|
||
const alignmentsRow = rows[1].split('|').map((align) => align.trim()).filter(Boolean); | ||
const columnAlignments = alignmentsRow.map((alignment) => { | ||
if (alignment.startsWith(':') && alignment.endsWith(':')) { | ||
return 'center'; | ||
} else if (alignment.startsWith(':')) { | ||
return 'left'; | ||
} else if (alignment.endsWith(':')) { | ||
return 'right'; | ||
} else { | ||
return 'left'; | ||
} | ||
}); | ||
|
||
const data = rows.slice(2).map(row => row.split('|').map(cell => cell.trim()).filter(Boolean)); | ||
|
||
return { headers, data, columnAlignments }; | ||
}; | ||
|
||
const SortableTable = ({ children }) => { | ||
const { headers, data: initialData, columnAlignments } = useMemo( | ||
() => parseMarkdownTable(children), | ||
[children] | ||
); | ||
|
||
const [data, setData] = useState(initialData); | ||
const [sortConfig, setSortConfig] = useState({ key: '', direction: 'asc' }); | ||
|
||
const sortTable = (keyIndex) => { | ||
const newDirection = (sortConfig.key === keyIndex && sortConfig.direction === 'asc') ? 'desc' : 'asc'; | ||
setSortConfig({ key: keyIndex, direction: newDirection }); | ||
|
||
const sortedData = [...data].sort((a, b) => { | ||
const aVal = stripMarkdown(a[keyIndex]); | ||
const bVal = stripMarkdown(b[keyIndex]); | ||
if (aVal < bVal) return newDirection === 'asc' ? -1 : 1; | ||
if (aVal > bVal) return newDirection === 'asc' ? 1 : -1; | ||
return 0; | ||
}); | ||
|
||
setData(sortedData); | ||
}; | ||
|
||
return ( | ||
<table> | ||
<thead> | ||
<tr> | ||
{headers.map((header, index) => ( | ||
<th | ||
key={index} | ||
onClick={() => sortTable(index)} | ||
style={{ | ||
cursor: 'pointer', | ||
position: 'relative', | ||
textAlign: columnAlignments[index], | ||
padding: '10px' | ||
}} | ||
> | ||
<div style={{ | ||
display: 'flex', | ||
alignItems: 'center', | ||
justifyContent: columnAlignments[index] === 'center' ? 'center' : columnAlignments[index] | ||
}}> | ||
<span style={{ marginRight: '5px' }}>{header}</span> | ||
<span style={{ | ||
opacity: sortConfig.key === index && sortConfig.direction === 'asc' ? 1 : (sortConfig.key === index ? 0.5 : 0.5) | ||
}}> | ||
↑ | ||
</span> | ||
<span style={{ | ||
marginLeft: '5px', | ||
opacity: sortConfig.key === index && sortConfig.direction === 'desc' ? 1 : (sortConfig.key === index ? 0.5 : 0.5) | ||
}}> | ||
↓ | ||
</span> | ||
</div> | ||
</th> | ||
))} | ||
</tr> | ||
</thead> | ||
<tbody> | ||
{data.map((row, rowIndex) => ( | ||
<tr key={rowIndex}> | ||
{row.map((cell, cellIndex) => ( | ||
<td | ||
key={cellIndex} | ||
style={{ | ||
textAlign: columnAlignments[cellIndex], | ||
padding: '8px' | ||
}} | ||
> | ||
<Markdown>{cell || '\u00A0'}</Markdown> | ||
</td> | ||
))} | ||
</tr> | ||
))} | ||
</tbody> | ||
</table> | ||
); | ||
}; | ||
|
||
export default SortableTable; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified
BIN
-5.22 MB
(6.1%)
website/static/img/docs/dbt-cloud/change_user_to_read_only_20221023.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+321 KB
website/static/img/docs/dbt-cloud/change_user_to_read_only_20221023.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters