Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding configurable delimiters #4265

Merged
merged 34 commits into from
Oct 13, 2023
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
1586627
Adding delimiter
runleonarun Oct 12, 2023
9ed18a9
adding delimiter page
runleonarun Oct 12, 2023
8bd37ef
Apply suggestions from code review
runleonarun Oct 12, 2023
4ce3545
Apply suggestions from code review
runleonarun Oct 12, 2023
f233765
Apply suggestions from code review
runleonarun Oct 12, 2023
52a6916
adding examles
runleonarun Oct 12, 2023
4a75514
Update website/docs/reference/resource-configs/delimiter.md
runleonarun Oct 12, 2023
ad16abc
Update website/docs/reference/resource-configs/delimiter.md
runleonarun Oct 12, 2023
ffa3c81
Apply suggestions from code review
runleonarun Oct 12, 2023
bc8b6ec
Update website/docs/reference/resource-configs/delimiter.md
runleonarun Oct 12, 2023
6f7e161
adding examles
runleonarun Oct 12, 2023
7eacd47
Merge branch 'current' into delimiter-adding
runleonarun Oct 12, 2023
87576e6
Apply suggestions from code review
runleonarun Oct 12, 2023
3d27d14
Apply suggestions from code review
runleonarun Oct 12, 2023
bec181a
Update delimiter.md
runleonarun Oct 12, 2023
cde62d0
Update delimiter.md
runleonarun Oct 12, 2023
cd69a56
adding versioning note
runleonarun Oct 12, 2023
f4e8e71
fixing sentence
runleonarun Oct 12, 2023
111472d
addign link to migration guide
runleonarun Oct 12, 2023
03be195
fixing note title
runleonarun Oct 12, 2023
29a2306
Apply suggestions from code review
runleonarun Oct 12, 2023
b0b1607
Merge branch 'current' into delimiter-adding
runleonarun Oct 12, 2023
bc638a9
addign to examples
runleonarun Oct 12, 2023
c739349
Merge branch 'delimiter-adding' of github.com:dbt-labs/docs.getdbt.co…
runleonarun Oct 12, 2023
462f4c7
fixing wording adding csv
runleonarun Oct 12, 2023
211a847
Merge branch 'current' into delimiter-adding
matthewshaver Oct 12, 2023
e5f602f
Apply suggestions from code review
runleonarun Oct 13, 2023
3042c76
Update website/docs/reference/resource-configs/delimiter.md
runleonarun Oct 13, 2023
7ce357d
Update delimiter.md
runleonarun Oct 13, 2023
4eccbfc
Merge branch 'current' into delimiter-adding
runleonarun Oct 13, 2023
5670118
Merge branch 'current' into delimiter-adding
runleonarun Oct 13, 2023
e1283c2
Merge branch 'current' into delimiter-adding
runleonarun Oct 13, 2023
6fbffaa
Update delimiter.md
runleonarun Oct 13, 2023
03fb18a
Update delimiter.md
runleonarun Oct 13, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 88 additions & 0 deletions website/docs/reference/resource-configs/delimiter.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
---
resource_types: [seeds]
datatype: <string>
---

## Description

Optionally specify a delimiter for [seed](/docs/build/seeds), by providing a string value. The defaults to comma.
runleonarun marked this conversation as resolved.
Show resolved Hide resolved

runleonarun marked this conversation as resolved.
Show resolved Hide resolved
Without specifying this, dbt will infer the datatype based on the column values in your seed file.
runleonarun marked this conversation as resolved.
Show resolved Hide resolved

runleonarun marked this conversation as resolved.
Show resolved Hide resolved
## Usage
Specify column types in your `dbt_project.yml` file:

<File name='dbt_project.yml'>

```yml
seeds:
jaffle_shop:
country_codes:
+column_types:
country_code: varchar(2)
country_name: varchar(32)
+delimiter: ","
runleonarun marked this conversation as resolved.
Show resolved Hide resolved
```

</File>

runleonarun marked this conversation as resolved.
Show resolved Hide resolved


Or:
runleonarun marked this conversation as resolved.
Show resolved Hide resolved

<File name='seeds/properties.yml'>

```yml
version: 2

seeds:
- name: country_codes
config:
column_types:
country_code: varchar(2)
country_name: varchar(32)
delimiter: ","
```

</File>

If you have previously run `dbt seed`, you'll need to run `dbt seed --full-refresh` for the changes to take effect.

Note that you will need to use the fully directory path of a seed when configuring `delimiter`. For example, for a seed file at `seeds/marketing/utm_mappings.csv`, you will need to configure it like so:

<File name='dbt_project.yml'>

```yml
seeds:
jaffle_shop:
marketing:
utm_mappings:
+delimiter:
...

```

</File>

## Examples

### Use a varchar column type to preserve leading zeros in a zipcode
runleonarun marked this conversation as resolved.
Show resolved Hide resolved
(Note: preservation of leading zeros works for v0.16.0 onwards)
runleonarun marked this conversation as resolved.
Show resolved Hide resolved
<File name='dbt_project.yml'>

```yml
seeds:
jaffle_shop: # you must include the project name
warehouse_locations:
+delimiter: ","
```

</File>

## Recommendation

runleonarun marked this conversation as resolved.
Show resolved Hide resolved
Use this configuration only when required. Otherwise you can omit this configuration.

## Troubleshooting

The `delimiter` configuration should contain a value when set and not be empty. This value should not be set to a semicolon.
runleonarun marked this conversation as resolved.
Show resolved Hide resolved
2 changes: 2 additions & 0 deletions website/docs/reference/seed-configs.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ seeds:
[<resource-path>](/reference/resource-configs/resource-path):
[+](/reference/resource-configs/plus-prefix)[quote_columns](/reference/resource-configs/quote_columns): true | false
[+](/reference/resource-configs/plus-prefix)[column_types](/reference/resource-configs/column_types): {column_name: datatype}
[+](/reference/resource-configs/plus-prefix)[delimiter](/reference/resource-configs/delimiter): <string>

```

Expand All @@ -43,6 +44,7 @@ seeds:
config:
[quote_columns](/reference/resource-configs/quote_columns): true | false
[column_types](/reference/resource-configs/column_types): {column_name: datatype}
[delimiter](/reference/resource-configs/grants): <string>

```

Expand Down
Loading