-
Notifications
You must be signed in to change notification settings - Fork 984
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add athena adapter preview #5858
Merged
Merged
Changes from all commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
6d5e60f
Add athena adapter preview
matthewshaver f366b16
Fixing ID
matthewshaver 8d677b5
Merge branch 'current' into athena-preview
amychen1776 125980c
Update website/docs/docs/dbt-versions/release-notes.md
matthewshaver cee6f4c
Update connect-amazon-athena.md
matthewshaver 3cf8382
Update release-notes.md
matthewshaver aaa105b
Update website/docs/docs/cloud/connect-data-platform/about-connection…
matthewshaver 059bad5
Merge branch 'current' into athena-preview
matthewshaver 4c849b0
Adding to sidebar
matthewshaver bdf92ea
Merge branch 'current' into athena-preview
mirnawong1 11baf5e
Update website/docs/docs/cloud/connect-data-platform/connect-amazon-a…
matthewshaver 6d88ad9
Update website/docs/docs/cloud/connect-data-platform/connect-amazon-a…
matthewshaver File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
44 changes: 44 additions & 0 deletions
44
website/docs/docs/cloud/connect-data-platform/connect-amazon-athena.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
--- | ||
title: "Connect Amazon Athena" | ||
id: connect-amazon-athena | ||
description: "Configure the Amazon Athena data platform connection in dbt Cloud." | ||
sidebar_label: "Connect Amazon Athena" | ||
--- | ||
|
||
# Connect Amazon Athena <Lifecycle status="beta" /> | ||
|
||
:::note beta | ||
|
||
This is a beta feature with limited availability. A public preview will follow shortly, for wider early access. For more information, check out our [product lifecycle](/docs/dbt-versions/product-lifecycles#dbt-cloud) page. | ||
|
||
::: | ||
|
||
Your environment(s) must be on ["Keep on latest version"](/docs/dbt-versions/versionless-cloud) to use the Amazon Athena connection. | ||
matthewshaver marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
Connect dbt Cloud to Amazon's Athena interactive query service to build your dbt project. The following are the required and optional fields for configuring the Athena connection: | ||
|
||
| Field | Option | Description | Type | Required? | Example | | ||
| ----------------------------- | ---------------- | ----------------------------------------------------------------------------------- | ------ | --------- | ------- | | ||
| AWS region name | region_name | AWS region of your Athena instance | String | Required | eu-west-1 | | ||
| Database (catalog) | database | Specify the database (Data catalog) to build models into (lowercase only) | String | Required | awsdatacatalog | | ||
| AWS S3 staging directory | s3_staging_dir | S3 location to store Athena query results and metadata | String | Required | s3://bucket/dbt/ | | ||
| Athena workgroup | work_group | Identifier of Athena workgroup | String | Optional | my-custom-workgroup | | ||
| Athena Spark workgroup | spark_work_group | Identifier of Athena Spark workgroup for running Python models | String | Optional | my-spark-workgroup | | ||
| AWS S3 data directory | s3_data_dir | Prefix for storing tables, if different from the connection's s3_staging_dir | String | Optional | s3://bucket2/dbt/ | | ||
| AWS S3 data naming convention | s3_data_naming | How to generate table paths in s3_data_dir | String | Optional | schema_table_unique | | ||
| AWS S3 temp tables prefix | s3_tmp_table_dir | Prefix for storing temporary tables, if different from the connection's s3_data_dir | String | Optional | s3://bucket3/dbt/ | | ||
| Poll interval | poll_interval | Interval in seconds to use for polling the status of query results in Athena | Integer| Optional | 5 | | ||
| Query retries | num_retries | Number of times to retry a failing query | Integer| Optional | 3 | | ||
| Boto3 retries | num_boto3_retries| Number of times to retry boto3 requests (e.g. deleting S3 files for materialized tables)| Integer | Optional | 5 | | ||
| Iceberg retries | num_iceberg_retries| Number of times to retry iceberg commit queries to fix ICEBERG_COMMIT_ERROR | Integer | Optional | 0 | | ||
|
||
### Development credentials | ||
|
||
Enter your _development_ (not deployment) credentials with the following fields: | ||
|
||
| Field | Option | Description | Type | Required | Example | | ||
| --------------------- | --------------------- | -------------------------------------------------------------------------- | ------ | -------- | -------- | | ||
| AWS Access Key ID | aws_access_key_id | Access key ID of the user performing requests | String | Required | AKIAIOSFODNN7EXAMPLE | | ||
| AWS Secret Access Key | aws_secret_access_key | Secret access key of the user performing requests | String | Required | wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY | | ||
| Schema | schema | Specify the schema (Athena database) to build models into (lowercase only) | String | Required | dbt | | ||
| Threads | threads | | Integer| Optional | 3 | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
qq - is it preview or beta?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the beta