Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug/persist-docs #4

Merged
merged 9 commits into from
Jan 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .github/workflows/auto-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
name: 'auto release'
on:
pull_request:
types:
- closed
branches:
- main

jobs:
call-workflow-passing-data:
if: github.event.pull_request.merged
uses: fivetran/dbt_package_automations/.github/workflows/auto-release.yml@main
secrets: inherit
7 changes: 6 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
# dbt_qualtrics_source v0.1.1
[PR #4](https://github.com/fivetran/dbt_qualtrics_source/pull/4) includes the following update:
## Bug fix
- Shortened the documentation description for field `distribution_status`, so no character limit error occurs when persisting docs.

# dbt_qualtrics_source v0.1.0
This is the initial release of the Qualtrics dbt source package!

Expand All @@ -10,4 +15,4 @@ This is the initial release of the Qualtrics dbt source package!
- Datetime fields are renamed to `<event happened>_at`.
- Adds column-level testing where applicable. For example, all primary keys are tested for uniqueness and non-null values.
- Generates a comprehensive data dictionary of your Qualtrics data through the [dbt docs site](https://fivetran.github.io/dbt_qualtrics_source/).
- These tables are designed to work simultaneously with our [Qualtrics transformation package](https://github.com/fivetran/dbt_qualtrics).
- These tables are designed to work simultaneously with our [Qualtrics transformation package](https://github.com/fivetran/dbt_qualtrics).
2 changes: 1 addition & 1 deletion dbt_project.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: 'qualtrics_source'
version: '0.1.0'
version: '0.1.1'
config-version: 2
require-dbt-version: [">=1.3.0", "<2.0.0"]

Expand Down
2 changes: 1 addition & 1 deletion docs/catalog.json

Large diffs are not rendered by default.

24 changes: 12 additions & 12 deletions docs/index.html

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/manifest.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/run_results.json

Large diffs are not rendered by default.

10 changes: 5 additions & 5 deletions integration_tests/ci/sample.profiles.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,13 @@ integration_tests:
pass: "{{ env_var('CI_REDSHIFT_DBT_PASS') }}"
dbname: "{{ env_var('CI_REDSHIFT_DBT_DBNAME') }}"
port: 5439
schema: qualtrics_source_integration_tests
schema: qualtrics_source_integration_tests_02
threads: 8
bigquery:
type: bigquery
method: service-account-json
project: 'dbt-package-testing'
schema: qualtrics_source_integration_tests
schema: qualtrics_source_integration_tests_02
threads: 8
keyfile_json: "{{ env_var('GCLOUD_SERVICE_KEY') | as_native }}"
snowflake:
Expand All @@ -33,7 +33,7 @@ integration_tests:
role: "{{ env_var('CI_SNOWFLAKE_DBT_ROLE') }}"
database: "{{ env_var('CI_SNOWFLAKE_DBT_DATABASE') }}"
warehouse: "{{ env_var('CI_SNOWFLAKE_DBT_WAREHOUSE') }}"
schema: qualtrics_source_integration_tests
schema: qualtrics_source_integration_tests_02
threads: 8
postgres:
type: postgres
Expand All @@ -42,13 +42,13 @@ integration_tests:
pass: "{{ env_var('CI_POSTGRES_DBT_PASS') }}"
dbname: "{{ env_var('CI_POSTGRES_DBT_DBNAME') }}"
port: 5432
schema: qualtrics_source_integration_tests
schema: qualtrics_source_integration_tests_02
threads: 8
databricks:
catalog: "{{ env_var('CI_DATABRICKS_DBT_CATALOG') }}"
host: "{{ env_var('CI_DATABRICKS_DBT_HOST') }}"
http_path: "{{ env_var('CI_DATABRICKS_DBT_HTTP_PATH') }}"
schema: qualtrics_source_integration_tests
schema: qualtrics_source_integration_tests_02
threads: 2
token: "{{ env_var('CI_DATABRICKS_DBT_TOKEN') }}"
type: databricks
11 changes: 8 additions & 3 deletions integration_tests/dbt_project.yml
Original file line number Diff line number Diff line change
@@ -1,13 +1,18 @@
name: 'qualtrics_source_integration_tests'
version: '0.1.0'
version: '0.1.1'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we also persist the docs in the integration tests to confirm that this works as expected across all other warehouses as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fivetran-joemarkiewicz Good idea! I have added, and it passed integrations tests. One caveat--I had to turn off the persist_docs: columns for databricks to get it to pass. This is not us since I'm seeing there are known issues on dbt's end with persist_docs for databricks.

profile: 'integration_tests'
config-version: 2

vars:
qualtrics_schema: qualtrics_source_integration_tests
qualtrics_schema: qualtrics_source_integration_tests_02
qualtrics__using_core_contacts: true
qualtrics__using_core_mailing_lists: true


models:
+persist_docs:
relation: true
columns: "{{ false if target.type in ['spark','databricks'] else true }}"

dispatch:
- macro_namespace: dbt_utils
search_order: ['spark_utils', 'dbt_utils']
Expand Down
34 changes: 17 additions & 17 deletions models/docs.md
Original file line number Diff line number Diff line change
Expand Up @@ -663,23 +663,23 @@ The time a survey was sent to the respondent.
{% enddocs %}

{% docs distribution_status %}
One of:
- `Pending` - The distribution is scheduled but has yet to be sent.
- `Success` - The distribution was successfully delivered to the contact.
- `Error` - An error occurred while attempting to send the distribution.
- `Opened` - The distribution was opened by the contact.
- `Complaint` - The contact complained that the distribution was spam.
- `Skipped` - The contact was skipped due to contact frequency rules or blacklisted contact information.
- `Blocked` - The distribution failed to send, because the contact blocked it or the email was caught by the spam circuit breaker.
- `Failure` - The distribution failed to be delivered.
- `Unknown` - The distribution failed for an unknown reason.
- `SoftBounce` - The distribution bounced but can be retried.
- `HardBounce` - The distribution bounced and should not be retried.
- `SurveyStarted` - The contact started the survey distributed.
- `SurveyPartiallyFinished` - The contact submitted a partially completed survey response.
- `SurveyFinished` - The contact submitted a completed survey response.
- `SurveyScreenedOut` - The contact screened out while taking the survey.
- `SessionExpired` - The contact's survey session has expired.
One of ([full descriptions](https://api.qualtrics.com/fc8017650d0b9-distribution-status)):
- `Pending`
- `Success`
- `Error`
- `Opened`
- `Complaint`
- `Skipped`
- `Blocked`
- `Failure`
- `Unknown`
- `SoftBounce`
- `HardBounce`
- `SurveyStarted`
- `SurveyPartiallyFinished`
- `SurveyFinished`
- `SurveyScreenedOut`
- `SessionExpired`
{% enddocs %}

{% docs survey_link %}
Expand Down