Skip to content

Commit

Permalink
add dep and update dbt config (#27096)
Browse files Browse the repository at this point in the history
## Summary & Motivation
Minor updates to project dependencies and dbt configuration (number of
threads and cleaner source file.
## How I Tested These Changes

## Changelog

> Insert changelog entry or delete this section.
  • Loading branch information
C00ldudeNoonan authored Jan 14, 2025
1 parent 5e9b1e9 commit a32c7a7
Show file tree
Hide file tree
Showing 3 changed files with 6 additions and 7 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,11 @@ version: 2

sources:
- name: r2_bucket
meta:
external_location: "read_ndjson_objects('r2://dagster-demo/atproto_{name}/**/*.json', filename=true)"
tables:
- name: actor_feed_snapshot
description: "external r2 bucket with json files of actor feeds"
meta:
external_location: "read_ndjson_objects('r2://dagster-demo/atproto_actor_feed_snapshot/**/*.json', filename=true)"
- name: starter_pack_snapshot
description: "external r2 bucket with json files for feed snapshots"
meta:
external_location: "read_ndjson_objects('r2://dagster-demo/atproto_starter_pack_snapshot/**/*.json', filename=true)"

6 changes: 3 additions & 3 deletions examples/project_atproto_dashboard/dbt_project/profiles.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ bluesky:
type: duckdb
schema: bluesky_dev
path: "local.duckdb"
threads: 16
threads: 1
extensions:
- httpfs
settings:
Expand All @@ -16,8 +16,8 @@ bluesky:
prod:
type: duckdb
schema: bluesky
path: "md:prod_bluesky?MOTHERDUCK_TOKEN={{ env_var('MOTHERDUCK_TOKEN') }}"
threads: 16
path: "md:prod_bluesky"
threads: 1
extensions:
- httpfs
settings:
Expand Down
1 change: 1 addition & 0 deletions examples/project_atproto_dashboard/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ dependencies = [
"dagster-dbt",
"dagster-duckdb",
"dagster-powerbi",
"dagster-webserver",
"dbt-duckdb",
"tenacity",
]
Expand Down

0 comments on commit a32c7a7

Please sign in to comment.