Skip to content

Commit

Permalink
Format yaml
Browse files Browse the repository at this point in the history
  • Loading branch information
PedramNavid committed Feb 28, 2024
1 parent 1788201 commit 6fe87b0
Show file tree
Hide file tree
Showing 6 changed files with 11 additions and 11 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,4 @@ streams:
sql: |
select all_user_id, name
from public."all_Users"
object: public.all_users # need to add 'object' key for custom SQL
object: public.all_users # need to add 'object' key for custom SQL
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,4 @@ streams:
sql: |
select all_user_id, name
from public."all_Users"
object: public.all_users # need to add 'object' key for custom SQL
object: public.all_users # need to add 'object' key for custom SQL
Original file line number Diff line number Diff line change
Expand Up @@ -34,4 +34,4 @@ streams:

env:
SLING_LOADED_AT_COLUMN: true # adds the _sling_loaded_at timestamp column
SLING_STREAM_URL_COLUMN: true # if source is file, adds a _sling_stream_url column with file path / url
SLING_STREAM_URL_COLUMN: true # if source is file, adds a _sling_stream_url column with file path / url
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
source: SLING_FILE
source: SLING_FILE
target: SLING_SQLITE

defaults:
mode: full-refresh
object: "{stream_table}"

streams: {}
streams: {}
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
This is some helper code for internally testing the integration with Sling.

Run `make run` to build and run a simple Postgres instance that is fed with some
sample data.
sample data.

Run the job in `sling_dag.py` with `dagster dev -f sling_dag.py` to see Dagster
load the assets from the replication file, and sync data from PG to DuckDB using
Sling.

You can interact with the duckdb instance which defaults to /var/tmp/duckdb.db

This folder is not currently used for automated testing.
This folder is not currently used for automated testing.
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,16 @@ source: MY_POSTGRES
target: MY_DUCKDB

defaults:
mode: full-refresh
mode: full-refresh

object: '{stream_schema}_{stream_table}'
object: "{stream_schema}_{stream_table}"

streams:
public.accounts:
public.users:
disabled: true
public.finance_departments_old:
object: 'departments' # overwrite default object
object: "departments" # overwrite default object
source_options:
empty_as_null: false
meta:
Expand All @@ -21,7 +21,7 @@ streams:
mode: incremental # overwrite default mode
primary_key: id
update_key: last_updated_at

public.all_users:
sql: |
select all_user_id, name
Expand Down

0 comments on commit 6fe87b0

Please sign in to comment.