Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: Fix 'No relation found' error for models using alias #196

Merged
merged 1 commit into from
Jan 11, 2025

Conversation

civitaspo
Copy link
Contributor

@civitaspo civitaspo commented Jan 10, 2025

Hi maintainers, this is my first contribution, so if there are any guidelines or best practices I should follow, please let me know 😄

Since dbt-osmosis v1.x.x, the method for generating the CatalogKey has changed. In the current implementation, when adapter.get_relation is called for a dbt model that uses an alias, None is returned.

Until v0.14.0, if an alias was used, the alias would take priority, and that behavior was correct.
(ref. https://github.com/z3z1ma/dbt-osmosis/blob/v0.14.0/src/dbt_osmosis/core/osmosis.py#L324-L328)

So, I have modified it so that if it is neither a BaseRelation nor a NodeType.Source, the alias is used.


Test

Prepare Environment

Add dbt model files using alias
$ cat <<EOF > demo_duckdb/models/jaffle_shop/main/sample_alias.yml
---
version: 2
models:
  - name: sample_alias
    columns:
      - name: order_id
        data_type: INTEGER

EOF

$ cat <<EOF > demo_duckdb/models/sample_alias.sql
{{ config(
    alias="a_sample",
) }}

with final as (

    select order_id from {{ ref('stg_orders') }}

)

select * from final

EOF
Build the deme_duckdb project
$ uv run dbt build --project-dir demo_duckdb --profiles-dir demo_duckdb
00:04:52  Running with dbt=1.9.1
00:04:52  Registered adapter: duckdb=1.9.1
00:04:52  Found 7 models, 3 seeds, 20 data tests, 424 macros
00:04:52  
00:04:52  Concurrency: 4 threads (target='dev')
00:04:52  
00:04:52  1 of 30 START seed file main.raw_customers ..................................... [RUN]
00:04:52  2 of 30 START seed file main.raw_orders ........................................ [RUN]
00:04:52  3 of 30 START seed file main.raw_payments ...................................... [RUN]
00:04:52  2 of 30 OK loaded seed file main.raw_orders .................................... [INSERT 99 in 0.07s]
00:04:52  3 of 30 OK loaded seed file main.raw_payments .................................. [INSERT 113 in 0.07s]
00:04:52  4 of 30 START sql view model main.stg_orders ................................... [RUN]
00:04:52  5 of 30 START sql view model main.stg_payments ................................. [RUN]
00:04:52  1 of 30 OK loaded seed file main.raw_customers ................................. [INSERT 100 in 0.08s]
00:04:52  6 of 30 START sql view model main.stg_customers ................................ [RUN]
00:04:52  4 of 30 OK created sql view model main.stg_orders .............................. [OK in 0.05s]
00:04:52  6 of 30 OK created sql view model main.stg_customers ........................... [OK in 0.03s]
00:04:52  7 of 30 START test accepted_values_stg_orders_status__placed__shipped__completed__return_pending__returned  [RUN]
00:04:52  8 of 30 START test not_null_stg_orders_order_id ................................ [RUN]
00:04:52  9 of 30 START test unique_stg_orders_order_id .................................. [RUN]
00:04:52  5 of 30 OK created sql view model main.stg_payments ............................ [OK in 0.06s]
00:04:52  10 of 30 START test not_null_stg_customers_customer_id ......................... [RUN]
00:04:52  10 of 30 PASS not_null_stg_customers_customer_id ............................... [PASS in 0.03s]
00:04:52  8 of 30 PASS not_null_stg_orders_order_id ...................................... [PASS in 0.05s]
00:04:52  7 of 30 PASS accepted_values_stg_orders_status__placed__shipped__completed__return_pending__returned  [PASS in 0.05s]
00:04:52  9 of 30 PASS unique_stg_orders_order_id ........................................ [PASS in 0.05s]
00:04:52  11 of 30 START test unique_stg_customers_customer_id ........................... [RUN]
00:04:52  12 of 30 START test accepted_values_stg_payments_payment_method__credit_card__coupon__bank_transfer__gift_card  [RUN]
00:04:52  13 of 30 START test not_null_stg_payments_payment_id ........................... [RUN]
00:04:52  14 of 30 START test unique_stg_payments_payment_id ............................. [RUN]
00:04:52  13 of 30 PASS not_null_stg_payments_payment_id ................................. [PASS in 0.03s]
00:04:52  14 of 30 PASS unique_stg_payments_payment_id ................................... [PASS in 0.02s]
00:04:52  11 of 30 PASS unique_stg_customers_customer_id ................................. [PASS in 0.03s]
00:04:52  12 of 30 PASS accepted_values_stg_payments_payment_method__credit_card__coupon__bank_transfer__gift_card  [PASS in 0.03s]
00:04:52  15 of 30 START sql table model main.a_sample ................................... [RUN]
00:04:52  16 of 30 START sql table model main.customers .................................. [RUN]
00:04:52  17 of 30 START sql table model main.orders ..................................... [RUN]
00:04:52  18 of 30 START sql table model main.orders_prefix .............................. [RUN]
00:04:52  15 of 30 OK created sql table model main.a_sample .............................. [OK in 0.05s]
00:04:53  17 of 30 OK created sql table model main.orders ................................ [OK in 0.05s]
00:04:53  18 of 30 OK created sql table model main.orders_prefix ......................... [OK in 0.06s]
00:04:53  19 of 30 START test accepted_values_orders_status__placed__shipped__completed__return_pending__returned  [RUN]
00:04:53  20 of 30 START test not_null_orders_amount ..................................... [RUN]
00:04:53  21 of 30 START test not_null_orders_bank_transfer_amount ....................... [RUN]
00:04:53  16 of 30 OK created sql table model main.customers ............................. [OK in 0.07s]
00:04:53  22 of 30 START test not_null_orders_coupon_amount .............................. [RUN]
00:04:53  20 of 30 PASS not_null_orders_amount ........................................... [PASS in 0.03s]
00:04:53  21 of 30 PASS not_null_orders_bank_transfer_amount ............................. [PASS in 0.03s]
00:04:53  22 of 30 PASS not_null_orders_coupon_amount .................................... [PASS in 0.01s]
00:04:53  19 of 30 PASS accepted_values_orders_status__placed__shipped__completed__return_pending__returned  [PASS in 0.03s]
00:04:53  23 of 30 START test not_null_orders_credit_card_amount ......................... [RUN]
00:04:53  24 of 30 START test not_null_orders_customer_id ................................ [RUN]
00:04:53  25 of 30 START test not_null_orders_gift_card_amount ........................... [RUN]
00:04:53  26 of 30 START test not_null_orders_order_id ................................... [RUN]
00:04:53  25 of 30 PASS not_null_orders_gift_card_amount ................................. [PASS in 0.03s]
00:04:53  26 of 30 PASS not_null_orders_order_id ......................................... [PASS in 0.03s]
00:04:53  24 of 30 PASS not_null_orders_customer_id ...................................... [PASS in 0.03s]
00:04:53  23 of 30 PASS not_null_orders_credit_card_amount ............................... [PASS in 0.03s]
00:04:53  27 of 30 START test unique_orders_order_id ..................................... [RUN]
00:04:53  28 of 30 START test not_null_customers_customer_id ............................. [RUN]
00:04:53  29 of 30 START test relationships_orders_customer_id__customer_id__ref_customers_  [RUN]
00:04:53  30 of 30 START test unique_customers_customer_id ............................... [RUN]
00:04:53  29 of 30 PASS relationships_orders_customer_id__customer_id__ref_customers_ .... [PASS in 0.02s]
00:04:53  27 of 30 PASS unique_orders_order_id ........................................... [PASS in 0.03s]
00:04:53  28 of 30 PASS not_null_customers_customer_id ................................... [PASS in 0.03s]
00:04:53  30 of 30 PASS unique_customers_customer_id ..................................... [PASS in 0.03s]
00:04:53  
00:04:53  Finished running 3 seeds, 4 table models, 20 data tests, 3 view models in 0 hours 0 minutes and 0.49 seconds (0.49s).
00:04:53  
00:04:53  Completed successfully
00:04:53  
00:04:53  Done. PASS=30 WARN=0 ERROR=0 SKIP=0 TOTAL=30

Before fixing

WARNING ⚠ No relation found => CatalogKey(database='jaffle_shop', schema='main', name='sample_alias') is shown.

$ uv run dbt-osmosis yaml refactor --project-dir ./demo_duckdb --profiles-dir ./demo_duckdb
INFO     🌊 Executing dbt-osmosis                                                                               main.py:243
                                                                                                                           
INFO     👋 Creating DBT project context using config => DbtConfiguration(project_dir='./demo_duckdb',       osmosis.py:266
         profiles_dir='./demo_duckdb', target=None, profile=None, threads=None, single_threaded=None,                      
         vars={}, quiet=True, disable_introspection=False)                                                                 
INFO     📑 Registering adapter as part of project context creation.                                         osmosis.py:271
00:07:49  Registered adapter: duckdb=1.9.1
INFO     🔄 Loaded the dbt project manifest!                                                                 osmosis.py:280
INFO     ✨ DbtProjectContext successfully created!                                                          osmosis.py:290
INFO     📓 Osmosis ThreadPoolExecutor max_workers synced with dbt => 4                                      osmosis.py:531
INFO     🏭 Creating missing source YAMLs (if any).                                                         osmosis.py:1008
INFO     💡 Drafting restructure delta plan for the project.                                                osmosis.py:1306
INFO     🌐 Building YAML file mapping. create_missing_sources => False                                     osmosis.py:1149
INFO     🌟 Draft plan creation complete => 0 operations                                                    osmosis.py:1325
INFO     ✅ No changes needed in the restructure plan.                                                      osmosis.py:1550
INFO                                                                                                        osmosis.py:1864
         ⚙ Running pipeline with => 5 operations ['Inject Missing Columns', 'Remove Extra Columns',                        
         'Inherit Upstream Column Knowledge', 'Sort Columns', 'Synchronize Data Types']                                    
                                                                                                                           
INFO     ⚙  Starting to Inject Missing Columns                                                              osmosis.py:1872
INFO     👋 Injecting missing columns for all matched nodes.                                                osmosis.py:1990
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main',                osmosis.py:855
         name='raw_customers')                                                                                             
INFO     🔧 Instantiating new adapter because none is currently set.                                         osmosis.py:227
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main',                osmosis.py:855
         name='raw_payments')                                                                                              
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main',                osmosis.py:855
         name='raw_orders')                                                                                                
INFO     🔧 Successfully acquired new adapter connection for thread => 6140260352                            osmosis.py:233
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main',                osmosis.py:855
         name='stg_customers')                                                                                             
INFO     🔧 Refreshing db connection for thread => 6157086720                                                osmosis.py:238
INFO     🔧 Refreshing db connection for thread => 6173913088                                                osmosis.py:238
INFO     🔧 Refreshing db connection for thread => 6190739456                                                osmosis.py:238
INFO     🔍 Introspecting columns in warehouse for => "jaffle_shop"."main"."raw_orders"                      osmosis.py:901
INFO     🔍 Introspecting columns in warehouse for => "jaffle_shop"."main"."stg_customers"                   osmosis.py:901
INFO     🔍 Introspecting columns in warehouse for => "jaffle_shop"."main"."raw_payments"                    osmosis.py:901
INFO     🔍 Introspecting columns in warehouse for => "jaffle_shop"."main"."raw_customers"                   osmosis.py:901
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main',                osmosis.py:855
         name='stg_payments')                                                                                              
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main',                osmosis.py:855
         name='stg_orders')                                                                                                
INFO     🔍 Introspecting columns in warehouse for => "jaffle_shop"."main"."stg_payments"                    osmosis.py:901
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main',                osmosis.py:855
         name='sample_alias')                                                                                              
INFO     🔍 Introspecting columns in warehouse for => "jaffle_shop"."main"."stg_orders"                      osmosis.py:901
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main',                osmosis.py:855
         name='orders_prefix')                                                                                             
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main',                osmosis.py:855
         name='customers')                                                                                                 
INFO     🔍 Introspecting columns in warehouse for => "jaffle_shop"."main"."orders_prefix"                   osmosis.py:901
WARNING  ⚠ No relation found => CatalogKey(database='jaffle_shop', schema='main', name='sample_alias')       osmosis.py:897
INFO     🔍 Introspecting columns in warehouse for => "jaffle_shop"."main"."customers"                       osmosis.py:901
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main', name='orders') osmosis.py:855
INFO     🔍 Introspecting columns in warehouse for => "jaffle_shop"."main"."orders"                          osmosis.py:901
INFO     ✨ Done with Inject Missing Columns in 0.17s                                                       osmosis.py:1879
                                                                                                                           
INFO     ⚙  Starting to Remove Extra Columns                                                                osmosis.py:1872
INFO     👋 Removing columns not in DB across all matched nodes.                                            osmosis.py:2031
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main',                osmosis.py:855
         name='sample_alias')                                                                                              
WARNING  ⚠ No relation found => CatalogKey(database='jaffle_shop', schema='main', name='sample_alias')       osmosis.py:897
INFO     🚫 No columns discovered for node => model.jaffle_shop_duckdb.sample_alias, skipping cleanup.      osmosis.py:2044
INFO     ✨ Done with Remove Extra Columns in 0.00s                                                         osmosis.py:1879
                                                                                                                           
INFO     ⚙  Starting to Inherit Upstream Column Knowledge                                                   osmosis.py:1872
INFO     👋 Inheriting column knowledge across all matched nodes.                                           osmosis.py:1942
INFO     🧬 Inheriting column knowledge for => seed.jaffle_shop_duckdb.raw_customers                        osmosis.py:1950
INFO     🧬 Inheriting column knowledge for => seed.jaffle_shop_duckdb.raw_payments                         osmosis.py:1950
INFO     🧬 Inheriting column knowledge for => seed.jaffle_shop_duckdb.raw_orders                           osmosis.py:1950
INFO     🧬 Inheriting column knowledge for => model.jaffle_shop_duckdb.stg_customers                       osmosis.py:1950
INFO     🧬 Inheriting column knowledge for => model.jaffle_shop_duckdb.stg_payments                        osmosis.py:1950
INFO     🧬 Inheriting column knowledge for => model.jaffle_shop_duckdb.stg_orders                          osmosis.py:1950
INFO     🧬 Inheriting column knowledge for => model.jaffle_shop_duckdb.sample_alias                        osmosis.py:1950
INFO     🧬 Inheriting column knowledge for => model.jaffle_shop_duckdb.orders_prefix                       osmosis.py:1950
INFO     🧬 Inheriting column knowledge for => model.jaffle_shop_duckdb.customers                           osmosis.py:1950
INFO     🧬 Inheriting column knowledge for => model.jaffle_shop_duckdb.orders                              osmosis.py:1950
INFO     ✨ Done with Inherit Upstream Column Knowledge in 0.01s                                            osmosis.py:1879
                                                                                                                           
INFO     ⚙  Starting to Sort Columns                                                                        osmosis.py:1872
INFO     👋 Sorting columns as configured across all matched nodes.                                         osmosis.py:2114
INFO     🔢 Sorting columns by warehouse order => seed.jaffle_shop_duckdb.raw_customers                     osmosis.py:2072
INFO     🔢 Sorting columns by warehouse order => seed.jaffle_shop_duckdb.raw_payments                      osmosis.py:2072
INFO     🔢 Sorting columns by warehouse order => seed.jaffle_shop_duckdb.raw_orders                        osmosis.py:2072
INFO     🔢 Sorting columns by warehouse order => model.jaffle_shop_duckdb.stg_customers                    osmosis.py:2072
INFO     🔢 Sorting columns by warehouse order => model.jaffle_shop_duckdb.stg_payments                     osmosis.py:2072
INFO     🔢 Sorting columns by warehouse order => model.jaffle_shop_duckdb.stg_orders                       osmosis.py:2072
INFO     🔢 Sorting columns by warehouse order => model.jaffle_shop_duckdb.sample_alias                     osmosis.py:2072
INFO     🔢 Sorting columns by warehouse order => model.jaffle_shop_duckdb.orders_prefix                    osmosis.py:2072
INFO     🔢 Sorting columns by warehouse order => model.jaffle_shop_duckdb.customers                        osmosis.py:2072
INFO     🔢 Sorting columns by warehouse order => model.jaffle_shop_duckdb.orders                           osmosis.py:2072
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main',                osmosis.py:855
         name='sample_alias')                                                                                              
WARNING  ⚠ No relation found => CatalogKey(database='jaffle_shop', schema='main', name='sample_alias')       osmosis.py:897
INFO     🚫 No columns discovered for node => model.jaffle_shop_duckdb.sample_alias, skipping db order      osmosis.py:2075
         sorting.                                                                                                          
INFO     ✨ Done with Sort Columns in 0.01s                                                                 osmosis.py:1879
                                                                                                                           
INFO     ⚙  Starting to Synchronize Data Types                                                              osmosis.py:1872
INFO     👋 Populating data types across all matched nodes.                                                 osmosis.py:2134
INFO     🔢 Synchronizing data types => seed.jaffle_shop_duckdb.raw_customers                               osmosis.py:2140
INFO     🔢 Synchronizing data types => seed.jaffle_shop_duckdb.raw_payments                                osmosis.py:2140
INFO     🔢 Synchronizing data types => seed.jaffle_shop_duckdb.raw_orders                                  osmosis.py:2140
INFO     🔢 Synchronizing data types => model.jaffle_shop_duckdb.stg_customers                              osmosis.py:2140
INFO     🔢 Synchronizing data types => model.jaffle_shop_duckdb.stg_payments                               osmosis.py:2140
INFO     🔢 Synchronizing data types => model.jaffle_shop_duckdb.stg_orders                                 osmosis.py:2140
INFO     🔢 Synchronizing data types => model.jaffle_shop_duckdb.sample_alias                               osmosis.py:2140
INFO     🔢 Synchronizing data types => model.jaffle_shop_duckdb.orders_prefix                              osmosis.py:2140
INFO     🔢 Synchronizing data types => model.jaffle_shop_duckdb.customers                                  osmosis.py:2140
INFO     🔢 Synchronizing data types => model.jaffle_shop_duckdb.orders                                     osmosis.py:2140
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main',                osmosis.py:855
         name='sample_alias')                                                                                              
WARNING  ⚠ No relation found => CatalogKey(database='jaffle_shop', schema='main', name='sample_alias')       osmosis.py:897
INFO     ✨ Done with Synchronize Data Types in 0.01s                                                       osmosis.py:1879
                                                                                                                           
INFO     🏁 Manifest transformation pipeline completed in => 0.20s                                          osmosis.py:1897
INFO     ⌛ Committing all changes to YAML files in batch.                                                  osmosis.py:1903
INFO     👋 No single node specified; synchronizing all matched nodes.                                      osmosis.py:1461
INFO     📥 Committing YAML doc changes for => seed.jaffle_shop_duckdb.raw_customers                        osmosis.py:1541
INFO     📥 Committing YAML doc changes for => seed.jaffle_shop_duckdb.raw_payments                         osmosis.py:1541
INFO     📥 Committing YAML doc changes for => seed.jaffle_shop_duckdb.raw_orders                           osmosis.py:1541
INFO     📥 Committing YAML doc changes for => model.jaffle_shop_duckdb.stg_customers                       osmosis.py:1541
INFO     📥 Committing YAML doc changes for => model.jaffle_shop_duckdb.stg_payments                        osmosis.py:1541
INFO     📥 Committing YAML doc changes for => model.jaffle_shop_duckdb.stg_orders                          osmosis.py:1541
INFO     📥 Committing YAML doc changes for => model.jaffle_shop_duckdb.sample_alias                        osmosis.py:1541
INFO     📥 Committing YAML doc changes for => model.jaffle_shop_duckdb.orders_prefix                       osmosis.py:1541
INFO     📥 Committing YAML doc changes for => model.jaffle_shop_duckdb.customers                           osmosis.py:1541
INFO     📥 Committing YAML doc changes for => model.jaffle_shop_duckdb.orders                              osmosis.py:1541
INFO     🏁 YAML commits completed in => 0.04s                                                              osmosis.py:1907

After fixing

$ uv run dbt-osmosis yaml refactor --project-dir ./demo_duckdb --profiles-dir ./demo_duckdb
INFO     🌊 Executing dbt-osmosis                                                                               main.py:243
                                                                                                                           
INFO     👋 Creating DBT project context using config => DbtConfiguration(project_dir='./demo_duckdb',       osmosis.py:266
         profiles_dir='./demo_duckdb', target=None, profile=None, threads=None, single_threaded=None,                      
         vars={}, quiet=True, disable_introspection=False)                                                                 
INFO     📑 Registering adapter as part of project context creation.                                         osmosis.py:271
00:08:36  Registered adapter: duckdb=1.9.1
INFO     🔄 Loaded the dbt project manifest!                                                                 osmosis.py:280
INFO     ✨ DbtProjectContext successfully created!                                                          osmosis.py:290
INFO     📓 Osmosis ThreadPoolExecutor max_workers synced with dbt => 4                                      osmosis.py:531
INFO     🏭 Creating missing source YAMLs (if any).                                                         osmosis.py:1008
INFO     💡 Drafting restructure delta plan for the project.                                                osmosis.py:1306
INFO     🌐 Building YAML file mapping. create_missing_sources => False                                     osmosis.py:1149
INFO     🌟 Draft plan creation complete => 0 operations                                                    osmosis.py:1325
INFO     ✅ No changes needed in the restructure plan.                                                      osmosis.py:1550
INFO                                                                                                        osmosis.py:1864
         ⚙ Running pipeline with => 5 operations ['Inject Missing Columns', 'Remove Extra Columns',                        
         'Inherit Upstream Column Knowledge', 'Sort Columns', 'Synchronize Data Types']                                    
                                                                                                                           
INFO     ⚙  Starting to Inject Missing Columns                                                              osmosis.py:1872
INFO     👋 Injecting missing columns for all matched nodes.                                                osmosis.py:1990
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main',                osmosis.py:855
         name='raw_customers')                                                                                             
INFO     🔧 Instantiating new adapter because none is currently set.                                         osmosis.py:227
INFO     🔧 Successfully acquired new adapter connection for thread => 6118076416                            osmosis.py:233
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main',                osmosis.py:855
         name='raw_payments')                                                                                              
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main',                osmosis.py:855
         name='raw_orders')                                                                                                
INFO     🔧 Refreshing db connection for thread => 6134902784                                                osmosis.py:238
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main',                osmosis.py:855
         name='stg_customers')                                                                                             
INFO     🔧 Refreshing db connection for thread => 6151729152                                                osmosis.py:238
INFO     🔧 Refreshing db connection for thread => 6168555520                                                osmosis.py:238
INFO     🔍 Introspecting columns in warehouse for => "jaffle_shop"."main"."raw_orders"                      osmosis.py:901
INFO     🔍 Introspecting columns in warehouse for => "jaffle_shop"."main"."stg_customers"                   osmosis.py:901
INFO     🔍 Introspecting columns in warehouse for => "jaffle_shop"."main"."raw_customers"                   osmosis.py:901
INFO     🔍 Introspecting columns in warehouse for => "jaffle_shop"."main"."raw_payments"                    osmosis.py:901
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main',                osmosis.py:855
         name='stg_payments')                                                                                              
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main',                osmosis.py:855
         name='stg_orders')                                                                                                
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main',                osmosis.py:855
         name='orders_prefix')                                                                                             
INFO     🔍 Introspecting columns in warehouse for => "jaffle_shop"."main"."stg_payments"                    osmosis.py:901
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main',                osmosis.py:855
         name='customers')                                                                                                 
INFO     🔍 Introspecting columns in warehouse for => "jaffle_shop"."main"."stg_orders"                      osmosis.py:901
INFO     🔍 Introspecting columns in warehouse for => "jaffle_shop"."main"."orders_prefix"                   osmosis.py:901
INFO     🔍 Introspecting columns in warehouse for => "jaffle_shop"."main"."customers"                       osmosis.py:901
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main',                osmosis.py:855
         name='a_sample')                                                                                                  
INFO     🔍 Introspecting columns in warehouse for => "jaffle_shop"."main"."a_sample"                        osmosis.py:901
INFO     🔎 Collecting columns for table => CatalogKey(database='jaffle_shop', schema='main', name='orders') osmosis.py:855
INFO     🔍 Introspecting columns in warehouse for => "jaffle_shop"."main"."orders"                          osmosis.py:901
INFO     ✨ Done with Inject Missing Columns in 0.14s                                                       osmosis.py:1879
                                                                                                                           
INFO     ⚙  Starting to Remove Extra Columns                                                                osmosis.py:1872
INFO     👋 Removing columns not in DB across all matched nodes.                                            osmosis.py:2031
INFO     ✨ Done with Remove Extra Columns in 0.00s                                                         osmosis.py:1879
                                                                                                                           
INFO     ⚙  Starting to Inherit Upstream Column Knowledge                                                   osmosis.py:1872
INFO     👋 Inheriting column knowledge across all matched nodes.                                           osmosis.py:1942
INFO     🧬 Inheriting column knowledge for => seed.jaffle_shop_duckdb.raw_customers                        osmosis.py:1950
INFO     🧬 Inheriting column knowledge for => seed.jaffle_shop_duckdb.raw_payments                         osmosis.py:1950
INFO     🧬 Inheriting column knowledge for => seed.jaffle_shop_duckdb.raw_orders                           osmosis.py:1950
INFO     🧬 Inheriting column knowledge for => model.jaffle_shop_duckdb.stg_customers                       osmosis.py:1950
INFO     🧬 Inheriting column knowledge for => model.jaffle_shop_duckdb.stg_payments                        osmosis.py:1950
INFO     🧬 Inheriting column knowledge for => model.jaffle_shop_duckdb.stg_orders                          osmosis.py:1950
INFO     🧬 Inheriting column knowledge for => model.jaffle_shop_duckdb.orders_prefix                       osmosis.py:1950
INFO     🧬 Inheriting column knowledge for => model.jaffle_shop_duckdb.customers                           osmosis.py:1950
INFO     🧬 Inheriting column knowledge for => model.jaffle_shop_duckdb.sample_alias                        osmosis.py:1950
INFO     🧬 Inheriting column knowledge for => model.jaffle_shop_duckdb.orders                              osmosis.py:1950
INFO     ✨ Done with Inherit Upstream Column Knowledge in 0.05s                                            osmosis.py:1879
                                                                                                                           
INFO     ⚙  Starting to Sort Columns                                                                        osmosis.py:1872
INFO     👋 Sorting columns as configured across all matched nodes.                                         osmosis.py:2114
INFO     🔢 Sorting columns by warehouse order => seed.jaffle_shop_duckdb.raw_customers                     osmosis.py:2072
INFO     🔢 Sorting columns by warehouse order => seed.jaffle_shop_duckdb.raw_payments                      osmosis.py:2072
INFO     🔢 Sorting columns by warehouse order => seed.jaffle_shop_duckdb.raw_orders                        osmosis.py:2072
INFO     🔢 Sorting columns by warehouse order => model.jaffle_shop_duckdb.stg_orders                       osmosis.py:2072
INFO     🔢 Sorting columns by warehouse order => model.jaffle_shop_duckdb.stg_payments                     osmosis.py:2072
INFO     🔢 Sorting columns by warehouse order => model.jaffle_shop_duckdb.orders_prefix                    osmosis.py:2072
INFO     🔢 Sorting columns by warehouse order => model.jaffle_shop_duckdb.stg_customers                    osmosis.py:2072
INFO     🔢 Sorting columns by warehouse order => model.jaffle_shop_duckdb.customers                        osmosis.py:2072
INFO     🔢 Sorting columns by warehouse order => model.jaffle_shop_duckdb.sample_alias                     osmosis.py:2072
INFO     🔢 Sorting columns by warehouse order => model.jaffle_shop_duckdb.orders                           osmosis.py:2072
INFO     ✨ Done with Sort Columns in 0.01s                                                                 osmosis.py:1879
                                                                                                                           
INFO     ⚙  Starting to Synchronize Data Types                                                              osmosis.py:1872
INFO     👋 Populating data types across all matched nodes.                                                 osmosis.py:2134
INFO     🔢 Synchronizing data types => seed.jaffle_shop_duckdb.raw_customers                               osmosis.py:2140
INFO     🔢 Synchronizing data types => seed.jaffle_shop_duckdb.raw_payments                                osmosis.py:2140
INFO     🔢 Synchronizing data types => seed.jaffle_shop_duckdb.raw_orders                                  osmosis.py:2140
INFO     🔢 Synchronizing data types => model.jaffle_shop_duckdb.stg_customers                              osmosis.py:2140
INFO     🔢 Synchronizing data types => model.jaffle_shop_duckdb.stg_payments                               osmosis.py:2140
INFO     🔢 Synchronizing data types => model.jaffle_shop_duckdb.stg_orders                                 osmosis.py:2140
INFO     🔢 Synchronizing data types => model.jaffle_shop_duckdb.orders_prefix                              osmosis.py:2140
INFO     🔢 Synchronizing data types => model.jaffle_shop_duckdb.customers                                  osmosis.py:2140
INFO     🔢 Synchronizing data types => model.jaffle_shop_duckdb.sample_alias                               osmosis.py:2140
INFO     🔢 Synchronizing data types => model.jaffle_shop_duckdb.orders                                     osmosis.py:2140
INFO     ✨ Done with Synchronize Data Types in 0.00s                                                       osmosis.py:1879
                                                                                                                           
INFO     🏁 Manifest transformation pipeline completed in => 0.20s                                          osmosis.py:1897
INFO     ⌛ Committing all changes to YAML files in batch.                                                  osmosis.py:1903
INFO     👋 No single node specified; synchronizing all matched nodes.                                      osmosis.py:1461
INFO     📥 Committing YAML doc changes for => seed.jaffle_shop_duckdb.raw_payments                         osmosis.py:1541
INFO     📥 Committing YAML doc changes for => seed.jaffle_shop_duckdb.raw_customers                        osmosis.py:1541
INFO     📥 Committing YAML doc changes for => seed.jaffle_shop_duckdb.raw_orders                           osmosis.py:1541
INFO     📥 Committing YAML doc changes for => model.jaffle_shop_duckdb.stg_customers                       osmosis.py:1541
INFO     📥 Committing YAML doc changes for => model.jaffle_shop_duckdb.stg_payments                        osmosis.py:1541
INFO     📥 Committing YAML doc changes for => model.jaffle_shop_duckdb.stg_orders                          osmosis.py:1541
INFO     📥 Committing YAML doc changes for => model.jaffle_shop_duckdb.orders_prefix                       osmosis.py:1541
INFO     📥 Committing YAML doc changes for => model.jaffle_shop_duckdb.customers                           osmosis.py:1541
INFO     📥 Committing YAML doc changes for => model.jaffle_shop_duckdb.sample_alias                        osmosis.py:1541
INFO     📥 Committing YAML doc changes for => model.jaffle_shop_duckdb.orders                              osmosis.py:1541
INFO     🏁 YAML commits completed in => 0.03s                                                              osmosis.py:1907

@civitaspo civitaspo marked this pull request as ready for review January 10, 2025 23:31
@z3z1ma
Copy link
Owner

z3z1ma commented Jan 11, 2025

Thanks for the detailed write up and the catch here! This LGTM, lets merge it and get a patch release out tomorrow.

@z3z1ma z3z1ma merged commit 21f5264 into z3z1ma:main Jan 11, 2025
7 checks passed
@civitaspo
Copy link
Contributor Author

thank you so much for releasing this!!
https://github.com/z3z1ma/dbt-osmosis/releases/tag/v1.1.9

@civitaspo civitaspo deleted the fix-CatalogKey-generation-for-alias branch January 17, 2025 10:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants