Merge pull request #21 from maddoc1/main

Add support for query tags from profiles.yml and environment variables

.changes/unreleased/Features-20240114-205231.yaml

@@ -0,0 +1,6 @@
+kind: Features
+body: Support for query tags from profiles.yml and environment variables
+time: 2024-01-14T20:52:31.0834294+02:00
+custom:
+  Author: maddoc1
+  PR: "21"

README.md

@@ -104,9 +104,11 @@ To extend the information added in the query comments, use [meta](https://docs.g
 
 ### Query tags
 
-To extend the information added in the query tags, set the [query_tag](https://docs.getdbt.com/reference/resource-configs/snowflake-configs#query-tags) config value to a mapping type. Examples:
+To extend the information added in the query tags, there are a few options:
 
-#### Compatible config
+#### Model config
+
+Set the [query_tag](https://docs.getdbt.com/reference/resource-configs/snowflake-configs#query-tags) config value to a mapping type. Example:
 
 Model
 ```sql
@@ -117,16 +119,7 @@ Model
 select ...
 ```
 
-Results in a final query tag of
-```
-'{"team": "data", "app": "dbt", "dbt_snowflake_query_tags_version": "2.3.1", "is_incremental": true}'
-```
-
-the additional information is added by this package.
-
-#### Incompatible config
-
-Using a non-mapping type in the `query_tag` config will result in a warning, and the config being ignored.
+Note that using a non-mapping type in the `query_tag` config will result in a warning, and the config being ignored.
 
 Model
 ```sql
@@ -137,17 +130,43 @@ Model
 select ...
 ```
 
-Leads to a warning
+Warning:
 ```
 dbt-snowflake-query-tags warning: the query_tag config value of 'data team' is not a mapping type, so is being ignored. If you'd like to add additional query tag information, use a mapping type instead, or remove it to avoid this message.
 ```
 
-Results in a final query tag of
+This results in a final query tag without 'data team' being present.
+
+#### Profiles.yml
+
+Additionally, you can set the `query_tag` in the `profiles.yml`. This must be a valid json object.
+
+profiles.yml
+```yml
+default:
+  outputs:
+    dev:
+      query_tag: '{"team": "data"}'
+      ...
+  target: dev
 ```
-'{"app": "dbt", "dbt_snowflake_query_tags_version": "2.3.1", "is_incremental": false}'
+
+#### Environment variables
+
+Another option is to use the optional project variable `env_vars_to_query_tag_list` to provide a list of environment variables to pull query tag values from.
+
+Example:
+
+dbt_project.yml:
+```yml
+  vars:
+    env_vars_to_query_tag_list: ['TEAM','JOB_NAME']
 ```
 
-Note that the query_tag value of 'data team' is not present.
+Results in a final query tag of
+```
+'{"team": "data", "job_name": "daily", "app": "dbt", "dbt_snowflake_query_tags_version": "2.3.1", "is_incremental": true}'
+```
 
 ## Contributing
 

macros/query_tags.sql

@@ -3,6 +3,27 @@
 {%- endmacro %}
 
 {% macro default__set_query_tag() -%}
+    {# Get session level query tag #}
+    {% set session_query_tag = get_current_query_tag() %}
+    {% set session_query_tag_parsed = {} %}
+
+    {% if session_query_tag %}
+        {% if fromjson(session_query_tag) is mapping %}
+            {% set session_query_tag_parsed = fromjson(session_query_tag) %}
+        {% else %}
+            {% do log("dbt-snowflake-query-tags warning: the session level query tag value of '{}' is not a mapping type, so is being ignored. If you'd like to add additional query tag information, use a mapping type instead, or remove it to avoid this message.".format(session_query_tag), True) %}
+        {% endif %}
+    {% endif %}
+
+    {# The env_vars_to_query_tag_list should contain an environment variables list to construct query tag dict #}
+    {% set env_var_query_tags = {} %}
+    {% if var('env_vars_to_query_tag_list', '') %} {# Get a list of env vars from env_vars_to_query_tag_list variable to add additional query tags #}
+        {% for k in var('env_vars_to_query_tag_list') %}
+            {% set v = env_var(k, '') %}
+            {% do env_var_query_tags.update({k.lower(): v}) if v %}
+        {% endfor %}
+    {% endif %}
+
     {# Start with any model-configured dict #}
     {% set query_tag = config.get('query_tag', default={}) %}
 
@@ -11,6 +32,8 @@
     {% set query_tag = {} %} {# If the user has set the query tag config as a non mapping type, start fresh #}
     {% endif %}
 
+    {% do query_tag.update(session_query_tag_parsed) %}
+    {% do query_tag.update(env_var_query_tags) %}
 
     {%- do query_tag.update(
         app='dbt',