snowflakedb · sfc-gh-osalazarlizano · Jun 12, 2024 · Jun 12, 2024 · Jun 19, 2024 · Jun 19, 2024
diff --git a/snowflake-cortex/.gitignore b/snowflake-cortex/.gitignore
@@ -0,0 +1,4 @@
+snowflake.local.yml
+output/
+**/__pycache__/
+**/.pytest_cache/
diff --git a/snowflake-cortex/README.md b/snowflake-cortex/README.md
@@ -0,0 +1,95 @@
+## Introduction
+
+This is an example template for a Snowflake Native App project which demonstrates the use of Python extension code and adding Streamlit code. This template is meant to guide developers towards a possible project structure on the basis of functionality, as well as to indicate the contents of some common and useful files. 
+
+Since this template contains Python files only, you do not need to perform any additional steps to build the source code. You can directly go to the next section. However, if there were any source code that needed to be built, you must manually perform the build steps here before proceeding to the next section. 
+
+Similarly, you can also use your own build steps for any other languages supported by Snowflake that you wish to write your code in. For more information on supported languages, visit [docs](https://docs.snowflake.com/en/developer-guide/stored-procedures-vs-udfs#label-sp-udf-languages).
+
+## Run the app
+Create or update an application package in your Snowflake account, upload application artifacts to a stage in the application package, and create or update an application object in the same account based on the uploaded artifacts.
+```
+snow app run
+```
+
+For more information, please refer to the Snowflake Documentation on installing and using Snowflake CLI to create a Snowflake Native App.  
+# Directory Structure
+## `/app`
+This directory holds your Snowflake Native App files.
+
+### `/app/README.md`
+Exposed to the account installing the application with details on what it does and how to use it.
+
+### `/app/manifest.yml`
+Defines properties required by the application package. Find more details at the [Manifest Documentation.](https://docs.snowflake.com/en/developer-guide/native-apps/creating-manifest)
+
+### `/app/setup_script.sql`
+Contains SQL statements that are run when a consumer installs or upgrades a Snowflake Native App in their account.
+
+## `/scripts`
+You can add any additional scripts such as `.sql` and `.jinja` files here. One common use case for such a script is to add shared content from external databases to your application package. This allows you to refer to the external database in the setup script that runs when a Snowflake Native App is installed.
+_Note: As of now, `snow app init` does not render these jinja templates for you into the required files, if you decide to use them. You will have to manually render them for now._
+
+
+## `/src`
+This directory contains code organization by functionality, such as one distinct module for Streamlit related code, and another module for "number add" functionality, which is used an example in this template. 
+```
+/src
+   |-module-add
+   |          |-main
+   |          |    |-python
+   |          |           |-add.py
+   |          |
+   |          |-test
+   |               |-python
+   |                      |-add_test.py
+   |
+   |-module-ui
+   |         |-src
+   |             |-ui.py
+   |             |-environment.yml
+   |         |-test
+   |              |-test_ui.py
+```
+
+## `snowflake.yml.jinja`
+While this file exists as a Jinja template, it is the only file that is automatically rendered as a `snowflake.yml` file by the `snow app init` command, as described in the [README.md](../README.md). Snowflake CLI uses the `snowflake.yml` file  to discover your project's code and interact with Snowflake using all relevant privileges and grants. 
+
+For more information, please refer to the Snowflake Documentation on installing and using Snowflake CLI to create a Snowflake Native App. 
+
+## Adding a snowflake.local.yml file
+Although your project directory must have a `snowflake.yml` file, an individual developer can choose to customize the behavior of Snowflake CLI by providing local overrides to the `snowflake.yml` file, such as a new role to test out your own application package. This is where you can use the `snowflake.local.yml` file, which is not a version-controlled file.
+
+For more information, please refer to the Snowflake Documentation on installing and using Snowflake CLI to create a Snowflake Native App. 
+
+## Unit tests
+To set up and run unit tests, please follow the steps below.
+
+### Set up testing conda environment (First Time setup)
+
+Go to the project's root directory where you can find `local_test_env.yml` and run the following command once to set up a conda environment with the correct packages. Please note that the version of test packages may differ from the version of packages in Snowflake, so you will need to be careful with any differences in behavior.
+
+```
+conda env create --file local_test_env.yml
+```
+
+This will create a conda environment with the name `streamlit-python-testing`.
+
+### Run unit tests
+To run unit tests, follow these steps:
+
+#### Activate conda environment
+You will need to activate this conda environment once per command line session:
+```
+conda activate streamlit-python-testing
+```
+To deactivate and use your current command line session for other tasks, run the following:
+```
+conda deactivate
+```
+#### Run Pytest
+To run the example tests provided, execute the following command from the project's root:
+```
+pytest
+```
+Note that there is a pytest.ini file specifying the location of the source code that we are testing.
diff --git a/snowflake-cortex/app/README.md b/snowflake-cortex/app/README.md
@@ -0,0 +1,3 @@
+# Snowflake Native App - Snowflake Cortex
+
+This is a sample Snowflake Native App that shows the use of Snowflake Cortex inside Native Apps.
diff --git a/snowflake-cortex/app/manifest.yml b/snowflake-cortex/app/manifest.yml
@@ -0,0 +1,17 @@
+# For more information on creating manifest, go to https://docs.snowflake.com/en/developer-guide/native-apps/creating-manifest
+manifest_version: 1
+
+version:
+  name: Dev
+  label: "Dev Version"
+  comment: "Default version used for development. Override for actual deployment."
+
+artifacts:
+  setup_script: setup_script.sql
+  readme: README.md
+  default_streamlit: core.ui
+  extension_code: true 
+
+privileges:
+  - IMPORTED PRIVILEGES ON SNOWFLAKE DB:
+      description: "Imported privileges to use cortex DB"
diff --git a/snowflake-cortex/app/setup_script.sql b/snowflake-cortex/app/setup_script.sql
@@ -0,0 +1,27 @@
+-- This is the setup script that runs while installing a Snowflake Native App in a consumer account.
+-- For more information on how to create setup file, visit https://docs.snowflake.com/en/developer-guide/native-apps/creating-setup-script
+
+-- A general guideline to building this script looks like:
+-- 1. Create application roles
+CREATE APPLICATION ROLE IF NOT EXISTS app_public;
+
+-- 2. Create a versioned schema to hold those UDFs/Stored Procedures
+CREATE OR ALTER VERSIONED SCHEMA core;
+GRANT USAGE ON SCHEMA core TO APPLICATION ROLE app_public;
+-- -- 3. Create a view referencing the shared view in the package shared-content.sql.
+-- CREATE VIEW IF NOT EXISTS core.shared_view
+-- AS SELECT * FROM package_shared.PROVIDER_SPOTIFY_VIEW;
+
+-- -- 4. Grant appropriate privileges over the view to your application roles. 
+-- GRANT SELECT ON VIEW core.shared_view TO APPLICATION ROLE app_public;
+
+-- 3. Create a streamlit object using the code you wrote in you wrote in src/module-ui, as shown below. 
+-- The `from` value is derived from the stage path described in snowflake.yml
+CREATE STREAMLIT core.ui
+     FROM '/streamlit/'
+     MAIN_FILE = 'ui.py';
+
+-- 4. Grant appropriate privileges over these objects to your application roles. 
+GRANT USAGE ON STREAMLIT core.ui TO APPLICATION ROLE app_public;
+
+-- A detailed explanation can be found at https://docs.snowflake.com/en/developer-guide/native-apps/adding-streamlit 
diff --git a/snowflake-cortex/data/spotify_consumer.csv b/snowflake-cortex/data/spotify_consumer.csv
@@ -0,0 +1,101 @@
+track_name,artists_name,artist_count,released_year,released_month,released_day,in_spotify_charts,streams,danceability_%
+"Too Many Nights (feat. Don Toliver & with Future)","Future, Metro Boomin, Don Toliver",3,2022,12,2,58,286400165,68
+"Chemical","Post Malone",1,2023,4,14,39,172825906,50
+"Mockingbird","Eminem",1,2004,1,1,61,1241559043,62
+"New Jeans","NewJeans",1,2023,7,7,35,29562220,81
+"Primera Cita","Carin Leon",1,2022,4,20,27,77309611,54
+"Cold Heart - PNAU Remix","Dua Lipa, Elton John, Pnau",3,2017,11,10,52,1605224506,80
+"Dandelions","Ruth B.",1,2017,4,28,21,1116995633,61
+"Bones","Imagine Dragons",1,2021,3,11,44,838079900,77
+"Set Fire to the Rain","Adele",1,2011,1,1,43,1163620694,61
+"Money Trees","Kendrick Lamar, Jay Rock",2,2012,1,1,32,1093605526,74
+"Tak Segampang Itu","Anggi Marito",1,2022,12,2,6,179659294,51
+"LAGUNAS","Jasiel NuÔøΩÔøΩez, Peso P",2,2023,6,22,18,39058561,77
+"Mine (Taylors Version)","Taylor Swift",1,2023,7,7,15,36912123,65
+"Everybody Wants To Rule The World","Tears For Fears",1,1985,2,17,25,1205951614,64
+"No Role Modelz","J. Cole",1,2014,12,9,36,1791000570,70
+"Tattoo","Loreen",1,2023,2,25,59,201660859,55
+"Rara Vez","Taiu, Milo j",2,2023,2,8,38,248088961,84
+"VAGABUNDO","Sebastian Yatra, Manuel Turizo, BeÔøΩÔ",3,2023,5,12,34,90839753,82
+"august","Taylor Swift",1,2020,7,24,22,607123776,51
+"LUNA","Junior H, Peso Pluma",2,2023,6,22,11,55842345,75
+"Miracle (with Ellie Goulding)","Calvin Harris, Ellie Goulding",2,2023,3,10,48,211050784,64
+"Nonsense","Sabrina Carpenter",1,2022,7,15,27,342897938,74
+"Que Vuelvas","Carin Leon, Grupo Frontera",2,2022,12,9,26,2762,49
+"Por las Noches","Peso Pluma",1,2021,6,11,24,330346424,81
+"Feliz CumpleaÔøΩÔøΩos Fe","Feid",1,2022,8,19,38,601863821,87
+"Can't Hold Us (feat. Ray Dalton)","Ray Dalton, Ryan Lewis, Macklemore",3,2011,8,16,52,1953533826,63
+"Watermelon Sugar","Harry Styles",1,2019,11,17,34,2322580122,55
+"lovely - Bonus Track","Billie Eilish, Khalid",2,2017,8,11,30,2355719893,35
+"Rauw Alejandro: Bzrp Music Sessions, Vol. 56","Rauw Alejandro, Bizarrap",2,2023,6,21,32,66902503,78
+"Queencard","(G)I-DLE",1,2023,5,15,33,96273746,82
+"OMG","NewJeans",1,2023,1,2,27,430977451,80
+"Radio","Lana Del Rey",1,2011,1,1,46,284819874,42
+"Shakira: Bzrp Music Sessions, Vol. 53","Shakira, Bizarrap",2,2023,1,11,44,721975598,78
+"505","Arctic Monkeys",1,2007,4,20,25,1217120710,52
+"Calling (Spider-Man: Across the Spider-Verse) (Metro Boomin & Swae Lee, NAV, feat. A Boogie Wit da Hoodie)","Swae Lee, A Boogie Wit da Hoodie, Metro Boomin, NAV",4,2023,6,2,16,109276132,63
+"Trance (with Travis Scott & Young Thug)","Travis Scott, Young Thug, Metro Boomin",3,2022,12,2,46,276259178,75
+"Tere Vaaste (From 'Zara Hatke Zara Bachke')","Sachin-Jigar, Shadab Faridi, Altamash Faridi, Amitabh Bhattacharya, Varun Jain",5,2023,5,22,8,54225632,76
+"Perfect","Ed Sheeran",1,2017,1,1,13,2559529074,60
+"Romantic Homicide","d4vd",1,2022,7,20,23,681583126,56
+"Believer","Imagine Dragons",1,2017,1,31,23,2594040133,77
+"Novo BalanÔøΩ","Veigh, Bvga Beatz, Supernova Ent, Prod Malax",4,2023,5,19,7,81102253,84
+"Gol Bolinha, Gol Quadrado 2","Mc Pedrinho, DJ 900",2,2023,6,1,8,11956641,93
+"Without Me","Eminem",1,2002,1,1,43,1687664027,92
+"QUEMA","Sog, Ryan Castro, Peso Pluma",3,2023,7,13,31,11599388,79
+"Stargirl Interlude","The Weeknd, Lana Del Rey",2,2016,11,24,32,611700552,59
+"Ojitos Lindos","Bomba Estereo, Bad B",2,2022,5,6,38,1133865788,65
+"Somewhere Only We Know","Keane",1,2004,1,1,16,1089402494,45
+"Those Eyes","New West",1,2019,5,10,14,411747614,60
+"El Gordo Trae El Mando","Chino Pacas",1,2023,1,27,21,255932395,74
+"Mi Bello Angel","Natanael Cano",1,2023,6,30,8,31873544,81
+"Bye","Peso Pluma",1,2023,5,26,14,95053634,78
+"Danza Kuduro","Don Omar, Lucenzo",2,2010,1,1,37,1279434863,47
+"Nosso Quadro","Ana Castela, AgroPlay",2,2023,2,2,9,233801632,69
+"Locked Out Of Heaven","Bruno Mars",1,2012,12,5,9,1481349984,73
+"Un Finde | CROSSOVER #2","Big One, FMK, Ke personajes",3,2023,4,4,14,142095275,50
+"Jimmy Cooks (feat. 21 Savage)","Drake, 21 Savage",2,2022,6,17,27,618885532,54
+"Counting Stars","OneRepublic",1,2013,1,1,43,2011464183,66
+"Ghost","Justin Bieber",1,2021,3,19,24,1167330737,61
+"Under The Influence","Chris Brown",1,2019,10,4,26,929964809,73
+"PRC","Natanael Cano, Peso Pluma",2,2023,1,23,26,436027885,78
+"Gasolina","Daddy Yankee",1,2004,7,13,18,657723613,86
+"One Dance","Drake, WizKid, Kyla",3,2016,4,4,24,2713922350,77
+"Enchanted","Taylor Swift",1,2010,1,1,16,621660989,45
+"Save Your Tears","The Weeknd",1,2020,3,20,13,1591223784,68
+"Sure Thing","Miguel",1,2010,5,25,19,950906471,68
+"Every Breath You Take - Remastered 2003","The Police",1,1983,1,6,19,1593270737,82
+"The Night We Met","Lord Huron",1,2015,2,2,35,1410088830,45
+"We Found Love","Rihanna, Calvin Harris",2,2011,1,1,21,1235005533,73
+"When I Was Your Man","Bruno Mars",1,2012,12,5,11,1661187319,60
+"Let Me Down Slowly","Alec Benjamin",1,2018,5,25,19,1374581173,65
+"Am I Dreaming (Metro Boomin & A$AP Rocky, Roisee)","A$AP Rocky, Metro Boomin, Roisee",3,2023,6,2,16,94186466,60
+"Do I Wanna Know?","Arctic Monkeys",1,2013,1,1,26,1788326445,55
+"Demons","Imagine Dragons",1,2012,1,1,13,1840364617,33
+"Idol","YOASOBI",1,2023,4,12,16,143573775,57
+"Reminder","The Weeknd",1,2016,11,25,17,684675814,71
+"Shake It Off","Taylor Swift",1,2014,1,1,13,1113838873,65
+"Why'd You Only Call Me When You're High?","Arctic Monkeys",1,2013,1,1,29,1267333350,70
+"SNAP","Rosa Linn",1,2022,3,19,18,726307468,56
+"Shape of You","Ed Sheeran",1,2017,1,6,10,3562543890,83
+"Night Changes","One Direction",1,2014,11,17,18,1131090940,67
+"Fin de Semana","Oscar Maydon, Junior H",2,2023,1,13,14,307370144,70
+"Creep","Radiohead",1,1992,9,21,7,1271293243,53
+"Car's Outside","James Arthur",1,2019,10,18,10,265882712,34
+"Apocalypse","Cigarettes After Sex",1,2017,3,21,17,841749534,37
+"Cheques","Shubh",1,2023,5,19,8,47956378,74
+"Pink + White","Frank Ocean",1,2016,8,20,30,806397070,54
+"Circles","Post Malone",1,2019,8,30,16,2132335812,70
+"Just The Way You Are","Bruno Mars",1,2010,1,1,13,1641426668,63
+"Take Me To Church","Hozier",1,2013,9,13,31,2135158446,57
+"Bebe Dame","Fuerza Regida, Grupo Frontera",2,2022,12,16,22,367316268,54
+"You Belong With Me (TaylorÔøΩÔøΩÔøΩs Ve","Taylor Swift",1,2021,4,9,12,350381515,63
+"Titi Me PreguntÔø","Bad Bunny",1,2022,5,6,42,1264310836,65
+"Better Than Revenge (Taylor's Version)","Taylor Swift",1,2023,7,7,11,30343206,50
+"Shut up My Moms Calling","Hotel Ugly",1,2020,2,10,14,405136812,48
+"Have You Ever Seen The Rain?","Creedence Clearwater Revival",1,1968,7,1,14,1145727611,74
+"Es un Secreto","Plan B",1,2010,7,20,36,540654286,84
+"POLARIS - Remix","Feid, Mora, Saiko, Quevedo",4,2023,6,8,33,57312735,62
+"Ditto","NewJeans",1,2022,12,19,22,397582059,81
+"Take On Me","a-ha",1,1984,10,19,17,1479115056,57
+"Annihilate (Spider-Man: Across the Spider-Verse) (Metro Boomin & Swae Lee, Lil Wayne, Offset)","Swae Lee, Lil Wayne, Offset, Metro Boomin",4,2023,6,2,4,86773632,61