Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-1482571: Add CI pipeline for apps unit testing #3

Merged
merged 69 commits into from
Jun 19, 2024
Merged
Show file tree
Hide file tree
Changes from 68 commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
d3d88df
Create python-app.yml
sfc-gh-osalazarlizano Jun 17, 2024
97ea167
change yml name
sfc-gh-osalazarlizano Jun 17, 2024
8de4c42
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
a2c9798
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
f933890
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
e2c6f1e
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
069f8c1
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
24bdb0b
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
e9a44b8
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
74533cf
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
ff8b032
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
dd87a03
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
40bac16
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
bb247a3
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
b49be4b
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
931c619
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
ada646b
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
49573ae
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
e2cd7f5
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
68abfa9
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
ef65f6e
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
b83eb70
add changed files to the pipeline
sfc-gh-osalazarlizano Jun 17, 2024
5e6872e
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
ac8aacc
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
f342d8f
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
ec550f6
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
09118f5
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
97f9f85
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
52e8283
Update ci.yml
sfc-gh-osalazarlizano Jun 17, 2024
dcfc4b7
Create test.sh
sfc-gh-osalazarlizano Jun 17, 2024
7572218
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
622c9f5
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
34eae21
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
778417e
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
11a221a
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
09af8bb
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
e7a4542
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
7439255
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
4a38651
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
480b9ad
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
bb184b9
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
7c0b384
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
a03f9e6
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
4c7edd8
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
34075bd
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
875da9b
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
6c8e068
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
f69bfa5
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
e232c05
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
1522cbd
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
5c0ea12
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
9b8fb93
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
c895600
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
72fd108
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
61bef88
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
daab66d
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
8223238
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
1b32603
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
01d45b3
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
2a4c688
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
c0db3f1
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
7130100
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
e065feb
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
82f5a26
Delete test.sh
sfc-gh-osalazarlizano Jun 18, 2024
6fb36d9
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
d272767
Update customers.sql
sfc-gh-osalazarlizano Jun 18, 2024
e77d17b
Update customers.sql
sfc-gh-osalazarlizano Jun 18, 2024
de51b2e
Update ci.yml
sfc-gh-osalazarlizano Jun 18, 2024
6b1c110
apply code suggestions
sfc-gh-osalazarlizano Jun 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 103 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
name: native-app-examples

on:
pull_request:
types:
- opened
- edited
- labeled
- unlabeled
- synchronize

permissions:
contents: read

jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
include:
- environment-file: local_test_env.yml
defaults:
run:
shell: bash -l {0}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: ${{ github.event_name == 'pull_request' && 2 || 0 }}
- name: Set up Python 3.10
uses: actions/setup-python@v3
with:
python-version: "3.10"
- name: Get changed files
id: changed-files
run: |
if ${{ github.event_name == 'pull_request' }}; then
echo "changed_files=$(git diff --name-only -r HEAD^1 HEAD | xargs)" >> $GITHUB_OUTPUT
else
echo "changed_files=$(git diff --name-only ${{ github.event.before }} ${{ github.event.after }} | xargs)" >> $GITHUB_OUTPUT
fi
- name: Determine tests to run
uses: actions/github-script@v7
id: tests_to_run
env:
CHANGED_FILES: ${{ steps.changed-files.outputs.changed_files }}
with:
script: |
const { CHANGED_FILES } = process.env;
const fs = require('fs');
const path = require('path');

function getPytestPaths(dir, callback)
{
const files = fs.readdirSync(dir, { withFileTypes: true });
for (const file of files)
{
if (file.isDirectory())
{
getPytestPaths(path.join(dir, file.name), callback);
}
else
{
extension = path.extname(file.name);

if (extension == '.py' && !file.name.startsWith('test'))
sfc-gh-osalazarlizano marked this conversation as resolved.
Show resolved Hide resolved
{
callback(dir);
}
}
}
}

const paths = new Set(CHANGED_FILES.split(" ")
.map(x => x.substring(0, x.indexOf("/") + 1))
.filter(x => x.length > 0 && !x.startsWith('.')));

const pytestPaths = new Set()
for (const rootPath of paths)
{
getPytestPaths(rootPath, x => pytestPaths.add(x))
}

core.setOutput('pytestPaths', [...pytestPaths].join(' '));
core.setOutput('pytestArgs', [...paths].join(' '));

- name: Setup test environment
uses: conda-incubator/setup-miniconda@v2
with:
environment-file: ${{ matrix.environment-file }}
- name: Install dependencies
run: |
printf "[pytest]\npythonpath=${{ steps.tests_to_run.outputs.pytestPaths }}" > pytest.ini
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be enough to enforce that each individual app has its own pytest.ini and we can simply delegate to it. I don't think it's necessary to only run changed files -- unless I'm misreading this. Does this just delegate to individual inis?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are dynamically creating that in the root folder since the pytest command needs it in the folder to be executed. We are running tests from modified sub-repos, and this is why the pytest command is run globally and not in the individual app.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah got it. OK, that makes sense to me!

python -m pip install pytest
- name: Run tests
run: |
args=${{ steps.tests_to_run.outputs.pytestArgs }}
if [ -z "${args}" ]; then
echo “Nothing to test”
else
pytest $args
fi


15 changes: 15 additions & 0 deletions local_test_env.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# This file is used to install packages for local testing
sfc-gh-osalazarlizano marked this conversation as resolved.
Show resolved Hide resolved
name: native-apps-examples-testing
channels:
- snowflake
dependencies:
- python=3.8
- pip
- pip:
- git+https://github.com/snowflakedb/snowflake-telemetry-python.git
- snowflake-native-apps-permission-stub
- snowflake-snowpark-python>=1.15.0
- snowflake-cli-labs>=2.0.0
- pytest
- streamlit>=1.26.0

Loading