Refresh vocabulary tables #18
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: Refresh vocabulary tables | |
| on: | |
| schedule: | |
| ## - cron: '0 0 * * 0' # weekly, sunday 12am | |
| - cron: '0 8 * * *' # every day 8am GMT (3/4am EST/EDT) | |
| workflow_dispatch: | |
| jobs: | |
| refresh-vocab-tables: | |
| # runs-on: BIDS-Premium-Action-Runners | |
| runs-on: ubuntu-latest | |
| # fail-fast: At least shows "Error: Process completed with exit code 143." instead of "exit code 1", for slightly more information about unexpected exits: https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs#handling-failures | |
| strategy: | |
| fail-fast: false | |
| steps: | |
| # Set up | |
| - name: Checkout repository and submodules | |
| uses: actions/checkout@v2 | |
| with: | |
| submodules: recursive | |
| - name: Set up Python version | |
| uses: actions/setup-python@v2 | |
| with: | |
| python-version: '3.10' | |
| - name: Print commit hash & branch for rollbacks & troubleshooting | |
| run: | | |
| echo "Commit hash: ${{ github.sha }}" | |
| echo "Branch: ${{ github.ref }}" | |
| - name: 'Create env file' | |
| run: | | |
| mkdir env | |
| echo "${{ secrets.ENV_FILE }}" > env/.env | |
| - name: Create and start virtual environment | |
| run: | | |
| python3 -m venv venv | |
| source venv/bin/activate | |
| - name: Install dependencies | |
| run: | | |
| # Reactivate install upgrades of pip, wheel, &/or setuptools if needed | |
| # python -m pip install --upgrade pip | |
| # pip install --upgrade wheel | |
| # pip install --upgrade setuptools | |
| # VS Hub SDK w/ OAuth disabled: https://github.com/jhu-bids/TermHub/issues/863 | |
| # TOKEN=$(grep '^PALANTIR_ENCLAVE_AUTHENTICATION_BEARER_TOKEN=' env/.env | cut -d'=' -f2) | |
| # python3 -m pip install vshub_sdk --upgrade --extra-index-url "https://:${TOKEN}@unite.nih.gov/artifacts/api/repositories/ri.artifacts.main.repository.9bc9cc56-4b8c-4560-9bfb-fba8ade55246/contents/release/pypi/simple" --extra-index-url "https://:${TOKEN}@unite.nih.gov/artifacts/api/repositories/ri.foundry-sdk-asset-bundle.main.artifacts.repository/contents/release/pypi/simple" | |
| pip install -r requirements.txt | |
| # Refresh | |
| - name: Refresh vocab | |
| run: make refresh-vocab | |
| # Test | |
| # todo: This won't actually run, because the Python script will throw an error at the end (only if successful | |
| # refresh happened, which will notify that it completed successfully. So the GitHub action will only get this far | |
| # if there was never any data. However, this test does run daily in a separate action. | |
| - name: Test | |
| run: python -m unittest test.test_backend.db.test_refresh_dataset_group_tables.TestCurrentDatasetGroupSetup.test_current_vocab |