Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MAINT: use Github based GPU instance #181

Merged
merged 15 commits into from
Jun 13, 2024
24 changes: 2 additions & 22 deletions .github/workflows/cache.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,30 +4,10 @@ on:
branches:
- main
jobs:
deploy-runner:
runs-on: ubuntu-latest
steps:
- uses: iterative/setup-cml@v2
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
- name: Deploy runner on EC2
env:
REPO_TOKEN: ${{ secrets.QUANTECON_SERVICES_PAT }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
run: |
cml runner launch \
--cloud=aws \
--cloud-region=us-west-2 \
--cloud-type=p3.2xlarge \
--labels=cml-gpu \
--cloud-hdd-size=40
cache:
needs: deploy-runner
runs-on: [self-hosted, cml-gpu]
runs-on: ubuntu-latest-gpu
container:
image: docker://mmcky/quantecon-lecture-python:cuda-12.3.1-anaconda-2024-02-py311
image: ghcr.io/quantecon/lecture-python-container:cuda-12.5.0-anaconda-2024-02-py311
options: --gpus all
steps:
- uses: actions/checkout@v4
Expand Down
26 changes: 3 additions & 23 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,30 +1,10 @@
name: Build Project [using jupyter-book]
name: Build Preview [using jupyter-book]
on: [pull_request]
jobs:
deploy-runner:
runs-on: ubuntu-latest
steps:
- uses: iterative/setup-cml@v2
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
- name: Deploy runner on EC2
env:
REPO_TOKEN: ${{ secrets.QUANTECON_SERVICES_PAT }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
run: |
cml runner launch \
--cloud=aws \
--cloud-region=us-west-2 \
--cloud-type=p3.2xlarge \
--labels=cml-gpu \
--cloud-hdd-size=40
preview:
needs: deploy-runner
runs-on: [self-hosted, cml-gpu]
runs-on: ubuntu-latest-gpu
container:
image: docker://mmcky/quantecon-lecture-python:cuda-12.3.1-anaconda-2024-02-py311
image: ghcr.io/quantecon/lecture-python-container:cuda-12.5.0-anaconda-2024-02-py311
options: --gpus all
steps:
- uses: actions/checkout@v4
Expand Down
26 changes: 3 additions & 23 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,31 +4,11 @@ on:
tags:
- 'publish*'
jobs:
deploy-runner:
runs-on: ubuntu-latest
steps:
- uses: iterative/setup-cml@v2
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
- name: Deploy runner on EC2
env:
REPO_TOKEN: ${{ secrets.QUANTECON_SERVICES_PAT }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
run: |
cml runner launch \
--cloud=aws \
--cloud-region=us-west-2 \
--cloud-type=p3.2xlarge \
--labels=cml-gpu \
--cloud-hdd-size=40
publish:
if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags')
needs: deploy-runner
runs-on: [self-hosted, cml-gpu]
runs-on: ubuntu-latest-gpu
container:
image: docker://mmcky/quantecon-lecture-python:cuda-12.3.1-anaconda-2024-02-py311
image: ghcr.io/quantecon/lecture-python-container:cuda-12.5.0-anaconda-2024-02-py311
options: --gpus all
steps:
- name: Checkout
Expand All @@ -46,7 +26,7 @@ jobs:
run: pip list
# Download Build Cache from cache.yml
- name: Download "build" folder (cache)
uses: dawidd6/action-download-artifact@v2
uses: dawidd6/action-download-artifact@v3
with:
workflow: cache.yml
branch: main
Expand Down
23 changes: 11 additions & 12 deletions environment.yml
Original file line number Diff line number Diff line change
@@ -1,27 +1,26 @@
name: lecture-jax
name: quantecon
channels:
- default
dependencies:
- python=3.11
- anaconda=2023.09
- anaconda=2024.02
- pip
- pip:
- jupyter-book==0.15.1
- docutils==0.17.1
- quantecon-book-theme==0.7.1
- sphinx-tojupyter==0.3.0
- sphinxext-rediraffe==0.2.7
- sphinx-reredirects==0.1.3
- sphinx-exercise==0.4.1
- ghp-import==1.1.0
- sphinxcontrib-youtube==1.1.0
- sphinx-togglebutton==0.3.1
- arviz==0.13.0
- array-to-latex
- prettytable
- kaleido
# Sandpit Requirements
# - quantecon
# - array-to-latex
# - PuLP
# - cvxpy
# - cvxopt
# - cylp
# - prettytable
- arviz
# Docker Requirements
- pytz
# Docutils Issue (https://github.com/mcmtroffaes/sphinxcontrib-bibtex/issues/322)
- docutils==0.17.1

5 changes: 2 additions & 3 deletions lectures/status.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,8 @@ This table contains the latest execution statistics.

(status:machine-details)=

These lectures are built on `linux` instances through `github actions` and `amazon web services (aws)` to
enable access to a `gpu`. These lectures are built on a [p3.2xlarge](https://aws.amazon.com/ec2/instance-types/p3/)
that has access to `8 vcpu's`, a `V100 NVIDIA Tesla GPU`, and `61 Gb` of memory.
These lectures are built on `linux` instances through `github actions` that has
access to a `gpu`. These lectures make use of the nvidia `T4` card.

You can check the backend used by JAX using:

Expand Down
Loading