Merge pull request #7 from asiripanich/automate

feat: Add CI and a python notebook to make the CSV files fully reproducible.

.github/workflows/tidy.yml

@@ -0,0 +1,47 @@
+name: Tidy ANZSIC
+
+on:
+  push:
+
+jobs:
+  tidy:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with: 
+          python-version: 3.x
+
+      - name: Install dependencies
+        run: pip install -r requirements.txt
+
+      - name: Run notebook
+        run: papermill tidy.ipynb tidy-rendered.ipynb
+
+      - name: Upload tidy-rendered.ipynb
+        uses: actions/upload-artifact@v4
+        with:
+          name: tidy-rendered.ipynb
+          path: tidy-rendered.ipynb
+
+      - name: Upload anzsic_1993.csv
+        uses: actions/upload-artifact@v4
+        with:
+          name: anzsic_1993.csv
+          path: anzsic_1993.csv
+
+      - name: Upload anzsic_2006.csv
+        uses: actions/upload-artifact@v4
+        with:
+          name: anzsic_2006.csv
+          path: anzsic_2006.csv
+
+      - name: Upload anzsic_1993_to_2006_conversion_table.csv
+        uses: actions/upload-artifact@v4
+        with:
+          name: anzsic_1993_to_2006_conversion_table.csv
+          path: anzsic_1993_to_2006_conversion_table.csv
+
+
+

.gitignore

@@ -0,0 +1 @@
+.DS_Store

README.md

@@ -1,5 +1,16 @@
 # anzsic codes
+[![Tidy ANZSIC](https://github.com/asiripanich/anzsic/actions/workflows/tidy.yml/badge.svg)](https://github.com/asiripanich/anzsic/actions/workflows/tidy.yml)
+
 ANZSIC codes in a standard table format for people who are sick of ABS for sharing data in non-standard formats.
 
+
+## Data sources
+
+| Filename | Source | Downloaded date |
+|----------|----------|-------|
+| `data/1292.0.55.005_ anzsic 2006 - correspondence tables.xls`   | https://www.abs.gov.au/AUSSTATS/[email protected]/DetailsPage/1292.0.55.0052006?OpenDocument   | 2024-10-23 |
+| `data/1292.0.55.002_anzsic 2006 - codes and titles.xls`   | https://www.abs.gov.au/AUSSTATS/[email protected]/DetailsPage/1292.0.55.0022006?OpenDocument   | 2024-10-23 |
+
+
 # Known issues
 Please note that the 'nfd' (not further defined) classes are missing from the anzsic tables, as repoted by @baslat in https://github.com/asiripanich/anzsic/issues/1. PRs to fix are welcome. 

env.yml

@@ -0,0 +1,9 @@
+name: anzsic
+dependencies:
+  - python=3.13.0
+  - pip=24.2
+  - pip:
+    - polars==1.10.*
+    - pyarrow==17.0.0
+    - fastexcel==0.12.0
+    - papermill==2.6.0

requirements.txt

@@ -0,0 +1,6 @@
+polars==1.10.0
+pyarrow==17.0.0
+fastexcel==0.12.0
+ipython 
+ipykernel
+papermill==2.6.0