Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workflow #64

Open
wants to merge 78 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
f0ebf55
add ptolemy as a dep
gidden Mar 30, 2023
2c803de
add initial gridding routine
gidden Apr 3, 2023
53131c8
some clean up with remaining iso col
gidden Apr 3, 2023
d4ca804
stickler
gidden Apr 3, 2023
83e8c6c
update for xarray reqs
gidden Apr 11, 2023
00d96f2
add example notebook
gidden Apr 11, 2023
2bd9a28
add docstrings for grid
gidden Apr 11, 2023
32a0051
add docstring for check_coord_overalp
gidden Apr 12, 2023
c5cab38
update xlsxwriter save -> close
gidden Apr 12, 2023
338a277
add plots in notebook
gidden Apr 12, 2023
bca3416
Add downscaling module
coroa Mar 3, 2023
3f26663
Improve harmonization bits with pandas_indexing
coroa Apr 3, 2023
8b421ab
Extend harmonization method selection to downscaling
coroa Apr 5, 2023
1b8e20e
Add method selection to downscaling
coroa Apr 5, 2023
82452d3
Add proxy checking
coroa Apr 7, 2023
fc5a8c1
Create downscaling context early
coroa Apr 7, 2023
676309a
Fix downscaling methods
coroa Apr 7, 2023
4813d93
Make linter happy
coroa Apr 7, 2023
588c6b5
Fix tox typo
coroa May 25, 2023
f05c1e1
Add simple_proxy downscaling method
coroa May 25, 2023
e55f1d2
Revise how arguments to default_methods are handled
coroa May 26, 2023
73b1659
Fix ExcelWriter usage
coroa May 26, 2023
edfb273
Merge branch 'pr/gidden/58' into workflow
coroa May 26, 2023
22f9ceb
Fix pandas-indexing imports
coroa May 26, 2023
23c0073
grid: Generalize check_coord_overlap
coroa May 31, 2023
9aff824
Add Gridder
coroa May 31, 2023
6567c37
Assume years as integers
coroa Jul 19, 2023
2d6581c
fix(intensity_convergence): Avoid creating duplicates
coroa Jul 19, 2023
90d7526
Add semijoin call into where to help pandas
coroa Aug 10, 2023
f79fcee
fix(downscaling): Cope with zero model emissions in intensity converg…
coroa Aug 16, 2023
f0c2772
add global trajectories to gridding, allow missing data that is in pr…
gidden Aug 11, 2023
487e41b
dropnas in tabular when proxy has more dims than data, add compression
gidden Aug 11, 2023
50ef375
add chunking option for proxy dimensions and provide explicit iter_le…
gidden Aug 12, 2023
d2c5632
rm print
gidden Aug 12, 2023
3b78655
add global_only cfg column for global sectors
gidden Aug 14, 2023
884693f
add comment about chunk_proxy_dims
gidden Aug 14, 2023
088d3ce
allow for inspection of datasets to be generated with write arg, prov…
gidden Aug 16, 2023
b992c35
add a verification function to confirm global yearly sums
gidden Aug 16, 2023
cbd94c8
revert global_only changes, to be checked
gidden Aug 17, 2023
8c265c7
change chunk_proxy_dims to a map instead of a list
gidden Aug 17, 2023
7b223db
bugfix chunks
gidden Aug 17, 2023
04a33a3
Convert output verification into a dask task (#63)
coroa Aug 17, 2023
3654362
Fix CI/CD
coroa May 28, 2023
2b98cd6
add pycountry to deps
gidden Aug 18, 2023
faa03b6
verify output now works as expected, allow skipping of write
gidden Aug 18, 2023
c27099e
black file
gidden Aug 18, 2023
3d4c107
additional bugfix for downscaled data in verify_global_values
gidden Aug 18, 2023
9c69460
support multiple methods in verify_global_values
gidden Aug 18, 2023
17f3641
make luc_sectors explicit and allow Gridder to be instatiated with them
gidden Aug 19, 2023
ab43e18
grid: Fix verify_global_values for multiple scenarios
coroa Aug 21, 2023
4d39847
fix(test_harmonize): Specify string-based harmonization year explicitly
coroa Aug 21, 2023
a3f78a4
black test harmonize
gidden Aug 21, 2023
1185c2d
Fix docs CI
coroa Aug 21, 2023
8f434ed
update base_year_pattern to support multiple scenarios/models
gidden Aug 25, 2023
a2de080
enh(downscaler): Check downscaling results for correctness
coroa Aug 25, 2023
34100b1
Add pre-commit config
coroa Aug 25, 2023
347fcbf
fix(downscaling): Make Downscaler use year-argument
coroa Aug 30, 2023
023fa90
Add additional check whether downscaled starts from history
coroa Aug 30, 2023
8600e13
add capability to skip executing a gridding operation if the output f…
gidden Aug 28, 2023
0d4d8c9
remove share separation logic
gidden Oct 3, 2023
12619b0
change complevel to 2
gidden Oct 4, 2023
238b2a4
first cut at multiproxy
gidden Oct 4, 2023
6479a9d
slight tweaks
gidden Oct 4, 2023
5b3a502
rearrange try except in open_and_normalize, bugfix for getting option…
gidden Oct 4, 2023
9e9e9c5
add fillvalue
gidden Oct 4, 2023
7ab762d
fixup pr
gidden Oct 4, 2023
8949072
aneris updates for dressing up gridded files
gidden Oct 5, 2023
3dd850a
remove template kwargs
gidden Oct 5, 2023
33da47a
blacked
gidden Oct 5, 2023
aafde73
Update src/aneris/grid.py
gidden Oct 5, 2023
d262217
Update src/aneris/grid.py
gidden Oct 5, 2023
d851f66
Loosen downscaling check thresholds
coroa Oct 3, 2023
f86527c
Cope with empty methods better
coroa Nov 24, 2023
a2f1591
Blackify
coroa Feb 22, 2024
1cec507
Update pyproject.toml ruff settings
coroa Feb 22, 2024
89df851
enh(downscaling): Switch downscaling regionmap to MultiIndex
coroa May 14, 2024
08b95d9
Make compatible with pyarrow-backed DataFrames
coroa Aug 23, 2024
14729ae
Update pandas-indexing
coroa Aug 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .coveragerc
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
[run]
omit = aneris/tutorial.py
omit = aneris/tutorial.py
9 changes: 8 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,18 +16,25 @@ jobs:
python-version: [3.8, 3.11] # to minimise complexity we only test a min and a max version
include:
# on all platforms and versions do everything
- tox-envs: [docs, lint, build, test]
- tox-envs: [lint, test, docs, build]

runs-on: ${{ matrix.platform }}

steps:
- name: Checkout
uses: actions/checkout@v3

# pandoc is required by nbsphinx for building the notebook-based docs
- name: Setup pandoc for building docs
uses: r-lib/actions/setup-pandoc@v2
with:
pandoc-version: '2.19.2'

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'

- name: Install tox
run: |
Expand Down
77 changes: 68 additions & 9 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,13 +1,72 @@
#*
aneris/_version.py
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

*#
# C extensions
*.so

# editors
*.swp
*~
*.pyc
build
dist
*.egg-info

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
_version.py

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
.*
nosetests.xml
coverage.xml
*.cover
.hypothesis/
.pytest_cache/

# Sphinx documentation
docs/_build/
docs/html
docs/latex

# Jupyter Notebook
.ipynb_checkpoints

# pyenv
.python-version

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

venv
# Editor settings
.spyderproject
.spyproject
.ropeproject
.vscode
61 changes: 61 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
hooks:
- id: check-merge-conflict
- id: end-of-file-fixer
#- id: fix-encoding-pragma # ruff does not thing this makes sense
- id: mixed-line-ending
- id: trailing-whitespace
- id: check-added-large-files
args: ["--maxkb=2000"]

# # Convert relative imports to absolute imports
# - repo: https://github.com/MarcoGorelli/absolufy-imports
# rev: v0.3.1
# hooks:
# - id: absolufy-imports

# Find common spelling mistakes in comments and docstrings
- repo: https://github.com/codespell-project/codespell
rev: v2.2.2
hooks:
- id: codespell
args: ['--ignore-regex="(\b[A-Z]+\b)"', '--ignore-words-list=fom'] # Ignore capital case words, e.g. country codes
types_or: [python, rst, markdown]
files: ^(scripts|doc)/

# Make docstrings PEP 257 compliant
- repo: https://github.com/PyCQA/docformatter
rev: v1.5.1
hooks:
- id: docformatter
args: ["--in-place", "--make-summary-multi-line", "--pre-summary-newline"]

- repo: https://github.com/keewis/blackdoc
rev: v0.3.8
hooks:
- id: blackdoc

# Formatting with "black" coding style
- repo: https://github.com/psf/black
rev: 23.1.0
hooks:
# Format Python files
- id: black
# Format Jupyter Python notebooks
- id: black-jupyter

# Linting with ruff
- repo: https://github.com/charliermarsh/ruff-pre-commit
# Ruff version.
rev: 'v0.0.245'
hooks:
- id: ruff
args: [--fix, --exit-non-zero-on-fix]

# # Check for FSFE REUSE compliance (licensing)
# - repo: https://github.com/fsfe/reuse-tool
# rev: v1.1.2
# hooks:
# - id: reuse
2 changes: 1 addition & 1 deletion .readthedocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,4 @@ python:
install:
- method: pip
path: .
extra_requirements: [docs]
extra_requirements: [docs]
2 changes: 1 addition & 1 deletion README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

**To reproduce harmonization routines from [Gidden et al.
(2019)](https://gmd.copernicus.org/articles/12/1443/2019/), use `v0.3.2` (or
earlier). Subsequent versions introduce backwards incompatibilities.**
earlier). Subsequent versions introduce backwards incompatibilities.**

Documentation
-------------
Expand Down
2 changes: 1 addition & 1 deletion ci/.coveragerc
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[report]
omit =
aneris/_version.py
aneris/_version.py
2 changes: 1 addition & 1 deletion ci/py2/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ FROM gidden/python2-viz

COPY . /aneris
WORKDIR /
RUN cd /aneris && python2 setup.py install
RUN cd /aneris && python2 setup.py install
2 changes: 1 addition & 1 deletion ci/py3/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ FROM gidden/python3-viz

COPY . /aneris
WORKDIR /
RUN cd /aneris && python3 setup.py install
RUN cd /aneris && python3 setup.py install
1 change: 0 additions & 1 deletion ci/travis-install-miniconda.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,3 @@ fi

# update conda
conda update --yes conda

2 changes: 1 addition & 1 deletion doc/.gh-config
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ include:
- _static
- _modules
- _templates
- _downloads
- _downloads
2 changes: 1 addition & 1 deletion doc/source/_bib/index.bib
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ @article{Gidden:2019:aneris
volume = {105},
journal = {Environmental Modelling & Software},
doi = {10.1016/j.envsoft.2018.04.002}
}
}
Loading
Loading