Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge development for v1.1.0 #28

Merged
merged 98 commits into from
Feb 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
98 commits
Select commit Hold shift + click to select a range
accd313
FEAT: Add PSpline smoothing to utils
derb12 Apr 2, 2023
b57fce6
FEAT: Add Liliand's customized baseline correction algorithm
derb12 Apr 3, 2023
a6b7103
FEAT: Add pspline_mpls
derb12 Apr 8, 2023
3bcb94e
TESTS: Add tests for mpls_pspline for half_window inputs
derb12 Apr 22, 2023
9079524
Merge branch 'main' into development
derb12 May 7, 2023
a2e9c9d
MAINT: Fix removal of necessary modules
derb12 May 7, 2023
8dca50f
OTHER: Added pspline knots, coefs, and degree from pspline_smooth
derb12 May 27, 2023
829de65
FEAT: Add rubberband method
derb12 Dec 27, 2023
c714706
MAINT: Change keys for custom_bc output params
derb12 Dec 28, 2023
b185acf
BREAK: Remove config.py
derb12 Dec 28, 2023
8c1a885
OTHER: Initialize whittaker system for fabc and rubberband
derb12 Dec 28, 2023
3c5539b
OTHER: Vender scipy's cwt and ricker functions
derb12 Dec 28, 2023
d4edcdc
MAINT: Fixed doc references to pentapy's solver
derb12 Dec 28, 2023
4671317
MAINT: Improve display of lam values in example
derb12 Dec 29, 2023
61a85dc
MAINT: Add python 3.12 as supported
derb12 Dec 31, 2023
b97cc5e
OTHER: Use a slightly faster calculation for _convert_coef
derb12 Jan 1, 2024
c4e7a76
DOCS: Ignore sphinx-gallery timings file
derb12 Jan 4, 2024
d3ff449
DOCS: Fix duplicate object warnings when building docs
derb12 Jan 4, 2024
20ea790
MAINT: Remove internal code for kwargs with optimizers
derb12 Jan 7, 2024
6f30cc2
MAINT: Handle deprecation of numpy.trapz
derb12 Jan 10, 2024
02ef6b3
TEST: Fixed x and y ordering for spline and whittaker tests
derb12 Jan 13, 2024
f7748c2
OTHER: Add add_diagonal method to PenalizedSystem
derb12 Jan 15, 2024
5f9e655
TEST: Ignore NumbaDeprecationWarning for nopython kwarg
derb12 Jan 20, 2024
385ddbc
MAINT: Use internal methods for solver whittaker and spline fits
derb12 Jan 21, 2024
0383117
MAINT: Allow skipping sorting inputs and outputs
derb12 Jan 27, 2024
2978e0f
MAINT: Use binning instead of direct truncation for custom_bc
derb12 Jan 28, 2024
1fe1caf
FEAT: Add 2D versions of some morphological algorithms
derb12 Apr 8, 2023
e781028
FEAT: Add 2D versions for most polynomial algorithms
derb12 Apr 17, 2023
8031abc
FEAT: Added 2D version of quant_reg
derb12 Apr 22, 2023
8de57b0
FEAT: Yeehaw finally figured out 2D penalized splines
derb12 Apr 26, 2023
e82cbb2
OTHER: Use a more efficient method for 2D psplines
derb12 Apr 27, 2023
e8a3bf9
FEAT: Implemented 2D version of mixture_model
derb12 Apr 29, 2023
09ebad2
MAINT: Fix removal of necessary modules
derb12 May 7, 2023
c554df1
FEAT: Implemented some 2D Whittaker baselines
derb12 Sep 3, 2023
ac8f2e1
FEAT: Allow separate polynomial orders for x and z
derb12 Sep 10, 2023
5c7149c
MAINT: Improve polynomial cross term matching
derb12 Sep 10, 2023
bd4018b
OTHER: Allow converting polynomial coefficients in 2D
derb12 Jan 2, 2024
84e4212
OTHER: Only retain sort order if needed
derb12 Jan 2, 2024
42d74bb
MAINT: Moved _sort_array to utils and added tests
derb12 Jan 6, 2024
451e447
MAINT: Finished code for initializing Baseline2D
derb12 Jan 6, 2024
dbd33e0
TESTS: Finished tests for two_d algorithm_setup
derb12 Jan 9, 2024
f00e8f7
TESTS: Finished tests for PSpline2D
derb12 Jan 9, 2024
bb48564
TEST: Add base class for testing 2D algorithms
derb12 Jan 12, 2024
8e24f74
MAINT: Change dimensionality of Baseline2D
derb12 Jan 12, 2024
280f6b4
MAINT: Fixed 2D Whittaker functions
derb12 Jan 12, 2024
860c499
TEST: Finished tests for 2D polynomial algorithms
derb12 Jan 13, 2024
88ffea5
TEST: Finished tests for 2D Whittaker algorithms
derb12 Jan 13, 2024
d77f8ae
MAINT: Fix validation.yxz_arrays
derb12 Jan 13, 2024
d9561cd
TEST: Add tests for 2D spline and morphological algorithms
derb12 Jan 13, 2024
8c74173
FEAT: Added 2D versions of collab_pls and adaptive_minmax
derb12 Jan 17, 2024
5765527
FEAT: Add 2D version of noise_median
derb12 Jan 17, 2024
a793bd5
TEST: Add comparison test to statsmodels for 2D quantile regression
derb12 Jan 17, 2024
07e6fe7
FEAT: Solve some 2D Whittaker baselines as banded systems
derb12 Jan 17, 2024
9b41267
TEST: Reduce size of 2D datasets for testing
derb12 Jan 17, 2024
1fb5ef2
MAINT: Clean up leftover code
derb12 Jan 17, 2024
598eb91
MAINT: Make PSpline2D subclass of PenalizedSystem2D
derb12 Jan 18, 2024
28c29b3
FEAT: Added 2D version of pspline_iasls
derb12 Jan 18, 2024
1518ca7
MAINT: Fixed handling of window values in 2d
derb12 Jan 23, 2024
d836853
DOCS: Fix method references in docs to point to Baseline or Baseline2D
derb12 Jan 26, 2024
ca2f74f
MAINT: Allow skipping sorting inputs and outputs for 2D
derb12 Jan 30, 2024
4c264fd
MAINT: Handle array-like smooth half window for 2d rolling_ball
derb12 Feb 1, 2024
3adb73e
MAINT: Updated all 2D docstrings to show the correct typing
derb12 Feb 1, 2024
147f4ab
MAINT: Specify rows and columns for 2D pspline
derb12 Feb 3, 2024
72855bc
MAINT: Remove the 2d version of goldindec
derb12 Feb 3, 2024
b288ddb
MAINT: Update CI and drop python 3.6 and 3.7
derb12 Feb 4, 2024
3baaa97
DOCS: Add algorithm section for 2D
derb12 Feb 4, 2024
587afec
MAINT: Ignore x order for weight inputs test
derb12 Feb 4, 2024
60ca5eb
MAINT: Add _get_method helper method to Baseline and Baseline2D
derb12 Feb 9, 2024
523749e
MAINT: Raise an error when 1d data is used for Baseline2D
derb12 Feb 9, 2024
f2a351e
ENH: Allow eigendecomposition for most 2D whittaker algorithms
derb12 Feb 9, 2024
9c2df3b
MAINT: Modified PSpline2D tck to work with scipy's NdBSpline
derb12 Feb 9, 2024
ba7c95c
DOCS: Speed up the whittaker solver example
derb12 Feb 9, 2024
0f37804
MAINT : Was not updating alpha_matrix for 2D aspls
derb12 Feb 10, 2024
c622ed5
OTH: Internally use scipy's sparse arrays when available
derb12 Feb 10, 2024
4cb72cb
MAINT: Move all metadata to pyproject and update CI
derb12 Feb 10, 2024
6d23edc
TEST: Add CI for testing against nightly numpy and scipy
derb12 Feb 10, 2024
9c0c728
MAINT: Extend _check_scalar_variable to allow 2d inputs
derb12 Feb 11, 2024
cb3e18a
MAINT: Update contributor guide
derb12 Feb 11, 2024
e200d59
DOCS: Finish the 2D algorithms section for whittaker and splines
derb12 Feb 12, 2024
1ca950d
DOCS: Finalize 2D algorithms explanation
derb12 Feb 12, 2024
d677d71
DOCS: Add docs and example for custom_bc
derb12 Feb 12, 2024
a5c4791
TEST: Finished most tests for 2D whittaker smoothing with eigendecomp…
derb12 Feb 13, 2024
86fb254
MAINT: Fix linting errors and bump numpy version
derb12 Feb 13, 2024
c0f1057
Merge pull request #27 from derb12/2D
derb12 Feb 13, 2024
7393b71
MAINT: Bump lowest pentapy version to 1.1
derb12 Feb 13, 2024
65aa82c
ENH: Use expectation-maximization for mixture_model
derb12 Feb 15, 2024
6802b82
MAINT: Renamed eigenvalues to num_eigens for 2d whittakers
derb12 Feb 15, 2024
34c1d5a
ENH: Added individual_axes to fit 2D data along each axis
derb12 Feb 16, 2024
1febf6b
TEST: Added tests for individual_axes
derb12 Feb 17, 2024
c5f3efa
DOCS: Pin transitive dependencies for requirements
derb12 Feb 17, 2024
995e087
DOCS: Made example showing eigendecomposition for 2d Whittaker
derb12 Feb 17, 2024
1a668d1
MAINT: Remove old quickstart from docs
derb12 Feb 18, 2024
21f2262
DOCS: Add example for using individual_axes
derb12 Feb 18, 2024
2c1abb2
TEST: Finished remaining tests for WhittakerSystem2D
derb12 Feb 18, 2024
7ad9dbd
MAINT: Sort imports
derb12 Feb 18, 2024
c7e2eed
DOCS: Update changelog and final updates to docs
derb12 Feb 18, 2024
3c0b8a4
Bump version v1.0.0 -> v1.1.0
derb12 Feb 18, 2024
bc3b8e8
MAINT: Fix linting errors
derb12 Feb 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 0 additions & 24 deletions .bumpversion.cfg

This file was deleted.

9 changes: 4 additions & 5 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,8 @@
are required and/or what problems they fix. Link or add the issue number
for any existing issues that the pull request solves.

Note that unsolicited pull requests will most likely be closed.
Please file an issue first, so that details can be discussed/finalized
before a pull request is created.-->
Note that it is preferred to file an issue first, so that details can be
discussed/finalized before a pull request is created.-->


### Type of Pull Request
Expand All @@ -24,9 +23,9 @@ before a pull request is created.-->
To run tests locally, type the following command within the pybaselines
directory: pytest .

To lint files using flake8 to see if they pass PEP 8 standards and that
To lint files using ruff to see if they pass PEP 8 standards and that
docstrings are okay, run the following command within the pybaselines
directory: flake8 .
directory: ruff check .

To build documentation locally, type the following command within the
docs directory: make html
Expand Down
54 changes: 54 additions & 0 deletions .github/workflows/python-test-latest.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# For testing the nightly builds of numpy and scipy so that any new changes will not be
# a surprise.

# Will only trigger if there is a change within pybaselines or tests directories.

name: test-latest-dependencies

on:
# allow manually activating the workflow
workflow_dispatch:

push:
branches: [ main ]
paths:
- 'pybaselines/**'
- 'tests/**'
- '.github/workflows/**'

pull_request:
# always trigger on a pull request, regardless of the branch
paths:
- 'pybaselines/**'
- 'tests/**'
- '.github/workflows/**'

jobs:
test-nightly:

runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
# Choose the latest stable python version
python-version: ['3.12']

steps:
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install required dependencies
run: |
python -m pip install --upgrade pip
python -m pip install pytest
python -m pip install -i https://pypi.anaconda.org/scientific-python-nightly-wheels/simple numpy
python -m pip install -i https://pypi.anaconda.org/scientific-python-nightly-wheels/simple scipy

- name: Test with required dependencies
# use -Werror so that any warnings will show up as errors -> want to be as stringent
# as possible
run: python -Werror -m pytest .
71 changes: 36 additions & 35 deletions .github/workflows/python-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,87 +26,88 @@ on:
jobs:
test:

runs-on: ubuntu-20.04
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
# Use strings since yaml considers 3.10 equal to 3.1
python-version: ['3.6', '3.7', '3.8', '3.9', '3.10', '3.11']
python-version: ['3.8', '3.9', '3.10', '3.11', '3.12']

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install required dependencies
run: |
python -m pip install --upgrade pip
python -m pip install "numpy>=1.14" "scipy>=1.0" pytest

# Only lint a single version; pick a recent, stable version
- name: Install linting dependencies
id: install-linters
if: matrix.python-version == '3.9'
run: |
python -m pip install flake8 flake8-comprehensions flake8-docstrings

- name: Lint
if: steps.install-linters.outcome == 'success'
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings.
flake8 . --count --exit-zero --statistics
python -m pip install "numpy>=1.20" "scipy>=1.5" pytest

- name: Test with required dependencies
run: pytest .

- name: Install optional dependencies
id: install-optional
# uncomment below in case this step ever needs skipped again
#if: matrix.python-version != '3.11'
run: python -m pip install "pentapy>=1.0" "numba>=0.45"
# uncomment below to allow skipping future versions
#if: matrix.python-version != '3.13'
run: python -m pip install "pentapy>=1.1" "numba>=0.49"

- name: Test with optional dependencies
# uncomment below in case this step ever needs skipped again
if: steps.install-optional.outcome == 'success'
run: pytest .


test-min-dependencies:

runs-on: ubuntu-20.04
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ['3.6']
python-version: ['3.8']

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install minimum dependencies
# Use numpy 1.14.5 rather than 1.14.0 since the optional
# dependency pentapy requires numpy>=1.14.5; no relevant difference
# between 1.14.0 and 1.14.5.
run: |
python -m pip install --upgrade pip
python -m pip install numpy==1.14.5 scipy==1.0 pytest
python -m pip install "numpy==1.20" "scipy==1.5" pytest

- name: Test with minimum required dependencies
run: pytest .

- name: Install minimum optional dependencies
# Have to pin llvmlite to 0.30.0 since it otherwise gets a more recent
# version that is imcompatible with numba v0.45
run: python -m pip install pentapy==1.0 numba==0.45 llvmlite==0.30.0
run: python -m pip install "pentapy==1.1" "numba==0.49"

- name: Test with minimum optional dependencies
run: pytest .

lint:

runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ['3.11']

steps:
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install linting dependencies
run: python -m pip install ruff

- name: Lint
run: ruff check .
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ docs/_build/

# sphinx-gallery output
docs/examples/
sg_execution_times.rst

# sphinx-autoapi output
docs/api
Expand Down
2 changes: 1 addition & 1 deletion .readthedocs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ version: 2
build:
os: ubuntu-22.04
tools:
python: "3.9"
python: "3.11"

# Path to sphinx's configuration file
sphinx:
Expand Down
56 changes: 56 additions & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,62 @@
Changelog
=========

Version 1.1.0 (2024-02-18)
--------------------------

This is a minor version with new features, deprecations,
and documentation improvements.

New Features
~~~~~~~~~~~~

* Added two dimensional versions of various baseline correction algorithms,
with the focus on Whittaker-smoothing-based, spline, and polynomial methods.
These can be accessed using the new `Baseline2D` class.
* Added the `Baseline2D.individual_axes` method, which allows fitting each row and/or
column in two dimensional data with any one dimensional method in pybaselines.
* Added a version of the rubberband method to pybaselines.misc which allows fitting
individual segments within data to better fit concave-shaped data.
* Added the Customized Baseline Correction (custom_bc) method to
pybaselines.optimizers, which allows fitting baselines with controllable
levels of stiffness in different regions.
* Added a penalized version of mpls (pspline_mpls) to pybaselines.spline.
* Updated spline.mixture_model to use expectation-maximization rather than the previous
nieve approach of fitting the histogram of the residuals with the probability density
function. Should reduce calculation times.
* Added a function for penalized spline (P-spline) smoothing to pybaselines.utils,
`pybaselines.utils.pspline_smooth`, which will return a tuple of the smoothed input and
the knots, spline coefficients, and spline degree for any further use with
SciPy's BSpline.

Other Changes
~~~~~~~~~~~~~

* Officially list Python 3.12 as supported.
* Updated lowest supported Python version to 3.8
* Updated lowest supported dependency versions: NumPy 1.20, SciPy 1.5,
pentapy 1.1, and Numba 0.49
* Use SciPy's sparse arrays when the installed SciPy version is 1.12 or newer. This
only affects user codes if using functions from the pybaselines.utils module.
* Vendor SciPy's cwt and ricker functions, which were deprecated from SciPy in version 1.12.

Deprecations/Breaking Changes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* Deprecated passing `num_bins` to spline.mixture_model. The keyword argument will
be removed in version 1.3.
* Removed the pybaselines.config module, which was simply used to set the pentapy solver.
The same behavior can be done by setting the `pentapy_solver` attribute of a `Baseline`
object after initialization.

Documentation/Examples
~~~~~~~~~~~~~~~~~~~~~~

* Added a section of the documentation explaining the extension of baseline correction for
two dimensional data.
* Added new examples for 2D baseline correction and for custom_bc.


Version 1.0.0 (2022-10-26)
--------------------------

Expand Down
2 changes: 1 addition & 1 deletion CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,4 @@ repository-artifact: https://pypi.org/project/pybaselines
repository-code: https://github.com/derb12/pybaselines
title: "pybaselines: A Python library of algorithms for the baseline correction of experimental data"
type: software
version: 1.0.0
version: 1.1.0
2 changes: 1 addition & 1 deletion LICENSE.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
BSD 3-Clause License

Copyright (c) 2021-2023, Donald Erb
Copyright (c) 2021-2024, Donald Erb
All rights reserved.

Redistribution and use in source and binary forms, with or without
Expand Down
6 changes: 3 additions & 3 deletions LICENSES_bundled.txt
Original file line number Diff line number Diff line change
Expand Up @@ -162,11 +162,11 @@ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.


Source: SciPy (https://github.com/scipy/scipy, last accessed November 6, 2021)
File: pybaselines._spline_utils.py
Source: SciPy (https://github.com/scipy/scipy, last accessed December 28, 2023)
Files: pybaselines._spline_utils.py, pybaselines.classification.py
License: 3-clause BSD

Copyright (c) 2001-2002 Enthought, Inc. 2003-2019, SciPy Developers.
Copyright (c) 2001-2002 Enthought, Inc. 2003-2023, SciPy Developers.
All rights reserved.

Redistribution and use in source and binary forms, with or without
Expand Down
17 changes: 0 additions & 17 deletions MANIFEST.in

This file was deleted.

Loading