Skip to content

Commit

Permalink
Merge pull request #60 from OnroerendErfgoed/release/0.2.1
Browse files Browse the repository at this point in the history
Release/0.2.1
  • Loading branch information
dieuska authored Sep 10, 2024
2 parents ddb2734 + 0177950 commit 97314d6
Show file tree
Hide file tree
Showing 15 changed files with 289 additions and 173 deletions.
58 changes: 29 additions & 29 deletions .github/workflows/python-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,34 +17,34 @@ jobs:

strategy:
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12"]
python-version: [ "3.9", "3.10", "3.11", "3.12" ]

steps:
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
pip install --upgrade pip
pip install pip-tools
pip-sync requirements-dev.txt
- name: Black code style check
run: |
# stop the build if there are files that need to reformatted
black --check .
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Test with pytest
run: |
pytest
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
pip install --upgrade pip
pip install pip-tools
pip-sync requirements-dev.txt
- name: Black code style check
run: |
# stop the build if there are files that need to reformatted
black --check .
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Test with pytest
run: |
pytest
17 changes: 13 additions & 4 deletions CHANGES.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,19 @@
# 0.1.0

- Initial version
- Initial version

# 0.2.0

- add GRB update detection functions and logic
- refactor loader code and other non generic parts ([#17](https://github.com/OnroerendErfgoed/brdr/issues/17), ([#7](https://github.com/OnroerendErfgoed/brdr/issues/7))
- add functionality to set a maximum feature size for input features (([#50](https://github.com/OnroerendErfgoed/brdr/issues/50))
- fix bug reulting from overlapping features in thematic layer ([#46](https://github.com/OnroerendErfgoed/brdr/issues/46))
- refactor loader code and other non generic
parts ([#17](https://github.com/OnroerendErfgoed/brdr/issues/17), ([#7](https://github.com/OnroerendErfgoed/brdr/issues/7))
- add functionality to set a maximum feature size for input
features (([#50](https://github.com/OnroerendErfgoed/brdr/issues/50))
- fix bug reulting from overlapping features in thematic
layer ([#46](https://github.com/OnroerendErfgoed/brdr/issues/46))

# 0.2.1

- fixed last_version_date in aligner.get_formula()
- fixed logic of evaluate() in grb.py
- added function to transform geojson to consistent geometry-type (MultiPolygon)
160 changes: 96 additions & 64 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,43 +19,60 @@ Quick links:
## Description

### Intro
`brdr` is a Python package that assists in aligning geometric boundaries to reference boundaries. This is an important task in geographic data management to enhance data quality.
* In the context of geographic data management, it is important to have accurate and consistent boundaries for a variety of applications such as calculating areas, analyzing spatial relationships, and visualizing and querying geographic information.
* When creating geographic data, it is often more efficient to derive boundaries from existing reference data rather than collecting new data in the field.
* `brdr` can be used to align boundaries from new data to reference data, ensuring that the boundaries are accurate and consistent.

`brdr` is a Python package that assists in aligning geometric boundaries to reference boundaries. This is an important
task in geographic data management to enhance data quality.

* In the context of geographic data management, it is important to have accurate and consistent boundaries for a variety
of applications such as calculating areas, analyzing spatial relationships, and visualizing and querying geographic
information.
* When creating geographic data, it is often more efficient to derive boundaries from existing reference data rather
than collecting new data in the field.
* `brdr` can be used to align boundaries from new data to reference data, ensuring that the boundaries are accurate and
consistent.

### Example

The figure below shows:
* the original thematic geometry (blue),
* A reference layer (yellow-black).

* the original thematic geometry (blue),
* A reference layer (yellow-black).
* The resulting geometry after alignment with `brdr` (green)

![](docs/figures/example.png)
![](docs/figures/example.png)

### Functionalities

`brdr` provides a variety of side-functionalities to assist in aligning boundaries, including:

* Loading thematic data ((Multi-)Polygons): as a dict, geojson or Web Feature Service (WFS-url)
* Loading reference data ((Multi-)Polygons): as a dict, geojson or Web Feature Service (WFS-url)
* (Flanders-specific) Download reference data from GRB-Flanders
* (Flanders-specific) Download reference data from GRB-Flanders
* Align thematic boundaries to reference boundaries with a specific relevant distance (process_dict_thematic)
* Align thematic boundaries to reference boundaries with a series of specified relevant distances (process_series)
* Make use of a 'predictor'-function that aligns thematic boundaries to reference boundaries for 'predicted' interesting relevant distances (predictor)
* Make use of a 'predictor'-function that aligns thematic boundaries to reference boundaries for 'predicted' interesting
relevant distances (predictor)
* Calculating a descriptive formulation of a thematic boundary based on a reference layer

### Possible application fields

* Geodata-management:
* Implementation of `brdr` in business-processes and tooling
* Bulk geodata-alignment
* Alignment after reprojection of data
* Cleaning data: In a postprocessing-phase, the algorithm executes sliver-cleanup and validity-cleaning on the resulting geometries
* ...
* Implementation of `brdr` in business-processes and tooling
* Bulk geodata-alignment
* Alignment after reprojection of data
* Cleaning data: In a postprocessing-phase, the algorithm executes sliver-cleanup and validity-cleaning on the
resulting geometries
* ...
* Data-Analysis: Investigate the pattern in deviation and change between thematic and reference boundaries
* Update-detection: Investigate the descriptive formulation before and after alignment to check for (automatic) alignment of geodata
* Update-detection: Investigate the descriptive formulation before and after alignment to check for (automatic)
alignment of geodata
* ...

### QGIS-script

An implementation of `brdr` for QGIS can be found at [GitHub-brdrQ](https://github.com/OnroerendErfgoed/brdrQ/).
This QGIS- script provides a User Interface to align thematic data to a reference layer, showing the results in the QGIS Table of Contents.
This QGIS- script provides a User Interface to align thematic data to a reference layer, showing the results in the QGIS
Table of Contents.

## Installation

Expand Down Expand Up @@ -103,112 +120,127 @@ print("removed area: " + process_result["theme_id_1"]["result_diff_min"].wkt)
# thematic_dict,
# reference_dict)
```

The resulting figure shows:

* the reference polygon (yellow-black)
* the original geometry (blue)
* the resulting geometry (green line)
* the resulting geometry (green line)
* the added zone (green squares)
* the removed zone (red squares)
<img src="docs/figures/basic_example.png" width="100%" />
<img src="docs/figures/basic_example.png" width="100%" />

More examples can be found in [Examples](https://github.com/OnroerendErfgoed/brdr/tree/main/examples)

## Workflow

(see also Basic example)

To use `brdr`, follow these steps:

* Create a Aligner-class with specific parameters:
* relevant_distance (m) (default: 1): Distance-parameter used to decide which parts will be aligned, and which parts remain unchanged.
* od_strategy (enum) (default: SNAP_SINGLE_SIDE): Strategy to align geodata that is not covered by reference-data
* threshold_overlap_percentage (%)(0-100) (default 50)
* crs: The Coordinate Reference System (CRS) (default: EPSG:31370 - Belgian Lambert72)
* relevant_distance (m) (default: 1): Distance-parameter used to decide which parts will be aligned, and which parts
remain unchanged.
* od_strategy (enum) (default: SNAP_SINGLE_SIDE): Strategy to align geodata that is not covered by reference-data
* threshold_overlap_percentage (%)(0-100) (default 50)
* crs: The Coordinate Reference System (CRS) (default: EPSG:31370 - Belgian Lambert72)
* Load thematic data
* Load reference data
* Process (align) the thematic data
* Results are returned:
* Resulting geometry
* Differences: parts that are 'different' from the original geometry (positive or negative)
* Positive differences: parts that are added to the original geometry
* Negative differences: parts that are removed form the original geometry
* Relevant intersections: relevant intersecting parts of the reference geometries
* Relevant differences: relevant differences of the reference geometries
* Resulting geometry
* Differences: parts that are 'different' from the original geometry (positive or negative)
* Positive differences: parts that are added to the original geometry
* Negative differences: parts that are removed form the original geometry
* Relevant intersections: relevant intersecting parts of the reference geometries
* Relevant differences: relevant differences of the reference geometries

## The `brdr`-algorithm
## The `brdr`-algorithm

The algorithm for alignment is based on 2 main principles:

* Principle of intentionality: Thematic boundaries can consciously or unconsciously deviate from the reference borders. The algorithm should keep notice of that.
* Selective spatial conservation of shape: The resulting geometry should re-use the shape of the reference borders where aligned is of relevance.
* Principle of intentionality: Thematic boundaries can consciously or unconsciously deviate from the reference borders.
The algorithm should keep notice of that.
* Selective spatial conservation of shape: The resulting geometry should re-use the shape of the reference borders where
aligned is of relevance.

The figure below shows a schematic overview of the algorithm:
![](docs/figures/algorithm.png)

The algorithm can be split into 3 main phases:
* Initialisation:
* Deciding which reference polygons are candidate-polygons to re-use its shape.
The reference candidate polygons are selected based on spatial intersection with the thematic geometry.
* Processing:
* Process all candidate-reference polygons one-by-one
* Calculate relevant zones for each candidate-reference-polygon
* relevant intersections: zones that must be present in the final result
* relevant differences: zones that must be excluded from the final result
![](docs/figures/relevant_zones.png)
* Evaluate each candidate based on their relative zones: which parts must be kept and which parts must be excluded
![](docs/figures/evaluate_candidates.png)
* Union all kept parts to recompose a resulting geometry
* Post-processing:
* Validation/correction of differences between the original input geometry and the composed intermediate resulting geometry after processing the algorithm
* Technical validation of inner holes and multipolygons that are created by processing the algorithm
* Clean-up slivers
* Make the resulting geometry valid

RESULT:

* Initialisation:
* Deciding which reference polygons are candidate-polygons to re-use its shape.
The reference candidate polygons are selected based on spatial intersection with the thematic geometry.
* Processing:
* Process all candidate-reference polygons one-by-one
* Calculate relevant zones for each candidate-reference-polygon
* relevant intersections: zones that must be present in the final result
* relevant differences: zones that must be excluded from the final result
![](docs/figures/relevant_zones.png)
* Evaluate each candidate based on their relative zones: which parts must be kept and which parts must be excluded
![](docs/figures/evaluate_candidates.png)
* Union all kept parts to recompose a resulting geometry
* Post-processing:
* Validation/correction of differences between the original input geometry and the composed intermediate resulting
geometry after processing the algorithm
* Technical validation of inner holes and multipolygons that are created by processing the algorithm
* Clean-up slivers
* Make the resulting geometry valid

RESULT:

A new resulting output geometry, aligned to the reference-polygons

## Development

### pip-compile

```sh
PIP_COMPILE_ARGS="-v --strip-extras --no-header --resolver=backtracking --no-emit-options --no-emit-find-links"
pip-compile $PIP_COMPILE_ARGS
pip-compile $PIP_COMPILE_ARGS -o requirements-dev.txt --all-extras
```

### tests

```python
python -m pytest --cov=brdr tests/ --cov-report term-missing
```

## Motivation & citation
A more in-depth description of the algorithm can be found in the following article (in dutch):

- Dieussaert, K., Vanvinckenroye, M., Vermeyen, M., & Van Daele, K. (2024). Grenzen verleggen.
Automatische correcties van geografische afbakeningen op verschuivende
onderlagen *Onderzoeksrapporten Agentschap Onroerend Erfgoed*,
332. <https://doi.org/10.55465/SXCW6218>.
A more in-depth description of the algorithm can be found in the following article (in dutch):

- Dieussaert, K., Vanvinckenroye, M., Vermeyen, M., & Van Daele, K. (2024). Grenzen verleggen.
Automatische correcties van geografische afbakeningen op verschuivende
onderlagen *Onderzoeksrapporten Agentschap Onroerend Erfgoed*,
332. <https://doi.org/10.55465/SXCW6218>.

## Comments and contributions

We would love to hear from you and your experiences with
`brdr` or its sister project [`brdrQ`](https://github.com/OnroerendErfgoed/brdrQ). The [discussions forum](https://github.com/OnroerendErfgoed/brdr/discussions/) is the place to be when:
`brdr` or its sister project [`brdrQ`](https://github.com/OnroerendErfgoed/brdrQ).
The [discussions forum](https://github.com/OnroerendErfgoed/brdr/discussions/) is the place to be when:

- You have any questions on using `brdr` or `brdrQ` or their
applicability to your use cases
- Want to share your experiences with the library
- Have any suggestions for improvements or feature requests
- You have any questions on using `brdr` or `brdrQ` or their
applicability to your use cases
- Want to share your experiences with the library
- Have any suggestions for improvements or feature requests

If you have discovered a bug in the `brdr` library you can report it here:

<https://github.com/OnroerendErfgoed/brdr/issues>
<https://github.com/OnroerendErfgoed/brdr/issues>

We try to keep the list of issues as clean as possible. If
We try to keep the list of issues as clean as possible. If
you're unsure whether something is a bug, or whether the bug is in `brdr`
or `brdrQ`, we encourage you to go through the [discussions forum](https://github.com/OnroerendErfgoed/brdr/discussions) first.
or `brdrQ`, we encourage you to go through the [discussions forum](https://github.com/OnroerendErfgoed/brdr/discussions)
first.

## Acknowledgement

This software was created by [Athumi](https://athumi.be/en/), the Flemish data utility company, and [Flanders Heritage Agency](https://www.onroerenderfgoed.be/flanders-heritage-agency).
This software was created by [Athumi](https://athumi.be/en/), the Flemish data utility company,
and [Flanders Heritage Agency](https://www.onroerenderfgoed.be/flanders-heritage-agency).

![https://athumi.be/en/](docs/figures/athumi_logo-250x84.png)
![https://www.onroerenderfgoed.be/flanders-heritage-agency](docs/figures/Vlaanderen_is_erfgoed-250x97.png)
2 changes: 1 addition & 1 deletion brdr/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@
datefmt="%d-%b-%y %H:%M:%S",
)

__version__ = "0.2.0"
__version__ = "0.2.1"
Loading

0 comments on commit 97314d6

Please sign in to comment.