Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup testing #67

Merged
merged 52 commits into from
Jul 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
d2d5e08
fix tutorials paths in gitignore
elbeejay Mar 7, 2024
13a5385
fix step-1 resolve deprecation warnings
elbeejay Mar 7, 2024
289a025
run step_2, remove deprecations including dict structure for CRS
elbeejay Mar 8, 2024
8e82465
get Step 3 working
elbeejay Mar 8, 2024
4b3641a
fix examples notebooks
elbeejay Mar 8, 2024
c0cc725
fix crs in doc-strings and as defaults to str
elbeejay Mar 8, 2024
aa3393c
reduce number of paths for example speed
elbeejay Mar 8, 2024
25aedbb
lengthen timeout for docs to run
elbeejay Mar 8, 2024
9309e23
text, formatting for step-1 tutorial
elbeejay Mar 16, 2024
44194bf
formatting step-2 tutorial
elbeejay Mar 17, 2024
0b8add4
formatting for step-3 notebook
elbeejay Mar 17, 2024
baa775d
update example_finding notebook
elbeejay Mar 17, 2024
8c15ce3
fix doc-strings ex-fixing
elbeejay Mar 18, 2024
65ca02f
update ex gravity notebook
elbeejay Mar 18, 2024
fb63f41
start testing
elbeejay Mar 24, 2024
ce945bb
placeholder tests
elbeejay Mar 26, 2024
d929bbd
add tests
elbeejay Mar 27, 2024
0b2aa66
one more test, coverage to 9%
elbeejay Mar 27, 2024
627db39
try miniconda ci
elbeejay Mar 27, 2024
2db9290
non-interactive conda install
elbeejay Mar 27, 2024
96ba8db
try to fix ci job
elbeejay Mar 27, 2024
de4059c
specify coverage rcfile
elbeejay Mar 27, 2024
6bae815
dont report coverage for now
elbeejay Mar 27, 2024
25dbddc
remove flag
elbeejay Mar 27, 2024
00f3e07
add tests
elbeejay Mar 28, 2024
c4fbcc8
add tests
elbeejay Mar 29, 2024
00b761a
core funcs and tests
elbeejay Apr 2, 2024
9e97e8e
finish tests for calculate_od_raw
elbeejay May 12, 2024
9d5b04f
more tests
elbeejay May 12, 2024
0ba15f4
add integrated test
elbeejay May 23, 2024
1e9e17c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 23, 2024
16c0722
more tests and edits for new networkx versions
elbeejay May 23, 2024
22a316e
Merge branch 'setup-testing' of github.com:worldbank/GOSTnets into se…
elbeejay May 23, 2024
1d6d454
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 23, 2024
cb22de6
add tests
elbeejay May 24, 2024
0b26687
stop building on macos
elbeejay May 24, 2024
0ebbde7
add test
elbeejay May 30, 2024
4e4fb5b
updates
elbeejay Jun 14, 2024
19d2dbe
Merge branch 'main' of github.com:worldbank/GOSTnets into setup-testing
elbeejay Jun 15, 2024
724c012
fix and test for length of multilinesegment
elbeejay Jun 15, 2024
6e2a130
more tests, use sample data in tests
elbeejay Jun 15, 2024
a5cd34b
more tests and fixes to core fcts
elbeejay Jun 16, 2024
7014b97
add test
elbeejay Jun 18, 2024
c3889f8
cleanup tests, test isochrone function
elbeejay Jul 1, 2024
e85cb16
more tests and edits to core functions
elbeejay Jul 4, 2024
27c1f7d
more testing of core functions
elbeejay Jul 9, 2024
3e5c6f5
add more tests
elbeejay Jul 11, 2024
7d2b7a0
more tests, up to 70% test coverage
elbeejay Jul 12, 2024
864347a
add more tests
elbeejay Jul 15, 2024
b1601d9
minor edits
elbeejay Jul 18, 2024
7f09eb6
clean up tests a bit, add pooch for some data
elbeejay Jul 20, 2024
ff46df4
remove cov report
elbeejay Jul 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version: ["3.8"]
os: [ubuntu-latest, windows-latest]
python-version: ["3.8", "3.9", "3.10"]
steps:
- name: Checkout code
uses: actions/checkout@v4
Expand Down
31 changes: 31 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: tests

on:
push:
pull_request:

jobs:
ubuntu-build:
runs-on: ${{ matrix.os }}
defaults:
run:
shell: bash -l {0}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest]
env:
OS: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: conda-incubator/setup-miniconda@v3
with:
auto-update-conda: false
activate-environment: test-env
auto-activate-base: false
- name: Install dependencies then test
run: |
conda install -c conda-forge gdal
conda install -c conda-forge pip
pip install .[dev]
coverage run -m pytest
99 changes: 85 additions & 14 deletions Tutorials/EXAMPLE_Finding_links_between_pairs.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,32 @@
"For most network analysis we do not consider actual geographic connections - this notebook focuses on how to calculate those direct connections between points and collect the geospatial information"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Import Libraries and Define Input Data\n",
"\n",
"First we'll import the necessary libraries and define the input data."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Libraries\n",
"\n",
"We'll use the following libraries:\n",
"\n",
"- `os` for setting the working directory\n",
"- `pickle` to load the data\n",
"- `networkx` to calculate travel times through the graph\n",
"- `geopandas` to work with the geospatial data\n",
"- `pandas` to work with the data\n",
"- `shapely.geometry` to work with the geometry of the data\n",
"- `GOSTnets` to apply custom functions to the network"
]
},
{
"cell_type": "code",
"execution_count": null,
Expand All @@ -21,17 +47,20 @@
"import geopandas as gpd\n",
"import pandas as pd\n",
"\n",
"from shapely.geometry import MultiLineString"
"from shapely.geometry import MultiLineString\n",
"\n",
"# import the GOSTnet library\n",
"import GOSTnets as gn"
]
},
{
"cell_type": "code",
"execution_count": null,
"cell_type": "markdown",
"metadata": {},
"outputs": [],
"source": [
"# import the GOSTnet library\n",
"import GOSTnets as gn"
"### Load Input Data\n",
"\n",
"We define the path to the Iceland network tutorial data, and load the network graph information.\n",
"Then we load the origins and destination data, so that we're ready to calculate travel times between the points."
]
},
{
Expand Down Expand Up @@ -59,6 +88,16 @@
"in_churches = in_churches.to_crs(rek_grid.crs)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Calculate Shortest Paths Between Loaded Origins and Destinations\n",
"\n",
"We'll calculate the shortest paths between the loaded origins and destinations.\n",
"To do this we first need to snap the origins and destinations to the network, only then can we calculate the shortest paths between them."
]
},
{
"cell_type": "code",
"execution_count": null,
Expand Down Expand Up @@ -112,7 +151,29 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Calculate line strings connecting all origins to all destinations"
"## Calculate line strings connecting all origins to all destinations\n",
"\n",
"We'll calculate the line strings connecting all origins to all destinations.\n",
"For the sake of the example, we'll only calculate the line strings between the first 10 origins and the first 10 destinations."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# for the sake of the example we truncate the number of origins and destinations\n",
"# to make the computation faster. We will only use the first 10 of each\n",
"origins = origins[:10]\n",
"destinations = destinations[:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Practically we perform this calculation by looping through all origins and destinations and calculating the line string between them."
]
},
{
Expand Down Expand Up @@ -155,6 +216,16 @@
" all_res.append([org, dest, MultiLineString(all_edges)])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Write the data to a file\n",
"\n",
"Finally, we'll write the data to a file.\n",
"First we write all connection data to a CSV file."
]
},
{
"cell_type": "code",
"execution_count": null,
Expand All @@ -166,6 +237,13 @@
"all_results.to_csv(os.path.join(pth, \"tutorial_outputs\", \"all_OD_links.csv\"))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Then we write the connections information out to a CSV file."
]
},
{
"cell_type": "code",
"execution_count": null,
Expand All @@ -191,13 +269,6 @@
" os.path.join(pth, \"tutorial_outputs\", \"OD_links_usage.csv\")\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
Expand Down
75 changes: 68 additions & 7 deletions Tutorials/EXAMPLE_Fixing_your_road_network.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,36 @@
"source": [
"# Fixing your road network graph\n",
"\n",
"*Note: This notebook includes processes covered in the [Step 2](Step_2-Clean_Network.ipynb) Tutorial Notebook.*\n",
"\n",
"The following notebook showcases some common practices to inspect the quality of your road-network graph, and possibilities to improve it."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Import Libraries and Load Data\n",
"\n",
"We will start by importing the necessary libraries and loading the data."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Import Libraries\n",
"\n",
"We will use the following libraries:\n",
"\n",
"- `os` to set the working directory and handle file paths\n",
"- `networkx` to handle the graph data structure\n",
"- `osmnx` to plot the network graph\n",
"- `pickle` to load the network data\n",
"- `matplotlib` to help plot and format the graph\n",
"- `GOSTnets` to access custom GOSTnets functionality"
]
},
{
"cell_type": "code",
"execution_count": null,
Expand All @@ -18,17 +45,20 @@
"import os\n",
"import networkx as nx\n",
"import osmnx as ox\n",
"import pickle as pkl"
"import pickle as pkl\n",
"import matplotlib.pyplot as plt\n",
"\n",
"# import the GOSTnets library\n",
"import GOSTnets as gn"
]
},
{
"cell_type": "code",
"execution_count": null,
"cell_type": "markdown",
"metadata": {},
"outputs": [],
"source": [
"# import the GOSTnets library\n",
"import GOSTnets as gn"
"### Load Data\n",
"\n",
"Define the path to the tutorial data and then load the network."
]
},
{
Expand All @@ -44,6 +74,24 @@
"G = pkl.load(open(os.path.join(data_pth, \"iceland_network_clean.pickle\"), \"rb\"))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Graph processing\n",
"\n",
"In this notebook the first processing step we take is to load the largest subgraph."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Identify the largest subgraph\n",
"\n",
"We will start by identifying the largest subgraph in the network."
]
},
{
"cell_type": "code",
"execution_count": null,
Expand Down Expand Up @@ -75,6 +123,15 @@
"print(G_largest)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Plot the largest subgraph\n",
"\n",
"We will then plot the largest subgraph to visualize the network."
]
},
{
"cell_type": "code",
"execution_count": null,
Expand All @@ -85,8 +142,12 @@
"G_largest.graph[\"crs\"] = \"epsg:32646\"\n",
"G_largest.graph[\"name\"] = \"Iceland\"\n",
"\n",
"fig, ax = plt.subplots(figsize=(10, 14))\n",
"ax.set_facecolor(\"k\")\n",
"ax.set_title(\"Iceland - The Largest Network\")\n",
"# largest connected subgraph\n",
"ox.plot_graph(G_largest, figsize=(10, 14), edge_linewidth=1, node_size=7)"
"fig, ax = ox.plot_graph(G_largest, ax=ax, edge_linewidth=1, node_size=7)\n",
"plt.show()"
]
},
{
Expand Down
Loading
Loading