notebooks/0. Complete solution.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "consecutive-interference",
   "metadata": {},
   "source": [
    "# Urban PointCloud Processing\n",
    "\n",
    "This notebook shows a \"complete solution\" in which a single point cloud file is automatically labeled using the various tools available in this repository. For clarity we skip [preprocessing of AHN data](1.%20AHN%20preprocessing.ipynb) and assume all necassary data files are already available.\n",
    "\n",
    "We build a `Pipeline` with different `Processors`, each of which labels a particular type of object. The result is a labelled pointcloud, where labels are stored in the LAS extra_dim `label`.\n",
    "\n",
    "The `Pipeline` supports processing a single file, or batch-processing a folder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "055462da-8dd8-4e26-a4a5-773fd613b6d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uncomment to load the local package rather than the pip-installed version.\n",
    "# Add project src to path.\n",
    "import set_path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d6514388-d99e-4c90-a8fb-e3256bb2c8a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import modules.\n",
    "import logging\n",
    "\n",
    "import upcp.fusion as fusion\n",
    "import upcp.region_growing as growing\n",
    "import upcp.utils.ahn_utils as ahn_utils\n",
    "import upcp.utils.bgt_utils as bgt_utils\n",
    "import upcp.utils.plot_utils as plot_utils\n",
    "import upcp.utils.log_utils as log_utils\n",
    "from upcp.pipeline import Pipeline\n",
    "from upcp.labels import Labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0ec7ecd1-c736-4777-9bf5-03abb54f7849",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set-up logging.\n",
    "logfile = 'upcp.pipeline.log'\n",
    "log_utils.reset_logger()\n",
    "\n",
    "# INFO messages will be printed to console.\n",
    "log_utils.add_console_logger()\n",
    "# DEBUG messages will be printed to a file. Use `clear_log` flag to clear the log file, if desired.\n",
    "log_utils.add_file_logger(logfile, clear_log=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6cf2883a-1fd1-4c20-849f-5f7103c802a8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set-up data files.\n",
    "\n",
    "# AHN data folder.\n",
    "ahn_data_folder = '../datasets/ahn/'\n",
    "# File with BGT road polygons.\n",
    "bgt_road_file = '../datasets/bgt/bgt_roads_demo.csv'\n",
    "# File with BGT building polygons.\n",
    "bgt_building_file = '../datasets/bgt/bgt_buildings_demo.csv'\n",
    "# File with <x,y> coordinates of pole-like objects.\n",
    "bgt_pole_file = '../datasets/bgt/custom_poles_demo.csv'\n",
    "# File with <x,y> coordinates of street furniture objects.\n",
    "bgt_street_furniture_file = '../datasets/bgt/bgt_street_furniture_demo.csv'\n",
    "# File with BGT tram track lines.\n",
    "bgt_tramtrack_file = '../datasets/bgt/bgt_tramtracks_demo.csv'\n",
    "# File with BAG building polygons\n",
    "bag_building_file = '../datasets/bag/bag_buildings_demo.csv'\n",
    "\n",
    "# Initialize AHN reader for elevation data.\n",
    "ahn_reader = ahn_utils.NPZReader(ahn_data_folder)\n",
    "\n",
    "# Initialize BGT readers for BGT polygon and point data.\n",
    "bgt_building_reader = bgt_utils.BGTPolyReader(bgt_file=bgt_building_file)\n",
    "bgt_road_reader = bgt_utils.BGTPolyReader(bgt_file=bgt_road_file)\n",
    "bgt_pole_reader = bgt_utils.BGTPointReader(bgt_file=bgt_pole_file)\n",
    "bgt_street_furniture_reader = bgt_utils.BGTPointReader(bgt_file=bgt_street_furniture_file)\n",
    "bgt_tramtrack_reader = bgt_utils.BGTLineReader(bgt_file=bgt_tramtrack_file)\n",
    "\n",
    "# Initialize BAG reader for building outlines.\n",
    "bag_building_reader = bgt_utils.BGTPolyReader(bgt_file=bag_building_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "turned-softball",
   "metadata": {},
   "source": [
    "---\n",
    "## Data Fusion\n",
    "\n",
    "### Ground and Buildings\n",
    "First, we use data fusion to automatically label ground and building points.\n",
    "\n",
    "For details and more options, see notebook [2. Ground and Buildings](2.%20Ground%20and%20Buildings.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "operational-maple",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Ground fuser using pre-processed AHN data.\n",
    "npz_ground_fuser = fusion.AHNFuser(Labels.GROUND, ahn_reader, target='ground', epsilon=0.2, refine_ground=True)\n",
    "\n",
    "# Building fuser using BGT building footprint data and AHN data for the maximum height, if available.\n",
    "bgt_building_fuser = fusion.BGTBuildingFuser(Labels.BUILDING, bgt_reader=bgt_building_reader, offset=0.25, ahn_reader=ahn_reader)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "embedded-anthropology",
   "metadata": {},
   "source": [
    "### Road segments\n",
    "Ground points that fall within certain road parts are labelled as such."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fancy-legislation",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Road fuser using BGT road polygon data.\n",
    "road_part_fuser = fusion.BGTRoadFuser(Labels.ROAD, bgt_reader=bgt_road_reader)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8056c332",
   "metadata": {},
   "source": [
    "### Noise filter\n",
    "Before actually performing the building fusion, we detect and remove outliers and noise (i.e. wrong measurements) from the point cloud."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "94aad450",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Noise filter using a clustering algorithm and AHN elevation data.\n",
    "noise_filter = fusion.NoiseFilter(Labels.NOISE, ahn_reader, epsilon=0.2, min_component_size=100)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3fefc49d",
   "metadata": {},
   "source": [
    "### Suspended cables and streetlights\n",
    "Next we label suspended utility/tram cables and streetlights.\n",
    "For details and on how to generate the BGT tramtrack csv file, see notebook [3. Cables](3.%20Cables.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "94a304aa",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Road fuser using BGT road polygon data.\n",
    "cable_fuser = fusion.CableFuser(Labels.CABLE, Labels.CABLE, Labels.TRAM_CABLE, Labels.ARMATUUR, ahn_reader, bag_reader=bag_building_reader, bgt_tram_reader=bgt_tramtrack_reader)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "secret-packaging",
   "metadata": {},
   "source": [
    "### Cars\n",
    "\n",
    "Then we label cars by searching for 'car-like' clusters above road segments.\n",
    "\n",
    "For details and on how to generate the BGT road csv file, see notebook [4. Cars](4.%20Cars.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "intense-summit",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Dimension ranges of passenger cars.\n",
    "car_params = {'min_height': 1.2, 'max_height': 2.4, 'min_width': 1.4, 'max_width': 2.4, 'min_length': 3.0, 'max_length': 6.0}\n",
    "\n",
    "# Car fuser using a clustering algorithm and BGT road data.\n",
    "car_fuser = fusion.CarFuser(Labels.CAR, ahn_reader, bgt_reader=bgt_road_reader, exclude_types=['fietspad'],\n",
    "                            grid_size=0.05, min_component_size=5000, params=car_params)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7bac6c70-c67c-440f-9c93-7790b172db05",
   "metadata": {},
   "source": [
    "### Pole-like objects\n",
    "\n",
    "Next, we look for 'pole-like' objects such as trees, street lights and traffic signs.\n",
    "\n",
    "For details and more options, see notebook [5. Pole-like objects](5.%20Pole-like%20objects.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f7f0a685-2525-40ab-bd8f-eff7f94a97fc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Parameter settings for each type of object.\n",
    "tree_params = {'seed_height': 1.5, 'min_points': 500, 'max_r': 0.5, 'label_height': 5.}\n",
    "light_params = {'seed_height': 1.75, 'min_points': 400, 'max_r': 0.2, 'label_height': 5.}\n",
    "sign_params = {'seed_height': 1.5, 'min_points': 200, 'max_r': 0.2, 'min_height': 1.2, 'z_max': 2., 'label_height': 3.}\n",
    "\n",
    "# Fusers for BGT point data.\n",
    "tree_fuser = fusion.BGTPoleFuser(Labels.TREE, bgt_type='boom', bgt_reader=bgt_pole_reader, ahn_reader=ahn_reader, params=tree_params)\n",
    "light_fuser = fusion.BGTPoleFuser(Labels.STREET_LIGHT, bgt_type='lichtmast', bgt_reader=bgt_pole_reader, ahn_reader=ahn_reader, params=light_params)\n",
    "sign_fuser = fusion.BGTPoleFuser(Labels.TRAFFIC_SIGN, bgt_type='verkeersbord', bgt_reader=bgt_pole_reader, ahn_reader=ahn_reader, params=sign_params)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2282f973",
   "metadata": {},
   "source": [
    "### Street furniture\n",
    "\n",
    "Finally we label street furniture objects by matching clusters above the ground with reference BGT points.\n",
    "\n",
    "For details and more options, see notebook [6. Street furniture](6.%20Street%20furniture.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e63a9ec9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Object type-specific parameters.\n",
    "rubbish_bin_params = {'min_height': 0.5, 'max_height': 1.5, 'min_width': 0.3, 'max_width': 1.5, 'min_length': 0.3, 'max_length': 1.5}\n",
    "city_bench_params = {'min_height': 0.5, 'max_height': 1.5, 'min_width': 0.3, 'max_width': 2.0, 'min_length': 1.0, 'max_length': 3.5}\n",
    "\n",
    "# Create fusers for BGT point data.\n",
    "rubbish_bin_fuser = fusion.BGTStreetFurnitureFuser(Labels.RUBBISH_BIN, bgt_type='afvalbak', bgt_reader=bgt_street_furniture_reader,\n",
    "                                                   ahn_reader=ahn_reader, params=rubbish_bin_params)\n",
    "city_bench_fuser = fusion.BGTStreetFurnitureFuser(Labels.CITY_BENCH, bgt_type='bank', bgt_reader=bgt_street_furniture_reader,\n",
    "                                                  ahn_reader=ahn_reader, params=city_bench_params)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "literary-source",
   "metadata": {},
   "source": [
    "---\n",
    "## Region Growing\n",
    "\n",
    "We use region growing based on labelling of connected components. For details and other options see notebook [Region growing](Region%20growing.ipynb).\n",
    "\n",
    "### Buildings\n",
    "\n",
    "We use region growing to refine the buildings. This will make sure that protruding elements such as balconies are labelled correctly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bfe6c884-319c-42e1-9534-b438c323646a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# We do this separately for the top and bottom since the best settings for each might differ.\n",
    "building_top = {'bottom': 12., 'grid_size': 0.1, 'threshold': 0.5}\n",
    "building_bottom = {'bottom': 0., 'top': 12., 'grid_size': 0.05, 'threshold': 0.5}\n",
    "\n",
    "building_grower = growing.LayerLCC(Labels.BUILDING, ahn_reader, \n",
    "                                   params=[building_top, building_bottom])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eec09ab7-c0d3-4e0f-9412-fd714c05e1d2",
   "metadata": {},
   "source": [
    "### Pole-like objects\n",
    "\n",
    "We also use region growing to refine the pole-like objects. We do this separately for the top and bottom since the best settings for each might differ.\n",
    "\n",
    "For details and more options, see notebook [5. Pole-like objects](5.%20Pole-like%20objects.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "48e1670c-58c8-476a-ae85-5b0ab7ec241c",
   "metadata": {},
   "outputs": [],
   "source": [
    "light_bottom = {'top': 1.75, 'grid_size': 0.05, 'threshold': 0.5}\n",
    "light_top = {'bottom': 1.75, 'grid_size': 0.1, 'threshold': 0.05}\n",
    "\n",
    "sign_bottom = {'top': 1.5, 'grid_size': 0.05, 'threshold': 0.5}\n",
    "sign_top = {'bottom': 1.5, 'grid_size': 0.1, 'threshold': 0.05}\n",
    "\n",
    "tree_bottom = {'top': 1.75, 'grid_size': 0.05, 'threshold': 0.8}\n",
    "tree_middle = {'bottom': 1.75, 'top':10.0, 'grid_size': 0.1, 'threshold': 0.01}\n",
    "tree_top = {'bottom': 6.0, 'grid_size': 0.2, 'threshold': 0.01}\n",
    "\n",
    "light_grower = growing.LayerLCC(Labels.STREET_LIGHT, ahn_reader, \n",
    "                                params=[light_bottom, light_top])\n",
    "sign_grower = growing.LayerLCC(Labels.TRAFFIC_SIGN, ahn_reader, \n",
    "                               params=[sign_bottom, sign_top])\n",
    "tree_grower = growing.LayerLCC(Labels.TREE, ahn_reader, reset_noise=True,\n",
    "                               params=[tree_bottom, tree_middle, tree_top])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c2427a7-6113-44fe-9ca9-fd74ebb08659",
   "metadata": {},
   "source": [
    "---\n",
    "## Set-up the pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "820b612b-375e-48f6-a8bb-9261fa556fc1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set-up pipeline.\n",
    "process_sequence = (npz_ground_fuser, road_part_fuser, noise_filter, bgt_building_fuser, cable_fuser,\n",
    "                    car_fuser, tree_fuser, light_fuser, sign_fuser, rubbish_bin_fuser, \n",
    "                    city_bench_fuser, building_grower, light_grower, sign_grower, tree_grower)\n",
    "pipeline = Pipeline(processors=process_sequence, ahn_reader=ahn_reader, caching=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "collaborative-durham",
   "metadata": {},
   "source": [
    "---\n",
    "## Process a single file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "14fcbc07-1514-4037-8f1f-2ea4beaefc1c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Select the file to process.\n",
    "tile_code = '2386_9702'  # Demo tile 1.\n",
    "# tile_code = '2397_9705'  # Demo tile 2.\n",
    "\n",
    "in_file = '../datasets/pointcloud/filtered_' + tile_code + '.laz'\n",
    "out_file = '../datasets/pointcloud/processed_' + tile_code + '.laz'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "involved-palestine",
   "metadata": {},
   "outputs": [],
   "source": [
    "pipeline.process_file(in_file, out_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "343b1355-d6a7-406f-ab95-e75bee962229",
   "metadata": {},
   "source": [
    "---\n",
    "## Process a folder\n",
    "\n",
    "It is also possible to process all LAS files in a folder in one batch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b8c1b35f-f83b-4ce8-9733-217ab9b554a0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Select the folder to process.\n",
    "las_folder = '../datasets/pointcloud/'\n",
    "out_folder = '../datasets/pointcloud/'\n",
    "# Only files starting with the specified prefix will be processed.\n",
    "in_prefix = 'filtered_'\n",
    "# The prefix will be replaced for processed files (otherwise the input might be overwritten).\n",
    "out_prefix = 'processed_'\n",
    "\n",
    "# Suppress logging other than ERROR messages to console.\n",
    "log_utils.set_console_level(level=logging.ERROR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9d5a34cc-97d2-4711-81d2-048ffc62336b",
   "metadata": {},
   "outputs": [],
   "source": [
    "pipeline.process_folder(in_folder=las_folder, out_folder=out_folder, in_prefix=in_prefix, out_prefix=out_prefix)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "660ca8b6-26a2-4ee3-87b1-30e3bcda1104",
   "metadata": {},
   "source": [
    "---\n",
    "## View the result\n",
    "\n",
    "To get a quick look at the result, we can plot a slice of the labelled point cloud at a specified height."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "44123d20-b21e-422c-915a-05baa4da56c4",
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib widget\n",
    "plot_utils.plot_bgt_and_cloudslice(tile_code, out_file, ahn_reader, building_file=bgt_building_file,\n",
    "                                   road_file=bgt_road_file, pole_file=bgt_pole_file,\n",
    "                                   street_furniture_file=bgt_street_furniture_file,\n",
    "                                   plane_height=.75, hide_noise=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e25448fe-02bf-4223-87bb-01b195bf088a",
   "metadata": {},
   "source": [
    "This will look like:\n",
    "\n",
    "![BGT and LAS labels](https://github.com/Amsterdam-AI-Team/Urban_PointCloud_Processing/raw/main/media/examples/bgt_cloud_slice.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "814efd8d-09cf-4697-9e58-3c8d2ae3e984",
   "metadata": {},
   "source": [
    "### In CloudCompare\n",
    "The final result can also be viewed in CloudCompare. Open the out_file (e.g. datasets/pointcloud/labelled_2386_9702.laz) and change `Colors` from \"RGB\" to \"Scalar field\". For best results, use our [custom color scale](https://github.com/Amsterdam-AI-Team/Urban_PointCloud_Processing/raw/main/media/cc_color_scale.xml), and set \"steps\" to 101.\n",
    "\n",
    "The result should look like this:\n",
    "![Demo result](https://github.com/Amsterdam-AI-Team/Urban_PointCloud_Processing/raw/main/media/examples/demo_result.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "hawaiian-inside",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  },
  "vscode": {
   "interpreter": {
    "hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}