Point-Voxel CNN TensorFlow

This repo is a TensorFlow 2 Implementation of Point-Voxel CNN for Efficient 3D Deep Learning (see arXiv paper, MIT HAN Lab Repo).

Development is currently in progress!

Prerequisites

This code is built on Google Colab (see build.ipynb). The following libraries must be installed in the Colab environment:

• Python >= 3.7
• TensorFlow == 2.6.2
• TensorFlow I/O == 0.21
• numpy
• tqdm
• plyfile
• h5py

Data Preparation

S3DIS

I re-use the data pre-processing used in PVCNN (see data/s3dis/). One should first download the S3DIS dataset from here, then run

python data/s3dis/prepare_data.py -d [path to unzipped dataset dir]

You can run s3dis_viz.py for a visualizaion of the dataset. Here is one example output with clutter, ceiling, floor, and wall points removed:

Performance of Pretrained Models

This project is still a work in progress. Numerical instabilities while training have impeded full training of the model to obtain performance results. However, the approximate 4x reduction in loss and 35% mean IoU accuracy acheived in the first 2500 iterations of the first epoch (see Figure below) suggests that the model was in fact learning up until crashing (i.e. NaN tensors).

For more details, please see the "Experiment Results and Dicussion" section of the final paper associated with this project.

Evaluating Pretrained Models

In Progress.

Training

In Progress.

License

This repository is released under the MIT license. This includes the license from the original authors. See LICENSE for additional details.

Acknowledgement

The following modules / code-segments were adapted from the official PVCNN implementation (MIT License):

• Data pre-processing.
• Experiment configuration.
• Utility functions used in training (get_configs and get_save_path)