Active panoptic mapping

Setup

Dependencies

This project uses Robostack so only a minimal conda installation such as mambaforge is required to get started.

Installation

1. Create a directory tree for catkin workspace:

   mkdir -p catkin_ws/src 
   cd catkin_ws/src
   

2. Clone this repository in the src directory:

   git clone [email protected]:giuliano-97/active_panoptic_mapping.git
   

3. Make sure mamba is install in your base environment (not needed when using mambaforge), then create the conda environment by running:

   mamba env create -f environment.yml
   

4. Activate the create conda environment:

   conda activate active_panoptic_mapping_env
   

5. Initialize the catkin workspace:

   cd .. && catkin init
   catkin config --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo
   catkin config --merge-devel
   cd src
   

6. Install package dependencies using ros install:

   • If you created a new workspace:

   wstool init . /active_panoptic_mapping/active_panoptic_mapping_ssh.rosinstall
   wstool update
   

   • If you use an existing workspace. Notice that some dependencies require specific branches that will be checked out.

   wstool merge -t . ./active_panoptic_mapping/active_panoptic_mapping.rosinstall
   wstool update
   

7. Apply some minor patches to the eigen_catkin and opencv3_catkin packages to avoid compilation errors:

   bash active_panoptic_mapping/apply_patches.sh
   

8. Compile and source the ros environment:

   catkin build active_panoptic_mapping_utils
   source ../devel/setup.bash
   

Examples

Replica Demo

1. Download the Replica dataset by following the download instructions on the GitHub page.

2. Launch the active_panoptic_mapping_node on the Replica "FRL-Apartment-0" scene:

   roslaunch active_panoptic_mapping_ros run.launch datasets_dir:=$(dirname <PATH_TO_REPLICA_DIR>)
   
   

3. Start publishing simulated sensor data:

   rosservice call /habitat_sim_node/toggle_pub_sensor_data true
   

4. Start the planner:

   rosservice call /active_panoptic_mapping_node/toggle_running true
   

5. By default, this will use the ground truth panoptic labels provided by the simulation. To use panoptic labels predicted online by Mask2Former, you need to additionally pass the use_ground_truth:=false argument:

   roslaunch active_panoptic_mapping_ros run.launch datasets_dir:=$(dirname <PATH_TO_REPLICA_DIR>) use_ground_truth:=false