Merge branch 'feature-omni-2022' into develop

README.md

@@ -1,4 +1,4 @@
-<p align=center><strong>~ Our <a href="https://eval.ai/web/challenges/challenge-page/1614/overview">Robotic Vision Scene Understanding (RVSU) Challenge is live on EvalAI</a> ~<br>(prizes include RTX A6000 NVIDIA GPUs and $2,500USD cash)</strong></p>
+<p align=center><strong>~ Our <a href="https://eval.ai/web/challenges/challenge-page/1614/overview">Robotic Vision Scene Understanding (RVSU) Challenge is live on EvalAI</a> ~<br>(prizes include RTX A6000 NVIDIA GPUs)</strong></p>
 <p align=center><strong>~ BenchBot is now powered by <a href="https://developer.nvidia.com/nvidia-omniverse-platform">NVIDIA Omniverse</a> and <a href="https://developer.nvidia.com/isaac-sim">Isaac Sim</a>. We are aware of some issues, please <a href="https://github.com/qcr/benchbot/issues">report any you do encounter</a>. ~</strong></p>
 <p align=center><strong>~ Our <a href="https://github.com/qcr/benchbot/wiki/Tutorial:-Performing-Semantic-SLAM-with-Votenet">BenchBot tutorial</a> is the best place to get started developing with BenchBot ~</strong></p>
 
@@ -25,18 +25,18 @@ This repository contains the software stack needed to develop solutions for Benc
 
 The BenchBot software stack is designed to run seamlessly on a wide number of system configurations (currently limited to Ubuntu 18.04+). System hardware requirements are relatively high due to the software run for 3D simulation (e.g. NVIDIA Omniverse-powered Isaac Sim):
 
-- Nvidia Graphics card (GeForce RTX 2070 minimum, GeForce GTX 3080+ / RTX A6000+ recommended)
+- Nvidia Graphics card (GeForce RTX 2080 minimum, Quadro RTX 5000 recommended)
 - CPU with multiple cores (Intel i7-6800K 7th Generation minimum)
 - 32GB+ RAM
 - 64GB+ spare storage (an SSD storage device is **strongly** recommended)
 
 Having a system that meets the above hardware requirements is all that is required to begin installing the BenchBot software stack. The install script analyses your system configuration and offers to install any missing software components interactively. The list of 3rd party software components involved includes:
 
-- NVIDIA GPU Driver (470.57+ recommended)
+- NVIDIA GPU Driver (520.60.11+ recommended)
 - CUDA with GPU support (10.0+ required, 10.1+ recommended)
 - Docker Engine - Community Edition (19.03+ required, 19.03.2+ recommended)
 - NVIDIA Container Toolkit (1.0+ required, 1.0.5+ recommended)
-- Isaac 2021.2 Omniverse simulator (when installing `sim_omni`)
+- Isaac 2022.2.1 Omniverse simulator (when installing `sim_omni`)
 
 ## Managing your installation
 
@@ -170,6 +170,21 @@ To use our system, we just ask that you cite our paper on the BenchBot system. T
 }
 ```
 
+If you use our benchbot environments for active robotics (BEAR) which are installed by default, we ask you please cite our data paper on BEAR. Citation details are as follows:
+
+```
+@article{hall2022bear,
+author = {David Hall and Ben Talbot and Suman Raj Bista and Haoyang Zhang and Rohan Smith and Feras Dayoub and Niko Sünderhauf},
+title ={BenchBot environments for active robotics (BEAR): Simulated data for active scene understanding research},
+journal = {The International Journal of Robotics Research},
+volume = {41},
+number = {3},
+pages = {259-269},
+year = {2022},
+doi = {10.1177/02783649211069404},
+}
+```
+
 ## Components of the BenchBot software stack
 
 The BenchBot software stack is split into a number of standalone components, each with their own GitHub repository and documentation. This repository glues them all together for you into a working system. The components of the stack are:

bin/benchbot_install

@@ -25,7 +25,7 @@ $URL_DEBUG $HOSTNAME_DEBUG"
 # Defaults for the installation process
 ADDONS_DEFAULT='benchbot-addons/ssu'
 CUDA_DEFAULT='cuda'
-NVIDIA_DEFAULT='nvidia-driver-510'
+NVIDIA_DEFAULT='nvidia-driver-525'
 SIMULATOR_DEFAULT='sim_omni'
 
 ################################################################################
@@ -150,13 +150,13 @@ checks_list_pre=(
   "nvidiacard"  # Is a valid graphics card available?
   "nvidiadriver"  # Is the driver available?
   "nvidiaversion"  # Is the driver version valid?
-  "nvidiappa"  # Does the driver come from a supported PPA?
-  "cudadriversavailable" # Is cuda-drivers package available?
-  "cudadriversversion"  # Is cuda-drivers version valid?
-  "cudadriversppa"  # Is cuda-drivers from the Nvidia PPA?
-  "cudaavailable"  # Is CUDA available?
+  # "nvidiappa"  # Does the driver come from a supported PPA?
+  #"cudadriversavailable" # Is cuda-drivers package available?
+  #"cudadriversversion"  # Is cuda-drivers version valid?
+  # "cudadriversppa"  # Is cuda-drivers from the Nvidia PPA?
+  #"cudaavailable"  # Is CUDA available?
   "cudaversion"  # Is the cuda version valid?
-  "cudappa"  # Is cuda from the Nvidia PPA?
+  # "cudappa"  # Is cuda from the Nvidia PPA?
   "Running Docker related system checks:"
   "dockeravailable"  # Is Docker available?
   "dockerversion"  # Is Docker version > 19.0?
@@ -244,31 +244,31 @@ chk_nvidiaversion_issue="\
 chk_nvidiaversion_fix="$chk_nvidiadriver_fix"
 chk_nvidiaversion_reboot=0
 
-chk_nvidiappa_name='NVIDIA driver from a standard PPA'
-chk_nvidiappa_pass='PPA is valid'
-chk_nvidiappa_fail='PPA is invalid'
-chk_nvidiappa_check='version=$(apt list --installed 2>/dev/null | '\
-'grep nvidia-driver- | sed '"'"'s/.*now \([^ ]*\).*/\1/'"'"'); '\
-'apt download --print-uris nvidia-driver-$(echo $version | '\
-'cut -d'"'"'.'"'"' -f1)=$version 2>/dev/null | cut -d'"'"' '"'"' -f1'
-chk_nvidiappa_eval=\
-'[[ "$check_result" =~ .*('"'"'developer.download.nvidia.com/'"'"'| \
-'"'"'ppa.launchpad.net/graphics-drivers/ppa/ubuntu'"'"').* ]]'
-chk_nvidiappa_issue="\
-  The installed NVIDIA driver was detected to have come from somewhere other
-  than the NVIDIA / Ubuntu graphics-drivers PPAs. The driver was sourced from:
-    "'$check_result'"
-
-  Our Docker container only supports official drivers, & GPUs only work in 
-  containers if the driver version exactly matches the version of the host.
-  Please use a driver from either of the following locations (the NVIDIA PPA is
-  preferred):
-    http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64
-    http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu"
-chk_nvidiappa_fix=\
-'sudo apt remove -y nvidia-driver-* && sudo apt -y autoremove; '\
-"$chk_nvidiadriver_fix"
-chk_nvidiappa_reboot=0
+# chk_nvidiappa_name='NVIDIA driver from a standard PPA'
+# chk_nvidiappa_pass='PPA is valid'
+# chk_nvidiappa_fail='PPA is invalid'
+# chk_nvidiappa_check='version=$(apt list --installed 2>/dev/null | '\
+# 'grep nvidia-driver- | sed '"'"'s/.*now \([^ ]*\).*/\1/'"'"'); '\
+# 'apt download --print-uris nvidia-driver-$(echo $version | '\
+# 'cut -d'"'"'.'"'"' -f1)=$version 2>/dev/null | cut -d'"'"' '"'"' -f1'
+# chk_nvidiappa_eval=\
+# '[[ "$check_result" =~ .*('"'"'developer.download.nvidia.com/'"'"'| \
+# '"'"'ppa.launchpad.net/graphics-drivers/ppa/ubuntu'"'"').* ]]'
+# chk_nvidiappa_issue="\
+#   The installed NVIDIA driver was detected to have come from somewhere other
+#   than the NVIDIA / Ubuntu graphics-drivers PPAs. The driver was sourced from:
+#     "'$check_result'"
+
+#   Our Docker container only supports official drivers, & GPUs only work in 
+#   containers if the driver version exactly matches the version of the host.
+#   Please use a driver from either of the following locations (the NVIDIA PPA is
+#   preferred):
+#     http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64
+#     http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu"
+# chk_nvidiappa_fix=\
+# 'sudo apt remove -y nvidia-driver-* && sudo apt -y autoremove; '\
+# "$chk_nvidiadriver_fix"
+# chk_nvidiappa_reboot=0
 
 chk_cudadriversavailable_name='CUDA drivers installed'
 chk_cudadriversavailable_pass='Drivers found'
@@ -300,30 +300,30 @@ chk_cudadriversversion_fix=\
 "${chk_nvidiadriver_fix//$NVIDIA_DEFAULT/cuda-drivers}"
 chk_cudadriversversion_reboot=1
 
-chk_cudadriversppa_name='CUDA drivers from the NVIDIA PPA'
-chk_cudadriversppa_pass='PPA is valid'
-chk_cudadriversppa_fail='PPA is invalid'
-chk_cudadriversppa_check=\
-'installed=$(apt list --installed 2>/dev/null | grep -E "cuda-drivers/" | \
-awk '"'"'{print $2}'"'"'); \
-apt download --print-uris cuda-drivers=\
-$(echo $installed | sed '"'"'s/.*now \([^ ]*\).*/\1/'"'"') 2>/dev/null | \
-cut -d'"'"' '"'"' -f1'
-chk_cudadriversppa_eval=\
-'[[ "$check_result" =~ .*('"'"'developer.download.nvidia.com/'"'"').* ]]'
-chk_cudadriversppa_issue="\
-  CUDA drivers were detected to have come from somewhere other than the NVIDIA 
-  PPAs. The CUDA drivers were sourced from:
-    "'$check_result'"
-
-  Our Docker container only supports official CUDA installs, & GPUs only work
-  in containers if the CUDA drivers version exactly matches the version of the
-  host.  Please install CUDA drivers from the official NVIDIA repository:
-    http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64"
-chk_cudadriversppa_fix=\
-'sudo apt remove -y cuda-drivers && sudo apt -y autoremove; '\
-"${chk_nvidiadriver_fix//$NVIDIA_DEFAULT/cuda-drivers}"
-chk_cudadriversppa_reboot=1
+# chk_cudadriversppa_name='CUDA drivers from the NVIDIA PPA'
+# chk_cudadriversppa_pass='PPA is valid'
+# chk_cudadriversppa_fail='PPA is invalid'
+# chk_cudadriversppa_check=\
+# 'installed=$(apt list --installed 2>/dev/null | grep -E "cuda-drivers/" | \
+# awk '"'"'{print $2}'"'"'); \
+# apt download --print-uris cuda-drivers=\
+# $(echo $installed | sed '"'"'s/.*now \([^ ]*\).*/\1/'"'"') 2>/dev/null | \
+# cut -d'"'"' '"'"' -f1'
+# chk_cudadriversppa_eval=\
+# '[[ "$check_result" =~ .*('"'"'developer.download.nvidia.com/'"'"').* ]]'
+# chk_cudadriversppa_issue="\
+#   CUDA drivers were detected to have come from somewhere other than the NVIDIA 
+#   PPAs. The CUDA drivers were sourced from:
+#     "'$check_result'"
+
+#   Our Docker container only supports official CUDA installs, & GPUs only work
+#   in containers if the CUDA drivers version exactly matches the version of the
+#   host.  Please install CUDA drivers from the official NVIDIA repository:
+#     http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64"
+# chk_cudadriversppa_fix=\
+# 'sudo apt remove -y cuda-drivers && sudo apt -y autoremove; '\
+# "${chk_nvidiadriver_fix//$NVIDIA_DEFAULT/cuda-drivers}"
+# chk_cudadriversppa_reboot=1
 
 chk_cudaavailable_name='CUDA is installed'
 chk_cudaavailable_pass='CUDA found'
@@ -352,29 +352,29 @@ chk_cudaversion_issue="\
 chk_cudaversion_fix="${chk_nvidiadriver_fix//$NVIDIA_DEFAULT/$CUDA_DEFAULT}"
 chk_cudaversion_reboot=1
 
-chk_cudappa_name='CUDA is from the NVIDIA PPA'
-chk_cudappa_pass='PPA is valid'
-chk_cudappa_fail='PPA is invalid'
-chk_cudappa_check=\
-'installed=$(apt list --installed 2>/dev/null | grep -E "cuda-$(\
-/usr/local/cuda/bin/nvcc --version | grep release | \
-sed '"'"'s/.*release \([0-9\.]*\).*/\1/'"'"')"); \
-apt download --print-uris "$(echo "$installed" | sed '"'"'s/\([^\/]*\).*/\1/'"'"')=$(echo "$installed" | cut -d " " -f 2)" 2>/dev/null'
-chk_cudappa_eval=\
-'[[ "$check_result" =~ .*('"'"'developer.download.nvidia.com/'"'"').* ]]'
-chk_cudappa_issue="\
-  CUDA was detected to have come from somewhere other than the NVIDIA PPAs. The 
-  CUDA package was sourced from:
-    "'$check_result'"
-
-  Our Docker container only supports official CUDA installs, & GPUs only work in 
-  containers if the CUDA version exactly matches the version of the host.
-  Please install CUDA from the official NVIDIA repository:
-    http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64"
-chk_cudappa_fix=\
-'sudo apt remove -y cuda-10* && sudo apt -y autoremove; '\
-"${chk_nvidiadriver_fix//$NVIDIA_DEFAULT/$CUDA_DEFAULT}"
-chk_cudappa_reboot=1
+# chk_cudappa_name='CUDA is from the NVIDIA PPA'
+# chk_cudappa_pass='PPA is valid'
+# chk_cudappa_fail='PPA is invalid'
+# chk_cudappa_check=\
+# 'installed=$(apt list --installed 2>/dev/null | grep -E "cuda-$(\
+# /usr/local/cuda/bin/nvcc --version | grep release | \
+# sed '"'"'s/.*release \([0-9\.]*\).*/\1/'"'"')"); \
+# apt download --print-uris "$(echo "$installed" | sed '"'"'s/\([^\/]*\).*/\1/'"'"')=$(echo "$installed" | cut -d " " -f 2)" 2>/dev/null'
+# chk_cudappa_eval=\
+# '[[ "$check_result" =~ .*('"'"'developer.download.nvidia.com/'"'"').* ]]'
+# chk_cudappa_issue="\
+#   CUDA was detected to have come from somewhere other than the NVIDIA PPAs. The 
+#   CUDA package was sourced from:
+#     "'$check_result'"
+
+#   Our Docker container only supports official CUDA installs, & GPUs only work in 
+#   containers if the CUDA version exactly matches the version of the host.
+#   Please install CUDA from the official NVIDIA repository:
+#     http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64"
+# chk_cudappa_fix=\
+# 'sudo apt remove -y cuda-10* && sudo apt -y autoremove; '\
+# "${chk_nvidiadriver_fix//$NVIDIA_DEFAULT/$CUDA_DEFAULT}"
+# chk_cudappa_reboot=1
 
 chk_dockeravailable_name='Docker is available'
 chk_dockeravailable_pass='Found'

bin/benchbot_run

@@ -395,7 +395,7 @@ while [ -n "$(docker ps -q -f 'name='$HOSTNAME_SUPERVISOR)" ] && \
   [ -n "$(docker ps -q -f 'name='$HOSTNAME_ROBOT)" ]); do
   sleep 1
 done
-# sleep infinity
+#sleep infinity
 
 if [ "$simulator_required" -eq 0 ] && \
     [ -z "$(docker ps -q -f 'name='$HOSTNAME_ROBOT)" ]; then

docker/core.Dockerfile

@@ -1,5 +1,5 @@
 # Start from the official Ubuntu image
-FROM ubuntu:bionic
+FROM ubuntu:focal
 
 # Setup a base state with needed packages & useful default settings
 SHELL ["/bin/bash", "-c"]
@@ -19,10 +19,10 @@ ARG CUDA_VERSION
 ENV NVIDIA_VISIBLE_DEVICES="all"
 ENV NVIDIA_DRIVER_CAPABILITIES="compute,display,graphics,utility"
 RUN add-apt-repository ppa:graphics-drivers && \
-    wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin && \
-    mv -v cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600 && \
-    apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub && \
-    add-apt-repository -n "deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/ /" && \
+    wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin && \
+    mv -v cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600 && \
+    apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub && \
+    add-apt-repository -n "deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /" && \
     apt update
 RUN CUDA_NAME="cuda-$(echo "${CUDA_VERSION}" | \
     sed 's/\([0-9]*\)\.\([0-9]*\).*/\1\.\2/; s/\./-/')" && \

docker/shared_tools.Dockerfile

@@ -6,21 +6,21 @@
 ENV BENCHBOT_DIR="/benchbot"
 RUN mkdir -p $BENCHBOT_DIR
 
-# Install ROS Melodic
+# Install ROS Noetic
 RUN apt update && apt install -y curl && \
-    echo "deb http://packages.ros.org/ros/ubuntu bionic main" > \
+    echo "deb http://packages.ros.org/ros/ubuntu focal main" > \
     /etc/apt/sources.list.d/ros-latest.list && \
     curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | \
     apt-key add - && apt update && \
-    apt install -y ros-melodic-ros-base python-rosdep \
-    python-rosinstall python-rosinstall-generator python-wstool \
-    python-catkin-tools python-pip build-essential \
-    ros-melodic-tf2-ros ros-melodic-tf
+    apt install -y ros-noetic-ros-base python3-rosdep \
+    python3-rosinstall python3-rosinstall-generator python3-wstool \
+    python3-catkin-tools python3-pip build-essential \
+    ros-noetic-tf2-ros ros-noetic-tf
 
 # Build a ROS Catkin workspace
 ENV ROS_WS_PATH="$BENCHBOT_DIR/ros_ws"
 RUN rosdep init && rosdep update && mkdir -p $ROS_WS_PATH/src && \
-    source /opt/ros/melodic/setup.bash && \
+    source /opt/ros/noetic/setup.bash && \
     pushd $ROS_WS_PATH && catkin_make && source devel/setup.bash && popd 
 
 # Add BenchBot's common ROS packages
@@ -37,8 +37,12 @@ ARG BENCHBOT_CONTROLLER_HASH
 ENV BENCHBOT_CONTROLLER_PATH="$BENCHBOT_DIR/benchbot_robot_controller"
 RUN git clone $BENCHBOT_CONTROLLER_GIT $BENCHBOT_CONTROLLER_PATH && \
     pushd $BENCHBOT_CONTROLLER_PATH && git checkout $BENCHBOT_CONTROLLER_HASH && \
-    pip install -r requirements.txt && pushd $ROS_WS_PATH && \
-    pushd src && git clone https://github.com/eric-wieser/ros_numpy && popd && \
+    pip install -r requirements.txt && \
+    sed -i 's/np.float/float/g' /usr/local/lib/python3.8/dist-packages/transforms3d/quaternions.py && \ 
+    pushd $ROS_WS_PATH && \
+    pushd src && git clone https://github.com/eric-wieser/ros_numpy && \
+    sed -i 's/np.float/float/g' /benchbot/ros_ws/src/ros_numpy/src/ros_numpy/point_cloud2.py && \
+    popd && \
     ln -sv $BENCHBOT_CONTROLLER_PATH src/ && source devel/setup.bash && catkin_make
 
 # Create a place to mount our add-ons, & install manager dependencies

docker/sim_omni.Dockerfile

@@ -1,15 +1,18 @@
 # Extend NVIDIA's official Docker Image for Isaac Sim. Download instructions:
 #   https://catalog.ngc.nvidia.com/orgs/nvidia/containers/isaac-sim
-FROM  nvcr.io/nvidia/isaac-sim:2021.2.1
+FROM  nvcr.io/nvidia/isaac-sim:2022.2.1
 
 # Fix to address key rotation breaking APT with the official Isaac Sim image
 #   https://developer.nvidia.com/blog/updating-the-cuda-linux-gpg-repository-key/
-RUN apt-key adv --fetch-keys \
-    https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub
+# RUN apt-key adv --fetch-keys \
+#     https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub
 
 # Fix scripts provided with image
-RUN sed -i 's/$@/"\0"/' python.sh
-RUN sed -i 's/sleep/# \0/' start_nucleus.sh
+# RUN sed -i 's/$@/"\0"/' python.sh
+# RUN sed -i 's/sleep/# \0/' start_nucleus.sh
+
+RUN echo "$TZ" > /etc/timezone && ln -s /usr/share/zoneinfo/"$TZ" \
+    /etc/localtime && apt update && apt -y install tzdata
 
 # Overrides to make things play nicely with the BenchBot ecosystem
 SHELL ["/bin/bash", "-c"]

docker/submission.Dockerfile

@@ -4,7 +4,7 @@ FROM benchbot/core:base
 # Install some requirements for BenchBot API & visualisation tools
 # (BenchBot supports both python2 & python3, but python3 is preferred)
 RUN apt update && apt install -y libsm6 libxext6 libxrender-dev python3 \
-    python3-pip python3-tk python-pip python-tk
+    python3-pip python3-tk
 
 # Upgrade to latest pip (OpenCV fails to install because the pip installed by
 # Ubuntu is so old). See following issues for details:
@@ -16,7 +16,7 @@ RUN apt update && apt install -y libsm6 libxext6 libxrender-dev python3 \
 #     https://github.com/pypa/pip/issues/5599
 # I'll move on rather than digressing into how stupid it is that that's the
 # state of things...
-RUN pip3 install --upgrade pip
+#RUN pip3 install --upgrade pip
 
 # Install BenchBot API
 ARG BENCHBOT_API_GIT