A Stereo Perception Framework with Depth Estimation for Autonomous Vehicles: Developed a stereo vision-based perception framework for autonomous vehicles using three deep neural networks simultaneously to perform free space detection, lane boundary detection, and object detection on image frames captured using the stereo camera. @This is the implementation of "A Stereo Perception Framework for Autonomous Vehicles", Narsimlu Kemsaram, Anweshan Das, Gijs Dubbelman.
- NVIDIA DRIVE™ PX 2 platform with the latest PDK flashed in the system.
- Desktop PC running Linux x86/x64
These are the basic prerequisites for Linux:
-
Ubuntu Linux 16.04 or 14.04 (out of the box installation)
-
GCC >= 4.8.X && GCC <= 4.9.x (for 16.04, use GCC 4.9.3 or greater)
-
cmake version >= 3.3
By default, Ubuntu 14.04 installs cmake version 2.8. For guidance on installing cmake 3.x, see:
http://askubuntu.com/questions/6102 91/how-to-install-cmake-3-2-on-ubuntu-14-04 -
NVIDIA® CUDA® Toolkit version 9.0 or later
-
NVIDIA DRIVE™ 5.0 Linux SDK/PDK installation on the Linux Host
-
You may also need to install (using
apt-get install) the following packages:libx11-dev libxrandr-dev libxcursor-dev libxxf86vm-dev libxinerama-dev libxi-dev libglu1-mesa-dev libglew-dev
Desktop development relies on NVCUVID for video decoding, which is included with
the NVIDIA drivers. In general, the cmake build scripts can find NVCUVID
installation. However, if this fails, you must set a symbolic link
/usr/lib/nvidia-current pointing to your NVIDIA driver lib, for instance to
/usr/lib/nvidia-384.
The DriveWorks SDK samples use a standard cmake-based build system. The default
approach is to create a build folder for an "out of the source tree build"
and to point cmake to build the samples from there.
-
On the host system, enter:
$ mkdir build $ cd build $ cmake -DCMAKE_BUILD_TYPE=Release /path/to/driveworks/samples/folder $ make -j
To compile the samples or any user code using DriveWorks, you must use cross-compilation from a Linux host. For this, in addition to the requirements above, you must have the additional components:
- DriveWorks SDK cross-compilation package
- NVIDIA DRIVE SDK/PDK installation for NVIDIA DRIVE PX 2 in the Linux Host
- CUDA cross-compilation libraries.
-
On the host system, install CUDA cross-compilation libraries.
$ sudo dpkg --add-architecture arm64 $ sudo apt-get update $ sudo apt-get install cuda-cross-aarch64-8-0@note Some repositories may not have binaries for arm64 and apt-get update causes errors. To prevent these errors, edit
/etc/apt/sources.listand files under/etc/apt/sources.list.d/and add[arch=amd64,i386]into each line starting withdeb. Ensure you specifyamd64and notarm64. -
Configure the location of the NVIDIA DRIVE SDK/PDK.
Set the environment variable
DriveSDKto point to the SDK/PDK folders:/path/to/drive-t186ref-linux (NVIDIA DRIVE PX 2)-or-
Define this location as a
cmakeparameter:-DVIBRANTE_PDK:STRING=/path/to/drive-t186ref-linux -
Cross-compile.
$ mkdir build $ cd build $ cmake -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_TOOLCHAIN_FILE=/path/to/samples/cmake/Toolchain-V4L.cmake \ -DVIBRANTE_PDK:STRING=/path/to/drive-t186ref-linux \ /path/to/driveworks/samples/folder $ make -j $ make installTo streamline the deploy process, you can set the
cmakedefinitions as environment variables.
After cross-compiling your custom DriveWorks sample on the Linux host system, you must load it to the target platform. You can use automatic upload (recommended) or manually upload it.
The package-based installation will install libraries, binaries and resources in
/usr/local/driveworks-<version>. However, if you build a custom DriveWorks
sample, you should upload those files to a separate directory such as:
/home/nvidia/driveworks
The above path assumes your user name is "nvidia". Of course, you would use
a name more appropriate to your situation. Using this path rather than
/usr/local/<name> avoids security constraints that can make the upload process
more difficult.
@note Important: The correct rpath for the binaries is set only during
installation. As such, NVIDIA discourages copying pre-install binaries to the
board because this could result in undefined behavior.
An upload target is provided as a convenience. Invoking this target uses
rsync to sync the builds to a specified location on the device. target is
set to depend on building and packaging.
The following cmake parameters configure the upload target:
VIBRANTE_HOST: IP address of the board, the default value is192.168.10.10.VIBRANTE_INSTALL_PATH: destination folder of the compiled products. The default value is/home/nvidia/driveworks.VIBRANTE_USER: Specifies the user that will be used torsyncthe binaries. The default value isnvidia.VIBRANTE_PASSWORD: Password forVIBRANTE_USER. The default value isnvidia.VIBRANTE_PORT: Port on whichsshis listening on the target board. The default is22.
-
Cross-compile the binaries as described in Cross Compiling for the NVIDIA DRIVE Platform.
-
On the Linux system, enter:
$ make upload
@note If the VIBRANTE_HOST parameter is set to the IP address of a running
NVIDIA DRIVE PX 2 board, users can call the upload target to rsync the build folder
with a folder on the board.
When you manually upload the sample to the target device, you copy the cross-
compiled binaries to a device with IP DPX_IP as user nvidia.
-
Cross-compile the binaries as described in Cross Compiling for the NVIDIA DRIVE Platform
-
On the host system, execute:
$ scp -r install/bin/* nvidia@DPX_IP:/home/nvidia/<destination_dir>/
If you use this code for your research, please kindly cite:
@INPROCEEDINGS{9128899, author={Kemsaram, Narsimlu and Das, Anweshan and Dubbelman, Gijs}, booktitle={2020 IEEE 91st Vehicular Technology Conference (VTC2020-Spring)}, title={A Stereo Perception Framework for Autonomous Vehicles}, year={2020}, volume={}, number={}, pages={1-6}, doi={10.1109/VTC2020-Spring48590.2020.9128899}}