Integration of Curiosity Rover demo with Nvidia's Isaac Sim

[franklinselva]

Feature description

Feature description

The feature submission is to extend the Curiosity Rover demos to Nvidia's Isaac Sim. The gazebo simulation has been one of the most common simulators when it comes to ROS applications while Nvidia's Isaac Sim has started to receive better traction towards robotics applications. The idea here is to extend the existing demos to the new simulation tool which uses PhysX as its engine for high-performance real-time simulations.

Below is the simulation snapshots acquired from Nvidia's Isaac Sim for the feature.

The integration allows to control the rover using teleoperation while the tool arm and the mast arm subscribes to Joint states. The camera allow streaming of photorealistic images to ROS2 interface

Meta Info

This feature submission is part of the NASA Space ROS SIM Challenge
freelancer id: selvakumarhs

Implementation considerations

There are a few considerations taken into account while developing the features which are listed below.

• Gazebo is known for its CPU-based physics calculations and uses GPU for visual rendering. While this doesn't affect the simulations in several use cases, there are a few scenarios where the simulation hits its limits like in photo-realistic rendering and GPU-based physics. On the other hand, Nvidia's IsaacSim supports photorealism and GPU-based physics while it requires an RTX Graphics Card for better simulation results.
• While Gazebo is completely open-source, IsaacSim has most of the components available in open-source.
• While Space applications mostly still lean towards deterministic systems for their safety-critical needs, there are approaches explored using non-deterministic approaches with Machine learning and AI. The AI and perception can be leveraged by photo-realistic simulations offered by Nvidia.
• The tradeoff between GPU and CPU computations depends on the scale of the application being tested and this feature allows the user to choose the computation necessary for the applications.

To get the best of both worlds when it comes to simulation, I have proposed a common interface to allow the demo packages to be run on both Gazebo and IsaacSim.

graph LR
    A[GazeboSim] <--> B[Ignition ROS2]
    B <--ROS2 control--> C[Common Package]
    C <--> D[Demo Packages]
    D <--> E[Space ROS2]

    C <--Isaac ROS2 Bridge--> F[Isaac Sim ROS2 Extension]
    F <--> G[Isaac Sim]


    subgraph Host Machine - GazeboSim
        A
        B
    end

    subgraph Host Machine - Isaac Sim
        F
        G
    end

    subgraph Space ROS Container
        C
        D
        E
    end

Loading

The above graph presents the design choices made for the feature to allow the demo packages to be run over more than a single simulation ecosystem. Moreover, the above approach also takes containerization into account providing directions to replace the Gazebo and IsaacSim with its containers.