image generated by DALL-E

When working on my final year project, I stumbled upon some NVIDIA applications that made me really curious. It was a bit confusing at first, but I’ve figured it out now. In this article, I’ll explain what each of these products does, how they’re different from each other, and help you decide which one you might want to use.

Nvidia Omniverse Link to heading

NVIDIA Omniverse™ is a computing platform that enables individuals and teams to develop Universal Scene Description (OpenUSD)-based 3D workflows and applications. – omniverse website

Omniverse Launcher

Think of it as a unique kind of app store, but it’s filled with NVIDIA’s specialized applications, all designed to support and enhance your development process, especially if you’re working with Pixar’s Universal Scene Description (OpenUSD). In essence, Omniverse can be seen as a vital hub for anyone involved in projects like remastering classic games, conducting robotic simulations, and more.

Nvidia Isaac Sim Link to heading

NVIDIA Isaac Sim™ is an extensible robotics simulator that gives you a faster, better way to design, test, and train AI-based robots. It’s powered by Omniverse™ to deliver scalable, photorealistic, and physically accurate virtual environments for building high-fidelity simulations. – isaac sim

Omniverse Isaac Sim

Isaac Sim is a simulator that you can get from the Omniverse Launcher, it is specifically designed for the intricacies of robotics simulation. While you might find some similarities with familiar environments like Unity, Unreal Engine, or Gazebo, Isaac Sim takes a unique stance, especially catering to the needs of the robotics community.

One of the distinctive features of Isaac Sim is its heavy reliance on powerful hardware. Unlike Unity or Unreal Engine, it leverages RTX technology even during the editing process. This ensures that you get to experience and work within highly realistic simulation environments, so you might need a beefy workstation to use it.

Scripting in Isaac Sim takes a different path too. Instead of using C++ or C#, it opts for Python. This flexibility is a boon for developers and researchers who prefer Python for its ease of use and extensive libraries.

Omniverse Isaac Sim

Moreover, Isaac Sim seamlessly integrates with ROS and ROS2, enhancing its utility for robotics applications. This compatibility ensures that developers can easily incorporate Isaac Sim into their existing workflows, making it a powerful tool for a wide range of robotics simulation tasks, from testing algorithms to developing new robotic capabilities.

Isaac Gym Link to heading

When diving into Isaac Gym, we need to distinguish between the two versions that share the name but serve different purposes.

Omniverse Isaac Sim Gym Envs

Firstly, we have the Omniverse Isaac Gym Environment. This version is a sophisticated reinforcement learning environment developed on top of Isaac Sim. Essentially, it’s a specialized scene crafted within Isaac Sim designed specifically for reinforcement learning projects. This integration with Isaac Sim means users can leverage the advanced simulation capabilities of Isaac Sim, tailored for the intricate needs of robotics training and development.

On the other hand, there’s the standalone application known simply as Isaac Gym. This version operates independently from the Omniverse ecosystem and, it’s important to note, is considered outdated. For those keen on reinforcement learning and simulation technology for robotics, the standalone Isaac Gym may not be the best choice due to its lack of updates and support.

For developers and researchers focusing on reinforcement learning within Isaac Sim, the clear path forward is with the Omniverse Isaac Sim Gym environment. This version not only benefits from continuous support and updates but is also seamlessly integrated with the broader Omniverse ecosystem.

Orbit Link to heading

Orbit is a unified and modular framework for robot learning that aims to simplify common workflows in robotics research (such as RL, learning from demonstrations, and motion planning). It is built upon NVIDIA Isaac Sim to leverage the latest simulation capabilities for photo-realistic scenes, and fast and efficient simulation. – isaac orbit

Isaac Orbit

Orbit has a simplified interface for env design and support for many robotics workflows - RL, Motion planning, teleoperation, imitation learning/behavior cloning, and real robot operation. This unification is the USP of orbit as compared to other interfaces.

Importantly Orbit is designed such that it can accept community contributions with open licensing. In the future, Orbit will be the environment zoo for IsaacSim with contributions from the community as well as internal development.

Orbit also has modified env interfaces. However, Orbit is open source, and users can modify & suggest change to these interfaces, as needed.

Summary Link to heading

Nvidia Omniverse is like a unique app store, but for 3D workflows and applications based on Pixar’s Universal Scene Description (OpenUSD). While Isaac Sim is like a game engine, tailored for robotics, and accessible via the Omniverse Launcher, that provides a photorealistic and physically accurate virtual environment. Omniverse Isaac Gym Envs and Orbit are just a reinforcement learning environemnt and framework built atop Isaac Sim. So if I were to choose a starting point for working on robotics reinforcement learning problems, my go-to would be Orbit.

Hope this article sheds light on NVIDIA’s powerful tools, helping you navigate your options and select the right platform for your project needs.

References Link to heading