Unity-Robotics-Hub/tutorials/pick_and_place

Pick-and-Place Tutorial

Unity's tools for robotic simulation enable users to integrate Unity with ROS-based workflows. ROS (Robot Operating System) provides services such as message-passing, package management, low-level device control, and hardware abstraction. Unity's robotics tools are able to support importing URDF files and sending and receiving messages between ROS and Unity. This tutorial will go through the steps necessary to integrate ROS with Unity, from installing the Unity Editor to creating a scene with an imported URDF to completing a pick-and-place task with known poses using MoveIt trajectory planning.

This tutorial is designed such that you do not need prior experience with Unity or C# in order to follow the scene setup steps, and you do not need prior robotics experience to get started with ROS integration. The tutorial is divided into high-level phases, from basic Unity and ROS initial setup through executing a pick-and-place task.

Want to skip the tutorial and run the full demo? Check out our Quick Demo

Note: This project has been tested with Python 2 and ROS Melodic, as well as Python 3 and ROS Noetic.

Pick-and-Place Tutorial

• Requirements
• Part 0: ROS Setup
• Part 1: Create Unity scene with imported URDF
• Part 2: ROS–Unity Integration
• Part 3: Pick-and-Place In Unity
• Part 4: Pick-and-Place on the Real Robot

Requirements

This repository provides project files for the pick-and-place tutorial, including Unity assets, URDF files, and ROS scripts. Clone this repository to a location on your local machine:

git clone --recurse-submodules https://github.com/Unity-Technologies/Unity-Robotics-Hub.git

Part 0: ROS Setup

This part provides two options for setting up your ROS workspace: using Docker, or manually setting up a catkin workspace.

Part 1: Create Unity scene with imported URDF

This part includes downloading and installing the Unity Editor, setting up a basic Unity scene, and importing a robot--the Niryo One--using the URDF Importer.

Part 2: ROS–Unity Integration

This part covers creating a TCP connection between Unity and ROS, generating C# scripts from a ROS msg and srv files, and publishing to a ROS topic.

Part 3: Pick-and-Place In Unity

This part includes the preparation and setup necessary to run a pick-and-place task with known poses using MoveIt. Steps covered include creating and invoking a motion planning service in ROS, moving a Unity Articulation Body based on a calculated trajectory, and controlling a gripping tool to successfully grasp and drop an object.

Part 4: Pick-and-Place on the Real Robot

This part is going to be a little different than the previous tutorials in that it will utilize a real Niryo One robot. We do not assume that everyone has access to a Niryo One outside of simulation. As such this tutorial should mostly be used as a reference for how to move from executing commands on a simulated robot to a real one.