University of Louisville
Autonomous Pick-and-Place Tasks with Synthetic Data and Path Planning
Grade Level at Time of Presentation
Senior
Major
Computer Science & Engineering
Minor
Entrepreneurship
KY House District #
41
KY Senate District #
33
Faculty Advisor/ Mentor
Sumit Kumar Das, PhD; Dan O. Popa, PhD.
Department
Computer Science & Engineering
Abstract
In the healthcare field, autonomous robotics has immense potential to be a valuable asset in assisting with a variety of repetitive tasks, specifically with transporting medical equipment and dispensing medications. To alleviate the workload on healthcare workers and allow them to focus on more skill-intensive tasks, the work presented here utilizes deep learning with synthetic data and path planning to perform autonomous pick-and-place tasks on known objects while avoiding obstacles in any given environment.
The Adaptive Robot Nursing Assistant (ARNA), equipped with a Kinova Gen3 Robot arm, was programmed to perform such tasks using the Robot Operating System (ROS) middleware and the Deep Object Pose Estimation (DOPE) convolutional neural network that is trained on domain-randomized and photorealistic synthetic data through Unreal Engine. Unlike manually pre-labeled data in deep learning, synthetic data can be labeled automatically, which is faster, more cost-effective, and ultimately capable of generating a large amount of data with a high degree of variability.
Our proposed approach combines synthetic data and path planning algorithms through the OpenRAVE environment to reduce pose estimation error margins and improve confidence levels by 18% when compared to results from a manually trained framework. Once the DOPE network recognizes objects using the arm’s built-in Intel RealSense camera and depth sensor, ROS topics can retrieve a specific pose and orientation, based on the user’s selection, through a publisher-subscriber methodology to formulate the optimal trajectory for grasping the object at those coordinates using the Kinova API and OpenRAVE.
The above work aims to improve shortcomings in the object recognition and deep learning domains when dealing with different lighting conditions, poses, and environments without external tags or references. Implementing such autonomous solutions will improve the efficiency of the current healthcare system and lead to better patient outcomes while reducing the risk of human error.
Autonomous Pick-and-Place Tasks with Synthetic Data and Path Planning
In the healthcare field, autonomous robotics has immense potential to be a valuable asset in assisting with a variety of repetitive tasks, specifically with transporting medical equipment and dispensing medications. To alleviate the workload on healthcare workers and allow them to focus on more skill-intensive tasks, the work presented here utilizes deep learning with synthetic data and path planning to perform autonomous pick-and-place tasks on known objects while avoiding obstacles in any given environment.
The Adaptive Robot Nursing Assistant (ARNA), equipped with a Kinova Gen3 Robot arm, was programmed to perform such tasks using the Robot Operating System (ROS) middleware and the Deep Object Pose Estimation (DOPE) convolutional neural network that is trained on domain-randomized and photorealistic synthetic data through Unreal Engine. Unlike manually pre-labeled data in deep learning, synthetic data can be labeled automatically, which is faster, more cost-effective, and ultimately capable of generating a large amount of data with a high degree of variability.
Our proposed approach combines synthetic data and path planning algorithms through the OpenRAVE environment to reduce pose estimation error margins and improve confidence levels by 18% when compared to results from a manually trained framework. Once the DOPE network recognizes objects using the arm’s built-in Intel RealSense camera and depth sensor, ROS topics can retrieve a specific pose and orientation, based on the user’s selection, through a publisher-subscriber methodology to formulate the optimal trajectory for grasping the object at those coordinates using the Kinova API and OpenRAVE.
The above work aims to improve shortcomings in the object recognition and deep learning domains when dealing with different lighting conditions, poses, and environments without external tags or references. Implementing such autonomous solutions will improve the efficiency of the current healthcare system and lead to better patient outcomes while reducing the risk of human error.