industry news
Subscribe Now

New Method Developed by GIST Researchers Allow Robot Vision to Identify Occluded Objects

Much like human vision, the method allows for the detection of visible, partially occluded, and unseen objects in a single framework
When artificial intelligence systems encounter scenes where objects are not fully visible, it has to make estimations based only on the visible parts of the objects. This partial information leads to detection errors, and large training data is required to correctly recognize such scenes. Now, researchers at the Gwangju Institute of Science and Technology have developed a framework that allows robot vision to detect such objects successfully in the same way that we perceive them.
 
Robotic vision has come a long way, reaching a level of sophistication with applications in complex and demanding tasks, such as autonomous driving and object manipulation. However, it still struggles to identify individual objects in cluttered scenes where some objects are partially or completely hidden behind others. Typically, when dealing with such scenes, robotic vision systems are trained to identify the occluded object based only on its visible parts. But such training requires large datasets of objects and can be pretty tedious.
Associate Professor Kyoobin Lee and Ph.D. student Seunghyeok Back from the Gwangju Institute of Science and Technology (GIST) in Korea found themselves facing this problem when they were developing an artificial intelligence system to identify and sort objects in cluttered scenes. “We expect a robot to recognize and manipulate objects they have not encountered before or been trained to recognize. In reality, however, we need to manually collect and label data one by one as the generalizability of deep neural networks depends highly on the quality and quantity of the training dataset,” says Mr. Back.
In a new study accepted at the 2022 IEEE International Conference on Robotics and Automation, a research team led by Prof. Lee and Mr. Back developed a model called “unseen object amodal instance segmentation” (UOAIS) for detecting occluded objects in cluttered scenes. To train the model in identifying object geometry, they developed a database containing 45,000 photorealistic synthetic images containing depth information. With this (limited) training data, the model was able to detect a variety of occluded objects. Upon encountering a cluttered scene, it first picked out the object of interest and then determines if the object is occluded by segmenting the object into a “visible mask” and an “amodal mask.”
The researchers were excited by the results. “Previous methods are limited to either detecting only specific types of objects or detecting only the visible regions without explicitly reasoning over occluded areas. By contrast, our method can infer the hidden regions of occluded objects like a human vision system. This enables a reduction in data collection efforts while improving performance in a complex environment,” comments Mr. Back.
To enable “occlusion reasoning” in their system, the researchers introduced a “hierarchical occlusion modeling” (HOM) scheme, which assigned a hierarchy to the combination of multiple extracted features and their prediction order. By testing their model against three benchmarks, they validated the effectiveness of the HOM scheme, which achieved state-of-the-art performance.
The researchers are hopeful about the future prospects of their method. “Perceiving unseen objects in a cluttered environment is essential for amodal robotic manipulation. Our UOAIS method could serve as a baseline on this front,” says Mr. Back.
It certainly looks like a giant leap for robotic vision!
Reference
Title of original paper: Unseen Object Amodal Instance Segmentation via Hierarchical Occlusion Modeling
Journal: IEEE International Conference on Robotics and Automation
 
 
About the Gwangju Institute of Science and Technology (GIST)
The Gwangju Institute of Science and Technology (GIST) was founded in 1993 by the Korean government as a research-oriented graduate school to help ensure Korea’s continued economic growth and prosperity by developing advanced science and technology with an emphasis on collaboration with the international community. Since that time, GIST has pioneered a highly regarded undergraduate science curriculum in 2010 that has become a model for other science universities in Korea. To learn more about GIST and its exciting opportunities for researchers and students alike, please visit: http://www.gist.ac.kr/.
About the authors
Kyoobin Lee is an Associate Professor and Director of the AI laboratory at GIST. His group is developing AI-based robot vision and deep learning-based bio-medical analysis methods. Before joining GIST, he obtained a Ph.D. in Mechatronics from KAIST and completed a postdoctoral training program at the Korea Institute of Science and Technology (KIST). The author can be reached at kyoobinlee@gist.ac.kr
Seunghyeok Back is a Ph.D. Student at the AI laboratory at GIST. His research is on robot vision for unseen object manipulation using deep neural networks and Sim2Real transfer. He received a B.S. degree in mechanical engineering from GIST. The author can be reached at shback@gm.gist.ac.kr

Leave a Reply

featured blogs
Dec 6, 2022
Explore quantum computing's impact on cryptography and learn how to prepare SoC designs for post-quantum computing and evolving cryptographic standards. The post Why Now Is the Time to Address Quantum Computing's Impact on Cryptography appeared first on From Silicon To Softw...
Dec 5, 2022
It's a new week in CFD and time to refresh our memories about what's going on here at Cadence Fidelity CFD. From The Blogs Cadence and McLaren Accelerate Innovation During a recent visit to the Cadence office in Austin, Texas, McLaren CEO Zak Brown discussed the Cad...
Nov 30, 2022
By Joe Davis Sponsored by France's ElectroniqueS magazine, the Electrons d'Or Award program identifies the most innovative products of the… ...
Nov 18, 2022
This bodacious beauty is better equipped than my car, with 360-degree collision avoidance sensors, party lights, and a backup camera, to name but a few....

featured video

How to Harness the Massive Amounts of Design Data Generated with Every Project

Sponsored by Cadence Design Systems

Long gone are the days where engineers imported text-based reports into spreadsheets and sorted the columns to extract useful information. Introducing the Cadence Joint Enterprise Data and AI (JedAI) platform created from the ground up for EDA data such as waveforms, workflows, RTL netlists, and more. Using Cadence JedAI, engineering teams can visualize the data and trends and implement practical design strategies across the entire SoC design for improved productivity and quality of results.

Learn More

featured chalk talk

Power Multiplexing with Discrete Components

Sponsored by Mouser Electronics and Toshiba

Power multiplexing is a vital design requirement for a variety of different applications today. In this episode of Chalk Talk, Amelia Dalton chats with Talayeh Saderi from Toshiba about what kind of power multiplex solution would be the best fit for your next design. They discuss five unique design considerations that we should think about when it comes to power multiplexing and the benefits that high side gate drivers bring to power multiplexing.

Click here for more information about Toshiba Gate Driver + MOSFET for 5-24V Line Power MUX