industry news
Subscribe Now

New Method Developed by GIST Researchers Allow Robot Vision to Identify Occluded Objects

Much like human vision, the method allows for the detection of visible, partially occluded, and unseen objects in a single framework
When artificial intelligence systems encounter scenes where objects are not fully visible, it has to make estimations based only on the visible parts of the objects. This partial information leads to detection errors, and large training data is required to correctly recognize such scenes. Now, researchers at the Gwangju Institute of Science and Technology have developed a framework that allows robot vision to detect such objects successfully in the same way that we perceive them.
 
Robotic vision has come a long way, reaching a level of sophistication with applications in complex and demanding tasks, such as autonomous driving and object manipulation. However, it still struggles to identify individual objects in cluttered scenes where some objects are partially or completely hidden behind others. Typically, when dealing with such scenes, robotic vision systems are trained to identify the occluded object based only on its visible parts. But such training requires large datasets of objects and can be pretty tedious.
Associate Professor Kyoobin Lee and Ph.D. student Seunghyeok Back from the Gwangju Institute of Science and Technology (GIST) in Korea found themselves facing this problem when they were developing an artificial intelligence system to identify and sort objects in cluttered scenes. “We expect a robot to recognize and manipulate objects they have not encountered before or been trained to recognize. In reality, however, we need to manually collect and label data one by one as the generalizability of deep neural networks depends highly on the quality and quantity of the training dataset,” says Mr. Back.
In a new study accepted at the 2022 IEEE International Conference on Robotics and Automation, a research team led by Prof. Lee and Mr. Back developed a model called “unseen object amodal instance segmentation” (UOAIS) for detecting occluded objects in cluttered scenes. To train the model in identifying object geometry, they developed a database containing 45,000 photorealistic synthetic images containing depth information. With this (limited) training data, the model was able to detect a variety of occluded objects. Upon encountering a cluttered scene, it first picked out the object of interest and then determines if the object is occluded by segmenting the object into a “visible mask” and an “amodal mask.”
The researchers were excited by the results. “Previous methods are limited to either detecting only specific types of objects or detecting only the visible regions without explicitly reasoning over occluded areas. By contrast, our method can infer the hidden regions of occluded objects like a human vision system. This enables a reduction in data collection efforts while improving performance in a complex environment,” comments Mr. Back.
To enable “occlusion reasoning” in their system, the researchers introduced a “hierarchical occlusion modeling” (HOM) scheme, which assigned a hierarchy to the combination of multiple extracted features and their prediction order. By testing their model against three benchmarks, they validated the effectiveness of the HOM scheme, which achieved state-of-the-art performance.
The researchers are hopeful about the future prospects of their method. “Perceiving unseen objects in a cluttered environment is essential for amodal robotic manipulation. Our UOAIS method could serve as a baseline on this front,” says Mr. Back.
It certainly looks like a giant leap for robotic vision!
Reference
Title of original paper: Unseen Object Amodal Instance Segmentation via Hierarchical Occlusion Modeling
Journal: IEEE International Conference on Robotics and Automation
 
 
About the Gwangju Institute of Science and Technology (GIST)
The Gwangju Institute of Science and Technology (GIST) was founded in 1993 by the Korean government as a research-oriented graduate school to help ensure Korea’s continued economic growth and prosperity by developing advanced science and technology with an emphasis on collaboration with the international community. Since that time, GIST has pioneered a highly regarded undergraduate science curriculum in 2010 that has become a model for other science universities in Korea. To learn more about GIST and its exciting opportunities for researchers and students alike, please visit: http://www.gist.ac.kr/.
About the authors
Kyoobin Lee is an Associate Professor and Director of the AI laboratory at GIST. His group is developing AI-based robot vision and deep learning-based bio-medical analysis methods. Before joining GIST, he obtained a Ph.D. in Mechatronics from KAIST and completed a postdoctoral training program at the Korea Institute of Science and Technology (KIST). The author can be reached at kyoobinlee@gist.ac.kr
Seunghyeok Back is a Ph.D. Student at the AI laboratory at GIST. His research is on robot vision for unseen object manipulation using deep neural networks and Sim2Real transfer. He received a B.S. degree in mechanical engineering from GIST. The author can be reached at shback@gm.gist.ac.kr

Leave a Reply

featured blogs
May 20, 2022
This year's NASA Turbulence Modeling Symposium is being held in honor of Philippe Spalart and his contributions to the turbulence modeling field. The symposium will bring together both academic and... ...
May 19, 2022
Learn about the AI chip design breakthroughs and case studies discussed at SNUG Silicon Valley 2022, including autonomous PPA optimization using DSO.ai. The post Key Highlights from SNUG 2022: AI Is Fast Forwarding Chip Design appeared first on From Silicon To Software....
May 12, 2022
By Shelly Stalnaker Every year, the editors of Elektronik in Germany compile a list of the most interesting and innovative… ...
Apr 29, 2022
What do you do if someone starts waving furiously at you, seemingly delighted to see you, but you fear they are being overenthusiastic?...

featured video

Building safer robots with computer vision & AI

Sponsored by Texas Instruments

Watch TI's demo to see how Jacinto™ 7 processors fuse deep learning and traditional computer vision to enable safer autonomous mobile robots.

Watch demo

featured paper

5 common Hall-effect sensor myths

Sponsored by Texas Instruments

Hall-effect sensors can be used in a variety of automotive and industrial systems. Higher system performance requirements created the need for improved accuracy and more integration – extending the use of Hall-effect sensors. Read this article to learn about common Hall-effect sensor misconceptions and see how these sensors can be used in real-world applications.

Click to read more

featured chalk talk

Just 1-Wire to Power and Operate I2C or SPI Endpoints

Sponsored by Mouser Electronics and Analog Devices

If you are working on a connection or IO constrained design, a one wire solution could be a great way for you to power and operate your I2C or SPI endpoints. In this episode of Chalk Talk, Amelia Dalton chats with Scott Jones from Maxim Integrated about the DS28E18 communications bridge: a one wire solution that can help you address a variety of system level challenges including protocol conversion, wiring limitations, and communication distance concerns.

Click here for more information about the Maxim Integrated DS28E18EVKIT Evaluation System