Open containers, i.e., containers without covers, are an important and ubiquitous class of objects in human life. In this work, we propose a novel method for robots to “imagine” the open containability affordance of a previously unseen object via physical simulations. We implement our imagination method on a UR5 manipulator. The robot autonomously scans the object with an RGB-D camera. The scanned 3D model is used for open containability imagination which quantifies the open containability affordance. This quantification is used for open-container vs. non-open-container binary classification. If the object is classified as an open container, the robot further imagines pouring into the object, again using physical simulations, to obtain the pouring position and orientation for real robot autonomous pouring. We evaluate our method on open container classification and autonomous pouring of granular material on a dataset containing 130 previously unseen objects with 57 object categories. Although our proposed method uses only 11 objects for simulation calibration (training), its open container classification aligns well with human judgements. In addition, our method endows the robot with the capability to autonomously pour into the 55 containers in the dataset with a very high success rate.


Introductory Video


Hongtao Wu, Gregory Chirikjian

Laboratory for Computational Sensing and Robotics (LCSR), Johns Hopkins University

Department of Mechanical Engineering, National University of Singapore

Supplementary Videos

In this section, we show more demos on open containability imagination and autonomous pouring.

The simulation videos are outputted by the Pybullet. Note that the runtimes for open containability imagination and pouring imagination reported in the paper are evaluated without video visualization.

I) Open Containability Imagination Demo

1) Plate (Detailed with Explanations)

Labeled as an open container by the human annotator

2) Chef Hat

Labeled as an open container by the human annotator

3) Cup

Labeled as an open container by the human annotator

4) Book

Labeled as a non-open container by the human annotator

II) Autonomous Pouring Demo

All the objects evaluated on the autonomous pouring are labeled as open containers by the human annotator.

1) Candlestick (Detailed with Explanations)

2) Gravy Boat

3) Baking Cup

4) Origami Cup

5) Chef Hat