Section Timestamps

• 0:55 Rotation prediction pipeline
• 1:23 Improving object generalization
• 1:42 Handling multimodality in training set
• 1:56 Experimental setup
• 2:27 Prior work on stacking
• 2:40 Stacking results
• 3:20 Real world results
• 3:43 Quantitative results

We present a system for accurately predicting stable orientations for diverse rigid objects. We propose to overcome the critical issue of modelling multimodality in the space of rotations by using a conditional generative model to accurately classify contact surfaces. Our system is capable of operating from noisy and partially-observed pointcloud observations captured by real world depth cameras. Our method substantially outperforms the current state-of-the-art systems on a simulated stacking task requiring highly accurate rotations, and demonstrates strong sim2real zero-shot transfer results across a variety of unseen objects on a real world reorientation task.

We show the performance of our system on stacking towers of two blocks. We compare against two baselines - a state of the art implicit rotation prediction method as well as an oriented bounding box baseline. In the examples shown, our rotation prediction pipeline is able to predict the rotation to sufficient accuracy such that the tower stabilizes, while the towers produced by the two baseline methods topple over.

We characterize the performance of our trained model on 3D-printed, real world versions of the unseen test objects. Our evaluation metric is the success rate, where success is determined by a human evaluator as to whether the predicted contact plane closely (within 15 degrees) matches one of the height-maximizing contact planes of the true object.

We have released the PyTorch based implementation and environment on the Github page. Try our code!

@inproceedings{li22stablereorientation,
  Author = {Li, Richard and
  Esteves, Carlos and Makadia, Ameesh and Agrawal, Pulkit},
  Title = {Stable Object Reorientation using Contact Plane Registration},
  Booktitle = {arXiv preprint arXiv:XXXX},
  Year = {2022}
}

Acknowledgements

We thank Branden Romero for helping 3D print the physical objects, and Abhishek Gupta for comments on the draft. We thank the NSF Institute of Artificial Intelligence and Fundamental Interactions and Sony for their financial support.