3D Object Pose Estimation and Recognition Part of the EU project PACO-PLUS. ← Back to research project list.

This work introduces a generative object model for 6D pose estimation in stereo views of cluttered scenes [detry2009a]. We model an object as a hierarchy of increasingly expressive object parts, where parts represent the 3D geometry and appearance of object edges. At the bottom of the hierarchy, each part encodes the spatial distribution of short segments of object edges of a specific color. Higher-level parts are formed by recursively combining more elementary parts together, the top-level part representing the whole object. The hierarchy is encoded in a Markov random field whose edges parametrize relative part configurations.

Pose inference is implemented with generic probability and machine learning techniques including belief propagation, Monte Carlo integration, and kernel density estimation. The model is learned autonomously from a set of segmented views of an object. A 3D object model is a useful asset in the context of robotic grasping, as it allows for aligning a grasp model to arbitrary object positions and orientations. Several aspects of this work are inspired by biological examples, which makes it a good building block for cognitive robotic platforms.

    

Pose estimation. The right image shows the maximum-likelihood pose for the toy pan, which is extracted from the largest mode of the pose distribution over the scene shown on the left.

Part of the EU project PACO-PLUS.

Supported by the Belgian National Fund for Scientific Research (FNRS).

Code based on the Nuklei library.

Main references:

detry2009a 

R. Detry, N. Pugeault and J. Piater, A Probabilistic Framework for 3D Visual Object Representation. In IEEE Trans. Pattern Anal. Mach. Intell., 31 (10): 1790–1803, 2009.
doi; pdf; show/hide bibtex

@article{detry2009a,
	author = {Renaud Detry and Nicolas Pugeault and Justus Piater},
	doi = {10.1109/TPAMI.2009.64},
	journal = {{IEEE} Trans. Pattern Anal. Mach. Intell.},
	number = {10},
	pages = {1790--1803},
	publisher = {IEEE Computer Society},
	title = {A Probabilistic Framework for {3D} Visual Object Representation},
	volume = {31},
	year = {2009}}

detry2010c 

R. Detry and J. Piater, Continuous Surface-point Distributions for 3D Object Pose Estimation and Recognition. In Asian Conference on Computer Vision, pages 572–585, 2010.
doi; pdf; show/hide bibtex

@inproceedings{detry2010c,
	author = {Renaud Detry and Justus Piater},
	booktitle = {Asian Conference on Computer Vision},
	doi = {10.1007/978-3-642-19318-7_45},
	pages = {572--585},
	title = {Continuous Surface-point Distributions for {3D} Object Pose Estimation and Recognition},
	year = {2010}}

Papers covering this topic:

detry2007a 

R. Detry and J. H. Piater, Hierarchical Integration of Local 3D Features for Probabilistic Pose Recovery. In Robot Manipulation: Sensing and Adapting to the Real World (Workshop at Robotics, Science and Systems), 2007.
pdf; show/hide bibtex

@inproceedings{detry2007a,
	author = {Renaud Detry and Justus H. Piater},
	booktitle = {Robot Manipulation: Sensing and Adapting to the Real World (Workshop at Robotics, Science and Systems)},
	title = {Hierarchical Integration of Local {3D} Features for Probabilistic Pose Recovery},
	year = {2007}}

detry2008a 

R. Detry, N. Pugeault and J. H. Piater, Probabilistic Pose Recovery Using Learned Hierarchical Object Models. In International Cognitive Vision Workshop (Workshop at the 6th International Conference on Vision Systems), pages 107–120, Springer-Verlag, Berlin, Heidelberg, 2008.
doi; pdf; show/hide bibtex

@inproceedings{detry2008a,
	author = {Renaud Detry and Nicolas Pugeault and Justus H. Piater},
	booktitle = {International Cognitive Vision Workshop (Workshop at the 6th International Conference on Vision Systems)},
	doi = {10.1007/978-3-540-92781-5_9},
	pages = {107--120},
	publisher = {Springer-Verlag},
	title = {Probabilistic Pose Recovery Using Learned Hierarchical Object Models},
	year = {2008}}

detry2009a 

R. Detry, N. Pugeault and J. Piater, A Probabilistic Framework for 3D Visual Object Representation. In IEEE Trans. Pattern Anal. Mach. Intell., 31 (10): 1790–1803, 2009.
doi; pdf; show/hide bibtex

@article{detry2009a,
	author = {Renaud Detry and Nicolas Pugeault and Justus Piater},
	doi = {10.1109/TPAMI.2009.64},
	journal = {{IEEE} Trans. Pattern Anal. Mach. Intell.},
	number = {10},
	pages = {1790--1803},
	publisher = {IEEE Computer Society},
	title = {A Probabilistic Framework for {3D} Visual Object Representation},
	volume = {31},
	year = {2009}}

detry2010c 

R. Detry and J. Piater, Continuous Surface-point Distributions for 3D Object Pose Estimation and Recognition. In Asian Conference on Computer Vision, pages 572–585, 2010.
doi; pdf; show/hide bibtex

@inproceedings{detry2010c,
	author = {Renaud Detry and Justus Piater},
	booktitle = {Asian Conference on Computer Vision},
	doi = {10.1007/978-3-642-19318-7_45},
	pages = {572--585},
	title = {Continuous Surface-point Distributions for {3D} Object Pose Estimation and Recognition},
	year = {2010}}

detry2010d 

R. Detry, Learning of Multi-Dimensional, Multi-Modal Features for Robotic Grasping. Ph.D. Thesis, University of Liège, 2010.
pdf; show/hide bibtex

@phdthesis{detry2010d,
	author = {Renaud Detry},
	school = {University of Liège},
	title = {Learning of Multi-Dimensional, Multi-Modal Features for Robotic Grasping},
	year = {2010}}

piater2008a 

J. Piater, F. Scalzo and R. Detry, Vision as Inference in a Hierarchical Markov Network. In International Conference on Cognitive and Neural Systems, 2008.
pdf; show/hide bibtex

@inproceedings{piater2008a,
	author = {Justus Piater and Fabien Scalzo and Renaud Detry},
	booktitle = {International Conference on Cognitive and Neural Systems},
	title = {Vision as Inference in a Hierarchical Markov Network},
	year = {2008}}

piater2008b 

J. Piater and R. Detry, 3D Probabilistic Representations for Vision and Action. In Robotics Challenges for Machine Learning II (Workshop at the IEEE/RSJ International Conference on Intelligent Robots and Systems), 2008.
pdf; show/hide bibtex

@inproceedings{piater2008b,
	author = {Justus Piater and Renaud Detry},
	booktitle = {Robotics Challenges for Machine Learning II (Workshop at the IEEE/RSJ International Conference on Intelligent Robots and Systems)},
	title = {{3D} Probabilistic Representations for Vision and Action},
	year = {2008}}

piater2009a 

J. Piater, S. Jodogne, R. Detry, D. Kraft, N. Krüger, O. Kroemer and J. Peters, Learning Visual Representations for Interactive Systems. In International Symposium on Robotics Research, 2009.
doi; pdf; show/hide bibtex

@inproceedings{piater2009a,
	author = {Justus Piater and Sébastien Jodogne and Renaud Detry and Dirk Kraft and Norbert Krüger and Oliver Kroemer and Jan Peters},
	booktitle = {International Symposium on Robotics Research},
	doi = {10.1007/978-3-642-19457-3_24},
	title = {Learning Visual Representations for Interactive Systems},
	year = {2009}}

Many of these publications are copyrighted by their respective publishers. Downloadable versions are not necessarily identical to the published versions. They are made available here for personal use only.