Part-based Grasp Generalization Part of the EU project CogX. Part of the EU project TOMSY. ← Back to research project list.

We present a real-world robotic agent that is capable of transferring grasping strategies across objects that share similar parts. The agent transfers grasps across objects by identifying, from examples provided by a teacher, parts by which objects are often grasped in a similar fashion. It then uses these parts to identify grasping points onto novel objects. While prior work in this area focused primarily on shape analysis (parts identified, e.g., through visual clustering, or salient structure analysis), the key aspect of this work is the emergence of parts from both object shape and grasp examples. As a result, parts intrinsically encode the intention of executing a grasp.

We devise a similarity measure that reflects whether the shapes of two parts resemble each other, and whether their associated grasps are applied near one another. We discuss a nonlinear clustering procedure that allows groups of similar part-grasp associations to emerge from the space induced by the similarity measure. We present an experiment in which our agent extracts five prototypical parts from thirty-two grasp examples, and we demonstrate the applicability of the prototypical parts for grasping novel objects.

Part of the EU project CogX.

Part of the EU project TOMSY.

Supported by the Belgian National Fund for Scientific Research (FNRS).

Supported by the Swedish Foundation for Strategic Research (SSF).

Code based on the Nuklei library.

Main reference:

detry2013a 

R. Detry, C. H. Ek, M. Madry and D. Kragic, Learning a Dictionary of Prototypical Grasp-predicting Parts from Grasping Experience. In IEEE International Conference on Robotics and Automation, 2013.
doi; pdf; show/hide bibtex

@inproceedings{detry2013a,
	author = {Renaud Detry and Carl Henrik Ek and Marianna Madry and Danica Kragic},
	booktitle = {{IEEE} International Conference on Robotics and Automation},
	doi = {10.1109/ICRA.2013.6630635},
	title = {Learning a Dictionary of Prototypical Grasp-predicting Parts from Grasping Experience},
	year = {2013}}

Papers covering this topic:

detry2010d 

R. Detry, Learning of Multi-Dimensional, Multi-Modal Features for Robotic Grasping. Ph.D. Thesis, University of Liège, 2010.
pdf; show/hide bibtex

@phdthesis{detry2010d,
	author = {Renaud Detry},
	school = {University of Liège},
	title = {Learning of Multi-Dimensional, Multi-Modal Features for Robotic Grasping},
	year = {2010}}

detry2011b 

R. Detry and J. Piater, Grasp Generalization Via Predictive Parts. In Austrian Robotics Workshop, 2011.
pdf; show/hide bibtex

@inproceedings{detry2011b,
	author = {Renaud Detry and Justus Piater},
	booktitle = {Austrian Robotics Workshop},
	title = {Grasp Generalization Via Predictive Parts},
	year = {2011}}

detry2012a 

R. Detry, C. H. Ek, M. Madry, J. Piater and D. Kragic, Generalizing Grasps Across Partly Similar Objects. In IEEE International Conference on Robotics and Automation, 2012.
doi; pdf; show/hide bibtex

@inproceedings{detry2012a,
	author = {Renaud Detry and Carl Henrik Ek and Marianna Madry and Justus Piater and Danica Kragic},
	booktitle = {{IEEE} International Conference on Robotics and Automation},
	doi = {10.1109/ICRA.2012.6224992},
	title = {Generalizing Grasps Across Partly Similar Objects},
	year = {2012}}

detry2012b 

R. Detry, C. H. Ek, M. Madry and D. Kragic, Compressing Grasping Experience into a Dictionary of Prototypical Grasp-predicting Parts. In International Workshop on Human-Friendly Robotics, 2012.
pdf; show/hide bibtex

@inproceedings{detry2012b,
	author = {Renaud Detry and Carl Henrik Ek and Marianna Madry and Danica Kragic},
	booktitle = {International Workshop on Human-Friendly Robotics},
	title = {Compressing Grasping Experience into a Dictionary of Prototypical Grasp-predicting Parts},
	year = {2012}}

detry2013a 

R. Detry, C. H. Ek, M. Madry and D. Kragic, Learning a Dictionary of Prototypical Grasp-predicting Parts from Grasping Experience. In IEEE International Conference on Robotics and Automation, 2013.
doi; pdf; show/hide bibtex

@inproceedings{detry2013a,
	author = {Renaud Detry and Carl Henrik Ek and Marianna Madry and Danica Kragic},
	booktitle = {{IEEE} International Conference on Robotics and Automation},
	doi = {10.1109/ICRA.2013.6630635},
	title = {Learning a Dictionary of Prototypical Grasp-predicting Parts from Grasping Experience},
	year = {2013}}

hjelm2013a 

M. Hjelm, C. H. Ek, R. Detry, H. Kjellström and D. Kragic, Sparse Summarization of Robotic Grasping Data. In IEEE International Conference on Robotics and Automation, 2013.
doi; show/hide bibtex

@inproceedings{hjelm2013a,
	author = {Martin Hjelm and Carl Henrik Ek and Renaud Detry and Hedvig Kjellström and Danica Kragic},
	booktitle = {{IEEE} International Conference on Robotics and Automation},
	doi = {10.1109/ICRA.2013.6630707},
	title = {Sparse Summarization of Robotic Grasping Data},
	year = {2013}}

Many of these publications are copyrighted by their respective publishers. Downloadable versions are not necessarily identical to the published versions. They are made available here for personal use only.