Generalizing Task Parameters Part of the EU project TOMSY. ← Back to research project list.

We address the problem of generalizing manipulative actions across different tasks and objects. Our robotic agent acquires task-oriented skills from a teacher, and it abstracts skill parameters away from the specificity of the objects and tools used by the teacher. This process enables the transfer of skills to novel objects. Our method relies on the modularization of a task's representation. Through modularization, we associate each action parameter to a narrow visual modality, therefore facilitating transfers across different objects or tasks. We present a simple experiment where the robot transfers task parameters across three tasks and three objects.

Transferring task parameters. This figure is organized around a simplified representation of the product space of tasks and objects. The robot is taught two task instances, namely pour from a bottle, and store a carton in a fridge door. The robot is then asked to store the bottle into the fridge door, a task/object combination that it has not been taught. However, from its experience with the carton, the robot learned that, in order to store, it needs to grasp the object near its top, to avoid colliding with the door while inserting the object. “Grasping near the top” is a task constraint that is potentially transferrable to other objects, including a bottle. Yet, the experience acquired with the carton will not allow the robot to adequately grasp the bottle, as the shape of the carton and the bottle differ substantially. A set of finger placements that are compatible with the cylindric shape of the bottle are instead derived from the first task instance that has been taught to the robot. From its experience with the bottle, the robot has learned how to place its fingers around a cylindric shape. The placement of the fingers around the cylindric shape of the bottle is not necessarily specific to the action of pouring liquid. The robot eventually combines parts of the experience gained through two different task instances to plan a grasp for a previously unobserved task/action combination.

Part of the EU project TOMSY.

Supported by the Belgian National Fund for Scientific Research (FNRS).

Supported by the Swedish Research Council (VR).

Code based on the Nuklei library.

Main reference:

detry2013b 

R. Detry, M. Hjelm, C. H. Ek and D. Kragic, Generalizing Task Parameters Through Modularization. In Autonomous Learning Workshop (Workshop at ICRA 2013), 2013.
pdf; show/hide bibtex

@inproceedings{detry2013b,
	author = {Renaud Detry and Martin Hjelm and Carl Henrik Ek and Danica Kragic},
	booktitle = {Autonomous Learning Workshop (Workshop at ICRA 2013)},
	title = {Generalizing Task Parameters Through Modularization},
	year = {2013}}

Papers covering this topic:

detry2013b 

R. Detry, M. Hjelm, C. H. Ek and D. Kragic, Generalizing Task Parameters Through Modularization. In Autonomous Learning Workshop (Workshop at ICRA 2013), 2013.
pdf; show/hide bibtex

@inproceedings{detry2013b,
	author = {Renaud Detry and Martin Hjelm and Carl Henrik Ek and Danica Kragic},
	booktitle = {Autonomous Learning Workshop (Workshop at ICRA 2013)},
	title = {Generalizing Task Parameters Through Modularization},
	year = {2013}}

hjelm2014a 

M. Hjelm, R. Detry, C. H. Ek and D. Kragic, Representations for Cross-task, Cross-object Grasp Transfer. In IEEE International Conference on Robotics and Automation, 2014.
doi; show/hide bibtex

@inproceedings{hjelm2014a,
	author = {Martin Hjelm and Renaud Detry and Carl Henrik Ek and Danica Kragic},
	booktitle = {{IEEE} International Conference on Robotics and Automation},
	doi = {10.1109/ICRA.2014.6907697},
	title = {Representations for Cross-task, Cross-object Grasp Transfer},
	year = {2014}}

Many of these publications are copyrighted by their respective publishers. Downloadable versions are not necessarily identical to the published versions. They are made available here for personal use only.