Research

Research Interests

My core research interest is in adaptive robotics, with a special interest in space applications. I develop robotic systems that cope with the complexity of our world by using cameras and other sensors to observe and parse their surroundings and learn to adjust their behavior accordingly. My research aims to bring autonomous robotic assistance to sectors where working conditions are dangerous or impractical for humans. This includes space robotics and planetary exploration, robotic assistance for people suffering from limited mobility, as well as manufacturing, construction, or agriculture. My work is at the intersection of robotics and machine learning, which is reflected in my dual appointment within KU Leuven’s departments of electrical and mechanical engineering.

My work focuses on casting robotics problems to a form that is catalytic to data-driven modeling. Much of the creativity involved here goes into crafting… models that allow us to leverage state-of-art machine learning tools to solve a given task. I study multimodal perceptual models that enable artificial decision-making in open-ended environments, with a focus on machine vision and robotics, including planning and control for manipulation, mobility, and task modeling. At the core of these models, I study perception-action embeddings that are capable of expressing the constraints and objectives of a task while maximizing invariance to inconsequential perceptual parameters, with the purpose of generalizing to new data. My scientific interests lie in the development of such representations to address problems that cannot be efficiently modeled through simpler means.

A defining aspect of my work is grounding in sensorimotor experience: I study models that run on physical systems. My PhD and postdoc (ULiege and KTH) focused primarily on manipulation systems composed of industrial arms and grippers equipped with visual and tactile sensors, and targeting home or industrial applications. During my tenure at NASA/JPL, I extended my reach to legged/wheeled mobility, situational awareness, and field robots that implement the hardware and software stacks needed to accomplish a real-world task in a relevant open-ended environment. I worked on self-sufficient autonomous mobile manipulation systems and fielded them in extreme natural settings including caves, deserts, and turbid water. [show more]

Keywords: Robotics, computer vision, robot learning, space robotics.

Projects

active

H2XO (2024–2028)

Relieving heavy-duty work with a haptically-controlled robot assistant that learns on the job.

active

Contact (2023–2025)

We leverage afferent visuo-tactile data and a world model to continuously refine a grasping point during approach.

active

NESP (2022–2026)

Neural Estimation of Spacecraft Pose.

active

Neurobotics (2022–2026)

Blended brain-machine control for human assistance using hybrid smart systems.

active

RoBétArmé (2022–2026)

Autonomous robotic concrete projection.

TubePicker (2021–2022)

Detect and retrieve sample tubes on Mars.

SatBuilder (2018–2019)

Autonomous visual manipulation for in-space assembly.

Mars Sample Return (2016–2021)

Return Martian soil samples to Earth.

FreeClimber (2016–2021)

Learning to climb rock walls of unknown geometry.

RCTA/RoMan (2016–2021)

Autonomous manipulation and mobility in unstructured terrain.

InVADER (2016–2021)

In-situ vent analysis divebot for exobiology research.

UMM (2016–2019)

Autonomous mobile manipulation in turbid water.

Plexegen (2012–2016)

Grasping novel objects with tactile-based grasp stability estimates.

Grasp (2011–2015)

Task-oriented Grasping: learning to use new tools and objects.

CogX (2008–2012)

Cognitive Systems that Self-Understand and Self-Extend.

Feature Hierarchies (2006–2010)

Learning of multi-Dimensional, multi-modal features for robotic grasping.

PACO-PLUS (2006–2010)

Perception, action and cognition through learning of object-action complexes.

Ph.D. Students

Current students:

• Ze Fu (pending arrival)
• Yutong Hu
• Elias De Smijter
• Zhongxi Li
• Yurui Du
• Pinhao Song
• Antoine Legrand
• Louis Hanut

Past students:

• Deniz Soysal
• Joseph Bowkett (informal co-supervision with Joel Burdick, Caltech)
• Emil Hyttinen (co-supervision with Danica Kragic, KTH)
• Martin Hjelm (informal co-supervision with Danica Kragic, KTH)
• Alexander Rietzler (informal co-supervision with Justus Piater, UIBK)
• Ashley Liddiard (informal co-supervision with Bryan Tripp, UWaterloo)
• Yasemin Bekiroglu (informal co-supervision with Danica Kragic, KTH)

Software

My group’s software is available on our GitLab portal. Older software from my Ph.D. and postdoc is freely available on this page.

Jobs

See this page.

Videos

See this page.

Events

See this page.