New IROS paper: Task-Aware Shared Control for Teleoperated Manipulation!

TASC: VLM-grounded shared control for relational telemanipulation 🤖 [1]

✅ Builds an open-vocabulary interaction graph from RGB-D + VLM reasoning, encoding task-relevant object–object relations and spatial affordances rather than relying on predefined templates.
✅ Uses the inferred relation to generate axis-alignment constraints between the grasped object and target object, then optimizes the gripper orientation during post-grasp interaction.
✅ In many tasks, the user can provide translational/positional input while TASC supplies the task-aware rotational assistance.

Check out the code & videos: https://fitz0401.github.io/tasc-page/

References

[1]

TASC: Task-Aware Shared Control for Teleoperated Manipulation.

Ze Fu, Pinhao Song, Yutong Hu, and Renaud Detry.

In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2026.

@inproceedings{fu2025a,
  author = {Fu, Ze and Song, Pinhao and Hu, Yutong and Detry, Renaud},
  title = {TASC: Task-Aware Shared Control for Teleoperated Manipulation},
  year = {2026},
  archiveprefix = {arXiv},
  booktitle = {{IEEE/RSJ} International Conference on Intelligent Robots and Systems},
  primaryclass = {cs.RO},
  url = {https://arxiv.org/abs/2509.10416}
}