TR2023-023

Tactile Tool Manipulation


Abstract:

Humans can effortlessly perform very complex, dexterous manipulation tasks by reacting to sensor observations. In contrast, robots can not perform reactive manipulation and they mostly operate in open-loop while interacting with their environment. Consequently, the current manipulation algorithms either are inefficient in performance or can only work in highly structured environments. In this paper, we present closed-loop control of a complex manipulation task where a robot uses a tool to interact with objects. Manipulation using a tool leads to complex kinematics and contact constraints that need to be satisfied for generating feasible manipulation trajectories. We first present an open-loop controller design using Non-Linear Programming (NLP) that satisfies these constraints. In order to design a closed-loop controller, we present a pose estimator of objects and tools using tactile sensors. Using our tactile estimator, we design a closed-loop controller based on Model Predictive Control (MPC). The proposed algorithm is verified using a 6 DoF manipulator on tasks using a variety of objects and tools. We verify that our closed-loop controller can successfully perform tool manipulation under several unexpected contacts.

 

  • Related News & Events

  • Related Video

  • Related Publications

  •  Shirai, Y., Jha, D.K., Raghunathan, A., Hong, D., "Closed-Loop Tactile Controller for Tool Manipulation", ICRA 2023 Workshop on Embracing contacts. Making robots physically interact with our world, May 2023.
    BibTeX TR2023-043 PDF Video
    • @inproceedings{Shirai2023may,
    • author = {Shirai, Yuki and Jha, Devesh K. and Raghunathan, Arvind and Hong, Dennis},
    • title = {Closed-Loop Tactile Controller for Tool Manipulation},
    • booktitle = {ICRA 2023 Workshop on Embracing contacts. Making robots physically interact with our world},
    • year = 2023,
    • month = may,
    • url = {https://www.merl.com/publications/TR2023-043}
    • }
  •  Shirai, Y., Jha, D.K., Raghunathan, A., Hong, D., "Tactile Tool Manipulation", arXiv, January 2023.
    BibTeX arXiv
    • @article{Shirai2023jan,
    • author = {Shirai, Yuki and Jha, Devesh K. and Raghunathan, Arvind and Hong, Dennis},
    • title = {Tactile Tool Manipulation},
    • journal = {arXiv},
    • year = 2023,
    • month = jan,
    • url = {https://arxiv.org/abs/2301.06698}
    • }