Robotics
Where hardware, software and machine intelligence come together.
Our research is interdisciplinary and focuses on sensing, planning, reasoning, and control of single and multi-agent systems, including both manipulation and mobile robots. We strive to develop algorithms and methods for factory automation, smart building and transportation applications using machine learning, computer vision, RF/optical sensing, wireless communications, control theory and signal processing. Key research themes include bin picking and object manipulation, sensing and mapping of indoor areas, coordinated control of robot swarms, as well as robot learning and simulation.
Quick Links
-
Researchers
Devesh K.
Jha
Diego
Romeres
Daniel N.
Nikovski
Arvind
Raghunathan
Stefano
Di Cairano
Siddarth
Jain
William S.
Yerazunis
Radu
Corcodel
Yebin
Wang
Toshiaki
Koike-Akino
Yuki
Shirai
Abraham P.
Vinod
Tim K.
Marks
Avishai
Weiss
Scott A.
Bortoff
Chiori
Hori
Ye
Wang
Anoop
Cherian
Jonathan
Le Roux
Matthew
Brand
Philip V.
Orlik
Alexander
Schperberg
Bingnan
Wang
Abraham
Goldsmith
Jianlin
Guo
Sameer
Khurana
Jing
Liu
Hassan
Mansour
Pedro
Miraldo
Saviz
Mowlavi
Anthony
Vetro
James
Queeney
-
Awards
-
AWARD University of Padua and MERL team wins the AI Olympics with RealAIGym competition at IROS24 Date: October 17, 2024
Awarded to: Niccolò Turcato, Alberto Dalla Libera, Giulio Giacomuzzo, Ruggero Carli, Diego Romeres
MERL Contact: Diego Romeres
Research Areas: Artificial Intelligence, Dynamical Systems, Machine Learning, RoboticsBrief- The team composed of the control group at the University of Padua and MERL's Optimization and Robotic team ranked 1st out of the 4 finalist teams that arrived to the 2nd AI Olympics with RealAIGym competition at IROS 24, which focused on control of under-actuated robots. The team was composed by Niccolò Turcato, Alberto Dalla Libera, Giulio Giacomuzzo, Ruggero Carli and Diego Romeres. The competition was organized by the German Research Center for Artificial Intelligence (DFKI), Technical University of Darmstadt and Chalmers University of Technology.
The competition and award ceremony was hosted by IEEE International Conference on Intelligent Robots and Systems (IROS) on October 17, 2024 in Abu Dhabi, UAE. Diego Romeres presented the team's method, based on a model-based reinforcement learning algorithm called MC-PILCO.
- The team composed of the control group at the University of Padua and MERL's Optimization and Robotic team ranked 1st out of the 4 finalist teams that arrived to the 2nd AI Olympics with RealAIGym competition at IROS 24, which focused on control of under-actuated robots. The team was composed by Niccolò Turcato, Alberto Dalla Libera, Giulio Giacomuzzo, Ruggero Carli and Diego Romeres. The competition was organized by the German Research Center for Artificial Intelligence (DFKI), Technical University of Darmstadt and Chalmers University of Technology.
-
AWARD Honorable Mention Award at NeurIPS 23 Instruction Workshop Date: December 15, 2023
Awarded to: Lingfeng Sun, Devesh K. Jha, Chiori Hori, Siddharth Jain, Radu Corcodel, Xinghao Zhu, Masayoshi Tomizuka and Diego Romeres
MERL Contacts: Radu Corcodel; Chiori Hori; Siddarth Jain; Devesh K. Jha; Diego Romeres
Research Areas: Artificial Intelligence, Machine Learning, RoboticsBrief- MERL Researchers received an "Honorable Mention award" at the Workshop on Instruction Tuning and Instruction Following at the NeurIPS 2023 conference in New Orleans. The workshop was on the topic of instruction tuning and Instruction following for Large Language Models (LLMs). MERL researchers presented their work on interactive planning using LLMs for partially observable robotic tasks during the oral presentation session at the workshop.
-
AWARD Joint University of Padua-MERL team wins Challenge 'AI Olympics With RealAIGym' Date: August 25, 2023
Awarded to: Alberto Dalla Libera, Niccolo' Turcato, Giulio Giacomuzzo, Ruggero Carli, Diego Romeres
MERL Contact: Diego Romeres
Research Areas: Artificial Intelligence, Machine Learning, RoboticsBrief- A joint team consisting of members of University of Padua and MERL ranked 1st in the IJCAI2023 Challenge "Al Olympics With RealAlGym: Is Al Ready for Athletic Intelligence in the Real World?". The team was composed by MERL researcher Diego Romeres and a team from University Padua (UniPD) consisting of Alberto Dalla Libera, Ph.D., Ph.D. Candidates: Niccolò Turcato, Giulio Giacomuzzo and Prof. Ruggero Carli from University of Padua.
The International Joint Conference on Artificial Intelligence (IJCAI) is a premier gathering for AI researchers and organizes several competitions. This year the competition CC7 "AI Olympics With RealAIGym: Is AI Ready for Athletic Intelligence in the Real World?" consisted of two stages: simulation and real-robot experiments on two under-actuated robotic systems. The two robotics systems were treated as separate tracks and one final winner was selected for each track based on specific performance criteria in the control tasks.
The UniPD-MERL team competed and won in both tracks. The team's system made strong use of a Model-based Reinforcement Learning algorithm called (MC-PILCO) that we recently published in the journal IEEE Transaction on Robotics.
- A joint team consisting of members of University of Padua and MERL ranked 1st in the IJCAI2023 Challenge "Al Olympics With RealAlGym: Is Al Ready for Athletic Intelligence in the Real World?". The team was composed by MERL researcher Diego Romeres and a team from University Padua (UniPD) consisting of Alberto Dalla Libera, Ph.D., Ph.D. Candidates: Niccolò Turcato, Giulio Giacomuzzo and Prof. Ruggero Carli from University of Padua.
See All Awards for Robotics -
-
News & Events
-
TALK [MERL Seminar Series 2024] Samuel Clarke presents talk titled Audio for Object and Spatial Awareness Date & Time: Wednesday, October 30, 2024; 1:00 PM
Speaker: Samuel Clarke, Stanford University
MERL Host: Gordon Wichern
Research Areas: Artificial Intelligence, Machine Learning, Robotics, Speech & AudioAbstract
Acoustic perception is invaluable to humans and robots in understanding objects and events in their environments. These sounds are dependent on properties of the source, the environment, and the receiver. Many humans possess remarkable intuition both to infer key properties of each of these three aspects from a sound and to form expectations of how these different aspects would affect the sound they hear. In order to equip robots and AI agents with similar if not stronger capabilities, our research has taken a two-fold path. First, we collect high-fidelity datasets in both controlled and uncontrolled environments which capture real sounds of objects and rooms. Second, we introduce differentiable physics-based models that can estimate acoustic properties of objects and rooms from minimal amounts of real audio data, then can predict new sounds from these objects and rooms under novel, “unseen” conditions.
-
TALK [MERL Seminar Series 2024] Zhaojian Li presents talk titled A Multi-Arm Robotic System for Robotic Apple Harvesting Date & Time: Wednesday, October 2, 2024; 1:00 PM
Speaker: Zhaojian Li, Mivchigan State University
MERL Host: Yebin Wang
Research Areas: Artificial Intelligence, Computer Vision, Control, RoboticsAbstract
Harvesting labor is the single largest cost in apple production in the U.S. Surging cost and growing shortage of labor has forced the apple industry to seek automated harvesting solutions. Despite considerable progress in recent years, the existing robotic harvesting systems still fall short of performance expectations, lacking robustness and proving inefficient or overly complex for practical commercial deployment. In this talk, I will present the development and evaluation of a new dual-arm robotic apple harvesting system. This work is a result of a continuous collaboration between Michigan State University and U.S. Department of Agriculture.
See All News & Events for Robotics -
-
Internships
-
CA0055: Internship - Human-Collaborative Loco-Manipulation Robots
MERL seeks graduate students passionate about robotics to contribute to the development of a framework for legged robots with manipulator arms to collaborate with human in executing various tasks. The work will involve multi-domain research including planning and control, manipulation, and possibly vision/perception. The methods will be implemented and evaluated in high performance simulators and (time-permitting) in actual robotic platforms. The results of the interns are expected to be published in top-tier robotic conferences and/or journal.
The internship should start in January 2025 (exact date is flexible) with an expected duration 3-6 months depending on agreed scope and intermediate progress.
Required Specific Experience
- Current/Past enrollment in a PhD program in Mechanical, Aerospace, Electrical Engineering, with a concentration in Robotics
- 2+ years of research in at least some of: machine learning, optimization, control, path planning, computer vision
- Experience in design and simulation tools for robotics such as ROS, Mujoco, Gazebo, Isaac Lab
- Strong programming skills in Python and/or C/C++
Additional Desired Experience
- Development of planning and control methods in robotic hardware platforms
- Acquisition and processing of multimodal sensor data, including force/torque and proprioceptive sensors
- Prior experience in human-robot interaction, legged locomotion, mobile manipulation
-
CI0067: Internship - IoT Network Design methodology
MERL is seeking a highly motivated and qualified intern to carry out research on mobile IoT network design methodology. The candidate is expected to develop innovative mobile network technologies to support UAV assisted IoT networks. The candidates should have knowledge of mobile network technologies such as path planning and cooperative network operations. Knowledge of UAV technology and mobility management is a plus. Candidates in their junior or senior years of a Ph.D. program are encouraged to apply. Start date for this internship is flexible and the duration is 3 months.
The responsibilities of this intern position include (i) research on UAV assisted network design methodology; (ii) develop network configuration technologies to support UAV cooperative network operations; (iii) simulate and analyze the performance of developed technology.
-
CA0095: Internship - Infrastructure monitoring using quadrotors
MERL seeks graduate students passionate about robotics to collaborate and develop a framework for infrastructure monitoring using quadrotors. The work will involve multi-domain research, including multi-agent planning and control, SLAM, and perception. The methods will be implemented and evaluated on an actual robotic platform (Crazyflies). The results of the internship are expected to be published in top-tier conferences and/or journals. The internship will take place during summer 2025 (exact dates are flexible) with an expected duration of 3-4 months.
Please use your cover letter to explain how you meet the following requirements, preferably with links to papers, code repositories, etc., indicating your proficiency.
Required Specific Experience
- Current enrollment in a PhD program in Mechanical, Electrical Engineering, Computer Science, or related programs, with a focus on Robotics and/or Control Systems
- Experience in some/all of these topics: multi-agent motion planning, constrained control, SLAM, computer vision
- Experience with ROS2 and validation of algorithms on robotic platforms, preferably quadrotors
- Strong programming skills in Python and/or C/C++
Desired Specific Experience
- Experience with Crazyflie quadrotors and the Crazyswarm library
- Experience with the SLAM toolbox in ROS2
- Experience in convex optimization and model predictive control
- Experience with computer vision
See All Internships for Robotics -
-
Openings
See All Openings at MERL -
Recent Publications
- , "Chance-Constrained Optimization for Contact-rich Systems using Mixed Integer Programming", Nonlinear Analysis: Hybrid Systems, DOI: 10.1016/j.nahs.2024.101466, Vol. 52, December 2024.BibTeX TR2024-008 PDF
- @article{Shirai2024dec,
- author = {Shirai, Yuki and Jha, Devesh K. and Raghunathan, Arvind and Romeres, Diego},
- title = {Chance-Constrained Optimization for Contact-rich Systems using Mixed Integer Programming},
- journal = {Nonlinear Analysis: Hybrid Systems},
- year = 2024,
- volume = 52,
- month = dec,
- doi = {10.1016/j.nahs.2024.101466},
- issn = {1751-570X},
- url = {https://www.merl.com/publications/TR2024-008}
- }
- , "Memory-Based Learning of Global Control Policies from Local Controllers", 21st International Conference on Informatics in Control, Automation and Robotics (ICINCO'24), November 2024.BibTeX TR2024-158 PDF
- @inproceedings{Nikovski2024nov,
- author = {{Nikovski, Daniel N. and Zhong, Junmin and Yerazunis, William S.}},
- title = {Memory-Based Learning of Global Control Policies from Local Controllers},
- booktitle = {21st International Conference on Informatics in Control, Automation and Robotics (ICINCO'24)},
- year = 2024,
- month = nov,
- url = {https://www.merl.com/publications/TR2024-158}
- }
- , "Open Human-Robot Collaboration Systems (OHRCS): A Research Perspective", IEEE International Conference on Cognitive Machine Intelligence (CogML 2024), October 2024.BibTeX TR2024-150 PDF
- @inproceedings{Suresh2024nov,
- author = {{Suresh, Prasanth and Romeres, Diego and Dosh, Prashant and Jain, Siddarth}},
- title = {Open Human-Robot Collaboration Systems (OHRCS): A Research Perspective},
- booktitle = {IEEE International Conference on Cognitive Machine Intelligence (CogML 2024)},
- year = 2024,
- month = oct,
- url = {https://www.merl.com/publications/TR2024-150}
- }
- , "Insert-One: One-Shot Robust Visual-Force Servoing for Novel Object Insertion with 6-DoF Tracking", 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2024), October 2024.BibTeX TR2024-137 PDF
- @inproceedings{Chang2024oct,
- author = {Chang, Haonan and Boularias, Abdeslam and Jain, Siddarth}},
- title = {Insert-One: One-Shot Robust Visual-Force Servoing for Novel Object Insertion with 6-DoF Tracking},
- booktitle = {2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2024)},
- year = 2024,
- month = oct,
- url = {https://www.merl.com/publications/TR2024-137}
- }
- , "Learning control of underactuated double pendulum with Model-Based Reinforcement Learning", Competition: AI Olympics With RealAIGym, October 2024.BibTeX TR2024-142 PDF
- @inproceedings{DallaLibera2024oct,
- author = {Dalla Libera, Alberto and Turcato, Niccolò and Giacomuzzo, Giulio and Carli, Ruggero and Romeres, Diego}},
- title = {Learning control of underactuated double pendulum with Model-Based Reinforcement Learning},
- booktitle = {Competition: AI Olympics With RealAIGym},
- year = 2024,
- month = oct,
- url = {https://www.merl.com/publications/TR2024-142}
- }
- , "DECAF: a Discrete-Event based Collaborative Human-Robot Framework for Furniture Assembly", IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October 2024.BibTeX TR2024-138 PDF
- @inproceedings{Giacomuzzo2024oct,
- author = {Giacomuzzo, Giulio and Terreran, Matteo and Jain, Siddarth and Romeres, Diego}},
- title = {DECAF: a Discrete-Event based Collaborative Human-Robot Framework for Furniture Assembly},
- booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
- year = 2024,
- month = oct,
- url = {https://www.merl.com/publications/TR2024-138}
- }
- , "Open Human-Robot Collaboration using Decentralized Inverse Reinforcement Learning", 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2024), October 2024.BibTeX TR2024-135 PDF
- @inproceedings{Suresh2024oct,
- author = {Suresh, Prasanth and Jain, Siddarth and Doshi, Prashant and Romeres, Diego}},
- title = {Open Human-Robot Collaboration using Decentralized Inverse Reinforcement Learning},
- booktitle = {2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2024)},
- year = 2024,
- month = oct,
- url = {https://www.merl.com/publications/TR2024-135}
- }
- , "Autonomous Robotic Assembly: From Part Singulation to Precise Assembly", IEEE/RSJ International Conference on Intelligent Robots and Systems., October 2024.BibTeX TR2024-133 PDF
- @inproceedings{Ota2024oct,
- author = {Ota, Kei and Jha, Devesh K. and Jain, Siddarth and Yerazunis, William S. and Corcodel, Radu and Shukla, Yash and Bronars, Antonia and Romeres, Diego}},
- title = {Autonomous Robotic Assembly: From Part Singulation to Precise Assembly},
- booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems.},
- year = 2024,
- month = oct,
- url = {https://www.merl.com/publications/TR2024-133}
- }
- , "Chance-Constrained Optimization for Contact-rich Systems using Mixed Integer Programming", Nonlinear Analysis: Hybrid Systems, DOI: 10.1016/j.nahs.2024.101466, Vol. 52, December 2024.
-
Videos
-
Software & Data Downloads
