TR2024-098
Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithms
-
- "Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithms", Learning for Dynamics & Control Conference (L4DC), July 2024, pp. 181-196.BibTeX TR2024-098 PDF
- @inproceedings{Zhang2024jul2,
- author = {Zhang, Xiangyuan and Mao, Weichao and Mowlavi, Saviz and Benosman, Mouhacine and Basar, Tamer}},
- title = {Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithms},
- booktitle = {Learning for Dynamics & Control Conference (L4DC)},
- year = 2024,
- pages = {181--196},
- month = jul,
- publisher = {PMLR},
- url = {https://www.merl.com/publications/TR2024-098}
- }
,
- "Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithms", Learning for Dynamics & Control Conference (L4DC), July 2024, pp. 181-196.
-
MERL Contacts:
-
Research Areas:
Artificial Intelligence, Computational Sensing, Dynamical Systems
Abstract:
We introduce controlgym, a library of thirty-six industrial control settings, and ten infinite- dimensional partial differential equation (PDE)-based control problems. Integrated within the OpenAI Gym/Gymnasium (Gym) framework, controlgym allows direct applications of standard reinforcement learning (RL) algorithms like stable-baselines3. Our control environments complement those in Gym with continuous, unbounded action and observation spaces, motivated by real-world control applications. Moreover, the PDE control environments uniquely allow the users to extend the state dimensionality of the system to infinity while preserving the intrinsic dy- namics. This feature is crucial for evaluating the scalability of RL algorithms for control. This project serves the learning for dynamics & control (L4DC) community, aiming to explore key questions: the convergence of RL algorithms in learning control policies; the stability and ro- bustness issues of learning-based controllers; and the scalability of RL algorithms to high- and potentially infinite-dimensional systems. We open-source the controlgym project at https://github.com/xiangyuan-zhang/controlgym.
Related Publication
- @article{Zhang2023nov,
- author = {Zhang, Xiangyuan and Mao, Weichao and Mowlavi, Saviz and Benosman, Mouhacine and Basar, Tamer},
- title = {Controlgym: Large-Scale Safety-Critical Control Environments for Benchmarking Reinforcement Learning Algorithms},
- journal = {arXiv},
- year = 2023,
- month = nov,
- url = {https://arxiv.org/abs/2311.18736}
- }