TR2024-098

Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithms


    •  Zhang, X., Mao, W., Mowlavi, S., Benosman, M., Basar, T., "Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithms", Learning for Dynamics & Control Conference (L4DC), July 2024, pp. 181-196.
      BibTeX TR2024-098 PDF
      • @inproceedings{Zhang2024jul2,
      • author = {Zhang, Xiangyuan and Mao, Weichao and Mowlavi, Saviz and Benosman, Mouhacine and Basar, Tamer}},
      • title = {Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithms},
      • booktitle = {Learning for Dynamics & Control Conference (L4DC)},
      • year = 2024,
      • pages = {181--196},
      • month = jul,
      • publisher = {PMLR},
      • url = {https://www.merl.com/publications/TR2024-098}
      • }
  • MERL Contacts:
  • Research Areas:

    Artificial Intelligence, Computational Sensing, Dynamical Systems

Abstract:

We introduce controlgym, a library of thirty-six industrial control settings, and ten infinite- dimensional partial differential equation (PDE)-based control problems. Integrated within the OpenAI Gym/Gymnasium (Gym) framework, controlgym allows direct applications of standard reinforcement learning (RL) algorithms like stable-baselines3. Our control environments complement those in Gym with continuous, unbounded action and observation spaces, motivated by real-world control applications. Moreover, the PDE control environments uniquely allow the users to extend the state dimensionality of the system to infinity while preserving the intrinsic dy- namics. This feature is crucial for evaluating the scalability of RL algorithms for control. This project serves the learning for dynamics & control (L4DC) community, aiming to explore key questions: the convergence of RL algorithms in learning control policies; the stability and ro- bustness issues of learning-based controllers; and the scalability of RL algorithms to high- and potentially infinite-dimensional systems. We open-source the controlgym project at https://github.com/xiangyuan-zhang/controlgym.

 

  • Related Publication

  •  Zhang, X., Mao, W., Mowlavi, S., Benosman, M., Basar, T., "Controlgym: Large-Scale Safety-Critical Control Environments for Benchmarking Reinforcement Learning Algorithms", arXiv, November 2023.
    BibTeX arXiv
    • @article{Zhang2023nov,
    • author = {Zhang, Xiangyuan and Mao, Weichao and Mowlavi, Saviz and Benosman, Mouhacine and Basar, Tamer},
    • title = {Controlgym: Large-Scale Safety-Critical Control Environments for Benchmarking Reinforcement Learning Algorithms},
    • journal = {arXiv},
    • year = 2023,
    • month = nov,
    • url = {https://arxiv.org/abs/2311.18736}
    • }