Software & Data Downloads — OFENet

This Python code implements an online feature extractor network (OFENet) that uses neural nets to produce good representations to be used as inputs to deep RL algorithms. Even though the high dimensionality of input is usually supposed to make learning of RL agents more difficult, by using this network, we show that the RL agents in fact learn more efficiently with the high-dimensional representation than with the lower-dimensional state observations. We believe that stronger feature propagation together with larger networks (and thus larger search space) allows RL agents to learn more complex functions of states and thus improves the sample efficiency. The code also contains several test problems. Through numerical experiments on these problems, we show that the proposed method outperforms several other state-of-the-art algorithms in terms of both sample efficiency and performance. Implementations of these algorithms are also included in the code.

• Related Publications

   Ota, K., Oiki, T., Jha, D.K., Mariyama, T., Nikovski, D.N., "Can Increasing Input Dimensionality Improve Deep Reinforcement Learning?", International Conference on Machine Learning (ICML), Daumé III , Hal and Singh, Aarti, Eds., June 2020, pp. 7424-7433.

  BibTeX TR2020-083 PDF Software

  • @inproceedings{Ota2020jun,
  • author = {Ota, Kei and Oiki, Tomoaki and Jha, Devesh K. and Mariyama, Toshisada and Nikovski, Daniel N.},
  • title = {Can Increasing Input Dimensionality Improve Deep Reinforcement Learning?},
  • booktitle = {International Conference on Machine Learning (ICML)},
  • year = 2020,
  • editor = {Daumé III , Hal and Singh, Aarti},
  • pages = {7424--7433},
  • month = jun,
  • publisher = {PMLR},
  • url = {https://www.merl.com/publications/TR2020-083}
  • }

Access software at https://github.com/merlresearch/OFENet.