TY - GEN
T1 - Federated Offline Reinforcement Learning for Autonomous Systems
AU - Park, Ju eun
AU - Woo, Honguk
N1 - Publisher Copyright:
© 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.
PY - 2022
Y1 - 2022
N2 - Offline reinforcement learning (RL) provides a safe learning method that can be applied to real-world applications through a data-driven learning process. In general, this process attributes to learning on large datasets, which is similar to conventional supervised learning techniques. In this paper, we propose a federated offline RL framework where policy parameters learned locally on separate datasets of RL agents are aggregated in communication rounds to achieve a high-performance global policy. The framework can solve the data insufficiency issue by performing offline RL among multiple agents without data sharing and integration. We show that using the framework, agents achieve a stable-performance control policy in a 2D navigation environment, even when each has only 1% of the required training trajectory amount (e.g., 10 trajectories).
AB - Offline reinforcement learning (RL) provides a safe learning method that can be applied to real-world applications through a data-driven learning process. In general, this process attributes to learning on large datasets, which is similar to conventional supervised learning techniques. In this paper, we propose a federated offline RL framework where policy parameters learned locally on separate datasets of RL agents are aggregated in communication rounds to achieve a high-performance global policy. The framework can solve the data insufficiency issue by performing offline RL among multiple agents without data sharing and integration. We show that using the framework, agents achieve a stable-performance control policy in a 2D navigation environment, even when each has only 1% of the required training trajectory amount (e.g., 10 trajectories).
KW - Autonomous systems
KW - Federated learning
KW - Offline reinforcement learning
UR - https://www.scopus.com/pages/publications/85140454483
U2 - 10.1007/978-3-031-17422-3_10
DO - 10.1007/978-3-031-17422-3_10
M3 - Conference contribution
AN - SCOPUS:85140454483
SN - 9783031174216
T3 - Communications in Computer and Information Science
SP - 107
EP - 117
BT - Computer and Communication Engineering - 2nd International Conference, CCCE 2022, Revised Selected Papers
A2 - Neri, Filippo
A2 - Du, Ke-Lin
A2 - Varadarajan, Vijaya Kumar
A2 - Angel-Antonio, San-Blas
A2 - Jiang, Zhiyu
PB - Springer Science and Business Media Deutschland GmbH
T2 - 2nd International Conference on Computer and Communication Engineering, CCCE 2022
Y2 - 11 March 2022 through 13 March 2022
ER -