Towards practical reinforcement learning for tokamak magnetic control
Reinforcement learning (RL) has shown promising results for real-time control systems, including the domain of plasma magnetic control. However, there are still significant drawbacks compared to traditional feedback control approaches for magnetic confinement. In this work, we address key drawbacks of the RL method; achieving higher control accuracy for desired plasma properties, reducing the steady-state error, and decreasing the required time to learn new tasks. We build on top of Degrave et al. (2022), and present algorithmic improvements to the agent architecture and training procedure. We present simulation results that show up to 65% improvement in shape accuracy, achieve substantial reduction in the long-term bias of the plasma current, and additionally reduce the training time required to learn new tasks by a factor of 3 or more. We present new experiments using the upgraded RL-based controllers on the TCV tokamak, which validate the simulation results achieved, and point the way towards routinely achieving accurate discharges using the RL approach.
WOS:001167503900001
2024-01-23
200
114161
REVIEWED
EPFL
Funder | Grant Number |
Kieran Milan | |
Swiss National Science Foundation | |
Euratom Research and Training Programme | 101052200 |
Swiss State Secretariat for Education, Research and Innovation (SERI) | |