Convergence of a model-free entropy-regularized inverse reinforcement learning algorithm

Renard, TitouanSchlaginhaufen, AndreasNi, TingtingKamgarpour, Maryam2025-03-252025-03-252025-03-242024-1210.1109/CDC56724.2024.108860012-s2.0-86000622747https://infoscience.epfl.ch/handle/20.500.14299/248233Given a dataset of expert demonstrations, inverse reinforcement learning (IRL) aims to recover a reward for which the expert is optimal. This work proposes a model-free algorithm to solve the entropy-regularized IRL problem. In particular, we employ a stochastic gradient descent update for the reward and a stochastic soft policy iteration update for the policy. Assuming access to a generative model, we prove that our algorithm is guaranteed to recover a reward for which the expert is -optimal using an expected number of O(1 / 2) samples of the Markov decision process (MDP). Furthermore, with an expected number of O(1 / 4) samples we prove that the optimal policy corresponding to the recovered reward is -close to the expert policy in total variation distance.enfalseConvergence of a model-free entropy-regularized inverse reinforcement learning algorithmtext::conference output::conference proceedings::conference paper