Average Reward Reinforcement Learning

Principal Investigator: Vaneet Aggarwal

Most real world problems have infinite horizon average reward objectives, while this case has not been as well understood. The key reason is that the contraction operation that gives the key results in the discounted setup no longer holds. In our work, we aim to give the foundations of average reward reinforcement learning. 

Representative Publications

• Bhrij Patel, Wesley A. Suttle, Alec Koppel, Vaneet Aggarwal, Brian M. Sadler, Amrit Singh Bedi, and Dinesh Manocha, "Global Optimality without Mixing Time Oracles in Average-reward RL via Multi-level Actor-Critic," in Proc. ICML, Jul 2024

• Qinbo Bai, Washim Uddin Mondal, and Vaneet Aggarwal, "Regret Analysis of Policy Gradient Algorithm for Infinite Horizon Average Reward Markov Decision Processes," in Proc. AAAI, Feb 2024. 

• Qinbo Bai, Washim Uddin Mondal, and Vaneet Aggarwal, "Learning General Parameterized Policies for Infinite Horizon Average Reward Constrained MDPs via Primal-Dual Policy Gradient Algorithm," in Proc. Neurips, Dec 2024 

• Swetha Ganesh, Washim Uddin Mondal, Vaneet Aggarwal, "Variance-Reduced Policy Gradient Approaches for Infinite Horizon Average Reward Markov Decision Processes," arXiv, Apr 2024

• Swetha Ganesh and Vaneet Aggarwal, "An Accelerated Multi-level Monte Carlo Approach for Average Reward Reinforcement Learning with General Policy Parametrization," arXiv, Jul 2024.