Events

Prof. Elynn Chen
Assistant Professor of Technology, Operations, and Statistics
Stern School of Business
New York University

In dynamic decision-making scenarios across business, healthcare, and education, leveraging data from diverse populations can significantly enhance reinforcement learning (RL) performance for specific target populations, especially when target samples are limited. We develop comprehensive frameworks for transfer learning in RL, addressing both stationary Markov decision processes (MDPs) with iterative Q*-learning and non-stationary finite-horizon MDPs with backward inductive Q*-learning.

For stationary MDPs, we propose an iterative Q*-learning algorithm with knowledge transfer, establishing theoretical justifications through faster convergence rates under similarity assumptions. For non-stationary finite-horizon MDPs, we introduce two key innovations: (1) a novel “re-weighted targeting procedure” that enables cross-satege transfer along multiple temporal steps, and (2) transfer deep Q*-learning that leverages neural networks as function approximators. We demonstrate that while naive sample pooling strategies may succeed in regression settings, they fail in MDPs, necessitating our more sophisticated approach. We establish theoretical guarantees for both settings, revealing the relationship between statistical performance and MDP task discrepancy. Our analysis illuminates how source and target sample sizes impact transfer effectiveness. The framework accommodates both transferable and non-transferable transition density ratios while assuming reward function transferability. Our analytical techniques have broader implications, extending to supervised transfer learning with neural networks and domain shift scenarios. Empirical evidence from both synthetic and real datasets validates our theoretical results, demonstrating significant improvements over single-task learning rates and highlighting the practical value of strategically constructed transferable RL samples in both stationary and non-stationary contexts.

This talk is based on the following three papers.
Transfer Q*-Learning for Offline Non-Stationary Reinforcement Learning: Manuscript on demand.
Data-Driven Knowledge Transfer in Batch Q* Learning: https://arxiv.org/abs/2202.04709
Transfer Q-Learning: https://arxiv.org/abs/2202.04709

Prof. Elynn Chen is an Assistant Professor of Technology, Operations and Statistics (TOPS) at NYU Stern School of Business. Her research focuses on developing innovative methodologies for data-driven decision-making and complex data analysis, with applications spanning business, economics, and healthcare domains.

Before joining NYU Stern in 2021, she completed postdoctoral fellowships at UC Berkeley’s EECS department with Prof. Michael I. Jordan, and at Princeton University’s ORFE department working with Prof. Jianqing Fan. During her academic journey, she also served as a Research Scholar at OpenAI in 2019. Her research contributions have been recognized with the NSF Postdoc Award.

Prof. Chen holds a Ph.D. in Statistics from Rutgers University, where she was advised by Prof. Rong Chen working in tensor time series. Her research spans three main areas: tensor learning, statistical reinforcement learning, and transfer learning. In tensor learning, she develops efficient algorithms for multi-dimensional data analysis, offering more natural representations of complex physical phenomena. Her work in statistical reinforcement learning focuses on designing algorithms for social applications across business, education, and healthcare sectors, with the goal of optimizing decision-making processes and improving outcomes. Additionally, she investigates transfer learning methods to enhance performance across related tasks, particularly in the contexts of reinforcement learning and tensor analysis. Her work has made impacts in various fields, including international trade, corporate finance, and clinical dynamic treatments. Through her research and teaching, she continues to bridge the gap between complex statistical methodologies and their practical applications in business and healthcare settings.