Personal Information
Hello, I am Hongyao Tang, 汤宏垚. I am an Associate Research Fellow at TJU RL Lab, College of Intelligence and Computing, Tianjin University.
Prior to this, I was a postdoctoral researcher with Professor Glen Berseth, in Robotics and Embodied AI Lab (REAL) at the Mila and Université de Montréal. I obtained my Ph.D. (Master's and Bachelor's Degree as well) in Deep Reinforcement Learning (DRL) Lab, Tianjin University advised by Professor Jianye Hao, Zhaopeng Meng, and Li Wang.
My research interests lie in unveiling the learning dynamics of Deep Reinforcement Learning (DRL) and realizing new approaches/paradigms for efficient, performant and generalizable agents. My current research focus is on Learning under Nonstationarity (i.e., one intriguing nature of RL), and concretely, I study Continual RL and RL problems in LLMs and embodied intelligence. I am also interested in Meta RL, MARL, Offline RL, Machine Unlearning, etc.
I have experience in applying DRL in practical problems like Electronic Design Automation (EDA), Drug Discovery, Online Games and etc. I am very willing to contribute to addressing real-world problems.
⚡️ Highlight
For prospective students (master, ph.d., RA, etc.) and collaboration partners, please contact me via my email (above) or contact via the homepage of TJU RL Lab.
欢迎有意向的同学（硕士、博士、研究助理等）以及有合作意向的伙伴通过上方邮箱联系我，或通过 天津大学强化学习实验室 主页与我们取得联系 🚗🚗🚗
🌟 News
- 2026.09 NeurIPS 2026 Eight papers accepted to NeurIPS 2026 on LLM question difficulty intuition 🔗, diffusion models for online RL 🔗, robustness-driven LLM self-training 🔗, multiagent RL at high replay ratios 🔗, monotonic inference policies for LLM RL 🔗, embodied foundation models 🔗, reformulating LLM RL under black-box discrepancy 🔗, and contact-rich manipulation 🔗!
- 2026.08 WarpSAC We propose WarpSAC (i.e., at "Warp Speed"🚀) that outperforms FlashSAC across 20+ CPU/GPU-scale environments, and points toward in-minutes sim-to-real deployment for Unitree G1!
- 2026.08 EMNLP 2026 One paper accepted to EMNLP 2026 (Findings) on critic-guided embodied agent 🔗!
- 2026.07 ACM MM 2026 One paper accepted to ACM MM 2026 on grounded embodied reasoning via pinned chain-of-thought 🔗!
- 2026.06 Embodied-R1.5 We released Embodied-R1.5🤖🧠, a 8B embodied foundation model that has comprehensive embodied reasoning capabilitys, which achieves SOTA performance on 16/24 embodied VLM benchmarks and can be fine-tuned into a VLA that outperforms π0.5 across 4 popular manipulation benchmark suites
- 2026.05 KDD 2026 One paper accepted to KDD 2026 on Code Representation for AutoML 🔗!
- 2026.04 ICML 2026 Two papers accepted as ICML 2026 Spotlight💎 on DRL scaling with MOE structure 🔗 and efficient non-forgetting network for continual RL 🔗!
- 2026.04 中国科学基金 Our survey report for Embodied Intelligence 🔗🤖 has been published on 《中国科学基金》!
- 2026.01 ICLR 2026 Three papers accepted to ICLR 2026 on off-policy RFT for LLM Reasoning 🔗, plasticity loss in RL 🔗 and reinforced embodied reasoning for general robotic manipulation 🔗!
- 2025.12 AAMAS 2026 One paper accepted as AAMAS 2026 Oral🌈 on a unified theory of policy abstraction and representation 🔗!
- 2025.09 NeurIPS 2025 Four papers accepted to NeurIPS 2025 on LLM-driven reward function design 🔗, multi-objective RL 🔗, offline RL 🔗, and EDA floorplanning 🔗!
- 2025.05 RLC 2025 One paper accepted to RLC 2025 on efficient cross-morphology adaptation 🔗!
- 2025.05 ICML 2025 Two papers accepted to ICML 2025 on continual RL 🔗 and LLM-driven reward design 🔗!
- 2025.01 Faculty Pos. Excited to join TJU RL Lab as an Associate Research Fellow!
- 2024.12 AAMAS 2025 One paper accepted to AAMAS 2025 on multi-agent RL with better sample efficiency 🔗!
- 2024.09 NeurIPS 2024 Two papers accepted to NeurIPS 2024 on network churn 🔗 and learning dynamics of DRL 🔗!
- 2023.11 PostDoc Excited to join Mila/UdeM as a Postdoctoral Research Fellow! Working with Prof. Glen Berseth.
- 2023.06 Ph.D. Got my Ph.D. at Tianjin University! Grateful to Prof. Jianye Hao and Prof. Zhaopeng Meng.
🛣 Education & Work Experiences
🧠 Selected Publications & Preprints
PS: Authors with equal contribution are marked by *. First / co-first authors played by me are marked by 🔥. Corresponding authors are marked by 💧.
arXiv preprint 2026
Can Jev be Your Q or Policy in Reinforcement Learning?
⚡️TL;DR: shows a frozen decision model can serve as a reference policy, exploration judge, and replay rater in RL training, beating standard RL learners on 9 MiniGrid tasks and 3 Atari games without the model itself being trained
arXiv preprint 2026
Code Owns the Simulation, Jev Owns the Evaluation
⚡️TL;DR: shows judgment models like Jev ace evaluation but fail at simulation, solving 99% of CRT questions yet walking an unwashed knife to the drawer, and that handing the prediction to code turns them into expert controllers
arXiv preprint 2026
WarpSAC: Towards the Pinnacle of Scalable Off-policy RL by Rethinking Exploration and Exploitation
⚡️TL;DR: We propose WarpSAC, a regime-aware family of off-policy RL algorithms that beats FlashSAC by 4.5%/23.1% AUC across 9 CPU-scale/14 GPU-scale envs and points towards an in-minutes sim-to-real deployment for Unitree G1 with a single high-end GPU like B200
NeurIPS 2026
Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models
⚡️TL;DR: Embodied-R1.5 is a unified Embodied Foundation Model that integrates comprehensive embodied reasoning capabilities within a single 8B-parameter architecture. It achieves SOTA on 16 out of 24 embodied VLM benchmarks and can be fine-tuned into a VLA that outperforms π0.5 across 4 popular manipulation benchmark suites
NeurIPS 2026
What Kind of Diffusion Models Do We Need in Online Reinforcement Learning?
⚡️TL;DR: proposes a taxonomy of existing diffusion RL methods into three families, discusses the limitations of each, and proposes Coupled Flow with tractable likelihood, beating diffusion-based RL baselines by 100% on HumanoidBench
arXiv preprint 2026
Generalized Agent Iteration: One Formal Framework for Iterative Policy Improvement and Recursive Self-Improvement
⚡️TL;DR: proposes Generalized Agent Iteration (GAI), a formal framework that casts iterative policy improvement and recursive self-improvement as one learning paradigm, with two dials placing existing systems on shared axes and making RSI's defects statable one condition at a time
NeurIPS 2026
The Mirage of Optimizing Training Policies: Monotonic Inference Policies as the Real Objective for LLM Reinforcement Learning
⚡️TL;DR: reveals that training-inference mismatch in LLM RL creates a persistent off-policy problem due to objective misalignment, and proposes MIPI objective with MIPU framework that selectively accepts synchronized policy updates via an inference-side gap proxy, achieving better reasoning performance and training stability
ICML 2026 (Spotlight💎)
ScaleMoE: Mixture-of-Experts for Scalable Continuous Control in Actor-Critic Reinforcement Learning
⚡️TL;DR: proposes a new way to scale deep AC methods with MoE architectures, outperforms SimBa and BRC in single-task RL and multi-task RL, respectively
ICML 2026 (Spotlight💎)
Neuro-evolutionary Continual Reinforcement Learning
⚡️TL;DR: dynamically allocates, reuses, and re-cycles neurons in a monolithic model, guarantees no forgetting, achieves SOTA results in ContinualWorld
KDD 2026
PACE: Unleashing the Power of Code Embeddings to Boost AutoML Agents
⚡️TL;DR: idenifies the structural locality in the multi-view low-d representation space of code solutions, leverages it to build admission strategies that realizes more efficient autoML
NeurIPS 2024
The Ladder in Chaos: Improving Policy Learning by Harnessing the Parameter Evolving Path in A Low-dimensional Space
⚡️TL;DR: identifies the low-rank param space of policy evolvement with temporal SVD, improves learning stability and efficiency by editing the SVD coeffs directly
ICLR 2026
Squeeze the Soaked Sponge: Efficient Off-policy Reinforcement Finetuning for Large Language Model
⚡️TL;DR: revisits Generalized PPO and extends it to LLM RL post-training, achieves SOTA level math reasoning results with a 30x-450x🚀 rollout cost reduction
AAMAS 2026 (Full Paper, Oral🌈)
Towards A Unified Policy Abstraction Theory and Representation Learning Approach in Markov Decision Processes
⚡️TL;DR: just as the title, proposes the systematic definition, hierarchy of policy abstraction and representation for the first time, a solute to Li et al. (2006)
AAAI 2022 Oral🌈 Presentation (< 5%)
What about Inputting Policy in Value Function: Policy Representation and Policy-Extended Value Function Approximator
⚡️TL;DR: proposes a new RL paradigm that allows value generalization among policies and utilize it to improve learning efficiency, learns low-d representation for NN policies
More Publications
Academic Service
Reviewer
RLC
2024 - 2026
NeurIPS
2021 - 2026 (Top Reviewer Award at NeurIPS 2022)
ICLR
2022 - 2027 (Highlighted Reviewer Award at ICLR 2022)
ICML
2021 - 2026
AAMAS
2021 - 2024
Nature Communications
Transactions on Machine Learning Research (TMLR)
IEEE Transactions on Neural Networks and Learning Systems (TNNLS)
Invited Talks
2025.03
RL in the Era of Large Models
2025 IEEE International Conference on Industrial Technology
2024.10
Where is the Road to Flawless RL? — Unsolved Problems and New Approaches
Google DeepMind, Discovery Team London (Remote)
2023.05
A Tale of Representations in Deep Reinforcement Learning
Robotics and Embodied AI Lab (REAL), the Université de Montréal
2022.07
Towards Understanding The Learning Dynamics of Deep Reinforcement Learning
Huawei Noah's Ark Lab, Decision-making and Reasoning Group (during internship)
2021.10
Self-supervised Reinforcement Learning — A Perspective of Representation
2021 TJU RL Summer Seminar
2020.11
State Abstraction and State Representation Learning in Reinforcement Learning
Huawei Noah's Ark Lab, Decision-making and Reasoning Group (during internship)
2019.08
Bias and Variance in Deep Reinforcement Learning
2019 TJU RL Summer Seminar