Zeyu Huang

I am a PhD student (2024.01 - Now) at the University of Edinburgh. I am very lucky to be supervised by Ivan Titov and Edoardo M. Ponti. I am honored to be named a 2026 Apple Scholar in AIML (1 of 20 globally).

I am broadly interested in different kinds of learning algorithms, with a particular interest in designing general lifelong learners that can efficiently adapt to new tasks and environments over time.

My current research mainly focuses on large language models (LLMs). I have been working on Model Editing, Mixture of Experts (MoE), and Reinforcement Learning (RL) plus LLMs. If you are interested in these topics, please feel free to reach out to me!!!

Email / CV / Google Scholar / GitHub / Twitter / LinkedIn

Selected Awards

2026 Apple PhD Scholar in AI/ML (1 of 20 globally)
NeurIPS 2025 Best Paper Award (4 / 21575 submissions), co-first-author
NAACL 2024 Outstanding Paper Award (6 / 2604 submissions), co-first-author
EPSRC DTA Scholarship (international flexibility), 2024-2027
National Scholarship for Graduate Students Award, 2021-2022 (3 / 291)
Outstanding Graduate Student of Beihang University, 2020-2021

Internships

Qwen Team, Alibaba Group - AliStar Intern (July 2026), Beijing
Advisor: Dr. Bo Zheng and Zihan Qiu
ByteDance Seed - TopSeed Intern (Mar. 2026 - Present), Shanghai
Advisors: Dr. Wenhao Zhu and Shanbo Cheng
Google DeepMind - Student Researcher (Aug. 2025 - Feb. 2026), London
Advisor: Dr. Marc'Aurelio Ranzato
INF Technology - Research Intern (Mar. 2024 - Aug. 2024), Remote
Advisor: Zili Wang
BAAI - Research Intern (Sep. 2022 - Jun. 2023), Beijing, China
Advisor: Dr. Jie Fu
WeChat AI - Research Intern (Nov. 2021 - May 2022), Beijing, China
Advisor: Dr. Yikang Shen

Selected Preprints

Context Training with Active Information Seeking
Zeyu Huang, Adhiguna Kuncoro, Qixuan Feng, Jiajun Shen, Lucio Dery, Arthur Szlam, Marc'Aurelio Ranzato.
arXiv, 2026

The Cancellation Hypothesis in Critic-Free RL: From Outcome Rewards to Token Credits
Tianhao Cheng*, Zeyu Huang*, Zihan Qiu, Yu Cheng, Edoardo Ponti, Yinghui Xu, Ivan Titov, Zenglin Xu.
arXiv, 2026

A Unified View of Attention and Residual Sinks: Outlier-Driven Rescaling is Essential for Transformer Training
Zihan Qiu*, Zeyu Huang*, Kaiyue Wen*, Peng Jin*, Bo Zheng*, ... , Dayiheng Liu, Jingren Zhou, Junyang Lin.
arXiv, 2026

Selected Publications

Blending Supervised and Reinforcement Fine-Tuning with Prefix Sampling
Zeyu Huang, Tianhao Cheng, Zihan Qiu, Zili Wang, Yinghui Xu, Edoardo M Ponti, Ivan Titov.
ICML 2026 | code

A Controllable Examination for Long-Context Language Models
Yijun Yang*, Zeyu Huang*, Wenhao Zhu, Zihan Qiu, Fei Yuan, Jeff Z Pan, Ivan Titov.
NeurIPS 2025 DB Track, 🏆 Spotlight (56 / 1995 submissions) | code

Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
Zihan Qiu*, Zekun Wang*, Bo Zheng*, Zeyu Huang*, Kaiyue Wen*, ... , Dayiheng Liu, Jingren Zhou, Junyang Lin
NeurIPS 2025, 🏆 Oral and Best Paper Award (4 / 21575 submissions) | code

Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models
Zihan Qiu*, Zeyu Huang*, Bo Zheng*, Kaiyue Wen, Zekun Wang, Rui Men, Ivan Titov, Dayiheng Liu, Jingren Zhou, Junyang Lin
ACL 2025 Main

Post-hoc Reward Calibration: A Case Study on Length Bias
Zeyu Huang, Zihan Qiu, Zili Wang, Edoardo M. Ponti, Ivan Titov
ICLR 2025 | code

Layerwise Recurrent Router for Mixture-of-Experts
Zihan Qiu*, Zeyu Huang*, Shuang Cheng, Yizhi Zhou, Zili Wang, Ivan Titov, Jie Fu
ICLR 2025 | code

Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training
Wenyu Du*, Tongxu Luo*, Zihan Qiu, Zeyu Huang, Yikang Shen, Reynold Cheng, Yike Guo, Jie Fu
NeurIPS 2024, 🏆 Spotlight (325 / 15671 submissions) | code

Unlocking Emergent Modularity in Large Language Models
Zihan Qiu*, Zeyu Huang*, Jie Fu
NAACL 2024, 🏆 Outstanding Paper (6 / 2604 submissions) | code

Transformer-Patcher: One Mistake worth One Neuron
Zeyu Huang, Yikang Shen, Xiaofeng Zhang, Jie Zhou, Wenge Rong, Zhang Xiong
ICLR 2023 | code

website credits