Zeyu Huang

I am a a PhD student (2024.01 - Now) at the University of Edinburgh. I am very lucky to be supervised by Ivan Titov and Edoardo M. Ponti. I am currently a Student Researcher at Google DeepMind London, advised by Marc'Aurelio Ranzato and Adhiguna Kuncoro.

Email / CV / Google Scholar / GitHub / Twitter / LinkedIn

Research

I am broadly interested in different kinds of learning algorithms, with a particular interest in designing general lifelong learners that can efficiently adapt to new tasks and environments over time.

My current research mainly focuses on large language models (LLMs). I have been working on Model Editing, Mixture of Experts (MoE), and Reinforcement Learning (RL) plus LLMs. If you are interested in these topics, please feel free to reach out to me!!!

Selected publications and preprints

Bending Supervised and Reinforcement Fine-Tuning with Prefix Sampling
Zeyu Huang, Tianhao Cheng, Zihan Qiu, Zili Wang, Yinghui Xu, Edoardo M Ponti, Ivan Titov.
arXiv, 2025 | code

A Controllable Examination for Long-Context Language Models
Yijun Yang*, Zeyu Huang*, Wenhao Zhu, Zihan Qiu, Fei Yuan, Jeff Z Pan, Ivan Titov.
arXiv, 2025 | code

Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
Zihan Qiu*, Zekun Wang*, Bo Zheng*, Zeyu Huang*, Kaiyue Wen, Songlin Yang, Rui Men, Le Yu, Fei Huang, Suozhi Huang, Dayiheng Liu, Jingren Zhou, Junyang Lin
arXiv, 2025 | code

Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models
Zihan Qiu*, Zeyu Huang*, Bo Zheng*, Kaiyue Wen, Zekun Wang, Rui Men, Ivan Titov, Dayiheng Liu, Jingren Zhou, Junyang Lin
ACL 2025 Main

Post-hoc Reward Calibration: A Case Study on Length Bias
Zeyu Huang, Zihan Qiu, Zili Wang, Edoardo M. Ponti, Ivan Titov
ICLR 2025 | code

Layerwise Recurrent Router for Mixture-of-Experts
Zihan Qiu*, Zeyu Huang*, Shuang Cheng, Yizhi Zhou, Zili Wang, Ivan Titov, Jie Fu
ICLR 2025 | code

Unlocking Emergent Modularity in Large Language Models
Zihan Qiu*, Zeyu Huang*, Shuang Cheng, Yizhi Zhou, Zili Wang, Ivan Titov, Jie Fu
NAACL 2024, 🏆 Outstanding Paper | code

Transformer-Patcher: One Mistake worth One Neuron
Zeyu Huang, Yikang Shen, Xiaofeng Zhang, Jie Zhou, Wenge Rong, Zhang Xiong
ICLR 2023 | code

website credits