Chenyang Lyu 吕晨阳

Staff Researcher at Alibaba and PhD from Ml-Labs, Dublin City University
Glasnevin, Dublin
Ireland

Email: lyuchenyang.dcu [at] gmail [dot] com
Google Scholar   Twitter   DBLP   LinkedIn

About

I am currently a Staff Researcher/Tech Lead at the Alibaba AI Business Group, where I head the speech team. Previously, I was a researcher at the Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), focusing on multilingual and multimodal large language models. I earned my Ph.D. in Machine Learning from Dublin City University's ML-Labs in 2023, following a Bachelor of Engineering from Northeastern University in China in 2018. My research lies primarily in natural language processing, especially the application of large models—including vision-language models—to multilingual and multimodal tasks. I have published over 30 papers at top-tier conferences such as ACL, EMNLP, NeurIPS, and ACM-MM, with my GPT4Video work nominated for the Best Paper Award at ACM-MM 2024. My Google Scholar citations exceed 1,600, with an h-index of 19, and the open-source projects I have led or contributed to have collectively garnered over 4k stars on GitHub. I won two championships and two runner-up prizes in the IWSLT 2025 speech translation competition. I also serve as an area chair, program committee member, shared task organizer, and reviewer for several leading conferences including ICLR, ACL, EMNLP, IJCAI, and ACM-MM. Prior to my current role, I gained extensive research experience in large language models through positions as a research assistant and visiting scholar at Tencent AI Lab, Japan's National Institute of Informatics (NII) and IBM Research-China. My work has been recognized with several awards, including the IWSLT 2025 Speech Translation Competition championship, the ACM-MM 2024 Best Paper nomination, the German DAAD AInet Fellowship, the 2023 Irish AI Young Talent of the Year Award and an SFI PhD Scholarship. Additionally, my research has been featured in media outlets such as Irish national broadcaster RTÉ, Slator, Irish Tech News, and Irish podcasts.

News

• [2026/02] One paper elastic transformer is accepted to CVPR 2026.
• [2026/01] Three papers including Marco-Voice,Long Speech, MECAP-R1 are accepted to ICASSP 2026.
• [2025/12] Invited Industry Expert Talk on Marco Models at ACM Multimedia Asia 2025, hosted by Dr. Jianquan Liu (NEC).
• [2025/11] Invited Talk on LLMs Hallucination Detection at Southern University of Science and Technology (SUSTech), hosted by Dr. Linyi Yang.
• [2025/07] Our team secured Two Championships and Two Runner-ups at the IWSLT 2025 International Speech Translation Competition.
• [2025/05] Four papers including multilingual LLMs, hallucination detection are accepted to ACL 2025.
• [2024/10] Marco-LLM featured by global media including Bloomberg, CNBC, and South China Morning Post (SCMP).

• [2023/06]: We have released our Multi-modal Large Language Models named Macaw-LLM!.

Educational Background

• Sep 2019 - Jul 2023, Doctor of Philosophy in Computer Science, ML-Labs, Dublin City University
• Oct 2014 - June 2018, Bachelor of Engineering in Computer Software Engineering, Northeastern University

Research Experience

• Research Intern, Huawei Noah's Ark Lab, May 2020 - May 2021
• Research Intern, IBM Research-China, July 2018 - December 2018

Selected Publications

• LongSpeech: A Scalable Benchmark for Transcription, Translation and Understanding in Long Speech
  Fei Yang, Xuanfan Ni, Renyi Yang, Jiahui Geng, Qing Li, Chenyang Lyu*, Yichao Du, Longyue Wang, Weihua Luo, Kaifu Zhang
  In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2026)
• MECap-R1: Emotion-aware Policy with Reinforcement Learning for Multimodal Emotion Captioning
  Haoqin Sun, Chenyang Lyu*, Xiangyu Kong, Shiwan Zhao, Jiaming Zhou, Hui Wang, Aobo Kong, Jinghua Zhao, Longyue Wang, Weihua Luo, Kaifu Zhang, Yong Qin
  In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2026)
• Marco-Voice Technical Report
  Fengping Tian, Chenyang Lyu*, Xuanfan Ni, Haoqin Sun, Qingjuan Li, Zhiqiang Qian, Haijun Li, Longyue Wang, Zhao Xu, Weihua Luo, Kaifu Zhang
  In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2026)
• New Trends for Modern Machine Translation with Large Reasoning Models
  Sinuo Liu, Chenyang Lyu*, Minghao Wu, Longyue Wang, Weihua Luo, Kaifu Zhang
  In International Conference on Language Resources and Evaluation (LREC 2026)
• GigaMoE: Sparsity-Guided Mixture of Experts for Efficient Gigapixel Object Detection
  Xiang Li, Wenxi Li, Yuetong Wang, Chenyang Lyu, Haozhe Lin, Guiguang Ding, Yuchen Guo
  In AAAI Conference on Artificial Intelligence (AAAI 2026)
• ElasticFormer: Detecting Objects in HRW Shots via Elastic Computing Vision Transformer
  Xiang Li, Wenxi Li, Yuetong Wang, Chenyang Lyu, Haozhe Lin, Guiguang Ding, Yuchen Guo
  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2026)
• Marco-Bench-MIF: On Multilingual Instruction-Following Capability of Large Language Models
  Wenxi Li, Jingchen Huang, Chenyang Lyu*, Mo-Ran Liu, Haozhe Lin, Guiguang Ding, Yuchen Guo
  In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025)
• HD-NDEs: Neural Differential Equations for Hallucination Detection in LLMs
  Qing Li, Jiahui Geng, Zongxiong Chen, Derui Zhu, Yuxia Wang, Congbo Ma, Chenyang Lyu, Fakhri Karray
  In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025)
• Marco-o1 v2: Towards Widening The Distillation Bottleneck for Reasoning Models
  Huifeng Yin, Yu Zhao, Minghao Wu, Xuanfan Ni, Bo Zeng, Hao Wang, Tianqi Shi, Liangying Shao, Chenyang Lyu, Longyue Wang, Weihua Luo, Kaifu Zhang
  In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025)
• VSCBench: Bridging the Gap in Vision-Language Model Safety Calibration
  Jiahui Geng, Qing Li, Zongxiong Chen, Yuxia Wang, Derui Zhu, Zhuohan Xie, Chenyang Lyu, Xiuying Chen, Preslav Nakov, Fakhri Karray
  In Findings of the Association for Computational Linguistics: ACL 2025
• EditEval: Towards Comprehensive and Automatic Evaluation for Text-guided Video Editing
  Bingshuai Liu, Ante Wang, Zijun Min, Chenyang Lyu, Longyue Wang, Zhihao Wang, Xu Han, Peng Li, Jinsong Su
  In Proceedings of the 33rd ACM International Conference on Multimedia (ACM-MM 2025)
• Enhancing Video-Text Matching via Sparse Stratified Sampling
  Chenyang Lyu, Wenxi Li, Tianbo Ji, Liting Zhou, Pintu Lohar, Yi Yu, Longyue Wang
  In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2025)
• Retrieval-augmented Multi-modal Chain-of-Thoughts Reasoning for Large Language Models
  Bingshuai Liu*, Chenyang Lyu*, Zijun Min, Zhanyu Wang, Jinsong Su, Longyue Wang
  In 2025 International Joint Conference on Neural Networks (IJCNN 2025)
• CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark
  David Romero*, Chenyang Lyu*, Haryo Akbarianto Wibowo, Teresa Lynn, Injy Hamed, Rada Mihalcea, Thamar Solorio, Alham Fikri Aji
  In Advances in Neural Information Processing Systems, Dataset and Benchmark Track (NeurIPS 2024) (Oral presentation)
• Reference-free Hallucination Detection for Large Vision-Language Models
  Qing Li, Chenyang Lyu, Jiahui Geng, Derui Zhu, Maxim Panov, Fakhri Karray
  In Findings of the Association for Computational Linguistics: EMNLP 2024
• GPT4Video: A Unified Multimodal Large Language Model for lnstruction-Followed Understanding and Safety-Aware Generation
  Zhanyu Wang, Longyue Wang, Zhen Zhao, Minghao Wu, Chenyang Lyu, Huayang Li, Deng Cai, Luping Zhou, Shuming Shi, Zhaopeng Tu
  In Proceedings of the 32nd ACM International Conference on Multimedia (ACM-MM 2024) Best Paper Awards Nomination
• Benchmarking and Improving Long-Text Translation with Large Language Models
  Longyue Wang, Zefeng Du, Wenxiang Jiao, Chenyang Lyu, Jianhui Pang, Leyang Cui, Kaiqiang Song, Derek F. Wong, Shuming Shi, Zhaopeng Tu
  In Findings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024)
• Semantic Enrichment For Video Question Answering With Gated Graph Neural Networks
  Chenyang Lyu, Wenxi Li, Tianbo Ji, Yi Yu, Longyue Wang
  In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2024)
• A Paradigm Shift: The Future of Machine Translation Lies with Large Language Models
  Chenyang Lyu, Zefeng Du, Jitao Xu, Yitao Duan, Minghao Wu, Teresa Lynn, Alham Fikri Aji, Derek F. Wong, Longyue Wang
  In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
• From Multiple-Choice to Extractive QA: A Case Study for English and Arabic
  Teresa Lynn, Malik H Altakrori, Samar Mohamed Magdy, Rocktim Jyoti Das, Chenyang Lyu, Mohamed Nasr, Younes Samih, Alham Fikri Aji, Preslav Nakov, Shantanu Godbole, Salim Roukos, Radu Florian, Nizar Habash
  In Proceedings of the 31st International Conference on Computational Linguistics (COLING 2025)

• Graph-Based Video-Language Learning with Multi-Grained Audio-Visual Alignment [PDF][Code][Bibtex]
  Chenyang Lyu, Wenxi Li, Tianbo Ji, Longyue Wang, Liting Zhou, Cathal Gurrin, Linyi Yang, Yi Yu, Yvette Graham, Jennifer Foster
  In Proceedings of the 31st ACM International Conference on Multimedia, ACM-MM 2023

• Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integration [PDF][Code][Bibtex]
  Chenyang Lyu, Minghao Wu, Longyue Wang, Xinting Huang, Bingshuai Liu, Zefeng Du, Shuming Shi, and Zhaopeng Tu.
  Preprint 2023 (1100+ stars on Github, 300,000 views and discussion on Twitter).

• Semantic-aware Dynamic Retrospective-Prospective Reasoning for Event-level Video Question Answering [PDF][Code][Bibtex]
  Chenyang Lyu, Tianbo Ji, Yvette Graham, and Jennifer Foster
  In Proceedings of the 61th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, ACL 2023

• Is a Video worth $n\times n$ Images? A Highly Efficient Approach to Transformer-based Video Question Answering [PDF][Code][Bibtex]
  Chenyang Lyu, Tianbo Ji, Yvette Graham, and Jennifer Foster
  In Proceedings of The Third Workshop on Simple and Efficient Natural Language Processing, ACL 2023

• Exploiting Rich Textual User-Product Context for Improving Personalised Sentiment Analysis [PDF][Code][Bibtex]
  Chenyang Lyu, Linyi Yang, Yue Zhang, Yvette Graham, Jennifer Foster
  In Findings of the 61th Annual Meeting of the Association for Computational Linguistics, ACL 2023

• New Trends in Machine Translation with Large Language Models [PDF] [Code][Bibtex]
  Chenyang Lyu, Zefeng Du, Jitao Xu, Derek F. Wong, Yitao Duan, Longyue Wang
  Symposium on Large Language Models at IJCAI2023

• Document-Level Machine Translation with Large Language Models [PDF] [Slides][Code][Bibtex]
  Longyue Wang*, Chenyang Lyu*, Tianbo Ji*, Zhirui Zhang*, Dian Yu, Shuming Shi, Zhaopeng Tu
  EMNLP 2023

• Dialogue-to-Video Retrieval [PDF] [Slides][Code][Bibtex]
  Chenyang Lyu, Manh-Duy Nguyen, Van-Tu Ninh, Liting Zhou, Cathal Gurrin, Jennifer Foster
  The 45th European Conference on Information Retrieval, ECIR 2023

• Extending the Scope of Out-of-Domain: Examining QA models in multiple subdomains [PDF] [Slides][Code][Bibtex]
  Chenyang Lyu, Jennifer Foster and Yvette Graham
  The 60th Annual Meeting of the Association for Computational Linguistics, ACL 2022, Workshop on Insights from Negative Results in NLP

• Achieving Reliable Human Assessment of Open-Domain Dialogue Systems [PDF][Slides][Code][Bibtex]
  Tianbo Ji, Yvette Graham, Gareth J. F. Jones, Chenyang Lyu and Qun Liu
  The 60th Annual Meeting of the Association for Computational Linguistics, ACL 2022

• Knowledge and Pre-trained Language Models Inside and Out: a deep-dive into datasets and external knowledge [PDF][slides][Bibtex]
  Chenyang Lyu
  PhD Transfer Report

• Improving Unsupervised Question Answering via Summarization-Informed Question Generation [PDF][Slides][Bibtex]
  Chenyang Lyu, Lifeng Shang, Yvette Graham, Jennifer Foster, Xin Jiang and Qun Liu
  The 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021

• Improving Document-Level Sentiment Analysis with User and Product Context [PDF][Slides] [Code][Bibtex]
  Chenyang Lyu, Jennifer Foster and Yvette Graham
  The 28th International Conference on Computational Linguistics, COLING 2020

Teaching Experience

• CA-146/297, Introduction to Programming, 2020, Dublin City University, Teaching Assistant
• CA-271, Machine Learning, 2022, Dublin City University, Guest Lecturer
• CA-168, Digital World, 2022, Dublin City University, Guest Lecturer

Professional Activities

• PC Member
  • The 45th European Conference on Information Retrieval, ECIR 2023
  • The 61st Annual Meeting of the Association for Computational Linguistics, ACL 2023
  • The 32nd International Joint Conference on Artificial Intelligence, IJCAI 2023
  • The 3rd Workshop on Financial Technology on the Web in conjunction with The Web Conference 2023, FinWeb 2023
• Conference Reviewer
  • The 60th Annual Meeting of the Association for Computational Linguistics, ACL 2022
  • The 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022
  • The 29th International Conference on Computational Linguistics, COLING 2022
  • The 32nd International Joint Conference on Artificial Intelligence, IJCAI 2023
• Workshop Reviewer
  • EvoNLP: The First Workshop on Ever Evolving NLP, EMNLP 2022
  • FinNLP: The Fourth Workshop on Financial Technology and Natural Language Processing, EMNLP 2022
  • FinWeb: The 3rd Workshop on Financial Technology on the Web, WWW 2023
• Journal Reviewer
  • IEEE Transaction on Multimedia, IEEE
  • Social Network Analysis and Mining, Springer
  • Connection Science, Taylor & Francis
• Regular Reviewer
  • ACL Rolling Review