 |
Lanyun Zhu 祝澜耘 Research Fellow Rapid-Rich Object Search (ROSE) Lab Nanyang Technological University Email: [email protected] CV / Google Scholar |
About Me
I am currently a Research Fellow in Rapid-Rich Object Search (ROSE) Lab, Nanyang Technological University, working with Professor Bihan Wen. Prior to this, I was a postdoctoral fellow at City University of Hong Kong, working with Professor Shiqi Wang. I received my bachelor’s degree from Beihang University in June 2020 and my Ph.D. from the Singapore University of Technology and Design (SUTD) in 2025, under the supervision of Professor Jun Liu. I also spent some wonderful times in Megvii and SenseTime. Currently, I work closely with NVIDIA, Alibaba (Professor Jieping Ye), and Tencent. My research directions are multimodal learning and computer vision. Currently, most of my works are focused multimodal large language models (MLLMs) and image segmentation. My research goal is to build efficient, trustworthy, and fine-grained multimodal systems that can process or integrate information from diverse modalities—such as text, images, videos, and data from other sensors—to effectively address a wide range of real-world industrial and scientific challenges. I believe that a practical multimodal system should be cheap—with lower training and deployment costs; powerful—with more comprehensive and fine-grained capabilities; and reliable—with stronger robustness and minimal instability. Currently, I am exploring new techniques to achieve these goals within MLLMs and to advance their applications in real-world industrial scenarios, such as online content safety, as well as in scientific domains such as healthcare and agriculture. I am always open to research collaborations. Please feel free to drop me an email if you are interested.Resume
[English Resume]Selected Publications
* refers to equal contribution; # refers to corresponding author Conference Papers[NeurIPS2025] Lanyun Zhu, Deyi Ji, Tianrun Chen, Haiyang Wu, Shiqi Wang, Retrv-R1: A Reasoning-Driven MLLM Framework for Universal and Efficient Multimodal Retrieval, Annual Conference on Neural Information Processing Systems (NeurIPS) 2025
[ICML2025] Lanyun Zhu, Deyi Ji, Tianrun Chen, Haiyang Wu, De Wen Soh, Jun Liu, CPCF: A Cross-Prompt Contrastive Framework for Referring Multimodal Large Language Models, International Conference on Machine Learning (ICML) 2025
[ICML2025] Qianxiong Xu, Lanyun Zhu, Xuanyi Liu, Guosheng Lin, Cheng Long, Ziyue Li, Rui Zhao, Unlocking the Power of SAM 2 for Few-Shot Segmentation, International Conference on Machine Learning (ICML) 2025
[CVPR2025] Lanyun Zhu, Tianrun Chen, Qianxiong Xu, Xuanyi Liu, Deyi Ji, Haiyang Wu, De Wen Soh, Jun Liu, POPEN: Preference-Based Optimization and Ensemble for LVLM-Based Reasoning Segmentation, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2025
[NeurIPS2024] Qianxiong Xu, Xuanyi Liu, Lanyun Zhu, Guosheng Lin, Cheng Long, Ziyue Li, Rui Zhao, Hybrid Mamba for Few-Shot Segmentation, Annual Conference on Neural Information Processing Systems (NeurIPS) 2024
[ICML2024] Deyi Ji, Feng Zhao, Lanyun Zhu, Wenwei Jin, Hongtao Lu, Jieping Ye, Discrete Latent Perspective Learning for Segmentation and Detection, International Conference on Machine Learning (ICML) 2024 (spotlight)
[CVPR2024] Lanyun Zhu, Tianrun Chen, Deyi Ji, Jieping Ye, Jun Liu, LLaFS: When Large Language Models Meet Few-Shot Segmentation, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2024
[CVPR2024] Lanyun Zhu, Tianrun Chen, Jianxiong Yin, Simon See, Jun Liu, Addressing Background Context Bias in Few-Shot Segmentation through Iterative Modulation, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2024
[ICCV2023] Lanyun Zhu, Tianrun Chen, Jianxiong Yin, Simon See, Jun Liu, Learning Gabor Texture Features for Fine-Grained Recognition, International Conference on Computer Vision (ICCV) 2023
[CVPR2023] Lanyun Zhu*, Tianrun Chen*, Jianxiong Yin, Simon See, Jun Liu, Continual Semantic Segmentation with Automatic Memory Sample Selection, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023
[CVPR2021] Lanyun Zhu*, Deyi Ji*, Shiping Zhu, Weihao Gan, Wei Wu, Junjie Yan, Learning Statistical Texture for Semantic Segmentation, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2021
[ICCV2023 Workshop] Tianrun Chen*, Lanyun Zhu*, Chaotao Ding, Runlong Cao, Shangzhan Zhang, Yan Wang, Zejian Li, Lingyun Sun, Papa Mao, Ying Zang, SAM-Adapter: Adapting Segment Anything in Underperformed Scenes, ICCV2023 1st Workshop on Visual Continual Learning
[3DV2022] Xiao Fu, Shangzhan Zhang, Tianrun Chen, Yichong Lu, Lanyun Zhu, Xiaowei Zhou, Andreas Geiger, Yiyi Liao., Panoptic NeRF: 3D-to-2D Label Transfer for Panoptic Urban Scene Segmentation, International Conference on 3D Vision (3DV) 2021
[TPAMI] Lanyun Zhu, Tianrun Chen, Deyi Ji, Peng Xu, Jieping Ye, Jun Liu, LLaFS++: Few-Shot Image Segmentation With Large Language Models, IEEE Transactions on Pattern Analysis and Machine Intelligence
[TPAMI] Lanyun Zhu, Tianrun Chen, Jianxiong Yin, Simon See, De Wen Soh, Jun Liu, Replay Master: Automatic Sample Selection and Effective Memory Utilization for Continual Semantic Segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence
[TIP] Lanyun Zhu, Tianrun Chen, Deyi Ji, Jieping Ye, Jun Liu, Not Every Patch is Needed: Towards a More Efficient and Effective Backbone for Video-based Person Re-identification, IEEE Transactions on Image Processing
[TVCG] Ying Zang, Yuanqi Hu, Xinyu Chen, Yuxia Xu, Suhui Wang, Chunan Yu, Lanyun Zhu, Deyi Ji, Xin Xu, Tianrun Chen, From Air to Wear: Personalized 3D Digital Fashion with AR/VR Immersive 3D Sketching, IEEE Transactions on Visualization and Computer Graphics
[TMM] Ying Zang, Runlong Cao, Jianqi Zhang, Yidong Han, Ziyu Cao, Wenjun Hu, Didi Zhu, Zejian Li, Lanyun Zhu, Deyi Ji, Tianrun Chen, Let human sketches help: Empowering challenging image segmentation task with freehand sketches, IEEE Transactions on Multimedia
[TMM] Tianrun Chen, Chaotao Ding, Lanyun Zhu, Ying Zang, Yiyi Liao, Zejian Li, and Lingyun Su, Reality3DSketch: Rapid 3D Modeling of Objects from Single Free-hand Sketches, IEEE Transactions on Multimedia
[TII] Tianrun Chen, Chunan Yu, Yuanqi Hu, Jing Li, Tao Xu, Runlong Cao, Lanyun Zhu, Ying Zang, Yong Zhang, Zejian Li, Linyun Sun, Img2CAD: Conditioned 3D CAD Model Generation from Single Image with Structured Visual Geometry, IEEE Transactions on Industrial Informatics
[TMI] Yan Wang, Jian Cheng, Yixin Chen, Shuai Shao, Lanyun Zhu, Zhenzhou Wu, Tao Liu, Haogang Zhu, FVP: Fourier Visual Prompting for Source-Free Unsupervised Domain Adaptation of Medical Image Segmentation, IEEE Transactions on Medical Imaging
[ACL2025 Industry Track] Deyi Ji, Yuekui Yang, Haiyang Wu, Shaoping Ma, Tianrun Chen, Lanyun Zhu#, RAVEN: Robust Advertisement Video Violation Temporal Grounding via Reinforcement Reasoning, Annual Meeting of the Association for Computational Linguistics (ACL) 2025 (working with Tencent)
[EMNLP2025 Industry Track] Deyi Ji, Yuekui Yang, Haiyang Wu, Shaogang Tang, Peng Shu, Xudong Chen, Shaoping Ma, Tianrun Chen, Lanyun Zhu#, RAVEN++: Pinpointing Fine-Grained Violations in Advertisement Videos with Active Reinforcement Reasoning, Conference on Empirical Methods in Natural Language Processing (EMNLP) 2025 (oral, working with Tencent)
Lanyun Zhu, Towards Data Efficient and Continual Semantic Segmentation. [Thesis]
Full list of publications in Google Scholar.
Experiences
- Research Intern | CCVL Lab, Johns Hopkins University| Apr 2020 - Sep 2021 Supervisor: Prof.Alan Yuille
- Research Intern | Sensetime, Beijing, China | June 2020 - July 2021 Mentor: Dr. Deyi Ji | Mr. Wei Wu
- Research Intern | Megvii, Beijing, China | Sep. 2019 - May 2020 Mentor: Dr. Zhikang Liu | Leader: Dr. Chi Zhang