 |
Lanyun Zhu 祝澜耘 Postdoctoral Fellow Email: [email protected] CV / Google Scholar |
About Me
I am currently a postdoctoral researcher. Prior to this, I obtained my Ph.D. from the Singapore University of Technology and Design (SUTD), under the supervision of Professor Jun Liu and Professor Soh De Wen. I received my bachelor’s degree from Beihang University in June 2020. I also spent some wonderful times in Megvii and SenseTime. Currently, I work closely with NVIDIA, Alibaba (Professor Jieping Ye), and Tencent. My research directions are multimodal learning and computer vision. Currently, most of my works are focused multimodal large language models (MLLMs) and image segmentation. My research goal is to build efficient, trustworthy, and fine-grained multimodal systems that can process or integrate information from diverse modalities—such as text, images, videos, and data from other sensors—to effectively address a wide range of real-world industrial and scientific challenges. I believe that a practical multimodal system should be cheap—with lower training and deployment costs; powerful—with more comprehensive and fine-grained capabilities; and reliable—with stronger robustness and minimal instability. Currently, I am exploring new techniques to achieve these goals within MLLMs and to advance their applications in real-world industrial scenarios, such as online content safety, as well as in scientific domains such as healthcare and agriculture. I am always open to research collaborations. Please feel free to drop me an email if you are interested.Resume
[English Resume] [中文简历]Selected Publications
* refers to equal contribution; # refers to corresponding author Conference Papers[ICML2025] Lanyun Zhu, Deyi Ji, Tianrun Chen, Haiyang Wu, De Wen Soh, Jun Liu, CPCF: A Cross-Prompt Contrastive Framework for Referring Multimodal Large Language Models, International Conference on Machine Learning (ICML) 2025
[ICML2025] Qianxiong Xu, Lanyun Zhu, Xuanyi Liu, Guosheng Lin, Cheng Long, Ziyue Li, Rui Zhao, Unlocking the Power of SAM 2 for Few-Shot Segmentation, International Conference on Machine Learning (ICML) 2025
[CVPR2025] Lanyun Zhu, Tianrun Chen, Qianxiong Xu, Xuanyi Liu, Deyi Ji, Haiyang Wu, De Wen Soh, Jun Liu, POPEN: Preference-Based Optimization and Ensemble for LVLM-Based Reasoning Segmentation, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2025
[ACL2025 Industry Track] Deyi Ji, Yuekui Yang, Haiyang Wu, Shaoping Ma, Tianrun Chen, Lanyun Zhu#, RAVEN: Robust Advertisement Video Violation Temporal Grounding via Reinforcement Reasoning, Annual Meeting of the Association for Computational Linguistics (ACL) 2025 (oral)
[NeurIPS2024] Qianxiong Xu, Xuanyi Liu, Lanyun Zhu, Guosheng Lin, Cheng Long, Ziyue Li, Rui Zhao, Hybrid Mamba for Few-Shot Segmentation, Annual Conference on Neural Information Processing Systems (NeurIPS) 2024
[ICML2024] Deyi Ji, Feng Zhao, Lanyun Zhu, Wenwei Jin, Hongtao Lu, Jieping Ye, Discrete Latent Perspective Learning for Segmentation and Detection, International Conference on Machine Learning (ICML) 2024 (spotlight)
[CVPR2024] Lanyun Zhu, Tianrun Chen, Deyi Ji, Jieping Ye, Jun Liu, LLaFS: When Large Language Models Meet Few-Shot Segmentation, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2024
[CVPR2024] Lanyun Zhu, Tianrun Chen, Jianxiong Yin, Simon See, Jun Liu, Addressing Background Context Bias in Few-Shot Segmentation through Iterative Modulation, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2024
[ICCV2023] Lanyun Zhu, Tianrun Chen, Jianxiong Yin, Simon See, Jun Liu, Learning Gabor Texture Features for Fine-Grained Recognition, International Conference on Computer Vision (ICCV) 2023
[CVPR2023] Lanyun Zhu*, Tianrun Chen*, Jianxiong Yin, Simon See, Jun Liu, Continual Semantic Segmentation with Automatic Memory Sample Selection, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023
[CVPR2021] Lanyun Zhu*, Deyi Ji*, Shiping Zhu, Weihao Gan, Wei Wu, Junjie Yan, Learning Statistical Texture for Semantic Segmentation, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2021
[ICCV2023 Workshop] Tianrun Chen*, Lanyun Zhu*, Chaotao Ding, Runlong Cao, Shangzhan Zhang, Yan Wang, Zejian Li, Lingyun Sun, Papa Mao, Ying Zang, SAM-Adapter: Adapting Segment Anything in Underperformed Scenes, ICCV2023 1st Workshop on Visual Continual Learning
[3DV2022] Xiao Fu, Shangzhan Zhang, Tianrun Chen, Yichong Lu, Lanyun Zhu, Xiaowei Zhou, Andreas Geiger, Yiyi Liao., Panoptic NeRF: 3D-to-2D Label Transfer for Panoptic Urban Scene Segmentation, International Conference on 3D Vision (3DV) 2021
[TPAMI] Lanyun Zhu, Tianrun Chen, Deyi Ji, Peng Xu, Jieping Ye, Jun Liu, LLaFS++: Few-Shot Image Segmentation With Large Language Models, IEEE Transactions on Pattern Analysis and Machine Intelligence
[TIP] Lanyun Zhu, Tianrun Chen, Deyi Ji, Jieping Ye, Jun Liu, Not Every Patch is Needed: Towards a More Efficient and Effective Backbone for Video-based Person Re-identification, IEEE Transactions on Image Processing
[TII] Tianrun Chen, Chunan Yu, Yuanqi Hu, Jing Li, Tao Xu, Runlong Cao, Lanyun Zhu, Ying Zang, Yong Zhang, Zejian Li, Linyun Sun, Img2CAD: Conditioned 3D CAD Model Generation from Single Image with Structured Visual Geometry, IEEE Transactions on Industrial Informatics
[TMM] Tianrun Chen, Chaotao Ding, Lanyun Zhu, Ying Zang, Yiyi Liao, Zejian Li, and Lingyun Su, Reality3DSketch: Rapid 3D Modeling of Objects from Single Free-hand Sketches, IEEE Transactions on Multimedia
[TMI] Yan Wang, Jian Cheng, Yixin Chen, Shuai Shao, Lanyun Zhu, Zhenzhou Wu, Tao Liu, Haogang Zhu, FVP: Fourier Visual Prompting for Source-Free Unsupervised Domain Adaptation of Medical Image Segmentation, IEEE Transactions on Medical Imaging
- Tianrun Chen, Chunan Yu, Jing Li, Jiangqi Zhang, Lanyun Zhu, Deyi Ji, Yong Zhang, Ying Zang, Zejian Li, Lingyun Sun, Reasoning3D - Grounding and Reasoning in 3D: Fine-Grained Zero-Shot Open-Vocabulary 3D Reasoning Part Segmentation via Large Vision-Language Models, Arxiv Preprint
Note: * indicates equal contribution.
Full list of publications in Google Scholar.
Experiences
- Research Intern | CCVL Lab, Johns Hopkins University| Apr 2020 - Sep 2021 Supervisor: Prof.Alan Yuille
- Research Intern | Sensetime, Beijing, China | June 2020 - July 2021 Mentor: Dr. Deyi Ji | Mr. Wei Wu
- Research Intern | Megvii, Beijing, China | Sep. 2019 - May 2020 Mentor: Dr. Zhikang Liu | Leader: Dr. Chi Zhang