Hanyu Zhou 周寒宇

Postdoctoral Research Fellow

School of Computing (SoC),
National University of Singapore (NUS),
11 Computing Drive, Singapore, 117416

Email: hyzhouboy@gmail.com, hy.zhou@nus.edu.sg

Biography

I am currently a postdoctoral research fellow at National University of Singapore (NUS), working closely with Prof. Gim Hee Lee. I was a research intern at Shanghai AI Lab in 2024, working closely with Bin Zhao.

I got the Ph.D. degree at Huazhong University of Science and Technology (HUST) in 2024, advised by Prof. Luxin Yan. Before that, I got the B.Eng. degree at Central South University (CSU) in 2019.

I am now working on 2D/3D/4D vision perception & understanding in adverse scenes, if you have an excellent project for collaboration, please email me!

Interests

Scene Perception & Understanding, Dynamic Modeling, Multimodal Learning, Event Camera, Domain Adaptation.

News

2025.02, Our event-based dense and continuous optical flow paper is accepted to CVPR'25.
2025.02, Our event-based frame interpolation paper is accepted to CVPR'25, congrating to Haoyue.
2025.02, Our event-based nighttime HDR imaging paper is accepted to TPAMI'25, congrating to Haoyue.
2024.12, I join the Computer Vision and Robotic Perception (CVRP) LAB as a postdoc research fellow.
2024.10, I pass my Ph.D. defense, and become a doctor of engineering.
2024.09, Our adverse weather optical flow paper is accepted to TPAMI'24.
2024.06, We have won 1st place in the track 'Text Recognition through Atmospheric Turbulence' in the CVPR'24 7th UG2+ Challenge.
2024.06, We have won 1st place in the track 'Coded Target Restoration through Atmospheric Turbulence' in the CVPR'24 7th UG2+ Challenge.
2024.02, Our VisMoFlow scene flow based on multimodal fusion is accepted to CVPR'24.
2024.02, Our NER-Net nighttime event reconstruction is accepted to CVPR'24.
2024.01, Our JSTR event-based moving object detection method is accepted to ICRA'24.
2024.01, Our ABDA-Flow optical flow under nighttime scene paper is accepted to ICLR'24 (Spotlight).
2023.02, Our UCDA-Flow optical flow under foggy scene paper is accepted to CVPR'23.
2022.11, Our HMBA-FlowNet optical flow under adverse weather paper is accepted to AAAI'23.
2021.01, Our JRGR derain paper is accepted to CVPR'21.

Researches

I like to explore how unmanned systems (e.g., uav and robotic) can emulate human behavior to relieve humans from manual tasks. Similar to humans, unmanned systems perceive the world via vision sensors and then perform corresponding actions in response. I realize that unmanned systems work well in ideal environments, but fail in adverse conditions, such as adverse weather, low light, fast motion and so on. Focusing on these challenging scenes, I have devoted myself to a new research topic: Multimodal-IPUP, i.e., scene imaging, scene perception, scene understanding and scene planning.

1. Regarding the issue of unmanned system, I construct a RGB-Event-LiDAR-IMU multimodality hardware system for 2D/3D/4D adverse scenes, and build a large-scale multimodal dataset under various time and various weather. The research outputs have been published in arXiv. More will be open soon.

2. Regarding the issue of scene imaging, I propose multimodal fusion methods for 2D HDR imaging, 2D frame interpolation and 3D/4D scene reconstruction. The research outputs have been published in CVPR 2025, TPAMI 2025, CVPR 2024.

3. Regarding the issue of scene perception, I propose multimodal adaptation frameworks for 2D/3D motion flow and detection. The research outputs have been published in CVPR 2025, TPAMI 2024, ICLR 2024, ICRA 2024, CVPR 2023, AAAI 2023.

4. Regarding the issue of scene understanding, I propose large multimodal vision-language models for 2D/3D/4D reasoning, VQA, dense caption and grounding. The research outputs have been published in arXiv. More will be coming soon.

5. Regarding the issue of scene planning, I will work on several action planning of robots based on multimodal vision.

Publications (Google Scholar)

(#: Co-First Author; *: Corresponding Author)

	Bridge Frame and Event: Common Spatiotemporal Fusion for High-Dynamic Scene Optical Flow. Hanyu Zhou, Haonan Wang, Haoyue Liu, Yuxing Duan, Yi Chang, Luxin Yan. IEEE Conference on Computer Vision and Pattern Recognition ( CVPR* ), 2025.* [arXiv]
	TimeTracker: Event-based Continuous Point Tracking for Video Frame Interpolation with Non-linear Motion. Haoyue Liu, Jinghan Xu, Yi Chang, Hanyu Zhou, Haozhi Zhao, Lin Wang, Luxin Yan. IEEE Conference on Computer Vision and Pattern Recognition ( CVPR* ), 2025.*
	NER-Net+: Seeing motion at nighttime with an event camera. Haoyue Liu, Jinghan Xu, Shihan Peng, Yi Chang, Hanyu Zhou, Yuxing Duan, Lin Zhu, Yonghong Tian, Luxin Yan. IEEE Transactions on Pattern Analysis and Machine Intelligence ( TPAMI* ), 2025.* [PDF]
	Adverse Weather Optical Flow: Cumulative Homogeneous-Heterogeneous Adaptation. Hanyu Zhou, Yi Chang, Zhiwei Shi, Wending Yan, Gang Chen, Yonghong Tian, Luxin Yan. IEEE Transactions on Pattern Analysis and Machine Intelligence ( TPAMI* ), 2024.* [PDF] [arXiv]
	Exploring the Common Appearance-Boundary Adaptation for Nighttime Optical Flow. Hanyu Zhou, Yi Chang, Haoyue Liu, Wending Yan, Yuxing Duan, Zhiwei Shi, Luxin Yan. International Conference on Learning Representations ( ICLR* ), Spotlight, 2024.* [PDF] [arXiv]
	Bring Event into RGB and LiDAR: Hierarchical Visual-Motion Fusion for Scene Flow. Hanyu Zhou, Yi Chang, Zhiwei Shi, Luxin Yan. IEEE Conference on Computer Vision and Pattern Recognition ( CVPR* ), 2024.* [PDF] [arXiv]
	Seeing Motion During Nighttime with Event Camera. Haoyue Liu, Shihan Peng, Lin Zhu, Yi Chang, Hanyu Zhou, Luxin Yan. IEEE Conference on Computer Vision and Pattern Recognition ( CVPR* ), 2024.* [PDF] [arXiv] [Code]
	JSTR: Joint Spatio-Temporal Reasoning for Event-Based Moving Object Detection. Hanyu Zhou, Zhiwei Shi, Hao Dong, Shihan Peng, Yi Chang, Luxin Yan. IEEE International Conference on Robotics and Automation ( ICRA* ), 2024.* [PDF] [arXiv]
	Unsupervised Cumulative Domain Adaptation for Foggy Scene Optical Flow. Hanyu Zhou, Yi Chang, Wending Yan, Luxin Yan. IEEE Conference on Computer Vision and Pattern Recognition ( CVPR* ), 2023.* [PDF] [arXiv] [Code]
	Unsupervised Hierarchical Domain Adaptation for Adverse Weather Optical Flow. Hanyu Zhou, Yi Chang, Gang Chen, Luxin Yan. AAAI Conference on Artificial Intelligence ( AAAI* ), 2023.* [PDF] [arXiv] [Code]
	Closing the Loop: Joint Rain Generation and Removal via Disentangled Image Translation. Yuntong Ye, Yi Chang, Hanyu Zhou, Luxin Yan. IEEE Conference on Computer Vision and Pattern Recognition ( CVPR* ), 2021.* [PDF] [arXiv]

Preprint Papers

Nighttime Scene Optical Flow: Common Spatio-Temporal Motion Adaptation.

Hanyu Zhou, Haonan Wang, Haoyue Liu, Wending Yan, Yuxing Duan, Yi Chang, Luxin Yan.

IEEE Transactions on Pattern Analysis and Machine Intelligence ( TPAMI ), 2025. (Under review)

LLaFEA: Frame-Event Complementary Fusion for Fine-Grained Spatiotemporal Understanding in LMMs.

Hanyu Zhou, Gim Hee Lee.

arXiv, 2025.

[arXiv]

CoSEC: A Coaxial Stereo Event Camera Dataset for Autonomous Driving.

Shihan Peng#, Hanyu Zhou#, Hao Dong, Zhiwei Shi, Haoyue Liu, Yuxing Duan, Yi Chang, Luxin Yan.

arXiv, 2024.

[arXiv]

Awards

2024.06, 1st Place of "Atmospheric Turbulence Mitigation" in CVPR 2024 7th UG2+ Challenge.

2023.11, Third Prize of "Robust Depth Estimation" in International Algorithm Case Competition.

2019.09-2023.09, Ph.D. Scholarships.

2018.04, Meritorious Winner in MCM/ICM.

Academic Services

Journal Reviewers: TPAMI, IJCV, TIP, TCSVT, TMM, TIM, Sensors, MTAP, Mathematics.
Conference Reviewers: NeurIPS'25, CVPR'23-25, ICLR'25, ICML'25, ICCV'25, ACMM'25, ECCV'24, AAAI'23-25, ICRA'24-25.