Biography

My name is Xiyang Wu. I am a Ph.D. student in Electrical and Computer Engineering at University of Maryland, College Park and a member of GAMMA group. My research advisor is Prof. Dinesh Manocha. I hold a Master’s degree from Georgia Institute of Technology, where I was working with Prof. Matthew Gombolay. Before that, I earned my Bachelor’s in Engineering from Tianjin University, supervised by Prof. Xiaodong Zhang.

My research interest lies in using the reasoning ability of neural networks and Large Language Models in robot decision-making and navigation tasks. Specifically, incorporating Large Language Models with reinforcement learning approaches in multi-agent coordination, scene understanding, and opponents’ intention inference when navigating and collaborating under complex and heterogeneous environments.

Please check list of publications here.

Research Interest

Reinforcement Learning
Autonomous Driving
Large Language Model
Robotics

Education

Ph.D. in Electrical and Computer Engineering, University of Maryland, College Park, 2021 - 2026 (Expected)
M.S. in Electrical and Computer Engineering, Georgia Institute of Technology, 2019 - 2021
B.Eng. in Electrical Engineering (Honors Class), Tianjin University, 2015 - 2019

News

Feb 2024:	We release an technical report highlighting the critical issues of robustness and safety associated with integrating large language models (LLMs) and vision-language models (VLMs) into robotics applications. The project webpage is released here.
Oct 2023:	We release an early report and analysis on failure modes of GPT-4V and LLaVA-1.5. Stay tuned on the release of our dataset HallusionBench!
Oct 2023:	iPLAN was award as Best Paper Award by at IROS 2023 MRS Workshop!
Sep 2023:	LANCAR was submitted to ICRA 2024
Aug 2023:	iPLAN was accepted to CoRL 2023 with Oral Presentation (Accept Rate: 6.6%) !
Jul 2023:	One paper was accepted to Digital Signal Processing!
Aug 2021:	Started Ph.D. at University of Maryland, College Park.
Aug 2019:	Started M.S. at Georgia Institute of Technology.

Selected Publications

	On the Safety Concerns of Deploying LLMs/VLMs in Robotics: Highlighting the Risks and Vulnerabilities Xiyang Wu, Ruiqi Xian, Tianrui Guan, Jing Liang, Souradip Chakraborty, Fuxiao Liu, Brian Sadler, Dinesh Manocha, Amrit Singh Bedi arXiv:2402.10340 (Under Review) 2024. [paper] [webpage] [code]
	HallusionBench: You See What You Think? Or You Think What You See? An Image-Context Reasoning Benchmark Challenging for GPT-4V(ision), LLaVA-1.5, and Other Multi-modality Models Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, Dinesh Manocha, Tianyi Zhou arXiv:2310.14566 (arXiv) 2023. [paper] [webpage] [code]
	LANCAR: Leveraging Language for Context-Aware Robot Locomotion in Unstructured Environments Chak Lam Shek, Xiyang Wu, Dinesh Manocha, Pratap Tokekar, Amrit Singh Bedi (* indicates equal contributions) arXiv:2310.00481 (Submitted to ICRA 2024) 2023. [paper] [webpage]
	iPLAN: Intent-Aware Planning in Heterogeneous Traffic via Distributed Multi-Agent Reinforcement Learning Xiyang Wu, Rohan Chandra, Tianrui Guan, Amrit Singh Bedi, Dinesh Manocha 7th Annual Conference on Robot Learning (CoRL) 2023. oral (6.6%) Abridged in IROS 2023 Advances in Multi-Agent Learning - Coordination, Peception and Control Workshop. Best Paper and Presentation Award. [paper] [webpage] [code]