Fuxiao Liu


Fuxiao Liu (刘赋骁)
Hi! I'm a 3rd-year CS Ph.D at University of Maryland, College Park, working with Abhinav Shrivastava and Yaser Yacoob.
I have broad interests in vision and language tasks, including image/video captioning, multimodal semantic alignment, fact-checking, document understanding. My recent focus is on building customizable large models that follow humans' intent.
Google Scholar/ LinkedIn/Github/Twitter/Instagram
🔥🔥🔥 I am on the job market and looking for a US-based Research/Applied Scientist position starting from Summer 2025. Feel free to contact me if you have any openings!
Experience
First-Author Publications
  1. New! EAGLE: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders
    Fuxiao Liu*, Min Shi*, Shihao Wang, Shijia Liao, Subhashree Radhakrishnan, De-An Huang, Hongxu Yin, Karan Sapra, Yaser Yacoob, Humphrey Shi, Bryan Catanzaro, Andrew Tao, Jan Kautz, Zhiding Yu*, Guilin Liu*
    Arxiv 2024 [paper] [code] [bibtex]

  2. New! Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning
    Fuxiao Liu*, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, Lijuan Wang

  3. New! HallusionBench: You See What You Think? Or You Think What You See? An Image-Context Reasoning Benchmark Challenging for GPT-4V(ision), LLaVA-1.5, and Other Multi-modality Models
    Fuxiao Liu*, Tianrui Guan*, Xiyang Wu, Ruiqi Xian, Xijun Wang, Zongxia Li, Lichang Chen, Yaser Yacoob, Dinesh Manocha, Tianyi Zhou

  4. New! MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuning
    Fuxiao Liu*, Xiaoyang Wang, Wenlin Yao, Jianshu Chen, Kaiqiang Song, Sangwoo Cho, Yaser Yacoob, Dong Yu
    NAACL 2024 [paper] [code] [bibtex]

  5. DocumentCLIP: Linking Figures and Main Body Text in Reflowed Documents
    Fuxiao Liu*, Hao Tan, Chris Tensmeyer
    ICPRAI 2023 [paper] [code] [bibtex]

  6. COVID-VTS: Fact Extraction and Verification on Short Video Platforms
    Fuxiao Liu*, Yaser Yacoob, Abhinav Shrivastava
    EACL 2023 (~Oral presentation) [paper] [code] [bibtex]

  7. Visual News: Benchmark and Challenges in News Image Captioning
    Fuxiao Liu*, Yinghan Wang, Tianlu Wang, Vicente Ordonez
    EMNLP 2021 (~Oral presentation) [paper] [code] [bibtex]

Other Publications
Service
    Conference Reviewer: AISTATS,CVPR,NAACL,ACL,IJCAI,ACMMM
    Journal Reviewer: JMIR
More About Myself
    I'm crazy about basketball since I was a little boy. I love it for its ultimate technical and mentality requirements. No one in the world can become a master without great talent and extensive training. My favorite basketball player is Kobe Bryant, who is noted for his rapid playing style, strong will, and his ambivalent relationship with the sport. I am always immersed in his phenomenal performance in the game.