My research focuses on computer vision with Vision Language Models, addressing a central question: "how to adapt pretrained foundation models to better serve specific downstream tasks, with none or limited labeled data?". This question drives my efforts to identify and analyze the limitations of foundation models, examining their pretraining data to understand the origins of these limitations. My prior work includes developing advanced prompting and retrieval-augmented learning techniques for zero-shot recognition, as well as stage-wise retrieval-augmented finetuning methods for few-shot recognition. I am also exploring the application of foundation models for video understanding, with specific interests in detecting unusual activities, such as failure actions or behaviors indicative of autism in children. In addition, I have broad interest in cyber-physical systems, including developing efficient machine learning systems on resource-constrained edge devices. Applications include efficient vision system for precision residential irrigation and voice assistance system for emergency medical services. Beyond these areas, my research extends to machine learning for healthcare and geoscience. This includes developing safe reinforcement learning algorithms for personalized medicine; enhancing the fairness of cardiovascular disease (CVD) risk prediction models for underrepresented populations; advancing geoscientific methods for characterizing subsurface fracture distribution and integrating 4D seismic data for more accurate reservoir model calibration. | ||
![]() |
Few-Shot Recognition via Stage-Wise Retrieval-Augmented Finetuning Tian Liu, Huixin Zhang, Shubham Parashar, Shu Kong[CVPR 2025] We explore retrieval-augmented learning for solving few-shot recognition, and propose stage-wise retrieval-Augmented fineTuning (SWAT) method to mitigate the imbalanced distribution and domain gap issues, outperforming SOTA methods by >6% accuracy. |
![]() |
UAL-Bench: The First Comprehensive Unusual Activity Localization Benchmark Hasnat Md Abdullah, Tian Liu, Kangda Wei, Shu Kong, Ruihong Huang[WACV 2025] We explore multimodal foundation models for Unusual Activities Localization (UAL) in video data. We compiled the first UAL benchmark dataset and proposed VLM-LLM framework to synergize multimodal foundational models for better video understanding. |
![]() |
ERIC: Estimating Rainfall with Commodity Doorbell Camera for Precision Residential Irrigation Tian Liu, Liuyi Jin, Radu Stoleru, Amran Haroon, Charles Swanson, Kexin Feng[ACM BuildSys 2024] Best paper award / TPC praise paper / arxiv / video presentation / slides / code We develop efficient vision system to estimate hyperlocal rainfall from doorbell camera for precision residential irrigation, saving > 9,000 gallons of water per month. |
![]() |
The Neglected Tails in Vision-Language Models Shubham Parashar*, Zhiqiu Lin*, Tian Liu* (*co-first authors), Xiangjue Dong, Yanan Li, Deva Ramanan, James Caverlee, Shu Kong[CVPR 2024, ICML 2024 DMLR Workshop Oral] paper / DMLR / arxiv / project / poster / code We expose the long-tailed concept distributions in VLMs' pretraining data and reveal failues of SOTA multimodal systems (e.g. GPT-4V, DALL-E 3). We propose retrieval-augmented learning, achieving SOTA zero-shot recognition performance. |
![]() |
EMSAssist: An End-to-End Mobile Voice Assistant at the Edge for Emergency Medical Services Liuyi Jin, Tian Liu, Amran Haroon, Radu Stoleru, Michael Middleton, Ziwei Zhu, Theodora Chaspari[ACM MobiSys 2023] paper / workshop paper / presentation / slides / app demo / code We build the first end-to-end mobile voice assistant system to assist Emergency Medical Technicians in selecting proper protocols for critical medical intervention. |
![]() |
Safe Reinforcement Learning with Contextual Information: Theory and Applications Junyu Cao, Esmaeil Keyvanshokooh, Tian Liu[Under review] We develop a safe reinforcement learning algorithm for personalized medical prescription considering patient's contextual information (e.g. age, gender, race etc.), achieveing sub-linear regret with zero safety violation. |
![]() |
Tian Liu, Ruxin Zhang [Journal of Computers and Geotechnics 2024] We develop Variational Autoencoder (VAE) model for fracture parameterization and distribution prediction using reservoir production data. |
![]() |
Tian Liu, Hongquan Chen, Gill Hetz, Akhil Datta-Gupta [Journal of Petroleum Science and Engineering 2020] 1st Place of TAMU Student Paper Contest, 3rd Place of International Championship JPSE journal paper / ATCE paper We develop onset-time approach for efficient and robust integration of 4D seismic data for reservoir model calibration, achieveing 2x error reduction and 6x speedup compared to traditional amplitude-matching methods. |
|