Research

Human-centered sensing in the real world.

We combine sensing, systems, and AI to build mobile and wearable technologies that are useful beyond the lab.

Research overview connecting data acquisition, intelligence, well-being, and feedback through the human-digital interface

01 / 08ArmTroi · Arm skeleton tracking

Human Behavior Sensing

To enable ubiquitous, practical, and user-friendly perception models, this research explores the full potential of existing wearable sensors (devices) to perceive and understand human behaviors [MobiSys’19, IMWUT’21, TMC’24, SenSys’26]. Rather than relying on excessive sensor deployment to cover all to-be-sensed areas of the human body, this line of work investigates a central question: Is it possible to perceive and understand fine-grained human behaviors using as few wearable sensors (devices) as possible?

ACM MobiSys 2019Real-time Arm Skeleton Tracking and Gesture Inference Tolerant to Missing Wearable SensorsOpen paper ↗

ACM IMWUT 2021WR-Hand: Wearable Armband Can Track User's HandOpen paper ↗

IEEE TMC 2024Finger Tracking Using Wrist-Worn EMG SensorsOpen paper ↗

ACM SenSys 2026NeuroPath: Practically Adopting Motor Imagery Decoding through EEG SignalsOpen paper ↗

Human Wellness Monitoring

This research gathers insights into human health [SmartWear’23, EarComp’24, BodySys’24, Pervasive and Mobile Computing Journal’24, IMWUT’24, NeurIPS’24, PerCom’25 Best Paper Award, PerCom’25, CHI’25, IMWUT’25, IMWUT’25, EMBC’25, ACM IASA’25, ACM EarComp’25, Nature Communications’26, ACM HotMobile’26, JAB’26, ACM SenSys’26, ACM SenSys’26, ACM IMWUT’26]. It primarily leverages earphones as versatile tools for tracking human wellness in diverse, real-world settings—enabling robust physiological and fitness monitoring and promoting healthier lifestyles. The work theoretically establishes and quantifies the relationships between physiological activities and biosignals collected near the ears, including acoustic and motion signals, and conducts in-depth analyses of interactions between physiological systems to uncover critical insights into their complex dynamics. These insights support the decoupling of intricate biosignals and the development of robust methods for accurate, non-invasive wellness monitoring.

ACM SmartWear 2023Yawning Detection using Earphone Inertial Measurement UnitsOpen paper ↗

ACM EarComp 2024BrushBuds: Toothbrushing Tracking Using Earphone IMUsOpen paper ↗

ACM BodySys 2024Detecting Foot Strikes during Running with EarbudsOpen paper ↗

Pervasive and Mobile Computing 2024An Evaluation of Heart Rate Monitoring with In-ear Microphones under MotionOpen paper ↗

ACM IMWUT 2024BreathPro: Monitoring Breathing Mode during Running with EarablesOpen paper ↗

NeurIPS 2024Towards Open Respiratory Acoustic Foundation Models: Pretraining and BenchmarkingOpen paper ↗

IEEE PerCom 2025 · Best PaperRespEar: Earable-Based Robust Respiratory Rate MonitoringOpen paper ↗

IEEE PerCom 2025WalkEar: Holistic Gait Monitoring using EarablesOpen paper ↗

ACM CHI 2025SmarTeeth: Augmenting Manual Toothbrushing with In-ear MicrophonesOpen paper ↗

ACM IMWUT 2025EarMeter: Continuous Respiration Volume Monitoring with EarablesOpen paper ↗

ACM IMWUT 2025HearForce: Force Estimation for Manual Toothbrushing with EarablesOpen paper ↗

IEEE EMBC 2025Deep-Learning Based Segmentation of In-Ear Cardiac SoundsOpen paper ↗

ACM IASA 2025IMUSteth: On-Body Stethoscope Localization with Inertial Sensing for Home Self-ScreeningOpen paper ↗

ACM EarComp 2025Earable-based Continuous Blood Pressure Monitoring via a Single-Point Flexible SensorOpen paper ↗

Nature Communications 2026Measuring Cardiac Stroke Volume Through In-ear Audio SensingOpen paper ↗

ACM HotMobile 2026EarCalo: Earable-Based Energy Expenditure Estimation While RunningOpen paper ↗

Journal of Applied Biomechanics 2026Ear-worn inertial sensors can predict gait metrics and reconstruct vertical ground reaction force curves during runningOpen paper ↗

ACM SenSys 2026EarSleeve: Transforming Everyday Earphones into a 12-Lead ECG Sensing PlatformOpen paper ↗

ACM SenSys 2026NutriEar: Robust Nutrition-Aware Food Classification from In-Ear Acoustic SignalsOpen paper ↗

ACM IMWUT 2026ImpactEar: Cross Activity Ground Reaction Force Estimation using Earable IMUsOpen paper ↗

Interaction with Physical World

This research also seeks to enhance human interactions with the physical world. First, by leveraging human-ambience interactions, it enables the capture and preservation of information about physical objects within digital spaces [TOSN’19, MobiSys’26]. Second, it develops and implements natural interaction technologies—such as gaze tracking via smartphones [SenSys’22, TMC’24] and sign language translation using smartwatches [INFOCOM’22, TMC’24]—to create more seamless and accessible ways for humans to engage with their surroundings. In addition, the work reveals and demonstrates critical privacy leakage issues that arise during interactions with the physical world, serving as a timely warning to the public [INFOCOM’18, TMC’19, ICDCS’20, TMC’21]. To address these concerns, a secure authentication system has been proposed to enhance user privacy and strengthen security [EarComp’23 Best Paper, ACM IASA’25].