FEU Institute of Technology

Educational Innovation and Technology Hub

Loading...

A Machine Vision-Based FSL Tutor with Static and Dynamic Gesture Recognition and Real-Time User Feedback Using MediaPipe Frameworks

TENCON 2025 - 2025 IEEE Region 10 Conference (TENCON), (2026), pp. 1215-1219

a Electronics Engineering, FEU Institute of Technology, Manila, Philippines

Abstract: Filipino Sign Language (FSL) is an invaluable tool for communication within the deaf and mute communities, yet there is a shortage of proficient special education teachers and accessible learning materials. Current research on FSL recognition is limited to basic detection, often invasive, and lacks comprehensive systems that provide feedback to users. Additionally, FSL incorporates distinctive static and dynamic gestures, including contractions, which set it apart from other sign languages. This study presents the development of a machine vision-based FSL tutor that leverages the MediaPipe framework-specifically, MediaPipe Hands for static gesture recognition and MediaPipe Holistic for full-body dynamic gesture tracking. LSTM networks were used to classify dynamic gestures based on sequential landmark data to capture temporal dependencies in sign execution. The system supports a desktop application platform enabling learners to engage in interactive modules with real-time feedback through visual prompts and audio cues. It utilizes 42 static hand feature landmarks and over 1,662 key points derived from hand, pose, and facial data to ensure accurate recognition and feedback. A total of 50 essential FSL gestures-aligned with the kindergarten curriculum-were modeled, covering alphabet knowledge, vocabulary development, self-introduction, and polite expressions. Performance evaluation using computer vision metrics demonstrated high recognition accuracy for both gesture types. In addition, the System Usability Scale (SUS) and statistical comparisons with traditional instruction methods confirmed the platform's effectiveness and user acceptability. The results validate the system as a comprehensive and accessible solution for FSL education, particularly suited for early learners and self-guided instruction.

Recommended Citation

Suamina, H. J. Y., Cruz, J. J. P., Gonzales, J. P. T., Pichay, K. M. T., & Loresco, P. J. (2026). A Machine Vision-Based FSL Tutor with Static and Dynamic Gesture Recognition and Real-Time User Feedback Using MediaPipe Frameworks. TENCON 2025 - 2025 IEEE Region 10 Conference (TENCON), 1215-1219. https://doi.org/10.1109/TENCON66050.2025.11375053
H. J. Y. Suamina, J. J. P. Cruz, J. P. T. Gonzales, K. M. T. Pichay, and P. J. Loresco, "A Machine Vision-Based FSL Tutor with Static and Dynamic Gesture Recognition and Real-Time User Feedback Using MediaPipe Frameworks," TENCON 2025 - 2025 IEEE Region 10 Conference (TENCON), pp. 1215-1219, 2026. doi: 10.1109/TENCON66050.2025.11375053.
Suamina, Hyan Jan Y., et al.. "A Machine Vision-Based FSL Tutor with Static and Dynamic Gesture Recognition and Real-Time User Feedback Using MediaPipe Frameworks." TENCON 2025 - 2025 IEEE Region 10 Conference (TENCON), 2026, pp. 1215-1219. https://doi.org/10.1109/TENCON66050.2025.11375053.
Suamina, H. J. Y., Cruz, J. J. P., Gonzales, J. P. T., Pichay, K. M. T., & Loresco, P. J.. 2026. "A Machine Vision-Based FSL Tutor with Static and Dynamic Gesture Recognition and Real-Time User Feedback Using MediaPipe Frameworks." TENCON 2025 - 2025 IEEE Region 10 Conference (TENCON): 1215-1219. https://doi.org/10.1109/TENCON66050.2025.11375053.

A Time Capsule Where Research Rests, Legends Linger, and PDFs Live Forever

Repository is the home for every research paper and capstone project created across our institution. It’s where knowledge kicks back, ideas live on, and your hard work finds the spotlight it deserves.

© 2026 Educational Innovation and Technology Hub. All Rights Reserved.