1. FingerSpeller: Camera-Free Text Entry Using Smart Rings for American Sign Language Fingerspelling Recognition.
- Author
-
Martin, David, Leng, Zikang, Gemicioglu, Tan, Womack, Jon, Heath, Jocelyn, Neubauer, William C, Kwon, Hyeokhyen, Ploetz, Thomas, and Starner, Thad
- Subjects
AMERICAN Sign Language ,HIDDEN Markov models ,WORD recognition ,WEARABLE technology ,CAMERAS ,ENCYCLOPEDIAS & dictionaries ,HANDWRITING recognition (Computer science) ,MOTION detectors - Abstract
Camera-based text entry using American Sign Language (ASL) fingerspelling has become more feasible due to recent advancements in recognition technology. However, there are numerous situations where camera-based text entry may not be ideal or acceptable. To address this, we present FingerSpeller, a solution that enables camera-free text entry using smart rings. FingerSpeller utilizes accelerometers embedded in five smart rings from TapStrap, a commercially available wearable keyboard, to track finger motion and recognize fingerspelling. A Hidden Markov Model (HMM) based backend with continuous Gaussian modeling facilitates accurate recognition as evaluated in a real-world deployment. In offline isolated word recognition experiments conducted on a 1,164-word dictionary, FingerSpeller achieves an average character accuracy of 91% and word accuracy of 87% across three participants. Furthermore, we demonstrate that the system can be downsized to only two rings while maintaining an accuracy level of approximately 90% compared to the original configuration. This reduction in form factor enhances user comfort and significantly improves the overall usability of the system. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF