Back to Search
Start Over
MagicHand: Interact with IoT Devices in Augmented Reality Environment
- Source :
- VR
- Publication Year :
- 2019
- Publisher :
- IEEE, 2019.
-
Abstract
- We present an Augmented Reality (AR) visualization and interaction tool for users to control Internet of Things (IoT) devices with hand gestures. Today, smart IoT devices are becoming increasingly ubiquitous with diverse forms and functions, yet most user controls over them are still limited to mobile devices and web interfaces. Recently, AR has been developed rapidly, and provided immersive solutions to enhance user experience of applications in many fields. Its capability to create immersive interactions allows AR to improve the way smart devices are controlled via more direct visual feedback. In this paper, we create a functional prototype of one such system, enabling seamless interactions with sound and lighting systems through the use of augmented hand-controlled interaction panels. To interpret users' intentions, we implement a standard 2D convolution neural network (CNN) for real-time hand gesture recognition and deploy it within our system. Our prototype is also equipped with a simple but effective object detector which can identify target devices within a proper range by analyzing geometric features. We evaluate the performance of our system qualitatively and quantitatively and demonstrate it on two smart devices.
- Subjects :
- business.industry
Computer science
Deep learning
Optical head-mounted display
020207 software engineering
02 engineering and technology
Visualization
User experience design
Human–computer interaction
Gesture recognition
0202 electrical engineering, electronic engineering, information engineering
Immersion (virtual reality)
020201 artificial intelligence & image processing
Augmented reality
Artificial intelligence
business
Internet of Things
Mobile device
Gesture
Subjects
Details
- Database :
- OpenAIRE
- Journal :
- 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)
- Accession number :
- edsair.doi...........4996ebeb5c6f68d2fff66ec3254574c0
- Full Text :
- https://doi.org/10.1109/vr.2019.8798053