Audio sensory substitution for human-in-the-loop force feedback of upper limb prosthetics

Users of prosthetic hands, such as upper limb amputees, require tactile feedback and sensation to successfully achieve complex gripping and grasping tasks. Whilst there are many methods of electronically capturing this interaction (through electronic pressure sensor arrays) there are limited methods of interfacing this data with the human brain. So-matotopical approaches do exist, however these are typically very invasive, and rely on access to the nerve. As an alternative approach, this paper investigates sensory substitution, whereby the user's sense of sound is exploited as a feedback interface between the sensors and the brain. A study, consisting of 8 participants, and a randomized trial method, is used to determine the perceptual latency and range sensitivity across a series of modulation techniques including frequency, volume, and beating. Two of these were used simultaneously to determine if two degrees of freedom were able to be comprehended simultaneously. It is found that multi-channel audio feedback is suitable for low bandwidth feedback applications so long as it can deal with latency of at least 600 ms. The capability of this interface has been captured in terms of time delay, learning curve, task correlation, and accuracy.