Object surface roughness/texture recognition using machine vision enables for human-machine haptic interaction

Tactile feedback can effectively improve the controllability of an interactive intelligent robot, and enable users to distinguish the sizes/shapes/compliance of grasped objects. However, it is difficult to recognize object roughness/textures through tactile feedback due to the surface features cannot be acquired with equipped sensors. The purpose of this study is to investigate whether different object roughness/textures can be classified using machine vision and utilized for human-machine haptic interaction. Based on practical application, two classes of specialized datasets, the roughness dataset consisted of different spacing/shapes/height distributions of the surface bulges and the texture dataset included eight types of representative surface textures, were separately established to train the respective classification models. Four kinds of typical deep learning models (YOLOv5l, SSD300, ResNet18, ResNet34) were employed to verify the identification accuracies of surface features corresponding to different roughness/textures. The human fingers' ability to objects roughness recognition also was quantified through a psychophysical experiment with 3D-printed test objects, as a reference benchmark. The computation results showed that the average roughness recognition accuracies based on SSD300, ResNet18, ResNet34 were higher than 95%, which were superior to those of the human fingers (94% and 91% for 2 and 3 levels of object roughness, respectively). The texture recognition accuracies with all models were higher than 84%. Outcomes indicate that object roughness/textures can be effectively classified using machine vision and exploited for human-machine haptic interaction, providing the feasibility of functional sensory restoration of intelligent robots equipped with visual capture and tactile stimulation devices.