Automatic Robotic Ultrasound for 3D Musculoskeletal Reconstruction: A Comprehensive Framework

Musculoskeletal ultrasound (US) imaging faces challenges such as operator experience, limited spatial flexibility, and high personnel costs. This study introduces an Automated Robotic Ultrasound Scanning (ARUS) system that integrates key technological advancements to automate the ultrasound scanning procedure with the robot, including anatomical target localization, automatic trajectory generation, deep-learning-based segmentation, and 3D reconstruction of musculoskeletal structures. The ARUS system consists of a robotic arm, ultrasound imaging, and stereo vision for precise anatomical area detection. A Graphical User Interface (GUI) facilitates a flexible selection of scanning trajectories, improving user interaction and enabling customized US scans. To handle complex and dynamic curvatures on the skin, together with anatomical area detection, the system employs a hybrid position–force control strategy based on the generated trajectory, ensuring stability and accuracy. Additionally, the utilized RA-UNet model offers multi-label segmentation on the bone and muscle tissues simultaneously, which incorporates residual blocks and attention mechanisms to enhance segmentation accuracy and robustness. A custom musculoskeletal phantom was used for validation. Compared to the reference 3D reconstruction result derived from the MRI scan, ARUS achieved a 3D reconstruction root mean square error (RMSE) of 1.22 mm, with a mean error of 0.94 mm and a standard deviation of 0.77 mm. The ARUS system extends 3D musculoskeletal imaging capacity by enabling both bones and muscles to be segmented and reconstructed into 3D shapes in real time and simultaneously. These features suggest significant potential as a cost-effective and reliable option for musculoskeletal examination and diagnosis in real-time applications.