Since a slight variance in production processes can make the entire production run defective, defect inspections are indispensable procedures in manufacturing processes to ensure high quality of each item before entering the next manufacturing step. Three-dimensional (3D) optical shape measurement technologies are widely applied for surface defect inspection of complex workpieces because of its high-accuracy and digitization. However, the complex surface structure and position of the test object can pose serious challenges, making inspections still relatively slow, expensive, and complicated in implementation and maintenance. In this work, we propose a real-time 360∘ 3D surface defect inspection approach based on fringe projection profilometry without any auxiliary equipment for position control. Firstly, a multi-view 3D measurement based on geometric constraints is employed to acquire high-accuracy depth information from different perspectives. Then, a cycle-positioning-based registration scheme with the establishment of the pose-information-matched 3D standard digital model is proposed to realize rapid alignment of the measured point cloud and the standard model. Finally, a minimum 3D distance search is driven by a dual-thread processing mechanism for simultaneous scanning and detecting to quantify and locate 3D surface defects in real time. Experimental results show that our method can accurately identify the surface defects of complicated objects in real time in an extremely simple (hand-held) manner, saving a lot of operational expenses on precision alignment and position-orientation adjustment. The proposed method holds tremendous potential for quality control in many facets of industry, such as product development, testing, and manufacturing.