Background A construction method has emerged in which a camera is installed around a construction machine, and the operator remotely controls the machine while synchronizing the vibration of the machine with the images seen from the operator's seat using virtual reality (VR) technology. Indices related to changes in heart rate (HR) and physical vibration, such as heart rate variability (HRV) and multiscale entropy (MSE), can then be measured among the operators. As these indices are quantitative measures of autonomic regulation in the cardiovascular system, they can provide a useful means of assessing operational stress. Objective In this study, we aimed to evaluate changes in HR and body vibration of machine operators and investigate appropriate methods of machine operation while considering the psychological load. Methods We enrolled 9 remote operators (18-50 years old) in the experiment, which involved 42 measurements. A construction machine was driven on a test course simulating a construction site, and three patterns of operation—riding operation, remote operation using monitor images, and VR operation combining monitor images and machine vibration—were compared. The heartbeat, body vibration, and driving time of the participants were measured using sensing wear made of a woven film-like conductive material and a three-axis acceleration measurement device (WHS-2). We used HRV analysis in the time and frequency domains, MSE analysis as a measure of the complexity of heart rate changes, and the ISO (International Standards Organization) 2631 vibration index. Multiple regression analysis was conducted to model the relationship among the low frequency (LF)/high frequency (HF) HRV, MSE, vibration index, and driving time of construction equipment. Efficiency in driving time was investigated with a focus on stress reduction. Results Multiple comparisons conducted via the Bonferroni test and Kruskal-Wallis test showed statistically significant differences (P=.05) in HRV-LF/HF, the vibration index, weighted acceleration, motion sickness dose value (MSDVz), and the driving time among the three operation patterns. The riding operation was found to reduce the driving time of the machine, but the operation stress was the highest in this case; operation based on the monitor image was found to have the lowest operation stress but the longest operation time. Multiple regression analysis showed that the explanatory variables (LH/HF), RR interval, and vibration index (MSDVz by vertical oscillation at 0.5-5 Hz) had a negative effect on the driving time (adjusted coefficient of determination R2=0.449). Conclusions A new method was developed to calculate the appropriate operating time by considering operational stress and suppressing the physical vibration within an acceptable range. By focusing on the relationship between psychological load and physical vibration, which has not been explored in previous studies, the relationship of these variables with the driving time of construction machines was clarified.