Contextual multimodal approach for recognizing concurrent activities of equipment in tunnel construction projects.

In order to accurately track progress and improve efficiency in complex construction projects, it's important to effectively monitor individual tasks and measure the time taken to complete a cycle of tasks. Tunnel construction involves a variety of activities, where multiple pieces of equipment are engaged in different actions that occur simultaneously or sequentially during a single activity. This study introduces a contextual audio-visual (multimodal) approach to better recognize multi-equipment activities in a tunnel construction site for monitoring purposes. By incorporating both audio and visual data, and by integrating both spatial and cyclical temporal contexts, the model accurately recognizes the activity being performed by multiple pieces of equipment more often than single-mode models. Tested against real-world operation data, the model achieved a remarkable F-score of 96.3% in recognizing construction activities, demonstrating its superiority over traditional methods in scenarios involving multiple, simultaneously operating pieces of equipment. The results emphasize the potential of contextual multimodal models in enhancing operational efficiency in complex construction sites. • An audio-visual approach is introduced for equipment monitoring in tunnel. • Concurrent activities with multiple pieces of equipment are recognized. • The model integrates spatial and cyclical temporal contexts of activities. • The model achieved a F-score of 96.3% in recognizing activities. [ABSTRACT FROM AUTHOR]