1. Attention-based image captioning for structural health assessment of apartment buildings.
- Author
-
Dinh, Nguyen Ngoc Han, Shin, Hyunkyu, Ahn, Yonghan, Oo, Bee Lan, and Lim, Benson Teck Heng
- Subjects
- *
CONVOLUTIONAL neural networks , *NATURAL language processing , *BUILDING inspection , *RECURRENT neural networks , *COMPUTER vision - Abstract
Automated visual assessment report generation in structural health monitoring (SHM) offers advantages for building inspections. However, current vision-based approaches that focus primarily on local surface detection cannot be directly used for inspection reports without further interpretation of the detected labels and coordinator metrics for an appropriate serviceability assessment. To address this gap, this paper presents an automated textual assessment framework for retrieving and generating linguistic descriptions of building component images. Six attention-based captioning methods were constructed based on convolutional neural network (Inception-V3, Xception, and ResNet50) and recurrent neural network (GRU, LSTM), and experimented via 7430 pairs of building component images and captions. The results indicated that the proposed methods had good predictive power and ResNet50-LSTM outperformed other methods with average precision, recall, and F1 scores of 0.84, 0.74, and 0.79, respectively. This paper highlights the potential of the image captioning approach for producing accurate and timely periodic structural assessment reports. • An image captioning approach is proposed for automating the structural monitoring health (SMH) of apartment buildings. • Six attention-based encoder-decoder models using CNNs and RNNs architectures were developed for comparative analysis. • A custom dataset of 7430 pairs of image-caption data was extracted from 563 safety inspection reports. • The ResNet50-LSTM model showed great superiority across all evaluation metrics, with an average F1 score of 0.79. • This demonstrates the potential of IC techniques to the current practices of SMH in the construction industry. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF