Multi-modal AI-based Water Pipeline Data Accumulation and Leakage Prediction Research using Image and Ultrasound Data

This study presents a multi-modal artificial intelligence (AI)-based water pipeline maintenance method based on RGB images and ultrasound data. Our methodology leverages the concurrent collection of visual and auditory data from pipelines to improve the detection and prediction of anomalies such as abnormal welds, corrosion, scale, and cracks. By converting ultrasound data into spectrogram images using short-time Fourier transform (STFT) and combining them with RGB images, we create a composite data input for a convolutional neural network (CNN) model. This model is trained to classify the condition of water pipelines into distinct categories based on multi-modal inputs. The fusion of these two data modalities aims to significantly enhance the accuracy of pipeline anomaly detection, offering a novel approach for predictive maintenance in water pipeline facilities. We tested on 6 different classes with each 100 pair of datasets, therefore a total of 600 pairs of RGB and spectrogram images, and achieved an average accuracy of 92.7%. Our research contributes the potential application of multi-modal AI in pipeline maintenance.