Automated construction safety reporting system integrating deep learning-based real-time advanced detection and visual question answering

The construction sector is globally acknowledged as one of the most hazardous industries, owing to the vulnerability of its workers to accidents, injuries, and even loss of life. Effective precautionary measures are necessary and ensuring the use of personal protective equipment (PPE) by workers is crucial for protecting them from accidents. Existing deep learning-based PPE detection systems mainly use simple vision-based target detection methods for tasks such as the identification of helmets or vests, and they tend to be task-specific. However, the identification of specific PPE based on respective job types and maintaining detailed safety records, requires further innovative approaches. In this paper, we propose an innovative intelligent system that not only accurately recognizes specific PPE according to the needs of different work types but also automatically generates safety inspection reports and establishes complete safety records, thus providing critical data to support accident investigations. The proposed system integrates a target detection model, visual question answering model, and text-based analysis of the relevant regulations to realize real-time detection of PPE and automatic generation of safety inspection reports. The experimental results show that the proposed YOLOv8n-DCA network strikes a good balance between performance and computational cost—, with a mAP value of 86%. Compared to the original YOLOv8n network, the mAP value is improved by 5.1%, while the model parameters and size are significantly reduced. Further, the visual question answering model exhibited a precision is 95.9. Finally, the automatic generation of safety inspection reports was successfully realized, verifying the feasibility of the developed system. This innovative system promises a comprehensive and efficient PPE management solution for the construction industry to ensure worker safety and provide strong data support.