D²-SPDM: Faster R-CNN-Based Defect Detection and Surface Pixel Defect Mapping with Label Enhancement in Steel Manufacturing Processes

The steel manufacturing process is inherently continuous, meaning that if defects are not effectively detected in the initial stages, they may propagate through subsequent stages, resulting in high costs for corrections in the final product. Therefore, detecting surface defects and obtaining segmentation information is critical in the steel manufacturing industry to ensure product quality and enhance production efficiency. Specifically, segmentation information is essential for accurately understanding the shape and extent of defects, providing the necessary details for subsequent processes to address these defects. However, the time-consuming and costly process of generating segmentation annotations poses a significant barrier to practical industrial applications. This paper proposes a cost-efficient segmentation labeling framework that combines deep learning-based anomaly detection and label enhancement to address these challenges in the steel manufacturing process. Using ResNet-50, defects are classified, and faster region convolutional neural networks (faster R-CNNs) are employed to identify defect types and generate bounding boxes indicating the defect locations. Subsequently, recursive learning is performed using the GrabCut algorithm and the DeepLabv3+ model based on the generated bounding boxes, significantly reducing annotation costs by generating segmentation labels. The proposed framework effectively detects defects and accurately defines them, even in randomly collected images from the steel manufacturing process, contributing to both quality control and cost reduction. This study presents a novel approach for improving the quality of the steel manufacturing process and is expected to enhance overall efficiency in the steel manufacturing industry.