Real-time monitoring of bolt loosening through deep learning-based detection and tracking methods

Structural bolts are critical components used in different structural elements, such as beam-column connections and friction damping devices. The clamping force in structural bolts is highly influenced by bolt rotation. Much of the existing vision-based research about bolt rotation estimation relies on traditional computer vision algorithms such as Hough Transform to assess static images of bolts. This requires careful image preprocessing, and it may not perform well in the situation of complicated bolt assemblies, or in the presence of surrounding objects and background noise, thus hindering their real-world applications. In this study, an integrated real-time detect-track method, namely RTDT-Bolt, is proposed to monitor the bolt rotation angle.

Bolt loosening monitoring pipeline

First, a real-time convolutional-neural-networks-based object detector, named YOLOv3-tiny, is established and trained to localize structural bolts.

Training and testing

Sample testing images

Then, a target-free object tracking algorithm based on optical flow is implemented, to continuously monitor and quantify the rotation of structural bolts. In order to enhance the tracking performance against background noise and potential illumination changes during tracking, the YOLOv3-tiny is integrated with the optical flow tracking algorithm to re-detect the bolts when the tracking gets lost.

Bolt loosening rotation tracking

Extensive parameter studies were conducted to identify optimal tracking performance and examine the potential limitations. The results indicate the RTDT-Bolt method can greatly enhance the tracking performance of bolt rotation, which can achieve over 90% accuracy using the recommended range for the parameters.

Team composition: Xiao Pan and T.Y. Yang

If you feel our work is inspiring or helpful to your research, please consider citing the following:

Reference: Pan, X., & Yang, T. Y. (2021). Image-based monitoring of bolt loosening through deep learning-based integrated detection and tracking. Computer‐Aided Civil and Infrastructure Engineering.