An end-to-end 3D vision-based pipeline for structural damage detection and quantification

We proposed an end-to-end 3D computer vision framework for more comprehensive structural damage detection and quantification. The proposed pipeline is the first-of-its-kind to create a 3D digital twin of structures using accurate and low-cost 3D computer vision methods and advanced robotic technologies, which can be used to detect various types of structural damages concurrently, and quantify the in-plane and out-of-plane deformations.

3D vision-based damage quantification framework


We examined the framework on steel plate structures including steel plate dampers and steel corrugated plate walls. Steel materials are widely used worldwide to develop and construct innovative energy dissipation devices and force-resisting components in civil structures. This research project aims to utilize state-of-the-art sensing techniques and AI algorithms to detect and quantify structural damage of critical steel structures. Currently, two full-scale experimental specimens tested at the UBC structural laboratory are selected to examine the techniques and algorithms.

The honeycomb structural fuse (HSF) utilizes commonly available welded wide flange sections with honeycomb-shaped perforations on the web. The HSF is designed to dissipate earthquake energy through plastic deformation of the web in shear, while the flanges remain elastic. The HSF can be fabricated into different shapes to fit different structural demands. The intent of the research is to develop damage detection techniques for critical damage patterns (e.g., fatigue damage, fracture, corrosion, delamination, buckling) experienced by such plated damping devices.

Honeycomb structural fuse (undamaged) Honeycomb structural fuse (damaged)


On the other hand, steel corrugated panels are widely used in Canada to construct long-span low-rise frameless systems. However, such systems are prone to have critical structural damages under extreme loading conditions such as earthquakes. Currently, extensive experiments are conducted on these panels to establish the hysteretic behaviour and understand the damage patterns. The intent of the research is to develop damage detection methods for such panels.

Steel panels tested at the UBC structural laboratory


The proposed framework consists of vision-based 3D scene reconstruction, multi-view steel component detection, and advanced 3D point cloud postprocessing for damage quantification.

Application on steel plate dampers


Application on steel corrugated plate walls


3D structural component detection


Advanced 3D point cloud processing


Quantification of the out-of-plane damage


The experimental results indicate the proposed framework can successfully reconstruct the steel plate structures, effectively localize the steel components from a 3D scene, and accurately quantify the out-of-plane displacements with an accuracy of about 1 mm. The main contributions and novelties of the paper are concluded:

(a) A novel 3D vision-based framework has been proposed to quantify out-of-plane displacements of steel plate structures.

(b) A novel vision-based multi-view 3D object detection method is proposed to detect structural components.

(c) The proposed method provides a non-contact measurement approach and a more economical solution in both equipment and setup cost, compared to traditional measurement methods using contact-type displacement sensors or laser devices;

(d) The proposed method provides finer measurement results, compared to traditional contact-type displacement sensors that measure displacement at limited sensor locations.

(e) The proposed method can provide information to perform geometry model updating of steel plate structures in the field after natural disasters.

(f) The concept of the proposed 3D vision-based displacement measurement can be generalized to quantify other structural damages such as spalling volume of concrete at high accuracy and low cost.


Team composition: Xiao Pan, Soham Vaze, Tianyi Li, and T.Y. Yang


If you feel our work is inspiring or helpful to your research, please consider citing the following:


[1] Pan, X., Yang, T. Y. (2022). 3D vision-based out-of-plane displacement quantification for steel plate structures using structure from motion, deep learning and point cloud processing. Computer-aided Civil and Infrastructure Engineering,

[2] Yang, T. Y., Li, T., Tobber, L., & Pan, X. (2020). Experimental and numerical study of honeycomb structural fuses. Engineering Structures, 204, 109814.

[3] Pan, X., Vaze, S., Xiao, Y., Tavasoli, S., Yang T.Y. “Structural damage detection of steel corrugated panels using computer vision and deep learning.” Canadian Society for Civil Engineering (CSCE) conference, 2022, Whistler, British Columbia, Canada. accepted.