Volume 29, Issue 9 e2993
RESEARCH ARTICLE

A two-stage data cleansing method for bridge global positioning system monitoring data based on bi-direction long and short term memory anomaly identification and conditional generative adversarial networks data repair

Kang Yang

Kang Yang

School of Civil Engineering, Southeast University, Nanjing, China

Key Laboratory of Concrete and Prestressed Concrete Structures of Ministry of Education, Southeast University, Nanjing, China

Search for more papers by this author
Youliang Ding

Corresponding Author

Youliang Ding

School of Civil Engineering, Southeast University, Nanjing, China

Key Laboratory of Concrete and Prestressed Concrete Structures of Ministry of Education, Southeast University, Nanjing, China

Correspondence

Youliang Ding, Key Laboratory of Concrete and Prestressed Concrete Structures of Ministry of Education, Southeast University, Nanjing 210096, China.

Email: [email protected]

Search for more papers by this author
Huachen Jiang

Huachen Jiang

School of Civil Engineering, Southeast University, Nanjing, China

Key Laboratory of Concrete and Prestressed Concrete Structures of Ministry of Education, Southeast University, Nanjing, China

Search for more papers by this author
Hanwei Zhao

Hanwei Zhao

School of Civil Engineering, Southeast University, Nanjing, China

Key Laboratory of Concrete and Prestressed Concrete Structures of Ministry of Education, Southeast University, Nanjing, China

Search for more papers by this author
Gan Luo

Gan Luo

School of Civil Engineering, Southeast University, Nanjing, China

Key Laboratory of Concrete and Prestressed Concrete Structures of Ministry of Education, Southeast University, Nanjing, China

Search for more papers by this author
First published: 11 May 2022
Citations: 15

Funding information: National Natural Science Foundation of China, Grant/Award Numbers: 20190013, 51978154, 52008099; Natural Science Foundation of Jiangsu Province, Grant/Award Number: BK20200369; Fund for Distinguished Young Scientists of Jiangsu Province, Grant/Award Number: BK20190013

Summary

Data cleansing is an essential approach for improving data quality. Therefore, it is the key to avoiding the false alarm of the monitoring system due to the anomaly of the data itself. Data cleansing consists of two parts: anomaly identification and anomaly repair. However, current research on data cleansing has mainly focused on anomaly identification and lacks efficient data repair methods. The key to data repair lies in sensor correlation models based on mapping relationships between sensors. To obtain a good inter-sensor relationship model, it is first necessary to exclude anomalous data from the training data set used for modeling. Therefore, a two-stage data cleansing framework for collaborative multi-sensor repair is proposed. First, based on the analysis of anomalous features of GPS data, a bidirectional long- and short-term memory (Bi-LSTM) neural network model is adopted for data anomalies classification and localization. As a result, the data segment to be repaired is determined. Then, on the basis of all sensor data in the time range of the day before the target repair data segment, the data set for data repair is constructed by excluding the anomaly data segments in the data set with the help of the above anomaly identification results. Then, a conditional generation adversarial network (CGAN) is proposed to achieve data repair. Experimental validation shows that the two-stage data cleansing method of identification followed by repair can accurately identify and repair GPS anomalies. Finally, several factors affecting the repair effect are discussed.

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available from the corresponding author upon reasonable request.

The full text of this article hosted at iucr.org is unavailable due to technical difficulties.