Volume 2022, Issue 1 9595286
Research Article
Open Access

Steel Plate Defect Recognition of Deep Neural Network Recognition Based on Space-Time Constraints

Chi Zhang

Corresponding Author

Chi Zhang

Department of Industrial Engineering and Management, Peking University, Beijing 100871, China pku.edu.cn

Search for more papers by this author
Zhiguang Wang

Zhiguang Wang

Science and Technology on Special System Simulation Laboratory, Beijing Simulation Center, Beijing 100854, China

Search for more papers by this author
Baiting Liu

Baiting Liu

Science and Technology on Special System Simulation Laboratory, Beijing Simulation Center, Beijing 100854, China

Search for more papers by this author
Wang Xiaolei

Wang Xiaolei

Science and Technology on Special System Simulation Laboratory, Beijing Simulation Center, Beijing 100854, China

Search for more papers by this author
First published: 10 March 2022
Citations: 1
Academic Editor: Qiangyi Li

Abstract

In order to improve the effect of real-time defect recognition in steel plate online production, this paper studies the method of steel plate defect recognition based on the deep neural network algorithm based on space-time constraints. Moreover, this paper improves the space-time constraint algorithm, optimizes the encryption structure of the traditional ABE scheme, and obtains a neural network feature recognition method based on space-time constraints. In order to process the massive image data stream generated instantaneously and ensure the real-time performance, accuracy, and stability of the detection system, this paper constructs a distributed parallel computing system structure based on the client/server (CC/S) model to obtain an intelligent recognition system. Through experimental research, it can be seen that the deep neural network recognition system based on space-time constraints proposed in this paper has a good effect in the recognition of steel plate defects.

1. Introduction

In the past few decades, steel plate rolling production technology and product quality have achieved rapid development. With the improvement of equipment and production technology, the dimensional accuracy, plate shape, mechanical properties, and other quality characteristics of the steel plate have been well controlled. However, the main downstream users of steel plates such as automobiles, aerospace, machinery and electronics, home appliances, and architectural decoration have higher and higher requirements for the surface quality of the steel plates, which makes the surface quality problems of the steel plates more and more prominent [1]. In the application, the surface defects of the steel plate affect the deep pressure of the processing, the coating effect of the final product, the electromagnetic characteristics, and the aesthetics. Therefore, whether it is a steel plate manufacturer or a steel plate user, it is necessary to attach great importance to the surface quality inspection of the steel plate [2].

Whether it is automobile production, shipbuilding, machinery manufacturing, or chemical, aerospace, and other industries, steel is used as raw material for processing and production [3]. The development and growth of various industries has made steel’s raw material status more and more important, and its demand is also expanding. However, in the actual production environment, the aging of rolling equipment, different processing techniques, and differences in raw materials may affect the quality of the steel plate, resulting in the appearance of various types of defects such as cracks, scars, and scratches on the surface of the steel plate. As a result, steel products are of low quality and poor performance. Therefore, we must attach great importance to the quality inspection of steel products to obtain high-quality steel products. At present, large-scale steel plants in China are mainly using machine inspection and manual inspection as a supplementary inspection method. Although this detection method has largely replaced with the traditional manual visual inspection and improved the labor force, most of the similar automatic defect detection equipment in China mostly rely on foreign technical support, and sufficient funds must be invested in repair and maintenance. For minor defects (such as scratches, etc.), manual inspection is still required, which does not meet the needs of production development. This increases our design to meet our own needs. The necessity of the defect detection system is on demand.

Based on the above analysis, this paper uses a deep neural network algorithm based on space-time constraints to build an intelligent system that can be used for steel plate defect recognition and improve the intelligent detection effect of steel plate defects.

2. Related Work

Literature [4] constructed an online surface inspection device for continuous casting slabs, which combines linear CCD chips with digital image processing components. Among them, the CCD chip enhances the effective pixels of the image while accelerating according to the frame transmission rate, which can effectively detect tiny defects. According to the frame transfer rate, the device can effectively detect tiny defects; literature [5] applies the iLearn learning classifier software system to the detection system and develops the iS2000 defect detection system, which uses two to increase A/D conversion. The CCD camera that automatically corrects the light source unit simultaneously acquires bright and dark areas with more uniform illumination. This device takes into account the impact of uneven illumination and effectively reduces the impact of this factor; literature [6] proposed a detection system, tried the theory of combining neural network classifier and CCD camera, and achieved certain success. Literature [7] developed an intelligent steel plate inspection system, which increased the defect data utilization function compared with the original system.

Although these foreign companies have studied steel plate defect detection systems relatively early and the detection technology is relatively mature, there are still problems such as low image acquisition quality, low system versatility, insufficient integration, and use of defect data, which require further research.

Literature [8] applied surface defect detection technology to the quality inspection of cold-rolled steel plate production and added a DSP image processing system based on the original research, using multiple array CCDs to obtain images. The more serious defects such as heavy skin, holes, and edge cracks are effective. However, due to the limitation of the imaging principle of the area array CCD, the use of multiple area array CCDs will produce a large amount of defect image redundant data, and the price is expensive, so this system has not been widely used; literature [9] developed an online inspection system that uses multiple area array CCDs as imaging modules and can be used for steel plate surface defect detection. The system can analyze and process the collected image data in parallel. Although the system guarantees real-time data processing functions, its versatility is low; literature [10] uses a new LED light source and FPGA embedded processing system to complete the design of a linear CCD steel plate defect detection system, which can store data by itself and function and display the test results on the screen. If there is a situation that does not meet the standard, the alarm function is completed, but as the function increases, it is not conducive to the production and development of the steel plate at the expense of reducing the production speed.

Literature [11] developed a system capable of detecting multiple functions of strip steel. Compared with the mainstream testing equipment in the world at that time, the system was comparable in detection rate and defect resolution, but the system needs to be improved. The point is the information on the surface of the strip can be used. Literature [12] uses the edge of the defect to detect the defect target and analyzes the texture feature through the PHOTO algorithm to determine the defect location, which improves the detection accuracy. However, the system has many detection characteristics and the system response time is slow.

Literature [13] designed a fuzzy neural network classifier. The feature of this classifier is to use a multilayer feedforward neural network model combined with a fuzzy algorithm. The classifier improves the classification accuracy while reducing the number of iterations, but the classifier has limitations in identifying defects; literature [14] designed the LVQ neural network classifier, which uses the “reward-punishment” iterative learning algorithm to speed up the classification speed of the strip surface defect detection system. However, the classification accuracy of this classification method is highly dependent on the amount of data; literature [15] studies genetic algorithm and BP neural network algorithm, using the global optimal characteristics of genetic algorithm and BP neural network local accuracy characteristics. With complementary advantages, a combined classifier is designed. However, whether it is genetic algorithm or BP neural network algorithm, more data is needed, which increases the amount of data processing; literature [16] uses the BP neural network classifier; the difference is that the high-order classifier is added to this classifier. High-order perturbation can prevent the training data from converging to a local extreme value and at the same time can improve the training speed. However, the addition of high-order disturbances has little contribution to the accuracy of defect detection. Literature [17] combined wavelet transform and BP neural network to the steel plate defect classification system, but the defect detection accuracy is about 90%, which needs to be improved. Compared with the traditional BP neural network, the improved neural network classification method has a more obvious classification effect on the trained data, but it often produces errors for the data outside the training library [18]. In practical applications, too large data set samples are not easy to obtain. Therefore, it is of great practical significance to study small sample data.

3. Deep Neural Network Recognition Based on Space-Time Constraints

For the attribute AiA of the user, it means that there are two sets of (li,j, li,k) and (li,j, li,k) that, respectively, represent the range of covering user attribute values, which correspond to two cryptographic values v{li, lk} and . If and li,kli,k exist, then it will be easy to calculate v{li, lk} from v{li, lk}, and vice versa.

In order to construct an MDRDF function for integer comparison of multiple attributes, this paper uses RSA and assumes that is a multiplicative cyclic group, where φ is a generator in the multiplicative cyclic group. n = pq and is a set of the largest random number in , and each λi and μi are prime numbers relative to the other elements of the set.

is an upper and lower bound of the attribute value corresponding to all attributes AiA, ψU is a cryptographic mapping about the order of user U, which is mapped to a value that reflects a cryptographic boundary of each attribute, and z is the largest integer value of user U. If and , this order-preserving cryptographic mapping means that there are two situations:
(1)
This article defines the formula of this mapping function ψ(⋅) from user set U to V as follows [19]:
(2)
If the following two conditions are met, the function F from Vv is based on the mapping of U to v.
  • (1)

    If , there are

    (3)

  • (2)

    If for a certain attribute AiA, there is or , then it is impossible for a polynomial algorithm to derive at any time [20].

(4)

Figure 1 shows that when (li,j, li,k)⊆(li,j, li,k), is mapped to v{li,j, li,k}AiA through the function F.

Details are in the caption following the image

In the process of data encryption, this article optimizes the encryption structure of the traditional ABE scheme. The traditional ABE scheme uses the structure of the access tree to encrypt the user’s attributes. Based on this, this paper redesigned the structure of the access tree. This paper considers embedding spatiotemporal constraints into the access tree structure and uses the multidimensional distance derivation function (MDRDF) combined with the threshold to determine the legitimacy of the user.

As shown in Figure 2, this paper uses the structure of the access tree to enhance fine-grained access. Among them, each leaf node represents the attributes A1,  A2, …, An owned by the user, and the nonleaf nodes represent logic gates (AND, OR, Threshold). In Figure 2, node x is represented by numx, and the threshold is represented by kx.

Details are in the caption following the image

In Figure 2, we consider embedding spatiotemporal constraints into any nonleaf node. At the same time, represents the time range constraint related to node x, and represents the location range constraint related to node x. In the structure of the access tree in Figure 2, the child access policy corresponding to node n1 needs to meet “A1∧A2”, and the child access policy corresponding to node n2 needs to meet “ + ”. If and only if the user wants to access the resource, it needs to meet the attribute requirement of “(A1∧A2)∨(A3∧A4) ”, and at the same time, his access time and space can be successfully accessed within the scope of [ta, tb][la, lb].

It chooses any two generators , so e(g, φ) = 1, but e(g, ω) ≠ 1. At the same time, three hash functions are used: and KGC arbitrarily selects two exponents and and generates h = ωβ, η = g1/β, ς = e(g, ω)α. Publishing system is as follows:
(5)
The master key of the system is as follows:
(6)
For each user there is a set of attributes Sgid; the key generation center KGC first selects and selects for each attribute iSgid belonging to the user set. Then calculate the key as follows:
(7)
We assume that the user gid is assigned time and space access rights [ta, tb] and [la, lb], where Ft and Floc represent the unique format of time and space in the system, respectively. It should be noted that [ta, tb] and [la, lb], respectively, represent the starting time and ending time and the starting position and ending position in the system, and all elements are discrete integers with a total order. KGC selects an arbitrary for each time attribute and space attribute and generates the corresponding time key and space key as shown below:
(8)

Finally, it outputs the user key SKgid = {SKAttr, DK[ta, tb], DKla,lb}.

Among them, is the parameter inherited from its parent node (or if x is the root node, it is used to process the root node), and is the parameter shared with its child nodes (or if x is a leaf node, then it is used for processing the relevant attributes of the user). If the time threshold or position threshold is related to node x, then it will be related to and , respectively. For any node, the value can be used to calculate as follows:
(9)
The polynomial qx arbitrarily chooses and dx = kx − 1. . Among them, χ is the collection of leaf nodes in the access tree, γ represents the attribute collection related to the time range [ta, tb], and Z represents the attribute collection related to the location range [la, lb]. Therefore, the structure of the ciphertext is as follows:
(10)
The cloud server will check whether the time tc and location lc of the node meet the time range constraint [ta, tb] and the location range constraint [la, lb]. If it is not satisfied, and are set to the special symbol ⊥. If it is satisfied:
(11)
Therefore, the final reencrypted ciphertext is
(12)

Finally, the cloud server sends the encrypted ciphertext CT, the current time tc, and the current location lc to the user.

Here, a recursive algorithm is used to define our decryption process.

First, pass a recursive algorithm DecrytNode(CT, SKgid, x) through the input cipher text CT and the private key SKgid and visit the node V in the tree x. If the node is a leaf node, then we set i = attr(x) and have the following definitions. If iSgid, then we have
(13)

If iSgid, then we define Decryt Node (CT, SKgid, x) = ⊥.

tc ∈ [ti, tj] and tc ∈ [ta, tb] are required to ensure safe time access control. It should be noted that At[ti, tj] is the access strategy for node y in the access tree, and At[ta, tb] is the access authority range of the time the user has. When the user’s access time is valid, it is calculated as follows:
(14)
Similarly, we will consider access to node ∀zZ in the tree and require lc ∈ [li, lj] and lc ∈ [la, lb] to ensure safe time access control. It should be noted that Al[tl, tl] is the access strategy for node z in the access tree, and Al[la, lb] is the access authority range of the time the user has. When the user’s access time is valid, it is calculated as follows:
(15)
Similarly, we will consider access to node ∀zZ in the tree and require lc ∈ [li, lj] and lc ∈ [la, lb] to ensure safe time access control. It should be noted that Al[tl, tl] is the access strategy for node z in the access tree, and Al[la, lb] is the access authority range of the time the user has.
(16)
If it has nothing to do with any time limit gate and position limit gate, then we can get the following formula:
(17)
We can get the following formula:
(18)
For any node x, if it is related to any position limit gate, then we can get the following formula:
(19)
With limit gate and position limit gate, then we can get the following formula:
(20)
Finally, we need to consider when x is a nonleaf node recursively; for node x, its child node set Sx composed of all its child nodes z has kx children. If there is no such set, the node is not satisfied and the decryption function returns ⊥. Otherwise, we will calculate
(21)
We can decrypt the plaintext as follows:
(22)

The DCNN-FDM deep convolutional neural network defect recognition model used in this paper is shown in Figure 3.

Details are in the caption following the image

4. Steel Plate Defect Recognition of Deep Neural Network Recognition System Based on Space-Time Constraints

In order to process the massive image data stream generated instantaneously and ensure the real-time performance, accuracy, and stability of the detection system, a distributed parallel computing system structure based on the client/server (C/S) model can be constructed, as shown in Figure 4. The overall architecture diagram of the online detection system for steel plate surface defects is shown. It can be seen from the figure that the entire detection system is composed of a client (lower computer), a server (upper computer), and a control terminal. Each camera is controlled by an independent client to ensure parallel processing of image data. The client receives related instructions from the control terminal to control the camera to realize the image collection function. At the same time, the client transmits the processed image data stream to the server and displays the relevant processing results in real time on the control terminal. The control terminal realizes the control of the entire detection system by setting the detection method and related parameters of the hardware device. Among them, the interaction of the image data stream between the server and the client can be carried out by using gigabit (1 Gb/s) industrial Ethernet, and the interaction of the control signal flow between the server and the client can be carried out by using a field bus.

Details are in the caption following the image

It is necessary to design a software system with a reasonable structure and clear levels to manage and deploy all hardware devices. At the same time, it is also necessary to design a fast, accurate, and stable image detection method to process and analyze the image, so that the entire detection system can run efficiently and in an orderly manner. Figure 5 shows the software architecture diagram of the online detection system for steel sheet surface defects.

Details are in the caption following the image

The image detection method of steel plate surface defects can be divided into two parts: online processing and offline processing. Among them, the online processing adopts a streamlined operation mode that combines real-time processing and quasi-real-time processing. Figure 6 shows the schematic flowchart of the image detection method for steel sheet surface defects. The preprocessing and screening operations of the steel plate surface image must be completed online and in real time in order to find and mark the steel plate surface defect image in time. However, operations such as segmentation and classification of the image of steel surface defects can be processed in the online quasi-real-time link, and the training of the classifier can be processed in the offline link.

Details are in the caption following the image

Due to the single color of the steel plate surface, the difference between the defective area and the normal area can be distinguished by the difference in light intensity. Therefore, this experimental platform uses a black and white line scan camera. Figure 7 shows the imaging schematic diagram of a line scan camera. The yellow area in the figure represents an image pixel (dot), rw represents the horizontal resolution, and rh represents the vertical resolution.

Details are in the caption following the image

The steel plate defect recognition is carried out through the system of this article, as shown in Figure 8. It is a schematic diagram of the steel plate defect recognition.

Details are in the caption following the image

The system introduces the overall structure, analyzes the light source lighting module and the image acquisition module, and effectively reduces the impact of the illumination angle on the image quality of the steel plate. Moreover, this article separately elaborates the main functions of the client, server, and monitoring console to complete the macrocontrol.

The data used to describe the defect information is huge. Therefore, accurate and effective recording of these data is very important for the system to quickly and accurately obtain the recognition results. The database management system is used to separate the detection system and the database, and the collected steel plate defect image information is converted into the set standard format and stored in the database. While ensuring real-time detection, it can ensure the safety and stability of steel plate image data and its dependence on historical data. In the steel plate surface defect detection system designed in this paper, the database is mainly used in two modules, one module is the application of real-time image acquisition, and the other module is the application of image punctual processing. In the database design process, for different types of data, a reasonable classification of the planning table is carried out to make the database hierarchical, organized, and easy to access and process the data. For different tables, this article uses reasonable definition of the fields of each table to avoid redundant data and combines the actual needs to determine the data type of each field reasonably and finally achieves the purpose of convenient retrieval and easy maintenance. Finally, this article chooses the only certain content as the primary key of the data table, and only one primary key can be defined in a table, and the connection between the tables is established through the primary key, thus forming a set of interconnected tables.

The simulation platform is used to construct a deep neural network recognition system based on space-time constraints. This paper selects 7 types of defect images collected at the production site. Table 1 shows the types and numbers of steel defect samples selected for testing.

1. Types and numbers of steel defect samples selected for testing.
Types of defects Category number
Scarring A
Transverse crack B
Horizontal scratch C
Longitudinal crack D
Longitudinal scratch E
Reticulated F
Pitting G

The defect identification of the system of this paper mainly includes two parts: preprocessing and defect splitting. Steel plate defect pretreatment detection is to screen out defective steel plates, and the steel plate classification setting classifies the defective steel plates, and the pretreatment results are shown in Table 2 below.

2. Statistical table of accuracy of steel plate defect pretreatment detection.
Group no Predetection accuracy rate (%) Group no Predetection accuracy rate (%)
1 98.92 16 96.97
2 96.18 17 98.92
3 96.48 18 96.61
4 97.19 19 98.69
5 98.68 20 97.54
6 97.50 21 96.98
7 97.94 22 96.33
8 98.75 23 97.40
9 98.55 24 97.86
10 97.03 25 98.01
11 98.75 26 97.02
12 96.23 27 96.01
13 97.73 28 97.27
14 98.01 29 96.44
15 98.21 30 97.08

Based on the above research, this paper evaluates the effect of steel plate defect classification on the deep neural network recognition system based on space-time constraints, and the results are shown in Table 3 and Figure 9.

3. Evaluation of the effect of steel plate defect classification.
Types of defects Category number Classification accuracy (%)
Scarring A 90.99
Transverse crack B 93.24
Horizontal scratch C 91.61
Longitudinal crack D 89.02
Longitudinal scratch E 89.95
Reticulated F 89.45
Pitting G 87.23
Details are in the caption following the image

From the above research, it can be seen that the deep neural network recognition system based on space-time constraints proposed in this paper has a good effect in the recognition of steel plate defects, and it can be applied to the automatic detection of steel plate online production.

5. Conclusion

The defects on the surface of the steel plate have different shapes and complex causes. According to the appearance characteristics of defects, it can be divided into point defects, line defects, and surface defects. Point defects appear on the surface of the steel plate in the form of dots. Some are single-point occurrences, some are continuous occurrences in relatively concentrated areas, some are periodic, such as roll prints, sticky rolls, etc., and some have no regularity, such as spot rust, pitting, etc. Line defects often appear in the form of lines. Some are continuous, some are intermittent, some are horizontal, and some are vertical, such as scratches, cracks, and streaks. Surface defects often appear in slices on local areas of the surface of the steel plate, which have a certain area, such as emulsion spots, embroidered spots, and oil stains. This article will build an intelligent system that can be used for steel plate defect recognition based on the deep neural network algorithm based on space-time constraints. Through experimental research, it can be seen that the deep neural network recognition system based on space-time constraints proposed in this paper has a good effect in the recognition of steel plate defects, and it can be applied to the automatic detection of steel plate online production.

Conflicts of Interest

The authors declare no competing interests.

Acknowledgments

This study was sponsored by Science and Technology on Special System Simulation Laboratory of Beijing Simulation Center and Peking University.

    Data Availability

    The labeled dataset used to support the findings of this study is available from the corresponding author upon request.

      The full text of this article hosted at iucr.org is unavailable due to technical difficulties.