Pipeline leakage detection is an integral part of pipeline integrity management. Combining AE (Acoustic Emission) with deep learning is currently the most commonly used method for pipeline leakage detection. However, this approach is usually applicable only to specific situations and requires powerful signal analysis and computational capabilities. To address these issues, this paper proposes an improved Transformer network model for diagnosing faults associated with abnormal working conditions in acoustic emission pipelines. First, the method utilizes the temporal properties of the GRU and the positional coding of the Transformer to capture and feature extract the data point sequence position information to suppress redundant information, and introduces the largest pooling layer into the Transformer model to alleviate the overfitting phenomenon. Second, while retaining the original attention learning mechanism and identity path in the original DRSN, a new soft threshold function is introduced to replace the ReLU activation function with a new threshold function, and a new soft threshold module and adaptive slope module are designed to construct the improved residual shrinkage unit (ASB-STRSBU), which is used to adaptively set the optimal threshold. Finally, pipeline leakage is classified. The experimental results show that the NDRSN model is able to make full use of global and local information when considering leakage signals and can automatically learn and acquire the important parameters of the input features in the spatial and channel domains. By optimizing the GRU improved Transformer network recognition model, the method significantly reduces the model training time and computational resource consumption while maintaining high leakage recognition accuracy. The average accuracy reached 93.97%. This indicates that the method has good robustness in acoustic emission pipeline leakage detection.
1.Introduction
Pipelines are commonly used to transport fluids such as water, oil, and natural gas [1]. However, sometimes they leak, resulting in environmental damage, economic losses, human deaths and other disasters [2]. Therefore, fault diagnosis has become the focus of attention. To prevent accidents and ensure the normal operation of the pipelines, it is necessary to conduct fault diagnosis [3].
Researchers have proposed a number of pipeline leak diagnosis methods, among which vibration-based and acoustic-based pipeline leak diagnosis methods are two popular types of solutions. The vibration based method uses the vibration signal collected from the inside of the pipe for fault detection, which is inexpensive and fast [4]. The acoustic-based approach utilizes acoustic signals collected by acoustic sensors installed near the pipe. Detection technology based on sound waves has attracted much attention due to its low cost and fast detection [5].
Acoustic emission detection utilizes acoustic signals generated by the pipe itself for nondestructive detection. It is independent of the geometry of the object under test and therefore requires less environmental proximity. This makes acoustic emission detection suitable for use in environments that are otherwise inaccessible and allows for early warning of pipeline leaks [6]. Moreover, since leakage acoustic emission waves captured by a data acquisition system are essentially digital signals, modern pattern recognition methods are generally adopted for data processing. To improve the accuracy of classification problems in leakage detection.
Many scholars have explored deep learning methods [7]. Deep learning differs from traditional machine learning methods in that it uses an end-to-end approach. This approach allows deep learning models to learn fault features directly from raw data without having to manually extract features. This ability to learn automatically helps to solve the problem of low generalization performance in traditional machine learning methods, thus improving the performance and accuracy of the model. Zhao et al [8] proposed a new hybrid scheme for the prediction of RULs with multiworking condition online regression CNNs of rolling bearings and modified the regression prediction results based on CNNs to improve the accuracy and smoothness of the prediction of RULs. Li et al [9] proposed a new method that combines a sparrow search algorithm and a convolutional neural network for detecting leaks in oil pipelines. This method is called SSA-CNN. Unlike traditional time-series data, this method transforms the input data into a two-dimensional form to better accommodate the processing of convolutional neural networks; however, the number of pipeline leakage experiment data samples was low, and the model could not be continuously updated while guaranteeing real-time detection. Spandonidis et al [10] proposed a 2D-convolutional neural network (CNN) model and long short-term memory autoencoder (LSTM AE), which directly receives signals from an accelerometer and provides an unsupervised leak detection solution. However, the signal before artificial leakage induction has the same order of magnitude as the noise generated on the pipe wall during small leakage, which makes leakage detection unidentifiable. A high performance of deep learning networks usually requires a large amount of labeled data so that the network can learn accurate patterns and features from the data. In addition, to ensure the ability of the network to generalize in real-world applications, it needs to be evaluated and tested using independent data that are distributed with the same data as the training data. In a real-world diagnostic task, meeting these two conditions is challenging.
Additionally, convolutional neural networks have strong advantages in terms of training capacity and hardware support, and are therefore widely adopted in various fields [11–13]. Due to the limitations of convolutional operations, the perception ability of convolutional networks is severely weakened when depth features are extracted. This leads to poor deep feature extraction ability and incomplete global modeling, which leads to the disadvantage of poor global information capture ability and limits the further development of convolutional networks. To effectively solve the remote capture capability of convolutional networks, several scholars have proposed long short-term memory (LSTM) [14] and gated recurrent unit (GRU) [15] networks as network prediction models for solving the gradient problem. Both are improved by the cyclic unit in the RNN. However, because the GRU has difficulty performing parallel computations, resulting in low time efficiency the GRU algorithm, is not used much in engineering. Therefore, how to capture the remote features of time series and perform parallel computations to ensure high time efficiency has always been a difficult problem faced by researchers. Under this requirement, the Transformer neural network was developed [16]. The Transformer model proposed in 2017 [17] supports parallel computing, provides faster training, and can model both long-term and short-term dependencies at the same time, showing good effects in processing time data series [18, 19]. Transformers have achieved good results in various fields of natural language processing [20]. Zhou et al [21] used an MCST-Transformer (multichannel spatiotemporal Transformer) to forecast traffic flow. Feng et al [22] proposed an end-to-end hybrid convolutional transformer (HCT) segmentation network for semantically based automatic segmentation and recognition of metal coupler fracture modes. Zhu et al [16] proposed a multiscale domain adaptive method based on Transformer-CNN for fault diagnosis when data are scarce. Liu et al [3, 23] proposed a dual encoder model based on transformers to predict the monthly runoff from the Yangtze River. Since the encoder and decoder of the Transformer use a self-attention mechanism, this leads to greater computational space complexity. In addition, due to the nature of the self-attention mechanism, the Transformer model has a relatively weak perception of local information features, which may increase the sensitivity of the model to anomalies. Therefore, we need to further consider how to optimize these issues.
To better apply the Transformer model to pipeline leakage detection, this paper couples the gated cycle unit (GRU) layer with a better predictive effect in the Transformer input part, improves the structure of the Transformer model, and builds a GRU improved Transformer network coupling model. Simulation detection of pipeline leakage was performed.
2.NDRSN denoising method
2.1.Deep residual shrinkage network model
The deep residual shrinkage network (DRSN), an improved version of ResNet, combines the characteristics of the residual network's identical cross-layer connections to enhance the stability of the deep learning model and improve the training efficiency. Moreover, two methods, soft thresholding and an attention mechanism [24], are introduced, and can effectively overcome the defects existing in the ResNet model. The operations in the particular model consist of performing absolute value operations on the feature map and then passing the result to the disparity layer to obtain image x. Next, image x is transformed into a one-dimensional vector and fed into a fully connected network with the same number of channels. To restrict the output to the range (0, 1), a sigmoid function is used for scaling.
where 
is the neuronal property, and 
is the scale factor. The threshold formula is as follows:
where 
is the threshold of the feature channel, 
is the width, and 
is the height.
Signals collected in real-world environments often contain high-intensity noise that can hinder the feature extraction capabilities of deep learning models. To solve this problem, the DRSN relies on its powerful noise signal feature extraction capability, which increases its ability to complete current fault diagnosis tasks in real industrial scenarios. The DRSN model architecture is shown in figure 1:
The basic architecture of the DRSN includes Conv, a residual subblock unit (RSBU), batch normalization (BN), a corrected linear unit (ReLU) activation function, global average pooling (GAP) and a cross entropy error function.
2.2.ASB-STRSBU module
The NDRSN consists of a convolution layer (Conv), an improved residual shrinkage unit (ASB-STRSBU), batch normalization (BN), global average pooling (GAP) and a cross entropy error function.
In the DRSN method, due to the limitation of the soft threshold, using a soft threshold to reduce noise will filter the fault-related features in the signal, thus causing signal distortion and reducing the fault diagnosis accuracy. In contrast to the ReLU activation function, the soft thresholding (ST) function no longer zeros unwanted features, but zeros near-zero features, and soft thresholding acts as a contraction function to help eliminate noise-related information, thereby improving the accuracy of signal analysis.
In signal processing, a soft threshold is an important part of noise reduction [25]. The key feature of the RSBU is the use of ReLU's nonlinear transformation and soft threshold function (ST). The method uses a gradient descent based approach to efficiently extract meaningful information and process noise. Below is the soft threshold function (ST):
where 
and 
are the relevant features, and 
is the threshold of a positive parameter.
Different from the traditional DRSN, in each RSBU, a new soft thresholding (NST) module was used to replace the ReLU activation function in the process of feature normalization, nonlinear transformation and convolution operation to obtain the feature maps of each group. On this basis, a new method is proposed, that is, by solving for each parameter, more new eigenvalues of the original signal are obtained. Second, an adaptive slope block (ASB) was designed to automatically adjust the optimal thresholds.
As shown in figure 2, the DRSN architecture was introduced as a new network modularization system. Moreover, the feature gradient can be updated faster due to the constant shortcut; therefore, the cross entropy error formula is:
where 
and 
are feature maps, and 
is the number of classes and is the actual probability of observing the target output. After the above conclusions are obtained, the parameters are optimized. This method is applied to a noise reduction test of pipeline leakage. As shown in figure 3, the original signal is uniformly decomposed into IMF components of different main frequency bands.
3.Acoustic emission pipeline leak diagnosis framework
3.1.Transformer network model
The Transformer network model is an encoder-decoder architecture based on an attention mechanism, and the original Transformer model needs to use positional encoding to capture the positional information of the data points for the translation task, otherwise, the translation result is affected. The Decoder is utilized to decode and generate the encoded information. However, pipeline leakage detection typically involves multivariate time series data, and defects are detected by multivariate inputs for individual flow values. Therefore, to better meet pipeline leakage detection requirements the original Transformer model is improved by adding a GRU layer and feature encoding layer as well as a feature decoding layer, and using a fully connected layer for the result output.
The model adopts the multihead scaled dot-product attention (MHDPA) mechanism from the original transformer; the output results are obtained by calculating the correlation of the query with each key and weighting its weight with the corresponding value. The weight depends on the similarity of the query and value. The outputs of the attention of the scaling points are fused with features to make the final output, and the output of each attention point is called the head. Figure 4 shows the structure of the multihead scaling dot product attention mechanism.
(1)
Zoom point attention
(2)
Long scale points of attention
where 
and 
denote the number of attention heads, set 
and 
is the dimensions of the vector, and 
denotes the transpose of the matrix.
3.2.Feature encoder
The feature encoder is responsible for encoding the small defect parameters of the positionally encoded pipeline as features for transmission to the feature decoder. The feature encoder consists of multiple identical encoder layers, Each encoder layer consists of a multihead self-attention module and a feedforward neural network module, and residual connectivity [26] and layer normalization [27] are performed between the modules.
The multihead self-attention mechanism is the key defining feature of the Transformer model, and the underlying mechanism can be regarded as learning alignment, which is a variant of the attention mechanism, i.e., it reduces the reliance on external information relative to the attention mechanism and is more adept at capturing the relevance within the information layer, mainly through the calculation of weight coefficients to characterize the relevance between the features and thus to achieve the effect of conveying important information. The multihead self-attention mechanism computes multiple attention heads in parallel, enabling the model to model and represent inputs from different perspectives and concerns. This approach increases the expressiveness of the model and captures input information from multiple perspectives. For each attention header, the input matrix Z is mapped to the query matrix O, the critical matrix P, and the value matrix R by independent linear transformations for subsequent attention calculations and weighted summation operations.
The trainable weights are 
and 
We then obtain the attention score by calculating the dot product between O and P, adjusting the dimension of P to maintain the stability of the gradient. The results are then normalized using the softmax function.
The feedforward neural network model is a two-layer linear network consisting of two fully connected layers. The input data are processed by the weighting and activation function of the first layer and then passed to the second layer for further weighting and activation function processing, which ultimately results in an output.
where 
is the normalized output matrix of the attention layer, 
is the weight vector, and 
is the bias term.
The residual connection structure is located between each submodule of the feature encoder and feature decoder. After the introduction of the residual network, the change in weights is more sensitive to the output, and the weight matrix. can be better optimized. Moreover, to increase the training speed and reduce overfitting, the results after residual connection are normalized, and the calculation process is as follows:
where 
is the input matrix of the encoder layer, 
is the feed-forward neural network input matrix, 
is the output matrix of the encoder layer, and 
is the layer normalization processing function.
3.3.Feature decoder
The feature decoder is responsible for receiving the output signal from the feature encoder for feature decoding and passing it to the output layer. The feature decoder consists of multiple identical decoder layers, and each decoder layer in the original Transformer architecture consists of a multihead self-attention module, a multihead encoding-decoding attention module, and a feedforward neural network module, Residual connectivity and normalization are performed between the modules. Different from the original model architecture, the architecture proposed in this paper adds a pooling layer to the decoder layer, which is located after the feedforward neural network module.
The stacking of decoder layers in the feature decoder is circular stacking, and its operation volume is proportional to the number of decoder layers, which leads to the problems of slow computation speed and parameter redundancy in the calculation. Therefore, in this paper, the pooling layer commonly used in convolutional neural networks is introduced into the feature decoder, and the position is located after the normalization layer, forming a decoder model with fused convolution. This structure can effectively remove redundant information, reduce memory consumption, and reduce the risk of overfitting on the basis of retaining the original features. The pooling layer adopts maximal pooling, which is a method of selecting the neuron's value in a particular step matrix region during forward propagation. the point with the largest value in a specific step matrix region; at the same time, the position information of the point is recorded. The calculation process is as follows:
where 
denotes the maximum pooled output value in the rectangular region 
associated with the kth feature and 
represents the element at 
in the rectangular area 
The process of pipeline leakage fault diagnosis training and testing proposed in this paper is shown in figure 5. This process mainly consists of two steps: the first step is data preprocessing, where the leakage data are first subjected to noise reduction and standardization, and the leakage labels are subsequently added to the data, which are divided into training and test sets in accordance with the working conditions. The second part is based on the coupled GRU to improve the transformer network pipeline acoustic emission leakage detection process. First, the coupled GRU improved transformer network model is constructed. By training and adjusting the parameters on the training set, we build a model that can be used for fault diagnosis and achieve good performance; moreover, the test set data are used for online prediction to output the test results.
3.4.GRU neural network
LSTM uses these gating mechanisms to determine how the currently input information should be remembered and retained, and when and how to output the memorized information. In this way, the LSTM network can effectively handle and remember long-term dependencies while also having the ability to forget unnecessary information. The GRU neural network is an improvement of the LSTM neural network. The forget gate and input gate are combined into a single update gate, and the three gates in the LSTM network are reduced to two gates; thus, the processing of information can be achieved by updating and resetting gates, which reduces the training time [28].
This paper presents a gated recurrent until (GRU)-based network for solving the gradient problem present in recurrent neural networks. Specifically, the GRU network utilizes a sigmoid function to convert stored states and current input values to values between 0 and 1 to determine what information should be retained and what information should be discarded. This mechanism allows the GRU network to deal effectively with long-term dependencies and to better solve the gradient problem in recurrent neural networks, thus enabling data to be remembered and discarded. The structure of the GRU is shown in figure 6, where 
is the activation state of the implicit layer at the current moment, 
is the excitation function, 
is the state of the implicit layer at the current moment, 
is the state of the implicit layer at the previous moment, 
is the reset gate, and 
is the update gate.
According to figure 5, the GRU network contains only two gating mechanisms, a reset gate and an update gate. Both the reset gate and update gate are obtained by weighted summation of the inputs of the current moment with the implicit layer state of the previous moment, processed by a weight matrix and a sigmoid function. In this case, the update gate indicates the extent to which information from the previous moment flows into the current moment, and a value close to 1 indicates that more information was retained during the previous time. The expression for the update gate can be described as follows:
where 
and 
are the weights, 
is the input at the current moment, 
is the sigmoid excitation function, and 
is the input to the excitation function.
The reset gate determines whether some of the information from the previous moment should be discarded at the current moment. The expression for a reset gate can be simply described as:
where 
and 
are the weights, and 
is the input to the excitation function.
After obtaining the update gate and output gate, the reset gate is applied to the implicit layer state at the previous moment, the result is combined with the input information at the current moment and the weight matrix is introduced for the summation process. Finally, the result of the summation process is transformed by the tanh excitation function to obtain the activation state of the implicit layer at the current moment. The specific calculation formula can be succinctly expressed as:
where 
and 
are the weights and 
is the Hadamard product of the matrix.
By updating the gate to the last moment of the implied layer state and the current moment of the implied layer activation state at the same time, the two results are summed, to obtain the current moment of the implied layer state. The mathematical expression can be expressed as follows.
The GRU updates and adjusts the state of the hidden layer at the current moment by combining the current and past hidden states and determining the degree of information selection and retention through a gating mechanism. This mechanism allows the GRU unit to process sequential data more efficiently and capture long-term dependencies that simultaneously forget and retain the input information and the historical information; moreover, the GRU unit relies only on a gating unit to select the information to be memorized, save operating memory, improve operating efficiency, and effectively increase the speed of the model's calculations.
Based on the input of the current moment, the implicit layer state of the previous moment and the result of the gating mechanism, the loop body unit obtains the implicit layer output of the current moment through a series of operations and activation functions. After obtaining the implied layer output, the output of the whole neural network can be obtained by calculating through the output layer of the neural network. The calculation formula of the output layer can be succinctly expressed as follows:
where, 
is the weight, 
is the output result of the neural network and 
is the input of the excitation function.
3.5.The GRU improved transformer network model
The feature extraction of time series data by coupling the GRU layer in the input part of the improved Transformer network model involves the processing and transformation of the raw data to obtain a more meaningful representation or structure that makes the features and relationships of the data more explicit; the GRU layer makes full use of the current data features and uses its gate structure to decide whether to remember or forget the previous features to construct the improved Transformer network model as shown in figure 7.
4.Experimental data processing and model construction
The devices used for this experiment were a LegionY7000P2020 computer with a 64-bit Windows 10 operating system and an Intel(R) Core (TM) i7–10750H CPU @2.60 GHz 2.59 GHz Dual Processor and an NVIDIA GeForce GTX1650Ti 4 GB graphics card. The program caning environment was PyCharm2022. The leak detection system used an acoustic emission sensor, type RS-2A 54678. The technical specifications of this sensor were as follow: (1) diameter: 18.8 mm; (2) height: 15 mm; (3) interface: M5-KY; (4) housing: stainless steel; (5) detection surface: ceramic; (6) frequency range: 50–400 KHz; (7) centre frequency: 150 KHz; (8) temperature: −20 °C–130 °C. (9) Weight: 20 g. The data collector used is equipped with a Windows operating system, a built-in wireless communication module, a high-performance laser scanning engine, a high-speed CPU processor with strong anti-interference ability, long-term stability and other capabilities. The main parameters of this data collector are as follow : (1) analogue input voltage range: −10 V∼10 V; (2) input signal channel: 8 analogue inputs, synchronous sampling is required; (3) sampling frequency: 1 MHz; (4) USB is used to communicate with the host computer; and (5) A/D conversion accuracy: 16 bits.
4.1.Experimental setup
Building the acoustic emission pipeline dataset is an important part of the work analysis. In this paper, signals from different working conditions of an acoustic emission pipeline are detected, and acquiring high-quality AE signals and building a corresponding database are especially important. The acquisition test platform is shown in figure 8. The test was performed in a test room, which included an acoustic emission sensor, a data collector and a PC. With such a test platform, the AE signals can be acquired and recorded in real time to ensure high-quality data acquisition. These data can be used to construct a corresponding database for subsequent applications, such as signal analysis, feature extraction, pattern recognition and fault diagnosis. This study provides a reliable foundation for further research and applications.
4.2.Dataset collection studies
Acoustic emission sensors were installed at each of the front and terminal ends of the pipeline to simulate leakage detection of the pipeline using a valve control device. The length of the experimental pipeline is 2800 m, and a pump is used to inject water into the pipeline to simulate pipeline transport. Manual control valves were installed at 500 m, 1100 m and 1700 m of the pipeline to simulate the leakage of the pipeline by using manual control valves, and the degree of opening of the valve control device simulated the leakage level, The acoustic emission signals were sent to the PC through the data collector, and a number of experiments were carried out. Three different types of leakage were collected, and the numbers of samples of leakage signals collected on the test platform were as follows: 750 samples were collected when there was no leakage at 0 mm; 810 samples were collected when there was leakage at 8 mm; 730 samples were collected when there was leakage at 10 mm; and 860 samples were collected when there was leakage at 20 mm. Approximately 24,356 signal-related data points were obtained after analysis. To simulate the different leakage levels of the actual pipeline, as shown in figure 9, the test considers a leakage aperture size of 0 mm as no leakage, 8 mm and 10 mm as medium leakage, and 20 mm as large leakage.
4.3.Analysis of the experimental results
The NDRSN model can denoise the input AE signal by learning and modelling the characteristics of the noise. The noise components are identified and filtered out by adaptively adjusting the weights and parameters of the network to extract valid information from the original signal. Figures 10 and 11 show the AE signals processed by the NDRSN model after removing the noise and retaining the useful signal features.
By introducing an attention mechanism and a nonlinear transformation layer, the NDRSN method yields significant results in denoising. The processed image has better signal quality and different levels are better distinguished, which helps to improve the accuracy and correctness of the subsequent work recognition.
The mean error loss function during pretraining gradually stabilizes, while the classifier cross-entropy loss gradually decreases and stabilizes during the second stage of training. The pretraining phase is longer while the training of the classifier is shorter because the classifier does not require feature reconstruction. The classifier is trained after the pretraining is completed.
To verify the adaptive feature extraction capability of the model, the original high-dimensional data of the test set are mapped to a two-dimensional space using the TSNE dimensionality reduction algorithm, and a scatter plot is drawn as shown in figure 12, where the original signals are randomly dispersed on a two-dimensional plane, and the three types of leaks, large, medium, small and small, are dispersed from each other, not uniformly distributed, and difficult to be distinguish. In addition, the features extracted from the pooling layer of the encoder are mapped to the 2D space for visualization as shown in figure 13. After the feature extraction by the improved Transformer network model, the three different leaks distributed on the two-dimensional space feature clustering are very effective, and easily seen classificale features, prove the effectiveness of the model in the extraction, and the average accuracy rate reaches 93.97%.
4.4.Comparison of experimental data
4.4.1.Comparison of denoising methods
We performed a series of noise reduction experiments and compared the original signal, wavelet packet denoising, WOA-VMD denoising, and CEEMDAN-wavelet packet denoising methods. Figure 14 shows the effectiveness of these methods for noise reduction.
By comparison with the other methods, we can see that the denoising model proposed in this paper yields better results in the noise reduction task.
Since it is not possible to obtain a pure signal, we use the denoised signal to calculate the metric values of different threshold functions to evaluate their denoising effect. Table 1 and figure 15 show the comparison results of these metric values. These metric values provide a rough estimate of the relative performance of the different methods in terms of noise suppression.
Table 1.The index values of three the threshold functions after denoising.
| The index | Hard threshold | Soft threshold | NDRSN |
|---|---|---|---|
| SNR | 32.1529 | 30.2709 | 34.2536 |
| RMSE | 1.5122 | 2.8962 | 1.243 |
| r | 0.8977 | 0.5766 | 0.9635 |
| R | 0.8834 | 0.9076 | 0.9869 |
By comparing the hard thresholding method, soft thresholding method and method proposed in this paper, it can be found that this paper's method performs best in denoising. Compared to the hard thresholding method, this paper's method is able to remove the noise signal better and maintain the boundary features of the original image, thus achieving a more desirable noise reduction effect.
4.4.2.Comparison under different types of noise
By adding Gaussian noise to the original test case, we can simulate an actual noise environment and use the signal-to-noise ratio to quantify the relative strength of the signal to the noise. This allows for a better evaluation of the performance of the denoising method in real noise environments. The SNR is defined as follows:
where 
and 
represent the effective power of the signal and noise, respectively
Figure 16 shows the performance comparisons of the LSTM-RNN, WDCNN, ResNet, WOA-VMD and NDRSN model under different SNR conditions. By observing figure 15, we can understand the denoising effect of these methods under different noise levels and evaluate and compare their performances.
Based on the results in figure 15 and table 2, we can conclude that for LSTM-RNN and WDCNN, increasing the depth of the network does not significantly improve the model's immunity to interference. As the noise energy increases, the accuracy of fault diagnosis gradually decreases. In contrast, the WOA-VMD method can significantly improve the accuracy of fault diagnosis by identifying and removing the noise information in the input data, although it cannot completely eliminate the noise interference. This indicates that the WOA-VMD method yields better results in suppressing noise and can improve the accuracy of fault diagnosis.
Table 2.Gaussian white noise comparison.
| LSTM-RNN | WDCNN | ResNet | WOA-VMD | NDRSN | |
|---|---|---|---|---|---|
| −4 dB | 93.08 ± 6.59 | 87.23 ± 10.45 | 98.23 ± 0.69 | 99.52 ± 0.76 | 99.95 ± 0.12 |
| −2 dB | 96.32 ± 4.32 | 88.22 ± 10.25 | 98.56 ± 0.42 | 99.68 ± 0.56 | 100.00 ± 0.00 |
| 0 dB | 97.65 ± 3.36 | 88.96 ± 10.36 | 99.65 ± 0.12 | 99.75 ± 0.36 | 100.00 ± 0.00 |
| 2 dB | 98.56 ± 0.65 | 89.69 ± 10.74 | 99.70 ± 0.25 | 99.80 ± 0.12 | 100.00 ± 0.00 |
| 4 dB | 98.97 ± 0.45 | 90.21 ± 9.25 | 99.89 ± 0.32 | 99.89 ± 0.25 | 100.00 ± 0.00 |
The NDRSN method is not only capable of integrating the overall and local characteristics of the leakage signal, but also capable of automatically learning and extracting the criticality of the nodes. By learning in both the airspace and channel dimensions, the NDRSN is able to select features more accurately with better generalizability ability and robustness. Compared to other methods, the NDRSN achieves higher accuracy in fault diagnosis tasks.
4.4.3.Comparison of methods
To verify the advantages of the GRU network in improving transformer model performance, LSTM, the GRU, the ANN, the WOA-GRU, GRU the improvement in the transformer performance and the average accuracy and time consumption are compared.
By comparing the data in table 3, we can see the difference in the average accuracy and time consumption of these five models for different leaks. The experiments with the models classified leaks into three categories-no leaks, medium leaks, and large leaks and classified predictions for these categories. Table 3, shows how each model performs in terms of average accuracy for different leakage scenarios and the time consumption needed to complete the prediction. The multiclassification methods of the five models all adopt the 'one-to-one' strategy, and the tenfold cross-validation method is used to train and test the dataset. The accuracy of the four experimental results of the five models and the time consumed by the model are shown in figure 17.
Table 3.Average accuracy and time consumption of the five models.
| Model | Accuracy (%) | Time consumption |
|---|---|---|
| LSTM | 82.95 ± 1.50 | 101.65 |
| GRU | 82.05 ± 1.33 | 62.26 |
| ANN | 87.75 ± 1.26 | 41.36 |
| WOA-GRU | 88.67 ± 1.59 | 34.18 |
| GRU Improvement of Transformer | 93.97 ± 0.91 | 18.16 |
By looking at the data in figure 17 and table 3, we can conclude that the GRU Improved Transformer network model performs better in all four experiments. It achieved a significant improvement in accuracy, reaching 93.97%, and a reduction in time consumption, with an average time consumption of 18.16 s. In addition to the advantages in accuracy and time consumption, the GRU improved transformer model also demonstrated a smaller standard deviation. Considering these factors together, we can conclude that the GRU improved Transformer network model is better in terms of integrated performance. Therefore, based on the results of this paper, it is wise to select the GRU improved transformer network model as the pipeline leakage identification model.
5.Conclusion
In this paper, an improved Transformer network model is proposed and applied to pipeline leakage diagnosis. To improve the speed of leakage detection, the fluidization property of the GRU and the position coding of the Transformer are utilized to capture and feature extract the position information of the data point sequence, and the method can reduce the data redundancy and retain the feature information. An improved noise reduction model for Deep Residual Shrinkage Network (DRSN) is proposed, which uses a new threshold function that is better than the soft threshold function in the DRSN and is able to remove noise while retaining fault-sensitive features more efficiently, which enhances the performance of the DRSN. The ASB-STRSBU module designed in this paper can adaptively set the threshold value, which further enhances the performance of the DRSN. The experimental results show that the constructed improved Deep Residual Shrinkage Network can maintain the model's higher recognition accuracy and significantly reduce the accuracy of the model's time consumption. The improved Transformer network leakage detection method significantly improves the accuracy. By continuously eliminating the identified unfavor factors, the risk can be controlled within a reasonable and acceptable range, and the goal of safe, economic and reliable operation of pipelines can be achieved.
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China under Grant 61673199, in part by the Natural Science Foundation of Liaoning Province under Grant 2019-BS-158, in part by the Scientific Research Funds of Liaoning Provincial Department of Education under Grant L2020017, in part by the China Postdoctoral Science Foundation under Grant 2020M670796, in part by the Talent Scientific Research under Grant 2019XJJL-008, and in part by the Talent Scientific Research Fund of Liaoning Petrochemical University under Grant 2019XJJL-008. T, in part by the Natural Science Foundation of Liaoning Province under Grant 2023-MS-289. in part by the Basic Research Program of Liaoning Provincial Department of Education under Grant JYTMS20231441. in part by the Liaoning Revitalization Talents Program under Grant XLYC2203160.
Data availability statement
The data cannot be made publicly available upon publication because no suitable repository exists for hosting data in this field of study. The data that support the findings of this study are available upon reasonable request from the authors.
