Volume 2022, Issue 1 6559868
Research Article
Open Access

Analysis and Research on Nonlinear Complex Function Approximation Problem Based on Deep Learning

Dapeng Zhou

Corresponding Author

Dapeng Zhou

School of Mathematics and Quantitative Economics, Shandong University of Finance and Economics, Jinan, Shandong 250014, China sdufe.edu.cn

Search for more papers by this author
First published: 2022
Citations: 3
Academic Editor: Sikandar Ali

Abstract

Shallow models have limited ability to express high-dimensional nonlinear complex functions. Based on deep learning, a Gaussian radial basis function neural network (RBFNN) is proposed, which is an analysis method of nonlinear complex function approximation based on the Gaussian-RBFNN model. The proposed method can approximate single-variable and binary complex nonlinear functions, and the approximation error is less than 0.1, which can achieve the ideal approximation effect. Finally, the proposed method is applied to analyze the nonlinear complex stock prediction approximation problem, and the effectiveness and practicality of the method are further verified. It can be seen that compared with the BP neural network model, this model performs better in average training time, training standard error, Ratt, RATR, and trmse For RBFNN; the training only needs 50 iterations, while the traditional BP needs 7100 training to gradually converge. In the actual function test, after 275 times of training, the convergence speed gradually tends to be stable, and the predicted value is closer to the actual value. Therefore, the model can be used to analyze practical nonlinear complex function approximation problems.

1. Introduction

With the development of intelligent technology and other high and new technologies, product functions gradually tend to be complex, resulting in the response time and parameters of the corresponding functions of products showing nonlinear and complex many-to-many characteristics. And the traditional shallow mathematical model cannot meet the modeling of the nonlinear system. For example, based on the BP neural network model and the network model with support vector machine (SVM), etc., there are only computing nodes of the hidden layer. And it can express the nonlinear simple mapping relationship, which can achieve ideal expression effect. However, when the training sample is small, the expression ability of the model has certain limitations [1, 2]. To solve this problem, some scholars try to improve the traditional shallow mathematical model to improve the approximation and expression of high-dimensional nonlinear complex functions. Xingqiang Chen and Yao Lu et al. used genetic algorithm (GA) to optimize BP neural network parameters, which realizes the improvement of the prediction accuracy of the model. Moreover, the validity of the model is proved by applying it in the examples of ship track prediction and CMT weld deformation prediction [3, 4]. Xu Ying et al. proposed an improved BP neural network method for beetle antenna search and verified the effectiveness of the method by applying it to predict multifactor gas explosion pressure. The results show that this method can improve the nonlinear systems modeling ability of BP neural network for complex many-to-many feature [5]. Razaque Abdul et al. combined radial basis function with SVM to propose a SVM-RBF and SVM-linear variant prediction method, which improved the reliability and fault tolerance of the SVM model, and made it have better approximation effect on nonlinear function problems [6]. To improve the classification accuracy of SVM model, Zhang Xiaoyan et al. proposed an improved SVM classification algorithm. The results show that this algorithm can solve the problem of low classification accuracy of SVM for nonlinear complex problems and provide a theoretical reference for the analysis of nonlinear complex function approximation problem [7]. Tian-bing Ma et al. used particle swarm optimization algorithm (PSO) to improve the SVM model. Features of each IMF component are extracted and fused; thus, the classification and recognition accuracy are improved [8]. Huang Wei, Dai Jianhua, etc. applied the nonlinear function to GDP prediction and achieved good results, which also laid the foundation for this study [913]. It provides a reference for the application of the traditional shallow mathematical model for the analysis of complex nonlinear problems. However, compared with the deep learning model, the improved shallow model still has some limitations in processing and analyzing nonlinear complex problems. Therefore, based on the RBFNN model, with Gaussian as kernel function, a Gaussian-RBFNN nonlinear complex function approximation method is proposed.

2. RBFNN Model

RBFNN is an artificial neural network which can approximate any nonlinear complex continuous function with any accuracy. It has the advantage of fast convergence speed and unique optimal approximation [14], which can solve the problem of local minimum. The basic structure of RBFNN is shown in Figure 1, which consists of input layer, hidden layer, and output layer.

Details are in the caption following the image
Structure diagram of RBFNN.

The common radial basis functions of RBFNN mainly include kernel functions, namely, Gaussian, multiquadric, and inverse multiquadric, and their radial basis function distribution is shown in Figure 2 [15]. It can be seen that, with the increase of center distance, multiquadric is a monotonically increasing function, while Gaussian and inverse multiquadric are monotonically decreasing functions. For monotonically decreasing function, when the variable is close to the function center, the function can obtain a large value; otherwise, its value tends to 0. Therefore, monotonically decreasing function has good local characteristics [16]. Thus, when RBFNN basis function is selected, monotonically decreasing function is usually used. Gaussian functions are characterized by simple form and good resolution [17]. Therefore, based on the advantages of Gaussian function, this paper uses it as the base function of RBFNN.

Details are in the caption following the image
Distribution of common radial basis functions.
The basis function of Gaussian-RBFNN model can be expressed as [18]
φ i x = exp X e i 2 2 σ i 2 , i = 1,2 , m , (1)
where φi(x) is the output of hidden layer node i and X = x 1 , x 2 , , x n T is the input sample. The Gaussian function center of node i is ei, and its variable is σi, namely, the base width.
The output of RBFNN is the linear combination of node output in the hidden layer [19]:
y k = i = 1 m w i k φ i x , k = 1,2 , , p , (2)
where w is the weight.

3. Nonlinear Complex Function Approximation Based on Gaussian-RBFNN

3.1. Multidimensional Gaussian-RBFNN Interpolation Approximation

The dataset is defined as (xi, fi), xi ∈ [a, b]k, fiR, i − 0,1, …, n. Multidimensional Gaussian-RBFNN is Wa(x, A), and accurate Gaussian-RBFNN is We(x, A). The only difference between the two is the external weights, which can be expressed as
W a x , A = j 0 n f j ϕ A x x j , W e x , A = j = 0 n k j ϕ A x x j . (3)

When A is large enough, Wa(x, A) can approximate We(x, A), and approximate dataset (xi, fi) ∈ RR+1, i = 0,1, …, n.

Theorem 1. When ij and xixj, dataset (xi, fi), xi ∈ [a, b]k, fiR, i = 0,1, …, n, can determine We(x, A) and Wa(x, A). For any ε > 0 and x ∈ [a, b]k, when a sufficiently large positive real variable A(A > A) is existing, there is

W e x , A W a x , A < ε . (4)

Prove that We(x, A) exists:

k f = k M k = I M k = I M k f + I M f , (5)
where k = k 0 , k 1 , k n T represents the external weight, I represents the identity matrix, and M represents the matrix. Therefore,
k f 1 I M 1 1 I M 1 f 1 , I M 1 = max 0 j n i j m i j . (6)

According to Lemma 1 [20, 21], when ϕ(0) = 1, ϕ(x) is a monotonically decreasing function on interval |0, +|, and there is

ϕ x < 1 , Due to Lim  m i j A = Lim A ϕ A x x j = 0 , i j . (7)

Therefore, in mij(ij), if ∀ε > 0, there is

ε = min 1 n + 1 ε + f 1 ε , 1 n + 1 . (8)

Thus, Aij > 0 is existing, and when A > Aij, there is

m i j < ε = min 1 n + 1 ε + f 1 ε , 1 n + 1 . (9)

Therefore, when A > A 1 = max i j A i j , i , j = 0,1 , , n , there is

I M 1 < n m i j < min 1 ε + f 1 , 1 . (10)

Considering that f(x) = x/1 − x is a monotonically increasing function on interval (−, 1) [22], there are

k f 1 I M 1 1 I M 1 f 1 < ε / ε + f 1 1 ε / ε + f 1 f 1 < ε , W e x , A W a x , A = i = 0 n k i f i ϕ A x x j k f 1 < ε . (11)

Therefore, it can be inferred that when ij and xixj, the dataset (xi, fi), xi ∈ [a, b]k, fiR, i = 0,1, …, n, determines Wa(x, A) and We(x, A). If ∀ε > 0, there is a sufficiently large positive real number A. When A > A, |fiWa(x, A)| < ε.

3.2. Gaussian-RBFNN Approximating Nonlinear Complex Function

Based on the above analysis, Gaussian-RBFNN is used to approximate the nonlinear complex function. The specific process is as follows:
  • (1)

    From 0 to 5, the nonlinear complex function is sampled every 0.1.

  • (2)

    Calculate the sample function value and establish the Gaussian-RBFNN model.

  • (3)

    The sample sets are divided into training sample set and testing sample set. The training sample set is adopted to train the model until the error is less than the set threshold or no longer changes, and the model is saved. After the training reaches the maximum number of iterations, if the error of the model is still greater than the set threshold or tends to decrease, the network structure of the model is adjusted, such as number of hidden layers and number of hidden layers’ neurons, and training continues until the requirements are met.

The newrb is used to construct a three-layer Gaussian-RBFNN. The used statements are as follows:
netl = newrb p , t ,goal,spread , net 2 = newff minmax p , n,1 , tansig , purelin , trainlm . (12)

In net1, p and t are input and output vectors, respectively, goal is the error square sum, and spread is the radial base walking constant. In net2, p is the input vector. Furthermore, n and tansig are the number of hidden layer nodes and transfer function, respectively, purelin is the transfer function of the output layer, and trainlm is the adopted learning algorithm.

4. Simulation Experiment

4.1. Construction of Experimental Environment

This experiment uses the neural network toolbox developed under MATLAB7.0 environment to establish the simulation experiment of nonlinear complex function approximation. The experiment is carried out on 64-bit Windows7 operating system with Centrino2D 2.1ghz CPU, Tesla K80 GPU, and 16G memory.

4.2. Data Sources and Preprocessing

In this experiment, single-variable nonlinear function y = 7sin3x+0.2 × 2+4.5, x ∈ [0,5], is selected for simulation, and its function image is shown in Figure 3 [23]. In addition, the binary nonlinear function is
f 2 x , y = x 2 + y 2 1020. cos 22 π x + cos π y + (13)
Details are in the caption following the image
Image of y = 7sin3x+0.2 × 2+4.5.

The nonlinear function is to be approximated.

4.3. Parameter Settings

Set the activation function of the proposed model as ϕ x = e x 2 .

Define S = {(x, y) : −1 ≤ x ≤ 1, −1 ≤ y ≤ 1}, and it satisfies
n 1 = 10 2 , A 100 = 8 , n 2 = 20 2 , A 400 = 20 , n 3 = 30 2 , A 900 = 40 , (14)
where A(n) is the time parameter of model training.

4.4. Evaluation Indexes

Average training time, training standard error, RATT, RATR, and TRMSE are also adopted to evaluate the model performance, and the calculation methods are as follows [2426]:
RATT = average training time of the proposed algorithm average training time of Gaussian RBFNN , RATR = average testing RMSE of the proposed algorithm average testing RMSE of Gaussian RBFNN , TRMSE = 1 n + 1 i = 0 n f x i , y i + W a x i , y i , A 2 . (15)

4.5. Experimental Results

4.5.1. Results of Single-Variable Nonlinear Function Approximation

(1) Convergence Speed. The training processes of the proposed model and the BP neural network model approximating single-variable nonlinear function y = 7sin3x+0.2 × 2+4.5 are shown in Figure 4. The proposed model only needs 50 steps to complete training. However, the BP neural network model needs 7100 steps. In addition, the accuracy is slightly higher than that of the BP neural network model. Thus, the proposed model has certain advantages, and its convergence speed of nonlinear functions approximation is better than the current BP network model.

Details are in the caption following the image
Training diagram of different models. (a) Training process of the proposed model. (b) Training process of the BP model.
Details are in the caption following the image
Training diagram of different models. (a) Training process of the proposed model. (b) Training process of the BP model.

(2) Approximation Ability. The comparison of errors in the training process of the proposed model and the BP model is shown in Figure 5, where the simulation error of the proposed model is close to 0, and the oscillation amplitude of the simulation error of the BP neural network model is significantly higher than that of the proposed model. Furthermore, the approximation accuracy of two models to training samples is 10−3. Thus, the proposed model has better approximation ability to training samples, almost reaching complete approximation.

Details are in the caption following the image
Comparison of approximation errors of different models to training samples.

The proposed model and the BP neural network model approximate nontraining samples, and the comparison of errors are shown in Figure 6. Compared with the BP neural network model, the proposed model has significantly better approximation ability to the nontraining samples, almost complete approximation. And the overall simulation error is less than 0.1. However, the BP neural network model has a large error fluctuation, and the maximum error is close to 0.5. This shows that the proposed model has better approximation ability to nontraining samples.

Details are in the caption following the image
Comparison of approximation errors of different models to nontraining samples.

The proposed model and the BP model are used to overall approximate y = 7sin3x+0.2 × 2+4.5, and the results are shown in Figure 7. Here, the proposed model is entirely consistent with y = 7sin3x+0.2 × 2+4.5. However, the BP model is inconsistent with some function segments of y = 7sin3x+0.2 × 2+4.5. Therefore, the overall approximation of the proposed model is better than that of the BP neural network model, and the overall effect is great.

Details are in the caption following the image
Comparison of overall approximation effects of different models.

(3) Network Structure. The number of hidden layer neurons is a main index to measure the complexity of the network structure. To compare the network structure of the proposed model and the BP neural network model, different training sample numbers are used to construct the proposed model and its comparison model, and the number of neurons of the proposed model and the comparison model is shown in Table 1. Here, under the same number of training samples, the number of hidden layer neurons of the proposed model is higher than that of the BP neural network model. It means that when training the same samples, the network structure of the proposed model is more complex than that of the BP neural network model.

Table 1. Comparison of parameter settings for different methods.
Sample number RBF neuron number BP neuron number
21 21 13
35 35 24
41 41 20
67 67 22
85 88 35

Therefore, the proposed model can well approximate the single-variable nonlinear function with high approximation accuracy, and the overall approximation error is less than 0.1.

4.5.2. Approximation Results of Binary Nonlinear Functions

Figure 8 shows the uniform approximation results of the proposed model to the binary nonlinear function f2(x, y) = x2 + y2 − 10(cos  2  πx + cos  2  πy) + 20. Figure 9 shows the relationship between A and n and TRMSE when the proposed model approximates f2(x, y) = x2 + y2 − 10(cos  2  πx + cos  2  πy) + 20. The proposed model has a good approximation effect for binary nonlinear complex function f2(x, y) = x2 + y2 − 10(cos  2  πx + cos  2  πy) + 20. With the increase of A and n, the model will uniformly converge to f2(x, y).

Details are in the caption following the image
Uniform approximation of the model to f2(x, y).
Details are in the caption following the image
The relationship between A and n and TRMSE when the model approximates f2(x, y).

Table 2 shows the comparison of approximation performance of the proposed model and BP neural network to binary nonlinear complex function f2(x, y) = x2 + y2 − 10(cos  2  πx + cos  2  πy) + 20. The performance of the proposed model is better than that of the BP neural network model on average training time, training standard error, RATT, RATR, and TRMSE, which indicates that the proposed model has better approximation performance.

Table 2. Performance comparison of different models.
Performance Training Test Parameter
Time RATT Time TRMSE Dev RATR n = 20
RBFNN 0.11875 1 0.115625 0.00375471 0 1 A = 40
BPNN 1.095313 9.223684 0.010938 0.07008864 0.04897677 18.66685 Node = 60

5. Application Analysis of the Gaussian-RBFNN Model

5.1. Data Source and Preprocessing

For the verification of the practical application effect of the proposed method, it is used to approximate stock price to predict the stock price. The Shanghai composite index of 300 consecutive trading days from April 1, 2018, to September 6, 2019, is selected as the experimental sample, the closing price is used as the prediction object, and the prediction period is 5 days [27]. In addition, datasets are divided into the training set and the testing set. There are 290 data in the training set and 10 data in the testing set.

To make the data samples meet the input and output requirements of the Gaussian-RBFNN model, before model prediction, premnmx and postmnmx functions in MATLAB7.0 are called to perform normalization and inverse normalization for the input and output models, respectively. The normalized data format of premnmx is shown in formula (16), and the reverse-normalized data format of postmnmx is shown in formula (17) [28, 29]:
p n , min p , max p , t π , min t , max t = premmmx P , T , (16)
P , T = postmnmx p n , min p , max p , t π , min t , max t , (17)
where P is the input data, minp and maxp are the minimum and maximum values of P, T is the output data, and mint and maxt are the minimum and maximum values of T.

5.2. Construction of the Stock Prediction Model

We use the newrb function in MATLAB7.0 to build the Gaussian-RBFNN model and train it automatically, and its format is shown as
Net = newrv p , t , goal , sperad , (18)
where p and t represent the input and output vectors of the model, respectively. With five days as a positive cycle, the Shanghai composite index from the first day to the fifth day is used to predict the Shanghai composite index of the sixth day, the Shanghai composite index from the second day to the sixth day is used to predict the Shanghai composite index of the seventh day, and so on. On the one hand, the closing prices of 226–279 trading days are constructed as a 5∗50 matrix, and the matrix is used as the input vector of model. On the other hand, the closing prices of 231–280 trading days are constructed as a 1∗50 matrix, and the matrix is used as the output vector of model. Then, y1 = sim(net, p1) is utilized for fitting prediction.

5.3. Results and Analysis

5.3.1. Analysis of Prediction Accuracy

Figure 10 shows the trend of training errors when the proposed model predicts stock prices. The training error of the proposed model is less than 0.1, which realizes the required prediction accuracy. The model completes the training through 275 steps, which means that the convergence speed of model is fast. Therefore, the prediction accuracy and convergence speed of the proposed model are effective.

Details are in the caption following the image
Trend diagram of the model training error.

5.3.2. Comparison of Prediction Effects

The effectiveness of the proposed method in practical application is further verified by comparing prediction results on the closing price of the stock exchange index for 50 trading days from June 29, 2018, to September 6, 2018. The results are shown in Figure 11. Compared with the comparison model, the predicted value of the proposed model is closer to the actual value, and the fitting effect of the predicted value curve and the actual value curve is better. Thus, the proposed model has higher prediction accuracy and better prediction effect.

Details are in the caption following the image
Comparison of the predicted and actual value of different models.

Figure 12 shows the comparison of absolute errors of the proposed model and BP neural network model predicting the closing price of Shanghai composite index in 50 trading days from June 29, 2020, to September 6, 2020. Here, the prediction absolute error of the proposed model has a small fluctuation range. There are about 72% absolute error values concentrated on interval [-20, 20], and the maximum error is less than ±40, which basically achieves the ideal prediction effect. However, the fluctuation range for absolute error of the BP neural network model is larger, which mainly focuses between [-40,-20] and [20,40], accounting for 48% of the total. In addition, there are a few absolute errors between [-60,-40] and [40,60], and one absolute error exceeds 80, which means that its prediction effect is poor. Thus, the proposed model has certain effectiveness and superiority in practical application.

Details are in the caption following the image
Comparison of prediction errors of different models.

6. Conclusion

To sum up, the proposed Gaussian-RBFNN model can approximate single-variable nonlinear complex functions and binary nonlinear complex functions, and the approximation effect is good. Compared with the BP neural network model, the proposed model performs better in average training time, training standard error, RATT, RATR, and TRMSE. In addition, the training speed of the proposed model is faster, and the approximation effect is better. The approximation error is less than 0.1, which can achieve the ideal approximation effect. In practical stock prediction application, the proposed model has good curve fitting effect of the predicted value and the actual value, and the training absolute error is less than 0.1, which can achieve the required prediction accuracy. Compared with the BP neural network model, the proposed model has certain advantages and precision in prediction accuracy and convergence speed. Moreover, a new idea and reference are provided for deep learning in the analysis of nonlinear complex function problems. As can be seen, some research achievements have been obtained in this paper, but because of the limitations of conditions, there are still some shortcomings to be improved. Here, only the Gaussian-RBFNN model is studied, and the prediction effect of the RBFNN model with other base function is not considered. Therefore, the next research is to find an optimal nonlinear complex function approximation method by deeply exploring this aspect.

Conflicts of Interest

The authors declare that they have no conflicts of interest regarding this work.

Data Availability

The experimental data used to support the findings of this study are available from the corresponding author upon request.

    The full text of this article hosted at iucr.org is unavailable due to technical difficulties.