The Levenberg-Marquardt-Type Methods for a Kind of Vertical Complementarity Problem
Abstract
Two kinds of the Levenberg-Marquardt-type methods for the solution of vertical complementarity problem are introduced. The methods are based on a nonsmooth equation reformulation of the vertical complementarity problem for its solution. Local and global convergence results and some remarks about the two kinds of the Levenberg-Marquardt-type methods are also given. Finally, numerical experiments are reported.
1. Introduction
By the above analysis, the purpose of this paper is to consider using the Levenberg-Marquardt-type methods for solving (1.1). We reformulate (1.1) as a system of nonsmooth equations and present two kinds of the Levenberg-Marquardt-type methods for solving it. The outline of the paper is as follows. In Section 2, we recall some background concepts and propositions which will play an important role in the subsequent analysis of convergence results. In Section 3, we give the two kinds of Levenberg-Marquardt-type methods and present their local and global convergence results and some remarks. Finally, numerical experiments and some final remarks about the numerical experiments are listed.
Throughout this paper, Rn denotes the space of n-dimensional real column vectors and T denotes transpose. For any differentiable function f : Rn → R, ∇f(x) denotes the gradient of f at x.
2. Preliminaries
exists for any h ∈ Rn. The class of semismooth functions is very broad; it includes smooth functions, convex functions, and piecewise smooth functions. Moreover, the sums, differences, products, and composites of semismooth functions are also semismooth. Maximum of a finite number of smooth functions are semismooth too.
Proposition 2.1. Function G in (1.4) is semismooth.
Proposition 2.2. The following statements are equivalent:
- (i)
H(x) is semismooth at x.
- (ii)
For V ∈ ∂H(x + h), h → 0.
- (iii)
One has
Proposition 2.3. The ∂⋆G(x) defined as (2.6) is nonempty and compact set for any x, and the point to set map is also upper semicontinuous.
Proposition 2.4. Suppose that G(x) and ∂⋆G(x) are defined by (1.4) and by (2.6), and all V ∈ ∂⋆G(x) are nonsingular. Then there exists a scalar β > 0 such that
By the continuously differentiable property of G in (1.1), the above Proposition 2.4 can be easily obtained.
Definition 2.5. One says that G is BD-regular at x if all elements in ∂G(x) are nonsingular.
Definition 2.6. M ∈ Rn×n is a P0 matrix if and only if there exists an index i ∈ {1,2, …, n} such that xi ≠ 0 and xi[Mx]i ≥ 0, for all x ∈ Rn, x ≠ 0.
3. The Levenberg-Marquardt-type Methods and Their Convergence Results
In this section, we present two kinds of the Levenberg-Marquardt-type methods for solving vertical complementarity problem (1.1). Firstly, we briefly recall some results on the Levenberg-Marquardt-type method for the solution of nonsmooth equations and their convergence results, see, for example, [7] and also [9, 10]. And we also give the new kinds of the Levenberg-Marquardt methods for solving vertical complementarity problem (1.1) and analyze their convergence results. We are now in the position to consider exact and inexact versions of the Levenberg-Marquardt-type method.
where rk is the vector of residuals, and we can assume for some αk ≥ 0.
We now give a local convergence Levenberg-Marquardt-type method (I) for (1.1) as follows.
3.1. The Levenberg-Marquardt Method (I)
Step 1. Given x0, ϵ > 0, , .
Step 2. Solve the system to get dk,
Step 3. Set xk+1 = xk + dk; if ∥G(xk)∥≤ϵ, terminate. Otherwise, let k : = k + 1, and go to Step 2.
Based upon the above analysis, we give the following local convergence result. Similar results have also been mentioned in [9].
Theorem 3.1. Suppose that {xk} is a sequence generated by the above method and there exist constants a > 0, αk ≤ a for all k, and there exist constant M > 0, . Let x⋆ be a BD-regular solution of G(x) = 0. Then the sequence {xk} converge Q-linearly to x⋆ for ∥x0 − x⋆∥≤ϵ.
Theorem 3.2. Suppose that {xk} is a sequence generated by the above method and there exist constants a > 0, αk ≤ a for all k. ∥rk∥ ≤ αk∥G(xk)∥, αk ≥ 0. And suppose that there exist constants M > 0, . Then the sequence {xk} converge Q-linearly to x⋆ for ∥x0 − x⋆∥ ≤ ϵ.
By Propositions 2.2, 2.3, and 2.4, we can get the proof of Theorem 3.2 similarly to [9, Theorem 3.1], so we omit it.
Remark 3.4. In the Levenberg-Marquardt method (I), if dk is computed by (3.3), Theorems 3.1, 3.2, and Remark 3.3 can also be obtained.
where Ψ is a continuously differentiable function. Now, we give the following Levenberg-Marquardt-type method (II).
3.2. The Levenberg-Marquardt Method (II)
Step 1. Given a staring vector x0 ∈ Rn, ρ > 0, p > 2, β ∈ (0, 1/2), ϵ ≥ 0, , .
Step 2. If Ψ(xk) ≤ ϵ, stop.
Step 3. Select an element Vk ∈ ∂G(xk), and find an approximate solution dk ∈ Rn of the system
Notice that if Vk is nonsingular in (3.9), the choice of , at each step, is allowed by the above algorithm. Then (3.9) is equivalent to the generalized Newton equation in [4]. In what follows, as usual in analyzing the behavior of algorithms, we also assume that the above Levenberg-Marquardt method (II) produces an infinite sequence of points. Based upon the above analysis, we can get the following global convergence results about vertical complementarity problem (1.1). The main proof of the following theorem is similar to [7, Theorem 12]. But the system (3.9), which is used for the solution of dk, is different from [7, Theorem 12].
Theorem 3.5. Suppose that there exist constants M > 0, . Then each accumulation point of the sequence {xk} generated by the above Levenberg-Marquardt method (II) is a stationary point of Ψ.
Proof. Assume that {xk} K → x⋆. If there are infinitely many k ∈ K such that dk = −∇Ψ(xk), then the assertion follows immediately from [5, Proposition 1.16]. Hence we can assume without loss of generality that if {xk} K is a convergent subsequence of {xk}, then dk is always given by (3.9). We show that for every convergent subsequence {xk} K for which
Remark 3.6. In the Levenberg-Marquardt method (II), (3.11) can be replaced by the following nonmonotone line search:
Remark 3.7. Let the assumptions of Theorem 3.5 hold. Equation (3.9) is replaced by (3.3). Then each accumulation point of the sequence {xk} generated by the above Levenberg-Marquardt method (II) is a stationary point of Ψ.
Remark 3.8. The assumption that there exist constants M > 0, in Theorems 3.1, 3.5, and Remark 3.7 can be easily obtained, for the continuously differentiable of F in (1.1).
Remark 3.9. Let x⋆ be a stationary point of Ψ, nonsingular, and a P0 matrix. Then x⋆ is a solution of GCP(F1, F2).
Remark 3.10. We can use φMS in [20] to construct merit function Ψ in the Levenberg-Marquardt method (II).
Remark 3.11. We can use a family of new NCP functions based on the Fischer-Burmeister function to construct merit function in the Levenberg-Marquardt method (II), which is defined by
4. Numerical Tests and Final Remarks
In this section, in order to show the performance of the above Levenberg-Marquardt type methods, we present some numerical results for the Levenberg-Marquardt method (I) and the Levenberg-Marquardt method (II). The results indicate that the Levenberg-Marquardt algorithms work quite well in practice. We coded the algorithms in Matlab 7.0. Some remarks are also attached.
Example 4.1. We consider the following vertical complementarity problem:
We use the Levenberg-Marquardt method (I) to compute Example 4.1. Results for Example 4.1 with initial point x0 = (0.1,0.7) T, αk ≡ 1 and λ1 = 0.01, λ2 = 0.01 are presented in Table 1.
Step | xk | G(x) |
---|---|---|
1 | (0.1000000,0.7000000)T | (0.0100000,0.0050000)T |
2 | (0.0500998,0.7000000)T | (0.0025099,0.0012549)T |
3 | (0.0250999,0.7000000)T | 1.0e − 003*(0.630004,0.315002)T |
4 | (0.0125749,0.7000000)T | 1.0e − 003*(0.158130,0.079065)T |
5 | (0.0063000,0.7000000)T | 1.0e − 004*(0.396906,0.198453)T |
6 | (0.0031563,0.7000000)T | 1.0e − 005*(0.996230,0.498115)T |
7 | (0.0015813,0.7000000)T | 1.0e − 005*(0.250052,0.125026)T |
8 | (0.0007922,0.7000000)T | 1.0e − 006*(0.627630,0.313815)T |
9 | (0.0003969,0.7000000)T | 1.0e − 006*(0.157534,0.078767)T |
10 | (0.0001988,0.7000000)T | 1.0e − 007*(0.395410,0.197705)T |
11 | (0.0000996,0.7000000)T | 1.0e − 008*(0.992475,0.496237)T |
We use the Levenberg-Marquardt method (I) to compute Example 4.1. Results for Example 4.1 with initial point x0 = (10,10) T, and x0 = (100,100) T, αk ≡ 1 and λ1 = 0.01, λ2 = 0.01 are presented in Table 2.
Step | xk | G(x) |
---|---|---|
x0 = (10,10) T | ||
1 | (10.000000,10.0000000)T | (100.000000,25.000000)T |
2 | (5.009980,10.0000000)T | (25.099900,12.549950)T |
3 | (2.509990,10.0000000)T | (6.300049;3.150024)T |
4 | (1.257499,10.0000000)T | (1.581306,0.7906530)T |
5 | (0.630004;10.0000000)T | (0.396906,0.198453)T |
6 | (0.315631,10.0000000)T | (0.099623,0.049811)T |
7 | (0.158130,10.0000000)T | (0.025005,0.012502)T |
8 | (0.079223,10.0000000)T | (0.006276, 0.003138)T |
9 | (0.039690,10.0000000)T | (0.001575,0.000787)T |
10 | (0.019884,10.0000000)T | 1.0e − 003*(0.395410,0.197705)T |
11 | (0.009962,10.0000000)T | 1.0e − 004*(0.992475,0.496237)T |
12 | (0.004991,10.0000000)T | 1.0e − 004*(0.249110,0.124555)T |
13 | (0.002500,10.0000000)T | 1.0e − 005*(0.625264,0.312632)T |
14 | (0.001252,10.0000000)T | 1.0e − 005*(0.156940,0.078470)T |
15 | (0.000627,10.0000000)T | 1.0e − 006*(0.393919,0.196959)T |
16 | (0.000314,10.0000000)T | 1.0e − 007*(0.988734,0.494367)T |
17 | (0.000157,10.0000000)T | 1.0e − 007*(0.248171,0.124085)T |
18 | (0.0000789,10.0000000)T | 1.0e − 008*(0.6229079,0.3114539)T |
x0 = (100,100) T | ||
21 | 1.0e + 002*(0.0000009,1.0000000)T | 1.0e − 008*(0.985008,0.492504)T |
When we compute dk by (3.3) in the Levenberg-Marquardt method (I) to compute Example 4.1. Results for Example 4.1 with initial point x0 = (0.1,0.7) T, x0 = (10,10) T, and x0 = (100,100) T, αk ≡ 1 and λ1 = 0.01, λ2 = 0.01 are presented in Table 3.
Step | xk | G(x) |
---|---|---|
x0 = (0.1,0.7) T | ||
40 | (0.007326,0.700000) T | 1.0e − 004*(0.536774,0.268387) T |
x0 = (10,10) T | ||
47 | (0.007301,10.000000) T | 1.0e − 004*(0.533144,0.266572) T |
x0 = (100,100) T | ||
50 | 1.0e + 002*(0.000073,1.000000) T | 1.0e − 004*(0.537728,0.268864) T |
Remark 4.2. From the numerical results for the Levenberg-Marquardt method (I) in Table 1, Table 2, and Table 3, we can see that the modification of (3.4) in the Levenberg-Marquardt method (I) works quite better in practice than (3.3) which has been used in [7].
Example 4.3. We consider the following vertical complementarity problem:
Now, we use the Levenberg-Marquardt method (II) to compute Example 4.3. For the special construction of Example 4.3, we can use (2.6) to get the Clarke generalized Jacobian in the Levenberg-Marquardt method (II). Results for Example 4.3 with initial point x0 = (1,1) T, x0 = (10,0.1) T, and x0 = (100,10) T and λ1 = 0.01, λ2 = 1 are presented in Table 4.
Step | xk | G(x) |
---|---|---|
x0 = (1,1) T | ||
3 | (−0.001250,1.000000) T | 1.0e − 005*(0.156444,0.156444) T |
x0 = (10,0.1) T | ||
15 | (0.099976,0.100000) T | (0.009995,0.009995) T |
x0 = (100,10) T | ||
17 | (0.096868,10.000000) T | (0.009383,0.009383) T |
Then we compute dk by (3.3) in the Levenberg-Marquardt method (II) to compute Example 4.3. Results for Example 4.3 with initial point x0 = (1,1) T, x0 = (10,0.1) T, and x0 = (100,10) T and λ1 = 0.01, λ2 = 1 are presented in Table 5.
Step | xk | G(x) |
---|---|---|
x0 = (1,1) T | ||
3 | (−0.001250,1.000000) T | 1.0e − 005*(0.156444,0.156444) T |
x0 = (10,0.1) T | ||
15 | (0.099989,0.100000) T | (0.009997,0.009997) T |
x0 = (100,10) T | ||
17 | (0.097467,10.000000) T | (0.009499,0.009499) T |
Acknowledgments
This work is supported by National Science Foundation of China (10971118, 11171221, 11101231), a project of Shandong province Higher Education Science and Technology Program (J10LA05), Shanghai Municipal Committee of Science and Technology (10550500800), Innovation Program of Shanghai Municipal Education Commission (10YZ99).