Volume 2011, Issue 1 161853
Research Article
Open Access

The Levenberg-Marquardt-Type Methods for a Kind of Vertical Complementarity Problem

Shou-qiang Du

Corresponding Author

Shou-qiang Du

College of Mathematics, Qingdao University, Qingdao 266071, China qdu.edu.cn

Search for more papers by this author
Yan Gao

Yan Gao

School of Management, University of Shanghai for Science and Technology, Shanghai 200093, China usst.edu.cn

Search for more papers by this author
First published: 01 December 2011
Citations: 3
Academic Editor: Chong Lin

Abstract

Two kinds of the Levenberg-Marquardt-type methods for the solution of vertical complementarity problem are introduced. The methods are based on a nonsmooth equation reformulation of the vertical complementarity problem for its solution. Local and global convergence results and some remarks about the two kinds of the Levenberg-Marquardt-type methods are also given. Finally, numerical experiments are reported.

1. Introduction

In this paper, we consider the following kind of vertical complementarity problem:
()
where the functions Fi(x) : RnRn are assumed to be continuously differentiable and denotes the jth component of the function Fi(x). The above vertical complementarity problem is of concrete background, for instance, the Karush-Kuhn-Tucker system of nonlinear programs, complementarity problems, and many problems in mechanics and engineering. When m = 2, (1.1) is the generalized complementarity problem, which has been considered in [1]. When x satisfied
()
x solves (1.1), where “min” denotes the componentwise minimum operator, , for, jJi, i = 1,2, …, n, are also continuously differentiable, and Ji, for i = 1,2, …, n, are finite index sets. Throughout this section, we denote
()
Thus, (1.2) can be briefly rewritten as
()
which is nonsmooth equation. Nonsmooth equations are much more difficult than smooth equations. Solving the above equations is a classical problem of numerical analysis, see [111]. Many existing classical methods for smooth equations cannot be extended to nonsmooth equations directly. Those difficult motivate us to seek higher-quality methods for nonsmooth equations. One of the classical iterative methods for solving (1.4) is the Levenberg-Marquardt-type method, which is based on the generalized Jacobian. The Levenberg-Marquardt method and its variants are of particular importance also because of their locally fast convergent rates. Levenberg-Marquardt method is also a famous method for nonlinear equations, which can be regarded as a modification of Newton method [1218]. In the smoothing-type methods, some conditions are needed to ensure that the Jacobian matrix is nonsingular; the Levenberg-Marquardt method does not need such conditions [1219].

By the above analysis, the purpose of this paper is to consider using the Levenberg-Marquardt-type methods for solving (1.1). We reformulate (1.1) as a system of nonsmooth equations and present two kinds of the Levenberg-Marquardt-type methods for solving it. The outline of the paper is as follows. In Section 2, we recall some background concepts and propositions which will play an important role in the subsequent analysis of convergence results. In Section 3, we give the two kinds of Levenberg-Marquardt-type methods and present their local and global convergence results and some remarks. Finally, numerical experiments and some final remarks about the numerical experiments are listed.

Throughout this paper, Rn denotes the space of n-dimensional real column vectors and T denotes transpose. For any differentiable function f : RnR, ∇f(x) denotes the gradient of f at x.

2. Preliminaries

A locally Lipschitz function H : RnRn is said to be semismooth at x provided that the following limit
()

exists for any hRn. The class of semismooth functions is very broad; it includes smooth functions, convex functions, and piecewise smooth functions. Moreover, the sums, differences, products, and composites of semismooth functions are also semismooth. Maximum of a finite number of smooth functions are semismooth too.

Proposition 2.1. Function G in (1.4) is semismooth.

Proposition 2.2. The following statements are equivalent:

  • (i)

    H(x) is semismooth at x.

  • (ii)

    For VH(x + h), h → 0.

()
  • (iii)

    One has

()
If for any VH(x + h), h → 0,
()
where 0 < p ≤ 1, then one says H(x) is p-order semismooth at x. Obviously, p-order semismoothness implies semismoothness. From [8], one knows that if H(x) is semismooth at x, then, for any h → 0
()

In [6], Gao considered a system of equations of max-type functions, which is the system of equations of smooth compositions of max-type functions. For solving the systems of equations, Gao take as a tool instead of the Clarke generalized Jacobian, B-differential, and b-differential. Based on [6], we give the following G(x) for G in (1.4):
()
Obviously, G(x) is a finite set of points and can be easily calculated by determining Ji(x), i = 1, …, n, and , jiJi(x), i = 1, …, n. In what follows, we use (2.6) as a tool instead of the Clarke generalized Jacobian and b-differential in the Levenberg-Marquardt method (I).

Proposition 2.3. The G(x) defined as (2.6) is nonempty and compact set for any x, and the point to set map is also upper semicontinuous.

Proposition 2.4. Suppose that G(x) and G(x) are defined by (1.4) and by (2.6), and all VG(x) are nonsingular. Then there exists a scalar β > 0 such that

()
holds for some constants γ > 0, ϵ > 0, and N(x, ϵ) is a neighbor of x.

By the continuously differentiable property of G in (1.1), the above Proposition 2.4 can be easily obtained.

Definition 2.5. One says that G is BD-regular at x if all elements in G(x) are nonsingular.

Definition 2.6. MRn×n is a P0 matrix if and only if there exists an index i ∈ {1,2, …, n} such that xi ≠ 0 and xi[Mx]i ≥ 0, for all xRn, x ≠ 0.

3. The Levenberg-Marquardt-type Methods and Their Convergence Results

In this section, we present two kinds of the Levenberg-Marquardt-type methods for solving vertical complementarity problem (1.1). Firstly, we briefly recall some results on the Levenberg-Marquardt-type method for the solution of nonsmooth equations and their convergence results, see, for example, [7] and also [9, 10]. And we also give the new kinds of the Levenberg-Marquardt methods for solving vertical complementarity problem (1.1) and analyze their convergence results. We are now in the position to consider exact and inexact versions of the Levenberg-Marquardt-type method.

Given a starting vector x0Rn, let
()
where dk is the solution of the system
()
In the inexact versions of this method, dk can be given by the solution of the system
()

where rk is the vector of residuals, and we can assume for some αk ≥ 0.

We now give a local convergence Levenberg-Marquardt-type method (I) for (1.1) as follows.

3.1. The Levenberg-Marquardt Method (I)

Step 1. Given x0, ϵ > 0, , .

Step 2. Solve the system to get dk,

()
for i = 1, ⋯n, and rk is the vector of residuals
()

Step 3. Set xk+1 = xk + dk; if ∥G(xk)∥≤ϵ, terminate. Otherwise, let k : = k + 1, and go to Step 2.

Based upon the above analysis, we give the following local convergence result. Similar results have also been mentioned in [9].

Theorem 3.1. Suppose that {xk} is a sequence generated by the above method and there exist constants a > 0, αka for all k, and there exist constant M > 0, . Let x be a BD-regular solution of G(x) = 0. Then the sequence {xk} converge Q-linearly to x for ∥x0x∥≤ϵ.

Theorem 3.2. Suppose that {xk} is a sequence generated by the above method and there exist constants a > 0, αka for all k. ∥rk∥ ≤ αkG(xk)∥, αk ≥ 0. And suppose that there exist constants M > 0, . Then the sequence {xk} converge Q-linearly to x for ∥x0x∥ ≤ ϵ.

By Propositions 2.2, 2.3, and 2.4, we can get the proof of Theorem 3.2 similarly to [9, Theorem 3.1], so we omit it.

Remark 3.3. Theorems 3.1 and 3.2 hold with ∥rk∥ = 0.

Remark 3.4. In the Levenberg-Marquardt method (I), if dk is computed by (3.3), Theorems 3.1, 3.2, and Remark 3.3 can also be obtained.

When m = 2 in (1.1), the vertical complementarity problem reduces to the well-known generalized complementarity problem(GCP) in [1]. The GCP is to find xRn satisfying
()
where Fi : RnRn, i = 1,2 and are continuously differentiable functions. If F1(x) = x, then the generalized complementarity problem (GCP) is the nonlinear complementarity problem (NCP). In the following, we give the Levenberg-Marquardt-type method (II) for the generalized complementarity problem  (GCP). The different merit function is based on the well-known F-B function
()
Then G(x) = φ(F1, F2), and we denote the corresponding merit function as
()

where Ψ is a continuously differentiable function. Now, we give the following Levenberg-Marquardt-type method (II).

3.2. The Levenberg-Marquardt Method (II)

Step 1. Given a staring vector x0Rn, ρ > 0, p > 2, β ∈ (0, 1/2), ϵ ≥ 0, , .

Step 2. If Ψ(xk) ≤ ϵ, stop.

Step 3. Select an element VkG(xk), and find an approximate solution dkRn of the system

()
where are the Levenberg-Marquardt parameters. If the condition
()
is not satisfied, set dk = −(Vk) TG(xk).

Step 4. Find the smallest ik ∈ {0,1, 2, ⋯} such that

()
Set , let k : = k + 1, and go to Step 2.

Notice that if Vk is nonsingular in (3.9), the choice of , at each step, is allowed by the above algorithm. Then (3.9) is equivalent to the generalized Newton equation in [4]. In what follows, as usual in analyzing the behavior of algorithms, we also assume that the above Levenberg-Marquardt method (II) produces an infinite sequence of points. Based upon the above analysis, we can get the following global convergence results about vertical complementarity problem (1.1). The main proof of the following theorem is similar to [7, Theorem 12]. But the system (3.9), which is used for the solution of dk, is different from [7, Theorem 12].

Theorem 3.5. Suppose that there exist constants M > 0, . Then each accumulation point of the sequence {xk} generated by the above Levenberg-Marquardt method (II) is a stationary point of Ψ.

Proof. Assume that {xk} Kx. If there are infinitely many kK such that dk = −∇Ψ(xk), then the assertion follows immediately from [5, Proposition 1.16]. Hence we can assume without loss of generality that if {xk} K is a convergent subsequence of {xk}, then dk is always given by (3.9). We show that for every convergent subsequence {xk} K for which

()
there holds
()
In the following, we assume that xkx. Suppose that x is not a stationary point of Ψ. By (3.9), we have
()
so
()
Note that the denominator in the above inequality is nonzero; otherwise by (3.14), we have ∥∇Ψ(xk)∥ = 0. xk would be a stationary point and the algorithm would have stopped. By assumption and Proposition 2.4, there exists a constant k1 > 0 such that
()
From the above inequality and (3.13), we get
()
Formula (3.13) now readily follows from the fact that we are assuming that the direction satisfies (3.10) with p > 2, while the gradient ∇Ψ(xk) is bounded on the convergent sequence {xk}. If (12) is not satisfied, there exists a subsequence of {xk} K
()
This implies, by (3.10), that . Together with (3.17) one implies
()
contradicting (3.12). The sequence {dk} is uniformly gradient related to {xk} according to the definition given in [5], and the assertion of the theorem also follows from [5, Proposition 1.16]. We completed the proof.

Remark 3.6. In the Levenberg-Marquardt method (II), (3.11) can be replaced by the following nonmonotone line search:

()
where m(0) = 0, m(k) = min {m(k − 1) + 1, M0}, and M0 is a nonnegative integer.

Remark 3.7. Let the assumptions of Theorem 3.5 hold. Equation (3.9) is replaced by (3.3). Then each accumulation point of the sequence {xk} generated by the above Levenberg-Marquardt method (II) is a stationary point of Ψ.

Remark 3.8. The assumption that there exist constants M > 0, in Theorems 3.1, 3.5, and Remark 3.7 can be easily obtained, for the continuously differentiable of F in (1.1).

Remark 3.9. Let x be a stationary point of Ψ, nonsingular, and a P0 matrix. Then x is a solution of GCP(F1, F2).

Remark 3.10. We can use φMS in [20] to construct merit function Ψ in the Levenberg-Marquardt method (II).

Remark 3.11. We can use a family of new NCP functions based on the Fischer-Burmeister function to construct merit function in the Levenberg-Marquardt method (II), which is defined by

()
where p is any fixed real number in the interval (1, +) and ∥(a, b)∥p denotes the p-norm of (a, b). Numerical results based on the function ϕp for the test problems from MCPLIB indicated that the algorithm has better performance in practice [21].

4. Numerical Tests and Final Remarks

In this section, in order to show the performance of the above Levenberg-Marquardt type methods, we present some numerical results for the Levenberg-Marquardt method (I) and the Levenberg-Marquardt method (II). The results indicate that the Levenberg-Marquardt algorithms work quite well in practice. We coded the algorithms in Matlab 7.0. Some remarks are also attached.

Example 4.1. We consider the following vertical complementarity problem:

()
where the functions
()
Both F1 and F2 are R2R2 continuously differentiable functions.

We use the Levenberg-Marquardt method (I) to compute Example 4.1. Results for Example 4.1 with initial point x0 = (0.1,0.7) T, αk ≡ 1 and λ1 = 0.01, λ2 = 0.01 are presented in Table 1.

Step xk G(x)
1 (0.1000000,0.7000000)T   (0.0100000,0.0050000)T
2 (0.0500998,0.7000000)T (0.0025099,0.0012549)T
3 (0.0250999,0.7000000)T 1.0e − 003*(0.630004,0.315002)T
4 (0.0125749,0.7000000)T 1.0e − 003*(0.158130,0.079065)T
5 (0.0063000,0.7000000)T 1.0e − 004*(0.396906,0.198453)T
6 (0.0031563,0.7000000)T 1.0e − 005*(0.996230,0.498115)T
7 (0.0015813,0.7000000)T 1.0e − 005*(0.250052,0.125026)T
8 (0.0007922,0.7000000)T 1.0e − 006*(0.627630,0.313815)T
9 (0.0003969,0.7000000)T 1.0e − 006*(0.157534,0.078767)T
10 (0.0001988,0.7000000)T 1.0e − 007*(0.395410,0.197705)T
11 (0.0000996,0.7000000)T 1.0e − 008*(0.992475,0.496237)T

We use the Levenberg-Marquardt method (I) to compute Example 4.1. Results for Example 4.1 with initial point x0 = (10,10) T, and x0 = (100,100) T, αk ≡ 1 and λ1 = 0.01, λ2 = 0.01 are presented in Table 2.

Step xk G(x)
x0 = (10,10) T
1 (10.000000,10.0000000)T (100.000000,25.000000)T
2 (5.009980,10.0000000)T (25.099900,12.549950)T
3 (2.509990,10.0000000)T (6.300049;3.150024)T
4 (1.257499,10.0000000)T (1.581306,0.7906530)T
5 (0.630004;10.0000000)T (0.396906,0.198453)T
6 (0.315631,10.0000000)T (0.099623,0.049811)T
7 (0.158130,10.0000000)T (0.025005,0.012502)T
8 (0.079223,10.0000000)T (0.006276, 0.003138)T
9 (0.039690,10.0000000)T (0.001575,0.000787)T
10 (0.019884,10.0000000)T 1.0e − 003*(0.395410,0.197705)T
11 (0.009962,10.0000000)T 1.0e − 004*(0.992475,0.496237)T
12 (0.004991,10.0000000)T 1.0e − 004*(0.249110,0.124555)T
13 (0.002500,10.0000000)T 1.0e − 005*(0.625264,0.312632)T
14 (0.001252,10.0000000)T 1.0e − 005*(0.156940,0.078470)T
15 (0.000627,10.0000000)T 1.0e − 006*(0.393919,0.196959)T
16 (0.000314,10.0000000)T 1.0e − 007*(0.988734,0.494367)T
17 (0.000157,10.0000000)T 1.0e − 007*(0.248171,0.124085)T
18 (0.0000789,10.0000000)T 1.0e − 008*(0.6229079,0.3114539)T
  
x0 = (100,100) T
21 1.0e + 002*(0.0000009,1.0000000)T 1.0e − 008*(0.985008,0.492504)T

When we compute dk by (3.3) in the Levenberg-Marquardt method (I) to compute Example 4.1. Results for Example 4.1 with initial point x0 = (0.1,0.7) T, x0 = (10,10) T, and x0 = (100,100) T, αk ≡ 1 and λ1 = 0.01, λ2 = 0.01 are presented in Table 3.

Step xk G(x)
x0 = (0.1,0.7) T
40 (0.007326,0.700000) T 1.0e − 004*(0.536774,0.268387) T
  
x0 = (10,10) T
47 (0.007301,10.000000) T 1.0e − 004*(0.533144,0.266572) T
  
x0 = (100,100) T
50 1.0e + 002*(0.000073,1.000000) T 1.0e − 004*(0.537728,0.268864) T

Remark 4.2. From the numerical results for the Levenberg-Marquardt method (I) in Table 1, Table 2, and Table 3, we can see that the modification of (3.4) in the Levenberg-Marquardt method (I) works quite better in practice than (3.3) which has been used in [7].

Example 4.3. We consider the following vertical complementarity problem:

()
where the functions
()
Both F1 and F2 are R2R2 continuously differentiable functions.

Now, we use the Levenberg-Marquardt method (II) to compute Example 4.3. For the special construction of Example 4.3, we can use (2.6) to get the Clarke generalized Jacobian in the Levenberg-Marquardt method (II). Results for Example 4.3 with initial point x0 = (1,1) T, x0 = (10,0.1) T, and x0 = (100,10) T and λ1 = 0.01, λ2 = 1 are presented in Table 4.

Step xk G(x)
x0 = (1,1) T
3 (−0.001250,1.000000) T 1.0e − 005*(0.156444,0.156444) T
  
x0 = (10,0.1) T
15 (0.099976,0.100000) T (0.009995,0.009995) T
  
x0 = (100,10) T
17 (0.096868,10.000000) T (0.009383,0.009383) T

Then we compute dk by (3.3) in the Levenberg-Marquardt method (II) to compute Example 4.3. Results for Example 4.3 with initial point x0 = (1,1) T, x0 = (10,0.1) T, and x0 = (100,10) T and λ1 = 0.01, λ2 = 1 are presented in Table 5.

Step xk G(x)
x0 = (1,1) T
3 (−0.001250,1.000000) T 1.0e − 005*(0.156444,0.156444) T
  
x0 = (10,0.1) T
15 (0.099989,0.100000) T (0.009997,0.009997) T
  
x0 = (100,10) T
17 (0.097467,10.000000) T (0.009499,0.009499) T

  

Remark 4.4. From the numerical results for in the Levenberg-Marquardt method (II) in Table 4 and Table 5, we can see that the modification of (3.9) in the Levenberg-Marquardt method (II) works quite as well as (3.3) which has been used in [7].

Acknowledgments

This work is supported by National Science Foundation of China (10971118, 11171221, 11101231), a project of Shandong province Higher Education Science and Technology Program (J10LA05), Shanghai Municipal Committee of Science and Technology (10550500800), Innovation Program of Shanghai Municipal Education Commission (10YZ99).

      The full text of this article hosted at iucr.org is unavailable due to technical difficulties.