Most of the direct methods solve optimal control problems with nonlinear programming solver. In this paper we propose a novel feedback control method for solving for solving affine control system, with quadratic cost functional, which makes use of only linear systems. This method is a numerical technique, which is based on the combination of Haar wavelet collocation method and successive Generalized Hamilton-Jacobi-Bellman equation. We formulate some new Haar wavelet operational matrices in order to manipulate Haar wavelet series. The proposed method has been applied to solve linear and nonlinear optimal control problems with infinite time horizon. The simulation results indicate that the accuracy of the control and cost can be improved by increasing the wavelet resolution.

1. Introduction

Optimal control is an important branch of mathematics and has been widely applied in a number of fields, including engineering, science, and economics. Although, the necessary and sufficient conditions for optimality have already been derived for H₂ and H_∞ optimal controls, they are only useful for finding analytical solutions for quite restricted cases. If we assume full-state knowledge, and if the optimal control problem is linear, then the optimal control is a linear feedback of the state, which is obtained by solving a matrix Riccati equation. However, if the system is nonlinear, then the optimal control is a state feedback function, which depends on the solution to a Hamilton-Jacobi-Bellman equation (HJB) or a Hamilton-Jacobi-Issac equation (HJI) for H₂ or H_∞ optimal control problem, respectively [1], and is usually difficult to solve analytically. Feng et al. [2] have solved an HJI equation iteratively by solving a sequence of HJB equation. In this paper, we are more concerned with approximate solution for HJB equation. Among numerous computational approach for solution of HJI equation, we refer in particular to [3–5]. Robustness of nonlinear state feedback is discussed in [6].

Broadly speaking, and in general, numerical methods for solving optimal control problem are divided into two categories: direct and indirect methods. The direct methods reduce optimal control problem to a nonlinear programming problem, by parameterizing or discretizing the infinite-dimensional optimal control problem, into finite-dimensional optimization problem. On the other hand, the indirect methods solve HJB equation or the first order necessary condition for optimality, which are obtained from Pontryagin minimum principle. Both these methods are important for solving optimal control problems; however, the difference between them is that the indirect methods are believed to yield more accurate result, whereas the direct methods tend to have better convergence properties. von Stryk and Bulirsch [7] have used both direct and indirect methods to solve optimal control problem for trajectory optimization in Apollo capsule. Beard et al. [8] have introduced Generalized Hamilton-Jacobi-Bellman equation to successively approximate solution of the HJB equation. Given an arbitrary stabilizing control law, their method can be used to improve the performance of the control. Moreover, Jaddu [9] has reported some numerical methods to solve unconstrained and constrained optimal control problems, by converting optimal control problems into quadratic programming problem. He has used a parameterization technique using the Chebyshev polynomials. Meanwhile, Beeler et al. [10] have performed a comparison study of five different methods for solving nonlinear control systems and studied the performance of the methods on several test problems. Park and Tsiotras [11] have proposed a successive wavelet collocation algorithm which used interpolating wavelets, to iteratively solve the Generalized Hamilton-Jacobi-Bellman equation and the corresponding optimal control law.

Wavelet basis that has compact support allows us to better represent functions with sharp spikes or edges than other bases. This property is advantageous in many applications in signal or image processing. In addition, the availability of fast transform makes it attractive as a computational tool. Numerical solutions of integral and differential equations have been discussed in many papers, which basically fall either in the class of spectral Galerkin and Collocation methods or finite element and finite difference methods.

Haar wavelet is the simplest orthogonal wavelet with a compact support. Chan and Hsiao [12] have used the Haar operational matrix method to solve lumped and distributed parameter systems. Hsiao and Wang [13] have solved optimal control of linear time-varying systems via Haar wavelets. Dai and Cochran Jr. [14] have considered a Haar wavelet technique to transform optimal control problems into nonlinear programming (NLP) parameters at collocation points. This NLP can be solved using nonlinear programming solver such as SNOPT.

In the present paper we have considered the method of Beard et al. [8] to successively approximate the solution of HJB equation. Instead of using the Galerkin method with polynomial basis, we have used collocation method with Haar wavelet basis to solve the Generalized Hamilton-Jacobi-Bellman equation. Galerkin method requires the computation of multidimensional integrals which makes the method impractical for higher order systems [15]. The main advantage of using collocation method in general is that computational burden of solving Generalized Hamilton-Jacobi-Bellman equation is reduced to matrix computation only. Our new successive Haar wavelet collocation method is used to solve linear and nonlinear optimal control problems. In the process of establishing the method we have to define new operational matrices of integration for a chosen stabilizing domain and new operational matrix for the product of two dimensions Haar wavelet functions.

2. Haar Wavelets

The orthogonal set of the Haar wavelets h_i(x) is a group of square wave over the interval x ∈ [τ₁, τ₂) defined as follows:

()

Other wavelets can be obtained by dilation and translation of the mother wavelet h₁(x). In general, h_i(x) = h₁(2^jx − k), where i = 2^j + k, j, k ∈ N ∪ {0}, and 0 ≤ k < 2^j.

Each f(x) ∈ L²([τ₁, τ₂)) can be expanded into Haar series of infinite terms:

()

If f(x) is approximated as piecewise constants then it can be decomposed as

()

where i = 2^j + k, j = 0,1, 2, …, log ₂m, and k = 0,1, 2, …, 2^j − 1.

The Haar coefficients that are

()

can be obtain by minimizing the integral square error

The sum in (3) can be compactly written in the form

()

where

is called the coefficient vector and h_m(x) = [h₀(x) h₁(x) ⋯ h_m−1(x)] ^T is the Haar function vector.

At collocation points x_j = (τ₁ + ((τ₂ − τ₁)/2m)(2j − 1)), j = 1,2, 3, …, m, the Haar function vector can be expressed in matrix form as

()

For instance, the fourth Haar wavelet matrix H₄ can be represented in matrix form as follows:

()

3. Haar Wavelet Operational Matrices

The integration of h_i(x) in the interval of [0, τ) can also be expanded into a Haar series, that is,

()

where the m × m matrix P_m is called the operational matrix of integration obtain recursively as

()

The formula in the interval of [0,1) was first given by Chen and Hsiao [12].

In order to solve nonlinear optimal control problem, it is essential to have the product of h(x) and h^T(x). The product of two functions f(x) = c^Th(x) and g(x) = d^Th(x) can be expanded into a Haar series with a Haar coefficient matrix M_m as

()

where M_m is an m × m matrix referred to as the product operational matrix. It was first given by Hsiao and Wu [16] as

()

where M₁ = c₀ and c_a = [c₀, …, c_m/2−1] ^T, c_b = [c_m/2, …, c_m−1] ^T.

Two-dimensional Haar wavelets basis can be formed by taking a tensor product of h_n(x) and h_m(x). Let the basis be {h_i(x₁)h_j(x₂)}, i = 1,2, …, n, j = 1,2, …, m. Then the two dimensions Haar function vector can be expressed as

()

Any function f ∈ L²([−τ₁, τ₁)×[−τ₂, τ₂)) can be written as

()

where C^T = [c₁₁ ⋯ c_1nc₂₁ ⋯ c_2n ⋯ c_m1 ⋯ c_mn]. Subsequently, we assume that n = m and τ₁ = τ₂ = τ, so that the operation matrix will be a square matrix. Let

where

is a m × m matrix. By using the Haar wavelet matrix in (6), the coefficient C^T in (13) can be obtained from

as follows:

()

and f_i,j = [f(x_i, x_j)], i, j = 1,2, …, m.

The integration of two dimensions Haar function vectors in [−τ, τ)×[−τ, τ) is

()

where Q_i and E_i for i = 1, 2 are the m² × m² operational matrices given as follows:

()

where ⊗ denotes the Kronecker product [17], I_m denotes m × m identity matrix, and

()

As in (10), we also required the product of H(x₁, x₂) and H^T(x₁, x₂). Let

()

The algorithm to obtain N_C is as follows.

Step 1. Let be a matrix of C, or equivalently .

Step 2. Compute , i = 1,2, …, m according to (11) using the column as the coefficient vector.

Step 3. For i = 1,2, …, m, compute .

Step 4. Form a big matrix by concatenating all vectors from Step 3; that is, .

Step 5. For each row k of matrix S, compute N_i,j according to (11) using the row S_k as the coefficient vector.

Step 6. Form the matrix as follows:

()

Step 7. End.

4. Problem Statement

The system to be controlled is given by the nonlinear differential equation of the form

()

where x(t) ∈ Ω ⊂ ℝⁿ is the state vector, u : Ω → ℝ^m is the control, f : Ω → ℝⁿ and g : Ω → ℝ^n×m are continuously differentiable with respect to all its arguments, x₀ is the initial condition vector, and Ω is domain of attraction.

The problem is to find the optimal control u^*(x) that minimizes the following performance index:

()

where Q ∈ ℝ^n×n is a positive semidefinite matrix and R ∈ ℝ^m×m is a positive definite matrix. Given an arbitrary control u, the performance of the control at x ∈ Ω ⊂ ℝⁿ is given by a Lyapunov function for the system [8]

()

where,

and l(x) = x^TQx. The optimal controller in feedback form is presented as follows [8]:

()

where V^*(x) is the solution to the following Hamilton-Jacobi-Bellman (HJB) equation

()

with boundary condition V^*(0) = 0; that is V(x^*, u^*) ≤ V(x, u) for all u, and x^*(t) is the solution of

. Basically, it is not so easy to solve the nonlinear partial differential equation in (24) for the purpose of obtaining V^*(x) and consequently u^*(x) from (23); rather the following two linear equations have been iterated by the algorithm proposed by [8]

()

with initial condition V⁽ⁱ⁾(0) = 0 and

()

Equation (25) is called the Generalized Hamilton-Jacobi-Bellman (GHJB) equation in [8]. In case of moderate presumptions, it has been established in [8] that the iteration between the GHJB (25) and the control (26) coincide with original HJB equation solution (24). If we can find a stabilizing control u⁽⁰⁾(x) to start off, it is possible to iteratively enhance the performance of this controller using (25), (26), and finally the optimal controller can be optimally approximated. Moreover, at each iteration step the controller u⁽ⁱ⁾ is a stable control.

5. The Successive Haar Wavelet Collocation Method

The following section describes the successive Haar wavelet collocation method (SHWCM) used for obtaining the two dimensional numerical solution to the HJB equation. In every step of this algorithm, an approximate solution to the GHJB equation (25) has been identified, namely, ∂V⁽ⁱ⁾/∂x, V⁽ⁱ⁾, and u⁽ⁱ⁾; all can be approximately expressed in term of Haar wavelets. As i → ∞, V⁽ⁱ⁾ and u⁽ⁱ⁾ will approach the optimal solution V^* and u^*, respectively.

Let us consider the following two-dimensional optimal feedback control problem

()

subject to the dynamics

()

where

, and u : Ω → ℝ.

Without loss of generality, the domain of attraction has been selected as Ω = [−τ, τ]×[−τ, τ] for the sake of convenience. The following equations express the pair of GHJB equation and the control law:

()

with initial condition V⁽ⁱ⁾(0) = 0 and

()

For (28), if initially u⁽⁰⁾ is a stabilizing control, then from (29) the solution to GHJB equation affiliated with u⁽⁰⁾ becomes a Lyapunov function for the system and equals to the cost associated with u⁽⁰⁾ as follows:

()

According to (13), function approximation for f₁(x) + g₁(x)u⁽⁰⁾(x), f₂(x) + g₂(x)u⁰(x) and x^TQx + u^(0)T(x)Ru⁽⁰⁾(x), can be written as

()

where the coefficient vectors, θ^T, μ^T, and k^T, can be calculate from (14). Since it is not possible to differentiate Haar functions, and as (29) only involves first-order derivatives of V, we assume that second-order partial derivative of V exists; that is,

()

for some coefficient vector ω.

With the assumption

()

the first-order partial derivative can be obtained by integrating (33), with respect to x₁ and x₂, respectively,

()

where

and

It should be noted that ω^T has m² unknown variables while

and

have only m unknown variables each. Now substituting (32) and (35) into (29), we have

()

Equation (36) is a system of underdetermined linear equations with m² equations and (m² + 2m) unknown variables which can solve for the unknown vectors ω^T,

, and

using Moore-Penrose pseudoinverse [18]. The underdetermined equation is expected because the Lyapunov function is not unique. The Moore-Penrose solution is the particular solution whose vector 2-norm is minimal.

By using the solution of GHJB equation (29), a feedback control law u⁽¹⁾ is constructed using (30), which improves the efficiency of u⁽⁰⁾. The solution of the Hamilton-Jacobi-Bellman equation is uniformly approximated by repeating the above process.

Knowing that

()

depends only on the initial and final points, not on the path followed, we can calculate the Lyapunov function V(x) by integrating parallel to the axes [19] as follows:

()

This gives

()

where

6. Numerical Examples

To show the efficiency of the proposed method, we applied our method to a linear quadratic optimal control problem and two nonlinear quadratic optimal control problems.

Example 1. Consider the following linear quadratic regulator (LQR):

()

subject to

()

To solve this problem we take the initial stabilizing control u⁽⁰⁾(x) = −x₁ − x₂. Tables 1 and 2 show sample iteration results for u⁽ⁱ⁾ and V⁽ⁱ⁾, respectively, when m = 8, x₁ = −1/8. The iteration is terminated when the difference between two successive controls is less than ϵ = 0.001. Subsequent, in order to display two-dimensional plots, we fix the value for x₁ at x₁[m/2] = −τ/m and x₂ ∈ [−1,1). Figure 1 shows that for the particular LQR problem, the usage of m = 16 is enough to approximate the exact optimal feedback control ; however, to approximate the exact cost function we require higher value of m as shown in Figure 2.

Table 1. Iteration results u⁽ⁱ⁾ for Example 1 when m = 8 and x₁ = −1/8.

x₂	u⁽⁰⁾	u⁽¹⁾	u⁽²⁾	u⁽³⁾	u⁽⁴⁾	u_exact
−7/8	1.0000	1.4463	1.3772	1.3786	1.3793	1.3624
−5/8	0.7500	1.0636	1.0114	1.0130	1.0136	1.0089
−3/8	0.5000	0.68889	0.6548	0.6548	0.6550	0.6553
−1/8	0.2500	0.3135	0.3027	0.3017	0.3015	0.3018
1/8	0	−0.0615	−0.0515	−0.0519	−0.0520	−0.0518
3/8	−0.2500	−0.4397	−0.4080	−0.4053	−0.4049	−0.4053
5/8	−0.5000	−0.8137	−0.7584	−0.7571	−0.7572	−0.7589
7/8	−0.7500	−1.1880	−1.1123	−1.1130	−1.1135	−1.1124

Table 2. Iteration results V⁽ⁱ⁾ for Example 1 when m = 8 and x₁ = −1/8.

x₂	V⁽⁰⁾	V⁽¹⁾	V⁽²⁾	V⁽³⁾	V_exact
−7/8	0.7051	0.6709	0.6712	0.6714	0.6618
−5/8	0.3914	0.3723	0.3722	0.3723	0.3654
−3/8	0.1723	0.1640	0.1637	0.1637	0.1574
−1/8	0.0470	0.0444	0.0442	0.0441	0.0377
1/8	0.0155	0.0130	0.0130	0.0130	0.0065
3/8	0.0781	0.0704	0.0701	0.0701	0.0636
5/8	0.2348	0.2162	0.2154	0.2153	0.2091
7/8	0.4850	0.4500	0.4492	0.4492	0.4431

Details are in the caption following the image — **Figure 1**
Open in figure viewer PowerPoint

Optimal feedback control for Example 1 via the SHWCM with m = 8,16 and x₁ = −0.1250, −0.0625, respectively.

Example 2. Consider the following nonlinear optimal control problem [15]:

()

subject to

()

The optimum solution for this problem is u^*(x) = −3x₂ and

. To solve this nonlinear optimal control problem, we started with the initial stabilizing control u⁽⁰⁾(x) = −1.8x₂. Figure 3 shows approximate optimal feedback control law u^* for m = 8,16, and 32. The graph for m = 64 overlaps with the exact optimal feedback control, and Figure 4 shows that the approximate cost function converges to the exact cost function as we increase the resolution. Figure 5 compares the exact state trajectories with approximate trajectories.

Example 3. Consider the following optimal control problem [8]:

()

subject to

()

The initial stabilizing control u⁽⁰⁾(x) = 0.4142x₁ − 1.3522x₂ can be obtained using feedback linearization method as outlined in [20]. The optimal feedback control and cost function obtained using SHWCM for various resolution m = 8, 16, and 32 are illustrated in Figures 6 and 7, respectively. We believe that, by increasing Haar wavelet resolution, the SHWCM will be capable of yielding more accurate results. Figure 8 shows simulation of the system trajectories.

7. Conclusion

In this paper we had proposed a new numerical method for solving the Hamilton-Jacobi-Bellman equation, which appears in the formulation of optimal control problems. Our approach uses a combination of successive Generalized Hamilton-Jacobi-Bellman equation and Haar wavelets operational matrix methods. The proposed approach is simple and stable and has been tested on linear and nonlinear optimal control problem in two-dimensional state space. Generally, by using our method, the approximate solutions for optimal feedback control require lower resolution, than the approximate solutions for the cost function. However, in both cases, it is clear that more accurate results can be obtained by increasing the resolution of Haar wavelet.

Acknowledgments

The authors are very grateful to the referees for their valuable comments and suggestions, which greatly improved the presentation of this paper. This research has been funded by University of Malaya, under Grant No. RG208-11AFR.

References

1 Beard R. W. and McLain T. W., Successive Galerkin approximation algorithms for nonlinear optimal and robust control, International Journal of Control. (1998) 71, no. 5, 717–743, 2-s2.0-0032202335.
10.1080/002071798221542
Web of Science® Google Scholar
2 Feng Y., Anderson B. D. O., and Rotkowitz M., A game theoretic algorithm to compute local stabilizing solutions to HJBI equations in nonlinear H_∞ control, Automatica. (2009) 45, no. 4, 881–888, 2-s2.0-61849156874, https://doi.org/10.1016/j.automatica.2008.11.006.
10.1016/j.automatica.2008.11.006
Web of Science® Google Scholar
3 Huang J. and Lin C.-F., Numerical approach to computing nonlinear H_∞ control laws, Journal of Guidance, Control, and Dynamics. (1995) 18, no. 5, 989–996, 2-s2.0-0029371239.
10.2514/3.21495
Web of Science® Google Scholar
4 Aliyu M. D. S., An approach for solving the Hamilton-Jacobi-Isaacs equation (HJIE) in nonlinear ℋ_∞ control, Automatica. (2003) 39, no. 5, 877–884, 2-s2.0-0037401851, https://doi.org/10.1016/S0005-1098(03)00025-6.
10.1016/S0005-1098(03)00025-6
Web of Science® Google Scholar
5 Abu-Khalaf M., Lewis F. L., and Huang J., Policy iterations on the Hamilton-Jacobi-Isaacs equation for H_∞ state feedback control with input saturation, IEEE Transactions on Automatic Control. (2006) 51, no. 12, 1989–1995, 2-s2.0-33845759425, https://doi.org/10.1109/TAC.2006.884959.
10.1109/TAC.2006.884959
Web of Science® Google Scholar
6 Glad S. T., Robustness of nonlinear state feedback—a survey, Automatica. (1987) 23, no. 4, 425–435, 2-s2.0-0023382820.
10.1016/0005-1098(87)90072-0
Web of Science® Google Scholar
7 von Stryk O. and Bulirsch R., Direct and indirect methods for trajectory optimization, Annals of Operations Research. (1992) 37, no. 1, 357–373, 2-s2.0-0001307871, https://doi.org/10.1007/BF02071065.
10.1007/BF02071065
Google Scholar
8 Beard R. W., Saridis G. N., and Wen J. T., Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation, Automatica. (1997) 33, no. 12, 2159–2176, 2-s2.0-0031332446.
10.1016/S0005-1098(97)00128-3
Web of Science® Google Scholar
9 Jaddu H. M., Numerical methods for solving optimal control problems using Chebyshev polynomials [Ph.D. thesis], 1998, School of Information Science, Japan Advanced Institute of Science and Technology.
Google Scholar
10 Beeler S. C., Tran H. T., and Banks H. T., Feedback control methodologies for nonlinear systems, Journal of Optimization Theory and Applications. (2000) 107, no. 1, 1–33, 2-s2.0-0034349664.
10.1023/A:1004607114958
Web of Science® Google Scholar
11 Park C. and Tsiotras P., Approximations to optimal feedback control using a successive wavelet collocation algorithm, 3, Proceedings of the American Control Conference, June 2003, 1950–1955, 2-s2.0-0142215883.
Google Scholar
12 Chen C. F. and Hsiao C. H., Haar wavelet method for solving lumped and distributed parameter systems, IEE Proceeding on Control Theory and Application. (1997) 144, no. 1, 87–94, https://doi.org/10.1049/ip-cta:19970702.
10.1049/ip-cta:19970702
Web of Science® Google Scholar
13 Hsiao C. H. and Wang W. J., Optimal control of linear time-varying systems via Haar wavelets, Journal of Optimization Theory and Applications. (1999) 103, no. 3, 641–655, 2-s2.0-0033269299.
10.1023/A:1021740209084
Web of Science® Google Scholar
14 Dai R. and CochranJ. E.Jr., Wavelet collocation method for optimal control problems, Journal of Optimization Theory and Applications. (2009) 143, no. 2, 265–278, 2-s2.0-70350173479, https://doi.org/10.1007/s10957-009-9565-9.
10.1007/s10957-009-9565-9
Web of Science® Google Scholar
15 Curtis J. W. and Beard R. W., Successive collocation: an approximation to optimal nonlinear control, 5, Proceeding of the American Control Conference, June 2001, 3481–3485, 2-s2.0-0034848079.
Google Scholar
16 Hsiao C. H. and Wu S. P., Numerical solution of time-varying functional differential equations via Haar wavelets, Applied Mathematics and Computation. (2007) 188, no. 1, 1049–1058, 2-s2.0-34248190325, https://doi.org/10.1016/j.amc.2006.10.070.
10.1016/j.amc.2006.10.070
Web of Science® Google Scholar
17 Brewer J. W., Kronecker products and matrix calculus in system theory, IEEE Transactions on Circuits and Systems. (1978) 25, no. 9, 772–781, https://doi.org/10.1109/TCS.1978.1084534, MR510703, ZBL0397.93009.
10.1109/TCS.1978.1084534
Web of Science® Google Scholar
18 Courrieu P., Fast computation of Moore-Penrose inverse matrices, Neural Information Processing-Letters and Reviews. (2005) 8, no. 2, 25–29.
Google Scholar
19 Slotine J.-J. and Li W., Applied Nonlinear Control, 1991, Prentice-Hall, Englewood Cliffs, NJ, USA.
Google Scholar
20 Isidori A., Nonlinear Control Systems, 1989, 2nd edition, Springer, New York, NY, USA, Communication and Control Engineering.
10.1007/978-3-662-02581-9
Google Scholar

Citing Literature

All articles

Feedback Control Method Using Haar Wavelet Operational Matrices for Solving Optimal Control Problems

Abstract

1. Introduction

2. Haar Wavelets

3. Haar Wavelet Operational Matrices

4. Problem Statement

5. The Successive Haar Wavelet Collocation Method

6. Numerical Examples

7. Conclusion

Acknowledgments

References

Citing Literature

Figures

References

Information

About Wiley Online Library

Help & Support

Opportunities

Connect with Wiley

Feedback Control Method Using Haar Wavelet Operational Matrices for Solving Optimal Control Problems

Abstract

1. Introduction

2. Haar Wavelets

3. Haar Wavelet Operational Matrices

4. Problem Statement

5. The Successive Haar Wavelet Collocation Method

6. Numerical Examples

7. Conclusion

Acknowledgments

References

Citing Literature

Figures

References

Related

Information