Research Article Volume 4 Issue 2
Solution of the Hamilton jacobi bellman uncertainties by the interval version of adomian decomposition method
Navid Razmjooy,1
Regret for the inconvenience: we are taking measures to prevent fraudulent form submissions by extractors and page crawlers. Please type the correct Captcha word to see email ID.
Mehdi Ramezani2
1Department of Electrical Engineering, University of Tafresh, Iran
2Department of Mathematics and Control, University of Tafresh, Iran
Correspondence: Navid Razmjooy, Department of Electrical Engineering, University of Tafresh, Iran, Tel +98-914-4539067
Received: June 16, 2017 | Published: March 27, 2018
Citation: Razmjooy N, Ramezani M. Solution of the Hamilton jacobi bellman uncertainties by the interval version of adomian decomposition method. Int Rob Auto J. 2018;4(2):113-117. DOI: 10.15406/iratj.2018.04.00104
Download PDF
Abstract
The main purpose of interval arithmetic is look into the uncertain-ties in the practical models with uncertain-but-bounded parameters, which only requires lower and upper bounds of uncertain parameters and with no in-formation about probability distributions. In this paper, an interval numerical method is proposed for solving the Hamilton Jacobi Bellman (HJB) Equations with uncertainties. An interval version of Adomian Decomposition Method is utilized for solving the interval model parameters in the partial differential equations of the HJB. This study assumes that the uncertainty can be happened in both differential coefficients and initial values. The proposed interval method is applied to solve the linear and nonlinear HJB equations in association with appropriate numerical solvers. A practical discussion problem is also solved to analyze the system robustness and to show that the proposed interval decomposition method is a powerful method for systems in the presence of uncertainties.
Keywords: interval arithmetic, Hamilton Jacobi bellman, partial differential equation, interval adomian decomposition method, nonlinear differential equation, optimal control
Introduction
Optimal control is the policy of getting the optimized control value for minimizing a predefined cost function. Recently, several optimization methods have been introduced for achieving this purpose.1–4 Among these methods, Pontryagins maximum principle5 and the Hamilton Jacobi Bellman equation6 are the most popular. In Pontryagins maximum principle, the optimal control problem will be converted to an ODE problem while the Hamilton Jacobi Bellman method converts the optimal control into a nonlinear partial differential equation. Hamilton Jacobi Bellman method can be utilized in different linear, non-linear and even distributed optimal control problems. Be-cause of difficulty in solving the HJB and the Pontryagins maximum principal method, it is usually necessary to employ the numerical methods to achieve the optimal solutions for the nonlinear practical models. One of the popular semi-numerical methods which is frequently used in the recent years is the Adomian decomposition method.7 Adomian decomposition method and its modifications have been efficiently employed to solve the ordinary and partial differential equations.8–10 Because the method uses no linearization or smallness assumptions in solving the differential equations, it has been an effective method among the other techniques. Generally, in designing the optimal control problems for engineering and practical applications the parameters considered as deterministic, but there is a great deal of uncertain parameters which can greatly affect the system performance. Such an uncertainty can be made by model simplification, manufacture error, design tolerance etc. If the number of the system uncertainties becomes very small, deterministic methods can be employed to solve these problems with little errors. If the number of uncertain parameters has been increased or the ranges of these parameters have become large, the deterministic methods might give the wrong answer. Three different methods have been introduced for solving these uncertain problems: Probabilistic methods, fuzzy methods and interval methods.11 The main purpose of this paper is to introduce an interval version of Adomian decomposition method to solve the Hamilton Jacobi Bellman equation. The proposed method is first applied on a linear optimal control with uncertainties. After that, it is utilized to solve a nonlinear optimal control with uncertain parameters. Finally, a practical distributed system is solved by the proposed method; in this problem, in addition to assuming the presence of uncertainty in parameters, the initial conditions are also considered with uncertainty.
Interval arithmetic
This section describes the basic concepts of the interval arithmetic which is necessary to use the proposed method.12 Generally, basic interval arithmetic operations have described to guarantee the interval results reliability. let:
and
be interval numbers as: X, Y ∈ R,
and the symbol ⋄ illustrates the basic mathematical operations, i.e. addition , subtraction, multiplication and division for real numbers. In other words,
.
By assuming the assumption above, we have:
, (1)
where 0
Y for preventing the singularity in the division. We can easily prove that the set IR of real compact intervals is closed with respect to the illustrated operations.13 from the explanation above, the basic interval arithmetic operations between two interval numbers X and Y and can be defined as follows:
(2)
(3)
(4)
(5)
, if
,
(6)
If in mathematical operations, a real number like x gets into operations, it can be described as
and called degenerate interval ; in other words, the interval form for this constant number is X ≡ [x, x].
Description of the Hamilton-Jacobi-Bellman (HJB) formulation under interval uncertainty
In this section, a brief illustration of the optimal control and how to use the Hamilton-Jacobi-Bellman for solving these problems are described. Consider a state space representation system as below:
(7)
where, x(t) is the space vector, u(t) is the control signal and
is a coefficient with interval uncertainties. The main purpose is to find a control signal to minimize the following index performance:
(8)
Here, g(.)is an arbitrary convex function, tf is final time of system operation
is a coefficient with interval uncertainties and J = [J, J] is the lower and upper bounds of the optimized performance index. By supposing the dynamic programming approach, we have:14
(9)
Since,
(10)
By considering the principle of optimality
(11)
And with applying Taylor series, we have:
(12)
By assuming
, then
[16]. Therefore:
(13)
By applying the division operation into both sides, we obtain:
(14)
The previous nonlinear time-variant differential equation forms the HJB equation. By using the Hamiltonian function, we have:
(15)
By substitution the Hamiltonian function into the Eq. (14), the nal interval HGB equation can be describes as below:
(16)
Interval adomian decomposition method (IADM )
In this section, a brief description of the interval Adomian method for HGB equation is introduced. Suppose that Lt = ∂t∂ . Let consider the HGB equation in the following form:
(17)
From Eq.15, we obtain:
(18)
Substituting Eq.16 into Eq.17 yields:
(19)
Since, the main equation can be considered as:
(20)
where
(21)
(22)
Here, R (Vx) and N (Vx) are the linear and nonlinear terms of the Hamiltonian function, respectively and [α] and [β] are uncertain coefficients for linear and nonlinear terms, respectively. Using the Adomian decomposition method, the nonlinear term (N(Vx)), can be achieved by:
(23)
where, An = An(V0x(x), V1x(x), . . . , Vnx(x)) are the Adomian polynomials:
(24)
By applying the inverse operator
into the both sides of the Eq. 19
(25)
Since, by considering the given conditions, the final equation can be achieved by:
(26)
Where
illustrates the initial condition interval by the presence of uncertainty. We consider the truncated V as:
(27)
By using Equation (28), we can compute Vi as follows:
(28)
Illustrative examples
The proposed interval method is studied by three examples to describe the robustness of the approach.
Example 1
Linear LQR system: In the first example, consider a linear system which has equal uncertainty in both control and state terms. Find the optimal control based on minimizing the performance index in below:
(29)
Subject to
(30)
Where
. The problem without uncertainty is solved in (14). From Eq. (15), the HJB equation is achieved as follows:
(31)
From the Eq. (27):
(32)
Where,
. the closed form of this system for n=3 is shown as follows: Figure 1. A random value for
is applied in the interval [0.1, 0.9] and the solution is stand between two interval. The intervals are characterized by grids.
Example 2
Non-linear quadratic regulation system: In this example, a non-linear LQR system with uncertainty is studied. The target is to find the optimal control based on minimizing the performance index is:
(33)
subject to:
(34)
where δ = [1, 2]. The HJB equation for this problem can be obtained as follows:
(35)
From the Eq.(27):
(36)
So we can compute and plot the closed form Eq.(25) for n = 3 as follows: Figure 2. A random value for δ is applied in the interval [1, 2] and the solution stands between two intervals. The random value solution is characterized by stars.
Figure 1 The Interval Adomian Solution for the linear LQR system with uncertainty by HGB method.
Figure 2 The Interval Adomian Solution for the nonlinear LQR system with uncertainty by HGB method.
Example 3
In this example, a class of nonlinear diffusion equations subject to initial and boundary conditions is studied which arises during Magnetic Resonance Imaging (MRI). The main reason for prosperity in diffusion MRI is in the powerful concept that during their diffusion-driven, random displacements molecules probe tissue structure at a microscopic scale well beyond the usual image res-olution.15,16 Here, a class of diffusion problem is considered which arises in MRI frequently. The equation is given as below:
(37)
Where f(x),
and
are known functions with uncertainty parameters, K is a known constant and p(x) and q(x) are control variables. For generating a HGB problem, we can have considered the problem below:
(38)
For instance, consider the equation below:
(39)
From the equation above, it can be seen that the initial condition has also uncertainty in this problem. From the Eq.27, the closed form solution can be achieved by
(40)
A random value for [δ] is applied in the interval [0.8, 1.2] and the solution is stand between two intervals. The intervals are characterized by grids (Figure 3).
Figure 3 The Interval Adomian Solution for distributed diffusion problem with uncertainty by HGB method.
Conclusion
In this paper, an uncertain analysis method is proposed for solving the Hamiltonian-Jacobi-Bellman, for systems involving uncertain parameters. The Adomian decomposition method is applied to deal with the interval method to handle the interval uncertainty. Three case studies including linear, nonlinear and a distributed nonlinear optimal control are studied for checking the system robustness. The main advantage of the proposed method over traditional numerical methods is that the proposed method is the first time which is used the interval arithmetic to provide a robust result for HGB equation with uncertain coefficients.
Acknowledgements
Conflict of interest
The authors declare there is no conflict of interest.
References
- Dehghan M, Allah Hamedi E, Khosravian-Arab H. A numerical scheme for the solution of a class of fractional variational and optimal control problems using the modi ed Jacobi polynomials. Journal of Vibration and Control. 2016;22(6).
- Sabeh Z, Shamsi M Dehghan M. Distributed optimal control of the viscous Burgers equation via a Legendre pseudospectral approach, Mathematical Methods in the Applied Sciences. 2015;39(12):3350–3360.
- Razmjooy N, Ramezani M. Analytical Solution for Optimal Control by the Second kind Chebyshev Polynomials Expansion. Iranian Journal of Science and Technology. 2016.
- Lions, Louis J. Optimal control of systems governed by partial differential equations. Springer; 1971:170.
- Pontryagin, Semenovich L. Boltyanskii VG. Mathematical theory of optimal processes. JSTOR. 1965;19(89):159–161.
- Feldbaum AA. Optimal Control Systems. Mathematics in science and engineering. 1965.
- Cherruault Y, Adomian G, Abbaoui K, et al. Further remarks on convergence of decomposition method. Int J Biomed Comput. 1995;38(1):89–93.
- Wazwaz AM. Adomian decomposition for a reliable treatment of the EmdenFowler equation. Applied Mathematics and Computation. 2005;161(2):543–560.
- Yucheng L. Adomian decomposition method with orthogonal polynomials: Legendre polynomials. Mathematical and Computer Modelling. 49(5-6):1268–1273.
- Dehghan M, Hamidi A, Shakourifar M. The solution of coupled Burgers equations using AdomianPade technique. Applied Mathematics and Computation. 2007;189(2):1034–1047.
- Wu J, Zhang Y, Chen L, et al. Uncertain analysis of vehicle handling using interval method, International journal of vehicle design. 2011;56(2011):81–105.
- Tangaramvong S, Tin-Loi F, Yang C, et al. Interval analysis of nonlinear frames with uncertain connection properties, International Journal of Non-Linear Mechanics. 2016;86:83–95.
- Alefeld G, Gnter M. Interval analysis: theory and applications. Journal of computational and applied mathematics. 121(1-2):421–464.
- Fakharian A, HBMT, Davari A. Solving the Hamilton Jacobi Bellman equation using Adomian decomposition method., International Journal of Computer Mathematics, 2010;87:2769–2785.
- Nouri K, Garshasbia J. Damirchi, Application of Adomian decomposition method to solve a class of di usion problem arises during MRI. Mathematical Sciences. 2014;2:207–218.
- Frankowska H. Lower semi continuous solutions of Hamilton-Jacobi-Bellman equations. SIAM Journal on Control and Optimization. 1993;31:257–272.
©2018 Razmjooy, et al. This is an open access article distributed under the terms of the,
which
permits unrestricted use, distribution, and build upon your work non-commercially.