Optimal control of nonlinear systems

690 [461 G. N. T. Lack and M. Enns, “Optimal control trajectories with minimax objective functions by linear programming,” IEEE Trans. Auromaric Contr. (Short Papers), vol. AC-12, pp. 749-752. December 1967. [47] S. Blum and K. A. Fegley, “A quadraticprogramming solution of the minimum energycontrol problem.’’ IEEE Trans. Aurornarir Conrr. (Cormpondence).vol. AC-13. pp. 20G207, April 1968. [a]D. Tabak, “Optimization of nuclear reactor Fuel recycle via linear and quadratic programming,” IEEE Trans. ~Vucl.Sri., vol. NS-15. pp. @”I,February 1968. [49] E. Polak. “On the removal of illsonditioning effects in the computation of optimal controls.“ Automarica, vol. 5. pp. 607-514. 1969. [SO] D.Tabak.“Constrained linear nonzero-sum difference games.“ presented at the 2nd Ann. Southeastern Ssmp. on System Theory, Gainsville, Fla.. March 1970.

IEEE TRANSACTIONS ON AUTOMATIC CO~XROL. DECEMBER

1970

I901 A. V. Fiaccoand G. P. McCormick. .\‘onlinear Programming: Sequenrial L;nconsrrained :Minimization Techniques. New York: Wiley. 1968. 1911 H. P. Kunzi. H. G . Tzschach. and C.A. Zehnder. ,Vnmeriral.3~fe~lerbods of,l.iarhenrariral Oprinzization. Yew, York: Academic Press, 1968. 1911 J . D. Pearson. “On variable metric methods of minimization.” Comprrt. J.. vol. 1 I . pp. 171-178. May 1969. [93] J . B. Rosen. “The gradient projection methodfornonlinearprogramming,pt. I: h e a r constraints.’‘ S I A M J . Appl. Marl!.. vol. 8. pp. 181-?17. 1960. .!MI ”The gradient projectlon method for nonlinear programming, pt. I I : nonlinear constralnts.” S I A M J . Appl. .Math.. vol. 9. pp. 514-32. 1961.

--.

E. .\‘onlinear Discrete-Time Sysrems

[SI] J. B. Rosen, “Optimal control and convex pro,gamming,” University of Wisconsin, Ma&son, Tech. Rep. 547, February 1965. [52] J. B. Pearson and R. Sridhar, “A discrete optimal control problem.” IEEE Trans. Arrromaric Conrr.. vol. AC-11. pp. 171-174, April 1966. IS31 0.L. Mangasarianand S. Fromovitr,“A maximum principle in mathematical progamming.” in .Cfarhemarical Theory of Conrrol. A. V. Balakrishnan and L. W. Neustadt, M s . New York: Academic Press, 1967. [54] M. D.Canon, C. Cullum, and E. Polak, “Constrained minimization problems in finite dimensional spaces,” S I A M 1. Conrr.. vol. 4. pp. 52?%547. 1966. [55] D. Tabak. “An algorithm nonlinear for process stabilization and control.” IEEE Trans. Compur.. vol. C-19. pp. 487-492, June 1970. I561 G . Porcelli, D. Tabak, and K. A. Fegley. “Deslgn of digital controllers for nonlinear systems by mathematical programming.” 1968 W € S C O N Rec.. sess. 14 (Los Angeles. Calif.). 1571 J. B. Rosen, “Iterative solution of nonlinear optimal control problem%,” S1A.M J . Conrr., vol. 4, pp. 223-244. 1966. [SS] R. R. Meyer, “The solution of non-convex optimization problems by iterative convex programming,” Ph.D. dissertation, University of Wisconsin, Madison, 1968. [59: -, “The talidity o f a family of optimization methods,” Department of Computer Sciences. University of Wisconsin. Madison. Tech. Rep. 28, 1968. [60] D. Tabak,“Optimalcontrolof nonlinear discrete time systems by mathematical programming..” J . Franklin lnsr., vol. 289, pp. 111-1 19. 1970. F. Stochasric S?srems

[61] A. J . and K. A. Fegley, “Quadraticprogramming in the statistical design or sampled-datacontrol systems,” IEEE Truns.Automatic Conlr. (Short Papers), vol. AC-13, pp. 77-80, February 1968. [621 D, and w. D, Lochrie, ..Nonlinear and recursive optimal =tima. tion.” Proc. 6111Ann. Allerron Conf. Circuirs and System Theory, October 1968. 1631 R. W . Harrison andK. A. Fegley, “Identification oflinear systems using mathematical programming.” 1968 I V E S C O N Rer., sess. 14 (Los Angeles, Calif.). 1641 R. W. Hamson. “Identification of linear systems using mathematical programming.” M.S. thesis. University of Pennsylvania, Philadelphia, May 1968. [65] J. F. Bums, “A mathematical progamming identification system IMPISI.” Ph.D. dissertation. University of Pennsylvania, Philadelphia. June 1969. [66] J . Marowitz, “Optimum stochastic and deterministic control-s>-stemsynthesis using quadraticprogramming,”Ph.D.dissertation. UniversityofPennsylvania. Philadelphia, 1969. [67l D. lsaacs and W . Lochrie. ”Estimation and identification of nonlmear systems with constraints,” Pror. 7111Ann. Allerron C o d Cirruir and Sysrem Theor?, October 1969. G. Disrribrrted Parameter Systems

[68] Y. Sakawa.“Solution of an optimalcontrol problem in adistributed-parameter system,“ IEEE Trans. Aurornarir Contr., vol. AC-9. pp. 42W26. October 1964. [69] V . Lorchirachoonkul. “Optimal sampled-datacontrol of dlstrlbutec-parameter s).stems.” Ph.D. dissertation. Montana State University, Bozeman. March 1967. [70] V. Lorchirachoonkul and D. A. Pierre, “Optimal control ofmultivariable distributedparameter system throu&linear programming.” Prac. 1967 J.4CC(Philadelphia. Pal. pp. 702-710.

Optimal Control in Nonlinear Systems Abstraef-Conditions on the nature of nonlinearities in higherorder systems which permitdeterminationof analytical solutions for the Hamiltow Jacobi equations arising in the classical optimal control problem of quadratic cost function minimization are presented. The possible applications of the study are illustrated with an example from the field of aerospace systems, namely, optimom stabilization of a rigid body with variable inertia.

INTRODUCTION Many of the studies [1]-[6] in optimal control of nonlinear systems lead to so-called specific optimal control or suboptimal solutions due to a variety of mathematically convenient approximations made in the analysis such as power series expansion [3] representation of the solution of the Hamilton-Jacobi eauation. assumption of form of suboptimal control law [2], [6], and use of an equivalent linear time varying model for the nonlinear system [4l, [SI. Also, the generality of the class of nonlinear systems to which the methods are applicable differs from case to case. In his paper on optimum stabilization of a satellite, Kumar [l] obtained an optimal control law for a nonlinear problem, namely, Euler’s equation of rotation of a rigid body rotating about its center of mass, which turns out to be similar to the solution of the linear regulator problem. An investigation as to the reasons for the ease with which solutions are obtained in such particular nonlinear problems led to certain conditions on the nature of nonlinearities in general nth-order nonlinear systems and corresponding performance indices that permit analytic solutions of the HamiltonJacobipartial differential equations. The earlier paper [l] considers a problem that automatically satisfies these conditions (althoughno specific mention of this fact is made in the paper) and, therefore, can be a good example for the analysis presented in the sequel. However, a modified case, namely, optimum stabilization of a rigid body with variable inertia, has been considered as an example to illustrate the possible applications of this study. ANALYSIS

H . .Mathemarical Programming: Theory and Algorithms [71] W. I. Zangvill. Manlinear Programming: A UnifiedApproach. Englewood Cliffs, N.J.: PrenticPHall, 1969. [72] 0. L. Mangasarian. XonlinearProgramming. New York: McGraw-Hill. 1969 [73] J. Abadie. Ed., Manlinear Programming. New York: Wiley. 1967. [74] G. Hadley, .\’onlinear and Dynamic Programming. Reading, Mass.: AddisonWesley, 1964. [75] H. P. Kunzi. W. Krelle. and W . Oettli. A‘onlinear Programming. Waltham, Mass.: Blaisdell, 1966. [76] S . Vajda, Marhemriral Programming. Reading. Mass.: Addison-Wesley. 1961 [77] R. L. Graves and P. Wolfe.Eds.. Recenr Adcances in Marhematical Programming. New York: McGraw-Hill, 1963. [78] G. Zoutendijk, .Merho& ofFeasible Direcrions. New York: Elsevier. 1960. [79] G. B. Dantzig. LinearProgramming and Exteusions. Princeton. N.J.: Princeton University Press. 1963. [SO] G. Hadley, LinearProgramming. Reading, Mass.: Addison-Wesley. 1962. [SI] M. Simonnard. LinearProgramming. Englewood Cliffs. N.J.: Prentie-Hall, 1966. [82] S . I. Gass, Linear Programming, 3rd ed. New York: McGrawHill. 1969. [83] R. W. Llewellyn, Linear Programming. New York: Holt. Rinehart. and Winston. 1964. [84] J. C. G. Boot, Quadratic Programming. Amsterdam. Holland:North-Holland Publishing Company, 1964. 1851 R.J. Duflin, E. L. Peterson. and C. M. Zener, Geometric Programming. New York: Wiley. 1967. [86] D. J . Wilde and C. S. Beightler, Forrndurion ofOprimiration. Englewood Cliffs. N.J.: Prentice-Hall, 1967. [87l M. L. Balinski, “Integer programming: Methods. uses. computation.” .%la,ragemenr Sri., vol. 12.pp. 253-313. 1965. 1881 J. F. Benders, “Partitioning procedures for solving mixed-variables programming problems.’‘ .\%mer. .Math., vol. 4. pp. 238-252. 1962. [89] M. W. Kuhn and A. W. Tucker, “Nonlinear programming.“ Proc. 2nd Berkeley S w n p . on Mathemarical Srarisrics and Probability. Berkeley, Calif.: University of California Press. 3951. pp. 481492.

Nonlinear Optimal Control Problem Formulation Consider a general nonlinear system described by a vector differential equation of the form i = F(x, t)

+ Bu

(la)

where x is an n vector and F(x, t) is an n-vector function that (without loss of generality) can be written as

F(x, t ) = a h

+ G(x, t)

(1b)

where a is ascalar and G(x,r) is a general nonlinear vector function. (When G(x, t) = ( A - aI)x,(1) assumes the general form of representation of a linear system.) B is an n x r (variable or constant) matrix and u is an r vector. Let the quadratic performance index to be extremized be of the form

Although this general optimal control problem cannot be analytically solved in all cases, imposition of certain conditions on the natureof nonlinearity in the system, i.e., form of the vector function G(x, t ) . permits Manuscript received November IO, 1969; revised March 26, 1970.

69 1

CORRESPONDENCE

analytical solution of the optimal control problem and the associated Hamilton-Jacobi partial differential equation. These conditions are stated in the form of a lemma.

Making a linear transformation,

I,w,

= x,

I2WY= x,

Lemma

13w, = x;

The sufficient conditions for a closed-loop control law of the form (3)

u(x. t ) = K,(t)x(t)

Equations (9aH9c) can be rewritten as

to be optimal (i.e., to be the solution of the optimal problem defined by (1) and (2)) are 1) x T G ( x , t ) = 0 2) Q and BR-'BT are scaled identity matrices.

Proof: Formulation of the associated Hamilton-Jacobi equation for the optimal control problem considered through standard optimization techniques [7] gives

SI.'

- + VV'[QIX dt

+ G ( s , t ) ] + tx'Qx

- fVV'BR-'B'VV

=0

and let the performance index (PI) be

where V ( x ,t ) is the minimum value of the performance index. When G(x, t ) = ( A - a l ) x , i.e., for a general linear optimal control problem, the solution to (4) can be written as

-/'* [c,(t)f' 1

(4)

PI =

2

.X:

10

,=I

f:

+ C2(t)

ui] dt.

(12)

i=1

Now ( 1 l a H l l c ) are in the form of (la) (along with (lb)). The condition 1) x'G(x, t ) = 0 is satisfied provided

p1 = a l l - i, p 2 = - i, ps = a13 - i,. with IC($) be

=

0 as the boundary condition, and the optimal controllaw will

u(x,t ) = KTx(t) where K , = - K B R - ' . For the nonlinear case, substitution of ( 5 ) in (4) leads to

X'AX + 2 x T K [ a I x + G(x, t)] + X'QX

-

x~KBR-~B'Kx= 0.

1) ~ ' G ( x , l )= 0 2) Q and BR-IB' are scaled identity matrices of the form

Q

=

q(t)I

t)= -

k,(t)

+ C , ( t )- (k$C,(t)) + 2a(t)k, = 0

(15)

ACKNOWL.EDGMENT The author wishes to thank Dr. A. D. Bond, Computer Sciences Corporation, for his constructive comments.

R- 'B'k(t)x(t) = K,(t)x(t).

Example An example of optimum stabilization of a rigid body writh variable inertia rotating about its center of mass is considered. The Euler equations of motion for a rigid body (rotating about its center of mass) with viscous damping and variable inertia can be written as

+ p l w x + u1 I,\;:? = ( I3 - I1)WZ\V~ + p2wv+ u2 13*= = (I' - I,)W,W, + p3wz + ug.

( 14)

CONCLUSIONS

The problem of optimum stabilization of a satellite solved by Kumar [l], [2] satisfies these conditions (although a specific mention of this fact is not made) and thus permits the analytical determination of the optimal control law. A modified version of the problem is solved as an example to show the possible application of the technique.

I,\bL., = ( I , - 13)wzw,

kn(t)xi:C2(t)

This correspondenceprovides an insight into the natureof nonlinearities in systems that admit analytical solutions to the Hamilton-Jacobi partial differentialequations representing theoptimal controlproblemofquadratic cost function minimization. For a given nonlinear system and performance index. it is sufficient to check for the conditions obtained in the preceding analysis that, if satisfied, lead to a closed-loop optimal controllaw expressed in terms of the solution of a relatively simple scalar Riccati equation. This study is expected to be ofinterest in view of the very fewpublished works' of this nature.

= p(t)l

and the closed-loop optimal control law is given by U(X,

-

where k,(t) is the solution of scalar Riccati equation

(8)

for G(x, t ) # 0. The sufficient conditions for (8) to be satisfied are a) K should be a scaled identity matrix such that K = k(t)l, where k(t) is an unspecified scalar time function, and b) xrG(x, t ) = 0. Condition a), when combined with (7).necessitates that Q and B R - ' B T also be scaled identity matrices. Hence effectively the sufficient conditions for thecontrol law u ( x , f ) (3) to be optimal are

BR- ' E r

u ~ ( xt,) =

(7)

For (7) to be variable separable (for an analytic solutionto be feasible),the nonlinear term x'KG(x, t ) should vanish, i.e., x T K G ( x ,t ) = 0

(This ineffect represents a supplementary control similar to the main control law and can, in fact, be combined with the otherin implementation. Such a scheme would correspond to a modified performance index [SI. In the earlier study [I], i, and z were zero.) The condition(2) is satisfied as Q,B, and R are all scaled identity matrices. Therefore. the optimal control law is of the form

BELURV. DASARATHY Computer Sciences Corp. Huntsville. Ala. REFERENCES [ I ] K. S .

P. Kumar. '-On the optimum stabilization of a satellite." IEEE

Trans. Aerosp.

EIPcfron. S p r . , "01. AES-I, pp. 82-83, October 1965.

12: K.S . P. Kumar and L. T e n s "Stabilization of a satellite via specific optimum control," I€€€ Trans. Aerosp. Electron. S p r . . 1.01. AES-2, pp. 46-449. July 1966. :3] W . L. Garrard, N. H. McClamroch, and L. G. Clark, "An approach to sub-optimal feedback control of nonlinear systems," Int. J . Conrr.. vol. 5 , pp. 4 2 5 4 3 5 , November 1967.

(94 (9b) (94

' The author wishes to point out that essent~allysimilar results obtalned throu& the Lagrangemultiplier technique and costate equations(un1ike theHamilton-Jacobiapproach used here) appeared after the submission of this correspondence in 191.

692

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, DECENBER

i4' . . J . D.P~rson:~Auuroximationmethodsinontimalcontrol." J . Elecrron. Conrr.,vol. 13.

pp. 45349.~196i.' 13: J . H. Burghart. "4 suboptimal feedback controller for nonlinear systems." IEEE Tram Aitrurnaric Cunrr. (Short Papers), vol. AC-14, pp. 53&533, October 1969. [6: A. G . Lon-muir and E. V. Bobn. "The synthesis of sub-optimalfeedback control laws." / E € € Trans. Auromaric Conrr. (Short Papers), vol. AC-12, pp. 755-758, December 1967. [T A. P.Sage, Oprinlurn S~vsternsConrrol. Englewood Cliffs, N.J.: Prentice-Hall, 1968. [SI B. V. Dasarathy, "On optimal stabilization of a rigid bod> with variable inertia." presented at the 13th hlidu*est Symp. Circuit Theory. May 1970. I91 hl. Sobral. Jr.. and 1.V . Bofii. "On the optimization of a certam claw of nonlineal systems," /€E€ Trans. Aurunlatic Conrr. (Correspondence). vol. AC-IS, pp. I1 1 - 1 12. Februar). 1970. .-

1970

This theorem gives a simple method for stabilizing a linear constant system. W ( T )is easy to compute by series summation and 7 = arbitrary may be chosen so that W(7)is well behaved numerically.

DAVID L. KLEXWAN Bolt. Beranek. and Newman. Inc. Cambridge. Mass. REFERENCES [I]D. L. Kleinman, "On an iterative technique for Riccati equationcomputations," IEEE Trans. Automaric Conrr. (Correspondence). vol. AC-13. pp. 1 1 4 - 1 15. February

1968. . Wonham. "On matrix quadratic equations and matrix Riccati equations," Center for Dynamical Systems, Brown University, Providence, R.I., Te&. Rep. TR67-5. 1967. 131 R. E. Kalman, Y . C. Ho, and K. S. Narendra, "Controllability of linear dynamical systems," in Conrriburions I O Dlferenrial Eqrrarions, vol. I , J. P. Lasalle and J.B. Diaz, Eds. New York: Interscience, 1963. [4] W . Hahn, T1ieor.v and Applicarion of Liapunor's Direcr Merlrod. Engiewood ClikTs, N.J.: Prentice Hall, 1963.

[Z] W . M

An Easy Way to Stabilize a Linear Constant System Abstracr-A constructive proof is given for hding constant feedback gains that stabilize a linear time-invariant controllable system. It is not necessary to transform variables or to specify polelocations. It is often desirable to find a control law of the form u(t) =

-Lx(t)

(1)

On Systems Described by the Companion Matrix

that stabilizes the linear constant controllable system i ( t ) = Ax(t)

+ Bu(t)

(2)

without having to transform A to a canonical form and without regard to explicit closed-loop pole assignment. Such situations exist in iterative methods for solving matrix quadraticequations [l]. [Z]. where any stabilizing control (1) suffices to initialize an algorithm. The object of this correspondence is to present a constructive method for easily finding a set of stabilizing feedback gains. The result is as follows.

Abstract-It is shown that general properties of analytical fnnctions of a l y system matrix (including the transition matrix) follow ~ t ~ d from consideration of the resolvent matrix and the Cauchy-integral formula. Previouslypublished results are extended to cover arbitrary analytical functions and allow for multiple eigenvalues.

I. IXTRODUCTION Linear time-invariant systems described by the state-variable equation

Theorent

If the system (2) is completely controllable, then u(t) = -Lx(r) is a stabilizing control law with

L = BW-'(T)

i= A X

(3)

W ( T )= Jre-"'Bly e-"" dr T

=

arbitrary

(4)

0

1.x..I:..:_.

A =

=

- e - ~ T ~ s e -+ ~ BE. 'r

(6)

- e - A T ~ e-A'r ~

- BE 2

-Q

(8)

where A = A - BEW-' and 0 2 0. Since W > 0 and Q 2 0, the Lyapunov stabdity theory [4]gua~ntees that 2 will have eigenvalues with negative real parts provided e"Q eA' g 0: for all f . Thus it is sufficient to show that e?'B is not identically zero. If this matrix were zero, then [B, AB,. . . ,2'B] = 0, which contradicts the complete controllability of the pair (7,B} (this pair iscontrollable if ( A , B} is controllable). Hence the stability ofd'and, in turn A, is established. Pre- and postmultiplying (8) by W-' establishes that Y(X) is a suitable Q.E.D. Lyapunov function. Manuscript received July 13. 1970.

...

--a1

and its Laplace transform @(s) = (SI

(7)

to both sides of (6) gives

WAS=

-an-'

1

' d -(e-"'BB e-*") d s dr

-2Bly = - WW-'BB' - BB'W-'W -

. ..

1

Jo

W is positive definite by complete controllability [ 3 ] ,so that adding

AW +

0

" '

general. A thorough understanding of the behavior of such systems is furnished throughthe concept of analytical functions of A. One such function (currently in vogue in system theory) is the transition matrix Q, defined by

Proqf: Pre- and postmultiplying W by A and A', respectively, gives

+ WA' = -

(1)

(5) receive considerable attention in control theory or systems theory in

w -'x

is a suitable Lyapunov function for the closed-loop system.

AW

1

,

-a,

and v(x) = x'

+ Bu(t)

where A is a companion matrix

- fQ-1.

'

(4)

In matrix theory (SI - A ) - is frequently called the resolcent of A. Since the companion form A is very simple, it is possible to derive many interesting relations involving A, Q,(r), and Q,(s). Such relations have been published and rediscovered on numerous occasions [1]-[4]. In this correspondence it will be shown that all these relations can be derived and generalized in a very simple manner. 11. THE C A U C H Y - ~ T T G R A LFORMULA The well-known Cauchy-integral formula

Manuscript received June 2. 1970.