Design of a proposed neural network control system

Available online at www.sciencedirect.com

Simulation Modelling Practice and Theory 16 (2008) 368–378 www.elsevier.com/locate/simpat

Design of a proposed neural network control system for trajectory controlling of walking robots S ß ahin Yildirim * Erciyes University, Engineering Faculty, Mechanical Engineering Department, 38039 Kayseri, Turkey Received 17 February 2006; received in revised form 3 November 2007; accepted 6 December 2007 Available online 1 January 2008

Abstract The use of a proposed recurrent hybrid neural network to control of walking robot with four legs is investigated in this paper. A neural networks based control system is utilized to the control of four-legged walking robot. The control system consists of four proposed neural controllers, four standard PD controllers and four-legged planar walking robot. The proposed neural network (NN) is employed as an inverse controller of the robot. The NN has three layers, which are input, hybrid hidden and output layers. In addition to feedforward connections from the input layer to the hidden layer and from the hidden layer to the output layer, there is also feedback connection from the output layer to the hidden layer and from the hidden layer to itself. The reason to use hybrid layer is that robot’s dynamics consists of linear and non-linear parts. The results show that the proposed neural control system has superior performance to control trajectory of walking robot with payload. Ó 2007 Elsevier B.V. All rights reserved. Keywords: Four-legged robot; Neural networks; PD controller; Recurrent neural network

1. Introduction Four-legged walking robot can be seen as an extension of two-legged bipedal robot system if the third dimension is ignored. The main problem with a bipedal robot, however, is that of balance control to prevent the robot from falling over and enable it to walk. Several researchers have proposed dynamic balance control systems for bipedal robots. State feedback laws have been suggested [3], since they are a natural control method for multi-variable systems. A method of deriving the linear quadratic regulator (LQR) for an n-legged robot that could be adapted for bipeds has been presented by Channon et al. [1]. A neural network which is recurrent hybrid network has been employed to the control of a bipedal robot. A neural inverse model control scheme has been described by Pham and Yıldırım [4]. The proposed network has two advantages: the recurrent connections provide an inherent dynamic *

Tel.: +90 352 4374901. E-mail address: [email protected]

1569-190X/$ - see front matter Ó 2007 Elsevier B.V. All rights reserved. doi:10.1016/j.simpat.2007.12.002

S ß . Yildirim / Simulation Modelling Practice and Theory 16 (2008) 368–378

369

memory which was facilitates modeling and the hybrid hidden layer enables real systems comprising linear and non-linear parts to be modeled accurately. This paper describes a proposed neural control scheme for four-legged walking robot. The scheme employs four neural controllers for stance legs and swing legs of the robot. The inverse neural networks is utilized as controllers of control system. Four-legged walking robot is used to overcome problem of balance. 2. The proposed neural network The neural network technique is a very effective tool for controlling complex non-linear systems when we have no complete model information, or even when considering a controlled plant as a black box. The neural networks employed in this work were of the recurrent type. Recurrent networks have the advantage of being able to model dynamics systems accurately and in a compact form. A recurrent network can be represented in a general schematic form as illustrated in Fig. 1. This diagram depicts the hybrid hidden layer as comprising a linear part and a non-linear part and shows that, in addition to the usual feedforward connections, the networks also have feedback connections from the output layer to the hidden layer and self-feedback connections in the hidden layer. The reason for adopting a hybrid linear/non-linear structure for the hidden layer will be evident later [5,7]. At a given discrete time t, let u(t) be the input to a recurrent hybrid network, y(t), the output of the network, x1(t) the output of the linear part of the hidden layer and x2(t) the output of the non-linear part of the hidden layer. The operation of the network is summarized by the following equations (see also Fig. 1): x1 ðt þ 1Þ ¼ WI1 uðt þ 1Þ þ bx1 ðtÞ þ aJ1 yðtÞ

ð1Þ

I2

x2 ðt þ 1Þ ¼ FfW uðt þ 1Þ þ bx2 ðtÞ þ aJ2 yðtÞg H1

ð2Þ

H2

yðt þ 1Þ ¼ W x1 ðt þ 1Þ þ W x2 ðt þ 1Þ

ð3Þ

where WI1 is the matrix of weights of connections between the input layer and the linear hidden layer, WI2 is the matrix of weights of connections between the input layer and the non-linear hidden layer, WH1 is the

y(t+1)

Output Layer

α

WH1

WH2

x1 (t+1) β

α

x2 (t+1)

Linear

Non-linear

Hidden Layer

WI1

WI2

Hidden Layer

β

Input Layer

u(t) Fig. 1. Schematic representation of the proposed three layered recurrent hybrid network.

370

S ß . Yildirim / Simulation Modelling Practice and Theory 16 (2008) 368–378

matrix of weights of connections between the linear hidden layer and the output layer, WH2 is the matrix of weights of connections between the non-linear hidden layer and the output layer, F{} is the activation function of neurons in the non-linear hidden layer and a and b are the weights of the self-feedback and output feedback connections. J1 and J2 are, respectively, nH1  nO and nH2  nO matrices with all elements equal to 1, where nH1 and nH2 are the numbers of linear and non-linear hidden neurons, and nO, the number of output neurons. If only linear activation is adopted for the hidden neurons, the above equations simplify to yðt þ 1Þ ¼ WH1 xðt þ 1Þ

ð4Þ

I1

xðt þ 1Þ ¼ W uðt þ 1Þ þ bxðtÞ þ aJ1 yðtÞ

ð5Þ

Replacing y(t + 1) by WH1 x(t + 1) in Eq. (5) gives xðt þ 1Þ ¼ ðbI þ aJ1 WH1 ÞxðtÞ þ WI1 uðt þ 1Þ

ð6Þ

where I is a nH1  nH1 identity matrix Eq. (6) is of the form xðt þ 1Þ ¼ AxðtÞ þ Buðt þ 1Þ H1

ð7Þ

I1

where A = bI+aJW and B= W Eq. (7) represents the state equation of a linear system of which x is the state vector. The elements of A and B can be adjusted through training so that any arbitrary linear system of order nH1 can be modeled by the given network. When non-linear neurons are adopted, this gives the network the ability to perform non-linear dynamics mapping and thus model non-linear dynamic systems. The existence in the recurrent network of a hidden layer with both linear and non-linear neurons facilitates the modeling of practical non-linear systems comprising linear and non-linear parts. The backpropagation (BP) algorithm is used to update weights of the proposed neural network. The BP algorithm is a method of supervised neural network learning. During training, the network is presented with a large number of input patterns. The experimental outputs are then compared to the neural network output nodes. The error between the experimental and neural network response is used to update the weights of the network inter-connections. This update is performed after each pattern presentation. One run through the entire pattern set is termed an epoch. The training process continues for multiple epochs, until a satisfactorily small error is produced. The test phase uses a different set if input patterns. The neural network outputs are again compared to a set of experimental outputs. This error is used to evaluate the networks ability to generalize. Usually, the training set and/or architecture needs to be assessed at this point. The BP algorithm is the most commonly used to update the weights of the neural networks. The weights between input layer and the hidden layer are updated as follows: DW ij ðtÞ ¼ g

oE2 ðtÞ þ aDW ij ðt  1Þ oW ij ðtÞ

ð8Þ

The weights between the hidden layer and the output layer are updated in the following equation: DW jk ðtÞ ¼ g

oE1 ðtÞ þ aDW jk ðt  1Þ oW jk ðtÞ

ð9Þ

where g is the learning rate, and a is the momentum term. E2(t) is the propagation error between hidden layer and output layer. E1(t) is the error between desired and neural network output signals. 3. Four-legged planar walking robot Fig. 2 illustrates four-legged planar walking robot, which has same leg length with other one. As can be seen from the figure, the torso has mass mt and inertia Jt measured about its center of mass, which is located at distance lt above the hip joints. The upper and lower leg elements are identical, with length 2le, mass me and inertia Je measured about their centers of mass. The feet have mass mf located at the ankle joint and are assumed to have negligible inertia. With both feet on the ground, the robot acts almost like the closed-loop linkage of four co-operating two

S ß . Yildirim / Simulation Modelling Practice and Theory 16 (2008) 368–378

lt

371

ψ

2l e α α

2

1

2l e β2 β1

Fig. 2. Four-legged planar walking robot.

planar SCARA robots. However, the interest in this paper is the control of the walking movements of the robot where only two foots remain on the ground. 3.1. Walking Walking commences at time t = 0 with one foot, called the swing foot, leaving the ground. Then, over a period T, the swing foot moves forward a distance 2L and the hip a distance L. Walking ends at time t = T, the instant just prior to the swing foot making contact with the ground. Throughout this period, the other foot, called the stance foot, remains fixed on the ground thus providing a single-support for the biped. At a given instant during the single-support phase, the position of the robot relative to the stance foot can be specified by the following co-ordinate vector H: H ¼ ½w1 a1 a2 b1 b2 w2 a3 a4 b3 b4 

T

ð10Þ

Here, w represents the angle of the torso with respect to the vertical, a1 and a2 are the angles of the upper leg elements relative to the vertical and b1 and b2 are the angles of the lower leg elements relative to the vertical, as shown in Fig. 2. Motors located at the robot’s hip, knees and stance foot generate torques s1 to s10. These can be grouped into vector s where: s ¼ ½s1 s2 s3 s4 s5 s6 s7 s8 s9 s10 

ð11Þ

Note that the torque arising from the swing foot ankle joint motor must be zero due to the assumption of negligible foot inertia. Hence it is not included in Eq. (11). € arising from the application of torques s can be derived from the following equaThe joint accelerations H tion [2,4]: € þ VðH; HÞ _ þ GðHÞ ¼ C1 F1 þ C2 F2 þ Ds MðHÞH

ð12Þ

372

S ß . Yildirim / Simulation Modelling Practice and Theory 16 (2008) 368–378

t=T

+

Swing Foots

Stance Foots

2L Fig. 3. Impact of swing foots with the ground as pointed out on the figure.

_ is a vector containing centripetal and Coriolis terms, G(H) is a vector Here, M(H) is the mass matrix, VðH; HÞ of torques arising from gravitational loading, F1 and F2 are the foot forces, C1 and C2 are the foot force transformation matrices and D is the joint transformation matrix. 3.2. Impact The second stage of a step results in the front swing and in the back stance foots hitting the ground and the front stance and the back swing foots breaking contact with the ground. This stage, termed impact, is assumed to be instantaneous, commencing at time t = T and finishing at some infinitesimally short time later t = T+ (Fig. 3). Consequently the position of the robot will not appreciably change during impact. 4. Control system Fig. 4 shows the control system adopted. The system comprises four PD controllers and four neural controllers, which are a recurrent hybrid networks used to model the inverse dynamics of the robot. A trajectory generator (not shown) produces the required sequence of rotations for each joint to yield the desired walking movements of the robot. The NNs are trained on-line during control to give the system the ability to adapt to change. Relative positions U of the robot joint angles are used as reference signals to the controller: U ¼ ½/1 /2 /3 /4 /5 /6 /7 /8 /9 /10 T /1 ¼ a1  w1 ; /6 ¼ a3  w2 ;

/2 ¼ a2  w1 ; /7 ¼ a4  w2 ;

/3 ¼ b1  a1 ; /8 ¼ b3  a3 ;

ð13Þ /4 ¼ b2  a2 ; /9 ¼ b4  a4 ;

/5 ¼ c1  b1 /10 ¼ c1  b3

ð14Þ

From Fig. 4, the vector of input torques to the robot is given by s ¼ sN þ sP

ð15Þ

where sN ¼ f ðU; eÞ sP ¼ KP e þ KD e_

ð16Þ ð17Þ

sN is the output of the NN controller (Fig. 4), which is seen as performing a non-linear mapping f on the set point U and the control error e. sP is the output of the PD controller. KP and KD are the gains of the controller.

S ß . Yildirim / Simulation Modelling Practice and Theory 16 (2008) 368–378

NC1 (STANCE LEG 1)

373

lt ψ

2l e

Φd

NC2 (SWING LEG 1)

τN

α2 α1

τ

2le β1

NC3 (STANCE LEG 2) NC4 (SWING LEG 2)

β2

Φ

τPD PD CONTROLLER 1 PD CONTROLLER 2 PD CONTROLLER 3 PD CONTROLLER 4

e

: MULTIPLE CONNECTION Fig. 4. Proposed neural control system for four-legged walking robot.

In this work, the following empirically determined values were adopted for KP and KD : KP ¼ diag½4000 4000 4000 4000 and KD ¼ diag½20 20 20 20. 4.1. Neural network controller The NN controller for walking robot hip and foot trajectory control has three layers as shown in Fig. 1. Also as depicted in Fig. 4, the first layer received reference inputs from the trajectory generator and also the control errors. Again, the second layer consisted of two parts, a linear part and a non-linear part. Finally, the third layer of the network produced outputs to drive the robot joints.

374

S ß . Yildirim / Simulation Modelling Practice and Theory 16 (2008) 368–378

The controller was trained by backpropagating the error e through the robot with a minor modification as shown below. Let Ek be the joint angle error for joint k. Using the chain rule, it follows that the gradient of Ek in weight space is: oEk oEk oUak osk osNk ¼ owij oUak osk osNk owij

ð18Þ

Nk ak In Eq. (18), os can be readily obtained through the BP algorithm but oU is difficult to determine exactly beowij os oUak k cause the robot dynamics are unknown. To overcome this problem, osk is computed as follows: X oUak oUak DUak ¼ Dsk þ DF m ð19:aÞ osk oF m m X oUak osk oUak DUak ¼ Dsk þ DF m ð19:bÞ osk osk oF m m oUak oUak X osk DUak ¼ Dsk þ DF m ð19:cÞ osk osk m oF m

oUak 1 ¼ Dsk P osk osk ðDUak þ m oF m

ð19:dÞ

DF m Þ DUak

5. Simulation results The proposed control system, which is controlled walking robot, illustrated in Fig. 4 was implemented on a PC using the MATLAB software [6]. The simulation results are presented below. They illustrate the effects of different torso masses on the performance of the control systems (Table 1). First, using the proposed control system of Fig. 4, the walking robot was trained for 35,000 iterations to carry a torso mass of 20 kg to 1000 points randomly located in the vertical (X–Y) plane. Following training, Table 1 Kinematic parameters of the bipedal robot Kinematics parameters Mass parameters Inertia parameters

lt = 0.25 m, le = 0.175 m mt = 20–200 kg, me = 5 kg, mf = 2 kg It = 1 kg m2, Ie = 0.05 kg m2

Table 2 Structural and training parameters of the proposed neural network NN

b

l

a

b

n

N

AF

RHNN

0.0001

0.01

0.8

0.8

12+12

35,000

HT

g: learning rate, l: momentum term, a: feedback gain from output layer to hidden layer b: feedback gain from hidden layer to itself, n: number of neurons in hidden layer N: iteration numbers, AF: activation function, RHNN: recurrent hybrid neural network, HT: hyperbolic tangent.

Table 3 RMS errors for various step lengths Step length (m)

PD controller (RMSE)

Neural control system (RMSE)

0.62 0.57 0.51 0.43

0.2067 0.2421 0.1787 0.2982

0.0582 0.0644 0.0443 0.0596

S ß . Yildirim / Simulation Modelling Practice and Theory 16 (2008) 368–378

375

Table 4 RMS errors for various stances Stance H (m)

PD controller (RMSE)

Neural control system (RMSE)

0.65 0.61 0.58 0.55

0.2173 0.2362 0.2376 0.2409

0.0583 0.0569 0.0581 0.0603

Table 5 RMS errors for different torso masses mt (kg)

PD controller (RMSE)

Neural control system (RMSE)

20 40 60 100 200

0.2638 0.4011 0.5821 35.8588 63.0643

0.0542 0.1022 0.1731 2.0092 4.7544

the robot was prescribed regular foot and hip trajectories on testing stage. After just one attempt, it was able to trace the desired trajectories with only very small errors (Tables 2–5). The robot was then made to carry a torso mass of 200 kg. The foot and hip trajectories of the robot on the second attempt, after the torso mass had increased to 200 kg, are shown in Fig. 5. For comparison, foot and hip trajectories obtained using the PD controller alone are depicted in Fig. 6 for the case with the increased torso mass respectively. Final positions of the robot NN and PD controllers are shown in Figs. 7 and 8. All results of the proposed control system are consisted of testing stage of the proposed neural network.

NNC .....A c tual ___ Desired

0.7

0.6

Y [m ]

0.5

0.4

0.3

0.2

0.1

0 0

0.1

0.2

0.3

0.4

0.5 X[m ]

0.6

0.7

0.8

0.9

Fig. 5. Foot and hip trajectories of four-legged walking robot using proposed NN controller for different torso masses (mt = 200 kg).

376

S ß . Yildirim / Simulation Modelling Practice and Theory 16 (2008) 368–378

P DC .....A c tual ___ Desired

0.7

0.6

Y [m ]

0.5

0.4

0.3

0.2

0.1

0 0

0.1

0.2

0.3

0.4

0.5 X[m ]

0.6

0.7

0.8

0.9

Fig. 6. Foot and hip trajectories of four-legged walking robot using PD controller for different torso masses (mt = 200 kg).

NNC .....A c tual ___ Desired

0.7

0.6

Y [m ]

0.5

0.4

0.3

0.2

0.1

0 0

0.1

0.2

0.3

0.4

0.5 X[m ]

0.6

0.7

0.8

Fig. 7. Final position of four-legged walking robot using proposed NN controller.

0.9

S ß . Yildirim / Simulation Modelling Practice and Theory 16 (2008) 368–378

377

P DC .....A c tual ___ Desired

0.7

0.6

Y [m ]

0.5

0.4

0.3

0.2

0.1

0 0

0.1

0.2

0.3

0.4

0.5 X[m ]

0.6

0.7

0.8

0.9

Fig. 8. Final position of four-legged walking robot using PD controller.

It can be noted from the figures presented that the neural control system performed better than the PD controller system. 6. Conclusion This paper has described neural control based system for four-legged walking robot. The system controls robot’s legs joints separately. Simulations have shown that neural controller is better than standard PD controllers at rejecting large load disturbances. The results obtained was supported the theory that the proposed network is able to represent different types of dynamics systems. The network is trainable using the simple backpropagation algorithm to control the inverse dynamics of variety non-linear multi-legged walking robots. Finally, the approach is improved that this kind of neural network based control system could be used in experimental robotic applications. Acknowledgements This research results consisted of a part of project Scientific & Technological Research Council of Turkey (TUBITAK). The author wishes to express his thanks to TUBITAK and Erciyes University for supporting this project. References [1] P.H. Channon, S.H. Hopkins, D.T. Pham, Optimal control of an n-legged robot, IMechE 210 (1996) 51–63. [2] J.J. Craig, Introduction to Robotics: Mechanics and Control, Addison-Wesley, Reading, MA, 1989. [3] R.E. Goddard, H. Hemami, F.C. Weimer, Biped side step in the frontal plane, IEEE Transactions on Automatic Control 28 (1985) 179–186.

378

S ß . Yildirim / Simulation Modelling Practice and Theory 16 (2008) 368–378

[4] D.T. Pham, S. Yıldırım, Control of the Trajectory of a planar robot using recurrent hybrid networks, International Journal of Machine Tools & Manufacture 39 (1999) 415–429. [5] S. Yıldırım, Neural network for control of bipedal robots, IEE Electronics Letters 35 (1999) 2064–2065. [6] The MathWorks Inc. 1998. Mathlab: The Language of Technical Computing, Version 5.3. [7] S. Yıldırım, Design of adaptive robot control system using recurrent neural network, Journal of Intelligent & Robotic Systems 44 (2005) 247–261.