Hierarchical genetic algorithms for optimal type-2 fuzzy system design

Hierarchical Genetic Algorithms for Optimal Type-2 Fuzzy System Design Patricia Melin, Daniela Sanchez, Leticia Cervantes Tijuana Institute Technology Tijuana, Mexico A. Granular Computing Abstract—In this paper we describe the application of genetic algorithms for optimal type-2 fuzzy system design. We illustrate the approach with two cases, one of designing optimal neural networks and the other of fuzzy control. Simulation results show the feasibility of the proposed approach of using hierarchical genetic algorithms for designing type-2 fuzzy systems. Keywords-component; Granular Type-2 Fuzzy system;Genetic Algorithm

I.

Computing;

INTRODUCTION

This paper is focused on granular computing and neural networks. These areas can work together to solve a great diversity of problems, i.e. the optimization for an application in the area of control and an application of human recognition with neural networks. We propose an algorithm to optimize a fuzzy system to control the Temperature in the Shower benchmark problem, in this application the fuzzy controller has two inputs: the water temperature and the flow rate. The controller uses these inputs to set the position of the hot and cold valves. In this part the genetic algorithm optimized the fuzzy system for control. We also used the same algorithm to optimize the integrator aggregating the responses of the neural network modules for the human recognition problem. This algorithm is based on the concept granular computing to design the optimal type-2 fuzzy system for the corresponding application. II.

BACKGROUND AND BASIC CONCEPTS

We provide in this section some basic concepts needed for this work.

Granular computing is based on fuzzy logic. There are many misconceptions about fuzzy logic. To begin with, fuzzy logic is not fuzzy. Basically, fuzzy logic is a precise logic of imprecision. Fuzzy logic is inspired by two remarkable human capabilities. First, the capability to reason and make decisions in an environment of imprecision, uncertainty, incompleteness of information, and partiality of truth. Second, the capability to perform a wide variety of physical and mental tasks based on perceptions, without any measurements and any computations. The basic concepts of graduation and granulation form the core of fuzzy logic, and are the main distinguishing features of fuzzy logic. More specifically, in fuzzy logic everything is or is allowed to be graduated, i.e., be a matter of degree or, equivalently, fuzzy. Furthermore, in fuzzy logic everything is or is allowed to be granulated, with a granule being a clump of attribute values drawn together by indistinguishability, similarity, proximity, or functionality. The concept of a generalized constraint serves to treat a granule as an object of computation. Graduated granulation, or equivalently fuzzy granulation, is a unique feature of fuzzy logic. Graduated granulation is inspired by the way in which humans deal with complexity and imprecision. The concepts of graduation, granulation, and graduated granulation play key roles in granular computing. Graduated granulation underlies the concept of a linguistic variable, i.e., a variable whose values are words rather than numbers. In retrospect, this concept, in combination with the associated concept of a fuzzy if–then rule, may be viewed as a first step toward granular computing[1][2][3][4]. Granular Computing (GrC) is a general computation theory for effectively using granules such as subsets, neighborhoods, ordered subsets, relations (subsets of products), fuzzy

978-1-61284-968-3/11/$26.00 ©2011 IEEE

sets(membership functions), variables (measurable functions), Turing machines (algorithms), and intervals to build an efficient computational model for complex with huge amounts of data, information and knowledge. Fuzzy and rough set theories, neutrosophic computing, quotient space, belief functions, machine learning, databases, data mining, cluster analysis, interval computing, more recently social computing, all involve granular computing. The topics and areas include: Computational Intelligence, in this area we have sub-areas such as: Neural Networks, Fuzzy Systems, Evolutionary Computation, Rough Sets and Formal Concept Analysis, etc. In the area of data mining and learning theory we have sub-areas such as: Stochastic learning, Machine learning, Kernel Machines, etc. In applications we can use it in bioinformatics, Medical Informatics and Chemical Informatics, e-Intelligence, Web Intelligence, Web Informatics, Web Mining and Semantic Web.[5]. One of the frequently asked questions about Granular Computing is whether it adds anything new to the established theory and practice of data clustering methods. To answer this question we must first reflect on the meaning of “granulation”. The commonly accepted definition of granulation as grouping together of elements based on their indistinguishability, similarity, proximity or functionality serves well the purpose of constructive generation of granules but it does little to differentiate granulation from clustering. To do so, we need to look beyond the algorithmic generation of granules and reflect on the semantics of granular entities. A different approach to the axiomatization of set theory designed to yield the same results as ZF theory but with a finite number of axioms (i.e without the reliance on axiom schemas) has been proposed by von Neumann in 1920 and subsequently has been refined by Bernays in 1937 and Goedel in 1940. The defining aspect of Von Neumann-Bernays- Goedel set theory (NBG) is the introduction of the concept of “class” in addition to the concept of “set” [6][7]. B. Type-2 Fuzzy System A type-2 fuzzy set expresses the degree of truth, non-deterministic imprecision and uncertainty with which an item belongs to the set. A type-2 fuzzy set, denoted by , is characterized by a type-2 membership function

where





If it is continuous is denoted by:

Where denotes the union of x and u. If discrete is denoted by:

is



Where denote the union between x and u. The membership functions generalized type-2 is parameterized with a type-1 primary membership function and type-1 secondary functions, fx (u), to define the footprint of uncertainty. If type-2 of membership function and

, it is

expressed by a lower and higher membership function and of type -1 and then it is called an interval type-2 fuzzy set [8]. C. Genetic Algorithms Genetic algorithms (GAs) are numerical optimization algorithms inspired by both natural selection and genetics. We can also say that the genetic algorithm is an optimization and search technique based on the principles of genetics and natural selection. A GA allows a population composed of many individuals to evolve under specified selection rules to a state that maximizes the “fitness”[9]. The method is a general one, capable of being applied to an extremely wide range of problems. The algorithms are simple to understand and the required computer code easy to write. GAs were in essence proposed by John Holland in the 1960's. His reasons for developing such algorithms went far beyond the type of problem solving with which this work is concerned. His 1975 book, Adaptation in Natural and Artificial Systems is particularly worth reading for its visionary approach. More recently others, for example De Jong, in a paper entitled Genetic Algorithms are NOT Function Optimizers , have been keen to remind us that GAs are potentially far more than just a robust method for estimating

a series of unknown parameters within a model of a physical system[10]. A typical algorithm might consist of the following: 1. Start with a randomly generated population of n l−bit chromosomes (candidate solutions to a problem). 2. Calculate the fitness ƒ(x) of each chromosome x in the population. 3. Repeat the following steps until n offspring have been created:  Select a pair of parent chromosomes from the current population, the probability of selection being an increasing function of fitness. Selection is done "with replacement," meaning that the same chromosome can be selected more than once to be-come a parent.  With probability Pc (the "crossover probability" or "crossover rate"), cross over the pair at a randomly chosen point (chosen with uniform probability) to form two offspring. If no crossover takes place, form two offspring that are exact copies of their respective parents. (Note that here the crossover rate is defined to be the probability that two parents will cross over in a single point. There are also "multipoint crossover" versions of the GA in which the crossover rate for a pair of parents is the number of points at which a crossover takes place.)  Mutate the two offspring at each locus with probability Pm (the mutation probability or mutation rate), and place the resulting chromosomes in the new population. If n is odd, one new population member can be discarded at random.  Replace the current population with the new population.  Go to step 2 [11][12]. D. Biometrics and control systems Biometrics plays an important role in the public security and information security domains. Using various physiological characteristics of the human, such as face, facial thermo grams, fingerprint, iris, retina, hand geometry etc., biometrics accurately identifies each individual [13][14] [15] [16] [17] [18] [19] [20] [21] [22]. Recent available real-world implementations indicate that biometric techniques are much more precise and accurate than the traditional

techniques. Other than precision, there have always been certain problems which remain associated with the existing traditional techniques [23]. As an example consider possession and knowledge. Both can be shared, stolen, forgotten, duplicated, misplaced or taken away. However the danger is minimized in case of biometric means [24] [25] [26]. Regarding the control area there are many applications where control is used to improve the performance of some applications such as flight control, temperature control, air flow control, all can be improved by fuzzy systems that can integrate plant simulation to see the behavior and improve your score. When performing a simulation in a control plant we have several options to start working with the fuzzy system, the number of inputs, the number of outputs, and the number of membership functions as well as the working range. Depending on all these values will be the outcome that is obtained and can be changing the parameters and types of functions according to the result that is generated. III.

RELATED WORKS

The field of fuzzy logic has been studied extensively over the years with the aim of further improving the application areas. There has been several research works in this area although each author takes their own approach to carry out the investigation. R. Sepulveda, et al. [27] realized an investigation about an efficient computational method to implement type-2 fuzzy logic in control applications. P. Melin and O. Castillo [28][29] in their work proposed the intelligent control of a stepping motor drive using an adaptive neurofuzzy inference system. In another investigation of the same authors an adaptive intelligence control of aircraft system with hybrid approach combining neural networks, fuzzy logic and fractal theory was proposed. The area of fuzzy control has success cases such as: Intelligence adaptive model-based control of robotic dynamic system with a hybrid fuzzy-neural approach in [30].Performance of a simple tuned fuzzy controller and PID controller on a DC motor, [31]. Intelligent control of aircraft dynamic system with a new hybrid neuro-fuzzy-fractal approach [32], and L. Aguilar [33] with his work of Intelligent control of a stepping motor drive using a hybrid

neuro-fuzzy ANFS approach. Other cases are: L. Cervantes and O. Castillo [34] in their work of Genetic Design of Type-1 and Type-2 Fuzzy Systems for Longitudinal Control of an Airplane where the control is achieved with a control and genetic algorithm to optimize the fuzzy system, D. Sanchez and P. Melin [22] with Modular neural networks with fuzzy response integration and its genetic optimization for human recognition based on iris, ear and voice biometrics , where she also worked with genetic algorithms to optimize in neural networks. When designing a fuzzy controller for achieving the best control possible or some application to improve the result, genetic algorithms can be a good alternative to use in the process of design. IV.

A. Human Recognition In this case we worked in the optimization for the fuzzy system that it is used for integration in the modular neural network (MNN). We used a database of the University of Science and Technology of Beijing [35]. The database consists of 77 people which contains 4 images per person (one ear), the image dimensions are 300 x 400 pixels, the format is BMP. 3 images were used for training and 1 for testing. A sample of ear images is shown in Figure 2.

PROBLEM DESCRIPTION Figure 2. Images of the ear

The problem was of developing a genetic algorithm to optimize the parameters of a fuzzy system that can be applied in 2 different applications in both the neural networks and fuzzy logic areas. The main goal was to achieve the best result in each application, in our case fuzzy control and recognition with neural networks. We design a genetic algorithm based on the ideas of granular computing; it helps to implement this algorithm for both solutions. We started to work with different membership functions in these cases and after performing the tests finally we took the best result. The genetic algorithm can change the number of inputs and outputs depending on that we need it for example (see Figure 1).

Figure 2 shows an example of pre-processing applied to each image in the ear, it is a manual cut to remove the parts of the image that are not in our interest, then makes a re-132 image size x 91 pixels and finally the image is automatically divided in 3 parts of interest (helix, lobe and shell).

Figure 3. Sample pre-processing done to the images of iris

We realized a number of tests with the genetic algorithm adding noise and the results are shown in Figure 4.

Figure 1.Fuzzy system Figure 4. Comparison between noise levels

The best fuzzy system with noise is shown in figure 5.

case of fuzzy control it was 2 inputs and 2 outputs. The best fuzzy system that we obtained in fuzzy control is shown in Figure 7.

Figure 7. Fuzzy system for control. Figure 5. Membership function of helix, shell, lobule

B. Fuzzy Control V. In this case we realized the simulation with the Simulink plant in Matlab. The problem was to improve temperature control in a shower example the original fuzzy system has two inputs to the fuzzy controller: the water temperature and the flow rate. The controller uses these inputs to set the position of the hot and cold valves. The plant that we used to simulate our problem is shown in figure 6.

CONCLUSIONS

With the previous results we can conclude that using the concept of granularity was very useful because we wanted to work with the same genetic algorithm to optimize the type-2 fuzzy system in 2 different areas of application. The observed results tell us that working with granularity we can optimize time and resources at the same time as it is a good option to divide the work required and get good results. REFERENCES

Figure 6. Simulation plant

When we simulated the fuzzy system the best result that we obtained was 0.000096 and when we used the genetic algorithm in case of human recognition we obtained 3 inputs and 1 output, in

[1] W.Pedrycz. A. Skowron, at al., handbook granular computing . Wiley-Interscience New York, NY, USA ©2008 ISBN:0470035544 9780470035542 2008 [2] A. Bargiela and W. Pedrycz. Granular Computing: An Introduction. Kluwer Academic Publishers, Dordercht, 2003. [3] L.A. Zadeh. Some reflections on soft computing, granular computing and their roles in the conception, design and utilization of information/intelligent systems. Soft Comput. 2 (1998) 23–25. [4] L.A. Zadeh. Outline of a new approach to the analysis of complex systems and decision processes. IEEE Trans.Syst. Man Cybern. SMC-3 (1973) 28–44. [5]Computer society, The 2011 IEEE Internaional Confenrece on Granular Computing Sapporo, Japan, August 11-13, 2011. IEEE GrC 2011 Indexed by EI (GrC 2005, 2006,2007, 2008, 2009 are indexed by EI).Call for Papers . [6] K. Goedel, The Consistency of the Axiom of Choice and of the Generalized Continuum Hypothesis with the Axioms of Set Theory. Princeton University Press, Princeton, NJ. , 1940. [7]A. Bargiela,, W. Pedrycz, , IEEE. The roots of Granular Computing. GrC 2006: 806-809. [8]J.R. Castro. Tutorial type-2 fuzzy logic theory and applications.IEEE Computational intelligence society mexico

chapter. 3rd nternational seminar on computational intelligence 2006. [9]R. Haupt., S. Haupt. “Practical Genetic Algorithm ”,WileyInterscience,Book,2004. [10] A. Coley.” An Introduction to Genetic Algorithms for Scientists and Engineers”,World Scientific,Book,1999 [11]M. Mitchell.” An Introduction to Genetic Algorithms” .Massachusetts Institute of Technology Book,1999. [12] R. Haupt, S. Haupt. “Practical Genetic Algorithm” ,Wiley- Interscience,Book,2004. [13] J. Daugman ,G.,“High Confidence Visual Recognition of Persons by a Test of Statistical Independence” . IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 15, nº 11. Noviembre 1993. pp. 1148-1161. [14] P. Melin , C. Felix, O. Castillo, Face recognition using modular neural networks and the fuzzy Sugeno integral for response integration. Int. J. Intell. Syst. 20(2): pp.275-29 (2005). [15] P. Melin , O. Mendoza. , O. Castillo, An improved method for edge detection based on interval type-2 fuzzy logic. Expert Syst. Appl. 37(12): pp. 8527-8535 (2010). [16] O. Mendoza ,P. Melin , G. Licea , A New method for Edge Detection in Image Processing Using Interval Type-2 Fuzzy Logic. GrC 2007: pp. 151-156. [17] O. Mendoza., P. Melin., G. Licea ., A hybrid approach for image recognition combining type-2 fuzzy logic, modular neural networks and the Sugeno integral. Inf. Sci. 179(13): pp. 2078-2101 (2009). [18]O. Mendoza , P. Melin , O. Castillo , Interval type-2 fuzzy logic and modular neural networks for face recognition applications. Appl. Soft Comput. 9(4): pp. 1377-1387 (2009). [19] ProQuest Information and Learning Company., "History of Voice Recognition Technology, The". Information Management Journal. 07 Oct, 2009. http://findarticles.com/ p/articles/mi_qa3937/is_200401/ai_n9383074/ [20] T. Salazar, P. Melin , O. Castillo , A New Biometric Recognition Technique Based on Hand Geometry and Voice Using Neural Networks and Fuzzy Logic. Soft Computing for Hybrid Intelligent Systems 2008: pp. 171-186. [21] B. Verma , M. Blumenstein , “Pattern Recognition Technologies and Applications”, Information Science Reference, Hershey, New York, 2008, pp. 90-91. [22] D. Sánchez and P. Melin. “Modular Neural Network with Fuzzy Integration and Its Optimization Using Genetic Algorithms for Human Recognition Based on Iris, Ear and Voice Biometrics”, Soft Computing for Recognition Based on Biometrics, SCI 312, Springer-Verlag Berlin Heidelberg 2010. pp. 85–102. [23]P. Melin., O. Castillo , “Hybrid Intelligent Systems for Pattern Recognition Using Soft Computing”, Springer-Verlag, Heidelberg, 2005, pp 2-3. [24] P. Melin , J. Urias , D. Solano , M. Soto ,et al., “Voice Recognition with Neural Networks, Type-2 Fuzzy Logic and Genetic Algorithms”, Jouranl of Engineering Letters 13(2): pp. 108-116 (2006). [25] B. Moreno, A. Sanchez, et al., “On the Use of Outer Ear Images for Personal Identification in Security Applications”, IEEE 33rd Annual International Carnahan Conference on Security Technology, pp. 469-476, 1999. [26] R. Muñoz., O. Castillo , P. Melin , Face, Fingerprint and Voice Recognition with Modular Neural Networks and Fuzzy Integration. Bio-inspired Hybrid Intelligent Systems for Image Analysis and Pattern Recognition 2009: pp. 69-79. [27]R. Sepulveda, O. Castillo, P. Melin, O. Montiel: “An efficient computational method to implement type-2 fuzzy logic in control application”.Analysis and design of intelligent system using soft computing techniques 2007, pp.45-52. [28]P. Melin , O. Castillo .”Intelligence control of a stepping motor drive using an adaptive neural fuzzy inference system ”. Inf. Sci. 170(2-4), pp. 133-151(2005).

[29] P. Melin, O. Castillo.”Adaptive intelligent control of aircraft system with a hybrid approach combining neural network, fuzzy logic and fractal theory.” Appl. Soft Comput.3(4), pp. 363-378(2003). [30]O. Castillo , P.Melin.”Intelligent adaptive model-based control of robotic dynamic system with a hybrid fuzzy-neural approach”. Appl. Soft Comput.3(4), pp. 363-378(2003). [31]O. Montiel, R. Sepulveda, P. Melin, O. Castillo. ” Performance of a simple tuned fuzzy controller and PID controller on a DC motor”. FOCI 2007, pp. 531-537. [32]P. Melin, O. Castillo.”Intelligence control of aircraft dynamic system with a new hybrid nweuro-fuzzy fractal approach”. Inf. Sci. 142(1-4), pp. 161-175(2002). [33]L. Aguilar, P. Melin, O. Castillo. “Intelligence control of a stepping motor drive using a hybrid neuro-fuzzy ANFIS approach”. Appl. Soft Comput. 3(3), pp.209-219(2003). [34]L. Cervantes, O. Castillo.” Design of a Fuzzy System for the Longitudinal Control of an F-14 Airplane”. Studies in Computational Intelligence, 2011, Volume 318/2011, pp.213224. [35] Database Ear Recognition Laboratory from the University of Science & Technology Beijing (USTB). Found on the Web page: http://www.ustb.edu.cn/resb/en/index.htm asp (Accessed 21 September 2009).