Fault diagnostic systems for agricultural machinery

biosystems engineering 106 (2010) 26–36

Available at www.sciencedirect.com

journal homepage: www.elsevier.com/locate/issn/15375110

Research Paper

Fault diagnostic systems for agricultural machinery Geert Craessaerts, Josse De Baerdemaeker, Wouter Saeys* Department of Biosystems, Katholieke Universiteit Leuven, Kasteelpark Arenberg 30, B-3001 Leuven, Belgium

article info Fault detection and diagnosis in process industry have attracted a lot of attention recently. Article history:

There is an abundance of literature on process fault diagnosis ranging from analytical methods

Received 14 April 2008

to artificial and statistical methods. From a modelling perspective, the methods can rely on

Received in revised form

quantitative, semi-quantitative and qualitative models. At the other end of the spectrum,

20 November 2009

there are historical data-based methods that do not make use of any form of model infor-

Accepted 7 December 2009

mation but rely only on historical process data. The basic aim of this study is to emphasize the

Published online 31 March 2010

importance of introducing more advanced multivariate fault diagnostic systems on agricultural machinery. Up till now, farmers and contractors still observe the process in order to detect process and sensor failures which can disturb the actions of the controllers and cause severe damage to the machine. In the future, the complete reliance on human operators for the correct functioning of these systems will become too risky, due to the increasing complexity of this type of machinery. A systematic and comparative study of various fault diagnostic methods, from an agricultural machinery perspective, is provided in this study. The different fault diagnostic techniques, investigated in scientific literature, are compared and evaluated on a common set of criteria. Typical requirements of a fault diagnostic system for agricultural machinery are adaptability to process changes, user-friendliness, quick detection and robustness. Based on these findings, a hybrid framework of qualitative model-based fault detection techniques and pattern recognition-based methods, which rely on historical process data, is proposed as the most suitable fault diagnostic technique. As a first step towards more advanced fault detection and isolation systems, the general applicability of intelligent neural network techniques like supervised self-organizing maps (SOMs) and back-propagation neural networks is illustrated for the detection and isolation of sensor failures on a New Holland CX combine harvester. Pattern recognition techniques, such as neural networks, were found to be very suitable for this kind of application because a lot of historical process data is available since the recent generation of combine harvesters is equipped with a wide range of sensors and actuators, which are continuously monitored. Moreover, these pattern recognition techniques allow quick detection, are easy to use and are able to adapt their structure and/or model parameters based on new measurement data. Since there is room for improvement of these standard techniques, suggestions for future research concerning fault diagnosis on agricultural machinery are given as well. ª 2009 IAgrE. Published by Elsevier Ltd. All rights reserved.

* Corresponding author. E-mail address: [email protected] (W. Saeys). 1537-5110/$ – see front matter ª 2009 IAgrE. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.biosystemseng.2009.12.004

biosystems engineering 106 (2010) 26–36

1.

Introduction

The introduction of process control has made a remarkable contribution to the world of agricultural technology. In the past, different processes on agricultural machinery were performed by human operators, but now the larger part is handled in an automatic manner by low and high-level control actions (Coen, Saeys, Missotten, & De Baerdemaeker, 2007; Coen, Vanrenterghem, Saeys, & De Baerdemaeker, 2008; Craessaerts, Saeys, Missotten, & De Baerdemaeker, in press). At a supervisory level, human operators still observe the process in order to detect process malfunctions, abnormal events and/or sensor failures which can disturb the actions of the controllers and cause severe damage to the whole process. However, this supervisory task becomes increasingly difficult for agricultural machinery operators due to the ever increasing workload and machine complexity they have to deal with. As a result, human operators often make erroneous decisions concerning the supervisory control of these machines which can have a significant economic, environmental and/or safety impact. Operating on uncertain or missing data may cause improper control actions and consequently the system will not be operating optimally. One of the next challenges for control engineers involved with the automation of agricultural machinery, will be the automation of fault detection and diagnosis to further lighten the job of the operator. In this context, a fault can be defined as a departure from an acceptable range of an observed variable or a calculated parameter associated with a process (Himmelblau, 1978). This defines a fault as a process abnormality or symptom, such as too high a pressure or too high a temperature of a hydrostatic pump. Faults can have different sources and can be classified into three classes of failures: caused by malfunctioning sensors and/or actuators, structural changes in the process or a sudden change of model parameters. The latter one is mainly caused by external disturbances whose dynamics are not taken into account in the process model. In this paper, an overview will be given of the different diagnostic techniques described in the literature for fault detection and diagnosis. Up till now, most of these techniques have been applied in the process industry because of the critical safety norms these processes deal with. It will be shown that fault diagnostic systems have not been given much attention yet in agricultural machinery research. However, these techniques could be of high value at a supervisory control level for agricultural machinery. Based on a formulation of the specific characteristics that a fault diagnostic system for agricultural machinery should include, a suggestion will be made of the most suitable diagnostic methods. Finally, the usefulness of artificial neural networks as a fault diagnostic tool for sensor failure detection will be investigated for an example case. This case study encompasses the detection and isolation of sensor failures on a New Holland CX combine harvester by means of self-organizing maps (SOMs) and back-propagation neural networks.

2.

27

Fault detection and isolation techniques

In the literature, fault diagnosis methods are broadly classified into three categories based on the type and amount of prior knowledge they use. A distinction can be made between quantitative model-based methods, qualitative model-based methods and process history-based methods (Venkatasubramanian, Rengaswamy, Yin, and Kavuri, 2003). The basic a priori knowledge that is needed for fault diagnosis is the set of possible failures and the relationship between the observations (symptoms) and the failures. This a priori domain knowledge may be derived from: a fundamental understanding of the process using firstprinciples models: such knowledge is referred to as causal or model-based knowledge, - historical process data: in this case, the knowledge is referred to as process history-based knowledge. -

The model-based a priori knowledge can be broadly classified as qualitative or quantitative. The model is usually developed based on some fundamental understanding of the physics of the process. In quantitative models this understanding is expressed in terms of a mathematical functional relationship between the inputs and outputs of the system. In contrast, in qualitative model equations these relationships are expressed in terms of heuristic functions centred around different units in a process. An excellent review of the different fault detection and isolation (FDI) techniques discussed in scientific literature is given by Venkatasubramanian, Rengaswamy, and Kavuri (2003); Venkatasubramanian, Rengaswamy, Kavuri, and Yin (2003); Venkatasubramanian, Rengaswamy, Yin, et al. (2003). In this section, these different techniques will be briefly communicated in order to highlight the advantages and shortcomings of the discussed techniques. This critical evaluation will be based on a formulation of the desirable characteristics the ideal FDI system should possess. The conclusions drawn from this review will be of high importance for readers wishing to implement a FDI system for their particular application.

2.1.

Desired characteristics of a fault diagnostic system

In Venkatasubramanian, Rengaswamy, Yin, et al. (2003), an overview is given of the characteristics the ideal FDI should possess: A quick detection and diagnosis of faults: a trade-off should be made between quick detection of faults and sensitivity to measurement noise. A high sensitivity to noise will lead to frequent false alarms during normal operation. - Isolation of faults: the fault diagnostic system should be able to make a distinction between different types of failures. - Robustness: the fault diagnostic system should be robust with respect to measurement noise and model uncertainties. -

28

biosystems engineering 106 (2010) 26–36

Novelty identification: the fault diagnostic system should be able to recognize the occurrence of novel faults and not misclassify these as one of the known malfunctions or as normal operation. - Classification error estimate: in order to make the system more reliable for the user, a prior estimate of the classification errors that can occur should be provided. - Adaptability: most processes in the real world are timevarying because of changes in environmental conditions and/or product characteristics. The diagnostic system should be adaptable to these changes. - Explanation facility: besides the ability of the system to identify the source of malfunctioning, the diagnostic system should also provide an explanation of how the fault originated and propagated into the current situation. - Low modelling requirements: the modelling effort for the development of the diagnostic classifier should be as low as possible. - Low computational requirements: with an eye on an implementation of the diagnostic classifier on a system with fast dynamics, the implementation algorithm should be of low complexity. - Multiple fault identification: the fault diagnosis system should be able to identify multiple faults occurring at the same time. -

2.2.

Quantitative model-based methods

In quantitative model-based FDI methods, one makes use of the inconsistencies, also called the residuals, between the actual and predicted process behaviour. As a first step, the residuals between the real system response and the modelled system response are calculated. Any inconsistency, expressed as residuals, can be used for detection and isolation purposes. The residuals should be close to zero when no fault occurs, but show ‘significant’ values when the underlying system changes. In a final step, a decision algorithm will make the appropriate fault diagnosis. As mentioned above, the generation of the diagnostic residuals requires an explicit mathematical model of the system. Consequently, the complexity and reliability of the resulting FDI system depends on the kind of modelling method and comparison strategy that was used (Venkatasubramanian, Rengaswamy, Yin, et al., 2003). Either first-principles models, black-box or statistical models can be used. First-principles models are based on a physical understanding of the process and are of high complexity when dealing with supervisory control and diagnosis of a whole plant which very often has non-linear characteristics. As a result, first-principle models are seldom used for fault diagnosis. Most of the FDI methods use discrete black-box and/or statistical plant models such as input–output or statespace models and assume linearity of the plant (Venkatasubramanian, Rengaswamy, Yin, et al., 2003). Process faults usually cause a change in the state variables, a change in the model parameters and/or a change in the output of the process. Based on the process model, one can estimate the non-measurable state variables or model parameters by the observed outputs and inputs using state estimation and parameter estimation methods. Typical state

estimation techniques used in fault diagnosis are the Kalman filter and the Luemberger observer (Clark, 1978; Frank, 1986; Patton, Chen, & Nielsen, 1995). These reconstruct the unknown states based on the measurements or subsets of the measurement data. The Luemberger observer is typically used in a deterministic setting while the Kalman filter is mainly used for stochastic processes (Betta and Pietrosanto, 2000). As a consequence, the deviations (residuals) of the model parameters and/or state variables from the normal situation can be used as a fault indicator. Similarly, parity relations (Gertler, 1995; Willsky, 1976) check the consistency of the modelled process output with the real measured process output. Any observed inconsistency would result in a high output residual and indicate the occurrence of a typical fault. Once the residuals are calculated, they have to be evaluated. When designing the decision algorithm, a trade-off should be made between fast and reliable fault detection. In most applications of residual observation, a simple threshold function is used. However, more scientific statistical and/or neural network classifiers are preferred (Koppen-Seliger, Frank, & Wolff, 1995). When evaluating quantitative model-based fault detection systems, it should be noted that these techniques require a high modelling effort and are generally restricted to linear systems and some specific non-linear systems. For a general non-linear system, linear approximations can be poor and hence the effectiveness of the method can be greatly reduced. However, thanks to the method of disturbance decoupling, the robustness can be maximized by minimizing the effect of unknown disturbances, like measurement and process noise, and unmodelled process behaviour. In this approach, all uncertainties are treated as disturbances and filters are designed to decouple the effects of faults and uncertainties such that these can be differentiated (Frank & Wunnenberg, 1989; Viswanadham & Srichander, 1987).

2.3.

Qualitative model-based methods

As noted above, when the a priori domain knowledge is developed from a fundamental understanding of the process by means of physical process knowledge, it is called causal model-based knowledge. When the physics of the process is expressed as mathematical functional relations between inputs, outputs and states of the system a quantitative modelling approach is used as mentioned in the previous section. When the physical relationships are expressed by means of qualitative, non-quantified functions the term qualitative modelling is used. A distinction should be made between the causal models and the abstraction hierarchies (Venkatasubramanian, Rengaswamy, & Kavuri, 2003). In a first attempt, knowledge-based expert systems, which mimic the fault detection by human experts, were investigated as a tool for fault diagnosis. However, the rule base, which consists of ‘if–then’ rules, grows rapidly with increasing complexity of the system. Another problem of this approach is the lack of insight into the physics of the system which means that it will fail when new conditions are encountered that were not defined in the rule base (Venkatasubramanian, Rengaswamy, & Kavuri, 2003). The need for a reasoning tool which can model the system in

biosystems engineering 106 (2010) 26–36

a qualitative way and describe it by a causal structure which is not as rigid as a numerical or analytical model has led to the development of different qualitative modelling methods, like digraphs and fault tree structures (Venkatasubramanian, Rengaswamy, & Kavuri, 2003). A digraph is a graph with directed arcs between the nodes which represents the cause–effect relation of a system. The directed arcs lead from the ‘cause’ nodes to the ‘effect’ nodes. As a result, it is an efficient way of representing the observed symptoms or patterns of a fault in a graphical way. Maurya, Rengeswany, and Venkatasubramanian (2007) proposed a digraph-based fault detection framework to select a possible candidate set of faults based on the incipient response of the process. Fault trees are mainly used in analyzing the system reliability and safety. The tree has different layers with nodes and at each node logic operations like AND and OR are performed for propagation. Fault trees serve to represent the propagation path of a fault from their origin to their top level of occurrence. Another way of presenting model-based knowledge is through the development of abstraction hierarchies. These are based on the decomposition of the process system into different subsystems. The main idea is to gain insight in the overall process behaviour by inspection of the laws governing the different subsystems. The failure of a higher-level subsystem will be caused by the failure of one or more of the subsystems. The main source of malfunctioning can then be found by making use of a bottom-up description, which describes what various units with certain functions are used for and how these serve the higher-level systems. When evaluating qualitative model-based fault detection systems, it can be concluded that these techniques are of high value when an abundance of process experience is available which is not numerically detailed. One of the main advantages of qualitative methods based on deep-knowledge is that they provide an explanation of the path of propagation. However, their complexity will increase very rapidly with the complexity of the system and, in comparison with quantitative model-based techniques, they suffer from the resolution problem because no detailed interval or order of magnitude information is available.

2.4.

Process history-based methods

In contrast to the model-based fault diagnosis approaches where a process model is needed a priori, only a large amount of historical process data is needed in process history-based fault diagnosis methods. Different kinds of features are then extracted from these historical process data. The extracted features can be of qualitative and/or quantitative nature (Venkatasubramanian, Rengaswamy, Kavuri, et al., 2003). In the former case a distinction can be made between expert systems and trend modelling methods. An expert system typically consists of a set of heuristic rules derived from a knowledge base. Since considerable process knowledge is often available from experienced engineers and/or operators of the process plant, this can be incorporated. A fuzzy rule base serves as the ideal framework for the incorporation of human knowledge into a fault

29

diagnosis system. Several authors have discussed expert system applications for fault diagnosis of specific systems (Chester, Lamb, & Dhurjati, 1984; Henley, 1984; Rich, Venkatasubramanian, Nasrallah, & Matteo, 1989). In the case of qualitative trend analysis, the different process signals are monitored and the qualitative analysis of their trends provides valuable information for the identification of underlying abnormalities in the process. These trends can be extracted from a qualitative analysis of the shape of the dynamics of a sensor signal. Venkatasubramanian, Rengaswamy, Kavuri, et al. (2003) state that a suitable classification and analysis of process trends can detect the fault earlier and lead to a quick repair of the faulty sensor. When extracting quantitative features from a historical data set, the fault diagnosis problem can be solved by pattern recognition techniques. The main goal of pattern recognition is to classify the quantitative features into different predetermined classes based on the interrelationship of these features. The number of classes equals n þ 1, with n the number of faults to be isolated. An extra class is needed to cluster the data points which correspond to the normal mode of operation. These pattern recognition techniques can be broadly classified into statistical and non-statistical (neural network) ones. The earliest attempts to use statistics in on-line process monitoring are the univariate quality control of a process by Shewhart (1931) and the cumulative sum charts (Page, 1954). However, most processes in industry are multivariate by nature and the use of univariate control charts for each of the variables can be confusing and may lead to incorrect conclusions (Venkatasubramanian, Rengaswamy, Kavuri, et al., 2003). As a result, multivariate tools like principal component analysis (PCA, Pearson, 1901) and partial least squares (PLS, Wold et al., 1984) were investigated to reduce the dimensionality of the data set and to extract the most essential features from large multivariate data sets. At first an ‘in control’ PCA/PLS model is built based on historical process data. By projecting the new observations onto the plane defined by the PCA/PLS loading vectors of the nominal model, the scores and the residuals can be calculated. As a result, the multivariate control chart based on the Hotelling T2-statistic (Hotelling, 1947) can be plotted in order to detect faulty sensor behaviour. Other commonly used statistical classifiers are the Bayesian classifier and clustering algorithms. Traditionally used neural network classifiers are the supervised back-propagation algorithm, self-organizing maps and support vector machines. Some of them will be investigated further in detail in Section 4.2. When evaluating process history-based fault detection systems, it can be concluded that these classifiers are easy to develop and perform well in terms of robustness to noise and isolatability requirements. They require a low modelling effort and a priori process knowledge which makes them attractive techniques from an industrial point of view. However, these techniques lack explanation properties. Moreover, the generalization ability of process history-based methods is limited by the data range of the training data. Nevertheless, by using radial and/or ellipsoidal neural

30

biosystems engineering 106 (2010) 26–36

network functions and by avoiding a decision when no similar training patterns are available in that region, novel faults can be detected. Because of the many possible multiple fault combinations, the search for multiple faults by specifying them explicitly as different classes and obtaining training patterns is not feasible for this kind of history-based fault diagnostic techniques. This is a drawback compared to the quantitative model-based methods where one could explicitly include multiple fault identification in the design procedure.

3. Condition monitoring of agricultural machinery 3.1.

Recent and future trends in agricultural technology

Mechanization in the agricultural sector has given rise to higher productivity per farmer. Total agricultural production grew exponentially during the past century while the number of people involved in agriculture decreased. The industrialization of the 18th and 19th century led to a migration of the work force from the agricultural sector towards industry. As a result, the number of farms decreased while the average farm size increased. The decreasing number of farms along with multi-farm machinery use caused a decrease of the quantity of machines sold per year. In order to enhance productivity and to allow scheduled work to be carried out in time, more advanced agricultural machinery with wider working elements and combination of different soil or crop cultivation work steps into one machine were introduced. As a result, the output of agricultural machinery has been rising during recent decades (Kutzbach, 2000). More complex machines in combination with longer working days make it difficult for an operator to remain as focused at the end as at the beginning of the working day. Moreover, in large grain farming areas, many contractors are active that own different machines, operated by less experienced drivers. The latter ones are not able to operate the machines at their maximal capacity. To lighten this burden on the operators, many operator tasks on agricultural machinery have been automated during the last decades. In addition to automatic cruise control (Coen et al., 2007), the automatic guidance of farming machines has become a hot topic during recent years. Different automatic steering systems for farm machines were investigated and commercialised like mechanical sensing bars (Busse, Coenenberg, Feldman, & Crusinberry, 1977; Parish & Goering, 1970), optical sensors (Harries & Ambler, 1981), laser beam scanners (Coen et al., 2008), machine vision (Gerrish, Stockmann, Mann, & Gongzhu, 1986; Gunderson, Kirk, & Wilson, 1982; Jahns, 1976) and the use of global positioning systems (GPS) (Elkaim, O’Connor, Bell, & Parkinson, 1997; Larsen, Nielsen, & Tyler, 1994). Nowadays, the workload on the machine operator is further reduced by the introduction of automation routines, like turning routines with implement operation, engine/ transmission control according to different strategies, automatic tuning of the machine settings for harvesting, etc. For example, the settings of the threshing, separation and

cleaning section can be automatically adjusted to obtain optimal machine performance for different grain varieties and conditions (Craessaerts et al., in press). In the future, driverless, autonomous vehicles might determine the character of intensive agriculture (Kutzbach, 2000). However, for the successful operation of these assistance systems, it is important to detect process upsets, equipment malfunctions or sensor failures as early as possible and to find and remove the factors causing those events. Nowadays, the detection of an abnormal event, diagnosis of its causal origin and repair of the system are still performed by human operators. In the future, the complete reliance on human operators for the correct functioning of these automation systems will become increasingly difficult, due to the higher complexity and multivariable character of agricultural machinery processes. Given this difficult condition, it can be expected that human operators would easily make erroneous decisions and lose their confidence in these automation systems. This shows the need for the introduction of fault detection and diagnostic systems on agricultural machinery in the near future. The challenge for control engineers working on agricultural machinery will be to develop abnormal event detection systems which can answer the needs imposed by the manufacturers and operators of this type of machines. The development of abnormal event management systems has been given considerable attention in the process industry because of the high economic losses the chemical, pharmaceutical and petrochemical industry suffer when plant failure occurs. As discussed in Section 2 a wide range of strategies for process fault diagnosis ranging from analytical methods to pattern recognition techniques were developed during the last decades, with applications especially in process industry. Such an extended collection of techniques often poses a challenge to process engineers not familiar with these techniques, but willing to implement an abnormal event management system for their particular system. Therefore, an overview of the limited literature dealing with the implementation of fault detection and diagnosis systems in the agricultural sector will be given in the next section. A possible explanation for the limited number of research papers on this topic might be that this is limited to the in-house development by agricultural machinery manufacturers. However, the limited literature shows the importance of defining the desirable characteristics of a fault detection and diagnosis system for agricultural technology machinery. Thus, the main goal of the following sections will be to make a suggestion of the most suitable diagnostic methods from an agricultural machinery perspective.

3.2. A review of fault detection and isolation systems in agricultural technology Most applications of fault diagnostic systems in the agricultural industry are found in greenhouses. This can be explained by the fact that the greenhouse culture is the most intensive type of agricultural practice. Consequently, when control system faults inhibit or reduce plant growth, the effect may be disastrous for the whole production and result in high economic losses.

biosystems engineering 106 (2010) 26–36

Beaulah, Chalabi, and Randle (1998) developed an expert knowledge-based system to detect, diagnose and replace faulty sensor readings. Its effectiveness was demonstrated in a real-time environmental control of the microclimate in greenhouses. Linker, Gutman, and Seginer (2000) investigated the use of a model-based method for the detection and identification of single failures in greenhouses. The method relies on climatic measurements currently available in commercial greenhouses and combines hybrid physical/neural-network models with robust failure detection and identification theory for non-linear systems. Automated detection of drip line damage and clogged or damaged emitters for irrigation systems was investigated by Coates, Delwiche, and Brown (2006). Based on the measurements of pressure fluctuations along the irrigation line and a knowledge-based rule base, a correct fault diagnosis and explanation could be formulated. From these studies it can be concluded that automated fault detection and correction for irrigation systems has the potential to reduce water and fertilizer losses, and labour costs.

3.3. Specific characteristics of a fault detection and isolation system for agricultural machinery Agricultural machinery, like combine harvesters, sugar beet and potato harvesters, and baling machines, destined for harvesting and/or processing biological crops, have to deal with time and place-specific conditions. This explains the time-variant character of these systems. A change in crop variety, crop moisture, field slope, temperature, etc. may result in a different process characteristic. As a consequence, the first requirement of a fault diagnostic system for agricultural machinery is adaptability to changes. It should be possible to gradually develop the scope of the system as new cases and problems emerge, and as more information becomes available. Besides adaptability, the fault diagnostic system should be easy to use and robust, and it should provide a quick diagnosis of the process malfunction. The user-friendliness of such a system is essential because, in large arable land areas, many contractors are active that own different machines, operated by less experienced drivers. Rapid diagnosis is of high importance in order to prevent severe machine damage and/ or high economic losses. It should be mentioned that a quick response to failure diagnosis and robustness during normal operation are two conflicting goals. A quick detection system will be sensitive to noise and can lead to frequent false alarms during normal operation. From a modelling perspective, agricultural machines are complex non-linear systems which consist of several subsystems interacting with each other as a result of biological material, energy and information flows. The complexity of these machines in combination with the time-variant nonlinear character of these processes makes it highly difficult to model them by quantitative modelling techniques. However, a lot of a priori physical process knowledge needed for fault diagnosis may be inferred from experienced machine operators and engineers. Such knowledge can be incorporated into a causal qualitative-based model. This type of model can provide an explanation of the path of propagation of the fault. On the other hand, a lot of process data is available since the

31

recent generation of agricultural machines is equipped with a wide range of sensors and actuators to monitor the different sub-processes. As a result, statistical and/or artificial neural network-based pattern recognition techniques could be very well suited to be fault diagnosis techniques for agricultural machinery. As previously mentioned in Section 2.4, artificial neural network techniques allow quick detection, are robust to noise, are easy to use and implement, but they lack the explanation properties. However, neither of these techniques, qualitative model-based FDI systems and process historybased FDI systems, is adequate to fulfil all the requirements for an ideal fault diagnosis system (Section 2.4). In a hybrid framework of qualitative models and pattern recognition-based methods, both techniques could complement each other and result in better diagnostic systems. Integrating their complementary features provides a way to develop hybrid methods that could overcome the limitations of the individual solution strategies.

4. Case study: identification of sensor faults on a combine harvester by the use of intelligent artificial neural network techniques In this section, the general applicability of intelligent artificial neural network techniques like supervised SOMs and backpropagation neural networks will be illustrated for the detection and isolation of sensor failures on a New Holland CX combine harvester. By comparing the signals of a machine running in normal and faulty conditions, detection of sensor failure becomes possible. A quick detection and diagnosis of sensor failures and/or process failures on a combine harvester during the harvest season is of high importance in order to avoid a long breakdown time and consequently high economic losses. The internal processes of a combine harvester are too complex to model in an analytical way because of the interaction with biological crop materials. Moreover, recent trends in the automation of agricultural machinery show an increasing interest in the field of data-based modelling and control techniques (Craessaerts, Saeys, Missotten, & De Baerdemaeker, 2008; Maertens, 2004). Consequently, a lot of historical process data is available which could be of high value for training neural network-based fault diagnostic systems. The ability of neural networks to learn, self-organize and adapt the model structure/parameters based on new up-to-date measurement data makes them very suitable for this kind of application. Moreover, an intuitive and interpretable presentation of highdimensional data is of high importance for data analysts and machine operators. SOMs are a data-mining tool for visualization and interpretation of large high-dimensional data sets on a 2-dimensional grid. Consequently, the visualization of a SOM on the user display of a combine harvester would reinforce the operator’s feeling for the multivariate process.

4.1.

The combine harvesting process

Combine harvesters are large, complex machines sent out to all corners of the world to harvest different types of crops under all possible environmental conditions. The working

32

biosystems engineering 106 (2010) 26–36

process of a combine harvester can be divided into four different sub-processes: 1) cutting of crop and collection of harvested material; 2) the separation of grain kernels from larger crop parts such as straw; 3) the cleaning process, separating grain kernels from other small particles such as chaff and short straw; and 4) the temporary storage of clean crop material in the grain bin. The combine harvesting process is a highly uncertain process asking for sensors that extract immediate information from the process. Different measurement devices are installed to extract information from the combine harvesting process for automation purposes. The sensors used in this study were installed on a New Holland CX test combine and were recorded during field experiments with the CANbus Control Design Interface program (Craessaerts, Maertens, & De Baerdemaeker, 2005), developed for on-the-go design and evaluation of combine automation systems. The sensors used in this study are: Machine speed sensor: the rotation speed of the driving wheels is commonly used to estimate ground speed. - Feedrate sensor: a sensor which measures the driving torque of the header as a measure of the total crop flow into the harvester. - Grain Pan Load sensor (4): these sensors follow the principle described by De Baerdemaeker, Srivastava, and Lindemans (1989) to measure the crop load on the grain pan. - Walker Loss sensor: an impact sensor, installed at the end of the walker section to estimate the amount of grain kernels leaving the machine together with the straw. -

Two different sensor failure cases will be investigated in this study: a failure of one of the Grain Pan Load sensors and a failure of the Walker Loss sensor. These failures were induced by disconnection of the appropriate power supply cables.

4.2.

Artificial neural network techniques

Artificial neural networks have emerged as a powerful tool for pattern recognition. Like other pattern recognition techniques, neural networks act on data by detecting some kind of underlying organization. The networks can recognize spatial, temporal, or other relationships and can perform such tasks as classification, prediction and function estimation.

4.2.1.

Supervised self-organizing maps

The SOM is an artificial neural network methodology developed by Kohonen that forms a two-dimensional presentation of a multi-dimensional data set (Kohonen, 1995). During this transformation, the topology of the data is retained in the presentation such that data vectors, which closely resemble one another, are located next to each other on the map. An important characteristic of the SOM is the generalization of the information, which enables the classification of data vectors not used in the training of the SOM. The SOM can thus serve as a clustering tool for high-dimensional data that were not included in the training data set. Each neuron i of the SOM is represented by an n-dimensional weight, or model vector, mi ¼ [mi1, . ,min]T (n is the

dimension of the input vector). The neurons are connected to adjacent neurons by a neighbourhood relation, which dictates the topology or structure of the map.

4.2.1.1. Training of the SOM. During the iterative training procedure, the SOM forms an elastic net that folds onto the ‘cloud’ formed by the input data. The net approximates the probability density of the data (Kohonen, 1995). The model vectors tend to drift where the data are dense, while there are only a few model vectors where data are sparse. In a first step of the training procedure, the SOM is initialized linearly. The initialization is made by calculating the eigenvalues and eigenvectors of the given data. The map weight vectors are initialized along the two greatest eigenvectors of the covariance matrix of the training data. After initialization, the map is trained in a supervised way by both sequential and batch algorithms. The output neurons of the SOM are labelled based on a voting procedure and the labels of the training data set; a ‘0’ label corresponds to a normal operating regime, ‘1’ label corresponds to a failure of a Grain Pan Load sensor, ‘2’ label corresponds to a failure of the Walker Loss sensor. The training length, the number of times that the training data are presented to the neural network algorithm for model identification, was set at 10 epochs and held constant during the tests. A standard hexagonal lattice and Gaussian neighbourhood function are used during all training procedures. 4.2.1.2. Sequential training algorithm. The SOM is trained iteratively. In each training step one sample vector x from the input data set is chosen randomly and the distances between that vector and all the weight vectors of the SOM (mi) are calculated by means of a distance measure. The neuron whose weight vector is closest to the input vector is called the Best-Matching Unit (BMU) denoted here by c: kx  mc k ¼ minfkx  mi kg i

(1)

where k.jj is the distance measure, typically Euclidean distance. After finding the BMU, the weight vectors of the SOM are updated such that the BMU is moved closer to the input vector of the input space. The topological neighbours of the BMU are treated similarly. The SOM update rule for the weight vector of unit i is: mi ðt þ 1Þ ¼ mi ðtÞ þ aðtÞhci ðtÞ ½xðtÞ  mi ðtÞ

(2)

where t denotes the time, x(t) is an input data vector randomly drawn from the input data set at time t, hci(t) is the neighbourhood kernel around the winner unit c and a(t) is the learning rate at time t. The training is usually performed in two phases. In the first phase, a relatively large initial learning rate a0 and neighbourhood radius s0 are used. In the second phase the learning rate and the neighbourhood radius become smaller. This procedure corresponds to first tuning the SOM approximately to the same space as the input data space and then fine-tuning the map.

4.2.1.3. Batch training algorithm. The batch algorithm is also iterative, but instead of using a single data vector at a time, the whole data set is presented to the map before any adjustments are made. In each training step, the data are partitioned

biosystems engineering 106 (2010) 26–36

according to the Voronoi regions (Voronoi, 1908), i.e. each data vector belongs to the data set of the map unit to which it is closest. After this, the new weight vectors are calculated as: Pn

j¼1

mi ðt þ 1Þ ¼ Pn

hc ðtÞxj

j¼1

hc ðtÞ

combine harvester during wheat harvest. A data set of 44163 samples was registered at a sample rate of 5 Hz. In order to achieve a good end result, the following pre-processing steps were applied to the data set:

(3)

where c ¼ arg mini fkxj  mi kg is the index of the BMU of data sample xj. The new weight vector is a weighted average of the data samples, where the weight of each data sample is expressed as the neighbourhood function value hc(t) at its BMU c.

4.2.2. Multilayer feed-forward networks with backpropagation Feed-forward neural networks (Rumelhart, Hinton, & Williams, 1986) provide a general framework for representing non-linear functional mappings between a set of input and output variables. Such a network consists of one or more hidden layers and an output layer. Different multilayer perceptron architectures were evaluated for the detection of sensor failures. During all the tests, a neural network with one input layer, one hidden layer and one output layer was used. The input layer had seven nodes representing the seven input signals. The number of neurons in the hidden layer was changed between 2 and 14. A maximum number of 14 hidden neurons was used because more complex architectures led to excessive training times. The standard way to train a multilayer feed-forward neural network is using a method called backpropagation (Bishop, 1995). The key to the back-propagation algorithm is its ability to change the values of the network weights in response to errors. In order to calculate the errors, the training data must contain a series of input patterns labelled with their target output patterns. In this study, the number of output nodes in the output layer was set at 2 and was constant during the tests. The target values of the output nodes could have only binary levels representing the normal and sensor failure conditions. During the training stage, the target values of the two output nodes for the normal operating condition were set at 0. For failing grain pan sensor conditions, the first output node was set at 1 and the second output node was set at 0. Similarly for failing Walker Loss sensor conditions, the first output node was set at 0 and the second output node was set at 1. During back-propagation training, the network passes the derivatives of the output errors back to the hidden layer, using the original weighted connections. This backward propagation of errors gives the algorithm its name. Each hidden node then calculates the weighted sum of the back-propagated errors to find its indirect contribution to the known output errors. After each output and hidden node have found their error value, the nodes adjust their weights to reduce their errors. The Levenberg–Marquardt (LM) algorithm was used as optimization routine in this study.

4.3.

33

Results and discussion

The neural network approach for sensor fault identification was applied on a data set gathered with a conventional CX

Filtering of the data by a second order low-pass filter with a cut-off frequency of 0.15 Hz. - Normalization of the data. - Labelling of the data: data were subdivided into three different regimes; normal operation; failure of a Grain Pan Load sensor; failure of the Walker Loss sensor. -

The data set was separated at random into a training (2/3) and test set (1/3). This selection procedure of training and test set along with the training and testing of the different neural network model structures was performed 20 times in order to cope with the intrinsic variability of neural networks like the random selection of training and test data, random initialization of weights, etc. After pre-processing of the data, an evaluation was made of both intelligent methods (SOMs and back-propagation neural networks) for detection and isolation of sensor failures.

4.3.1. maps

Sensor fault detection by supervised self-organizing

Different SOM configurations and training algorithms were evaluated for detection of sensor failure. In this section, an overview will be given of the performances of the different map configurations and training algorithms. The result of a sequential training procedure of a 5  5 SOM is shown in Fig. 1. The U-matrix is shown along with the seven component planes and the label grid. By making use of the Umatrix, the distances between each map unit and each of its immediate neighbours is calculated. Distance matrix methods show the borders between the different clusters. Each component plane consists of the values of a single vector component in all map units. Different map configurations and training procedures were tested by evaluating the classification accuracy for independent test data. These test data were classified into one of the three different regimes by an auto-labelling algorithm. This algorithm assigns a label to the test sample by locating the new data vector in the trained label grid. For every new data input vector, the algorithm searches the BMU from the set of weight vectors and assigns the corresponding label to the new data vector. The classification of test data was classified as correct if the label, assigned by the auto-labelling algorithm, corresponded with the real label, assigned during the preprocessing phase. These classification results are illustrated in Fig. 2. Analysis of the results in Fig. 2 shows that: A sequential training algorithm achieves better classification results than a batch algorithm. - The classification performance increases with the map size until overtraining occurs. This occurred at a map dimension of 60  60 and implies less generalisation power. - Best classification accuracy (82.37%) was achieved with a 50  50 SOM and a sequential training algorithm. -

34

biosystems engineering 106 (2010) 26–36

Fig. 1 – Visualization of a SOM trained by a sequential training algorithm. ‘0’ label corresponds with normal operating regime, ‘1’ label corresponds with a failure of a Grain Pan Load sensor, ‘2’ label corresponds with a failure of the Walker Loss sensor.

4.3.2. Sensor fault detection by multilayer feed-forward neural networks with backpropagation The classification performances of different neural network architectures, considering independent test data, are illustrated in Table 1. Analysis of the results in Table 1 shows that:

framework of monitoring and visualization of the different process regimes. Consequently, these techniques could be of high value for designers of industrial fault monitoring systems. However, there is still space for improvement of these standard techniques. In future research, it is suggested that historical process data-based classification techniques should be combined with

No significant increase of classification performance was obtained by an increase in the number of neurons in the hidden layer. - No significant increase of classification performance was obtained by an increase in the number of the training cycles. - No overtraining occurred during the tests. - A classification performance of approximately 80% was obtained with a rather simple network (4 hidden neurons) and training procedure (10 training epochs). -

4.4.

Conclusions and future perspectives

It should be noted that for safety-critical fault diagnosis, an accuracy of 82% would not be sufficient because of the possibility of misclassifying unsafe events as being normal. However, this case study shows that within a short period of development and training a first prototype of a fault monitoring system for non-safety-critical systems could be designed based on the proposed techniques. Neural network techniques like SOMs offer interesting properties within the

Fig. 2 – Classification performance of different SOM configurations and training algorithms.

biosystems engineering 106 (2010) 26–36

Table 1 – Percentage of correct classification of different process regimes observed in the test data by making use of a multilayer perceptron neural network. Number of hidden layer neurons 2 4 6 8 10 12 14

10 training epochs

30 training epochs

77.62% 80.34% 80.67% 80.49% 80.78% 80.74% 80.65%

79.29% 80.85% 81.01% 81.08% 81.14% 80.98% 80.98%

causal qualitative model-based fault diagnosis techniques. Qualitative expert-based models are available from experienced combine operators and engineers, and could provide an in depth explanation of the origin of the fault and its propagation path in time. Moreover, the fault signatures used in this case study are based on steady-state information. The use of steady-state fault signatures can lead to significant time delays in the fault isolation procedure. In order to reduce the time delay in isolating a fault and to identify it more accurately, it is advisable to make use of dynamic data collected during the evolution of the fault. Finally, incremental learning techniques could be of high value for improvement of the above mentioned fault diagnostic system. A gradual change of the machine behaviour and/or the occurrence and detection of new faults would lead to a failure of the standard non-adaptive fault diagnostic systems, as presented in Section 4.3. By incorporating learning techniques, which find the optimal parameters of the fault diagnostic system automatically by observing the machine behaviour over a number of cycles, the diagnostic system can anticipate changing environmental conditions. It could for example happen that a fault is detected by an experienced operator, but not by the diagnostic system because of a change in process operating conditions. This information can then be presented in a supervised way to the fault diagnostic system by the operator, so that it can be used to update the classification algorithm by means of learning techniques. In this way, novel faults, which were not modelled adequately, can also be recognized and incorporated in the diagnostic system. Robustness and accuracy of these techniques should be investigated in future research.

5.

Final conclusion

The basic aim of this paper was to reveal the importance of introducing fault diagnostic systems on agricultural machinery. Based on the recent and future trends in agriculture, it was shown that the next challenge for control engineers involved with the automation of agricultural machinery will be to automate the supervisory control of sensor and process failures. Typical requirements of a fault diagnostic system for agricultural machinery are the adaptability to process changes, user-friendliness, quick detection and

35

robustness. Therefore, a hybrid framework of qualitative models and pattern recognition-based methods has been suggested as the most suitable fault diagnostic platform for agricultural machinery. The potential of SOMs and multilayer feed-forward artificial neural networks as fault diagnostic tools has been illustrated in a case study concerning sensor failure detection on a combine harvester. An artificial neural network procedure was presented for sensor fault diagnosis on combine harvesters. Two intelligent methods, namely supervised SOMs and multilayer feed-forward neural networks with backpropagation were evaluated within the framework of the development of a supervisory fault detection system. Different configurations of both neural network types were trained several times by a random training set and their classification performance was tested on an independent test set. Both neural network types showed comparable classification results for new test data. However, best results were obtained with a supervised SOM of 50  50 dimension. With this neural network configuration, it became possible to classify 82% of the new test data correctly. Finally, suggestions for improvement of these techniques were made, proposing hybrid fault diagnostic systems with incremental learning techniques as an interesting future research topic.

Acknowledgements The authors gratefully acknowledge the I.W.T.-Flanders (Institute for the Promotion of Innovation through Science and Technology, I.W.T.-Vlaanderen) for the financial support through project ‘Automatische regeling van het reinigingsproces in maaidorsers’ project number 020750 and CNH Belgium for its cooperation. Wouter Saeys is funded as a Postdoctoral Fellow of the Research Foundation – Flanders (FWO). Josse De Baerdemaeker is a full professor at the Katholieke Universiteit Leuven.

references

Beaulah, S. A., Chalabi, Z. S., & Randle, D. G. (1998). A real-time knowledge-based system for intelligent monitoring in complex, sensor-rich environments. Computers and Electronics in Agriculture, 21, 53–68. Betta, G., & Pietrosanto, A. (2000). Instrument fault detection and isolation: state of the art and new research trends. IEEE Transactions on Instrumentation and Measurement, 49, 100–107. Bishop, C. M. (1995). Neural networks for pattern recognition. Oxford University Press. Busse, W., Coenenberg, H., Feldman, F., & Crusinberry, T. F. (1977). The first serial produced automatic steering system for corn combines and forage harvesters. In Proceedings of the international grain and forage harvesting conference, Ames, IA, pp. 43–47. Chester, D., Lamb, D., & Dhurjati, P. (1984). Rule-based computer alarm analysis in chemical process plants. In Proceedings of seventh micro-Delcon, pp. 22–29. Clark, R. N. (1978). Instrument fault detection. IEEE Transactions on Aerospace and Electronic Systems, AES-14(4), 558–563.

36

biosystems engineering 106 (2010) 26–36

Coates, R. W., Delwiche, M. J., & Brown, P. H. (2006). Control of individual microsprinklers and fault detection strategies. Precision Agriculture, 7, 85–99. Coen, T., Saeys, W., Missotten, B., & De Baerdemaeker, J. (2007). Cruise control on a combine harvester using model-based predictive control. Biosystems Engineering, 99, 47–55. doi:10. 1016/j.biosystemseng.2007.09.023. Coen, T., Vanrenterghem, A., Saeys, W., & De Baerdemaeker, J. (2008). Autopilot for a combine harvester. Computers and Electronics in Agriculture, 63(1), 57–64. Craessaerts, G., Maertens, K., & De Baerdemaeker, J. (2005). A design environment for combine automation via CANbus. Computers and Electronics in Agriculture, 49(2), 233–245. Craessaerts, G., Saeys, W., Missotten, B., & De Baerdemaeker, J. (2008). Identification of the cleaning process on combine harvesters. Part I: A fuzzy model for prediction of the material other than grain (MOG) content in the grain bin. Biosystems Engineering, 101(1), 42–49. Craessaerts, G., Saeys, W., Missotten, B., & De Baerdemaeker, J. Fuzzy control of the cleaning process on a combine harvester. Biosystems Engineering, in press. De Baerdemaeker, J., Srivastava, A. J., & Lindemans, P. (1989). Measuring the material flowrate on the preparation table of a combine. ASAE meeting paper, Louisiana. Elkaim, G., O’Connor, M., Bell, T., & Parkinson, B. (1997). System identification and robust control of farm vehicles using carrier-phase differential GPS. In ION conference, Kansas City, MO. Frank, P. M. (1986). Fault diagnosis in dynamic systems using analytical and knowledge-based redundancy – a survey and some new results. Automatica, 22, 333–344. Frank, P. M., & Wunnenberg, J. (1989). Robust fault diagnosis using unknown input observer schemes. Fault diagnosis in dynamic systems: Theory and applications. NY: Prentice-Hall. Gerrish, J. B., Stockmann, G. C., Mann, L., & Gongzhu, H. (1986). Pathfinding by image processing in agricultural field operations. ASAE paper 86-1455. St-Joseph, MI: American Society of Agricultural Engineers. Gertler, J. (1995). Diagnosing Parametric Faults – From Identification to Parity Relations. Proc. American Control Conference, Seattle, WA, June 1995. pp. 1615–1620. Gunderson, D. G., Kirk, T. G., Wilson, J. N. (1982). Feasibility study of closed circuit TV as a guidance assist for field equipment. ASAE paper NCR, 82–117. Harries, & Ambler. (1981). Guidance system using opto-electronic techniques and a microprocessor based controller. Journal of Agricultural Engineering Research, 26. Henley, E. J. (1984). Application of expert systems to fault diagnosis. In AIChE annual meeting, San Francisco, CA. Himmelblau, D. M. (1978). Fault detection and diagnosis in chemical ´ msterdam: Elsevier Press. and petrochemical processes. A Hotelling, H. (1947). Multivariate quality control illustrated by the testing of simple bombsights. Selected techniques of statistical analysis. NY: McGraw-Hill. pp. 119–184. Jahns, G. (1976). Automatic guidance of farm vehicles – A monograph. Agricultural Engineering Department series no. 1. Auburn, AL: Agricultural Experiment Station, Auburn University. Kohonen, T. (1995). Self-organizing maps. Berlin: Springer–Verlag. Koppen-Seliger, B., Frank, P. M., & Wolff, A. (1995). Residual evaluation for fault detection and isolation with RCE neural

networks. In Proceedings of the American control conference, Seattle, Washington, pp. 3264–3268. Kutzbach, H. D. (2000). Trends in power and machinery. Journal of Agricultural Engineering Research, 76, 237–247. Larsen, W. E., Nielsen, G. A., & Tyler, P. A. (1994). Precision navigation with GPS. Computers and Electronics in Agriculture, 11, 85–95. Linker, R., Gutman, P. O., & Seginer, I. (2000). Robust model-based failure detection and identification in greenhouses. Computers and Electronics in Agriculture, 26, 255–270. Maertens, K. (2004). Data-driven techniques for the on-the-go evaluation of separation processes in combine harvesters. Ph. D. thesis, KULeuven. Maurya, M. R., Rengeswany, R., & Venkatasubramanian, V. (2007). A signed directed digraph and qualitative trend analysisbased framework for incipient fault diagnosis. Chemical Engineering Research and Design, 85, 1407–1422. Page, E. S. (1954). Continuous inspection schemes. Biometrika, 41, 100–115. Parish, R. L., & Goering, C. E. (1970). Developing an automatic steering system for a hydrostatic vehicle. Transactions of the ASAE 523–527. Patton, R. J., Chen, J., & Nielsen, S. B. (1995). Model-based methods for fault diagnostic: some guide-lines. Transactions of the Institute of Measurement and Control, 17(2), 77–83. Pearson, K. (1901). On lines and planes of closest fit to systems of points in space. Philosophical Magazine Series B, 2, 559–572. Rich, S. H., Venkatasubramanian, V., Nasrallah, M., & Matteo, C. (1989). Development of a diagnostic expert system for a whipped toppings process. Journal of Loss Prevention in the Process Industries, 3, 145–154. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal representations by error propagation. In D. E. Rumelhart, & J. L. McClelland (Eds.), Parallel distributed processing (part 1) (pp. 318–362). Cambridge, USA: MIT Press. Shewhart, W. A. (1931). Economic control of quality of manufactured products. Princeton. Venkatasubramanian, V., Rengaswamy, R., & Kavuri, S. N. (2003). A review of process fault detection and diagnosis Part II: qualitative models and search strategies. Computers and Chemical Engineering, 27, 313–326. Venkatasubramanian, V., Rengaswamy, R., Kavuri, S. N., & Yin, K. (2003). A review of process fault detection and diagnosis Part III: process history based methods. Computers and Chemical Engineering, 27, 327–346. Venkatasubramanian, V., Rengaswamy, R., Yin, K., & Kavuri, S. N. (2003). A review of process fault detection and diagnosis Part I: quantitative model-based methods. Computers and Chemical Engineering, 27, 293–311. Viswanadham, N., & Srichander, R. (1987). Fault detection using unknown-input observers. Control Theory and Advanced Technology, 3, 91–101. Voronoi, G. F. (1908). Nouvelles applications des parame`tres continus a` la the´orie de formes quadratiques. Journal fu¨r die reine und angewandte Mathematik, 134, 198–287. Willsky, A. S. (1976). A survey of design methods for failure detection in dynamic systems. Automatica, 12, 601–611. Wold, S., Albano, C., Dunn, W. J., III, Edland, U., Esbensen, K., Geladi, P., et al. (1984). Multivariate data analysis in chemistry. Chemometrics, mathematics and statistics in chemistry. Dordrecht: Reidel Publishing Co.