Research Nural Network

Development of an Effective Artificial Neural Network Model for Water Quality Assessment Using Remote Sensing Data Hanan Farag1 and Hanan Ezzat2 ABSTRACT Based on satellite synchronous monitoring session, a back propagation (BP) and radial basis function (RBF) neural network models are developed to detect turbidity from Landsat data. This paper proposes an analytical approach to identify the appropriate combination of remote sensing data as input variables for the developed ANN water quality model. The sensitivity of the changes in the individual input reflectance channels (wavelength bands) is examined. The proposed ANN models are applied in a case study area, Lake Burullus in northern coast, Egypt. The RBF model has a higher accuracy compared to BP model since their validation efficiency around 94% approximately for the training sample, and the error is controlled to be less than 10%. The output accuracy of RBF prediction model had root mean square error around 0.99. The results of the case study are reliable based on the input variables selection and indicate that selecting the appropriate input variable could reduce computing time required and improve the predicting results when applying ANN water quality model.

1. INTRODUCTION The concern about water quality in inland water bodies such as lakes and reservoirs has been increasing. Owing to complexity associated with field collection of water quality samples and remote sensing techniques for water quality information retrieval have been employed. Due to the limitations of linear regression methods, many researchers have employed the Artificial Neural Network (ANN) technique for water quality assessment for its power of non-linear regression. For water color remote sensing study of inland water body, terrestrial satellite is often a good data source, because it has high spatial resolution. However, the precision of water color remote sensing inversion limits its application to water environmental monitoring and pollution analysis. Chlorophyll-a (chl-a), suspended matter (SM), and dissolved organic matters are optically active parameters of lake water quality. Several investigators have successfully used Landsat-MSS/TM imagery in inland and estuarine water quality monitoring. In case of inland water, the water dynamics is more complex to have a linear relationship between the satellite spectral signatures and the water quality parameters. Panda et al., 2004 has reported that there is considerable scattering (in all visible and even in nearIR bands) from the lake water with high sediment and chlorophyll content. For nonlinear environmental processes, artificial neural networks (ANN) could be used since it needed nonlinear geophysical transfer functions (Haykin, 1999). Through learning procedures, ANNs have the power to approximate any non-linear relationship that exists between a set of inputs and their corresponding set of outputs (Lacroix et al., 1997). The 1

Researcher, Environment and Climate Changes Research Institute (ECRI), National Water Research Center (NWRC), Cairo, Egypt. 2 Head of GIS and Data Management Unit, Ground Water Sector (GWS), Ministry of Water Resources and Irrigation (MWRI), Egypt.

1

ANN techniques are based on the configuration of several prediction, classification, and time series estimation techniques. Zhuang and Engel, 1990, Ranaweera et al. 1995, and Panda and Panigrahi, 2000, have provided research evidences regarding the superiority of ANN modeling technique over the statistical process in the case of nonlinear data modeling. Keiner and Yan 1998, and Baruah et al.,2001, have established the importance of back propagation (BP) neural networks over the multiple regression technique while predicting water quality from Satellite imagery. Several researches have been conducted to use ANN in retrieving the water quality from remote sensing data. Panda et al., 2004, has been developed and compared Radial basis function network (RBF) model with a statistical model to predict chl-a and SM concentration using the Landsat-TM satellite imagery and simultaneously in-situ analyzed spatially distributed water samples from Beaver Reservoir. The paper aims to develop and compare the radial bases function network (RBF) and back propagation algorithm (BP) neural models to predict turbidity concentration using landsatTM satellite and simultaneously in-situ water samples. The best ANN model efficiency is used in investigating the highly accurate band or band combinations that detect the turbidity.

2. METHODOLOGY The methodology of this paper described in details in the following sections. It included a brief description of neural network algorithm models BP and RBF. MATLAB, Neural Networks Toolbox® 7 was used for building the RBF and BP models.

2.1

Neural Network Algorithm

The most neural network algorithm used in detecting water quality parameters based on remote sensing data is the BP and RBF. A brief description of the mathematical equation sequences for both of these algorithms. 2.1.1

Back propagation Algorithm (BP)

The back propagation learning algorithm can be divided into two phases: propagation and weight update. Phase I: Propagation Each propagation involves: a)

Forward propagation of a training pattern's input through the neural network in order to generate the propagation's output activations.

b) Backward propagation of the propagation's output activations through the neural network using the training pattern's target in order to generate the deltas of all output and hidden neurons. Phase II: Weight update For each weight-synapse follow: a) Multiply back propagation network output delta (difference between desired and network outputs) and input activation function to get the gradient of the weight. b) Back propagate the weight in the opposite direction of the gradient by subtracting a ratio of it from the weight. This ratio influences the speed and quality of learning; it is called the learning rate. The sign of the gradient of a weight indicates where the error is increasing; this is why the weight must be updated in the opposite direction.

2

Repeat phase I and II until the performance of the network is satisfactory. The network learning process starts with randomly chosen weights, while mean square error is unsatisfactory and computational bounds are not exceeded, it is repeated each input pattern. Computations has been done of the following parameters: hidden node

net   , hidden node outputs x   , inputs to the outputs nodes net  , the network outputs O  , the error between the network outputs and finally the desired output d . Modifications of the weights between hidden and output nodes can be inputs

1 p, j

1 p, j

( 2) p ,k

p ,k

p ,k

calculated from equation (1)





Wk(,2j,1)   d p,k  o p,k  net (p2,k) x (p1,)j

(1)

Finally, the weights between input and hidden nodes should be modified by equation (2)









W j(,1i,0)    d p ,k  o p ,k  net (p2,k) Wk(,2j,1) net ((1p), j ) x p , j

(2)

K

Where

net    is the hidden node inputs x    is the hidden node outputs net  is the inputs to the outputs nodes O  is the network outputs d  is the desired outputs 1 p, j

1 p, j

( 2) p ,k

p ,k

p ,k

2.1.2

Radial Basis Function Network (RBF)

The (RBF) architecture classification is depicted as in (Figure 1). It consists of three layers: the input layer (I), the hidden layer (H), and the output layer (O).

Figure 1. The RBF network architecture (MIPPR, 2005)

3

The input layer relies on as many neurons as input features. Input neurons just propagate input features to the hidden layer. Each neuron in the hidden layer (H) is associated with a Gaussian Kernel functions as follows:



u j  exp  ( X  c j )T ( X  C j ) / 2 2j



j=1,2,……N

(3)

Where, uj denotes the output of the j-th hidden neuron, input features vector X= (x1, x2, x3, ……,xn)T ,cj, σj, represent the center and width of Gaussian Kernel function respectively, and N is the number of hidden neurons. Equation (3) indicates that the output range of hidden neurons is from 0 to 1, and as the distance between X and cj decreases, the output increases. The output layer is composed of as many neurons as output classes to be recognized. Each output neuron computes a simple weighted summation over the responses of the hidden neurons. Let Oi be the input of output layer, it is calculated by equation (4)

Oi   j 1 wij u j N

i= 1,2,……,m

(4)

Where Wij is weighted connection between hidden layer and output layer.

2.2

Satellite Images and Water Quality Data Acquisition

Three Thematic Mapper (TM) cloud free images from Path/Row (177/38) covering Burullus Lake were used in this study. The TM images were precision corrected including radiometric and geometric corrections. Six spectral bands were specified: Band

wavelength

band 1 (TM1)

0.45 – 0.52 μm

band 2 (TM2)

0.52 – 0.60 μm

band 3 (TM3)

0.63 – 0.69 μm

band 4 (TM4)

0.76 – 0.90 μm

band 5 (TM5)

1.55 – 1.75 μm

band 7 (TM7)

2.08 – 2.35 μm

Band 6 (57 m resolution) was not used because of the image resolution dissimilarity with other bands that were 28.5 m in resolution. Three water sampling campaigns were collected over the period from 2002 to 2010 with different spatial positions. The total number of water sampling is 54 points. The Remote sensing data was taken at the same time of water sampling dates. Three LandSat images were processed. Figure (1) represents map of the water quality sampling spatial distribution over Lake Burullus,

4

Figure 2.

2.3

General Layout of the Water Sampling Spatial Distribution

Input Dataset

Three different input data sets were prepared using DN values of different band of TM images. The dataset 1 included six DN numbers of band: B1, B2, B3, B4, B5 and B7. Dataset 2 included the proposed eleven band combinations (BC1, BC2,…) of different bands. Dataset 3 included the bands plus band combinations (datset1 plus dataset 2) which result in 17 inputs. Table (1) shows eleven band combinations of dataset 2. Table 1. Various Band Combinations considered in Input data Band combination

Expression

BC1

(B1+B2+B3)/3

BC2

B1/B2

BC3

B1/B3

BC4

B2/B3

BC5

(B5/B7)/2

BC6

(B4+B5+B7)/3

BC7

(B1-B3)/(B1+B3)

BC8

(B1-B2)/(B1+B2)

BC9

(B2-B3)/(B2+B3)

BC10

B2+B3

BC11

B4+B5+B7

5

2.4

Development of (ANN) Turbidity Model

The developed ANN models is based on selecting of Input-output variables; model structure and estimating of its parameters, and validation of the identified model. In the following section, architecture of the turbidity ANN models are described.

2.4.1

Architectures of Artificial Neural Networks Models Applied

A typical RBF and BP models consist of three different layers with each successive layer fully connected by feed forward. There is no provision of weight between the input layer and the hidden layer (prototype) in the ANN models. A nonlinear transfer function is used at the hidden layer in the ANN models. The output layer in ANN models is linear. The model architecture of both ANN models was defined by providing its layer numbers. The architecture of 6-5-0-1, which means the network, has 6 input neurons/input processing element (IPE), 5 prototype nodes in the prototype layer, zero neurons in the hidden layer, and 1 output neuron (OPE). For datasets 1, 2, and 3, the ANN models were constructed with an initial architecture of 610-0-1, 11-10-0-1, and 17-10-0-1, respectively. “Delta rule” was defined as activation function of BP network as learning rule. The activation function of RBF network was sigmoid (nonlinear) transfer function which used for model learning. The models were developed to obtain the most reliable, intelligent ANN architecture for detecting turbidity model. Identification of the number of hidden neurons in the network that was responsible for capturing the dynamic and complex relationships among input and output variables. The trial-and-error procedure started with two hidden neurons initially, and the number of hidden neurons was increased to 10 during the trials with a step size of one in each trial. For each set of hidden neurons, the network was trained in batch mode to minimize the mean square error at the output layer. The training was stopped when there was no significant improvement in the efficiency, and the model was then tested for its generalization properties.

2.4.2

Validation and Testing

In most ANN applications in water resources, the data are divided arbitrarily into the required subsets. However, previous studies have shown that the way of data dividing can have a significant impact on the generalization properties of the model, Tokar and Johnson, 1999. Therefore, developing an ANN model in most of researches are divided the available data into training and validation sets prior to the model building, and in some cases a cross validation set is also used. Moreover, selecting validation structure is based on minimum error and maximum efficiency during training as well as testing. In this paper, the method which is proposed by Sudheer and Jain, 2004, for data division into a training set and a validation set has employed. The method of data division ensures representative samples from all ranges of data. Since the sigmoid function is used as the activation function, the model output and input were scaled appropriately to fall within the function limits (0 to 1) to get over the “saturation” in training. Thirty nine samples are

6

considered for training and fifteen samples for validation. The scaling was performed using the maximum value of the output variable in the dataset. The convergence percentage and root mean square error RMSE were computed for both BP, RBF ANN models.

3. RESULTS AND DISCUSSIONS The two ANN models are tested to obtain the most reliable, intelligent and accurate network model for turbidity by comparing BP and RBF network for dataset 1, 2, and 3. The RBF is getting high accuracy than BP network. The convergence percentage was 94.8% for RBF network whereas BP was 91.54%. This result indicates that the RBF is an ideal tool for environmental application problems, because local approximation of nonlinear input-output mapping results in the RBF models of fast learning with reduced sensitivity as mention by Haykins, 1999. Table (2) summarizes the comparison between the radial basis function and back propagation network results. Table (2) Comparison between RBF and BP algorithm results for prediction Turbidity Radial Basis

Back-Propagation

training

testing

training

testing

No. of iterations

44444

24444

44444

24444

learning rate

4.9

4.97

4.85

4.93

RMSE

4.4170

4.4230

4.4195

4.4283

94.8%

Convergence percentage

91.54%

Applying developed RBF turbidity model to select the best band/band combinations from the three datasets 1, 2, and 3 for predicting turbidity concentrations. In this stage of the paper the RBF turbidity model has been learned, the number of iterations reached to 10,000 iterations for achieving stable trained intelligent network. Training and validation efficiency percentage for the three input dataset is summarized in table (3).

Table (3) The highest Training and Validation efficiency for the different Input Dataset Band/ band Ratio

Training Efficiency (%)

Validation Efficiency (%)

B1

TM1

98.15

98.08

B2

TM2

98.14

97.89

B3

TM3

98.11

97.99

7

B4

TM4

98.25

98.08

BC1

(TM1+TM2+TM3)/3

95.37

95.22

BC6

(TM4+TM5+TM7)/3

95.57

95.40

BC9

(TM2-TM3)/TM2+TM3

92.30

91.89

It noted that Band 1, 2 and 3 are the more significant in predicting turbidity, while high accuracy obtained at band combination consists of these bands BC1 and BC9. Figure 3-a and 3-b show the relation between measured and predicted turbidity values for BC1 and BC9. The obtained results agree with review researches: short wavelength radiation in the blue and green regions of the spectrum is commonly used to sense in-water constituents due to its strong capability of water penetration than longer wavelength radiation. In addition Liu et al., 2003 reported that the spectral behavior of water parameters varies with the type of constituent, increasing in suspended particulates lead to increases in overall brightness in the narrow band reflectance spectra and highly significant correlations exist between turbidity and TM data. The correlation coefficient between suspended sediment and remotely sensed data peaks in the green/red wavelengths, Novo et al., 1991.

Measured_Turbidity_NTU

140 BC1

120 y = 1.0007x - 0.1973 R2 = 0.9999

100 80 60 40

Ypredicted Linear (Ypredicted)

20 0 0

20

40

60

80

100

120

140

Predicted_Turbidity_NTU

Figure 3-a.

Measured and predicted turbidity concentration obtained from RBF (BC1).

8

Measured_Turbidty_NTU

140 BC9

120

y = 1.0009x - 0.2211 2

100

R = 0.9999

80 60 40 Ypredicted

20

Linear (Ypredicted)

0 0

20

40

60

80

100

120

140

Predicted_Turbidty_NTU

Figure 3-b.

4.

Measured and predicted turbidity concentration obtained from RBF (BC9).

CONCLUSIONS

Two ANN models Back Propagation (BP) and Radial Basis Function Network (RBF) were developed for obtaining the reliable robust, intelligent turbidity model. RBF model is the best for predicting turbidity concentrations in Lake Burullus using spectral information from Landsat imagery. In RBF better results obtained for dataset 2 (band combination) than dataset 1 (individual bands). It indicates that, increase in parameters calculated as a function of available parameters may not improve prediction accuracy for RBF model. The visible bands (band 1, 2 and 3) in combination have reliable results in predicting turbidity The regression between measured and predicted turbidity concentrations was higher in RBF model R2 is about 0.99.

REFERECES Baruah, P., Tumura, M., Oki, K. and Nishimura, H. (2001). Neural, network modeling of lake surface chlorophyll and sediment content from Landsat TM imagery, the 22nd Asian Conference on Remote Sensing, 5-9 Nov. 2001, Singapore. Haykin, S. (1999). Neural Networks: A Comprehensive Foundation, Second Edition, Prentice Hall Inc., Upper Saddle River, NJ. Keiner, L. and Yan, X. (1998). A Neural network model for estimating sea surface chlorophyll and sediments from Thematic Mapper imagery. Remote Sens. Environ. 66, 153-165. Lacroix, R., Salehi, F., Yang, X., Wade, K. (1997). Effects of data preprocessing on the performance of artificial neural networks for dairy yield prediction and cow culling classification. Trans. ASAE, 40(3), 839-846.

9

Liu,Y., Islam, M. and Gao, J. (2003). Quantification of shallow water quality parameters by means of remote sensing. Progress in Physical Geography, 27(1): 24–43. MATLAB, (2010), Neural Network ToolboxTM 7, Users Guide. MIPPR, (2005), “SAR and Multispectral Image Processing”. Novo, E., Steffen, C. and Braga, Z. (1991). Results of a laboratory experiment relating spectral reflectance to total suspended solids. Remote Sensing of Environment 36, 67–72.

Panda, S. and Panigrahi, S. (2000). Analysis of remotely sensed aerial images for precision farming. ASAE Paper No. 003055, St. Joseph, MI. Panda, S., V. Garg, and I. Chaubey, (2004). Artificial Neural Network Application in Lake Water Quality Estimation Using Satellite Imagery. Journal of Environmental Informatics 4(2):65-74. Panda, S., Panigrahi, S. and Derby, N. (2004). Application of vegetation indices for crop yield prediction using data mining and neural network techniques. Precis. Agric. Ranaweera, D.K., Hubele, N.F. and Papalexopoulos, A.D. (1995). Application of radial basis function neural network model for short-term load forecasting. IEEE Proc. Gener. Transm. Distrib., 142, 45-50. Sudheer, K. and A. Jain, (2004). Explaining the Internal Behavior of Artificial Neural Network River Flow Models. Ydrological Processes 18(4):833-844. Tokar, A. and A. Johnson, (1999). Rainfall-Runoff Modeling Using Artificial Neural Networks. Journal of Hydrologic Engineering, ASCE 4(3):232-239. Zhuang, X. and Engel, B. (1990). Classification of multi-spectral remote sensing data using neural network vs. statistical methods. ASAE Paper No. 90-7552, St. Joseph, MI.

14