Comparison of Multivariate Calibration Techniques Applied to Experimental NIR Data Sets

Comparison of Multivariate Calibration Techniques Applied to Experimental NIR Data Sets VIÂ TEÂZSLAV CENTNER, JORGE VERDUÂ -ANDREÂ S, BEATA W ALCZAK, * DELPHINE JOUAN-RIMBAUD, FREÂ DEÂ RIC DESPAGNE, LUISA PASTI, RONEI POPPI,² DEÂSIREÂ-LUC MASSART, ³ and ONNO E. DE NOORD ChemoAC, Vrije Universiteit Brussel, Laarbeeklaan 103, B-1090 Brussel, Belgium (V.C., J.V.-A., B.W., D.J.-R., FD., L.P., R.P., D.-L.M.); and Shell International Chemicals B.V, Shell Research and Technology Centre, Amsterdam, P.O. Box 38 000, 1030 BN Amsterdam, The Netherlands (O.E.D.)

The present study compares the perform ance of different multivariate calibration tech niques applied to four near-infrared data sets when test samples are well within the calibration domain. Three types of problem s are discussed: the nonlinear calibration, the calibration using heterogeneous data sets, and the calibration in the presen ce of irrelevant inform ation in the set of predictors. Recommendations are derived from the com parison, which should help to guide a nonchemometrician through the selection of an appropriate calibration method for a particular type of calibration data. A ¯ exible methodology is proposed to allow selection of an appropriate calibration tech nique for a given calibration problem. Index Head ings: Calibration; Multivariate; Method comparison; NIR; Nonlinearity; Clustering.

INTRODUCTIO N Many papers devoted to multivariate calibration start by assuming that the property of interest y (responseÐ e.g., concentration) is linearly related to the set of predictors X [ e.g., near-infrared (NIR) spectra ] ; that the samples collected to build the calibration m odel are nicely (norm ally or uniformly) distributed over the whole experimental dom ain; and that all important sources of variation needed to properly m odel y are included in X. However, in real-life situations, the assumption about the model linearity is not always m et. Similarly, the distribution of the calibration samples is not always controlled (designedÐ i.e., hom ogenous), but natural (e.g., clustered), since the collected samples come from production. It is also hardly known to what extent the information included in the X variables is relevant to model the response y. A frequent task of chemometricians is to cope with this situation and to build a calibration m odel that yields reliable predictions. If the assumptions described above are severely violated, then the choice of the optimal calibration m ethod is not trivial. W hen the model linearity does not hold, one can (1) still rely on linear methods to account partly for nonlinearities; (2) transform data (preprocessing, introducing nonlinear terms of the original or the latent variables to the calibration data) and apply the linear m ethods; or (3) use local (data splitting, local modeling) or (4) typically nonlinear calibration techniques. In case of clustering, one has to decide whether it is preferable to develop one global calibration model or to split the data and build Received 23 November 1998; accepted 6 December 1999. * Permanent address: Silesian University, Katowice, Poland. ² Permanent address: University of Campinas, Brazil. ³ Author to whom correspondence should be sent.

608

Volume 54, Number 4, 2000

separate m odels for each cluster. W hen the variables recorded to build a calibration m odel y 5 f(X ) are numerous, or when they are collected without suf® cient knowledge about the chemical (spectral) properties of the studied system, many of them (e.g., whole spectral regions) can be completely irrelevant for predicting the property of interest y. In this situation, a calibration combined with the elimination of uninform ative, or the selection of relevant, variables m ight have a profound in¯ uence on the predictive ability of the ® nal calibration m odel. The aim of this study is to compare performance of different calibration techniques when applied to different types of NIR data sets and to propose a methodology that would assist a novice in choosing the appropriate calibration technique for a given calibration problem. To cover the large domain of calibration m ethods, the following approaches were included in the com parison: c Standard linear full-spectrum m ethods [ principal component regression, (PCR); partial least-squares, (PLS) ] . c Nonstandard linear full-spectrum methods [ PCR using total least-squares (PCR-T LS) ] . c Linear methods combined with variable selection/elimination performed in the original or in a transformed domain [ stepwise m ultiple linear regression (MLR) p rocedur e (S tep w ise); genetic algor ithm (G A ); Brown’ s method; uninform ative variable elimination PCR or PL S (UV E-PCR or UVE-PL S); relevant com ponent extraction PLS (RCE-PLS) ] . c Full-spectrum methods based on PCR/PLS to cope with a nonlinear problem [ nonlinear (NL) variants of PCR and PLS ] . c Linear local m ethods [ locally weighted regression (LW R); K-nearest neighbors (KN N); radial basis function PLS (RBF-PL S) ] . c Nonlinear m ethods [ neural networks (NN ) ] . The NIR data sets analyzed here were selected so that they represent different types of calibration: (1) the data set is homogeneous or clustered; (2) most of the X variables are relevant to m odel y, or many variables are uninform ative; (3) the calibration problem is linear or nonlinear. One of the presented data sets (WHEAT) was published as an intended reference data set; 1 the remaining three were obtained from industry. Although this collection of data sets is not exhaustive, it covers m ost typical calibration problems that can be encountered in practice (see Table I). The performance of the investigated m ethods is compared in terms of predictive ability of the ® nal calibration

0003-7028 / 00 / 5404-0608$2.00 / 0 q 2000 Society for Applied Spectroscopy

APPLIED SPECTROSCOPY

TABLE I. Description of the four experim ental data sets. Data set

Linearity/ nonlinearity

Clustering

WHEAT POLY-DAT GASOLINE POLYMER

Linear Linear Slightly nonlinear Strongly nonlinear

Minor (2 clusters on PC3) Strong (2 clusters on PC1) Strong (3 clusters on PC2) Strong (4 clusters on PC1)

model. Other criteria, such as the speed of m odel development, the need for a large calibration data set, etc., are of a secondary interest. All calibration m ethods were applied to the same calibration set and test set. The results presented were obtained for the test set. THEO RY The applied calibration m ethods are described further in the subsection below. The validation of the developed calibration models (the quanti® cation of their predictive ability) is treated in the following subsection. The third subsection is concerned with the preprocessing of the NIR spectra. A brief description of how to choose the param eters that in¯ uence the predictive ability of the ® nal model (e.g., how to decide on the optimal m odel complexity) is given in the ® rst subsection. This approach allows one, to a large extent, to reproduce the presented results (e.g., PCR, PLS). In the cases where algorithms start using random num bers (e.g., weights in NN, the ® rst set of solutions in GA) this serves as a m ethodology which allows one to obtain calibration m odels of a comparable, but not exactly the same, predictive power. A bold upper-case letter (X ) denotes a matrix, a bold lower-case letter (y) a column vector, and an italic lowercase letter (h) a scalar. The ``hat’ ’ symbol (e.g., yÃi) refers to a predicted characteristic (e.g., to the predicted response of the object i) and 9 denotes the m atrix (vector) transpose. W hen useful, matrix dimensions are indicated between parentheses, e.g., X (n, p). Description of the Applied Calibration Techniques. Principal Component Regression and its Variants (PCRS, Poly-PCR, NL-P CR, TLS-PCR). PCR is one of the oldest and m ost frequently used m ultivariate calibration techniques. This m ethod includes two steps: (1) the approximation of the original (large) data m atrix X (n, p) by a sm all set of a orthogonal latent variables [ principal components (PCs) ] T (n, a); and (2) the development of a multiple linear regression model of y (n, 1) on T, y 5 f(T ). The original PCR algorithm introduces PCs successively into the m odel, according to the amount of variance they explain, i.e., PC1, PC2, etc. This approach is called PCR top-down (denoted PCR in this paper). Because all variance in X is not necessarily related to the variance in y, another variant, called best subset selection (PCRS), can be used instead. In this case, those PCs are selected and considered in the model that lead to the best predictive ability. Different approaches exist to perform the selection; 2±7 all of them are, however, related to the ranking of PCs according to their correlation with y, or ranking according to the improvement of prediction obtained by using the calibration set (cross-validation). In this study, the selection based on correlation is considered. In order to account partly for nonlinearities, several

modi® cations of the original linear PCR algorithm have been proposed. For instance, the squared (e.g., PC1 2, indicated in section results as 1 2) and the cross-product terms (e.g., PC1´PC2, indicated as 1´2) of the linear principal com ponent scores (Poly-PCR) 8 can be incorporated in the model. The poly-PCR algorithm applied here in addition includes the selection of the PC terms according to their correlations with y (which is similar to PCRS). Another alternative is to incorporate nonlinear transform s of the original variables into the data matrix X (in this study the squared original variables) and to perform the ordinar y PCR on the extended X matrix (NL -PCR).9 ±11 Once the optimal complexity of the PCR m odel has been estimated, an elimination of the variables that are irrelevant to m odel (and to predict) the property of interest y can be carried out. This capability is especially important in the case of NL -PCR, where the number of variables in X is double, compared to the original number. For m ore details on the variable elimination see below. There is also the possibility of replacing the ordinar y least-squares (OLS) regression in PCR by the total leastsquares (T LS-PCR). Although it is usually stated 12,13 that the TLS m odel does not give better prediction than the OLS model, we apply TLS here for reasons of comparison (for m ore details on TLS see Ref. 14). In general, it is expected that TLS-PCR and PCR should perform similarly, since the level of noise included in the ® rst (or in the ® rst selected) PCs is relatively low. The complexity a of the PCR models is usually determined by cross-validation. This method was applied also in this study. The global minimum on the root-meansquared error of prediction (RMSEP) vs. the model complexity plot, obtained by using leave-one-out cross-validation, was considered as a ® rst estimate of the optimal a. Then, the randomization test of van der Voet 15 has been applied to avoid a possible over® tting. This m ethodology was followed when using all PCR variants: PCR, PCRS, Poly-PCR, NL-PCR, and TLS-PCR. Partial Least-Squares and its Variants (PLS, SplinePLS). The basic idea of PLS 16 is similar to PCR, i.e., to extract the essential information from X in order to model y, and to discard the noise. Compared to PCR, PLS directly focuses only on the systematic variation in X that is correlated with y. This type of inform ation extraction can be interpreted as a com promise between PCR (maximizing the variance of scores t) and OLS (m aximizing the correlation between predictors and y). PLS can be mathematically described as follows (the original NIPALS algorithm17 for a multiple Y ): 1. Decompose (in each hth iteration, where h 5 a) the (centered) matrix X and Y: X 5

th p h 9 1

Eh

Y 5

uhqh 9 1

F h*

1, . . . , (1)

2. Establish the hth linear inner relation between t h and u h , i.e., the relation between scores from X and Y: u h 5 f(t h): uh 5

b hth 1

dh

(3)

3. Build a m odel of Y on T h, Q h 9 , and B h in order to obtain the matrix of the response residuals F h : APPLIED SPECTROSCOPY

609

Y 5 where T h 5 . . . b h]

[ t1

T hQ h 9 B h 1

. . . t h], Q h 5

Fh

[ q1

. . . q h ] and B h 5

(4) [ b1

4. To perform iteration h 1 1 go to point 1 and replace X of Eq. 1 by E h and Y of Eq. 2 by F h . The maximal number of iterations a (i.e., the complexity of the PLS m odel) m ust be speci® ed prior to starting the algorithm . For the sake of simplicity, it is usually stated that PLS maximizes the covariance between X and Y (m inimizes the norm of the residual F ). De Jong and Phatak 18 show that the maximized criterion is (u 9 t) 2. Since the original linear PLS algorithm is one of the most popular calibration techniques, there was some interest in developing a nonlinear variant of this method. For this reason, the Quadratic PLS,19 NLPLS,20 and PLS with spline inner relation (Spline-PLS) have been proposed. 21 Compared to the ordinary PLS, the only modi® cation lies in the inner (originally linear) relation of Eq. 3 uh 5

f(t h) 1

dh

(5)

In the case of Spline-PL S, which is considered here as the m ost often applied alternative, f(t h ) is no longer linear, but a spline functionÐ a piecewise polynomialÐ relating the X scores (t h ) to the Y scores (u h ) after extracting h latent variables. d h denotes the vector of residuals from the hth inner relation. The piecewise polynomial f(t h ) usually is of the second or the third degree. The individual polynomial pieces join at the knot with continuity constraints. 22 The performance of Quadratic PLS, NL PL S, Poly-PLS (which uses zero knots and the polynomial of the second degree), and NL-PLS (PL S on the original and the squared original variables) is not investigated here. It is assum ed that the m ore ¯ exible SplinePL S model gives predictions that are representative for the whole fam ily of nonlinear PLS m ethods. The optimization of the complexity of PL S models (PLS, Spline-PLS) developed in this study was carried out by means of (leave-one-out) cross-validation and the randomization test (see above). Variable Selection/Elimination. Brown’ s Method of Variable Selection (Brown’ s Method). The variable selection m ethod of Brown 23±25 consists of two steps. First, the variables are ranked by using the so-called signal-to-noise ratio: b j 2 /s j (e) 2 , where j 5 1, . . . , p. b j is the estimated slope in the univariate calibration model of x j on y and s j (e) 2 is the estimated variance of residuals (e) obtained from this model. This criterion is equivalent to sorting variables x j according to their correlation with y. 26 Second, the num ber (a) of variables that should be considered in the optimal model is determined by using the length of the con® dence interval. 23±25 The ® nal model is built as a weighted sum of the a selected variables. Stepwise Multiple Linear Regression (Stepwise). The most popular classical method to select a small subset of variables from the original set of predictors X is the stepwise MLR procedure. 27 In Stepwise, the variable x j that is most correlated with y is selected ® rst, the univariate regression model of y on x j is built, and the obtained regression coef® cient b j is tested for signi® cance by using a t-test at the considered critical level a 5 5 or 1% (in610

Volume 54, Number 4, 2000

dicated as Stepwise 5 or 1%, respectively in the tables). When b j ± 0, the selection proceeds by including a following variable in the model, x k ; namely, the one that yields the highest partial correlation coef® cient (PCC). This is called forward selection. The (forward) selection based on PCC is equivalent to examining the correlation between residuals (e) obtained from the m odel with all the variables already included y 5 b jx j 1 e and all the predictors not entered in regression at this stage (k 5 1, . . . , p where k ± j). The signi® cance of the regression terms entered into the model (b jx j and b k x k ) is then tested, and the nonsigni® cant terms are eliminated from the equation. This is called backward elimination. The forward selection and backward elimination are repeated until no signi® cant improvement of the m odel ® t can be achieved by including m ore variables, and all regression terms already involved are signi® cant. Since the criterion utilized in the Stepwise selection procedure is based on data ® tting, the ® nal MLR model may be over® tted. To decrease the danger of over® tting, we applied a (leave-one-out) cross-validation and the random ization test.15 The predictive ability of the model including all variables selected using Stepwise MLR and the m odel including all variables except the one yielding the lowest PCC (x z ) was evaluated. W hen the measure of prediction errors achieved with the less complex model was lower than the other one, or when the random ization test indicated that the predictions from both m odels are comparable, then the zth variable was removed. This elimination was repeated until no improvement could be reached by the variable exclusion. Finally, the shrunken Stepwise MLR m odel was applied to predict y of the test samples. Genetic Algorithm (GA , GA-FT ). Genetic algorithms are optimization tools, 28 simulating the process of life evolution, which can be used to select a sm all subset of original variables that should cover all sources of systematic variation in X, in order to m odel properly the response y. Such a well-chosen subset is further considered to build a multivariate calibration m odel using MLR. 29±33 The algorithm applied here33 starts from a set of random solutions (subset of variables, also called ``strings’ ’ , or ``chromosomes’ ’ ) and estimates the ® tness of each solution, i.e., its quality in its environment. In the case of feature selection, the ® tness is estimated in terms of cross-validated RMSE P. With a probability depending on their ® tness, pairs of solutions are then selected to undergo the two GA operators: cross-over (m ixing of the solutions) and m utation (random change in a solution, occurring with a very low probability). This procedure is iterated a certain number of times, until convergence to a good solution occurs. The GA applied here requires a num ber of input param eters. Depending on the com plexity of the m odeled system, the parameters have been adjusted as follows: the num ber of generations: 150 ±250; the num ber of strings in each generation: 20 ±50; the maximal number of variables in each string: the complexity of the optimal PL S model 1 1 (2); the maximal acceptable RMSEP value for the selected string: the optimal PL S RMSEP (1 10%); the frequency of cross-over: 50% , of the m utation: 1/(number of the original variables) and of the backward elimination: once after 50 generations.

Since data sets used for purposes of multivariate calibration usually contain m any correlated and m any irrelevant variables, one m ay prefer to transform the original data ® rst (so that the size of the data m atrix X is reduced), and only then to apply GA to select the optimal subset of variables. A typical example is the application of the power spectra (PS) coef® cients34 obtained by the Fourier transform ation (FT). In this case, the variable selection is carried out in the frequency domain, from the ® rst PS coef® cients (the ® rst 50 in this study) only (denoted as GA-FT ). Uninform ative Variables Elimination PL S (UV E-PL S, UVE -PCR, UVE-PCRS, NL-UV E-PCR). UVE -PLS35 is a variable elimination m ethod, aiming at improving the predictive ability of the ® nal bilinear model by rem oving the information from the set of predictors X that is not related to the m odeled property y. The criterion used to distinguish the inform ative from the uninform ative variables is the reliability (stability) of the PLS (PCR) regression coef® cients b: c j 5 b j /s(b j ), where b j 5 m ean (b j) and s(b j) 5 standard deviation of b j, obtained by (leave-one-out) jackkni® ng for each variable j. The (cutoff) level, below which the criterion c j is considered to be too small, indicating that the variable j is uninformative and should be removed, is estimated by using a matrix of random variables (R) attached arti® cially to the experimental data. The complexity a of the ® nal m odel developed on the subset of the retained inform ative variables is optimized by lowering a and computing RMSE P values as described in Ref. 35. The input parameters required by the algorithm, applied in the current version, are the cutoff level: 99%; the dimension of R: n 3 p (where p . 200); and the constant used to scale R to a sm all variance: 10 2 10 . R elevant Com pon en t E xtr action- P L S (R CE -P L S ). RCE-PL S36 is a m odi® cation of the previously described UVE -PLS algorithm. The criterion applied to select the relevant features is the same in both m ethods, i.e., the reliability of the b coef® cients. The difference lies in the data presentation and in the cutoff level estimation. In contrast to UVE, the selection of variables using RCE is performed in the wavelet domain, and the level at which the criterion is considered to be critical is estimated without adding the arti® cial random variables to the original data. The RCE-PLS m ethod proceeds as follows: First, the mean spectrum of the calibration set is used (1) to determine the optimal ® lter (among the 10 members of the Daubechies family of wavelets in this case) and (2) to distinguish the wavelet coef® cients signi® cant for the signal reconstruction, and the noise coef® cients (applying the minimum description length 37,38 ). Once the optimal ® lter (wavelet) is selected, all spectra are decomposed and presented in the optimal basis. The stability of the b coef® cient is then calculated for each wavelet coef® cient by leave-one-out jackkni® ng. The cutoff level is estimated by using b associated with the noise wavelet coef® cients. The ® nal model is built including only the reliable wavelet coef® cients. Local Modeling. K-Nearest Neighbors (KNN). KNN is a nonparametric method, well known in the ® eld of pattern recognition.39 The application of KNN for calibration purposes is relatively rare, 40 but possible. The algorithm starts searching in the X space for the K nearest calibra-

tion (x cal ) neighbors of each test object i (x test, i ), using a distance m easure (e.g., the Euclidean distance \x test 2 x cal \). Once the nearest neighbors are found (index j 5 1, . . . , K ), the yÃtest, i value is calculated by pooling (weighting) their corresponding y values (y cal, j ): yÃtest,i 5

O

K

O

j5 1

\ x test,i 2 K

j5 1

x cal, j \2 1 y cal, j (6)

\ x test,i 2

2 1

x cal, j \

In this study, the Euclidean distances were calculated in the original domain (a similar computation performed in the PC space would require an estimation of one additional parameter, i.e., the optimal number of PCs). The number of nearest neighbors (K ) was optimized by (leave-one-out) cross-validation. Locally Weighted Regression (LWR). Locally weighted regression was introduced into the ® eld of m ultivariate calibration as a m ethod of dealing with nonlinear problems using linear local (PCR) calibration models. 41,42 Similarly to KNN, LW R starts by ® nding those calibration samples for each new object i that are closest to it (the nearest neighbors). With this aim a distance measure such as the Euclidean or the Mahalanobis distance is applied. Only the selected neighbors are used further to build a local calibration model and to predict the property y of the new object i, i.e., yÃtest, i . Before the local m odel is built, the importance of the neighbors can be adjusted by weighting, using the distances calculated earlier. There exist several variants of LWR 41±45 that differ in the way in which the distances are measured and weighting is carried out. In the current study we show only the predictive ability of the best LWR model (obtained by comparing LWR models that use the Euclidean/Mahalanobis distance, no/cubic weighting, and PCR/PL S modeling, respectively). The com plexity of the LWR m odel and the num ber of nearest neighbors (K in the tables) was optimized by cross-validation. Radial Basis Functions - PLS (RBF-PL S). The idea of RBF-PL S46 is som ewhat similar to LW R. The algorithm starts by replacing the original data m atrix X (n, p) by a so-called activation matrix A (n, n). The elements of A (a ij) are nonlinear m easures of the distance between all pairs (i and j) of calibration objects. A could be called a nonlinearly transform ed distance matrix. Second, the ordinar y (linear) PLS model is developed to relate the transform ed distance measures (A) to the property of interest y. Accordingly, in the prediction phase, x test (1, p) is replaced by a test (1, n), where the elements a test, j are the transform ed distances between the actual test object and all calibration objects j. Finally, the above developed PL S model is applied to predict y test from the distance vector a test . Mathematically speaking, a ij and a test, j are determined by the output of the Gaussian functions (radial basis functions), where the number of Gaussians is equal to n, and the centers of these functions are de® ned by the coordinates of the calibration objects 2 a i j 5 e \ x i 2 x j \ /s j for i 5 a test, j 5

e \ x tes t 2

1, . . . , n

and

j 5

1, . . . , n

x j \ / s 2j

APPLIED SPECTROSCOPY

(7) (8) 611

\#\ denotes a distance m easure, here the Euclidean distance; x j is the center of the jth Gaussian function, s j describes its width, x i refers to the ith calibration object and x test to the test object; n denotes the number of objects in the calibration set. The activity matrix A is a squared matrix (n, n), having ones on the diagonal. The a ij coef® cients are linear combinations of the outputs of the Gaussian functions. One global RBF-PL S m odel is used for all x test , which is different from LW R. The local weighting in the global RBF-PLS model is achieved via the Gaussian functions. The RBF-PL S param eters to be optimized for each calibration problem are the width and the num ber of Gaussian functions (i.e., the complexity of the PL S model). Similarly as for NN, the optimization was carried out by means of prediction testing (training and monitoring set). Neural Networks (NN, OBS-NN). Neural networks belong to the fam ily of nonparametric calibration tools. The idea behind neural com putation is to mimic the massively parallel architecture of the hum an brain, by processing X data between m ultiple interconnected nonlinear units called nodes. In multivariate calibration, one takes advantage of the ability of NN to map complex functional relationships between X and y. NN are particularly useful when the functional relationship is unknown or nonlinear,47 but they can also correctly model linear data. X data are presented to the NN in the input layer and modeled by the hidden layer nodes. Responses of hidden nodes are then forwarded to the output layer where the NN produces its estimated response yÃ. The NN is trained to m inimize the residuals between the estimated and the reference y; i.e., e 5 S (y 2 yÃ) 2. Training samples are repeatedly presented to the NN, and after each iteration values of connections w i between the nodes (weights) are modi® ed with respect to the magnitude of the error e. The ¯ exibility of an NN is determined by its topology, i.e., by the number of nodes and weights in the NN, and by the nature of the transfer functions (linear or nonlinear) associated with the different nodes. The NN topology is de® ned by the user. Its optimization is a critical step in neural computation. The num ber of nodes and weights in the NN should be set as low as possible to reduce the num ber of degrees of freedom and to keep the problem overdetermined. This is why the NIR spectra analyzed here were compressed to a few PC scores, before inputting them to the NN. In order to reduce further the number of nodes and weights in the NN, different methods have been used: c A m ethod based on direct visualisation of hidden and input node contributions to the ® nal model, according to the relative magnitude of the weights and to the contribution of each input variable to the variance of the predicted response. 48 This m ethod will simply be referred to as ``NN’ ’ . c A new m ethod called ``Optimal Brain Surgeon’ ’ 49 that allows one to evaluate the effect of each weight deletion on the global error. This m ethod is called OBSNN. For both methods, weights updating was performed by the Levenberg±Marquardt50 optimization algorithm. Hyperbolic tangent was used in the hidden layer, when hy612

Volume 54, Number 4, 2000

perbolic tangent or linear functions were used in the output modes. O ptim ization of the Calibration Models and Validation of the Predictive Ability of the Optimized M odel. The developm ent of any m ultivariate calibration model should always include two basic steps: (1) the optimization of the calibration m odel for each calibration method (e.g., the optimization of the model complexity in PCR/PLS, of the number of neighbors in LWR or KNN, of the topology of NN, etc.) and (2) the validation (quanti® cation) of the predictive power of the developed model. There are two similar strategies of selecting the optimal m odel for one calibration method, namely, cross-validation51±55 (the term ``validation’ ’ is unfortunate but is often applied in this context) and the prediction testing (splitting data into the training and the monitoring set56±61). Depending on the calibration m ethod used, one of these approaches is applied here (see above). Cross-validation was used to optimize the complexity of the PCR (and its variants, including LW R) and PL S (and its variants) m odels, to select the small subset of variables with the Stepwise/GA algorithm, and to ® nd the optimal number of nearest neighbors in KNN and LW R. The prediction testing was utilized to reach the optimal topology of NN and to optimize the complexity of the PLS m odel and the width of the Gaussian functions in RBF-PLS. In both cases a test set was kept aside, which did not contain calibration (training or monitoring) objects. This set was used for the ® nal validation (quanti® cation) of the predictive power of the calibration models. In multivariate calibration, there is some concern about how to carr y out the data splitting 62 into the calibration set and test set. For this reason, we start from the of® cial recomm endations concerning the properties of an ideal calibration and test set, indicated in the ASTM guidelines. 63 The guidelines specify two groups of requirements, requirements about the sample’ s variability and the number of samples. The selected calibration set must span (exceed) the range of variation in the concentrations of all com ponents expected for samples which are to be analyzed in the future. This approach guarantees that the future predictions will involve only the interpolation of the developed calibration m odel. In addition, the calibration samples should be selected so that they cover the whole experimental domain and not only extremes. The ideal distribution of samples in the multidimensional space is uniform. The number of calibration samples, n, used to describe the relationship between the set of predictors X and the property of interest y has to be suf® ciently large. The ideal size of n, however, depends on the com plexity of the calibration problem and cannot be speci® ed before the modeling and the determination of the m odel complexity starts. Similarly, an ideal test set should contain samples that are uniformly spread over the whole expected range of variation in concentrations of all chemical com ponents present in future samples. The number of test samples should be also large enough. 63 In this study, the Kennard and Stone (KS) algorithm 64 was applied to split the considered data sets into the cal-

among the rem aining points are assigned to the test set, the next two again to the calibration set, etc. The RMSEP 16 was considered as a m easure of the predictive ability of the calibration models at the stage of the m odel optimization, and also in the ® nal validation using the test set: RMSEP 5

F IG . 1. Data set POLY-DAT. The position of the calibration (´) and test (C) samp les, selected using ( A) the Kennard and Stone and (B) the duplex algorithm , in the PC1±PC2 space.

ibration and test, and the training and monitoring set, respectively. This algorithm selects a set of objects that ful® lls the requirements on the calibration set speci® ed above: it selects extreme objects as calibration samples, and the rest of the experimental dom ain is uniformly covered by calibration samples. In doing so, we ensure that an optimal calibration situation is achieved, because we believe that analytical chemists will endeavour to reach such a situation when possibleÐ for instance by designing 65±67 a calibration set or by selecting samples for it on the basis of chemical knowledge. W hen the practical situation does not allow one to follow the AST M guidelines, the recomm ended approach described here does not constitute an optimal validation: the estimated m easure of the predictive ability m ight be over-optimistic. In an effort to study this possibility, in one case (data set W HE AT) both KS and the data splitting applied by Kalivas1 were compared, the latter requiring in the prediction phase a m ild extrapolation outside the calibration domain for a few objects. In another case (data set POLYDAT), KS selected for the test set a set of samples that was clearly too centrally located (see Fig. 1A). In that case we instead applied the duplex algorithm 61 to split the data set, because this algorithm ensures that the two sets (calibration and test) will span approximately the same range and will have similar statistical properties (see Fig. 1B). Similarly as for KS, the duplex algorithm starts by selecting the two calibration points that are farthest apart. In the next step, the two objects that are farthest apart

!O

nt i5 1

(yÃ2 i

y i ) 2 /n t

(9)

where n t refers to the num ber of objects in the test set, y i is the property of interest (e.g., concentration) obtained with a reference m ethod for the object i, and yÃi is the concentration of this object predicted by applying the developed calibration m odel. Data Preprocessing. NIR spectra often include information that is related to physical properties of the m easured samples, and not to the chemical properties of interest. A typical example is the particle size, causing a baseline shift of spectra. This shift is not relevant for the prediction of chemical composition and should be removed when the chemical composition is the subject of calibration. A number of m athematical m ethods were developed to rem ove the baseline shift/drift. The techniques applied in this study are the offset correction, the standard normal variate (SNV), and the ® rst derivative. In the offset correction, the row average of a few (e.g., ® ve) ® rst variables (columns) is calculated and subtracted from each element of the corresponding row. This procedure is repeated, row by row, through the whole matrix X. SNV corrects each spectrum i (row) separately by subtracting its row m ean from all elements of the spectrum, and by norm alization in the row direction. This approach enables one to eliminate the baseline shift as well as the drift. The ® rst-derivative spectrum is a vector, whose elements are obtained as the differences between the neighboring points within one raw spectrum (row). These differences are calculated on m athematically smoothed spectra, because, without the sm oothing, the obtained ® rst-derivative cur ves would be too noisy. The result of the ® rst-derivative data preprocessing is the elimination of the baseline shift and the separation of overlapping peaks. Proper selection of the pretreatm ent is an important step in the development of a m ultivariate calibration. The selection of the pretreatment might in som e cases have an effect on the selection of calibration m ethods, because, for instance, nonlinearities can be introduced or reduced. However, this is not the case for the data sets treated here. EXP ERIM ENTAL W HEAT. The data set WHEAT was submitted to the database of Chemometrics and Intelligent Laboratory Systems 1 by Kalivas as a proposed standard reference data set. It consists of 100 NIR spectra of wheat samples and their protein and m oisture content values. Because of the poor precision of the protein values, only m odels for moisture were developed and compared in this study. The moisture values range from 12.45 to 16.94%. Their precision was not reported by Kalivas, but it is known from other literature68 that it usually is better than 0.2%. Samples were measured in diffuse re¯ ectance as log(1/ R) from 1100 to 2500 nm in 2 nm intervals with the use APPLIED SPECTROSCOPY

613

of a Bran1 Luebbe instrum ent. The recorded spectra are shown in Fig. 2A. Visual inspection of the spectra shows that there is a clear baseline shift between the spectra. Since this source of variability is not related to m oisture, it should be removed. This can be achieved with several of the preprocessing methods described above: the offset correction, the SNV approach, or the ® rst derivative. There seem to be several outlying spectra in the data set. At this stage of data analysis it is, however, dif® cult to decide whether some of them should be removed. Principal component analysis indicates that there is a clustering in the data. W hen the spectra are offset-corrected, a clear separation into two subgroups appears in the PC3± PC1 plot (Fig. 2B). The highest loadings corresponding to PC3 (see Fig. 2C) are found in the spectral regions where water absorbs, i.e., around 1450 and 1920 nm. This is the reason why the scores on PC3 are strongly correlated with the moisture content. The correlation coef® cient is 0.84 (Fig. 2D). There is no evidence about nonlinearity between PC3 and m oisture. In order (1) to keep the comparability between the results obtained by Kalivas and results of this study, and (2) to apply a consistent methodology to all data sets included in this comparison, two different calibration setups were considered: 1. Kalivas’ setup: No data pretreatm ent (except centering); the use of the data splitting indicated by Kalivas in Ref. 1: calibration set with three objects entered twice, two test sets (see the list of indexes in Table II), and 13 objects excluded from the data set. The distribution of the calibration and test objects in the PC1±PC3 plane is indicated in Fig. 2E. PC1 and PC3 are shown because of their highest correlation with y. 2. Offset setup: Offset correction (and centering) with the data set split into the calibration and test, and training and monitoring set, respectively, with the use of the Kennard and Stone algorithm . As a result, 59 objects are included in the calibration and 40 in the test set (see Table II for object indexes). The clear outlier on the important PC3, i.e., object 100, was detected with the use of Rao’s statistics and Grubbs’ test. This result was con® rm ed by m odeling, 69 and the object was rem oved from the data set. All objects used in the calibration set were entered only once. The distribution of the calibration and test samples for the offset setup is shown in Fig. 2G. In this plot, a stronger clustering in the PC plane can be observed than for the Kalivas setup. This clustering turned out to be strongly correlated with the clustering in y (Fig. 2D). Most of the test objects are situated in the smaller cluster. However, except for the extremes on PC1, the whole dom ain is covered reasonably well. The results for both setups are reported separately below. PO LY-D AT. Data set POLY-D AT was collected with the aim of developing a calibration m odel that would allow one to predict the hydroxyl number of polyether polyol samples from their NIR spectra. The reference hydroxyl numbers were determined by using the ASTM D4274 method. They range from 10.9 to 133.6. The re614

Volume 54, Number 4, 2000

peatability of the m ethod is 1%; the reproducibility is 1.8%. The spectra were recorded in duplicates or triplicates from 1100 to 2158 nm (step 2 nm) with a NIRSystems 6250 instrument. The replicates were averaged. The offset correction has been applied to eliminate the baseline shift between spectra. The ® rst and last 15 variables, three replicates, and three object outliers were removed from the data set. 69 A previous investigation 33,69 and the PC1-PC2 plot shown in Fig. 1B indicate that the data set consists of several (at least two) clusters. Figure 3A shows that the calibration problem is linear.33,69 W hen the training set is selected by using KS, all extreme points (Fig. 1A) are included in the training set, and the leftover objects assigned to the test set do not cover the whole experimental domain. As a result, the corresponding RMSEP values are over-optimistic. For this reason, the duplex algorithm was applied instead of KS (see Fig. 1B). G ASOLINE. Data set GASOLINE concerns the determination of the octane numbers of gasolines by NIR. The reference method applied to obtain the octane num bers is the ASTM method D 2699-86: ``Standard Test Method for Knock Characteristics of Motor Fuels by the Research Method’ ’ . 70 The reference values range from 91.8 to 100.3. The estimated repeatability is 0.07; the estimated reproducibility is 0.25 octane. A Perkin-E lmer PIONIR 1024 spectrometer was used to collect data in the spectral range 800 ±1080 nm (step 0.5 nm). The ® rst derivatives of the NIR spectra were obtained to eliminate the baseline shift and to separate the overlapping peaks. A preliminar y data inspection (principal component analysis) has shown that the data set is ver y heterogeneous. It contains three big clusters, corresponding to three different product grades (see Fig. 4A). Since the differences between the clusters are not intrinsic, but only m arketing-related, the global calibration models should be able to cope with this type of heterogeneity. There is a slightly nonlinear relationship between the X variables and the octane num ber y. This nonlinearity can be seen in the PC1 octane num ber plot (Fig. 4B). The calibration set and test set were given by the manufacturer. They contain, respectively, 132 and 30 samples. Figure 4A shows that there are situations in practice when most of the new (test) samples fall close to the centroid of the calibration objects. Although in this case the test set is given by the manufacturer (and the test samples are projected to the PC space by using the loadings obtained on the calibration data), the PC plot looks similar, as if it were obtained with the Kennard and Stone selection algorithm. PO LYM ER. This data set concerns the determination of the amount of a m inor m ineral compound in a polymer by NIR. The reference values are in the range 0 ±1.75. The estimated precision of the reference m ethod is 0.009. The spectra were recorded from 1100 to 2498 nm (2 nm intervals). The explorator y data analysis9 has indicated that there is a baseline shift/drift between the spectra. The SNV transformation was therefore used to eliminate these effects.

F IG . 2. Data set WHEAT. (A) the raw spectra of all 100 wheat samples; (B) PC1±PC3 plot of the offset-corrected spectra after the elimination of the object 100; (C ) the loadings for PC3; (D ) PC3 vs. the m oisture plot; correlation coef® cient 5 0.84; (E ) the distribution of the calibration and test samples in the PC1±PC3 plane obtained for the Kalivas setup: calibration set (´), test set 1 ( 1 ), and test set 2 (C); (F ) outlying object number 100 on PC3 of the offset setup; (G ) the distribution of the calibration (´) and test (C) samp les in the PC1±PC3 plane for the offset setup.

APPLIED SPECTROSCOPY

615

TABLE II. List of the indexes of the calibration and validation samples for the W HEAT data set. Setup

Set

Kalivas

Calibration Test 1 Test 2

Offset

Calibration Test

Object indexes 10, 10, 12, 15, 16, 17, 22, 24, 25, 36, 37, 38, 39, 41, 43, 43, 44, 45, 49, 54, 55, 56, 57, 58, 60, 63, 64, 65, 69, 70, 72, 73, 77, 79, 82, 83, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 97, 97, 98, 99 1, 6, 13, 18, 21, 26, 34, 35, 46, 50, 51, 59, 62, 67, 74, 76, 80, 81, 84, 96 14, 19, 23, 29, 30, 31, 32, 33, 42, 47, 48, 52, 53, 61, 66, 68, 71, 75, 78, 95 1, 2, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 22, 23, 24, 25, 26, 27, 28, 30, 32, 33, 34, 37, 38, 39, 40, 41, 43, 44, 47, 48, 51, 56, 59, 63, 66, 67, 68, 70, 71, 72, 74, 77, 78, 80, 82, 83, 87, 89, 91, 94, 95, 97 3, 4, 17, 21, 29, 31, 35, 36, 42, 45, 46, 49, 50, 52, 53, 54, 55, 57, 58, 60, 61, 62, 64, 65, 69, 73, 75, 76, 79, 81, 84, 85, 86, 88, 90, 92, 93, 96, 98, 99

The data set consists of four clusters of objects (see Fig. 5A). The clustering is due to technological differences in the process, leading to different products. Moreover, the relationship between PC1 (or X ) and y is strongly nonlinear (see Fig. 5B). The Kennard and Stone algorithm has been applied to each cluster separately to obtain a training and test set, which cover all sources of variance in the X block. The ® nal sets contain 40 and 14 objects, respectively. The calibration and test samples are indicated in the PC1±PC2 plane of Fig. 5A. RESULTS AND DISCUSSIO N It should be emphasized that the present study compares only one aspect of m ultivariate calibration m ethodsÐ namely, the quality of prediction for test samples that are well within the calibration domain. Other important aspects, which we intend to investigate later, are the quality of prediction for test samples that require (mild) extrapolation outside the experimental domain, and the robustness towards instrumental changes and towards the presence of outliers in the calibration set. Although all examples discussed in this study com e from an undesigned population of calibration experiments, the application of the Kennard and Stone algorithm to select calibration samples ensures that the whole experimental domain is covered well and the ASTM requirements concerning the calibration set are ful® lled. Since all m easures of the predictive ability include the precision and bias of both the predicted and reference y, the RMSEP values depend on the reference m ethod. When the precision of this m ethod is poor or the bias signi® cant, then it is not possible to decide which of the

F IG . 3.

616

Data set POLY-DAT. PC1 vs. the hydroxyl number plot.

Volume 54, Number 4, 2000

properly functioning m ultivariate calibration methods performs best. From the preliminar y data analysis presented in the preceding section it should be clear that the data sets treated in this study are very different in nature (see Table I). The NIR spectra included in WHEAT and POLY-D AT seem to be linearly related to the m odelled property y. On the other hand, for GASOLINE and POLYMER the relation between X and y appears to be slightly, respectively strongly, nonlinear. The tendency to form subgroups (clustering) is a frequent feature of multivariate data sets, especially when samples come from production (POLY-D AT, GASOLINE , POLYM ER). This tendency can be much less apparent when samples are natural (W HE AT). However, as we discuss further, even the minor clustering (on PC3) can be important for m odeling. WH EAT. The proposed reference data set W HE AT 1 was analyzed with two different calibration setups. The

F IG . 4. Data set GASOLINE. (A) the distribution of the calibration (´) and test sam ples (C) given by the manufacturer in the PC1±PC2 score plot; (B) mild nonlinearity of the PC1±octane number relationship.

results obtained in both cases, for the setup applied by Kalivas and after the offset correction, are presented in Tables IIIA and IIIB, respectively. A ® rst comparison shows that the calibration models developed after the offset correction yield lower prediction errors. However, this result may be also due to the differences between the test sets obtained after the data splitting. A clear advantage of the offset correction is that it decreases the complexity of the calibration problem and leads to more parsimonious m odels. It is emphasised that, although the differences in RMSEP values achieved for the different test sets are relatively large, the relative differences in the perform ance of individual calibration techniques for the two setups are similar. This observation indicates that the m ethodology of the Kennard and Stone splitting, adopted in this study, can be used for the method comparison. The above described preliminar y data analysis has shown that the relationship between the NIR spectra and the moisture content is linear, and that the data set is clustered only on PC3. For this reason one should not expect a big improvement due to using nonlinear or local calibration m ethods. Indeed, MLR, PCR, PL S (and their variants except for Spline-PLS), NN, and LW R yield similar results. On the other hand, the prediction errors obtained by using Brown’ s method of variable selection, KNN, and SplinePLS are larger. It should be noted that there are relatively few variables that do not carr y any inform ation related to the m oisture content (all UVE-related methods retain more than 75% of all variables). For this reason one can select (with Stepwise or GA) quite different subsets of variables for the ® nal MLR m odel, yielding very similar predictions. It is remarkable that the lowest RMSEP after the offset correction was reached with the Stepwise MLR model using only two variables.

F IG . 5. (A) The strongly nonlinear relationship between the PC1 and PC2 object scores in case of the POLYMER data set. The position of the calibration (´) and test (C) objects obtained by applying the Kennard and Stone algorithm to each cluster separately is indicated. (B) Nonlinearity of the PC1 vs. y relationship.

TABLE IIIA. Data set WH EAT, Kalivas’ setup. Predictive ability, p 5 numbers of original, a 5 number of selected variables, and the list of selected variables for all compared calibration m ethods (RMSEP1 and RMSEP 2 were obtained for two different test sets). Method

RMSEP 1 /RMSEP 2

p

a

Selected variables

PCR PCR-TLS PCRS PCRS-TLS PLS

0.248/0.298 0.277/0.314 0.256/0.307 0.286/0.323 0.246/0.296

701 701 701 701 701

4 4 3 3 4

1±4 1±4 1, 3, 4 1, 3, 4 1±4

Brown Stepwise MLR 1 and 5% GA GA-FT UVE-PCR UVE-PCRS UVE-PLS RCE-PLS

1.147/1.323 0.227/0.300 0.233/0.305 0.242/0.302 0.247/0.298 0.249/0.301 0.245/0.297 0.248/0.327

701 701 701 701 633 626 627 65

200 4 5 4 4 3 4 4

NL-PCR NL-PCRS NL-UVE-PCR NL-UVE-PCRS Poly-PCR Spline-PLS

0.240/0.328 0.262/0.350 0.247/0.297 0.246/0.295 0.256/0.307 0.315/0.417

1402 1402 601 595 701 701

6 5 4 3 3 4

1±6 1, 2, 4 ±6 1±4 1, 3, 4 1, 3, 4 1±4

KNN LWR (K 5 RBF-PLS

1.164/1.246 0.246/0.296 0.286/0.320

701 701 701

Ð 4 Ð

3 NN 1±4 Ð

0.243/0.301 0.265/0.936

701 701

3 5

1, 3, 4 1±4, 10

50)

NN (topology: 3-1-1) NN-OBS (linear)

200 variables 258, 313, 352, 388 184, 304, 440, 506, 663 3, 4, 5, 8 1±4 1, 3, 4 1±4 1±4

APPLIED SPECTROSCOPY

617

TABLE IIIB. Data set W HEAT, offset setup. Predictive ability, p 5 selected variables for all compared calibration m ethods. Method

number of original, a 5

number of selected variables, and the list of

RMSEP

p

a

Selected variables

PCR PCR-TLS PCRS PCRS-TLS PLS

0.215 0.211 0.213 0.209 0.215

701 701 701 701 701

3 3 2 2 3

1±3 1±3 1, 3 1, 3 1±3

Brown Stepwise MLR 1 and 5% GA GA-FT UVE-PCR UVE-PCRS UVE-PLS RCE-PLS

1.425 0.186 0.213 0.255 0.227 0.225 0.222 0.213

701 701 701 701 547 543 578 36

199 2 5 5 3 2 3 3

NL-PCR NL-PCRS NL-UVE -PCR NL-UVE -PCRS Poly-PCR Spline-PLS

0.221 0.219 0.212 0.211 0.252 0.296

1402 1402 522 446 701 701

4 3 3 2 4 3

1±4 1, 3, 4 1±3 1, 3 1, 3, 3´11, 7´9 1±3

KNN LWR (K 5 RBF-PLS

1.050 0.215 0.197

701 701 701

Ð 3 Ð

3 NN 1±3 Ð

0.198 0.215

701 701

2 2

1, 3 1, 3

59)

NN (topology 2-1-1) NN-OBS (linear)

Figures 6A and 6B show the plot of the log(1/R) values of the calibration samples vs. their moisture content at the selected wavelengths 1982 and 2150 nm, respectively. One concludes that the selected single absorbance m easurements are poorly correlated with the moisture content, so that it is rather surprising that such good results are obtained from the bivariate m odel (see Fig. 6C). However, the regression coef® cients b corresponding to the Stepwise selected variables ( b 1982 5 139.5, b 2150 5 2 137.8) indicate that it is, in fact, the difference (contrast) between the two absorbance measurem ents that enables the good modeling (the m agnitude of the two b coef® cients is almost equal; only the signs are different). Indeed, Fig. 6D shows that the differences between absorbances x 1982 2 x 2150 are well correlated with the moisture content. One can interpret the subtraction of the absorbance at 2150 from the absorbance at 1982 nm as a background correction, yielding good predictions. Since the variables selected with the Stepwise MLR procedure are only two in this case, one can also apply the H-point method 71,72 to ® nd the best possible pair of variables to model and to predict the m oisture content. Interestingly, the H-point algorithm ® nds almost the same pair of variables as Stepwise; namely, the log(1/R) signals at 1980 and 2150 nm . It is concluded that the calibration using the original variables som etimes allows a better chemical (spectroscopic) interpretation of the m odel than the latent variable models. In the latter case, the interpretation is possible, but more dif® cult. PO LY-D AT. Similarly as in case of the WHEAT data set, the NIR spectra included in POLY-D AT are linearly related to the property y. The difference is that POLYDAT is a heavily clustered data set. It was seen 33 that some of the PCs or PLS latent variables describe differences due to clustering (e.g., PC1). Those PCs or PL S 618

Volume 54, Number 4, 2000

293±339, 494±645 442, 526 58, 280, 433, 620, 672 3, 5, 10, 13, 15 1±3 1, 3 1±3 1±3

latent variables can therefore be important for developing one general calibration model for all present subgroups. Table IV again indicates that m ost of the linear calibration m ethods lead to rather similar results. The optimal full spectrum PCR m odels (top down, selection, TLS) yield RMSEP values that range from 1.84 to 2.09. The RMSEP obtained with the slightly m ore parsimonious PL S model is 2.23. Although there seems to be an appreciable difference between the absolute RMSE P value achieved with PCR-TLS (1.84) and the RMSEP reached with PL S (2.23), the randomization test shows that this difference is not statistically signi® cant. Similar predictive ability is obtained by using the variable elimination methods; namely, UVE-PCRS, UVE -PLS, and RCEPL S, and also GA-FT. Compared to PCR/PL S, lower prediction errors were achieved by using the variable selection methods: Stepwise MLR at the 5% critical level and GA. However, according to the randomization test, these results are not signi® cantly better than the PCR results. Because the important variables describing the data clustering cannot be detected and selected by using correlation, Brown’ s method of variable selection does not work in this case. As explained earlier, the calibration problem is linear so that there does not seem to be a need to utilize nonlinear calibration. Indeed, NL-(UV E)-PCR provides predictions similar to those of the ordinary PCR. Slightly lower prediction errors obtained with NL-PCR and m ainly NL-UVE -PCRS can be also due to including higher PCs. Because such PCs often carr y information that is not robust with respect to sm all (instrumental) changes, one does not have to consider those two models as possible alternatives to PCR/PL S. As expected, Poly-PCR and the Spline-PL S model do not bring any improvement of prediction in this linear case, com pared to the performance of the ordinar y bilinear m ethods. The obtained

F IG . 6. Results of the Stepwise MLR variable selection procedure applied to data set WHEAT after the offset correction. A and B show the absorbance values obtained for the calibration samples at 1982 and 2150 nm, respectively, plotted vs. the moisture content. ( C ) the plot of the predicted moisture content vs. the moisture content obtained with the reference method for the test sam ples. (D ) The difference between the absorbance values at the selected wavelengths against the m oisture content.

TABLE IV. Data set POLY-DAT for the data splitting using the duplex algorithm. Predictive ability, p 5 of selected variables, and the list of selected variables for all com pared calibration methods. Method

number of original, a 5

number

RMSEP

p

a

Selected variables

PCR PCR-TLS PCRS PCRS-TLS PLS

2.07 1.84 2.09 1.85 2.23

499 499 499 499 499

9 9 8 8 7

1±9 1±9 1±6, 8, 9 1±6, 8, 9 1±7

Brown Stepwise MLR 5% Stepwise MLR 1% GA GA-FT UVE-PCR UVE-PCRS UVE-PLS RCE-PLS

7.04 1.73 1.93 1.71 2.15 2.48 2.05 2.10 2.38

499 499 499 499 499 154 121 187 37

16 10 8 8 8 7 8 7 6

479±494 54, 145, 146, 377, 389, 393, 456, 465, 489, 492 54, 145, 146, 377, 456, 465, 489, 492 96, 136, 146, 211, 276, 296, 386, 471 3, 4, 9, 13, 14, 20, 26, 33 1±7 1, 2, 4 ±7, 11, 13 1±7 1±6

NL-PCR NL-PCRS NL-UVE-PCR NL-UVE-PCRS Poly-PCR Spline-PLS

2.10 1.95 2.04 1.77 3.08 3.00

998 998 209 180 499 499

13 14 8 10 10 7

1±13 1±13, 21 1±8 1±8, 10, 13 1±6, 8, 3´7, 5´7, 6´10 1±7

KNN LWR (K 5 RBF-PLS

7.15 1.38 1.41

499 499 499

Ð 7 Ð

1 NN 1±7 Ð

2.18 1.74

499 499

8 11

1±6, 8, 9 1±6, 8±10, 16, 20

15)

NN (8-3-1) NN-OBS (11-3-1)

APPLIED SPECTROSCOPY

619

TABLE V. Data set GA SOLINE. Pred ictive ability, p 5 variables for all compared calibration m ethods. Method

number of original, a 5

number of selected variables, and the list of selected

RMSEP

p

a

Selected variables

PCR PCR-TLS PCRS PCRS-TLS PLS

0.143 0.232 0.163 0.252 0.146

561 561 561 561 561

15 15 11 11 9

1±15 1±15 1, 2, 4, 6, 7, 9, 11±15 1, 2, 4, 6, 7, 9, 11±15 1±9

Brown Stepwise MLR 1 and 5% GA GA-FT UVE-PCR UVE-PCRS UVE-PLS RCE-PLS

0.423 0.136 0.288 0.198 0.132 0.166 0.134 0.150

561 561 112 561 255 262 341 26

9 10 10 11 11 10 9 16

286±294 88, 265, 290, 306, 327, 361, 425, 481, 493, 534 111, 266, 291, 296, 301, 396, 426, 431, 466, 531 2, 7, 8, 10, 13, 15, 19, 27, 34, 37, 42 1±11 1, 2, 4, 6, 7, 9 ±12, 22 1±9 1±16

NL-PCR NL-PCRS NL-UVE -PCR NL-UVE -PCRS Poly-PCR Spline-PLS

0.153 0.176 0.140 0.175 0.174 0.193

1122 1122 312 189 561 561

20 14 15 10 9 8

1±20 1±9, 11, 14, 15, 18, 20 1±15 1±3, 5±9, 11, 13 1, 2, 4, 6, 7, 11±13, 1´2 1±8

KNN LWR (K 5 RBF-PLS

132)

0.351 0.146 0.133

561 561 561

Ð 9 Ð

3 NN 1±9 Ð

NN (13-6-1) NN-OBS (linear)

0.146 0.147

561 561

13 11

1±7, 9, 11±15 1, 2, 4, 6, 7, 9, 11±15

predictions are in fact worse. The application of the nonlinear versions of PCR/PLS can be therefore omitted in the POLY-DAT case. As stated in the theoretical section, it is shown that if the calibration problem is linear, NN can still perform quite well. In the POLY-DAT case the optimal NN model uses the same PCs as the PCRS m odel and gives almost the same RMSEP value as PCR/PLS. Although the NNOBS m odel leads to somewhat lower prediction errors, it also includes high PCs (namely, PC 10, 16 and 20). In practice, one would probably prefer to apply the m ore parsimonious and more robust NN m odel to predict the property of interest of future samples, excluding the high PCs. The local KNN m ethod predicts poorly since the selection of nearest neighbors in the original space using the Euclidean distance (as applied here) is in¯ uenced mainly by the information related to PC1 (clustering). The information included in the higher PCs, more correlated with y, has m uch less in¯ uence on the selection of nearest neighbors. On the other hand, the application of the local regression techniques m akes sense in this case and gives good predictions. The RBF-PLS m odel and the optimal LWR m odel (using PL S regression, the Mahalanobis distance as a distance m easure and no weighting) leads to RMSE P below 1.45. Both these results still are not considered to be signi® cantly better than PL S at the critical a level 5 5%, but they are at a 5 10%. It seems that both local m ethods partly overcom e the problem of data heterogeneity. G ASOLINE. This data set is the most complex example presented in this study. Gasolines are mixtures of hundreds of different compounds that contribute differently to the ® nal octane num ber. It is assumed therefore that the calibration model should be m ore complex than, 620

Volume 54, Number 4, 2000

e.g., the m odel developed to predict the m oisture content of WHEAT. Although the RMSEP values do not vary so much as in case of the other data sets, one can still see differences in the predictive power of the developed calibration m odels (see Table V). The data inspection has indicated a slightly nonlinear relationship between X and y. The results, however, do not support the idea of nonlinear m odeling. The typical representatives of nonlinear and local calibration m ethods, NN and LWR, yield the same prediction errors as PCR or PLS. The nonlinear variants of PCR and PLS result in even worse predictions. Apart from the two m ethods that generally do not seem to work very well, Brown’s m ethod and KNN, there are other techniques that give poor predictions in case of the GASOL INE data set: TLS-PCR and GA. There seems to be no particular reason for this obser vation. Slightly better results than those for PCR, PLS, and NN are obtained by the fam ily of variable selection/elimination methods: Stepwise MLR, UVE-PCR(PLS), RCEPL S, and also RBF-PL S. As explained above, this appears to be an example of a situation where the RMSEP value is limited by the imprecision and bias of the reference y, so that the possibility of com paring methods is restricted. PO LYM ER. The last case presented in this study is an example of a nonlinear calibration problem. In this instance it is expected that the linear methods such as PCR, PL S, and MLR will not perform as well as the nonlinear variants of PCR and PL S and, in particular, as well as the truly nonlinear (NN) and local (KNN) calibration m ethods. Concerning NN and the local m odeling techniques, the results presented in Table VI con® rm this expectation to a large extent. However, NL-PCR, PolyPCR, and Spline-PLS do not work better than the ordinary PCR/PLS.

TABLE VI. Data set POLYM ER. Predictive ability, p 5 variables for all com pared calibration methods. Method

number of original, a 5

number of selected variables, and the list of selected

RMSEP

p

a

Selected variables

PCR PCR-TLS PCRS PCRS-TLS PLS

0.0477 0.0500 0.0477 0.0500 0.0445

700 700 700 700 700

7 7 7 7 5

1±7 1±7 1±7 1±7 1±5

Brown Stepwise MLR 1% Stepwise MLR 5% GA GA-FT UVE-PCR UVE-PCRS UVE-PLS RCE-PLS

0.1871 0.0861 0.0797 0.0594 0.0515 0.0519 0.0498 0.0537 0.0485 0.0649

700 700 700 700 350 1 350 700 262 257 278 53

7 1 2 8 8 5 7 7 5 8

455±461 457 37, 457 39, 164, 238, 382, 409, 476, 510, 672 123, 133, 477, 535, 669, 9 2 , 123 2 , 4592 4, 5, 16, 21, 28 1±7 1, 2, 4, 6, 7, 10, 12 1±5 1±8

NL-PCR NL-PCRS NL-UVE-PCR NL-UVE-PCRS Poly-PCR Spline-PLS

0.0478 0.0546 0.0531 0.0597 0.0538 0.0576

1400 1400 833 398 700 700

8 5 7 3 3 1

1±8 1, 2, 5, 6, 8 1±7 1±3 1, 7, 1 2 1

KNN LWR (K 5 RBF-PLS

0.0061 0.0060 0.0078

700 700 700

Ð 2 Ð

2 NN 1±2 Ð

0.0096 0.0179

700 700

3 7

1, 2, 4 1±7

5)

NN (3-3-1) NN-OBS (nonlinear)

The PCR, PCRS, TLS-PCR, and PLS m odels yield similar predictions; the differences lie only in the model complexity. Seven PCs are needed to reach the m inimal prediction errors with PCR, whereas only ® ve latent variables are needed with PL S. The variable selection or elimination does not help in this case, since the m ain challenge of this data set is the model nonlinearity. Surprisingly, the inclusion of the squared terms of the original variables to the X m atrix for the selection with GA (350 original linear and 350 squared variables), and for the PCR m odeling (NL-PCR, NL-UV E-PCR), does not improve the precision of prediction compared to the ordinar y PCR/PL S. Poly-PCR and Spline-PLS also are not very successful; the only improvement is the reduced model complexity. Quite different predictions are achieved by using (OBS-)NN and the local calibration methods: KNN, LW R, and RBF-PL S. The RMSEP values obtained with these techniques are one order of magnitude lower than those achieved with the other m ethods. The power of neural networks to deal with nonlinear problems is known. The extremely good results obtained with KNN, LW R, and RBF-PL S are likely due to the other characteristic of this data set, namely, the extreme data clustering. The calibration models developed by using local techniques probably work well within the narrow ranges of the four clusters. However, in the case where future samples would be within the experimental range, but situated outside the four existing clusters (sample inliers), the prediction would not be as good as the prediction for objects within the clusters. This might be a problem in some cases, but not for the POLYMER data set. It is known that, for technological reasons, only the products belonging to one of the four clusters can be produced. G eneral Remarks on the Performance of Calibra-

tion Techniques. Although the examples presented in the current study do not cover all possible calibration situations, they give a reasonable idea of how the considered calibration techniques perform . The following conclusions can be drawn from the results. PCR, preferably with the PC subset selection, yields prediction results similar to those of PL S. The PLS models are of the same or smaller complexity than the PCR models. TLS-PCR has never been seen to give signi® cantly better results than the ordinar y PCR and does not need to be considered as an additional alternative to the established PCR/PL S m ethods. The variable selection/elimination can have a positive in¯ uence on the predictive ability of the calibration model. In particular, the perform ance of the Stepwise MLR variable selection procedure seems to be som ewhat underestimated by chemometricians. When the calibration problem is linear, Stepwise MLR regression yields predictions that are com parable to, and sometimes even better than, the predictions obtained by using the full-spectrum calibration m ethods. It should not be forgotten, however, that the MLR results m ay be more sensitive to model extrapolations, so that the conclusions presented here are valid only when the calibration set exceeds the range of variation of future samples.63 In addition, chance correlations can occur in Stepwise MLR. The UVE methods can be applied with the aim of improving the precision of prediction, but also as a diagnostic tool (a screening step) to see to what extent the variables included in X are relevant to predict the property y. Possible advantages of RCE-PLS over UVE -PLS have not been clearly seen, probably because the considered NIR data are very precise so that there is almost no noise to remove. All cited variable selection/elimination techAPPLIED SPECTROSCOPY

621

niques yield signi® cantly better predictions than the simple (univariate) variable selection method of Brown. In this study, there was no clear example that would demonstrate the usefulness of the fam ily of the PCR/PLS nonlinear methods. W hen the calibration problem was linear, the linear methods resulted in better predictions; when the problem was nonlinear, NN and some of the local calibration techniques outperform ed NL -PCR, PolyPCR, and Spline-PL S. The better perform ance of NN compared to the nonlinear PCR/PLS is likely due to a much higher ¯ exibility of NN. The disadvantage of NN is that it cannot be applied when too few calibration samples are available, since one needs a m onitoring set at the stage of the m odel development. Another m ethod that does not seem to be of interest is KNN. This technique suffers from its nonparametric character since it gives the same importance to the relevant and to irrelevant sources of variance in X to determine y. It should be noted that, although the optimization of the LW R m odel is time consuming (PCR/PL S, Euclidean/Mahalanobis distance, weighting/no weighting for PCR), the ® nal LW R m odel is always at least as good as the ordinary PCR/PLS m odel, since, when necessary, all calibration samples can still be used to build the model (see WHEAT and GASOL INE as examples). On the other hand, sometimes there is no expectation that the LW R model will outperform the ordinary PCR/PL S; e.g., when the calibration problem is linear and the data set not very heterog en eou s ( W H E AT ) . Ind eed, regr ession theor y shows 27,70 that the number of calibration samples should always be as high as possible. Using sample subgroups in the m odeling step as is applied in LWR decreases the precision of the calibration param eters, the precision of prediction, and to some extent also the reliability of predictions. Compared to PCR/PLS, a better reliability of predictions can be achieved with LWR only when the decreased precision is compensated by less bias, caused by an intrinsically better ® t of the local m odels to the calibration data (see POLY-D AT and POLYMER). A Strategy to Select a Suitable Calibration Technique for a G iven Calibration Problem. The aim of this section is to propose a ¯ exible methodology on how to select an appropriate calibration technique for a given calibration problem. It is proposed that one start with PCRS and/or PL S, depending on whether one prefers a somewhat easier interpretation of the developed calibration m odel (PCRS) or the (sometimes) more parsimonious m odel (PL S). The achieved result(s) can be considered as a benchmark for comparison with other calibration techniques. W hen the calibration problem is linear, it is useful to apply the Stepwise (or GA) MLR selection procedure, especially when it is intended to interpret directly the original and not the latent variables. This approach m ay be appreciated mainly by spectroscopists without a particular background in chemometrics, although not all of the selected variables are always directly related to the modeled y, but may correct, e.g., for the background shift. A deeper insight into the linear m ultivariate calibration problems can be reached with a UVE method. The application of this approach can be treated as a part of the 622

Volume 54, Number 4, 2000

data investigation, giving an indication on how relevant the variables included in X are to model y. The feature elimination leads to better predictions in those cases, when m any variables can be removed from X that contain a lot of variability, which is not, however, related to the modeled property y.35 If the calibration problem is nonlinear and the new objects are well within the calibration domain, then NN can be considered as a good tool, enabling one to approximate any type of nonlinearity. Since NN with the proper topology can very well m odel also completely linear problems, an alternative strategy might be to apply NN from the start in any application of multivariate calibration, assuming the new samples are within the calibration space and enough calibration samples were measured. In this study and under those conditions we never obtained catastrophic predictions with NN. On the contrar y, the prediction errors from the NN models were never signi® cantly larger than those from the PCR/PL S m odels. The disadvantage is that NN functions as a black box. If m ore diagnostics can be developed, so as to allow better understanding of the relationship within the data, one might consider replacing PCR/PL S by NN. At ® rst sight, a third alternative strategy m ight be to always work with LW R. In many cases, where the relationship between X and y can be locally approximated by a linear ® t, LWR can be ver y helpful. One of the reasons to apply local modeling may also be the presence of clusters in the data. The literature on these m ethods is, however, ver y limited, and m ore work needs to be done to develop a simple approach to method optimization Moreover, the m ethod can be contra-indicated for small data sets, because of the loss of precision mentioned earlier, so that, at least for relatively small sets as used here, the m ethod should be applied with caution. CONCLUSIO N The aim of the present study was to compare the quality of prediction obtained with several m ultivariate calibration m ethods for test samples within the calibration dom ain. Second, a ¯ exible methodology was proposed to choose an appropriate calibration m ethod for a given calibration problem. It should be mentioned that the proposed m ethodology cannot be considered general, as it is based on a limited, though relatively representative, set of data. However, it m ay serve as a starting point for such a m ethodology. It has been shown that a correct application of many calibration methods (MLR to the selected variables, PCR, PL S, NN) to a linear problem yields results of similar quality. Surprisingly good results (and simple m odels) can often be reached with the Stepwise MLR regression procedure. The calibration m ethods that were designed (or meant) for a particular type of problem generally perform better than the other ones in the desired situation: NN in nonlinear calibration, LW R and RBF-PLS in case of data clustering/nonlinearity, Stepwise MLR and UVEmethods when irrelevant inform ation is included in the matrix of predictors X. Among the m ethods that do not always work, or do not bring special advantages compared to MLR, PCR, PL S, and NN, are TLS-PCR, Brown’ s m ethod, KNN, and

the nonlinear variants of PCR/PLS (NL -PCR, Poly-PCR, Spline-PLS). The conclusions we made are valid only for the calibration situation studied, i.e., when new samples will be situated within the calibration domain. We intend to carr y out a similar comparison study for the situation where one cannot be sure that this condition will be true, focusing on the perform ance of the methods and the quality of prediction when (mild) extrapolation is needed, as may be the case in prediction for the m ore extreme objects in a population. Moreover, the evaluation of the robustness with respect to instrum ental changes and the presence of outliers is being undertaken. 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31.

J. Kalivas, Chemom. Intell. Lab. Syst. 37, 255 (1997). A. M. C. Davies, Spectrosc. Eur. 7/4, 36 (1995). J. Sun, J. Chemom. 9, 21 (1995). J. A. Cowe and J. W. McNicol, Appl. Spectrosc. 39, 257 (1985). I. T. Jolliffe, Appl. Stat. 31, 300 (1982). J. M. Sutter, J. H. Kalivas, and P. M. Lang, J. Chem om. 6, 217 (1992). J. Verdu -Andre s and D. L. Massart, Appl. Spectrosc. 52, 1425 (1998). N. B. Vogt, Chemom. Intell. Lab. Syst. 7, 119 (1989). J. Verdu -Andre s, D. L. Massart, C. Menardo, and C. Sterna, Anal. Chim. Acta 389, 115 (1999). O. M. Kvalheim, Chem om. Intell. Lab. Syst. 8, 59 (1990). A. Berglund and S. J. Wold, J. Chem om. 11/2, 141 (1997). S. D. Hodges and P. G. Moore, Appl. Stat. 21, 185 (1972). W. A. Fuller, M easurement Error Models (Wiley, New York, 1987). S. Van Huffel and J. Vandewalle, The Total Least Squares Problem: Computational Aspects and Analysis (SLAM, Philadelphia, 1988). H. van der Voet, Chemom. Intell. Lab. Syst. 25, 313 (1994). H. Martens and T. Nñ s, Multivariate Calibration (Wiley, Chichester, 1989). P. Geladi and B. R. Kowalski, Anal. Chim. Acta 185, 1 (1986). S. de Jong and A. Phatak, ``Partial Least Squares Regression’ ’ , Procedings of the 2nd International Workshop on TLS and Errorin-Variables Modelling, Leuven, Belgium (1996). S. Wold, N. Kettaneh-Wold, and B. Skagerberg, Chemom. Intell. Lab. Syst. 7, 53 (1989). J. E. Frank, Chemom. Intell. Lab. Syst. 8, 109 (1990). S. Wold, Chemom. Intell. Lab. Syst. 14, 71 (1992). S. Wold, Technometrics 16, 1 (1974). P. J. Brown, C. H. Spiegelman, and M. C. Denham, Phil. Trans. R. Soc. A 337, 311 (1991). P. J. Brown, J. Chemom. 6, 151 (1992). P. J. Brown, Measurement, Regression, and Calibration (Clarendon Press, Oxford, 1993). D. Jouan-Rimbaud, B. Walczak , D. L. Massart, I. R. Last, and K. A. Prebble, Anal. Chim. Acta 304, 285 (1995). N. R. Draper and H. Smith, Applied Regression Analysis (Wiley, New York, 1981), 2nd ed. D. E. Goldberg, Genetic Algorithms in Search, Optimization and M achine Learning (Addison-Wesley Publishing Company, Reading, Massachusetts, 1989). C. B. Lucasius and G. Katem an, Chemom. Intell. Lab. Syst. 19, 1 (1993). C. B. Lucasius and G. Katem an, Chemom. Intell. Lab. Syst. 25, 99 (1994). R. Leardi, R. Boggia, and M. J. Terrile, J. Chemom. 6, 267 (1992).

32. R. J. Leardi, Chemom. 8, 65 (1994). 33. D. Jouan-Rimbaud, D. L. Massart, R. Leardi, and O. E. de Noord, Anal. Chem. 67, 4295 (1995). 34. L. Pasti, D. Jouan-Rimbaud, D. L. Massart, and O. E. de Noord, Anal. Chim. Acta 364, 253 (1998). 35. V. Centner, D. L. Massart, O. E. de Noord, S. de Jong, B. M. Vandeginste, and C. Stema, Anal. Chem. 68, 3851 (1996). 36. D. Jouan-Rimbaud, B. Walczak, R. J. Poppi, O. E. de Noord, and D. L. Massart, Anal. Chem. 69, 4317 (1997). 37. N. Saito, ª Simultaneous Noise Suppression and Signal Compression Using a Library of Orthonormal Bases and the Minimum Description Length Criterionº , in W avelets in Geophysics, E. Foufoula-Georgion and P. Kumar, Eds. (Academic Press, New York, 1994). 38. B. Walczak and D. L. Massart, Chemom. Intell. Lab. Syst. 36, 81 (1997). 39. D. L. Massart, B. Vandeginste, Y. Michotte, L. Kaufm an, and S. Dem ing, Chemometrics: A Textbook (Elsevier, Amsterdam , 1988). 40. C. J. Stone, Ann. Statist. 5(4), 595 (1977). 41. T. Nñ s, T. Isaksson, and B. R. Kowalaki, Anal. Chem . 62, 664 (1990). 42. T. Nñ s and T. Isaksson, Appl. Spectrosc. 62, 34 (1992). 43. Z. Wang, T. Isaksson, and B. R. Kowalski, Anal. Chem. 66, 249 (1994). 44. T. Nñ s and T. Isaksson, NIR News 5(4), 7 (1994). 45. T. Nñ s and T. Isaksson, NIR News 5(5), 8 (1994). 46. B. Walczak and D. L. Massart, Anal. Chim Acta 331, 177 (1996). 47. J. R. M. Smits, W. J. Melssen, L. M. C. Buydens, and G. Kateman, Chemom. Intell. Lab. Syst. 22, 165 (1994). 48. F. Despagne and D. L. Massart, Chem. Intell. Lab. Syst. 40, 145 (1998). 49. R. J. Poppi and D. L. Massart, Anal. Chim. Acta 375, 187 (1998). 50. R. Fletcher, Practical M ethods of Optimization (Wiley, New York, 1987). 51. S. Wold, Technometrics 20, 395 (1974). 52. H. T. Eastman and W. J. Krzanowski, Technometrics 24, 73 (1982). 53. J. Shao, JASA 88(422), 486 (1992). 54. D. W. Osten, J. Chemom. 2, 39 (1988). 55. B. Efron and G. Gong, Am . Statistician 37, 36 (1983). 56. T. Nñ s, J. Chemom. 1, 121 (1987). 57. G. Puchwein, Anal. Chem. 60, 569 (1988). 58. J. FerreÂand F. X. Rius, Anal. Chem. 68, 1565 (1996). 59. J. FerreÂand F. X. Rius, Trends Anal. Chem . 16, 70 (1997). 60. D. E. Honigs, G. M. Hieftje, H. L. Mark, and T. B. Hirschfeld, Anal. Chem. 57, 2299 (1985). 61. R. D. Snee, Technometrics 19, 415 (1977). 62. T. Fearn, NIR News 8(1), 7 (1997). 63. ``Standard Practices for Infrared, Multivariate, Quantitative Analysis’ ’ , American National Standard ASTM, E 1655-94 (Am erican Society for Testing and Materials, Philadelphia, 1994). 64. R. W. Kennard and L. A. Stone, Technometrics 11, 137 (1969). 65. S. N. Deming and S. L. Morgan, Experim ental Design: A Chemometrics Approach (Elsevier, Amsterdam, 1993) 2nd ed. 66. R. Carlson, Design and Optimisation in Organic Synthesis (Elsevier, Amsterdam, 1982). 67. P. J. Gem perline and A. Salt, J. Chemom. 3, 343 (1989). 68. B. G. Osborne, T. Fearn, and P. H. Hindle, Practical NIR Spectroscopy with Applications in Food and Beverage Analysis (Longm an Singapore Publishers, Singapore, 1993). 69. V. Centner, D. L. Massart, and O. E. de Noord, Anal. Chim. Acta 330, 1 (1996). 70. ``Determination of the Knock Numbers by Research Methods,’’ Am erican National Standard ASTM, D 2699-86 (American Society for Testing and Materials, Philadelphia, 1986). 71. F. Bosch-Reig and P. Cam pins-Falco, Analyst 113, 1011 (1988). 72. F. Bosch-Reig and P. Cam pins-Falco, Analyst 115, 111 (1989).

APPLIED SPECTROSCOPY

623