second-order calibration: bilinear least squares regression and a simple alternative

Ž .Chemometrics and Intelligent Laboratory Systems 42 1998 159–178

Second-order calibration: bilinear least squares regression and asimple alternative

Marie Linder ), Rolf SundbergMathematical Statistics, Stockholm UniÕersity, S-106 91 Stockholm, Sweden

Received 6 June 1997; accepted 12 February 1998

Abstract

We consider calibration of second-order, or hyphenated instruments generating bilinear two-way data for each specimen.The bilinear regression model is to be estimated from a number of specimens of known composition. We propose a simple

Žestimator and study how it works on real and simulated data. The estimator, which we call the SVD singular value decom-.position estimator is usually not much less efficient than bilinear least squares. The advantages of our method over bilinear

Ž .least squares are that it is faster and more easily computed, its standard errors are explicit and derived in the paper , and ithas a simpler correlation structure. q 1998 Elsevier Science B.V. All rights reserved.

Keywords: Multivariate calibration; Bilinear regression; Hyphenated instruments; Matrix data; Multilinearity; Singular value decompositionŽ .SVD ; SVD estimator

1. The bilinear model

A hyphenated instrument, consisting of two instruments in series, yields a matrix of data for each specimen.Given a number of calibration specimens with several constituents of known concentration, how do we calibratesuch instruments? How do we resolve the pure response of each constituent?

We will use LC–UV as a model example. Each pure constituent is characterized by an elution profile and aUV-spectrum. The LC separation gives rise to a new spectrum for each chromatographic time, i.e., we get amatrix of absorbances for each specimen when we collect the response in the UV-instrument. This matrix will

Ž . Ž . Žk .be indexed by time from the LC profile and wavelength from the UV spectra . We use the notation z fori j

the absorbance z at time i and wavelength j in specimen k; constituent is indexed by r. We prefer the wordspecimen, instead of sample, because they are thought of as generated from designed mixtures of known con-centrations and not from random sampling.

We assume proportionality in concentration for each of the instruments separately. For a UV instrument thiscan be motivated by Lambert–Beer’s law. For an LC instrument the elution peak area is expected to increase

Ž .linearly with moderately increasing concentration of the constituent. Examples of other instrument combina-

) Corresponding author.

0169-7439r98r$19.00 q 1998 Elsevier Science B.V. All rights reserved.Ž .PII: S0169-7439 98 00032-X

( )M. Linder, R. SundbergrChemometrics and Intelligent Laboratory Systems 42 1998 159–178160

tions which yield bilinear data matrices are excitation-emission fluorescence instruments and LC– or GC–MS,whereas MS–MS and 2D–NMR do not yield bilinear data matrices. We also assume that responses for bothinstruments are additive over constituents, in the sense that the response for a mixture is the sum of the individ-ual constituent responses. From above we note that the concentrations must not be too high for proportional lin-earity to be realistic. The final assumption behind our model is that the background has been separately elimi-nated. If the background had not been eliminated, we would have to add some type of intercept-parameter to ourmodel to make it valid.

From the assumption of linearity in each instrument response, we conclude that data should follow a bilinearregression model. In its most intuitive form, this model is written as ZŽk .sÝR cŽk .a b T, with elements Z Žk .

rs1 r r r r i j

sÝa cŽk .b for specimen k. In the LC–UV example cŽk . is the concentration of constituent r in specimen k,i r r r jr r r

a is the elution profile of constituent r, and b is the spectrum of constituent r. The model can be written inr r

condensed matrix form as

Model for data: ZŽk .sACŽk .BT qEŽk . , 1Ž .where ZŽk . is the observed data matrix for specimen k, A is the pure response of the first instrument, CŽk . is adiagonal matrix with the concentrations for calibration specimen k on the diagonal, B is the pure response of thesecond instrument and EŽk . is the noise matrix for specimen k. At present we make no specific assumptionsabout the noise, in Section 3 it will be assumed uncorrelated and homoscedastic. For an example of simulateddata matrices, see Fig. 1.

Fig. 1. Pure and noisy data structures for two specimens with the same constituents but with different concentrations.

( )M. Linder, R. SundbergrChemometrics and Intelligent Laboratory Systems 42 1998 159–178 161

The model description above is not unique since multiplication of a and division of b by the same scalarr r

will not change their product a b T. In order to make it unique in a symmetric way we write Z Ž k . sr r

ÝR cŽk .g a b T, or equivalentlyrs1 r r r r r r

Žk . Žk . T Žk . < < < <Symmetrically constrained model: Z sAC GB qE , a s b s1, 2Ž .r r

where columns a and b for constituent r of matrices A and B represent the profiles from the first and ther r

second instrument, respectively, normalized to unit length. Here G is a diagonal matrix of scale parameters with� 4elements g , the products of the original lengths of a and b . We need to estimate the parameters A, B, Gr r r r

of the calibration model, which we can then use for future predictions. One possibility is bilinear least squaresŽ .BLLS , which we try in this paper. We also present an alternative, a simple estimator, which we call the singu-

Ž .lar value decomposition SVD estimator. This estimator is formed by singular value decomposition of suitablyweighted sums of the observed data matrices and can be interpreted as obtained from reweighted least squaresnormal equations. For the SVD estimator we can explicitly write down the variance–covariance matrix. We makenumerical comparisons of efficiency with the BLLS estimator, and try both methods on a real data example.

Among alternative methods for solving the bilinear regression problem, we first note the Generalized RankŽ . w xAnnihilation Method GRAM , see Sanchez and Kowalski 1 , even though it is based on just one calibration

specimen which is compared with one unknown specimen. Other methods are based on the PARAFAC model,which is obtained by regarding also the concentration matrices CŽk . in model 1 as unknown parameters, see Brow x Ž .2 for a review. The parameters in this model can be estimated by, for example, Alternating Least Squares ALS

Ž . w xor by the methods known as TriLinear Decomposition methods TLD , see Sanchez and Kowalski 3 or Bur-w xdick et al. 4 . Note that trilinear here means a bilinear instrument with more than one specimen used. TLD

Ž .somewhat arbitrarily uses the GRAM idea on two selected specimens or two selected linear combinations . Theestimated concentrations in the PARAFAC model are finally regressed on the known concentrations to build a

w x w xcalibration model. PLS-based methods have been proposed and used by Wold et al. 5 and Bro 6 . Wold et al.use unfolding to transform three-way data to two-way data, whereas Bro refrains from that. Bro’s method has

w x w x Ž .recently been discussed by Smilde 7 and de Jong 8 . For more general views on bilinear and multilinearw x w xmodels and methods for decompositions of such data sets, see Geladi 9 and Leurgans and Ross 10 . Not much

Ž .is known about how different bilinear multilinear methods compare in precision.

2. Estimation procedures

We assume that the calibration is done with at least as many specimens as constituents. This condition is notreally necessary for the quantitative prediction of the same constituents in a new specimen, as shown by theGRAM method, and it is not always necessary for the existence of a unique solution to the estimation problem,

w xsee Kruskal 11 .

2.1. The BLLS estimator

The least squares method for fitting a model to data, be it linear or nonlinear, requires that the sum of squareddeviations between measured and expected responses is made as small as possible. An ideal situation for leastsquares is that the noise is normally distributed, uncorrelated and homoscedastic. Then the least squares methodis equivalent to the maximum likelihood method. To obtain the least squares solution we minimize the residualsum of squares with respect to the parameters. This can be done by differentiation with respect to the parame-ters. Equating these derivatives to zero we get the normal equations. Solving them yields the parameter esti-mates.


The residual sum of squares for bilinear models and three-dimensional arrays is naturally expressed by sum-ming over specimens and using the trace of the resulting matrix, or in formulas

KTŽ . Ž .k k T Žk . Žk . TTr Z yAC GB Z yAC GB . 3Ž . Ž .Ž .Ýž /

ks1

� 4 < < < <This has to be minimized with respect to A, B, G , under the constraints a s b s1 from the assumedr r

normalization. Using Lagrange’s multiplier method for minimization under constraints we find the normal equa-tions

g ADŽ r .GBTb yg 2 dŽ r .a yg T Ž r .b yl a s0r r r r r r r r r r r r r

TŽ r . T 2 Ž r . Ž r .g BGD A a yg d b yg T a ym b s0Ž .r r r r r r r r r r r r r

b T BGDŽ r .ATa ya T T Ž r .b s0,r r r r

where l and m are the Lagrangian multipliers, and T Ž r . and DŽ r . are defined byr r

K KŽ r . Žk . Žk . Ž r . Žk . Žk .T s c Z , D s c C .Ý Ýr r r r

ks1 ks1

The matrix statistics T Ž r . and the coefficient matrices DŽ r . are of great importance for the sequel. They areweighted sums of data matrices and concentration matrices, respectively. Note that each constituent, rs1, . . . , Rrequires its own weights, namely its concentrations in the different specimens. The matrices T Ž r . carry all infor-mation in data about the bilinear structure, in a sense that may be specified in terms of the concept of suffi-ciency.

After simplification of the normal equations we get

Result 2.1: Normal equations for BLLS.

The normal equations, over rs1, . . . , R, can be written:

ADŽ r .GBTb sT Ž r .br r4Ž .TŽ r . T Ž r .BGD A a s T a .Ž .r r

Remark: In the special case when we have only one constituent, the matrices A and B reduce to vectors and the< < < <diagonal matrices G and D reduce to scalars. Using the property a s b s1, the system of normal equations

reduces to the eigen-problem

2TTT as g d aŽ .2TT Tbs g d b .Ž .

This reduction cannot be performed when we have more than one constituent. The following argument explainswhy. For one constituent the term ATAsa Tas1, but more generally ATA is a matrix in which only the diago-nal elements are fixed and equal to one. The unknown off-diagonal elements will typically not be zero, sincethey represent the angles between the profile vectors for the different constituents. B

The system of normal equations can be solved iteratively in analogy with the PARAFAC model. The follow-ing alternating least squares procedure, with starting values for either A or B, is natural and usually works slowly


Ž .but safely. We express A as a function of B and vice versa in the following reformulation of Eq. 4 , cf. ModelŽ .Eq. 1 :

y1Ž .T T 1d b b PPP d b b T ba 11 1 1 1 R 1 R 11

. . . .. . . . .. s 5Ž .. . . ..� 0 � 0 � 0T T ŽR.a R d b b PPP d b b T bR1 R 1 R R R R R

y1 TŽ .T 1d PPP d a a T ab Ž .11 1 R 1 R 11. . . .. . . . .. s . 6Ž .. . . ..� 0 � 0T T� 0b ŽR.d a a PPP dR R1 R 1 R R T aŽ . R

Here we have incorporated g in b and only assume a normalized to unit length in each step, for unique-r r r rŽ . Ž .ness. The iteration procedure works by alternating updating of A from Eq. 5 and B from Eq. 6 . When the

ˆ< <iteration procedure has converged we get the symmetrically constrained solution by setting g s b and renor-ˆr r rˆmalizing b by its length. This BLLS algorithm has only linear convergence and can be quite slow when ther

Ž .calibration is imprecise, so that the function 3 is relatively flat in some directions from the minimum.

2.2. The SVD estimator

ŽWe now introduce and motivate our proposed alternative to the BLLS estimator, the SVD single value de-.composition estimator.

Result 2.2: Construction of the SVD estimator.

The SVD estimator is constructed from singular value decomposition of weighted data matrices by

RŽ .T y1 r T1ˆg a b sSVD D T susz ,Ž .ˆ ˆ r rÝr r r r 1 1

r s11

ˆ w xwhich yields a su, b sz and g ss. Here SVD PPP means the first singular value component of a ma-ˆ ˆr r r r 1

trix to be constructed below.

Justification: To derive and motivate this estimator we start from the same matrices T Ž r . and DŽ r . as in the leastsquares procedure. As mentioned in Section 2.1, the matrices T Ž r . carry all information in data about the bilinearstructure, in a sense that may be specified in terms of the concept of sufficiency. First, note that

K K KŽ r . Žk . Žk . Žk . Žk . T Žk . Ž r . T Žk . Žk .T s c Z s c AC GB qE sAD GB q c E ,Ž .Ý Ý Ýr r r r r r

ks1 ks1 ks1

so T Ž r . has the expected value ADŽ r .GBT sÝR d g a b T. Now we simply identify T Ž r . with its ex-r s1 r r r r r r1 1 1 1 1 1

pected value. In matrix-like form for all constituents simultaneously, where the vectors have matrix elements,this yields the estimating equations

Ž . T1 g a bd . . . dT 11 1 111 1 R. .. . .. .. . .s ,. .. . .� 0� 0 � 0TŽ .R d . . . d g a bT R1 R R R R R R

T Ž y1 . Ž1.or in brief notation T s Dv. Inverting the matrix D to solve for v, we obtain g a b s D Tr r r r r1Ž y1 . ŽR. Ž r .q PPP q D T . Thus if the T were error-free, the right-hand side would be exactly rank one. Allow-rR


ing noise in the T Ž r . we estimate g a b T by the first component of the singular value decomposition, whichr r r rTT y1 Tˆis the natural rank one approximation. This means that g a b sSVD D t susz . Since we alwaysŽ .ˆ ˆ rr r r r 1

ˆchoose a and b as vectors of unit length, we get the estimates a su, b sz and g ss by identifyingˆ ˆr r r r r r

terms in the two expressions. B

We propose the following algorithm for calculating the SVD estimates.

SVD algorithm:K

Žk . Žk .D s c c Compute the matrix D.Ýr r r r r r1 2 1 1 2 2ks1

KŽ r . Žk . Žk . Ž r .T s c Z Compute the vector t of matrices T .Ý r r

ks1

y1vsD t Invert the matrix D and solve for the vector of matrices v .Ts u z sSVD v Select the first singular value component for each matrix component of v .Ž .r r r r 1 r

ˆa su , b sz , g ss Identify components in the singular value decomposition.ˆ ˆr r r r r r r r

In the SVD step the singular vectors u and z can be rotated to point in the opposite direction, in principle.1 1

However, since a and b should have non-negative components there should be only one natural choice. Ifr r

this is not the case we must question the model for this data set. A few small negative estimated componentsmay be replaced by zeros or kept. Another indication of model invalidity is that the matrices forming Dy1 T arenot close to rank one. B

Remark: With only one constituent present it is easy to show that the SVD and BLLS estimators are identical.In the case of two constituents, which is taken to represent the general case, we find for SVD

Ž . Ž .1 2a and b are the first singular vectors of d T yd TŽ .1 1 22 12

Ž . Ž .1 2a and b are the first singular vectors of yd T qd T .Ž .2 2 21 11

In this way the SVD estimator puts explicit weights on the T Ž r . matrices. The weights for the BLLS estimator,Ž .on the other hand, are parameter dependent as seen from the normal equations, Eq. 4 , and therefore only im-

plicitly determined. Hence the BLLS estimator is generally not identical with the SVD estimator. B

3. Precision

For the new SVD estimator explicit, approximate error-bounds can be written down, see below. We willcompare with the BLLS estimator, for which we use approximate least squares results for the linear case withlinear constraints. For this section we assume uncorrelated homoscedastic noise for simplicity. Uncorrelatedmeans lack of correlation between measurements taken for different specimens, at different wavelengths or atdifferent chromatographic times. Assuming that the number of calibration specimens is large, or more realisti-cally that s is small, so that the estimated model falls near the true model, we may linearize to find the approxi-mate precision.

3.1. The BLLS estimator

We briefly indicate how an approximate variance–covariance matrix for the BLLS estimator can be com-puted. Under the above assumption the bilinear model may be linearized in terms of the full parameter us


T ˆŽ .a , . . . ,a ,b , . . . ,b ,g , . . . ,g in a neighborhood of the true u that includes the estimate u . If it were1 R 1 R 11 R R

not for the constraints on the parameters, the ordinary theory of linear regression models could then be applied.Ž 2 .It tells that if we have a linear problem y, Xu , s I , where y is the vector of responses, X is the matrix of

explanatory variables, and the noise is homoscedastic and uncorrelated, then the variance of the least-squaresˆ ˆ 2 T y1Ž . Ž .estimate u is Var u ss X X . In order to cope with the parameter constraints, we linearize them as well,T Ž w x.to read H ush , for some matrix H and vector h . The following result see section 4a.9 of Ref. 12 tells how

to combine the linearized design matrix X with the linearized constraints matrix H to obtain the variance–co-variance matrix for u .

Result 3.1: Asymptotic Õariance–coÕariance matrix of the BLLS estimator.

The approximate variance–covariance matrix for BLLS is given byy1

TX X H2ˆVar u ssŽ .T½ 5ž /H 0 11

The interpretation of this formula is that we first take the inverse of the given matrix and then extract its upperleft part, of the same size as XT X.

For a mathematical justification of Result 3.1, see Appendix A.

3.2. The SVD estimator

Ž .In this section we first approximately solve the eigenvector problem induced by the singular value decom-position. We then linearize the expressions for the SVD estimator found in this way and conclude by finding theasymptotic variance–covariance matrix.

Straightforward computation shows that the SVD estimator for the pure response g a b T of constituent rr r r r

is given by the first term of the singular value decomposition ofR K R

y1 Ž r . T y1 Žk . Žk . T Ž r .1D T sg a b q D c E sg a b qj ,Ž . Ž .r r r rÝ Ý Ýr r r r r r r r r r1 1 1 1r s1 ks1 r s11 1

where j Ž r . is a noise term consisting of a linear combination of the noise matrices EŽk . from the different speci-ˆmens. From the definition of singular value decomposition, a and b are also the first eigenvectors u and zˆ r r

of the eigen-problemsT TŽ . Ž . Ž . Ž .2 T T r r T r rg a a qg a b j qg j b a qj j uslu 7Ž .Ž . Ž .ž /r r r r r r r r r r r r

andT TŽ . Ž . Ž . Ž .2 T T r r T r rg b b qg b a j qg j a b q j j zslz, 8Ž .Ž . Ž .ž /r r r r r r r r r r r r

respectively. In Appendix B we solve these equations in detail to find that the SVD estimates can be expressedas

Ž .T r Ž r .a fa 1ya j b rg qj b rgˆ Ž .r r r r r r r r r

TŽ .T r Ž r .b̂ fb 1ya j b rg q j a rgŽ .Ž .r r r r r r r r r

g fg qa Tj Ž r .b ,ˆr r r r r r

Ž r . K R Ž y1 . Žk . Žk .where j sÝ Ý D c E as before. This linearization of the estimates allows us to calculateks1 r s1 r r r r1 1 1 1

variances and covariances. In the case of uncorrected homoscedastic noise the result is particularly simple and isgiven by the following result.


Result 3.2: Precision of the SVD estimator.

For uncorrelated homoscedastic noise with variance s2 the approximate covariances and variances of the SVD

estimator are

2 y1 T T TCov a ,a ss D Iya a b b Iya a rg gŽ .ˆ ˆ r r Ž . Ž .r r r r r r r r r r r r1 21 2 1 1 1 2 2 2 1 1 2 2

2 y1 T T TˆCov a ,b ss D Iya a a b Iyb b rg gŽ .ˆ r r Ž . Ž .r r r r r r r r r r r r1 21 2 1 1 2 1 2 2 1 1 2 2

2 y1 T TCov a ,g ss D Iya a b b a rgŽ .ˆ ˆ r r Ž .r r r r r r r r r r1 21 2 2 1 1 1 2 2 1 1

2 y1 T T Tˆ ˆCov b ,b ss D Iyb b a a Iyb b rg gŽ . r r Ž . Ž .r r r r r r r r r r r r1 21 2 1 1 1 2 2 2 1 1 2 2

2 y1 T TˆCov b ,g ss D Iyb b a a b rgŽ .ˆ r r Ž .r r r r r r r r r r1 21 2 2 1 1 1 2 2 1 1

2 y1 T TCov g ,g ss D a b b a ,Ž .ˆ ˆ r rr r r r r r r r1 21 1 2 2 1 1 2 2

where I is the identity matrix of suitable size. In particular the variances and the zero covariances within con-stituent are:

2 y1 T 2Var a ss D Iya a rgŽ .ˆ Ž .r rr r r r r

2 y1 T 2ˆVar b ss D Iyb b rgŽ . Ž .r rr r r r r

2 y1Var g ss DŽ .ˆ r rr r

ˆ ˆCov a ,b sCov a ,g sCov b ,g s0.ˆ ˆ ˆ ˆŽ .Ž . ž /r r r r r r r r

ˆ ˆŽ . Ž . < < < <Remark: The matrices Var a and Var b are singular due to the constraints a s b s1. For derivationˆ ˆr r r r

of the variances and covariances, see Appendix B. Note that within constituent many covariances are zero, sothe correlation structure is relatively simple. Also covariances between constituents will typically be relativelysmall. However, all covariances will vanish only if all profiles are mutually orthogonal, a situation not likely tooccur in practice. B

4. A simulation study

We carried out an extensive simulation study to answer the following important questions. How does the SVDestimator perform relative to the BLLS estimator? Do the approximate precision formulas work well with fewspecimens and large or moderate noise? Can the theoretical formulas be verified empirically? Which factors havethe highest influence on the precision of the estimates?

4.1. Design

To set up a design for the simulation study we needed those factors which influence the precision. From Re-sult 3.2 for the SVD estimator we see that the precision depends on the following factors:Ž .i s , i.e., size of noise in data,Ž .ii D, i.e., choice of concentrations, number of specimens and quality of the calibration set, andŽ . � 4iii a , b , g , i.e., the true profiles, which are influenced by the number of wavelengths, the number ofr r r r

chromatographic times, and the degree of overlap between profiles.


Fig. 2. Factor 4: elution profiles, moderately and much overlapping, respectively.

To illustrate the performance of the SVD estimator versus the BLLS estimator we simulated an LC–UV ex-Ž .ample with two constituents Rs2 . In accordance with the above discussion, we chose to investigate the fol-

lowing factors, at two levels each in a 26y1 factorial design.

Factor y level q levelŽ .1 number of chromatographic times and wavelengths 20 10Ž . Ž .2 number of specimens K 4 2Ž . Ž .3 size of noise in data s 0.0025f10% 0.0075f30%Ž .4 degree of overlap in true profiles moderately overlapping much overlappingŽ .5 degree of overlap in true spectra moderately overlapping much overlappingŽ .6 quality of the calibration design more informative less informative

ŽFactor 1 determines the size of the data matrix, which is number of times=number of wavelengths 20=20.or 10=10 . For factor 2, the q level was the minimum number of specimens possible to investigate, since we

had two constituents, see Section 2. The relative size of noise in data was of magnitude 10% and 30% of theaverage response, respectively. Figs. 2 and 3 illustrate factors 4 and 5, the form and degree of overlap in thetrue, pure responses.

Since we had only two constituents, the choice of concentration combinations can be illustrated in the xy-Ž .plane, see Fig. 4. For the more informative set y level we first chose two points well apart for the Ks2

Ž . Ž .specimens level, and then for each such point x, y also the point u x, u y with us2 to form the set for fourŽ .specimens. The less informative set q level was formed in the same way, but starts with two points closer

together. The advantage of this choice is that we can predict the difference in precision between two and fourŽ 2 .specimens. The D-matrices with Ks2 and Ks4, D and D are simply related by D s 1qu D . We se-2 4 4 2

Fig. 3. Factor 5: spectra, moderately and much overlapping, respectively.


Fig. 4. Calibration designs.

Ž . Ž . Ž . Ž .lected the points 1,2 and 2,1 for the informative set and the points 1.5,2 and 2,1.5 for the less informativeset, with us2.

The design chosen for the study was a factorial 26y1 design with the usual confounding pattern for half-frac-tions, that is the highest order interaction aliased with the identity. We made 100 replicates in each point of thefactorial experiment, and as much as feasible we let the noise matrices EŽk . be based on the same random num-bers over different design points.

4.2. Response

As measures of closeness between estimated and true elution profiles we use the statisticsp1 1 22a < <m s a ya s a yaŽ .ˆ ˆÝr r r i r i rp p is1

for each of the two elution profiles a and a , where p is the number of chromatographic times. Analogously1 2b ˆ 2 q ˆ 2< < Ž .we use the statistics m s b yb rqsÝ b yb rq for each of the two spectra b and b , wherer r r js1 jr jr 1 2

g Ž .2 2q is number of wavelengths, and m s g yg rg for each of the two lengths g and g . Using Resultˆr r r r r r r 11 22

3.2 we can calculate the expected values of ma , m b and mg for the SVD estimator. For example the expectedr r r

value of ma isr

p p 2 y11 1 1 s DŽ . r r2a TE m s E a ya f Var a s Tr I ya aŽ . Ž . Ž .ˆ ˆ Ž .Ý Ýr i r i r i r p=p r r2ž / ž /p p p gr ris1 is19Ž .

2 y1 t 2 y1s D pya a py1 s DŽ . Ž .r r r rr rs s .2 2p pg gr r r r

In the same way the other expected values are found as

qy1 s 2 Dy1 s 2 Dy1Ž . Ž .r r r rb gE m s , E m s . 10Ž .Ž .Ž .r r2 2q g gr r r r

Ž .For BLLS we do not have an explicit asymptotic covariance matrix, but we can compute ÝVar a rp,ˆ i rˆ 2Ž . Ž .ÝVar b rq and Var g rg numerically from Result 3.1.ˆi r r r r r


The mean responses in the factorial experiment are listed for constituent No. 1 in Tables 1 and 2, as 12qa b gŽ . Ž . Ž .ln m , 12q ln m and 12q ln m , where m is the mean value over the 100 replicates of the experiment. The1 1 1

constant 12 was added for convenience to make all response values positive. The logarithmic transformationchanged the previously multiplicative character of the precision indices into an additive one, so that any substan-tial interactions were neither expected nor found for the logarithmic responses. The logarithms of the approxi-

Ž Ž . Ž ..mate expected values Eqs. 9 and 10 were used for theoretical comparison with the observed responses inthe factorial experiment, by inserting the known values from the design. The effects in the factorial experimentwere estimated as signed means and thus correspond to half the effect of changing the current factor from itsminus level to its plus level. The theoretical effects are calculated from the asymptotic variances according to

Ž . Ž .Eqs. 9 and 10 . Theoretical and observed results are discussed in Section 4.3 below.

Table 1Simulation results for elution profiles and for spectra

No. Elution profile Spectrum

Ž . Ž .BLLS SVD ln ratio r2 BLLS SVD ln ratio r2

Observed Theoretical Observed Theoretical Observed Theoretical Observed Theoretical Observed Theoretical Observed Theoretical

1 0.8 1.1 1.1 1.1 y0.13 0.00 0.6 0.9 1.1 1.1 y0.28 y0.122 1.0 1.2 1.1 1.1 y0.04 0.04 0.9 1.1 1.1 1.1 y0.10 y0.013 2.7 3.0 2.7 2.7 y0.03 0.12 2.2 2.5 2.8 2.7 y0.27 y0.104 2.4 2.6 2.7 2.7 y0.15 y0.08 2.6 2.7 2.8 2.7 y0.09 y0.035 2.4 2.7 2.4 2.4 y0.04 0.12 1.9 2.3 2.4 2.4 y0.25 y0.086 2.1 2.2 2.4 2.4 y0.17 y0.11 2.3 2.4 2.4 2.4 y0.08 y0.037 3.8 4.0 4.1 4.0 y0.17 y0.02 3.6 3.8 4.0 4.0 y0.22 y0.118 4.0 4.1 4.1 4.0 y0.04 0.03 3.9 4.1 4.0 4.0 y0.06 0.019 2.0 2.3 2.4 2.4 y0.20 y0.06 0.7 1.0 2.4 2.4 y0.87 y0.69

10 1.3 1.4 2.4 2.4 y0.56 y0.51 1.6 1.7 2.4 2.4 y0.43 y0.3411 2.9 3.2 4.0 4.0 y0.57 y0.42 2.4 2.7 4.1 4.0 y0.84 y0.6712 3.7 3.7 4.0 4.0 y0.16 y0.14 3.3 3.3 4.1 4.0 y0.39 y0.3413 2.6 2.9 3.8 3.7 y0.60 y0.40 2.2 2.5 3.7 3.7 y0.76 y0.6314 3.3 3.4 3.8 3.7 y0.23 y0.16 3.0 3.1 3.7 3.7 y0.37 y0.3115 5.0 5.2 5.4 5.3 y0.21 y0.06 3.8 4.0 5.3 5.3 y0.73 y0.6616 4.2 4.2 5.4 5.3 y0.61 y0.55 4.7 4.7 5.3 5.3 y0.32 y0.3217 3.2 3.5 3.3 3.3 y0.03 0.12 2.7 3.1 3.3 3.3 y0.28 y0.1018 3.0 3.1 3.3 3.3 y0.14 y0.08 3.1 3.2 3.3 3.3 y0.10 y0.0319 4.7 4.9 4.9 4.9 y0.13 0.00 4.5 4.7 5.0 4.9 y0.25 y0.1220 5.0 5.0 4.9 4.9 0.01 0.04 4.9 4.9 5.0 4.9 y0.06 y0.0121 4.3 4.6 4.6 4.6 y0.16 y0.02 4.2 4.4 4.6 4.6 y0.23 y0.1122 4.6 4.7 4.6 4.6 y0.02 0.03 4.5 4.6 4.6 4.6 y0.06 0.0123 6.3 6.5 6.3 6.2 y0.01 0.12 5.9 6.1 6.2 6.2 y0.16 y0.0824 6.0 6.0 6.3 6.2 -0.13 -0.11 6.2 6.2 6.2 6.2 -0.02 0.0325 3.6 3.8 4.6 4.6 y0.53 y0.42 3.0 3.3 4.6 4.6 y0.83 y0.6726 4.3 4.3 4.6 4.6 y0.16 y0.14 3.9 3.9 4.6 4.6 y0.38 y0.3427 6.1 6.1 6.3 6.2 y0.10 y0.06 5.0 4.8 6.3 6.2 y0.63 y0.6928 5.4 5.2 6.3 6.2 y0.41 y0.51 5.7 5.5 6.3 6.2 y0.30 y0.3429 5.6 5.8 6.0 5.9 y0.18 y0.06 4.5 4.6 5.9 5.9 y0.72 y0.6630 4.9 4.8 6.0 5.9 y0.55 y0.55 5.3 5.3 5.9 5.9 y0.31 y0.3231 6.8 6.7 7.6 7.5 y0.42 y0.40 6.5 6.3 7.6 7.5 y0.52 y0.6332 7.2 7.2 7.6 7.5 y0.22 y0.16 7.1 6.9 7.6 7.5 y0.21 y0.31

0.03 0.03 0.02 0.03 0.03 0.020.09 0.05 0.05 0.10 0.05 0.05

The table gives values of 12qthe logarithm of the precision indices m of Section 4.2, and in the last two lines lower and upper bounds fortheir empirical standard errors. For the factorial design, see Table 2.


Table 2Simulation results for the scale parameter, g , in analogy with Table 111

Ž .No. The factorial design BLLS SVD ln ratio r2

s Design No. of points No. of specimen Form 1 Form 2 Observed Theoretical Observed Theoretical Observed Theoretical

1 y y y y y y 1.2 1.1 1.3 1.2 y0.05 y0.042 y y y y q q 1.3 1.1 1.3 1.2 y0.01 0.003 y y y q y q 2.9 2.7 2.9 2.8 0.00 y0.014 y y y q q y 2.9 2.7 2.9 2.8 0.00 y0.015 y y q y y q 2.5 2.5 2.5 2.5 y0.03 y0.016 y y q y q y 2.5 2.5 2.5 2.5 y0.03 y0.027 y y q q y y 4.2 4.1 4.3 4.1 y0.06 y0.048 y y q q q q 4.3 4.1 4.3 4.1 y0.02 0.009 y q y y y q 2.6 2.4 2.6 2.5 y0.01 y0.02

10 y q y y q y 2.6 2.4 2.6 2.5 y0.03 y0.0311 y q y q y y 4.1 3.9 4.2 4.1 y0.05 y0.0712 y q y q q q 4.2 4.0 4.2 4.1 y0.01 y0.0113 y q q y y y 3.6 3.7 3.8 3.8 y0.11 y0.0814 y q q y q q 3.8 3.8 3.8 3.8 y0.03 y0.0115 y q q q y q 5.5 5.4 5.6 5.4 y0.07 y0.0216 y q q q q y 5.5 5.4 5.6 5.4 y0.07 y0.0317 q y y y y q 3.5 3.3 3.5 3.4 y0.02 y0.0118 q y y y q y 3.5 3.3 3.5 3.4 y0.03 y0.0119 q y y q y y 5.1 4.9 5.2 5.0 y0.04 y0.0420 q y y q q q 5.2 5.0 5.1 5.0 0.00 0.0021 q y q y y y 4.6 4.6 4.7 4.7 y0.06 y0.0422 q y q y q q 4.7 4.7 4.7 4.7 y0.02 0.0023 q y q q y q 6.5 6.3 6.6 6.3 y0.06 y0.0124 q y q q q y 6.5 6.3 6.6 6.3 y0.07 y0.0225 q q y y y y 4.7 4.5 4.9 4.7 y0.11 y0.0726 q q y y q q 4.8 4.6 4.9 4.7 y0.04 y0.0127 q q y q y q 6.4 6.2 6.8 6.3 y0.20 y0.0228 q q y q q y 6.3 6.2 6.8 6.3 y0.23 y0.0329 q q q y y q 5.9 6.0 6.1 6.0 y0.08 y0.0230 q q q y q y 6.0 6.0 6.1 6.0 y0.06 y0.0331 q q q q y y 7.6 7.5 8.1 7.6 y0.27 y0.0832 q q q q q q 7.7 7.6 8.1 7.6 y0.16 y0.01

0.12 0.11 0.080.18 0.18 0.13

4.3. Results

For an illustration of the estimates from BLLS and SVD with their errors, see Figs. 5–7. These estimatesŽ .represent the design point yqy yqy , that is they were generated with moderately overlapping profiles

and spectra, two specimens, 20 chromatographic times and 20 wavelengths, the more informative calibration setand the larger s , i.e., overall about 30% noise added to the theoretical responses.

Our simulation results for constituent No. 1 are listed in Tables 1 and 2, constituent No. 2 gave similar re-sults. Given in the tables are the mean values of 12q the logarithm of the precision indices m, see Section 4.2,

Ž . Ž .for a , b and g , both empirical as observed in the simulations and asymptotic theoretical values . For the1 1 11

empirical values we give the range of the standard error at the bottom of the column. For comparisons betweenthe two methods we also give half the difference between the logarithms of the precision indices m for BLLSand SVD. The main effects of the factorial experiment are given in Table 3. All interactions were negligible.The first row of Tables 1 and 2 is the intuitively most precise point, i.e., the smaller s , four specimens, 20


Fig. 5. Estimation of scaled elution profiles; two constituents.

points, more informative design and moderately overlapping profiles and spectra. This point is coded by onlyminus signs, the remaining points of the 26y1 factorial experiment follow in standard order, see Table 2.

A first, overall impression from the tables is that there is good agreement between the observed simulationresults and the asymptotic theory, not least for the SVD estimator. This confirms that the approximate varianceformulas well represent the true precision for both BLLS and SVD. Surprisingly neither BLLS nor SVD losemuch precision when we let the elution profiles or the spectra be much overlapping.

From the given tables, we see that the SVD estimator is usually not much less efficient than BLLS for uncor-related homoscedastic noise. SVD works significantly worse than BLLS only under the less informative design,as on rows 9–16 and 25–32 in Table 1, so that in the case of a really bad calibration design the SVD estimatorshould perhaps not be advocated. The efficiency of SVD compared to BLLS is seen more clearly from Table 3,where we list theoretical and observed main effects from the factorial experiment. All interactions were negligi-ble. The tables also show a general similarity between BLLS and SVD. However, they do not behave preciselythe same, and we now focus on some of the differences.

Fig. 6. Estimation of scaled spectra; two constituents.


Ž .Fig. 7. Estimation of scale factors g values ; two constituents with "2 S.E.M., true values, BLLS and SVD, respectively.

4.3.1. Influence from designŽBLLS is much less sensitive to the choice of calibration design than SVD for the estimation of a and b but

.not g . The fact that SVD is much worse than BLLS for the less informative design may be related to the needfor inverting the matrix D, which depends on the calibration set C. For the less informative calibration set thematrix D is closer to singular than for the more informative calibration set.

4.3.2. Influence from formIn particular the SVD method is remarkably insensitive to the form factors. The moderately overlapping and

the much overlapping levels of these factors are formed from each other by moving only the profile or spectrumfor constituent No. 2, so we expect no significant changes in the estimates for constituent No. 1. We use thelog-normal distribution for the elution profiles, see Fig. 2. Some small negative main effects appearing for thefactor ‘form 1’ can be explained by a scale change induced by the log-normal distribution.

5. Real data

We have also tried the SVD and BLLS methods on real data, which need not satisfy our model assumptions.Ž .Data were generated from nine specimens with different mixtures of four polyaromatic hydrocarbons PAH :

anthracene, fluoranthene, chrysene and perylene. The instrumentation was LC–UV, measuring chromatographictimes 0–2.10 min with step about 0.36 s and wavelengths 209.5–401.5 nm with step 2 nm. We used the chro-matographic time interval 1.30–2.10 and the whole wavelength region in our analyses. With the particular LC-column the PAHs were expected to overlap to some extent, and did so, see Fig. 8.

()

M.L

inder,R.Sundberg

rC

hemom

etricsand

IntelligentLaboratory

Systems

421998

159–

178173

Table 3Main effects from the 26y 1 experiment; observed and theoretical effects for the logarithmic responses

Ž .Response method s No. of specimen No. of points Design Form 1 Form 2

Observed Theoretical Observed Theoretical Observed Theoretical Observed Theoretical Observed Theoretical Observed Theoretical

Ž .a BLLS 1.2 1.1 0.8 0.8 0.6 0.6 0.4 0.4 0.0 y0.1 0.2 0.21Ž .a SVD 1.1 1.1 0.8 0.8 0.7 0.7 0.7 0.7 0.0 0.0 0.0 0.01Ž .a BLLS 1.2 1.1 0.8 0.8 0.6 0.6 0.4 0.3 y0.1 y0.1 0.3 0.32Ž .a SVD 1.1 1.1 0.8 0.8 0.7 0.7 0.7 0.7 y0.1 y0.1 0.0 0.02Ž .b BLLS 1.2 1.1 0.9 0.8 0.7 0.7 0.3 0.2 0.3 0.2 0.0 0.01Ž .b SVD 1.1 1.1 0.8 0.8 0.6 0.7 0.7 0.7 0.0 0.0 0.0 0.01Ž .b BLLS 1.2 1.1 0.9 0.8 0.7 0.7 0.3 0.2 0.1 0.2 0.0 0.02Ž .b SVD 1.1 1.1 0.8 0.8 0.6 0.7 0.7 0.7 y0.1 y0.1 0.0 0.02Ž .g BLLS 1.1 1.1 0.9 0.8 0.6 0.7 0.6 0.6 0.0 0.0 0.0 0.011Ž .g SVD 1.2 1.1 0.9 0.8 0.7 0.7 0.7 0.7 0.0 0.0 0.0 0.011Ž .g BLLS 1.1 1.1 0.9 0.8 0.6 0.7 0.6 0.6 y0.1 y0.1 0.1 0.122Ž .g SVD 1.2 1.1 0.9 0.8 0.6 0.7 0.7 0.7 y0.1 y0.1 0.0 0.022


Fig. 8. SVD estimate of the elution profiles.

The purely random additive noise seemed to be very small, but the uncertainty in data was far from negligi-ble. The LC turned out to be affected by drift, so there were problems with centering the peaks on the correctretention times. We tried to set the retention times so that the maximums of the first peak were at the sameretention time for all specimens. This ad hoc solution helped to some extent. The pre-processing also includedsome necessary data cleaning. The time intervals were still not of exactly constant length within and betweenspecimens. Such problems are outside the scope of the present paper, however.

The resulting estimates of the elution profiles from SVD and BLLS are mostly quite good, but with someextra ‘ghost-peaks’. The result for SVD can be seen in Fig. 8, the results for BLLS looks almost the same. Wethink that the ghost-peaks are due to the drift in the LC. From inspection of data the retention times seem to beabout 1.40, 1.48, 1.58 and 1.86 min, respectively for anthracene, fluoranthene, chrysene and perylene. BLLSyields the estimates 1.40, 1.48, 1.59 and 1.93 min, and SVD differs only in the estimate of the retention time forperylene, where the estimate is 1.95 min.

The estimates of the spectra also differ very little between the SVD and BLLS method. The estimates of theŽ .spectra for chrysene can be seen in Fig. 9. The bad resolution ghost-peaks in the retention times probably yields

some mixing of the spectra. The estimated spectra for chrysene is the one that differs most between the two

Ž . Ž .Fig. 9. Spectra for chrysene; BLLS and SVD - - - .


methods. The differences in the other spectrum cannot be seen in pictures like Fig. 9, because they are to simi-lar.

6. Conclusions and further research

Our tests on real and simulated data confirm that the BLLS and SVD estimators behave quite similarly. TheSVD method is typically slightly less efficient than BLLS, or in a badly designed calibration clearly less effi-cient, under uncorrelated, homoscedastic noise. A compensating advantage of the SVD estimator is that it doesnot require iterations. Under unfavourable conditions the alternating BLLS algorithm described in Section 2.1shows a very slow convergence. Furthermore the SVD estimator has explicit asymptotic variance formulas, incontrast to the BLLS estimator, and the approximate standard errors work well even with few samples, bad cali-bration designs and very large noise-levels. We therefore advocate the use of the SVD estimator for bilinearregression as a reasonable alternative to bilinear least squares.

We are next carrying out more extensive method tests and method comparisons, on real and simulated data,including TLD and multilinear PLS type methods. We will also study how well the calibration model works forpredicting the constituents of a new sample. Other important aspects are diagnostics for detecting the presenceof unexpected interfering constituents and the treatment of other deviations from the bilinear model which mayappear in practice, like in the example of Section 5.

Acknowledgements

The second author is grateful to the Swedish Natural Science Research Council for financial support. We arealso grateful to Ludvig Moberg, Department of Analytical Chemistry, Stockholm University, who generated thePAH data and to Goran Robertsson, Scotia LipidTeknik, who found a way to translate those data from an old¨Hewlett-Packard system into Matlab.

Appendix A. Linearization and variance–covariance matrix for BLLS

In this appendix we linearize the bilinear model with its nonlinear constraints to fit it into a linear frameworkand use this to find the approximate variance–covariance matrix of Result 3.1. The set of parameters can be

Ž Ž . Ž . Ž ..expressed as a column vector us Vec A ; Vec B ; Diag G . The Vec-operator transforms a matrix to a vectorŽ . Ž .Tby putting its column vectors on top of each other, i.e., Vec M s m , . . . ,m . We also use the notation1 n

Ž . Ž .TDiag M to get the vector m , . . . ,m of diagonal elements from a n=n quadratic matrix M. We now make11 nnŽ Žk . T .a first-order Taylor expansion of Vec AC GB with respect to the parameter vector u . The necessary deriva-Ž w x.tives can be derived by tensor algebra see Graham 13 and are given by

E Vec ACŽk .GBTŽ .Žk . Žk . TD s sC GB mIA p=pE Vec AŽ .

E Vec ACŽk .GBTŽ .Ž .Žk . T k TD s sU I mGC AŽ .B q=qE Vec BŽ .

E Vec ACŽk .GBTŽ .Žk . T Žk . TD Gs sP B mC A .Ž .

E Diag GŽ .


Ž . Ž T .Here U is the permutation matrix associating Vec X with Vec X . The matrix P picks out the rows numberedŽ .rq ry1 R for rs1, . . . , R, where R equals the number of constituents. Vectorizing all observation matrices

ZŽk . jointly, this leads to a linearization of the whole three-dimensional array

Ž .Ž . Ž .1 Ž1. Ž1.1 1D D D GVec Z Vec EŽ . Ž .Vec AŽ .A B. .. . .. .. . . Vec BŽ .yzs q ,. .. . . � 0� 0� 0 � 0ŽK . ŽK . ŽK .Ž . Ž .K KDiag GŽ .D D D GVec Z Vec EŽ . Ž .A B

where z is a constant, depending only on the central point of the linearization. We also have to linearize theŽ T .constraints, which for A can be written as Diag A A s1 for A. Here the required derivative is

a T 0 PPP 01. .TT 0 . .E Diag A AŽ . . .s2 s2d .. . Až / . . 0E Vec AŽ . . .� 0T0 PPP 0 a R

The calculation for B is analogous and the linearized constraints can be written jointly as

1d 0 Vec AŽ . .A .s ..ž / ž /0 d Vec BŽ .B � 01

We have thus transformed the original bilinear model with its nonlinear constraints into the linear frameworkŽ . TYsVec Z yzsXuqe with linear constraints H ush , say.

By the Lagrange multiplier method, with multiplier l, the solution to this linear problem is found as root tothe equation system

XT X H Tu X Ys ,T ž / ž /hž / lH 0

that is

y1T T TG Gˆ 11 12u X X H X Y X Ys s .

T ž / ž /h hž /ž / ž /G Gˆ H 0 21 22l

ˆThe variance for the estimator u is finally obtained as

ˆ T T 2 T 2Var u sVar G X YqG h sVar G X Y ss G X XG ss G ,Ž . Ž . Ž .11 12 11 11 11 11

where we use the symmetry and the particular form of the matrix inverted. A slightly more general matrix resultw xis found in exercise 15, page 69 of Rao and Mitra 14 .

Appendix B. Linearization and variance–covariance matrix for SVD

Ž .In this appendix we first approximately solve the eigenvector problem induced by the singular value de-composition. We then linearize the expressions for the SVD estimator found in this way and conclude by find-ing the asymptotic variance–covariance matrix.


ˆŽ . Ž . Ž .We first concentrate on a from Eq. 7 , since an analogous reasoning works for b from Eq. 8 . In Eq. 7ˆ r r

the first three terms are rank one. The first two of them have a as eigenvector and the third has eigenvectorr

j Ž r .b , withr

K RŽ r . y1 Žk . Žk .j s D c E . 11Ž . Ž .r rÝ Ý r r1 1 1

ks1 r s11

Ž r .Ž Ž r ..TWe linearize by neglecting the second-order error term j j and conclude that the linearized form of Eq.Ž .7 has two non-zero eigenvalues, and that the corresponding eigenvectors are of the form

uska qmj Ž r .b , 12Ž .r r

Ž . Ž . T Ž r .for some coefficients k and m. Inserting Eq. 12 in the linearized Eq. 7 , letting s sa j b and t sr r r rTŽ Ž r ..Tb j b and identifying terms, we get the equation systemr r

kg 2 qkg s qmg 2 s qmg t sklr r r r r r r r r r r

kg qmg s smlr r r r r

k 2 q2kms qm2 t s1.r r

The last equation is a constraint to unit length of z. Solving this equation system for l, k and m we find

2 4 3 2 2 2(lsg r2qg s " g r4qs g qg t fg r2qg s " g r2qg sŽ .r r r r r r r r r r r r r r r r r r r r r r r

2 2(ks" lrg ys r lrg q t ysŽ . Ž .r r r r r r r

2 2(ms"1r lrg q t ys ,Ž .r r r r

with k and m of the same sign. The root approximation in l is based on the fact that t and s2 should both ber r

small relative to s . This yields one solutionr

l fg 2 q2g s1 r r r r r

2 2(k f" g qs r g q2 s q t ys f" 1ys rgŽ . Ž . Ž .1 r r r r r r r r r r r

2 2(m f"1r g q2 s q t ys f"1r g q2 s ,Ž . Ž .1 r r r r r r r r

2(and another with eigenvalue l f0. Now g f l f g q2g s fg qs . The estimate a is the eigen-(ˆ ˆ2 r r 1 r r r r r r r r rŽ . Ž r .vector of Eq. 7 corresponding to the largest eigenvalue, i.e., a fk a qm j b . The analogous calculationˆ r 1 r 1 r

ˆfor b will be left out. We conclude that the SVD estimates can be expressed asr

g fg qa Tj Ž r .bˆr r r r r r

T Ž r . Ž r .a fa 1ya j b rg qj b rgˆ Ž . 13Ž .r r r r r r r r r

TT Ž r . Ž r .b̂ fb 1ya j b rg q j a rg .Ž .Ž .r r r r r r r r r

Ž Ž ..We now use the linearizations Eq. 13 of the SVD estimates, in combination with the assumption of uncor-related homoscedastic noise to find approximative covariances. The random parts of these expressions are

T Ž r . Ž T . Ž r . Ž T .Ž Ž r ..Ta j b , Iya a j b rg and Iyb b j a rg , respectively, so we need only the variancesr r r r r r r r r r r r


Ž r . Ž Ž r ..T Ž r . Ž Ž ..and covariances of the vectors j b and j a . From the definition of j Eq. 11 , we see that ther r

covariances needed can be traced back to

k Žk . 2 TŽ .1 2Cov E b ,E b ss d b b IŽ .r r k k r r1 2 1 2 1 2

T Tk Žk . 2 TŽ .1 2Cov E a , E a ss d a a IŽ .Ž . Ž .r r k k r r1 2 1 2 1 2

Tk Žk . 2 TŽ .1 2Cov E b , E a ss d a b ,Ž .r r k k r r1 2 1 2 2 1

Ž r . Ž Ž r ..Twhere d denotes Kronecker’s delta. The covariances for j b and j a are now found in this manner:i j r r

Ž .r Ž r . y1 k k y1 k kŽ . Ž . Ž . Ž .1 2 1 1 2 2Cov j b ,j b sCov D c E b , D c E bŽ . Ž .r r r rÝÝ ÝÝr r r r r r r r1 3 2 41 2 3 3 1 4 4 2r rk k3 41 2

y1 Žk . y1 Žk . 2 T1 2s D c D c s d b b IŽ . Ž .r r r rÝÝÝÝ Ž .r r r r k k r r1 3 2 43 3 4 4 1 2 1 2r rk k 3 41 2

Ž .k Žk . y1 y1 2 Ts c c D D s b b IŽ . Ž .r r r rÝÝÝ Ž .Ž .r r r r r r1 3 2 43 3 4 4 1 2r r k3 4

y1 y1 2 Ts D d D s b b IŽ . Ž .r r r rÝÝ Ž .r r r r1 3 4 23 4 1 2r r3 4

y1 y1 2 T 2 y1 Ts D DD s b b Iss D b b I.Ž . Ž .r r r rŽ . Ž .r r r r1 2 1 21 2 1 2

Analogously,TTŽ .r Ž r . 2 y1 T1 2Cov j a , j a ss D a a IŽ .Ž . Ž . r r Ž .r r r r1 21 2 1 2

andTŽ .r Ž r . 2 y1 T1 2Cov j b , j a ss D a b .Ž .Ž . r rr r r r1 21 2 2 1

Ž .Applied on expressions 13 we obtain the covariance formulas of Result 3.2.

References

w x Ž .1 Sanchez, Kowalski, Generalized rank annihilation factor analysis, Anal. Chem. 58 1986 496–499.w x Ž .2 Bro, PARAFAC. Tutorial and applications, Chemometr. Intelligent Lab. Syst. 38 1997 149–171.w x Ž .3 Sanchez, Kowalski, Tensorial resolution: a direct trilinear decomposition, J. Chemometr. 4 1990 29–45.w x4 Burdick, Tu, McGown, Millican, Resolution of multicomponent fluorescent mixtures by analysis of the excitation-emission-frequency

Ž .array, J. Chemometr. 4 1990 15–28.¨w x Ž .5 Wold, Geladi, Esbensen, Ohman, Multi-way principal components-and PLS-analysis, J. Chemometr. 1 1987 41–56.

w x Ž .6 Bro, Multiway calibration, multilinear PLS, J. Chemometr. 10 1996 47–62.w x Ž .7 Smilde, Comments on multilinear PLS, J. Chemometr. 11 1997 367–377.w x Ž .8 de Jong, Regression coefficients in multilinear PLS, J. Chemometr. 12 1998 77–81.w x Ž . Ž .9 Geladi, Analysis of multi-way multi-mode data, Chemometr. Intelligent Lab. Syst. 7 1989 11–30.

w x Ž .10 Leurgans, Ross, Multilinear models: applications in spectroscopy, Stat. Sci. 7 1992 289–319.w x Ž .11 Kruskal, Rank, Decomposition and uniqueness for 3-way and N-way arrays, in: R. Coppi, S. Bolasco Eds. , Multiway Data Analysis,

Amsterdam, 1989.w x12 Rao, Linear Statistical Inference and its Applications, 2nd edn., Wiley, USA, 1973.w x13 Graham, Kronecker Products and Matrix Calculus with Applications, Ellis Horwood, Chichester, 1981.w x14 Rao, Mitra, Generalized Inverse of Matrices and Its Applications, Wiley, USA, 1971.

second-order calibration: bilinear least squares regression and a simple alternative

Documents