Data Reconciliation under Fuzzy Constraints inMaterial Flow Analysis

Didier Dubois1, Hélène Fargier1 Dominique Guyonnet2

1IRIT, CNRS & Université de Toulouse, France2BRGM-ENAG, Orléans, France

Abstract

Data reconciliation consists in modifying noisy orunreliable data in order to make them consistentwith a mathematical model (herein a material flownetwork). The conventional approach relies onleast squares minimization. Here we show thatthe setting of fuzzy sets provides a generalized ap-proach that is more flexible and less dependenton oftentimes debatable probabilistic justifications.Moreover the proposed setting also encompassesconstraint-based formulations using intervals.

Keywords: Material flow analysis, data reconcilia-tion, least squares, fuzzy constraints

1. Introduction

Material flow analysis consists in calculating thequantities of a certain product transiting a networkof local entities referred to as processes, consideringinput and output flows and including the presenceof material stocks. The unknowns to be determinedare the values of the flows and stocks. The basicprinciple that provides constraints on the flows isthat what goes into a process must come out, upto the variations of stock. Flows and stocks mustbe balanced, through a set of linear equations. Themass-balance equation relative to a process with nflows in, k flows out and a stock level s is written:

n∑i=1

INi =k∑j=1

OUTj + ∆s (1)

where ∆s is the amount of stock variation (positiveif∑kj=1 OUTj <

∑ni=1 INi and negative otherwise).

Such flow balancing equations in a process net-work define a linear system of the form Ayt = B, ybeing the vector of N flows and stock variations. Inorder to evaluate balanced flows and stocks, datais collected regarding the material flow transitingthe network and missing flow or stock variation val-ues are calculated. But this task may face severaldifficulties:• There may not be sufficient information to de-termine all the missing flows or stock varia-tions.• There may be on the contrary too much in-formation available and the system of balanceequations may not have any solution.

• The available information is often not suffi-ciently reliable and precise.

In this paper we address the second and third cases.If the data are in conflict with the mass-balanceequations, it may be because they are erroneous andshould be corrected: this is the problem of data rec-onciliation, a well-known problem in statistics eversince its origins. As early as the end of the XVII-Ith century, this question was addressed using themethod of least-squares. It still is the case today,but the justification is statistical and usually basedon the Central Limit Theorem and the principle ofmaximum likelihood. In this paper, we examine thelimitations of this approach and propose a prelimi-nary discussion on alternative approaches that takeinto account more explicitly data uncertainty, us-ing intervals or fuzzy intervals. Following the latterapproaches, the problem can then be solved usingcrisp or fuzzy linear programming. We outline ageneral framework based on fuzzy intervals, under-stood as flexible constraints, that encompasses theleast-squares method.

2. Data reconciliation

Data reconciliation consists in modifying measuredor estimated quantities in order to balance the massflows in a given network. The vector of flows y issubdivided into two sub-vectors x and u, i.e., k in-formed quantities xi and N − k totally unknownquantities uj , to be determined. We denote by xthe vector of available measurements xi. In general,the system A(xu)t = B has no solution such thatx = x. This absence of solution is assumed to bedue to measurement errors or information defaults.The problem to be solved is to modify x, while re-maining as close as possible to x, such that the massbalance equations Ayt = B, with y = (xu), are sat-isfied.

2.1. The least-squares approach

The traditional approach to data reconciliation [16]considers that data come from measurements, andmeasurement errors follow a Gaussian distributionwith zero average and a diagonal covariance matrix.The precision of each measurement xi, understoodas a mean value, is characterized by its standarddeviation σi. Data reconciliation becomes a prob-

8th Conference of the European Society for Fuzzy Logic and Technology (EUSFLAT 2013)

© 2013. The authors - Published by Atlantis Press 25

lem of optimization under linear constraints. In thesimplest case (assuming no u):

Find x minimizingk∑i=1

wi(xi − xi)2

such that Axt = b

It is the method of weighted least-squares used inmany data reconciliation packages such as STAN[2]. The solution is known to be of the form [16]:

x∗ = x−W−1At(AW−1At)−1A(x− b),

where W is a diagonal matrix containing terms1/wi. Weights are often of the form wi = (σi)−2.Such packages sometimes also reconcile variances

as explained in [16]. It assumes that the vector ofestimated values x has a multivariate normal distri-bution characterized, by a covariance matrix C gen-eralizing W , whose diagonal contains the variancesσ2i . The balance flows being linear, the reconciled

values x∗ depend to the estimated values via a lin-ear transformation, say x∗ = Bx hence also have anormal distribution. The covariance matrix of x∗ isthen of the form C∗ = BCBt.

2.2. Limitations of the approach

The method of least-squares is often justified basedon the principle of maximum likelihood, applied tonormal distributions, in turn justified by the CentralLimit Theorem (CLT). If pi is the probability den-sity function associated with error εi = xi − xi, themaximum likelihood is calculated on the functionL(x) =

∏ki=1 pi(xi− xi). If the pi’s are normal with

average 0 and standard deviation σi, then pi(xi−xi)

is proportional to e− (xi−xi)2

σ2i . As a consequence, the

maximum of L(x) coincides with the solution tothe least squares method. The Gaussian assump-tion seems to be made because of the popularity ofGauss law (that computes an approximation of thestandard deviation of a function f(x1, . . . xk) in thevicinity of a measurement point based on the linearpart of its Taylor expansion). The universal char-acter of this approach, albeit reasonable in certainsituations, is nevertheless dubious:

• It is not consistent with the history of statis-tics [18]. The least-squares method, developedby Legendre (1805) and Gauss (end of XVII-Ith century), was discovered prior to the CLT,as is the Gauss function. Invented precisely tosolve a problem of data reconciliation in astron-omy, the least squares method sounded naturalsince it was in accordance with the Euclideandistance. Moreover, it led to solutions thatcould be calculated analytically and it couldjustify the use of the average in the estimationof quantities based on several independent mea-sures. The normal law was discovered by Gaussas the only error function that was compatible

with the average estimator. However, CLT isa mathematical result obtained independentlyby Laplace, who later on made the connectionbetween his mathematical result and the leastsquares method.• The CLT presupposes a statistical process witha finite average E and standard deviation σ. Inthis case, the average of n random variables vihas standard deviation σ/

√n and the distribu-

tion of the average∑n

i=1vi−nE√n

is asymptoti-cally Gaussian as n increases. The fundamen-tal hypothesis behind the normal distributionis the existence of a finite σ. In practice, thisimplies that for N observations ai of v, the em-

pirical variance msd =2∑

i<j(ai−aj)2

N(N−1) remainsbounded as N increases, which is neither al-ways true nor easily verifiable.• The Gaussian hypothesis is only valid in thecase of an unbounded random variable. Ifvi is positive or bounded, assuming that thequantity En =

∑n

i=1vi

n assymptotically followsa normal distribution with standard deviationσ/√n is an approximation that may be useful

in practice but does not constitute a generalprinciple.

Based on the remarks above, it is natural to lookfor alternative methods for reconciling data that donot come from a statistical measurement process.A first alternative consists in representing error-tainted data by means of intervals and checking thecompatibility between these intervals.

3. Interval reconciliation

In practice, information on mass flows is seldomprecise: the data-gathering process often relies onsubjective expert knowledge or on scarce measure-ments published in various documents that more-over might be obsolete. Each flow value providedby a source can be more safely represented by an in-terval Xi, which in a first stance, can be consideredas encompassing the actual flow value: of course,the less precise the available information, the widerthe interval. Missing values ui can also be takeninto account: we then select as its attached intervalthe domain of possible values of the correspondingparameter (for example, the unknown grade of anore extracted from a mine and sent to the treat-ment plant can, by default, be represented by theinterval [0 ,100]%). In the least-squares approach todata reconciliation, we use weights to reflect the as-sumed variance of a Gaussian phenomenon; if suchinformation on variances σ2

i are available, we canset Xi = [xi − 3σi, xi + 3σi], as the distribution ofxi is often assumed to be Gaussian. Thus each ofthe N variables yi of the vector y = xu is delimiteda priori by an interval Yi.The representation of flow data by intervals leads

26

us to consider the reconciliation as a problem ofconstraint satisfaction; the mass balance equationsmust be satisfied for flux and stock values that liewithin the specified intervals - or, to be more pre-cise, we can restrain these intervals to the sole valuesthat are compatible with the balancing model giventhe possible values of other variables. Formally, thereconciliation problem can be expressed as follows:For each i = 1, . . . N , find the smallest and largest

values for yi, such that :

Ayt = B

yi ∈ Yi, i = 1, . . . , N

The calculation of consistent minimum and max-imum values of yi is sufficient: since all the equa-tions are linear, we can show that if there exists twoflow vectors y and y′, each being a solution to theabove system of equations, then any vector v lyingbetween y and y′ componentwise is a solution of thesystem of equations Ayt = B. The problem can ofcourse be solved using linear programming.Due to the linearity of the constraints, it may also

be solved by methods based on interval propagation.For each variable yi, equation j of the system Ayt =B can be expressed as

yi =∑k 6=i bj − ajkyk

aji, i = 1, . . . , N.

We can then project this constraint on yi and findthe possible values of yi consistent with it. Dueto the m linear constraints, the values of yi can berestricted to lie in the interval:

Yi = Yi ∩ (∩j=1...,m

∑k 6=i bj − ajkYk

aji),

where∑

k 6=ibj−ajkYkaji

is calculated according to thelaws of interval arithmetic [12]; if the new intervalof possible values of yi is more precise (Yi ⊂ Yi), itis in turn propagated to the other variables. Thisprocedure, known as “arc consistency”, is iterateduntil intervals are stabilized; it converges within afinite number of steps to a unique set of intervals[13] (for additional details on interval propagationtechniques, see [1, 11]). Note that contrary to thestatistical approach, here the model constraints andthe imprecise data are handled on a par.

4. Modeling data using fuzzy intervals

In the framework of measurement problems, Mau-ris [14] has suggested that in the case of competingerror functions (empirical probability distributionspi, i = 1 . . . k), one may refrain from choosing oneof them and consider a family of probabilities P in-stead, to represent our knowledge about x, wherepi ∈ P,∀i. In general such a representation canbe extremely complex. For instance, P should beconvex, typically the convex hull of {pi, i = 1 . . . k}.

However a very simple and convenient representa-tion is via possibility distributions having the shapeof a fuzzy interval [4].

A possibility distribution is a mapping π : R →[0, 1] such that π(r∗) = 1 for some r∗ ∈ R. It repre-sents the current information on a quantity x. Theidea is that π(r) = 0 if and only if x = r is impos-sible, while π(r) = 1 if x = r is a totally normal,expected, unsurprizing value. One rationale for thisframework is that the set Iα = {r, π(r) ≥ α} (α-cut)contains x with level of confidence 1 − α, that canbe interpreted as a lower probability bound. In par-ticular, it is sure that x ∈ {r, π(r) > 0} = S(π), thesupport of the possibility distribution.

A fuzzy interval is a possibility distribution whoseα-cuts Iα are closed intervals. They form a nestedfamily of intervals containing the core C(π) ={r, π(r) ≥ α} = 1 and contained in the support.The simplest representation of a fuzzy interval isa trapezoid defined by the core and the support.Note that this format is very convenient to gatherinformation from experts in the form of confidenceintervals.

Given a possibility distribution π, the degree ofpossibility of an event A is Π(A) = supr∈A π(r).The degree of certainty of event A is N(A) =1 − Π(Ac), where Ac is the complement of A. Apossibility distribution can be viewed as encodinga convex probability family P(π) = {P, P (A) ≥N(A),∀A measurable} (see [4] for references).Functions Π and N can be shown to compute exactprobability bounds in the sense that

Π(A) = supP∈P(π)

P (A) and N(A) = infP∈P(π)

P (A).

When several error functions are possible, onemay choose to represent them by a possibility distri-bution that encompasses them. This is the idea de-veloped by Mauris [14]. Probabilistic inequalities isone example. For instance, knowing the mean valueand the standard deviation of a random quantity,Chebyshev inequality gives a possibility distributionthat encompasses all probability distributions hav-ing such characteristics [7]. Gauss inequality alsoprovides such possibility distributions encompass-ing probability distributions with fixed mode andstandard deviation (see [15]). It yields a triangular(bounded) fuzzy interval if probability distributionshave bounded support. Hence a possibility distri-bution may account for incomplete statistical data.

Conversely, if an expert provides a probabilitydistribution that represents subjective belief, it ispossible to reconstruct a possibility distributionthat fits the Laplace principle of indifference. Whenthe available knowledge is an interval [a, b], and theexpert is forced to propose a probability distribu-tion, the most likely proposal is a uniform distri-bution over [a, b] due to symmetry. If the availableknowledge is a possibility distribution π, this sym-metry argument leads to replace π by a probability

27

distribution constructed by (i) picking at random athreshold α ∈ [0, 1] and (ii) a number at random inthe α-cut Iα of π. One may argue that we shouldbet on the basis of this probability function in theabsence of any other information. Conversely, asubjective probability provided by an expert can berepresented by the (unique) possibility distributionthat would yield this probability distribution usingthis two-stepped random Monte-Carlo process [10].In summary, fuzzy intervals, and specifically tri-

angular or trapezoidal possibility distributions, mayaccount for various kinds of uncertain information.

5. Fuzzy interval reconciliation

The interval approach does not yield the same typeof answer as the least-squares method because itprovides only intervals rather than precise values.Such intervals can be compared to reconciled vari-ances provided by current software for MFA anddata reconciliation like STAN [2]. A natural wayto obtain both reconciled values and intervals is totolerate a certain level of flexibility on the flow esti-mates using the notion of fuzzy interval: the more-or-less possible values of each flow or stock yi willbe limited by a fuzzy interval Yi. For some of thesequantities, these constraints will be satisfied to acertain degree, rather than simply either satisfiedor violated. The problem of searching for a possiblesolution then becomes an optimization problem - weseek an optimal position within all the (fuzzy) inter-vals of possible values. If no solution provides entiresatisfaction for all intervals, some will be relaxed ifnecessary [5].

5.1. A general framework

In this approach, the linear equations describing thematerial flow for each process are considered as in-tegrity constraints that must necessarily be satis-fied, but the information relative to possible valuesof each flow or stock quantity yi is represented, notas a strict interval Yi but as a fuzzy interval Yi,interpreted as a possibility distribution πi. Thisinterval may still coincide with the domain of thequantity, in the case of total ignorance.An assignment y for all yi is possible, provided it

satisfies all the constraints. In other words, the de-gree of plausibility of an assignment y = xu can beobtained by a conjunctive aggregation of the localsatisfaction degrees. Namely it is ?Ni=1π(yi) if y sat-isfies the integrity constraints, and 0 otherwise - theoperation ? being associative, commutative and in-creasing on [0, 1] (a t-norm). We may then calculatethe most plausible reconciled vectors and the asso-ciated degree of possibility by solving the followingproblem:Find the values y = xu that maximize:

π?(y) = ?Ni=1πi(yi) such that Ayt = B

Rather than providing the user with one amongstseveral optimal solutions, it is often more informa-tive to have reconciled flows in the form of fuzzyintervals obtained by projection on the domain ofeach yi:

maxy s.t. yi=v and Ayt=B

?Nj=1πj(yj)

The operator models the fact that the yi must beplaced as close as possible to the cores of the fuzzyintervals - to their center if we use triangular (ortrapezoidal) representations. In the case of “classi-cal” intervals, i.e. when degrees of membership tothe Yi’s are 0 or 1, the ? operator performs a simpleconjunction and we come back to the formulation ofSection 3. The cases ? = min and ? = product con-stitute two basic modeling choices. We may also usethe Łukasiewicz t-norm : max(0, a + b − 1), whicheliminates as impossible some vectors y that havealbeit positive but too low scores. The first opera-tor, the minimum, nevertheless presents advantagesfrom a computation viewpoint: it allows the use oftools from linear programming or interval propaga-tion.

5.2. The max-min approach

The optimization problem to be solved takes theform:

Find y∗ that maximizes

πmin(y) =N

mini=1

πi(yi) where Ayt = B

with Ayt = B. Let α∗ = minNi=1 πi(y∗i ) where y∗is an optimal solution. This implies that we cannotaim at a plausibility value α > α∗, since there willbe no simultaneous choice of the yi in the α-cutsof Yi that will form a consistent vector in the senseof the network defined by Ayt = B, whereas thereexists at least one consistent assignment of flowsat the level α∗. Note that there may exist severalsolutions that allow a level of satisfaction α∗.Once α∗ is known, we can assign to each flow an

interval of optimal values (Yi)α∗ = {yi : πi(yi) ≥α∗} by solving for each yi the following intervalreconciliation problem: Find the minimum (resp.maximum) values of yi such that Ayt = B and:

πj(yj) ≥ α∗, j = 1, . . . , N.

The fuzzy reconciliation problem fails if α∗ = 0.The optimal supports of the optimal fuzzy intervalscontaining the yi’s can be obtained if we use thesupports of the Yi in the procedure of the previoussection. This program contains on the one hand themass flow model Ayt = B which, as seen previously,is linear, and then we force the yi to belong to thesupports [si, si], i = 1, . . . N of the fuzzy intervalsYi’s.

28

5.3. Resolution methods

From a technical standpoint, the fuzzy interval rec-onciliation problem can be solved using three alter-native approaches:

Using a fuzzy interval propagation algorithmAs in the crisp case, fuzzy intervals of possible val-ues Yi can be improved by projecting the fuzzy do-mains of other variables over the domain of yi viathe balancing equations:

Y ′i = Yi ∩ (∩j=1...,m

∑k 6=i bj − ajkYk

aji),

where∑

k 6=ibj−ajkYkaji

is a fuzzy interval Aj thatcan be easily obtained by means of fuzzy inter-val arithmetics [8] since equations are linear. Notethat Yi ∩ (∩j=1...,mAj) has possibility distributionπ′i = min(πi,minj=1...,m πAj ).The propagation algorithm iterates these up-

dates by propagating the new fuzzy intervals on allthe neighboring yi’s, until their domains no longerevolve. This procedure presupposes efficient fuzzyinterval representation schemes must be used. Typ-ically we should use piecewise linear fuzzy intervals[17] including subnormalized ones. Eventually, op-timal (maximally precise) fuzzy intervals Yi

∗ are ob-tained as resulting fuzzy domains of the reconciledflows. These fuzzy domains may be subnormalized:at least one of them has height hi = supyi π

∗i (yi) =

α∗ that may be less than 1 and may contain a sin-gle value. However the heights hj of other opti-mal fuzzy intervals may be greater that α∗. Theirhj-cuts contain the optimal values y∗i . However,this method will only provide the fuzzy intervalswith possibility distributions min(πi, α∗) since fuzzyarithmetic methods applied to fuzzy intervals of var-ious heights only preserve the least height [8].

Using α-cuts In order to take advantage of thecalculation power of modern linear programmingpackages, a simple solution is to proceed by di-chotomy on the α-cuts of the fuzzy intervals : onceeach Yi is cut at a given level α, we obtain a sys-tem of equations as in Section 3, replacing Yi by theinterval (Yi)α; this system can therefore be solvedby calling an efficient linear programming solver. Ifthe solver finds a solution, the level α is increased; ifnot, i.e., if it detects an inconsistency in the systemof equations, the value α is decreased, etc. until themaximum value α∗ is obtained with sufficient preci-sion, along with the corresponding intervals (Yi)α∗ .

Using fuzzy linear programming When thefuzzy intervals are triangular or trapezoidal (or evenhomothetic, as in the case of L-R fuzzy numbers),it is possible to write a (classical) linear programin order to obtain the value of α∗, then obtain theoptimal ranges (Yi)α∗ ’s for the reconciled flows. It

is necessary to model the fact that the global de-gree of plausibility of the optimal reconciled val-ues is the least among the local degrees of possi-bility, i.e., we should maximize a value less thanall the πi(yi), hence we should write N constraintsα ≤ πi(yi), i = 1, . . . , N . When the original fuzzyintervals are triangular with core yi, each constraintis written in the form of two linear inequalities, onefor each side of the fuzzy intervals. All these equa-tions being linear, we can then use a linear solverto maximize the value α such that:

Ayt = B

si ≤ yi ≤ si, i = 1, . . . , Nα(yi − si) ≤ yi − si, i = 1, . . . , Nα(si − yi) ≤ si − yi, i = 1, . . . , N

The same type of modeling yields the inf and suplimits of the α∗-cuts for the reconciled intervals Y ∗i(maximizing and minimizing yi, letting α = α∗ inthe constraints above). By virtue of the linearityof the system of equations and of the membershipfunctions, we can reconstruct the reconciled Y ∗i upto possibility level α∗ by linear interpolation be-tween the cores and the optimal supports obtainedby deleting the third and fourth constraints in theabove program (although strictly speaking the rec-onciled fuzzy intervals might only be piecewise lin-ear).

It is possible (and recommended) to iterate theabove procedure and refine the optimal intervals(Yi)α∗ by instanciating the quantities yi such that(Yi)α∗ reduces to a singleton {y∗i } as described in[6], leaving other variables in their optimal α∗-cuts.Namely, let V1 = {i : (Yi)α∗ = y∗i }. We can solvethe problem of maximizing the value α such that

Ayt = B

si ≤ yi ≤ si, i 6∈ V1

yi = y∗i , i ∈ V1

α(yi − si) ≤ yi − si, i 6∈ V1

α(si − yi) ≤ si − yi, i 6∈ V1

α ≥ α∗

Then we get an optimal value α∗1 > α∗, and we canlook for the optimal ranges (Yi)α∗1 , i ∈ V1 some ofwhich again reduce to singletons. So, at this secondstep we have instanciated a set V2 ⊃ V1 of variables.We can iterate this procedure until all variablesare instanciated, at various levels of optimal pos-sibilities. Eventually, it delivers precise reconciledvalues along with possibility distributions aroundthem. These values are Pareto-optimal in thesense of the vector-maximisation of the vectors(π1(y1), . . . πN (yN )).

Among the three approaches, the latter based onfuzzy linear programming looks like the most con-venient one.

29

y1 y2 y3 y4

Least squares methodOriginal data 24± 2 16± 3 15± 4 22± 5

Reconciliated values 23, 8 15, 5 15, 9 23, 4Reconciliated fuzzy intervals

Triangular fuzzy intervals (22, 24, 26) (13, 16, 19) (11, 15, 19) (17, 22, 27)α∗ : 11

14Reconciliated cores 23 + 4/7 15 + 5/14 15 + 6/7 23 + 1/14

Reconciliated supports [22, 26] [13, 19] [11, 19] [17, 27]

Table 1: Reconciliated flows for Example 1

6. Some examples

We present simple examples in order to compare thestatistical and fuzzy approaches

6.1. One-process case

We consider the example illustrated in Figure 1,which is composed of four flows (y1, y2, y3, y4) andone process (P1). Flows y1 and y2 enter the pro-cess, while y3 and y4 exit the process. There are nostocks. In this example we have symmetric trian-gular fuzzy intervals Y1 = 24± 2, Y2 = 16± 3, Y3 =15±4, Y4 = 22±5. In the least-squares method, weinterpret the half-length of the interval as a stan-dard deviation.

Figure 1: Example 1

With the fuzzy interval approach, the calculation ofα∗ using linear programming is obtained by solvingthe following linear problem: Maximize α such that:

y1 + y2 = y3 + y4

22 ≤ y1 ≤ 26α · (26− 24) ≤ 26− y1

α · (24− 22) ≤ y1 − 2213 ≤ y2 ≤ 19

α · (19− 16) ≤ 19− y2

α · (16− 13) ≤ y2 − 1311 ≤ y3 ≤ 19

α · (19− 15) ≤ 19− y3

α · (15− 11) ≤ y3 − 1117 ≤ y4 ≤ 27

α · (27− 22) ≤ 27− y4

α · (22− 17) ≤ y4 − 17

Figure 2: Example 2

The results obtained using the two methods(least-squares and fuzzy interval reconciliation) areprovided in Table 1. We note that the alpha-cutsof the fuzzy intervals at level α∗ after propagationare singletons and that the maximum deviation be-tween the initial and reconciled values is smaller inthe case of the fuzzy method than with the least-squares method.

6.2. Two-process example

We consider the example in Figure 2, composed offour flows (y1, y2, y3, y4) and two processes (P1 andP2). Flows y1 and y2 both enter process P1; y3exits P1 to enter P2, two flows exit P2: y4 and y2,while the latter is recycled into P1. In this exampleY1 = 20± 3, Y2 = 10± 2, Y3 ∈ 20± 4, Y4 = 16± 3.For the approach using fuzzy intervals, the calcu-

lation of α∗ by linear programming is obtained bysolving a system of equations similar to that of theprevious case. We obtain α∗ = 1/3. We can alsoobtain this value by calculating the height of Y1∩Y4.Indicated in Table 2 are the cuts at level 1/3 and thesupports of the reconciled fuzzy intervals. We notethat reconciled values obtained by least-squares areat the center of the supports of the reconciled inter-vals obtained using the fuzzy interval method.

However, it is possible to refine the remaining in-tervals. If we retain the information y∗1 = y∗4 = 18and run the fuzzy interval propagation procedureagain, we verify that the intersection Y2 ∩ (Y3− 18)has a height of unity, obtained for y2 = 10. We

30

can also fix y3 = 28 considering Y3 ∩ (Y2 + 18).We can therefore verify that π1(18) = π4(18) =1/3, π2(10) = π3(28) = 1 and therefore that theleast-squares solution coincides in this particular ex-ample with the Pareto-optimal solution of the fuzzydata reconciliation problem. The former exampleshows that this is not always the case.

6.3. Comparing reconciled values: a genericexample

Consider a single process with n inputs xi and asingle output x0 =

∑ni=1 xi. Suppose all mea-

sured inputs are xi = a > 0 while x0 = ka > 0.One may argue that, assuming the xi’s have thesame variance 1, x0 has variance equal to n. It iseasy to obtain least squares estimates, minimizing∑ni=1(xi − a)2 + (x0−a)2

n under the balancing con-straint. It is easy to find that xLS0 = a(k+n)

2 andxLSi = a

2 + ak2n . Note that limn→∞ xLSi = a/2 and in

fact a2 < xLSi ≤ a(k+1)

2 . All reconciled flows linearlyincrease to infinity if k increases.In the fuzzy interval approach we can assume tri-

angular membership functions : Xi has mode aand support [a − α, a + β], where the magnitudesof α, β depend on the available knowledge. Sup-pose that the relative error of the data is every-where the same so that X0 has mode ka and sup-port [k(a−α), k(a+β)]. The reconciled value for x0is obtained as the value for which the intersectionX0 ∩ nXi has maximal positive possibility degree.There are two cases

x∗0 ={nka(α+β)nα+kβ if k ≤ n and k(a+ β) > n(a− α)

nka(α+β)kα+nβ if k ≥ n and k(a− α) < n(a+ β).

It can be checked that the least squares solutionis encompassed by the fuzzy interval approach. Ifk ≤ n, x∗0 = xLS0 if and only if α, β are chosen suchthat nα = kβ > a(n−k)

2 (the inequality makes thefuzzy reconciliation problem feasible). Likewise, ifn ≥ k, the condition is kα = nβ > a(n−k)

2 .Finally we check when x∗0 is closer to the esti-

mated value ka than xLS0 . For instance, if k ≤ n,x∗0 > ka and xLS0 > ka, and xLS0 > x∗0 > ka pro-vided that kβ < nα.

7. A unified framework for least squaresand fuzzy interval reconciliation

If we select the product for operation ? in the gen-eral formulation of Section 5.1, the reconciliationproblem boils down to maximizing the expressionπ�(y) =

∏Ni=1 πi(yi) under constraints Ayt = B.

If in addition we choose to use Gaussian shapes

πi(y) = e− (yi−yi)2

σ2i for the fuzzy intervals, it becomes

clear that this formulation brings us precisely backto the maximum likelihood expression L(x). There-fore the fuzzy interval approach defined in Section

5.1 captures the least-squares method as a specialcase, minimizing the distance to estimated values inthe sense of the l2 norm.

With ? = min and triangular fuzzy intervals Yicentered around measured values yi, solving themax-min fuzzy constraint problem, reduces to min-imizing the maximal weighted absolute deviation:

e∞(y) = maxi=1,...,N

|yi − yi|σi

,

using an l∞ norm instead of the Euclidean l2 norm.Here σi is interpreted as the spread of the fuzzyinterval Yi.Similarly, choosing a ? b = max(0, a + b − 1) un-

der the same hypotheses comes down to minimizinga weighted sum of absolute errors, i.e., use the l1norm:

e1(y) =∑

i=1,...,N

|yi − yi|σi

.

More generally recent works on penalty-based ag-gregation [3] may help us find an even more generalsetting for devising reconciliation methods in termsof general penalty schemes when deviating from themeasured data flows.

8. Conclusion

In the context of the mass flow reconciliation prob-lem, we often deal with scarce data of various ori-gins, pertaining to different quantities, that we canhardly assume to be generated by a standard ran-dom process. It seems more natural to concentrateefforts on the choice of a distance (l1, l2, l∞, . . . ) forminimizing the error rather than to invoke the CLTto justify the least-squares method. A fuzzy-set ap-proach to data reconciliation has been proposed. Itsadvantages are:

• Its general character: in a formal sense, it gen-eralizes the least-squares method without be-traying the principle of maximum likelihood.Indeed, it is well-known that a likelihood func-tion is a special case of a possibility distribution[9] and we see clearly that the likelihood L(x)is a special case of π?(x), with ? = product.• Its clear conceptual framework, both for repre-senting uncertainty pervading the data (statis-tical or subjective) and for leaving the choiceof the distance that enters the error functionto the user. The reconciled ranges around thereconciled values are also more easy to inter-pret than the reconciled variances, as they re-sult from standard interval propagation.• The opportunity of solving the problem in themax-min case, using standard methods andsoftware.

However, our framework does not encompass theprobabilistic method of variance reconciliation de-scribed in Section 2.1, since the latter views the nor-mal distribution on flow measurements as a data

31

y1 y2 y3 y4

Least squares methodOriginal data 20± 3 10± 2 28± 4 16± 3

Reconciliated values 18 10 28 18Reconciliated fuzzy intervals

Triangular fuzzy intervals (17, 20, 23) (8, 10, 12) (24, 28, 32) (13, 16, 19)α∗ : 1

3Reconciliated cores [18, 18] [8 + 2

3 , 12− 23 ] [26 + 2

3 , 29 + 13 ] [18, 18]

Reconciliated supports [17, 19] [8, 12] [25, 31] [17, 19]Reconciliated cores: 2d round [18, 18] [10, 10] [28, 28] [18, 18]

Table 2: Reconciliated flows For Example 2

generation process and not as an additional con-straint. In contrast one could consider reconcilingvariances as a data fusion problem.Mind that minimizing a sum of absolute valued

deviations (and to a lesser extent quadratic), runsthe risk of making certain values of xi deviate sig-nificantly from the data xi, whereas the max-minapproach is designed to keep all of them as closeas possible to the initial data. The latter approachseems to be more reasonable if the data come assingle estimate from an expert or other sources foreach quantity. This approach is currently beingstudied for analyzing the material flow of rare earthelements in the anthroposphere of the EU-27.

Acknowledgements This work is supported bythe French National Research Agency (ANR), aspart of Project ANR-11-ECOT-002 ASTER “Sys-temic Analysis of Rare Earths - flows and stocks”.

References

[1] F. Benhamou, L. Granvilliers, F. Goualard. In-terval Constraints: Results and Perspectives.New Trends in Constraints, LNAI 1865, pages1-16, Springer, 2000. 22nd European Symposiumon Computer Aided Process Engineering (I.D.Lockart Bogle, M. Fairweather, Eds.), Elsevier2012, 122-126

[2] P.H. Brunner and H. Rechberger (2004). Practi-cal Handbook of Material Flow Analysis. LewisPublishers.

[3] T. Calvo, G. Beliakov Aggregation functionsbased on penalties, Fuzzy Sets and Systems,161(10), 2010, 1420-1436.

[4] D. Dubois Possibility theory and statistical rea-soning Comput. Stat. & Data Anal., 51, 47-69,2006

[5] D. Dubois, H. Fargier, H. Prade. Possibility the-ory in constraint satisfaction problems: Han-dling priority, preference and uncertainty. Ap-plied Intelligence, 6, 287-309, 1996.

[6] D. Dubois, P. Fortemps. Computing improvedoptimal solutions to max-min flexible constraint

satisfaction problems. Eur. J. of Operation Re-search, 118, 95-126, 1999.

[7] Dubois D. Foulloy L. Mauris G. Prade H.Probability-possibility transformations, trian-gular fuzzy sets, and probabilistic inequalities.Reliable Computing. 2004. 10, 273-297

[8] D. Dubois, E. Kerre, R. Mesiar, H. Prade.Fuzzy interval analysis. In: Fundamentals ofFuzzy Sets, Dubois, D. Prade, H., Eds: Kluwer,Boston, Mass, The Handbooks of Fuzzy Sets Se-ries, 483-581, 2000.

[9] D. Dubois, S. Moral, H. Prade. A semanticsfor possibility theory based on likelihoods. J. ofMath. Anal. Appl., 205, 359-380, 1997.

[10] D. Dubois, H. Prade, Philippe Smets: A def-inition of subjective possibility. Int. J. Approx.Reasoning 48(2): 352-364 (2008)

[11] L. Granvilliers, F. Benhamou. Algorithm 852:RealPaver: an interval solver using constraintsatisfaction techniques. ACM Trans. on Mathe-matical Software 32(1), 138-156, 2006.

[12] L. Jaulin, M. Kieffer, O. Didrit, E. Walter. Ap-plied Interval Analysis, Springer, London, 2001.

[13] O. Lhomme. Consistency Techniques for Nu-meric CSPs. Proc. Int. Joint Conf on ArtificialIntelligence(IJCAI 1993), pages 232-238.

[14] G. Mauris: Expression of Measurement Uncer-tainty in a Very Limited Knowledge Context:A Possibility Theory-Based Approach. IEEE T.Instrum. and Meas. 56(3): 731-735 (2007)

[15] G. Mauris: Possibility distributions: A unifiedrepresentation of usual direct-probability-basedparameter estimation methods, Int. J. of Ap-prox. Reasoning, 52(9), 1232-1242 (2011).

[16] S. Narasimhan, C. Jordache, Data reconcili-ation and gross error detection: an intelligentuse of process data, Gulf Publishing Company,Houston, 2000.

[17] H. Steyaert, F. Van Parys, R. Baekeland and E.Kerre. Implementation of piecewise linear fuzzyquantities, Int. J. Intelligent Systems, 10, 1049-1059, 1995.

[18] S. M. Stigler. The History of Statistics: TheMeasurement of Uncertainty before 1900. Belk-nap Press/Harvard University Press, 1990.

32