Multi-criteria analysis of ranking preferences on residential traits.

Dominique Fischer Professor, Property Studies Curtin University Perth, Western Australia

Pair-wise comparisons assume additivity, transitivity and equal weighting of properties’ traits. This paper explores multi-criteria treatments of pair wise comparisons in order to reveal the implicit hierarchical decision making process used by analysts who perform residential valuations. This treatment is presented as an extension of traditional grid treatments used extensively by professsional valuers.

Multiple criteria analysis

Multi-Criteria Decision Analysis (MCDA) is based on computational instruments used to support the subjective evaluation of a finite number of options under a finite number of performance criteria, by a single decision maker or by a group of decision makers.

Decisions require choices among a number of options. Yardsticks can be used to rank the available options under various performance criteria. Decision makers must judge the options performance under selected criteria and simultaneously they must weight the relative importance of the criteria in order to arrive at a global judgement. Finally, as a group, decision makers must weight the relative judgment of each group member. Moreover, in a group of decision makers each member faces the question of how to judge the quality of the other members and their relative power positions before an acceptable compromise solution emerges.

The MCDA technique has been an active part of the field of operation research for at least 30 years. From the pioneering work of (Keeney and Raiffa 1976),(Saaty 1980),(Roy 1985), (Winterfeldt and Edwards 1986),(French 1988), (Vincke 1989), (Stewart 1992); (Roy and Bouyssou 1993),(Russo and Schoemaker 1989)

Over the years the various emerging approaches have diverged. Four perspectives can now be identified.

1. A descriptive perpective that tells us how decisions are made by decision makers faced with complex set of options under conflicting viewpoints;

2. A normative perspective that suggests how decision makers should behave and how MCDA should work, via logical rules which are based upon certain fundamental axioms such as the transitivity of preferences. The typical products of the approach are Multi-Attribute Value Theory (MAVT) and Multi-Attribute Utility Theory (MAUT), to be used for decision problems with certain and with uncertain outcomes respectively. This perpective is usually described as the American branch of MCDA

3. A prescriptive approach that describes how decision makers could improve the decision process and the decisions themselves, and how MCDA could support such a process.

4. A constructive perspective that questions the existence of a coherent, well-ordered system of preferences and values in the decision maker's mind.The constructivists suggest that the decision maker's preferences and values, initially unstable will be shaped by MCDA. This constructive perspective seems to be favoured by French and European analysts (Roy 1985),(Roy and Bouyssou 1993) and Roy and (Roy and Vanderpooten 1996).

MCDA techniques have received very little attention for Urban and Real Estate researchers. An early working paper from this author (Achour-Fischer and Moffet 1982) used MCDA to analyse the decision to purchase office buildings by institutional investors. This paper - published in French - has had no significant impact.

More recently the approach has be used to explain housing choices (Ball and Srinivasan 1994) or to model environmental quality: (Bender, Din et al. 1999)), to determine preferences on office quality attributes (Ho 1999) or to asset stigma effects on land and property values (Chan 2002)

The present paper suggests that MCDA could facilitate the choice and treatment of comparable sales in the traditional grid analysis used in the market approach to valuation. It also suggest, that in a constructive perspective it can be used to clarify the decision makers (valuers) cognitive strategies.

Paper organisation

The paper is organised as follows:

1- With a brief review of the market approach and its academic modifications a very simple example is used to illustrate some approaches used in grid comparison (traits adjustments, weighted adjustments and quality grid optimisation);

2- The same example is then recycled to illustrate how MCDA can be used to rank the various comparables. The ranking is based on an a priori weighting of the criteria (the so called top-down approach);

3- A more complex example is finally illustrated to show the potential and limitations of the technique;

Predictably, the paper will conclude that MCDA is a powerful instrument that deserves further investigation. Less predictably it will conclude that practical pair-wise comparisons does not require such a heavy-duty tool and that a traditional Excel-aided grid adjustment treatment is a cost effective and more practical way to deal with this type of analysis.

Since this ‘paper’ is not yet a paper but a simple conference presentation, it should be read with the support of the attached Excel simulations used during the presentation.

The basic treatment of comparative pricing: keep it in the family.

Scenario 1: the “twin” trick

If my neighbour's house, perfectly identical to mine, was sold for 179 000 $ yesterday, my house should be worth around the same price.

This is the best and ideal application of the “market approach” Simple and cheap, this approach makes lots of sense and does not raise major theoretical issues. Comparable properties should command identical prices. Essentially this is how kilos of potatoes are sold on the local market and how shares are priced on the stock market. A kilo of identical potatoes, transacted now, is priced as any other kilo sold at the same time. One share transacted now, is priced as any other share transacted at the same time. One identical house transacted now… etc.

One observed property transaction is sufficient to infer the probable value of a “twin” property transaction. It cannot get any simpler... so simple in fact that one would hardly ever need the professional services of a superbly trained professional appraiser.

Fortunately, things are never that simple.

Scenario 2: the “sibling” trick

My neighbour's house, perfectly identical to mine, was sold for 179 000 $ yesterday. However his house has a carport, while mine does not. Thus my house should be valued a bit less. How much less?

In this case we need to adjust for a different trait (the carport) in order to bring both properties to the same level of comparison. When not dealing with twins — but mere siblings — the pricing requires some adjustments.

Scenario 3: Cousins

My neighbour's house (the one with a carport) was sold for 179 000 $ one year ago. According to the real estate agent, neighbourhood prices have been growing rapidly during that one-year period. Thus, my house should be worth a bit less because of the absence of a carport, and a bit more because of the one year during which prices have risen.”

Now things are getting a bit more interesting: You need to adjust (down) for the absence of a carport and also adjust (up) to take into account one year of price inflation.

Scenario 4: more distant cousins

My neighbour’s house (the one with a carport) was sold for 179 000 $ one year ago. According to the real estate agent, neighbourhood prices have been growing rapidly during that one-year period. Then (from my nosy butcher) I also learn that my neighbour has facilitated his transaction by granting a favourable vendor's mortgage to his buyer.

Now you must adjust for the carport, for the passage of time (1 year), and for the favourable vendor's mortgage... not so much fun anymore!

These simplistic tales are sufficient to describe the nature of the “market comparison technique” and to illustrate that the required adjustments may raise quite a few interesting difficulties. Let us now describe the general approach in more detail.

In a `normal’ market, goods that provide identical flows of services (or identical flows of income) have the same market value. An informed purchaser would not accept to pay more for a good than the price he would have to pay for an identical good. This was described above as the “substitution criterion”.

The price of a property cannot be higher or lower that the price paid for an identical available property. If the asking price were higher, then the potential buyer would purchase a “twin” at a better price. If the price were lower, a purchaser could snap it up and sell it immediately at market price. This is an illustration of price "arbitrage" that is practiced on the stock market.

In practice, this arbitrage may take several steps before it reaches the proper market price. Thus, the market, adjusting from one transaction to another, behaves as a rational filter: the “off-the-range” transactions are filtered out. Sellers and buyers confront their respective expectations and their eventual errors of appreciation are rapidly corrected by other similar transactions. With frequent and continuous arbitrage, a specific property will sell — thus should be valued — as a “twin” properties sold recently.

Less basic treatments for grid adjustments

The simple `twinning’ approach is the one favoured by practicing valuers. It has been frequently criticised and ‘improved’ by over 20 years of academic contributions.

One of the first significant article is (Colwell, Cannaday et al. 1983) who demonstrate the close similarity between grid treatment and ordinary least squares. The Colwell and al. adjustement is illlustrated below in our initial example.

Then, the ‘behavorists’ compared professional with expert behaviour (Diaz 1990), (Spence and Thorson 1998) when using simple grid analysis. The results favours the experts, but not by much.

Similar results can be found in (Northcraft and Neale 1986) and even more convincingly in (Kiel and Zabel 1999).

(Manaster 1991) demontrates the convergence of adjustment results on valuations performed by different valuers using slightly different adjustment techniques.

(Crookham 1995) examines the quality of commercially generated comparable sales databases commonly used by appraisers who apply the sales comparison approach, while (Boronico and Moliver 1997) explore the contradictory estimates obtained using different units of comparison.

(Isakson 1986) and (Isakson 1988) suggests a technique very close to our basic ‘twinning’ concept: the Nearest Neighbors Appraisal Technique (NNAT) in which the final estimate of value is calculated as a weighted average of the actual selling prices of the comparable properties.

(Vandell 1991) presents a minimum variance (among the adjusted values of the comparable sales) approach for selecting and weighting comparable properties, while (Gau, Lai et al. 1992) present a variation of Vandell’s techniques in which the coefficient of variation replaces Vandell’s variation as the measure to be minimized. In both techniques, the adjusted values of the comparable properties are calculated using the dollar additive technique and OLS derived adjustment factors.

(Green 1994) looks closely at the Vandell and Gau, et. al. techniques and shows that under classical OLS assumptions, Vandell’s approach is preferable. But, in the presence of omitted variables or heteroskedasticity, Green finds that the Gau, et. al. approach is better.

(Pace 1998) presents the sales comparison approach in a mathematical format (matrix algebra). In doing this, Pace is better able to focus upon some of the more interesting aspects of this approach. Unfortunately, Pace’s model does not completely capture the sales comparison approach as practiced by appraisers.

Various authors have also tried to account more explicitely with qualitative ranking adjustments notably (Dilmore 1984). An adaption of such technique is also illustrated below since it leads quite naturally to the mutiple criteria extension suggested in this paper.

Much of this literature can be found in the (Lentz and Wang 1998) review of 141 academic books and articles that deal, in one way or another, with the sales comparison approach.

Illustration of various grid adjustments (see the attached workbook)

1. Straight and simple twinning

The basic treatment starts with a filtering treatment of all relevant recent transactions in order to get as close a possible to the perfect twin. Since identical twins are rare, pairiwse additive adjustments are required to bring the cousins or siblings down or up to the subject property traits values.

With:

Vs the value of the subject property

Vc the value of a paired comparable

B the trait coefficient

Xs trait measurement for the subject

Xc trait mearurement for the comparable

p the percentage difference

Adjustement can be absolute and written as (additive dollar adjustment):

Vs = Vc + b (Xs – Xc)

Or relative and written as an additive percentage adjustment:

Vs = Vc + p × (Xs – X) × Vc

= Vc × (1 + p (Xs – Xc))

The following example illustrates the additive dollar adjustments of 5 traits (date of sale, land area, liveable area, number of bathrooms and number of bedrooms) from 4 comparable sales (table 1).

2. Collwell and al. weighting of family ties

Since the ideal comparisons should be drawn from ‘identical twins’, the relative importance of each comparable sales could be weighted to reflect the closeness between the subject and the paired sibling. Less comparability means a smaller weight and more comparability means a higher weight. Numerous techniques have been suggested to finesse such a weighting system. The following treatment is based on Colwell and al. weighting scheme. It illustrates two options applicable to a simple additive dollar adjustment grid: an absolute weighting of the traits and a squared weighting.

In the squared adjustment scheme the appropriate weights are computed as:

Where the double summation (over k and j) is the sum of the squared values of all adjustments made within the grid and where the simple summation) over j is the sum of all adjustments made for comparable i.

m is the number of traits requiring adjustments. Here m = 5 traits

n is sthe number of comparables. Here n = 4

The following table illustrates the procedures for the additive adjustement with absolute values (table 2) and the squared adjustments (table 3). The previous data are used again as a continous example:

Table 2

1 2 3 4

138 000.00 $ 112 600.00 $ 110 000.00 $ 146 800.00 $ 360 379 363 340

7 4 13 1110 68 98 102

3 2 2 31.5 1 1 1

15,939.00 $ 7,431.60 $ 23,595.00 $ 2,422.20 $12 000.00 $ .00 $ .00 $ 12 000.00 $ 4 000.00 $ .00 $ .00 $ .00 $ 4 521.00 $ 14 467.20 $ 904.20 $ 904.20 $

36 460.00 $ 21 898.80 $ 24 499.20 $ 15 326.40 $ 98 184.40 $ 61 724.40 $ 76 285.60 $ 73 685.20 $ 82 858.00 $

3 3 3 398 184.40 $ 98 184.40 $ 98 184.40 $ 98 184.40 $

0.20955 0.25899 0.25016 0.2813028 918.26 $ 29 161.99 $ 27 517.51 $ 41 294.93 $ 126 892.70 $ adjusted values

Differencesn-1sumweight

Total of absolute values for adjustments

Additive adjustments with absolute values - a la Colwell, Cannaday, Wu (1983)

Transaction price (in $)

The comparable

Land area (pi. Ca.)Recorded transaction (in month

For bathroomsFor liveable area

Indexation (for time of sale)For bedrooms

Number of bathroomsAdjustments ($)

Liveable area (in m2)Number of bedrooms

Table 3

1 2 3 4

138 000.00 $ 112 600.00 $ 110 000.00 $ 146 800.00 $ 360 379 363 340

7 4 13 1110 68 98 102

3 2 2 31.5 1 1 1

15,939.00 $ 7,431.60 $ 23,595.00 $ 2,422.20 $12 000.00 $ .00 $ .00 $ 12 000.00 $ 4 000.00 $ .00 $ .00 $ .00 $ 4 521.00 $ 14 467.20 $ 904.20 $ 904.20 $

434491 162.00 $ 264528 554.40 $ 557541 602.64 $ 150684 630.48 $ 1407245 949.52 $ 972754 787.52 $ 1142717 395.12 $ 849704 346.88 $ 1256561 319.04 $

3 3 3 31407245 949.52 $ 1407245 949.52 $ 1407245 949.52 $ 1407245 949.52 $

0.23042 0.27067 0.20127 0.29764 1.0000031 797.37 $ 30 477.97 $ 22 139.57 $ 43 693.67 $ 128 108.57 $

Indexation (for time of sale)For bedrooms

Number of bathroomsAdjustments ($)

Liveable area (in m2)Number of bedrooms

Total of squares of values for

Additive dollar squared adjustment method

Transaction price (in $)

The comparable

Land area (pi. Ca.)Recorded transaction (in month

For bathroomsFor liveable area

adjusted values

Differencesn-1sumweight

3. Modified (Fischer’s) quality grid using optimisation techniques

The previous treament deal adequatly with traits that can be expressed in an ordinal scale but cannot be applied to qualitative traits. Here again various techniques and non-parametric treatments have been suggested.

We will now illustrate an optimisation treatment of qualitative traits. Four comparable properties are now judged on 3 additional ‘qualities’ : location, landscaping and level of maintenance. This treatment is based on the initial Dilmore’s article (Dilmore 1984) but relies here on Excel capabilities to treat linear programming problems.

1- Choose the relevant traits, assign a weight that should reflect their relative importance and score the comparables with respect to the traits. Arbitrary but reasonable weights are chosen by the decision maker. More simply equal weights can be chosen intially.

2- Choose the most representative quantitive indicator of value (here liveable area) and compute the weighted scores. Thus at least one quantitative significant trait must be chosen.

3- Apply the resulting average price per m2 of points to the observed prices (comparables);

4- Using a constrained optimisation process find the weighting that minimize the squared error from observed prices under constraints of the weights being included between 1 and 0 and the sum of weights must equal to 1.

5- Applying the resulting weighting to the subject property

Weights Scoring scale

From 'less to more' 1 2 3 4

0.35 1 to 3 3 2 2 3

0.12 1 to 3 2 3 1 2

0.53 1 to 3 2 3 2 2

1.000001 7 8 5 7

Landscaping

Comparables

1- choose the relevant traits, assign a weight that should reflect their relative importance and score the comparables with respect to the traits.

Location

Level of maintainance

Qualitative traits

1 2 3 4 MeanStandard deviation

138 000.00 $ 112 600.00 $ 110 000.00 $ 146 800.00 $ 126 850.00 $ 18 342.21 $

110 68 98 102 94.50 $ 18.36 $

0.25 0.70 0.70 1.05

1.35 0.36 0.12 0.24

0.60 1.59 1.06 1.06

2.20 2.65 1.88 2.35

1 254.55 $ 1 655.88 $ 1 122.45 $ 1 439.22 $ 1 368.02 $ 231.74 $

570.25 $ 624.59 $ 597.38 $ 612.73 $ 601.24 $ 23.47 $

Weigthed scores

Liveable area (in m2)

Level of maintainance

Landscaping

Location

2- Choose the most representative quantitive indicator of value (here liveable area) and compute the weighted scores

Total weighted score

Price per m2 of liveable area

Price per m2 per point

Transaction price (in $)

4- Using a constrained optimisation process find the weighting that minimize the error from observed prices

Spread with observed prices 7 499.11 $ (4 210.90 $) 710.86 $ (2 752.98 $)

Squared spread 56236 724.57 $ 17731 662.49 $ 505 324.16 $ 7578 899.57 $ 9 058.29 $

The optimizing scores are

Landscaping 0.35

Location 0.12

Level of maintainance 0.53

1.000001

weighted score

100

1 0.35

3 0.36

2 1.06

6.00 1.77

601.24 $

106 549.13 $

Minimum 102 389.52 $

Maximum 110 708.75 $

5- Applying the resulting weighting to the subject property

Totals

Average per m2 per point

Estimated value

Range

Level of maintainance

Landscaping

Location

Liveable area (in m2)

Multiple-criteria approach to grid adjustments

The quality grid presented above illustrates some of the problems that can advantageously be addressed with the help of MCDA.

Two techniques could be used to deal with the issues of criteria selection and adjustments:

1. The Simple Multi-Attribute Rating Technique (SMART). The performance of the alternatives under the respective criteria, evaluated via a direct-rating procedure, is expressed in grades on a numerical scale.

2. The Analytic Hierarchy Process (AHP) where alternatives are considered in pairs. Their relative performance can be expressed as a ratio of subjective values (Multiplicative AHP) or as a difference of grades.

This last approach is used here because it mimics almost perfectly the valuers’ traditional approach and - even more importantly - because it has been packaged in an appropriate software package. (Expert Choice). The use of a software package is essential because of the fairly complex optimisation procedures used in the AHP approach.

A multiple criteria decision program: Expert Choice

Expert Choice is a commercially available software design to treat multiple objective decisions. The algorithm is based Saaty’s Analytical Hierarchy process (AHP) and is particularly well adapted to deal with multi-factor pair wise comparisons.

1. Expert choice offers two broad problem structuring approaches:

- A deductive approach (top-down) that relies on the a priori selection and weighting of the decision criteria that are used to rank alternatives. This approach will be illustrated with the same 3 criteria and 4 alternatives (comparables) examples used above.

- An inductive approach (bottom-up) where the criteria and their ranking emerge from the treatment of the alternatives. However, since in a typical valuation exercise, all the comparables are compared with respect with each and every trait, this approach may not be appropriate here.

In the deductive approach, the objectives are chosen and ranked initialy. Using the previous very basic example we can now frame the problem in its most intuitive format.

The optimisation goal is defined as: `Finding the best twin’ and the criteria (also called objectives in Expert Choice) are again, defined as: Location, Landscaping, Maintenance.

Then, 4 ‘alternative’ choices are chosen (the 4 comparable properties). The Subject property is also treated as an alternative but made dormant in the computation.

The Expert Choice initial results will look like:

In the deductive approach (defined as top-down) in the Expert Choice jargon, we now assign a ranking of importance between our objectives. This choice is crucial since it will indirectly affect the full optimisation procedure. Here, I simply use the optimal weighting obtained from our previous example. We will see later what would be the effects of changing this initial allocation.

Thus, entering directly the a priori importance ranking we also produce a weighting histogram:

The next step requires a pair-wise comparison of each alternative (comparables) with respect to each criterion. This exercise can use quantitative, verbal or graphical assessment of preferences between the comparables.

Here the verbal assessment is used and the question is – for example – with respect to location, is comparable 1 ‘better’ or ‘worse’ than comparable 2. Better or worse be interpreted here as ‘more comparable to the subject’ or less comparable to the subject.

The same question is then answered for comp 2 and comp 3, etc.

Then, again, the same pair-wise preference assessment will be applied to the criteria of maintenance and landscaping.

One of the difficulties here is to maintain consistency (transitivity) between the responses. This can be facilitated by the ‘consistency’ checks provided by Expert Choice.

A loading for the objective ‘location’ is summarize and adjusted to reduce the level of inconsistencies:

The same exercise is repeated three times and the results are now presented both in loading and some graphical reinforcements.

The loading details are presented below:

The loadings represent an explicit ranking of the comparables with respect with their subject for each of the traits. In a weighted pair-wise treatment these coefficient could thus be used in order to weight non-quantitative judgement in the same manner that was illustrated in the Colwell type of treatment.

This may represent an important advantage to this existing seat-of-the pant ranking and, mostly it could be used to reduce the almost unavoidable inconsistencies resulting from more subjective approaches.

However, the reassurance of using quantified coefficient, should not occultate two fundamental issues: 1 – The loadings are obtained by the application of more or less subjective pair wise assessments and 2 and more importantly, the overall results are dependent on the initial ranking of importance between the criteria.

A rudimentary sensitivity analysis will illustrate how the results will be modified by shifting our criteria loading of importance.

With our initial ranking (Maintenance: 0.53, Landscaping: 0.35, Location: 0.12), the marshalling of alternatives (comparable 4> comp 1 > comp 3 > comp 2) is graphed as below:

The left Y-axis indicates the initial criteria loadings and the right Y-axis indicates each comparable priorities for each criteria.

The overall ranking of comparables is illustrated in the graph’s right margin.

The analyst may decide that the initial ranking does not reflect her assessment for this particular market and reassign different criteria loadings such as:

Location 80%, Maintenance, 10% and Landscaping: 10%.

Then the ranking of comparables is modified (comparable 1 > comp 2 > comp 4 > comp 3) to reflect the greater importance of location.

This approach appears to be straightforward and intuitively very close to the way analysts may think when they treat simple adjustment grids including a mix of quantitative and qualitative criteria. The multiple criteria treatment illustrated here does facilitate the pair-wise judgments and may strengthen the analyst’s confidence in the coherence (the transitivity) of her results.

A more complex analysis is illustrated now, with a combined usage of standard weightings and MCDA derived loadings for a set of 13 criteria and 10 comparables.

At this level of complexity, the previous pair-wise treatment is replaced by a Expert Choice data grid treatment. With a large number of variables, the analyst should start with a data grid to have a better overview and then proceed to single pair-wise adjustment to refine or review the grid results.

Both the MCDA data grid results and a more traditional Excel based treatment will be illustrated on screen at the conference.

Conclusions:

1. MCDA is flexible enough to allow an almost perfect fit (in Dilmore’s type of optimisation procedure) by tweaking and cajoling your pairwise comparisons until you get the `right’ answer.

2. With a large number of variables and comparables the treatment can be quite time consuming (more or less 2 days of fidgeting in this case);

3. Very acceptable comparable results can be obtained with a standard Excel based treatment combined with the optimisation procedure (also in Excel) presented above.

4. Publicly available data-bases do not contain the type of qualitative information required for a useful MCDA analysis. This type of information would have to be collected individually and manually.

5. However, in Australia, fees for a standard residential valuation are less than 100 Euros, thus the treatment has to be simple and cost-efficient. A full fledged MCDA treatment is not simple nor cost efficient, and even the more advanced Excel based optimisation procedure may be too complex for practical daily usage. This could explain why valuers have not been significantly influenced by the academic tinkering with the tradiditonal grid adjustment parctice. They are very unlikely to be impressed by MCDA options.

MCDA appears to be a convincing tool to deal with complex pair-wise treatments such a market grid analysis. However, it is unlikely to become the practicising valuers’ favourite gimmick.

MCDA has a much more interesting future for other property related analysis such as: choice between complex investment and development projects, leasing decisions, choice between various mortgage packages, selection of services providers, selection of listed property trusts, policy analysis, urban planning decisions, assessment of environmental impacts, etc.

This partial list is more than a list: it could be a life time research agenda.

References

Achour-Fischer, D. and P. Moffet (1982). Acquisitions institutionnelles par l'approche multi-criteres. Quebec City, Laval University.

Ball, J. and V. C. Srinivasan (1994). "Using the Analytic Hierarchy Process in House selection." Journal of Real Estate Finance and Economics 9: 69 - 85.

Bender, A., A. Din, et al. (1999). "Environmental quality perceptions of urban commercial real estate." Journal of Property Investment & Finance 17(3): 280 - 296.

Boronico, J. S. and D. M. Moliver (1997). "Appraisal reliability and the sales comparison approach." The Appraisal Journal(Oct): 331-336.

Chan, N. (2002). "Stigma assessment: a multi-criteria decision-making approach." Pacific Rim Property Research 8(1): 29-47.

Colwell, P. F., R. E. Cannaday, et al. (1983). "The analytical foundations of adjustment grid methods." AREUEA Journal 11(1): 11 - 29.

Crookham, J. (1995). "Sales comparison approach: Revisited." The Appraisal Journal(April): 177-181.

Diaz, J. I. (1990). "The process of selecting comparable sales." The Appraisal Journal(July): 533-540.

Dilmore, G. (1984). Technology and information processing and data basing: implications for real estate valuation methodology. Valuation Colloquium. K. W.N. Boston, Oelgeschlager, Gunn and Hain.

French, S. (1988). Decision Theory, an Introduction to the Mathematics of Rationality. Chichester, UK, Ellis Horwood.

Gau, G. W., I. Lai, et al. (1992). "Optimal Comparable Selection and Weighting in Real Property Valuation: An Extension." Journal of the American Real Estate and Urban Economics Association 20(1): 107-123.

Green, R. K. (1994). "Optimal Comparable Selection and Weighting: A Comment." Journal of the American Real Estate and Urban Economics Association 22(4): 647-654.

Ho, D. (1999). "Preferences on Office Quality Attributes: A Sydney CBI Study." Australian Land Economics Review 5(2): 36-42.

Isakson, H. R. (1986). "The nearest neighbors appraisal technique: an alternative to the adjustment grid methods." Journal of the American Real Estate and Urban Economics Association 14(2): 274-286.

Isakson, H. R. (1988). "Valuation analysis of commercial real estate using the nearest neighbors appraisal technique." Growth and Change 19(2): 11-24.

Keeney, R. and H. Raiffa (1976). Decisions with Multiple Objectives: Preferences a Value Trade Offs. New York, Wiley.

Kiel, K. and J. Zabel (1999). "The accuracy of owner-provided house values: the 1978-1991 American housing survey." Real Estate Economics 27.

Lentz, G. H. and K. Wang (1998). "Residential appraisal and the lending process: A survey of issues." Journal of Real Estate Research 15(11-40).

Manaster, M. S. (1991). "Sales comparison approach: A comparative analysis of three appraisal reports on the same property." The Real Estate Appraiser(May): 12- 26.

Northcraft, G. and M. Neale (1986). Experts, amateurs and real estate an anchoring-and-adjustment perspective on property pricing decisions. Arizona, Department of Management and Policy, University of Arizona.

Pace, R. K. (1998). "Total grid estimation." Journal of Real Estate Research 15(1/2): 101-114.

Roy, B. (1985). L'analyse multi-critere d'aide a la decision. Paris, Economica.

Roy, B. and D. Bouyssou (1993). Aide Multicritere a la Decision: Methodes et Cas. Paris, Economica.

Roy, B. and D. Vanderpooten (1996). "The European School of MCDA.. Emergence, Basic Features, and Current Works." Journal of Multi-Criteria Decision Analysis 5: 22-37.

Russo, L. E. and P. J. H. Schoemaker (1989). Decision Traps. New York, Simon & Schuster.

Saaty, T. L. (1980). The Analytic Hierarchy Process, Planning, Priority Setting, and Resource Allocation. New York, McGraw Hill.

Spence, M. T. and J. Thorson (1998). "The effect of expertise on the quality of appraisal services." The Journal of Real Estate Research 15(1/2): 205-215.

Stewart, T. J. (1992). "A Critical Survey on the Status of Multiple Criteria Decision Making Theory and Practice." Omega 20: 569-586.

Vandell, K. D. (1991). "Optimal comparable selection and weighting in real property valuation." Journal of The American Real Estate and Urban Economics Association 19(2): 213-239.

Vincke, P. (1989). L'Aide Multicritere a la Decision. Belgique, Editions de l'Universite Libre Bruxelles.

Winterfeldt, D. v. and W. Edwards (1986). Decision Analysis and Behavioral Research. Cambridge, UK, Cambridge University Press.