Mathematical Ranking (and Consensus Forming) Method

Hiroaki Ishii Graduate School of Information Science and

Technology Osaka University, Japan

Decision making based on Mathematical evaluation

Data envelopment analysis （ DEA) －－－ mathematical evaluation method for measuring the efficiency of decision-making units (DMU) on the basis of the observed data practiced in comparable DMUs, such as public departments (governments, universities, libraries, hospitals, etc), banking, etc.

DEA is originated by Charnes et al. and extended by Banker et al. The basic DEA models are known as CCR and BCC named after the authors’ initials.

input

Virtual Input=

1 2( , , , )j j mjx x x

1 1 2 2j j m mjv x v x v x

Output weight

Virtual output

1 2( , , , )j j sjy y y

1 1 2 2j j s sju y u y u y virtual output

efficiencyvirtual input

Weight , 1, 2,...,iv i m

, 1, 2,...,ru r s

11 12 1

21 22 2

1 2

n

n

m m mn

x x x

x x xX

x x x

11 12 1

21 22 2

1 2

n

n

s s sn

y y y

y y yY

y y y

1 1 2 20

, 1 1 2

1 1 2 2

1 1 2

1 2 1 2

max

1, 1,2,...,

, , , 0, , ,...., 0

r i

o o s so

u v o o m mo

j j s sj

j j m mj

m s

u y u y u yFP

v x vx v x

u y u y u ysubject to j n

v x vx v x

v v v u u u

0 1 1 2 2,

1 1 2 20

1 1 2 2 1 1 2 2

1 2 1 2

max

1

,

1,2,...,

, , , 0, , ,...., 0

r io o s so

u v

o m mo

j j s sj j j m mj

m s

LP u y u y u y

subject to v x v x v x

u y u y u y v x v x v x

j n

v v v u u u

DEFINITION (CCR EFFICIENCY)

1. is CCR-efficient if and there exist at least one

optimal with

DMUo * 1

( *, *)v u *, * 0v u

2. Otherwise, is CCR-inefficient.DMUo

PRODUCTION POSSIBILITY SET P

1. The observed activities belong to P( , ), 1, 2,...,j jx y j n

2. If an activity (x, y) belongs to P, then activity (tx, ty) belongs to P

for any positive scalar t.

3. For an activity (x, y) in P, any semi-positive activity withˆ ˆ( , )x yˆ ˆ,x x y y is included in P.

4. Any semi-positive linear combination of activities in P belongs to P.

For ranking of alternatives, one of the most familiar methods is to compare the weighted sum of their votes, after determining suitable weights of each alternative. Borda initially proposed the “Method of Marks” more than two hundred years ago so as to obtain an agreement among different opinions. His

method is surely a useful method evaluating consumers’ preferences of commodities in marketing, or in ranking social policies in political sciences, for instance. It is, however, difficult to determine suitable weight of each alternative a priori. In this context, Cook and Kress formulated the measure to automatically decide on the total rank order weight in order to hold the advantage using Data Envelopment Analysis model.

Later, Green et al. evolved the measure so as to make it possible to decide on the total rank order of all candidates.

And so, the weight of first, second, and third place are set to be5,3, and 1, respectively, where these are given a priori. The totalScore of i-th candidate, , is given as follows: . Here, denote the number of i-th place votes earned by candidate i.Table 1 Voting data, MVP of CL (Central League) in JPB

i321 135 iiii vvv

ijv

Candidates First Second Third (5) (3) (1)

Total score

1.Noguchi (Dragons) 2.Uehara (Giants) 3.Sekikawa (Dragons)

59 72 41 80 35 33 55 62 43

552 538 504

6 59 3 72 1 41 611´ + ´ + ´ =6 80 3 35 1 33 618´ + ´ + ´ =

In this example, if different weights are given to places, then the result of ranking becomes different. Then, an important issue is how to determine proper weights of first, second, and third place.

Since all candidates want to be ranked first place, they wish each weight to be assigned so as to maximize their own composite score. So, Green’s method by using LP which can determine the value of weights, is a very useful method. However, their method has undesirable points from the viewpoint of application.

Rank ordering method by Green et al Each candidate j=1,2,…, m obtained number yj1 of vote as first place, yj2 as second

place, yjk of k-th place the weight of k-th place (k=1,2,…,K). 　Assign the each weight so as to maximize the weighted sum to her/his vote ． 　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 (1)

1

k

jj j jw y

2jy

jiw

jiw

Total Ranking Method based on DEA

Among m objects, n persons select till k ranks according to their preference

1

1

max

1,

1,..., , 1,..., ,

k

ji jijji

k

jq ji qii

w y

w y

q m i k

s.t.

1 1 2

1 2 3

0, 0

12 3 ,

( 1)

ji ji j j jk

j j j jk jk

w w w w w

w w w kw wk m

+- ³ ³ ³ ³ ³

³ ³ ³ ³ ³+ + ´

L

LL

Caseⅰ 　 (Green et al.)

Caseⅱ 　 (Noguchi et al.)

Weight×captured votes

1DMU

2DMU

・・・

1DMU 2DMU mean・・・・・・・・・・・・

・・・ ・・・・・・ ・・・・・・・

1121 22

1221

1

1 2 , 1, 2,...,mq q q qm q m

Geometric mean

mDMU 1m 2m mm mf

1mf

2m

mDMU

Cross evaluation

A B C D E F

A 1.000 1.000 1.000 1.000 1.000 1.000

B 1.000 0.980 1.000 1.000 0.980 0.980

C 0.961 0.939 0.961 0.967 0.939 0.939

D 0.716 0.714 0.716 0.710 0.714 0.714

E 0.480 0.490 0.480 0.484 0.490 0.490

F 0.353 0.367 0.353 0.355 0.367 0.367

Geometric 　mean

1.000

0.990

0.951

0.714

0.486

0.360

rank

1

2

3

4

5

6

Example Six DMU (A ～ F)

1

1

1 2

max

1, 1,2,...,

2 0

ks sjj j j

ks sjq j q

j j jk

w y

Subject to w y q m

w w kw

In case of many categories, we solve the following linear programming problems

1

1 2( )s s s mos o o mo

1

1

max

1 ,

1,2,...,

S

oo

S

l j

w

w

j m

f q

q

=

=

=

£

=

å

å

l ll

ll

s.t.

Evaluation of DMU based on various data

m DMU : S categories Efficiency of DEA

1 21

1 2 3

0, 0

12 3 ,

( 1)

S

S S

w w w w w

w w w Sw wS m

- - ³ ³ ³ ³ ³

³ ³ ³ ³ ³+ + ´

l l L

LLⅱ)

ⅰ)

Multiple choice

(Changing weight order)

Different type

Candidates are chosen.

1DMU

2DMU

・・・

1DMU 2DMU mean・・・・・・・・・・・・

・・・ ・・・・・・ ・・・・・・・

1121 22

1221

1DMU

2DMU

mDMU

・・・

1 2 S1w 2w Sw

・・・・・・・・・・・・

・・・

・・・ ・・・・・・ ・・・・

1121 k2

k1

2mq1mq

12

mSq

22

weight

category

Efficiency of DEA

1

S

o l oll

wf q=

= å

mDMU

mDMU

1m 2m mf

1m2m

mm

AHP

maxTheconsistency index C.I.=

1

n

n

max (consistent)n

Example Application to Apparel Maker Selction Problem 　

1. Criteria Selection, candidate of maker,

Hierarchy construction

2. Using DEA, making scores of makers from characteristics

3 ． Based on AHP, subjective evaluation with respect to Priority

4. Total Preference of maker

Selection of maker

Hierarchy of Apparel Maker Selection Problem -1

Level １

Purpose

Level ２

Evaluation Points

Level ３

Objects

Maker D

Selection of apparel maker

Sewing Design

Maker A

Maker B

Maker C

Assortment

　ＡＨＰ　Applicable of AHP directly

　ＡＨＰ？　Not Applicable

of AHP

Selection of Suit Brand (DEA+AHP)

Hierachy-2

Level １

Purpose

Level ２

Evaluation Points

Level ３

Objects

Maker D

Selection of Apparel maker

Sewing Design

Maker A

Maker B

Maker C

Assortment

　ＡＨＰ　Applicable of AHP directly

Vote by staff (Piecewise

comparison may be replaced by

Voting

Vote,First rank, second rank 　 DEA

Voting data (Input DATA)

1st 2nd 1st 2nd 1st 2ndMaker A 8 9 7 7 1 0Maker B 6 7 7 8 2 2Maker C 5 3 3 3 4 8Maker D 1 1 3 2 13 10

Sewing Design Assortment

Max 8*wsa1+9* wsa2

Subject to 8*wsa1+9* wsa2 1,≦

6*wsa1+7* wsa2 1,≦

5*wsa1+3* wsa2 1,≦

1*wsa1+1* wsa2 1,≦

wsa1 2≧ * wsa2.,

wsa2 2/{20*2(2+1)}.≧

Linear sum for maker A maximize by two weughts wsa1 wsa2

1st 2ndMaker A 8 9Maker B 6 7Maker C 5 3Maker D 1 1

Sewing

（ obtained votes as the first rank)×weight for the first rank ） + （ obtained votes as the second rank) x weight for the 2nd ra

nk

1.000.760.520.12

Voting Data Analysis from Sewing point of view

maximize A B C Dw1 0.08 0.08 0.1063 0.1063 meanw2

0.04 0.04 0.017 0.017Maker A 1.0000 1.00001.0000 1.000 1.000Maker B 0.76 0.76 0.757 0.7

570.758

Maker C 0.52 0.52 0.583 0.583

0.551Maker D 0.12 0.12 0.123 0.123 0.1215

Geometric mean of Preference ratejjji

Sewing Design AssortmentMaker A 1.0000 0.9761 0.0597Maker B 0.758 1.0000 0.1641Maker C 0.551 0.4183 0.4160Maker D 0.1215 0.3941 1.0000

Piecewise Comparison

Sewing Design Assortment PrioritySewing 1 5 3 0.6495Design 1/5 1 3 0.2295

Assortment 1/3 1/3 1 0.121

A maker ：0.650*1.0000+0.223*0.9761+0.121*0.0597=0.87

49

B maker ：0.650*0.758+0.223*1.0000+0.121*0.1641=0.735

6

C maker ：0.650*0.551+0.223*0.4183+0.121*0.4160=0.501

8

D maker ：0.650*0.1215+0.223*0.3941+0.121*1.0000=0.28

69

Selection of A maker is the most preferable

jppjiij xxx 11

Conjoint analysis is a scaling method developed in mathematical psychology by American psychologist Luce and Turkey in 1964. A model of consumer’s preference formation in common use is the simple additive model. In this model, we think that each possible level of an attribute has a “part worth” to a level of an attribute, and the sum of the part worthies of its attributes is the “total worth” to a consumer of a product. Generally, conjoint analysis introduces a part worth value of each attribute of each product based upon some goodness fit criterion from preference rank ordinal data. The rank ordinal data in that case is a result of people’s selection by individuals.

Part worth value

6(1)

5(2)

4(3)

3(4)

2(5)

1(6)

0

1

0

1

0

1

1

0

1

0

1

0

0

0

0

0

1

1

1

1

0

0

0

0

0

0

1

1

0

0

A

B

C

D

E

F

Condominium

Detached

house

Totally

European

Semi-European

style

Totally

Japanese

Forms Styles Scores z

(ranking

y)

Factors

Real estate

11b 13b 21b 22b

Example of Conjoint Analysis

12b

　 P 　　 　 　　Q 　

A 0 1 0 1 0

B 0 1 0 0 1

C 1 0 0 1 0

D 1 0 0 0 1

E 0 0 1 1 0

F 0 0 1 0 1

1p 2p 3p 1q 2q Rank

1

2

3

4

5

6

Monotonic TransformationZ

6

5

4

3

2

1

Part worth11b 12b 13b 21b 22b

Factor Analysis by Conjoint analysis

0-1 design matrix to indicate each level of products

:,,

:ˆ

:ˆ

:,

:

:,,

21

,21

1

111

21

n

Tn

nmn

m

Tn

bbb

zzz

dd

dd

yyy

b

cz

Dbz

z

D

y Ordinal scale of consumer’s preference for products

Order preserving transformation of y

An additive conjoint model

Average vector of z

The part worth values to be estimated

Fitting criterion for conjoint analysis

Quadratic fractional programmingQuadratic fractional programming

NumeratorNumerator

denominatordenominator

Difference between estimation and actual Difference between estimation and actual datadata

Variation of actual dataVariation of actual data

2 ,S f b To be minimizedTo be minimized

2 ,S f b Optimal bOptimal b

minimize

maximize

)()(

)()(

)ˆˆ()ˆˆ(

)ˆ()ˆ(),(2

cDbcDb

zDbzDb

zzzz

zzzzb

T

T

T

T

fS

2( , )S f b

Parametric quadratic function with

• Theorem

)()()()()( cDbcDbzDbzDbF TT

cczzcDbzDbDbDb TTTTTTTT 22)1(

Let

If

then

*

* 2

min{ ( )

min{

}

( |

|

0

) }

a

S

t

F F b

b b

F

Further, the minimizer of

solution of

)( *F is also an optimal

)(2 bS

Optimality conditions

0)(

b

F )(F is convex , Since

)2(0)( 　 　 zDbzzzzDbF TTTT

b )(FLet b, be , substitute into

From (1), (2), obtain the optimal part worth value

)1()1( 　 　zDcDDbD TTT

　 P 　　 　 　　Q 　

A 0 1 0 1 0

B 0 1 0 0 1

C 1 0 0 1 0

D 1 0 0 0 1

E 0 0 1 1 0

F 0 0 1 0 1

Part Worth 0 2 -2 0.5 -0.5

1p 2p 3p 1q 2q

2.5

1.5

0.5

-0.5

-1.5

-2.5

422 Pb 15050 ..bQ

Method B

RESEARCH PURPOSE

Conjoint Conjoint AnalysisAnalysis 　

Many Multi-purpose Problem ActuallyMany Multi-purpose Problem Actually

Conjoint Analysis combined with Conjoint Analysis combined with DEADEA

More ApplicableMore Applicable

MarketingConsumer Preference

Single 　 Objective

Multi-purpose

DEA ・ Conjoint Analysis

DEA ･･･　

Conjoint Analysis ･･　

　　　　　　　 　　　　　　　 multiple total evaluationmultiple total evaluationWeighting Total OrderingWeighting Total Ordering

Voting Data

　　　　　　　 　　　　　　　 from total evaluationfrom total evaluationFactor—part worth valueFactor—part worth value

Ordinal Data

Conjoint Analysis combined with DEA

sample category Colour First Second

Th

ird

Pencil SharpPencil

Ball point pen

red blue

A 1 0 0 1 0 2 2 6

B 0 1 0 0 1 5 3 2

C 0 0 1 1 0 3 5 2

〈〈 ExampleExample 〉〉 Voting Data Voting Data

MultipleMultiple evaluationevaluationTotal evaluationTotal evaluation

DEA

Conjoint AnalysisConjoint Analysis

Part worth value

sample order

A 3

B 1

C 2

DEA model

constraintconstraint

Objective functionObjective function 　　

1

1k

iq ij qjj

y w v

1,2, ,q m

1 2 32 3 0i i i ikw w w kw ;ijw 　

;ijv 　

Choose k preferences among m alternatives with rankingChoose k preferences among m alternatives with ranking

Weight of each rankWeight of each rank

Captured votesCaptured votes

Preferable Weighting For each alternative

1

k

ii ij ijj

y w v Maximize

Ranking by Cross-valuation

Application

DEA －Ｃｏｎｊｏｉｎｔ Analysis

Ｅｖａｌｕａｔｉｏｎ Method ｆｏｒ Voting Data

Ａｐｐｌｉｃａｔｉｏｎ to development plan ｏｆ ｎｅｗ medicine 　

　　 From activation values of 40 samples, From activation values of 40 samples, we find promising we find promising

Compounds from various aspect.Compounds from various aspect.

(single objective case )Mother compound

• Artificial example

NHCNHSO2

O N

N

CH

1R 2R

3R

The combination of substituents expected as the new medicines

5.410

6.49

4.98

5.17

5.16

5.95

6.34

6.63

7.12

7.61

Sample 1R 2R 3R

COOEt

COOEt

Cl

3CH

3COOCH

COOEt

2NO

3CH

Cl

COOEt

Cl

3OCH

3OCH

3OCH

3OCH

3CH

3CH

3CH

3CH

3OCH

3OCH

3OCH

H

H

H

H

H

H

H

H

Activationvalue

Application to the design matrix

5.4100100100010

6.410100100009

4.910100010008

5.110100000017

5.110100000106

5.910010100005

6.310010000014

6.610010001003

7.101010000102

7.601001000011

Activation value

Z3COOCH 2NO3CH Cl Cl 3OCH 3CH 3OCH HCOOEt

b 11b 12b 13b 14b 15b 21b 22b 23b 31b 32b

1R 2R 3R

Calculation Result

COOEt3CH 3COOCH Cl 2NO Cl 3OCH 3CH 3OCH H

44.5 00.5 22.6 83.4 94.5 55.0 44.0 0 77.1 0b

1R 2R 3R

Estimate a new medicine with combination of substituents of high part worth value

　 :　 :　 :

3COOCHCl

3OCH3

2

1

R

R

R Activation value

(estimation)8.54

Theme

Extending Distance Measure to Construct Joint Ballot Model

　　　 Single Ballot

Choosing “special and

the best” objects

　　　 Joint Ballot

Choosing “the most

favorable pairs”

Theme

constructing a model to reflect voter’s favorable pairs when they place objects in the order

Distance FunctionMinimizing “distance” which indicates degree of disagreement of voters

Cook&Kress’s Relative Distance

Position j Forward Indicator Vector

“degree of differences”

are expressed

Relative Distance

Position j Backward Indicator Vector

Consensus Formation

RelativeDistance

1 2 3 4Cocktail

0 1 0 0

Beer 0 0 1 0焼酎 1 0 0 0Wine 0 0 0 1

Definition

Relative Distance shows the Degree of Differences

Relative Distance Function

Ranking of Voter A

Joint Ballot ~ Basic Idea ~

Model to reflect “priority of viewpoints” and “patterns of objects”

1.Signifying viewpoints

2.Comparing objects relatively in each viewpoint

BrandDesign

Price

Functionality

2 steps

Joint Ballot

sital

~ Joint Ballot Model ~

1.Joint Ballot Model

Adding Weights wsj of rankings and viewpoints and Viewpoint to relative distance measure

2.Weights

(i) Viewpoints ws ： applying AHP

(ii) Rank wj

(iii) Integrating these 2 weights

Every object is not placed in the same standing

Every object is placed only in a standing

ssj jw ww=

Joint Ballot~ Procedure to Rank Ordering ~

Voters’ task

Specifying Viewpoints

Integrating Weights of Ranks and Viewpoints

Overall Ranking

pairwise Comparison of Importance between Viewpoints

Rank Ordering Objects in Each Viewpoint

Joint Ballot to

Choose n Objects:

Top n of Overall Ranking

Validation

~ Questionnaire ~

1.Situation

Chain restaurant “X” sticks to catering “fresh”, “tasty” dinning and “cozy” space at affordable price. “X” will target young people and open 3 new shops around “Y” University. It sends out a questionnaire to search the most effective combination of stores.

2.Objects

Bar ・ Chinese Restaurant ・ European Restaurant ・ Japanese Restaurant ・Burger Shop ・ Bakery ・ Café ・ Cake Shop ・ Taiwanese Traditional Tea Shop3.Viewponts

Occasion 　・　 Menu Item 　・ Number of People 　・ Freshness

4.Joint Ballot

In advance usual Joint ballot is given to compare our method. 3 stores are chosen.

Validation ~ Coincidence with Joint Ballot ~

Average of Data Spread ：4.2

Standard Deviation:3.17

Proposed Method Coincides with Joint Ballot to Some Degree

Cause

　 Similarity Effect

　 Difficulty to Weigh Viewpoints

Bar

Bakery

Cafe

Tea

European R

R27Burger

R4

Japanese R Chinese

R

Cake Shop

CausalSpecial

Combination of Restaurant and Light Meal

Combination of Light Meals

Light Meal

Full Meal

Validation ~ Ogawa and Ishii’s Count ~

Ogawa and Ishii’ Count

1. Priority as of viewpoints with AHP

2. Preference Rate with DEA

AHP Applied to Cook and Kress’s Count

3. Overall Ranking

Overall score of an object is product-sum of priority as and preference rate Zi.

Preference Rate Zi

Preference Rate Ziq to which Object q (q=1,2,…,m) is Applied

1

1

1 2 3

max

. . 1 ( 1,2,... )

2 3

1 2

(1 2 ) ( 1)

k

ii ij ijj

k

iq ij qjj

i i i ik

ik

Z w v

s t Z w v q m

w w w kw

wk n nk k

Validation ~ Comparison to Ogawa and Ishii’s Count ~

Meaning of Winner

Coincidence with Rank of Accumulated Ballots

　　Accumulated Ballots

OccasionMenu Item

PeopleFreshness

Ogawa

9 th 0.0167 0.7000 1.0000 0.9667

3 rd 0.9833 0.9667 0.9000 0.9500

1 st 0.9667 0.9333 0.9833 0.8667

Our Metho

d

9 th 0.1083 0.9333 0.8667 0.9000

3 rd 0.7250 0.7000 0.9083 0.9833

1 st 0.7250 0.5667 0.8417 0.9667

In Ogawa and Ishii’s count, ranking is likely to depend on number of ballots in high rank.

Ranking to Advantage of the Strong

Average of n th

rank

Bar

Chinese R

European R

Japanese R

Burger Shop

Bakery

Café

Cake Shop

Tea

Accu

mu

late

dB

allo

ts

Validation

Meaning of Winner Comparison to ranking of Neutral Point

Neutral Ranking which Covers Both of high and low Opinion

Approximately Consistent with Proposed Count

~ Comparison to Ogawa and Ishii’s Count ~

in Neutral

Agreement Disagreement

Bar

Chinese R

RankA

ccu

mu

late

d

Ballo

ts

Validation

　 Ogawa

Ours

Bar 2 9

Chinese R 4 3

European R 1 1

Japanese R 3 2

Burger Shop 7 4

Bakery 6 5

Café 5 6

Cake Shop 8 7

Tea Shop 9 8

Ranking

Quite Different in ranks of Bar and Burger Shop

~ Comparison to Ogawa and Ishii’s Count ~

Overall Ranking between Different Groups of Preferential Manner

BarBakery

Cafe

Tea

European R

R27 Burger

R4

Japanese R Chines

e R

Cake Shop

Group who Prefers Light Meal and Casual Restaurant

Group who Prefers Exclusive Restaurant

Validation

　 　 １ st

２nd

３rd

４ th

５ th

６ th

７ th

８ th

９ th

Occasion Bar 2 0 0 1 1 0 0 1 5

Occasion Burger Shop 0 2 2 1 1 3 0 0 1

Menu Item Bar 0 2 0 0 1 0 0 0 7

Menu Item Burger Shop 0 1 0 3 0 2 0 3 1

People Bar 5 1 0 0 2 1 0 1 0

People Burger Shop 0 1 2 6 0 0 0 1 0

Freshness Bar 4 0 1 1 0 0 0 0 4

Freshness Burger Shop 4 0 1 1 0 0 0 0 4

Burger ： Many Ballots at 3rd and 7th

Low Rank in Ogawa’s Count

Middle Rank in Proposed Count

Bar ： Many Ballots at 1st and 9th 　

Ballots at 1st 　　　　　 Ogawa’s Count

Ballots at 9th 　　　　　 Proposed Count

~ Comparison to Ogawa and Ishii’s Count ~

Overall Ranking between Different Groups of Preferential Manner

Validation

Ranking

Overall Ranking between Different Groups of Preferential Manner

　 Ogawa Ours

Bar 2 9

Chinese R 4 3

European R 1 1

Japanese R 3 2

Burger Shop 7 4

Bakery 6 5

Café 5 6

Cake Shop 8 7

Tea Shop 9 8

BarBakery

Cafe

Tea

European R

R27 Burger

R4

Japanese R Chines

e R

Cake Shop

~ Comparison to Ogawa and Ishii’s Count ~

Group who Prefers Light Meal and Casual Restaurant

Proposed Count Covers 2 Groups’ Preferential Zone

Approximate Ranking Covering Voter’s Favorable Combination

Group who Prefers Exclusive Restaurant

Summary

Proposed Method Coincides with Joint Ballot to Some Degree

Conclusion

Reasonable Approach with Basic Idea and Distance Function

Proposed Count Ishii and Ogawa’s Count

Neutral Ranking which covers Both of high and low Opinion

Ranking to Advantage of the Strong

Consensus Formation

Problem

Difficulty to Weight Viewpoints Pairwise Comparison with Cook&Kress’s Object Based Distance

Ranking which Takes Importance to Majority