reputation systems - university of california, san...

1

Reputation Systems

Andrew A. ChienMay 21, 2004

UCSD CSE225

CSE225 Lecture #15

Administrivia

• Project Presentations, 6/11, 2-4pm, location TBD• Next week:

» 5/26 meet at regular time» 5/28 meet ½ hour early (430pm)

2

CSE225 Lecture #15

Last Time

• Grids are varied in structure and relationship» Internet Desktop Grid, Enterprise Grid» Compute Resource Oriented, Data Resource Oriented

• Virtual Organizations are a Security Policy Overlay» Notion of Virtual Organization is diverse» GSI can be used to build VO’s based on individual identity

– Challenges in administration: local/VO, management, lack of group identity

» Community Authorization Service supports notion of group– Anonymous use, Group management– Lesser Audit and no fine-grained control

CSE225 Lecture #15

Today’s Readings

• Resnick, Paul, Zeckhauser, Richard, Friedman, Eric, and Kuwabara, Ko. Reputation Systems. Communications of the ACM, 43(12), December 2000, pages 45-48.

• Resnick, Paul and Richard Zeckhauser. Trust Among Strangers in Internet Transactions: Empirical Analysis of eBay's Reputation System. The Economics of the Internet and E-Commerce. Michael R. Baye, editor. Volume 11 of Advances in Applied Microeconomics. Amsterdam, Elsevier Science. 2002.

3

Reputation Systems

Courtesy: Paul ResnickUniv. of Michigan

School of [email protected]

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

Learning Objectives

Understand– What a reputation system is– Theory about when and why it should work– Open research questions

4


Outline

What is a reputation system?Theory: when/why they should workEmpirical resultsDesign space


Definition

A Reputation System…– Collects– Distributes– Aggregates

…information about behavior

5


Examples

BBBBizrateeBayExpertise sites– Epinions “top reviewers”– Slashdot karma system


6



7


http://cgi2.ebay.com/aw-cgi/eBayISAPI.dll?ViewFeedback&userid=the_sharper_image


8



9


Why Reputation Systems

Interacting with strangersSellers (Exchange Partners) Vary– Skill– Effort– Ethics


Other Trust-Inducing Mechanisms in E-commerce

InsuranceEscrowFraud Prosecution

10


How Reputation Systems Should Work

Information– Past interactions inform abilities and

dispositionIncentive– Reciprocity or retaliation: future behavior– “Shadow of the future”

Self-selection– Low quality (ratings) -> less return – High quality -> greater return


Basic Requirements

Entities are long-lived– Anonymity– Name changes– Name trades

Feedback about interactions is captured and distributed– Non-participation (effort)– Reluctance to record negative– Honesty?

Past feedback guides buyer decisions

11

Interaction Type

ID changes

Anonymous every xaction

Pseudonyms

at will

Identified never

Anonymity Analysis

Interaction type

ID changes

Reputation Sharing

Trust/ cooperation


Pseudonyms

at will

1L Pseudonyms each arena

Identified never

12

Interaction type

ID changes

Reputation Sharing

Trust/ cooperation


none none

Pseudonyms

at will + only + only

1L Pseudonyms each arena

Identified never + and – + and 0


1L Pseudonyms

Third-party issues pseudonyms– No cost– Not replaceable– Reveal name to third party– Don’t reveal mapping of name to

pseudonym

13

Interaction type

ID changes

Reputation Sharing

Trust/ cooperation


none none

Pseudonyms

at will + only + only

1L Pseudonyms

each arena

+ and – within arena

+ and 0 within arena

Identified never + and – + and 0


Empirical Results: eBay

Feedback is providedIt’s almost all positiveReputations are informativeReputation benefits– Effect on probability of sale– Effect on price

14


Provision of Feedback

Negatives: paid but did not receive; seller cancelled; not as advertised; communication Neutrals: slow shipping, not as advertised, communication

Buyer of Seller Seller of Buyer Frequency Percent Frequency Percent

negative 111 0.3 353 1.0neutral 62 0.2 60 0.2positive 18,569 51.2 21,560 59.5none 17,491 48.3 14,260 39.4Total 36,233 36,233


Feedback Profiles of Buyers and Sellers

Group N (Sellers)

Percent neutral and negative

(Sellers)

N (Buyers)

Percent neutral and negative

(Buyers) 0-9 positive 4,018 2.83% 13,306 1.99%

10-49 positive 3,932 1.25% 7,366 1.09% 50-199 positive 3,728 0.95% 3,678 0.76%

200-999 positive 1,895 0.79% 738 0.60% 1000+ 122 1.18% 15 0.92%

All 13,695 0.93% 25,103 0.83%

15


Predicting Problematic Transactions

Logistic Regression

f(0,0) = 1.91% f(100,0) = .18% f(100,3) = .53%

N = 36233Beginning Block Number 0. Initial Log Likelihood Function-2 Log Likelihood 2194.3468-2 Log Likelihood 2075.420

Dependent Variable.. NEGNEUT---------------------- Variables in the Equation -----------------------

Variable B S.E. Wald df Sig R Exp(B)

LNNPOS .7712 .1179 42.7907 1 .0000 .1363 2.1624LNPOS -.5137 .0475 116.8293 1 .0000 -.2288 .5983Constant -3.9399 .1291 931.3828 1 .0000


Predictive Value

1-specificity (% of unproblematic transactions rejected)

Sensitivity (% of problematic transactions rejected)

Cutoff predicted probability

% of accepted transactions that are problematic

75% 94.2% .20% .11% 50% 81.5% .31% .18% 25% 57.2% .54% .27% 10% 32.4% 1.09% .36% 0% 0% Accept all .48%

Predicting Problem Transactions

1 - Specificity

1.00.75.50.250.00

Sens

itivi

ty

1.00

.75

.50

.25

0.00

16


Interesting Dynamics

High Courtesy Equilibrium: SymmetrySeller Driven


Interesting Dynamics

Skunk at the party, Bad applesPaying Initiation dues (buildup; type)– Lower prob of sales; lower prices

Stoning Bad Behavior– Piling on after see others had a problem– Not clear if this happens

17


Design Space

Rating scalesAggregation of ratingsWho rates?Incentives for ratersIdentification/Anonymity– Exchange partners– Evaluation providers


Summary

Reputation Ssystems inform, incent, selectOpportunity for RS: interactions with strangersDesign space– Scales, aggregation, raters, incentives,

anonymity

18

CSE225 Lecture #15

Discussion

• What do reputation systems have to do with Grids?» Open system for establishing/manage pairwise trust» Doesn’t require a trusted oracle in control» Defaults can tie policy to reputation (effectively)» Can you use reputations predict resource properties?

• What capabilities do RS provide? Can this transaction view be translated to control resource use/transactions?

» Perhaps yes. Can do selection based on reputation» Combine reputation and price =>as basis for allocation

• Can data access be controlled based on reputation?» Scoping and contexts» Not typically, but maybe in some contexts.» Reputation could be one attribute, but there are issues of policy, identity

and other characteristics• Mechanism for generating peer trust, but doesn’t address the whole

security problem

CSE225 Lecture #15

Discussion II

• Do they provide the same capabilities as GSI?» No, GSI is mechanisms to implement policy based on some

predecided trust

• Do these capabilities fit together?» Basis of trust» Mechanisms

reputation systems - university of california, san...

Documents