aiguidelines · 2020. 2. 8. · opportunity analysis engagements with practitioners findings &...

https://aka.ms/aiguidelines

INITIALLY

DURING

INTERACTION

WHEN WRONG

OVER TIME

Make clear

how well the

system can

do what it

can do.

Make clear

what the

system

can do.

Time services

based on

context.

contextually

relevant

information.

relevant

social norms.

Mitigate

social biases.

Support

efficient

invocation.

Support

efficient

dismissal.

Support

efficient

correction.

services when

in doubt.

Make clear

why the

system did

what it did.

Remember

recent

interactions.

Learn from

user behavior.

Update and

cautiously.

Encourage

granular

feedback.

Provide

global

controls.

Notify users

changes.

Convey the

consequences

of user

actions.

Agenda

AI is fundamentally changing how we interact with

computing systems

MS Word

MS OneNote

MS PowerPoint

Disclaimers

INITIALLY

DURING

INTERACTION

WHEN WRONG

OVER TIME

Make clear

how well the

system can

do what it

can do.

Make clear

what the

system

can do.

Time services

based on

context.

contextually

relevant

information.

relevant

social norms.

Mitigate

social biases.

Support

efficient

invocation.

Support

efficient

dismissal.

Support

efficient

correction.

services when

in doubt.

Make clear

why the

system did

what it did.

Remember

recent

interactions.

Learn from

user behavior.

Update and

cautiously.

Encourage

granular

feedback.

Provide

global

controls.

Notify users

changes.

Convey the

consequences

of user

actions.

INITIALLY

DURING

INTERACTION

WHEN WRONG

OVER TIME

Make clear

how well the

system can

do what it

can do.

Make clear

what the

system

can do.

Time services

based on

context.

contextually

relevant

information.

relevant

social norms.

Mitigate

social biases.

Support

efficient

invocation.

Support

efficient

dismissal.

Support

efficient

correction.

services when

in doubt.

Make clear

why the

system did

what it did.

Remember

recent

interactions.

Learn from

user behavior.

Update and

cautiously.

Encourage

granular

feedback.

Provide

global

controls.

Notify users

changes.

Convey the

consequences

of user

actions.

Make clear

how well the

system can

do what it

can do.

Make clear

what the

system

can do.

*Dan Russell. The Joy of Search.

INITIALLY

DURING

INTERACTION

WHEN WRONG

OVER TIME

Make clear

how well the

system can

do what it

can do.

Make clear

what the

system

can do.

Time services

based on

context.

contextually

relevant

information.

relevant

social norms.

Mitigate

social biases.

Support

efficient

invocation.

Support

efficient

dismissal.

Support

efficient

correction.

services when

in doubt.

Make clear

why the

system did

what it did.

Remember

recent

interactions.

Learn from

user behavior.

Update and

cautiously.

Encourage

granular

feedback.

Provide

global

controls.

Notify users

changes.

Convey the

consequences

of user

actions.

services

based on

context.

contextually

relevant

information.

relevant

social

norms.

Mitigate

social

biases.

Remember to

call your mom

services

based on

context.

contextually

relevant

information.

relevant

social

norms.

Mitigate

social

biases.

services

based on

context.

contextually

relevant

information.

relevant

social

norms.

Mitigate

social

biases.

services

based on

context.

contextually

relevant

information.

relevant

social

norms.

Mitigate

social

biases.

6“Information is not subject to biases, unless

users are biased against fastest routes”

“There’s no way to set an avg walking speed.

[The product] assumes users to be healthy”

INITIALLY

DURING

INTERACTION

WHEN WRONG

OVER TIME

Make clear

how well the

system can

do what it

can do.

Make clear

what the

system

can do.

Time services

based on

context.

contextually

relevant

information.

relevant

social norms.

Mitigate

social biases.

Support

efficient

invocation.

Support

efficient

dismissal.

Support

efficient

correction.

services when

in doubt.

Make clear

why the

system did

what it did.

Remember

recent

interactions.

Learn from

user behavior.

Update and

cautiously.

Encourage

granular

feedback.

Provide

global

controls.

Notify users

changes.

Convey the

consequences

of user

actions.

Support

efficient

invocation.

Support

efficient

dismissal.

Support

efficient

correction.

services

when in

doubt.

Make clear

why the

system did

what it did.

Support

efficient

invocation.

Support

efficient

dismissal.

Support

efficient

correction.

services

when in

doubt.

Make clear

why the

system did

what it did.

Support

efficient

invocation.

Support

efficient

dismissal.

Support

efficient

correction.

services

when in

doubt.

Make clear

why the

system did

what it did.

INITIALLY

DURING

INTERACTION

WHEN WRONG

OVER TIME

Make clear

how well the

system can

do what it

can do.

Make clear

what the

system

can do.

Time services

based on

context.

contextually

relevant

information.

relevant

social norms.

Mitigate

social biases.

Support

efficient

invocation.

Support

efficient

dismissal.

Support

efficient

correction.

services when

in doubt.

Make clear

why the

system did

what it did.

Remember

recent

interactions.

Learn from

user behavior.

Update and

cautiously.

Encourage

granular

feedback.

Provide

global

controls.

Notify users

changes.

Convey the

consequences

of user

actions.

Cupcakesare

muffinsthat believed

in miracles

Remember

recent

interactions.

Learn from

user behavior.

Update and

cautiously.

Encourage

granular

feedback.

Provide global

controls.

Notify users

about changes.

Convey the

consequences of

actions.

Agenda

Findings &

Impact

Opportunity Analysis

Engagements with Practitioners

Findings &

Impact

Academia Practitioners Industry

CHI 2019 Best Paper Honorable Mention Being leveraged by product teams

across the company throughout the

design and development process

Cited and used in related organizations

Translated to other languages

Initial Impact

Findings &

Impact

Developing the Guidelines for Human-AI Interaction

Confirms that the guidelines

are applicable to a broad range

of products, features and

scenarios.

Applies Violates

Violations

Applications

Remember

recent

interactions.

Violations

Applications

Remember

recent

interactions.

Violations

Applications

Make clear

what the

system

can do.

Remember

recent

interactions.

Violations

Applications

Make clear

what the

system

can do.

contextually

relevant

information.

Provide

global

controls.

Violations

Applications

Provide

global

controls.

Violations

Applications

Make clear

why the

system did

what it did.

Provide

global

controls.

Violations

Applications

Make clear

why the

system did

what it did.

11Make clear

how well the

system can

do what it

can do.

Types of content: examples, patterns,

research, code

Tagged by guideline and scenario

with faceted search and filtering

Comments and ratings to support

learning

Grow with examples and case studies

submitted by practitioners

Initial Impact

Opportunity Analysis

Findings &

Impact

Workshops and Courses

Q & A Break

Agenda

How can I

implement the

HAI Guidelines? Interaction Design

AI System Engineering

Interaction Design

for AI requires

ML & Eng Support

Time services

based on

context.

Hard to implement if the logging infrastructure is

oblivious to context.

services when

in doubt.

Does the ML algorithm know or state that

it is “in doubt”?

Make clear

why the

system did

what it did.

Is the ML algorithm explainable?

Make clear

how well the

system can

do what it

can do.

Make clear

what the

system

can do.

Make clear

how well the

system can

do what it

can do.

Make clear

what the

system

can do.

Make clear

how well the

system can

do what it

can do.

Make clear

what the

system

can do.

[Buolamwini, J. & Gebru, T. 2018]

90% accuracy

Make clear

how well the

system can

do what it

can do.

Make clear

what the

system

can do.

Benchmark

Internal component features

Content features

countOBJECTS

people

Descriptive features Failure explanation models

with Pandora

[Nushi et. al. HCOMP 2018]

All (5.5%)

Women (11.5%)

All (5.5%)

No makeup (23.7%)

Women (11.5%)

All (5.5%)

Short hair (30.8%)

Women (11.5%)

No makeup (23.7%)

Not smiling (35.7%)

Women (11.5%)

No makeup (23.7%)

Short hair (30.8%)

All (5.5%)

Make clear

how well the

system can

do what it

can do.

Make clear

what the

system

can do.

[Nushi et. al. HCOMP 2018]

[Wu et. al. ACL 2019]

[Zhang et. al. IEEE TVCG 2018]

Make clear

how well the

system can

do what it

can do.

Make clear

what the

system

can do.

https://scikit-learn.org/stable/auto_examples/

calibration/plot_calibration_curve.html

[Platt et al., 1999; Zadrozny & Elkan, 2001]

[Gal & Ghahramani, 2016; Osband et al., 2016]

https://forecast.weather.gov/

Probably a yellow school bus driving down a street

Time services

based on

context.

Support

efficient

invocation.

Support

efficient

dismissal.

AI System

Automated

triggering

Explicit

invocation

Time services

based on

context.

Support

efficient

invocation.

Support

efficient

dismissal.

AI System

Automated

triggering

Explicit

invocation

Time services

based on

context.

Support

efficient

invocation.

Support

efficient

dismissal.

Recall

Learn from

user behavior.

Encourage

granular

feedback.

Update and

cautiously.

Learn from

user behavior.

Encourage

granular

feedback.

Update and

cautiously.

Learn from

user behavior.

Encourage

granular

feedback.

Update and

cautiously.

Encourage

granular

feedback.

Provide

global

controls.

Encourage

granular

feedback.

Provide

global

controls.

Encourage

granular

feedback.

Provide

global

controls.

Agenda

Learn from

user behavior.

Remember

recent

interactions.

Make clear

why the

system did

what it did.

Intelligible, Transparent, Explainable AI

Predictable ~ (Human) Simulate-able

Intelligible ~ Transparent

Explainable ~ Interpretable

Inscrutable ⊇ Blackbox

⊇⊇

Predict exactly what it will do

Answer counterfactual

predict how a change to model’s inputs

will change its output

Construct rationalization for why

(maybe) it did what it did

Inscrutable: too complex to understand

Blackbox: know nothing about it

Caveat: My take – No consensus here

Reasons for Wanting Intelligibility

1. The AI May be Optimizing the Wrong Thing

2. Missing a Crucial Feature

3. Distributional Drift

4. Facilitating User Control in Mixed Human/AI Teams

5. User Acceptance

6. Learning for Human Insight

7. Legal Requirements

[Weld & Bansal CACM 2019]

interactions, or computed in expectation from the system it-self.

In this paper, we argue that a classifier that is intended towork with a human should maximize the number of accurateexamples beyond the handover threshold by focusing the op-timization effort on improving theaccuracy of such examplesand putting less emphasis on examples within the threshold,since such examples would be inspected and overriden by thehuman anyways. We achieve this goal by proposing a newoptimization loss function named as negative log-utility loss(NELU) that optimizes for team utility rather than AI accu-racy alone. Current classifiers trained via log-loss for opti-mizing team accuracy do not take into account this aspect ofAI-advised decision making, and therefore only aim at maxi-mizing the number of instances on either side of the decisionboundary.

Take for instance, the illustrative example shown in Fig-ure 1. The left side shows a standard classifier (h1) trainedwith negative log-loss, while the right side shows a classifiertrained with negative log-utility loss (h2). The shaded grayarea shows the handover region for which it is rational forthe human to inspect the prediction and if inaccurate, over-ride thedecision. It is clear that although h1 ismore accurate(i.e., more accurate instances on either side of the decisionboundary), if it were used in an AI-advised decision makingsetting, its high accuracy in the handover region would notbe useful to the team because the human would not trust theclassifier on those instances and spend more time in solvingthe problem herself, which would then translate to subopti-mal team utility. For the case of h2 instead, even though theoverall accuracy is lower (B is on the wrong side of the de-cision boundary), if combined with a human it could achievea higher team utility because the human can safely trust themachine outside of the handover region and at the same timethere exist less data points to inspect within the handover re-gion (notice that the set of data points A has now moved out-side of this region).

While there exist other aspects of collaboration that canalso beaddressed via optimization techniques, such as modelinterpretability, supporting complementary skills, or enablinglearning among partners, the problem we address in this pa-per to account for team-based utility functions is a first steptowards human-centered optimization. 1

In sum, we make the following contributions:

1. We highlight a novel, important problem in the field ofhuman-centered artificial intelligence: the most accu-rate ML model may not lead to the highest team utilitywhen paired with ahuman overseer.

2. Weshow that log-loss, themost popular loss function, isinsufficient (as it ignores teamutility) and develop anewloss function to overcome its issues.

3. Show that on multiple real data sets and machine learn-ing models our new loss achieves higher utility than logloss.

1BUG: This is an attempt to say that we are just making a firststep here to set expectations right.

Figure 2: (a) A schematic of AI-advised decision making. (b) Tomake a decision, the human decision maker either accepts or over-rides a recommendation. The Over r i de meta-decision is costlierthan Accept .

4. Provide a thorough explanation on when and why it isbest to use the negative log-utility loss by showing ex-perimental results with varying domain related cost pa-rameters.

2 3 4 5

2 Problem Descr iption

We focus on a special case of AI-advised decision mak-ing where a classifier h gives recommendations to a hu-man decision maker d to help make decisions (Figure 2a).If h(x) denotes the classifier’s output, a probability distri-bution over Y , the recommendation r h (x) consists of a la-bel argmax h(x) and a confidence value max h(x), i.e.,r h (x) := (argmax h(x), max h(x)). Using this recommen-dation, the user computes a final decision d(x, r h (x)). Theenvironment, in response, returns a utility which depends onthe quality of the final decision and any cost incurred due tohuman effort. Let U denote the utility function. If the teamclassifiesasequenceof instances, theobjectiveof this team isto maximize the cumulative utility. Before deriving a closed

2BUG: Change. Weanalyze... with different costs....3BUG: - show connection with real world4BUG: say wemakeassumptions about collaboration, but its im-

portant5BUG: Work weneed to citehere to show that collaboration may

not work: [Zhang et al., 2020; Lai and Tan, 2018; Feng and Boyd-Graber, 2019].

2 3 4 5

Deploy

Autonomous AI

Intelligibility Useful in Both Cases

2 3 4 5

Human-AI Team

5. User Acceptance

Panda!

Firetruck?!?

AI Errors

Human Errors

But Humans Err as Well

Human Errors

AI Errors

AI-Specific

ErrorsJoint

Errors

Preventable Errors

Human Errors

AI Errors

AI-Specific

ErrorsJoint

Errors

Preventable Errors

Intelligible AI → Better Teamwork

A Simple Human-AI Team

Beyond Accuracy: TheRole of Mental Models in Human-AI Team Performance

Gagan Bansal1 Besmira Nushi2 EceKamar 2 Walter S. Lasecki3

Daniel S. Weld1,4 Eric Horvitz2

1University of Washington 2Microsoft Research 3University of Michigan 4Allen Institute for Artificial Intelligence

Abstract

Decisions made by human-AI teams (e.g., AI-advised hu-mans) are increasingly common in high-stakes domains suchas healthcare, criminal justice, and finance. Achieving highteam performance depends on more than just the accuracy ofthe AI system: Since the human and the AI may have differ-ent expertise, the highest team performance is often reachedwhen they both know how and when to complement one an-other. We focus on a factor that is crucial to supporting suchcomplementary: thehuman’smental model of theAI capabil-ities, specifically the AI system’s error boundary (i.e. know-ing “When does the AI err?”). Awareness of this lets the hu-man decidewhen to accept or override theAI’srecommenda-tion. Wehighlight two key properties of anAI’serror bound-ary, parsimony and stochasticity, and a property of the task,dimensionality. We show experimentally how these proper-ties affect humans’ mental models of AI capabilities and theresulting team performance. We connect our evaluations torelated work and propose goals, beyond accuracy, that meritconsideration during model selection and optimization to im-proveoverall human-AI team performance.

1 IntroductionWhile many AI applications address automation, numer-ous others aim to team with people to improve joint per-formance or accomplish tasks that neither the AI nor peo-ple can solve alone (Gillies et al. 2016; Kamar 2016;Chakraborti and Kambhampati 2018; Lundberg et al. 2018;Lundgard et al. 2018). In fact, many real-world, high-stakesapplications deploy AI inferences to help human expertsmake better decisions, e.g., with respect to medical diag-noses, recidivism prediction, and credit assessment. Rec-ommendations from AI systems—even if imperfect—canresult in human-AI teams that perform better than eitherthe human or the AI system alone (Wang et al. 2016;Jaderberg et al. 2019).

Successfully creating human-AI teamsputsadditional de-mands on AI capabilities beyond task-level accuracy (Grosz1996). Specifically, even though team performance dependson individual human and AI performance, the team perfor-mance will not exceed individual performance levels if hu-

Figure 1: AI-advised human decision making for read-mission prediction: The doctor makes final decisions us-ing the classifier’s recommendations. Check marks denotecases where the AI system renders a correct prediction, andcrosses denote instances where the AI inference is erro-neous. Thesolid line represents theAI error boundary, whilethedashed lineshowsapotential human mental model of theerror boundary.

man and the AI cannot account for each other’s strengthsand weaknesses. In fact, recent research shows that AI ac-curacy does not always translate to end-to-end team perfor-mance(Yin, Vaughan, and Wallach 2019; Lai and Tan 2018).Despite this, and even for applications where people are in-volved, most AI techniques continue to optimize solely foraccuracy of inferences and ignore team performance.

Whilemany factors influence team performance, westudyone critical factor in this work: humans’ mental model ofthe AI system that they are working with. In settings wherethe human is tasked with deciding when and how to makeuse of the AI system’s recommendation, extracting benefitsfrom the collaboration requires the human to build insights(i.e., a mental model) about multiple aspects of the capabil-ities of AI systems. A fundamental attribute is recognitionof whether the AI system will succeed or fail for a partic-ular input or set of inputs. For situations where the humanuses theAI’soutput to makedecisions, this mental model oftheAI’serror boundary—which describes theregionswhereit is correct versus incorrect—enables the human to predictwhen the AI will err and decide when to override the auto-mated inference.

We focus here on AI-advised human decision making, asimplebut widespread form of human-AI team, for example,

ML Model

Readmission Prediction

ClassifierHuman

Decision

When can I trust it?

How can I adjust it?

Small decision tree over semantically

meaningful primitives

Abstract

When can I trust it?How can I adjust it?

Linear model over semantically

meaningful primitives

Abstract

Figure 4: A part of Figure 1 from [5] showing 3 (of 56 total) components for a GA2M model, which was trained to predict a

patient’s r isk of dying from pneumonia. The two l ine graphs depict the contr ibution of individual features to risk : a) patient’s

age, and b) boolean variable asthma. The y-axis denotes i ts contr ibution (log odds) to predicted risk . The heat map, c, visual izes

the contr ibution due to pairwise interactions between age and cancer rate.

evidence from the presence of thousands of words, one could see

the e ect of a given word by looking at the sign and magnitude of

the corresponding weight. This answers the question, “What if the

word had been omitted?” Similarly, by comparing the weights asso-

ciated with two words, one could predict the e ect on the model of

substituting one for the other.

Ranking Intel l igible Models: Since one may have a choice of

intelligible models, it is useful to consider what makes one prefer-

able to another. Grice introduced four rules characterizing coop-

erative communication, which also hold for intelligible explana-

tions [11]. The maxim of quality says be truthful, only relating

things that are supported by evidence. The maxim of quantity says

to give as much information as is needed, and no more. The maxim

of relation: only say things that are relevant to the discussion. The

maxim of manner says to avoid ambiguity, being be as clear as

possible.

Miller summarizes decades of work by psychological research,

noting that explanations are contrastive, i.e., of the form “Why P

rather than Q?” The event in question, P, is termed the fact and Q is

called the foil [28]. Often thefoil isnot explicitly stated even though

it is crucially important to the explanation process. For example,

consider the question, “Why did you predict that the image depicts

an indigo bunting?” An explanation that points to the color blue

implicitly assumes that the foil is another bird, such as a chickadee.

But perhaps the questioner wonders why the recognizer did not

predict a pair of denim pants; in this case a more precise explana-

tion might highlight the presence of wings and a beak. Clearly, an

explanation targeted to the wrong foil will be unsatisfying, but the

nature and sophistication of a foil can depend on the end user’s

expertise; hence, the ideal explanation will di er for di erent peo-

ple [7]. For example, to verify that an ML system is fair, an ethicist

might generate more complex foils than a data scientist. Most ML

explanation systems have restricted their attention to elucidating

thebehavior of abinary classi er, i.e., wherethere isonly onepossi-

ble foil choice. However, as we seek to explain multi-class systems,

addressing this issue becomes essential.

Many systems are simply too complex to understand without

approximation. Here, the key challenge is deciding which details

to omit. After long study psychologists determined that several

criteria can beprioritized for inclusion in an explanation: necessary

causes (vs. su cient ones); intentional actions (vs. those taken

without deliberation); proximal causes vs. distant ones); details that

distinguish between fact and foil; and abnormal features [28].

According to Lombrozo, humans prefer explanations that are

simpler (i.e., contain fewer clauses), moregeneral, and coherent (i.e.,

consistent with what thehuman’s prior beliefs) [24]. In particular,

she observed the surprising result that humans preferred simple

(one clause) explanations to conjunctive ones, even when the prob-

ability of the latter was higher than the former) [24]. These results

raise interesting questions about the purpose of explanations in

an AI system. Is an explanation’s primary purpose to convince a

human to accept thecomputer’sconclusions (perhapsby presenting

a simple, plausible, but unlikely explanation) or is it to educate the

human about the most likely true situation? Tversky, Kahneman,

and other psychologists have documented many cognitive biases

that lead humans to incorrect conclusions; for example, people

reason incorrectly about the probability of conjunctions, with a

concrete and vivid scenario deemed more likely than an abstract

one that strictly subsumes it [17]. Should an explanation system

exploit human limitations or seek to protect us from them?

Other studies raise an additional complication about how to

communicate a system’s uncertain predictions to human users.

Koehler found that simply presenting an explanation asafact makes

people think that it is more likely to be true [18]. Furthermore,

explaining a fact in the same way as previous facts have been

explained ampli es this e ect [35].

3 INHERENTLY INTELLIGIBLE MODELS

Several AI systems are inherently intelligible. We previously men-

tioned linear models as an example that supports conterfactuals.

Unfortunately, linear models have limited utility because they of-

ten result in poor accuracy. More expressive choices may include

simple decision trees and compact decision lists. We focus on Gen-

eralized additive models (GAMs), which are a powerful class of

ML models that relate a set of features to the target using a linear

combination of (potentially nonlinear) single-feature models called

shape functions [25]. For example, if y represents the target and

{x1, . . . .xn } represents the features, then a GAM model takes the

form y = β0 +Õ

j f j (xj ), where the f i sdenote shape functions and

the target y is computed by summing single-feature terms. Popular

shape functions include non-linear functions such as splines and

decision trees. With linear shapefunctions GAMsreduce to a linear

Part of Fig 1 from R. Caruana, Y. Lou, J. Gehrke, P. Koch, M. Sturm, and N. Elhadad. “Intelligible models for healthcare:

Predicting pneumonia risk and hospital 30-day readmission.” In KDD 2015.

1 (of 56) components of learned GA2M: risk of pneumonia death

possible.

form y = β0 +Õ

possible.

form y = β0 +Õ

Abstract

Deep cascade of CNNs

Variational networks

Transfer learning

Kidney MRI From [Lundervold & Lundervold 2018]https://www.sciencedirect.com/science/article/pii/S0939388918301181

Input: Pixels

Features are not semantically

meaningful

Use DirectlyInteract with Simpler

NoIntelligible? •Explanations

•Controls

Map to Simpler Model

Inscrutable

Model Too Complex

Simplify by currying –> instance-specific explanation

Simplify by approximating

Features not Semantically Meaningful

Map to new vocabulary

Usually have to do both of these!

Usually have to do all of these!Simpler Explanatory

Inscrutable

LIME - Local Approximations

1. Sample points around p

2. Use complex model to predict labels for each sample

3. Weigh samples according to distance from p

4. Learn new simple modelon weighted samples(possibly using different features)

5. Use simple model as explaination

Slide adapted from Marco Ribeiro – see “Why Should I Trust You?: Explaining the Predictions of Any Classifier,” M. Ribeiro, S. Singh, C. Guestrin,

SIGKDD 2016

To explain prediction for point p …

To create features for explanatory classifier,

Compute `superpixels’ using off-the-shelf image segmenter

Hope that feature/values are semantically meaningful

To sample points around p, set some superpixels to grey

Explanation is set of superpixels with high coefficients…

Understandable

Over-Simplification

Accurate

Inscrutable

Any model simplification is a Lie

Inscrutable

Need Desiderata

Fewer conjuncts (regardless of probability)

Explanations consistent with listener’s prior beliefs

Presenting an explanation made people believe P was true

If explanation ~ previous, effect was strengthened

… Even when the explainer is wrong…

when not to trust

Abstract

Except…

In these papers, Accuracy(Humans) << Accuracy(AI)

So… the rational decision is to omit the humans (not explain)

2 3 4 5

Assistance Architecture

0) Solo Human (No AI)

1) AI Recommends

2) AI also gives its confidence

3) AI also explains (LIME-like)

4) AI gives human explanations

Explanations are Convincing

AI Correct

AI Incorrect

AI Correct

AI Incorrect

AI Correct

AI Incorrect

Better Explanations are More Convincing

AI Correct

AI Incorrect

That Other Question…

Abstract

• Deep neural paper embeddings

• Explain with linear bigrams

Beta: s2-sanity.apps.allenai.org

Tuning

Agents

If all one cared about was the explanatory model,

one could change this parameters… but not even the

features are shared with the neural model!

Tuning

Agents

Instead… We generate new training instances by

varying the feedback feature, weight by distance to

x’…

Tuning

Agents

Instead… We generate new training instances by

varying the feedback feature, weight by distance to

x’…

and Retrain.

Good News:

Less Good News:

No significant improvement on feed quality (team

performance) as measured by clickthru

Summary

Abstract

ML Model

Readmission Prediction

ClassifierHuman

Decision

2 3 4 5

Human-AI Team

Helpful

Use DirectlyInteract with Simpler

NoIntelligible? •Explanations

•Controls

Map to Simpler Model

INITIALLY

DURING

INTERACTION

WHEN WRONG

OVER TIME

Make clear

how well the

system can

do what it

can do.

Make clear

what the

system

can do.

Time services

based on

context.

contextually

relevant

information.

relevant

social norms.

Mitigate

social biases.

Support

efficient

invocation.

Support

efficient

dismissal.

Support

efficient

correction.

services when

in doubt.

Make clear

why the

system did

what it did.

Remember

recent

interactions.

Learn from

user behavior.

Update and

cautiously.

Encourage

granular

feedback.

Provide

global

controls.

Notify users

changes.

Convey the

consequences

of user

actions.

Thanks! Questions?

Tutorial website: https://www.microsoft.com/en-us/research/project/guidelines-for-human-ai-

interaction/articles/aaai-2020-tutorial-guidelines-for-human-ai-interaction/

Learn the guidelines

Introduction to guidelines for human-AI interaction

Interactive cards with examples of the guidelines in practice

Use the guidelines in your work

Printable cards (PDF)

Printable poster (PDF)

Find out more

Guidelines for human-AI interaction design, Microsoft Research Blog

AI guidelines in the creative process: How we’re putting the human-AI guidelines into practice at

Microsoft, Microsoft Design on Medium

How to build effective human-AI interaction: Considerations for machine learning and software

engineering, Microsoft Research Blog

aiguidelines · 2020. 2. 8. · opportunity analysis engagements with practitioners findings &...

Documents

engagements 2011 - refworld

weddings and engagements

201208 engagements

binaryflips engagements 2012

guide to compilation engagements - ifac€¦ · the guide...

standard on assurance engagements asae 3500 performance...

attest engagements - aicpa

engagements libqual

implementation tool for practitioners - ias plus...other...

attest engagements - · pdf fileattest engagements 1391...

advertisement no. 01/2019 - odisha govt jobs · 2019. 4....

audit engagements questionnaire

weddings, engagements & couples

afganistan engagements

kontakt, vertrieb & engagements

the terminology of issai 100 compared with the interna...

nos engagements

3 speaking engagements

les 60 engagements

professional engagements