the concrete substrates of abstract rule use bradley love

The Concrete Substrates of Abstract Rule Use

Bradley Love

www.ccc.utexas.edu

Three Domains

• Tracking basic statistics in our environment.

• Learning seemingly abstract rules.

• Learning and reasoning about future rewards in dynamic environments.

Basic Story

• Learning is constrained by– non-rational mechanisms– the nature of domain representations– trial-by-trial updates with regards to current

representations in memory.

• Seemingly abstract or rational

explanations prove not to be upon closer inspection.

Tracking Basic Statistics

• Variance• Mean

Tracking Variability

Rational Models

• Fried and Holyoak (1984)– Maximum likelihood– unbiased estimator/consistent

• Tenenbaum and Griffiths (2001)– Bayesian updating– Globally Bayesian

Tracking Variability

Obvious Mechanistic Approach

• Error driven learning of cluster position and variance.– updates are trial by trial– are with regard to a memory representation

of each category

Formally..

Mechanistic Model

Rational or Mechanistic?

• Trial by trial error-driven learning makes local updates.– all statistics are local in error-driven learning

• We can use this to disentangle the competing explanations.

Tracking Variability (locally)

What about the mean?

Tracking Statistics

• People appear to update with regards to representations in memory in a trial-by-trial fashion.

• Learning is error-driven and sensitive to the error term (i.e., task goal)– we see this in many domains (e.g., inference

vs. classification learning)

Verbal Rules?

• Certainly, we can implement and report strategies, but that doesn’t imply categories are represented as rules.

• Are rules more concrete than they appear?

Are people really using rules?

• Rules aren’t always rules (e.g., Allen & Brooks, 1991; Ramscar, 2002).

• One alternative is clusters with selective attention.– more schema-like

Sakamoto and Love (2004)

Value 2

2 1 1 1 2 A2 2 1 2 2 B2 1 2 1 1 B2 2 2 1 1 B2 1 1 2 2 B

Value 1

1 1 1 2 1 B1 2 1 2 2 A1 1 2 1 1 A1 2 2 1 1 A1 1 1 2 2 A1 2 1 1 1 A1 1 2 2 2 A1 1 2 1 2 A1 2 1 2 1 A


Value 2

2 1 1 1 2 A2 2 1 2 2 B2 1 2 1 1 B2 2 2 1 1 B2 1 1 2 2 B

Value 1

1 1 1 2 1 B1 2 1 2 2 A1 1 2 1 1 A1 2 2 1 1 A1 1 1 2 2 A1 2 1 1 1 A1 1 2 2 2 A1 1 2 1 2 A1 2 1 2 1 A

Rule Route

If small, then A.

If large, then B.

Exception Route

memorize item


Recognition

50

60

70

80

90

100

Small Large Small Large

Inconsistent Consistent

2AFC Accuracy

Value 2

2 1 1 1 2 A2 2 1 2 2 B2 1 2 1 1 B2 2 2 1 1 B2 1 1 2 2 B

Value 1

1 1 1 2 1 B1 2 1 2 2 A1 1 2 1 1 A1 2 2 1 1 A1 1 1 2 2 A1 2 1 1 1 A1 1 2 2 2 A1 1 2 1 2 A1 2 1 2 1 A

Violating-BFollowing-A Following-A


Recognition

50

60

70

80

90

100

Small Large Small Large

Inconsistent Consistent

2AFC Accuracy

Value 2

2 1 1 1 2 A2 2 1 2 2 B2 1 2 1 1 B2 2 2 1 1 B2 1 1 2 2 B

Value 1

1 1 1 2 1 B1 2 1 2 2 A1 1 2 1 1 A1 2 2 1 1 A1 1 1 2 2 A1 2 1 1 1 A1 1 2 2 2 A1 1 2 1 2 A1 2 1 2 1 A

“Abstract” Rules

• Abstract rules are not specified by fixed-values.– e.g., learning an abstract notion of same and

different

• Could abstract responding be grounded in concrete episodes and trial-by-trial learning?

Exemplar Model

Kruschke’s (1992) ALCOVE

Category association

Learns association and attention weights

Size

Lum

inan

ce

Attention

Size

Lum

inan

ce

Learns association and attention weights

Adding Structure:

ChaseChase

Chaser ChaserChased Chased

one-to-one correspondence

ChaseChase

One or the other, not both

Chaser ChaserChased Chased

Parallel Connectivity

ChaseChase

Perfect relational match, but feature mismatches.Parallel connectivity satisfied.

Parallel Connectivity

Two relational mismatches, but high feature match.Parallel Connectivity violated.

ChaseChase

Disambiguation: Attention

ChaseChase

Attention determines the trade-off.

BRIDGES

Learning Abstract Rules

• Marcus et al. showed infants can learn to distinguish simple grammars.

• AAB pattern vs. ABB pattern

• No diagnostic features• Infants habituated to one grammar, then

tested with novel sounds from both grammars

Do, Do, Re (pause) Me, Me, Fa (pause) Ti, Ti, Sa (pause) ...

Representation

• Attention on position constrained• Only type-token relation used

a a b

Type_of( , )

Type_of( , )

Type_of( , )

A

A

B

a1

a2

b1

Same grammar

• Relations: perfect similarity, parallel connectivity preserved

a a b

Type_of( , )

Type_of( , )

Type_of( 1, )

c c d

Type_of( , )

Type_of( , )

Type_of( , )

A

A

B

C

C

D

a1

a2

b

c1

c2

d1

Opposite grammar

• Relations: 1 mismatch, parallel connectivity not preserved

a a b

Type_of( , )

Type_of( , )

Type_of( , )

e f f

Type_of( , )

Type_of( , )

Type_of( , )

A

A

B

E

F

F

a1

a2

b1

e

f1

f2

Learning

• Attention shifts from inconsistent features to predictive relations

• Concrete exemplar-based memory and attention shifting to relations explains performance

Lurking Concreteness in Abstract Concepts

• Pigeons appear to learn the concept of same and different.

Same Different

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.








QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.











QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.







Young, Wasserman and others - 1997, 2001, 2004

Experimental Details

•16 “Same” grids, 16 “Different” grids -> 4 x 4•Pigeons trained to peck green for same, red for different•Then tested with novel icons and old icons•accuracy was 83% with old icons, 71% with new

BRIDGES captures this pattern...

Concrete basis revealed in graded structure

Shades of grey

Different






Same





































Determining relational similarity

Shades of grey

DifferentQuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.





Same





































MatchMatch

Mismatch

Mismatch

2 Mismatches

13 Mismatches

Relational categories are graded

Shades of grey

DifferentSame



























Shades of grey

















Shades of grey

DifferentSame









































Shades of grey

DifferentSame




































QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.




Shades of grey

DifferentSame






































Shades of grey

DifferentSame







































14 Mismatches 1 MismatchQuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

0

0.2

0.4

0.6

0.8

1

1.2

0.3 0.35 0.4 0.45 0.5 0.55 0.6

BRIDGES P(Different)

Pigeons P(Different)

• BRIDGES is able to attain an excellent fit ( R2 =.94)

• People respond similarly (Wasserman, Young, & Cook, 2004)

Learning Abstract Concepts

• Exemplar-based learning is sufficient– error-driven learning– trial-by-trial updates– attention shifting– analogy

• The concrete underbelly of abstract concepts shines through.– graded notion of same and different

Learning in Dynamic Environments

• How do people gain an understanding of complex systems that require ongoing interaction?

• Do people reason abstractly about these domains or does learning operate as in category learning?

• How strong of a constraint is people’s representations of the system’s state?

Melioration/Maximization

Studied extensively in a variety of animals (Herrnstein & Vaughan, 1980; Tunny & Shanks, 2002; Neth, Sims, Gray, 2006)Overwhelming result is animals prefer immediate gain and do not appear to maximize long term expected rewardsProblems for theories of rational choice?

Results

Conclusions

• Optimality with regards to people’s mental representation of state representation, not with regards to the environment.

• Trial-by-trial updates and bootstrap learning lead to insight into how the system operates.

Overall Conclusions

• Abstract responding is rooted in mechanisms that are concrete.

• Updates during learning are trial-by-trail, non-rational, and made with reference to current mental representations.

• This general approach to understanding cognition leads to rich predictions that are non-obvious.

Acknowledgements

• Category Learning– Matt Jones, Tyler Davis, Yasu Sakamoto

• BRIDGES/Analogy– Marc Tomlinson

• Dynamic Decision Making– Todd Gureckis

• Funding– AFOSR, ARL, NSF

Thank You.

Any Questions?

Sakamoto & Love (in prep)

Inference

Sakamoto & Love (in press)

Differentiation account:

Better memory for BX.

Isolation account:

Better memory for AX.


Following reconstruction, subjects classified the lines without corrective feedback.

The isolated exceptions were easier to identify (90% vs. 78%) than the differentiated exceptions, although the differentiated exceptions were more accurately reconstructed.

Beyond Classification Learning

• Subtle differences in the induction task can lead to large differences in learning difficulty and conceptual organization (Love, 2002; 2003).

• Current theories are not sensitive to these pressures.– e.g., Anderson’s (1991) rational model.– brittle, laboratory theories.

• Important because we interact in various ways with category information.

Inference vs. Classification

Sakamoto and Love (submit.)

• Predictions– inference learning will lead to better learning of correlated

features, even if not directly queried during training.– Knowledge should be gained more efficiently through

inference learning.– Better retention with inference learning.

• Training Implications

Sixgill

2 1 1 1 1 1 2 1 1 2 1 2 2 1 1 1 1 2 2 1 1 1 1 2 2

Sakamoto & Love (submit.)

Tiger

1 2 2 2 2 2 1 2 2 1 2 1 1 2 2 2 2 1 1 2 2 2 2 1 1

Sixgill

2 1 1 1 1 S1 2 1 1 2 S1 2 2 1 1 S1 1 2 2 1 S1 1 1 2 2 S

Tiger

1 2 2 2 2 T2 1 2 2 1 T2 1 1 2 2 T2 2 1 1 2 T2 2 2 1 1 T


Inference

Inference will lead to better learning of correlated features even if not directly queried during learning.


Sixgill

2 1 1 1 1 S1 2 1 1 2 S1 2 2 1 1 S1 1 2 2 1 S1 1 1 2 2 S

Classification

Tiger

1 2 2 2 2 T2 1 2 2 1 T2 1 1 2 2 T2 2 1 1 2 T2 2 2 1 1 T

Classification will lead to better learning of the first dimension.


Inference


Classification


ClassificationInference

Sixgill

2 1 1 1 1 S1 2 1 1 2 S1 2 2 1 1 S1 1 2 2 1 S1 1 1 2 2 S

Tiger

1 2 2 2 2 T2 1 2 2 1 T2 1 1 2 2 T2 2 1 1 2 T2 2 2 1 1 T

Sixgill

2 1 1 1 1 S

1 2 1 1 2 S1 2 2 1 1 S1 1 2 2 1 S1 1 1 2 2 S

Tiger

1 2 2 2 2 T

2 1 2 2 1 T2 1 1 2 2 T2 2 1 1 2 T2 2 2 1 1 T

2030405060708090

100

Inference

% Accuracy

Dim 1 Dim 2-4 Dim 5

Tiger

1 2 2 2 22 1 2 2 12 1 1 2 22 2 1 1 22 2 2 1 1


Sixgill

2 1 1 1 11 2 1 1 21 2 2 1 11 1 2 2 11 1 1 2 2

5th graders

2030405060708090

100

Inference Classification

% Accuracy

Dim 1 Dim 2-4 Dim 5

Tiger

1 2 2 2 22 1 2 2 12 1 1 2 22 2 1 1 22 2 2 1 1


Sixgill

2 1 1 1 11 2 1 1 21 2 2 1 11 1 2 2 11 1 1 2 2

5th graders

Undergraduates

2030405060708090

100


% Accuracy

Dim 1 Dim 2-4 Dim 5


2030405060708090

100


% Accuracy

Dim 1 Dim 2-4 Dim 5

5th graders


Inference leads to better learning of correlated features, even those not queried.

Children like sharks learned through inference better.

Classroom exercises and assignments should emphasize reasoning from the category to multiple properties rather than from a set of properties to the category.

the concrete substrates of abstract rule use bradley love

Documents

trial learning

trial errordriven learning

abstract rulesabstract

trial updates

trial fashion

rules arent

abstract notion

errordriven learningwe