machine reading and reasoning with neural program …

Machine Reading and Reasoning with Neural Program Interpreters

Sebastian Riedel

Machine Reading

@riedelcastro

Collaborators

!2

Tim RocktäschelMatko Bosnjak

Pontus Stenetorp

(now at Oxford)(UCL)

(UCL)

Johannes Welbl(UCL)

Jason Naradowsky(now Johns Hopkins University)

Marzieh Saeidi Max Bartolo

Guillaume Bouchard(Bloomsbury AI)

(Bloomsbury AI) (Bloomsbury AI)

!3

The “Classic” NLP Paradigm

!3

In January 1880, two of Tesla's uncles put together enough money to help him leave Gospić for Prague where he was to study. Unfortunately, he arrived too late to enrol at Charles-Ferdinand University; he never studied Greek, a required subject; and he was illiterate in Czech, another required subject. Tesla did, however, attend lectures at the universi ty, although, as an auditor, he did not receive grades for the courses.

[Language]


!3


[Language] [Meaning]


Tesla

PraguemovedTo

!3



What city did Tesla move to in 1880?

[Information Need]


Tesla

PraguemovedTo

!3




[Information Need]

Prague


Tesla

PraguemovedTo

!3




[Information Need]

Prague


Tesla

PraguemovedTo

+reusable cache+interpretable+supports reasoning

!3




[Information Need]

Prague


Tesla

PraguemovedTo


-misses details-error prone

-engineering heavy

“Should we separate meaning from language?”

!3

Chris Manning @AKBC 2013




[Information Need]

Prague

Tesla

PraguemovedTo



-engineering heavy


!3





[Information Need]

Prague

[Maybe not?]Tesla

PraguemovedTo



-engineering heavy


!4





[Information Need]

Prague

[Maybe not?]Tesla

PraguemovedTo

End-to-End Reading and Comprehension

!5

(Hermann et. al 2015, Seo et al., 2016, Rajpurkar et al., 2016, Weissenborn 2016…)


[Language]


Prague

[Information Need]

Rules: Entertainment

Offers of entertainment may be accepted or made where they are necessary to develop and maintain outside contacts relevant to work responsibilities… disproportionately lavish. Invitations to expensive or exclusive sporting or cultural events should not be accepted.

3. Invitations from institutions supervised by the Bank or the FCA, or from professional advisers, should not be accepted without prior agreement of an Executive Director or Governor ….

Machine Reading & Comprehension

!6

Would it be appropriate for me to accept court side seating tickets to an NBA event?





!6




Would it be appropriate for me to accept court side seating tickets to an NBA event?

Invitations to expensive or exclusive sporting or cultural events should not be accepted

…

Archiving Inbox Items

Once an inbox item has been archived it will no longer appear in the list of inbox items returned by cape.client.CapeClient.get_inbox(). To archive an item call cape.client.CapeClient.archive_inbox() with the ID of the inbox item to archive:

from cape.client import CapeClient

cc = CapeClient()cc.login('username', 'password')cc.archive_inbox('4123')


!7

Which function do I call for storing items in cape?

…





…






!7

Which function do I call for storing items in cape?

cape.client.CapeClient.archive_inbox()

How to read and reason end-to-end?

!8

Learning to Read and Aggregate

!9

Which medical specialty deals with pituitary ACTH hypersecretion?


!9

Which medical specialty deals with pituitary ACTH hypersecretion? Pituitary ACTH hypersecretion ... is a form of

hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary …


!9



A major organ of the endocrine system, the anterior pituitary is the glandular, anterior lobe that ...


!9




The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology, a branch of internal medicine.


!9


Endocrinology

Pituitary ACTH hypersecretion ... is a form of hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary …




!9


Endocrinology




Part 1


!9


Endocrinology




Part 1

NIPS 2017, TACL 2018

Learning to Read and Calculate

!10

How many pictures were in each of the albums? Isabel uploaded 2 pictures from her phone and 4 from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album.


!10

How many pictures were in each of the albums?

2

Isabel uploaded 2 pictures from her phone and 4 from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album.


!10


2


Part 2


!10


2


Part 2

ICML 2017

Learning to Interpret Rules

!11

I am working for an employer in Canada since December 2017. Do I need to carry on paying UK national insurance?

If you’re self-employed You might be able to carry on paying National Insurance if you’re: - usually self-employed in the UK - working abroad temporarily

If you do, you won’t have to pay social security contributions in the country where you’re working. You can check this by completing form CA9107.

Working in any other country You’ll carry on paying National Insurance for the first 52 weeks you’re abroad if you’re working for an employer outside the EEA.


!11


Yes





!11


Part 3

Yes




And Ask Clarification Questions

!12


Part 3If you’re self-employed You might be able to carry on paying National Insurance if you’re: - usually self-employed in the UK - working abroad temporarily




!12


Part 3

Have you been working abroad 52 weeks or less?





!12


Part 3


Yes





!12


Part 3


Yes

Yes





!12


Part 3


Yes

Yes

ONGOING




Part 1: Learning to Aggregate

!13

Tim RocktäschelJohannes WelblPontus Stenetorp

Neural Theorem Provers

!14






!15





Rocktäschel and Riedel NIPS 2017

Agent


!15






Agent

Program Interpreter


!15






Agent

Program Interpreter

Reader


!15






Agent

Program Interpreter

Reader

QuestionCode

[translates questions]


!16





Agent

Program Interpreter

Reader

Question

What is ACTH hypersecretion created by?


Code



!17





Agent

Program Interpreter

Reader

Question

Which specialty deals with pituitary ACTH hypersecretion?

Question

What is the pituitary a part of?


Code



!18





Agent

Program Interpreter

Reader

Question


Question


Question

What topic covers the endocrine system?


Code


Code

Vectors Correspond to Interpretable Rules

!19





Agent

Program Interpreter

Reader

Question


Question


Question

What topic covers the endocrine system?

X deals with Y if Y produced by Z Z is a part of U X deals with U


Catch: Currently only Works on Relational Data

!20

createdBy(hypersecretion, anterior pituitary)dealsWith(hypersecretion, X)

partOf(endocrine system, anterior pituitary)

dealsWith(endocrine system, endocrinology)


Agent

Program Interpreter

Reader

Question


Question


Question

covers(endocrine system, X)

Code

X deals with Y if Y produced by Z Z is a part of U X deals with U

Why Program Interpreter/Theorem Prover?

!21

createdBy(hypersecretion, anterior pituitary)




Agent

Amounts the Backward Chaining algorithm used in Prolog Interpreters

dealsWith(hypersecretion, X)

Why Neural? Supports Soft Unification

!22





Agent

Program Interpreter

Reader

Question


Question


Question

covers(endocrine system, X)

Code

X deals with Y if Y produced by Z Z is a part of U X covers U

Learn Rules by Learning Symbol Representations

!23





Agent

Program Interpreter

Reader

Question


Question


Question

rel62(endocrine system, X)

Code

X rel32 Y if Y rel42 Z Z rel51 U X rel62 U

Before Learning

!24

rel32


rel42rel51

rel62

X deals with Y if Y produced by Z Z is a part of U X deals with U deals with

produced by

is a part of

covers

After Learning

!25

rel32


rel42

rel51

rel62 X deals with Y if Y produced by Z Z is a part of U X deals with U

deals with

produced by

is a part of

covers

Results on Benchmark (Rank of Correct Answer)

!26Rocktäschel and Riedel NIPS 2017


!26

0

25

50

75

100

Countries S3 UMLS NationsComplex NTP Complex NTP Complex NTP



!26

0

25

50

75

100


8696

48.4



!26

0

25

50

75

100


8999

77.386

96

48.4



!26

0

25

50

75

100


8999

77.386

96

48.4

Comparable or Better than Baselines, and interpretable


Limitations

Scalability

Currently only works for KBs with < 10k facts (solved!)

Small proof depth (getting there)

Still requires relational representation (ongoing)

!27

How to create Datasets for Multihop Reading and Reasoning?

!28

Johannes WelblPontus Stenetorp

Dataset Construction Method

!29TACL 2018


!29

Unlabelled Text

Knowledge Base

TACL 2018


!29

Unlabelled Text

Knowledge Base


TACL 2018


!29

Unlabelled Text

Knowledge Base


TACL 2018

Multihop Dataset


!30

DocumentsEntitiesKB Triple Instance


!30

DocumentsEntities

Jamie Burnett, citizenship, Scotland

KB Triple Instance


!30

DocumentsEntities


What is the nationality of Jamie Burnett?

KB Triple Instance


!30

DocumentsEntities

Jamie Burnett



KB Triple Instance


!30

DocumentsEntities

Jamie Burnettdescribed in



KB Triple Instance


!30

DocumentsEntities


mentions



KB Triple Instance


!30

DocumentsEntities


mentions

HamiltonJamie Burnett, citizenship, Scotland


KB Triple Instance


!30

DocumentsEntities


mentions

described in



KB Triple Instance


!30

DocumentsEntities


mentions

described inmentions



KB Triple Instance


!30

DocumentsEntities

Jamie Burnett

Scotland

described in

mentions




KB Triple Instance

Scotland


!30

DocumentsEntities

Jamie Burnett

Scotland

described in

mentions



Yixing Open


KB Triple Instance

Scotland


!30

DocumentsEntities

Jamie Burnett

Scotland

described in

mentions



Yixing Open

China


KB Triple Instance

Scotland

China


!31

Unlabelled Text

Knowledge Base


[Conditionally Accepted to TACL]

Multihop Dataset


!32


WikiHopUnlabelled Text

Knowledge Base


!33


MedHopUnlabelled Text

Knowledge Base

Baseline Results

!34

Baseline Results

!34

Accu

racy

[%]

0

8.333

16.667

25

33.333

41.667

50

WikiHop MedHop

RandomTF-IDFBiDAF

Baseline Results

!34

Accu

racy

[%]

0

8.333

16.667

25

33.333

41.667

50

WikiHop MedHop

RandomTF-IDFBiDAF

13.9

11.5

Baseline Results

!34

Accu

racy

[%]

0

8.333

16.667

25

33.333

41.667

50

WikiHop MedHop

RandomTF-IDFBiDAF

9

25.6

13.9

11.5

Baseline Results

!34

Accu

racy

[%]

0

8.333

16.667

25

33.333

41.667

50

WikiHop MedHop

RandomTF-IDFBiDAF

47.8

42.9

9

25.6

13.9

11.5

Reduction to Traditional Machine Comprehension

!35


Endocrinology




Reduction to Traditional Machine Comprehension

!36


Endocrinology A major organ of the endocrine system, the anterior pituitary is the glandular, anterior lobe that ...



Do Neural Reading Models Aggregate?

!37


Endocrinology A major organ of the endocrine system, the anterior pituitary is the glandular, anterior lobe that ...



Removing Relevant Documents, Keep Answer Documents

!38

Accu

racy

[%]

25

30

35

40

45

50

55

WikiHop MedHop

30.4

44.6

33.7

54.5

BIDAFBIDAF doc-rem 10%

!39

TACL 2018

Phil’s View?

Good:

You cannot just do context matching

We spent a lot of efforts in removing biases

We have extra information at annotation time we don’t release

Bad:

It is still multiple choice

Test set comes from the same process as training (kinda—-see above)

!40

Part 2: Learning to Read and Calculate

!41

Tim RocktäschelMatko Bosnjak Jason Naradowsky

Machine Reading and Reasoning: Math

!42


2


Differentiable Program Interpreters

!43

Program Interpreter

Bosnjak et al. ICML 2017

Reader

Model




!43

Program Interpreter0 1 2 3 4 5 6 7 8 9

Heap


Reader

Model




!43

0 1 2 3 4 5 6 7 8 9

34

2

d

Stack


Heap


Reader

Model




!43

0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

34

2

d

Stack


Heap


Reader

Model



0 1 2 3 4 5 6 7 8 9

34

2

d

Stack0 1 2 3 4 5 6 7 8 9p

Code


!44

0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

42

3

d

Stack


Heap


Reader

Model



0 1 2 3 4 5 6 7 8 9

34

2

d

Stack0 1 2 3 4 5 6 7 8 9p

Code


!45

0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

42

3

d

Stack


Heap


Reader

Model



0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

63

d

Stack

0 1 2 3 4 5 6 7 8 9

34

2

d

Stack0 1 2 3 4 5 6 7 8 9p

Code


!46

0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

42

3

d

Stack


Heap


Reader

Model



0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

63

d

Stack0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

36

d

Stack

0 1 2 3 4 5 6 7 8 9

34

2

d

Stack0 1 2 3 4 5 6 7 8 9p

Code


!47

0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

42

3

d

Stack


Heap


Reader

Model



0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

63

d

Stack0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

36

d

Stack0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

2

d

Stack

0 1 2 3 4 5 6 7 8 9

34

2

d

Stack0 1 2 3 4 5 6 7 8 9p

Code

Training

!48

0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

42

3

d

Stack


Heap


Reader

Model



0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

63

d

Stack0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

36

d

Stack0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

2

d

Stack

0 1 2 3 4 5 6 7 8 9

34

2

d

Stack0 1 2 3 4 5 6 7 8 9p

Code

Training

!49

0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

42

3

d

Stack


Heap


Reader

Model



0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

63

d

Stack0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

36

d

Stack0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

2

d

Stack

0 1 2 3 4 5 6 7 8 9

34

2

d

Stack0 1 2 3 4 5 6 7 8 9p

Code

Training

!50

0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

42

3

d

Stack


Heap


Reader

Model



0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

63

d

Stack0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

36

d

Stack0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

2

d

Stack

0 1 2 3 4 5 6 7 8 9

34

2

d

Stack0 1 2 3 4 5 6 7 8 9p

Code

Training

!51

0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

42

3

d

Stack


Heap


Reader

Model



0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

63

d

Stack0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

36

d

Stack0 1 2 3 4 5 6 7 8 9p

Code0 1 2 3 4 5 6 7 8 9

2

d

Stackdef solve(x):

{+|-|%|*} solve(y)

Results on Math Problems (Accuracy)

!52Bosnjak et al. ICML 2017



50

61.5

73

84.5

96

Ours Seq2Seq



50

61.5

73

84.5

96

Ours Seq2Seq

96



50

61.5

73

84.5

96

Ours Seq2Seq

9596



50

61.5

73

84.5

96

Ours Seq2Seq

9596


(2 + 4) / 3

Seq2Seq solves simpler problem

Limitations

Continuous relaxations difficult to train

Hard to learn long programs

Hard to learn with recursive function calls

Need better understanding of SGD for networks with discrete/stochastic activations

!53

Part 3: Learning to Interpret Rules and Ask Clarification Questions

!54

Marzieh Saeidi Max BartoloGuillaume BouchardTim Rocktäschel


!55


Yes





!56



Yes

Yes




Generating a Corpus

!57





Generating a Corpus

!57



No




Generating a Corpus

!57



No

No




Generating a Corpus

!57



No

No




Yes

Generating a Corpus

!57



No

No




Yes

Are you working for an employer outside of the EEA?

Generating a Corpus

!57



No

No




Yes


No

Generating a Corpus

!57



No

No




Yes


No

No

Generating a Corpus

!57



No

No




Yes


No Yes

No

Generating a Corpus

!57



No

No




Yes


No Yes

No Yes

Generating a Corpus

!58






Yes


Yes

Yes

Generating a Corpus

!59






Yes


Yes

Yes

Generating a Corpus

!60





Yes

Generating a Corpus

!61






Yes


Yes

Yes

Generating a Corpus

!62






Yes


Yes

Yes

Generating a Corpus

!63






Yes

Yes

Generating a Corpus

!64






Yes

Yes

Generating a Corpus

!64






Yes

Yes

Created ~40k instances this way

Seq2Seq?

!65





Seq2Seq?

!65


Were you laid before earning at least the lower earnings limit?




Seq2Seq?

!65


Were you laid before earning at least the lower earnings limit?




Seq2Seq with attention (to your needs)

Conclusion

!66

Conclusion

Great Progress in End-to-End Reading Comprehension

!66

Conclusion


Reasoning (aggregation, calculation etc.) and conversation end-to-end is still very challenging

!66

Conclusion



Our Approaches

!66

Conclusion



Our Approaches

cast reasoning as program learning and execution

!66

Conclusion



Our Approaches


are end-to-end differentiable (can be trained on downstream loss)

!66

Conclusion



Our Approaches



are inspired and tied to traditional symbolic formalisms (Forth, Prolog/Datalog)

!66

Conclusion



Our Approaches




are learnt models are interpretable

!66

Conclusion



Our Approaches





allow injection of prior knowledge

!66

Conclusion



Our Approaches





allow injection of prior knowledge

create datasets

!66