designing precise pushdown higher-order flow analysesdeeglaze.github.io/ibmpl2012-slides.pdf · 2....

Designing Precise Pushdown

Higher-Order Flow Analyses

J. Ian Johnson, David Van Horn and Olin Shivers{ianj,dvanhorn,shivers}@ccs.neu.edu

Northeastern UniversityBoston, MA, USA

1

Flow analysis is indispensible

2

Flow analysis is indispensible

Every language should have one

3

Analysis is great

• Compiler optimizations

4

Analysis is great


• Type reconstruction

5

Analysis is great



• Deadlock detection

6

Analysis is great




• Data race detection

7

Analysis is great





• Code quality enforcement

8

Analysis is great





• Code quality enforcement

• Hundreds more

9

Analysis is terrible

• Annoying false positives

10



• Takes too long

11



• Takes too long

• Hard to implement

12



• Takes too long


• Hard to show correct

13



• Takes too long



• The good ones inherently first-order

14



• Takes too long




We can fix this

15



• Takes too long




We can fix thisFirst some history

16

Flow Graphs vs. Programs

First-order

(Hecht 77) CFG + lattice

Higher-order

17


First-order


(Sharir & Pnueli 81) CFG + lattice + summaries

Higher-order

(Jones & Muchnick 82) Program + set constraints

18


First-order



(Reps 95) PDS + idempotent semiring

Higher-order


(Shivers 91) CPS + set constraints

19


First-order




Higher-order



(Might 07) CPS + abstract small step semantics

20


First-order




Higher-order




(Might & Van Horn 10) Derived from concrete

21


First-order




Higher-order





(Vardoulakis & Shivers 10) CPS + summaries

(Earl et al. 10) ANF + Dyck state graph

22


First-order




Higher-order







Graphs are a bad abstraction

23


First-order




Higher-order







Graphs are a bad abstraction

24


First-order




Higher-order







25


First-order




Higher-order







?

26

Analyses are derivable [Might & Van Horn 2010]

• Start: concrete machine semantics

27



• Simple transforms: put semantics in right form

28



• Simple transforms: put semantics in right form

• Analysis: pointwise-abstraction

29

Functions Without Stack

(define (sqr x) (* x x))

(sqrt (+ (sqr y) (sqr z)))

30




31




0CFA: Not even terminating!

32

Pushdown Higher-Order Flow Analysis

33


34


35


36


37


38


39


40


41

Analysis is terrible salvagable


• Takes too long




42

Deriving Pushdown Analyses

• Start: Concrete machine semantics

43



• Simple transform: memoize functions

44




• Simple transform: store functions’ calling contexts

45





• Simple transform: heap-allocate data

46







47







Gives semantic account of summaries

48



• Takes too long




49

(app id 1)

Memo

Contexts

Environment:id ...app ...

(let* ([id (λ (x) x)][app (λ (f y) (f y))][n1 (app id 1)][n2 (app id 2)])

(+ n1 n2))

50

(f y)

Memo

Contexts⟨app id 1⟩ (let* (... [n1 •] ...) ...)

Environment:id ...f idy 1


(+ n1 n2))

51

x

Memo


⟨id 1⟩ (let* (... [app (λ (f y) •)] ...) ...)

Environment:x 1


(+ n1 n2))

52

x

Memo⟨id 1⟩ 1


⟨id 1⟩ (let* (... [app (λ (f y) •)] ...) ...)

Environment:(let* ([id (λ (x) x)][app (λ (f y) (f y))][n1 (app id 1)][n2 (app id 2)])

(+ n1 n2))

53

(f y)

Memo⟨id 1⟩ 1

⟨app id 1⟩ 1


⟨id 1⟩ (let* (... [app (λ (f y) •)] ...) ...)


(+ n1 n2))

54

(app id 2)

Memo⟨id 1⟩ 1

⟨app id 1⟩ 1


⟨id 1⟩ (let* (... [app (λ (f y) •)] ...) ...)

Environment:id ...app ...n1 1


(+ n1 n2))

55

(f y)

Memo⟨id 1⟩ 1

⟨app id 1⟩ 1


⟨id 1⟩ (let* (... [app (λ (f y) •)] ...) ...)

⟨app id 2⟩ (let* (... [n2 •]) ...)

Environment:id ...f idy 2


(+ n1 n2))

56

x

Memo⟨id 1⟩ 1

⟨app id 1⟩ 1


⟨id 1⟩ (let* (... [app (λ (f y) •)] ...) ...)

⟨app id 2⟩ (let* (... [n2 •]) ...)

⟨id 2⟩ (let* (... [app (λ (f y) •)] ...) ...)

Environment:x 2


(+ n1 n2))

57

x

Memo⟨id 1⟩ 1

⟨app id 1⟩ 1

⟨id 2⟩ 2


⟨id 1⟩ (let* (... [app (λ (f y) •)] ...) ...)

⟨app id 2⟩ (let* (... [n2 •]) ...)

⟨id 2⟩ (let* (... [app (λ (f y) •)] ...) ...)


(+ n1 n2))

58

(f y)

Memo⟨id 1⟩ 1

⟨app id 1⟩ 1

⟨id 2⟩ 2

⟨app id 2⟩ 2


⟨id 1⟩ (let* (... [app (λ (f y) •)] ...) ...)

⟨app id 2⟩ (let* (... [n2 •]) ...)

⟨id 2⟩ (let* (... [app (λ (f y) •)] ...) ...)


(+ n1 n2))

59

(+ n1 n2)

Memo⟨id 1⟩ 1

⟨app id 1⟩ 1

⟨id 2⟩ 2

⟨app id 2⟩ 2


⟨id 1⟩ (let* (... [app (λ (f y) •)] ...) ...)

⟨app id 2⟩ (let* (... [n2 •]) ...)

⟨id 2⟩ (let* (... [app (λ (f y) •)] ...) ...)

Environment:id ...app ...n1 1n2 2


(+ n1 n2))

60

(app id 1)

Memo

Contexts

Store: σ₀

Start Store: σ₀


(+ n1 n2))

61

(f y)

Memo

Contexts⟨app σ₁ ⟩ ((let* (... [n1 •] ...) ...) σ₀)

Store: σ₁

f id

y 1

Start Store: σ₁


(+ n1 n2))

62

x

Memo


⟨id σ₂ ⟩ ((let* (... [app (λ (f y) •)] ...) ...) σ₁)

Store: σ₂

f id

y 1

x 1

Start Store: σ₂


(+ n1 n2))

63

x

Memo⟨id σ₂⟩ (1 σ₂)


⟨id σ₂ ⟩ ((let* (... [app (λ (f y) •)] ...) ...) σ₁)

Store: σ₂

f id

y 1

x 1

Start Store: σ₂


(+ n1 n2))

64

(f y)


⟨app σ₁⟩ (1 σ₂)


⟨id σ₂ ⟩ ((let* (... [app (λ (f y) •)] ...) ...) σ₁)

Store: σ₂

f id

y 1

x 1

Start Store: σ₁


(+ n1 n2))

65

(app id 2)


⟨app σ₁⟩ (1 σ₂)


⟨id σ₂ ⟩ ((let* (... [app (λ (f y) •)] ...) ...) σ₁)

Store: σ₃

f id

y 1

x 1

n1 1

Start Store: σ₀


(+ n1 n2))

66

(f y)


⟨app σ₁⟩ (1 σ₂)


⟨id σ₂ ⟩ ((let* (... [app (λ (f y) •)] ...) ...) σ₁)

⟨app σ₄ ⟩ ((let* (... [n2 •]) ...) σ₀)

Store: σ₄

f id

y (1 2)

x 1

n1 1

Start Store: σ₄


(+ n1 n2))

67

x


⟨app σ₁⟩ (1 σ₂)


⟨id σ₂ ⟩ ((let* (... [app (λ (f y) •)] ...) ...) σ₁)

⟨app σ₄ ⟩ ((let* (... [n2 •]) ...) σ₀)

⟨id σ₅ ⟩ ((let* (... [app (λ (f y) •)] ...) ...) σ₄)

Store: σ₅

f id

y (1 2)

x (1 2)

n1 1

Start Store: σ₅


(+ n1 n2))

68

x


⟨app σ₁⟩ (1 σ₂)

⟨id σ₅⟩ (2 σ₅)


⟨id σ₂ ⟩ ((let* (... [app (λ (f y) •)] ...) ...) σ₁)

⟨app σ₄ ⟩ ((let* (... [n2 •]) ...) σ₀)

⟨id σ₅ ⟩ ((let* (... [app (λ (f y) •)] ...) ...) σ₄)

Store: σ₅

f id

y (1 2)

x (1 2)

n1 1

Start Store: σ₅


(+ n1 n2))

69

(f y)


⟨app σ₁⟩ (1 σ₂)

⟨id σ₅⟩ (2 σ₅)

⟨app σ₄⟩ (2 σ₅)


⟨id σ₂ ⟩ ((let* (... [app (λ (f y) •)] ...) ...) σ₁)

⟨app σ₄ ⟩ ((let* (... [n2 •]) ...) σ₀)

⟨id σ₅ ⟩ ((let* (... [app (λ (f y) •)] ...) ...) σ₄)

Store: σ₅

f id

y (1 2)

x (1 2)

n1 1

Start Store: σ₄


(+ n1 n2))

70

(+ n1 n2)


⟨app σ₁⟩ (1 σ₂)

⟨id σ₅⟩ (2 σ₅)

⟨app σ₄⟩ (2 σ₅)


⟨id σ₂ ⟩ ((let* (... [app (λ (f y) •)] ...) ...) σ₁)

⟨app σ₄ ⟩ ((let* (... [n2 •]) ...) σ₀)

⟨id σ₅ ⟩ ((let* (... [app (λ (f y) •)] ...) ...) σ₄)

Store: σ₆

f id

y (1 2)

x (1 2)

n1 1

n2 (1 2)

Start Store: σ₀


(+ n1 n2))

71

Heaps are imprecise

(+ n1 n2)


⟨app σ₁⟩ (1 σ₂)

⟨id σ₅⟩ (2 σ₅)

⟨app σ₄⟩ (2 σ₅)


⟨id σ₂ ⟩ ((let* (... [app (λ (f y) •)] ...) ...) σ₁)

⟨app σ₄ ⟩ ((let* (... [n2 •]) ...) σ₀)

⟨id σ₅ ⟩ ((let* (... [app (λ (f y) •)] ...) ...) σ₄)

Store: σ₆

f id

y (1 2)

x (1 2)

n1 1

n2 (1 2)

Start Store: σ₀


(+ n1 n2))

72

Fixed[Vardoulakis & Shivers 10]

Local storage (stack frames)

73

(define (foldr f b lst) (letrec

([loop(λ (lst)(cond [(empty? lst) b]

[else (cons (f (car lst))(loop (cdr lst)))]))])

(loop lst)))

74




(loop lst)))

75




(loop lst)))

76

Fixed

Instrument your escape analysis

77

Escape Analysis

Which addresses outlive their creator’s context?

78

Escape Analysis


We introduce three components:

• Escaped addresses ℰ

• Addresses owned by current function 𝒪

• The random access stack Ξ

79

Escape Analysis


We introduce three components:

• Escaped addresses ℰ

• Addresses owned by current function 𝒪

• The random access stack Ξ

Lookup is now defined with

ℒ(a,σ,Ξ,ℰ) = a ∈ ℰ → σ(a), Ξ(a)

80

⟨(x ,ρ),σ,Ξ,κ⟩ ⟼ ⟨v,σ,Ξ,κ⟩

where v ∈ ℒ(ρ(x) ,σ ,Ξ ,ℰ)

81

⟨(x ,ρ),σ,Ξ,κ⟩ ⟼ ⟨v,σ,Ξ,κ⟩


⟨v,σ,Ξ,fn((λ x.e ,ρ),κ)⟩ ⟼ ⟨c,σnext,Ξ[a ↦ v],[]c⟩

where c = (e ,ρ[x ↦ a])

σnext = σ[a ↦ v]

ℰ' = ℰ∩ ℛ(𝒯(c) ,σnext)

82

⟨(x ,ρ),σ,Ξ,κ⟩ ⟼ ⟨v,σ,Ξ,κ⟩






𝒪 ' = {a}

83

⟨(x ,ρ),σ,Ξ,κ⟩ ⟼ ⟨v,σ,Ξ,κ⟩






𝒪 ' = {a}

L' = L⊔[(c ,σnext) ↦ (κ ,σcur ,Ξ ,ℰ ,𝒪)]

84

⟨v,σ,Ξ,[]c⟩ ⟼ ⟨v,σ,Ξ',κ⟩

if (κ ,σnext ,Ξ' ,ℰ' ,𝒪) ' ∈ L(c ,σcur)

M' = M[(c ,σcur) ↦ (v ,σ)]

ℰ'' = ℰ' ∪ ((ℰ ∪ 𝒪) ∩ ℛ(𝒯(v),σ))

85



• Takes too long




86

We're not done yet

87

Too context-sensitive


⟨app σ₁⟩ (1 σ₂)

⟨id σ₅⟩ (2 σ₅)

⟨app σ₄⟩ (2 σ₅)


⟨id σ₂ ⟩ ((let* (... [app (λ (f y) •)] ...) ...) σ₁)

⟨app σ₄ ⟩ ((let* (... [n2 •]) ...) σ₀)

⟨id σ₅ ⟩ ((let* (... [app (λ (f y) •)] ...) ...) σ₄)

Store: σ₆

f id

y (1 2)

x (1 2)

n1 1

n2 (1 2)

Start Store: σ₀


(+ n1 n2))

88

First-Order Influence

First-order




Higher-order







Today's talk

?

89

First-Order Influence

First-order




(Oh et al. 12) CFG + lattice + bypassing

Higher-order







Today's talk

?

90

First-order




(Oh et al. 12) CFG + lattice + bypassing

Higher-order







Today's talk

?

91

To Conclude

• Design: Model abstract mechanisms concretely

92

To Conclude


• Pushdown: Memo and local continuation tables

93

To Conclude



• Precise analyses: Stack allocation ⟹ precision

94

To Conclude




• (Not shown) works for control operators / GC

95

To Conclude





• Future work: sparse analysis via time travel

96

To Conclude





• Future work: sparse analysis via time travel

Thank you

97

designing precise pushdown higher-order flow analysesdeeglaze.github.io/ibmpl2012-slides.pdf · 2....

Documents