intraprocedural optimizations

Intraprocedural Optimizations

Jonathan BachrachMIT AI Lab

Outline• Goal: eliminate abstraction overhead using static analysis and

program transformation• Topics:

– Intraprocedural type inference– Static method selection– Specialization and Inlining– Static class prediction– Splitting– Box/unboxing– Common Subexpression Elimination– Overflow and range checks– Partial evaluation revisited

• Partially based on: Chambers’ “Efficient Implementation of Object-oriented Programming Languages” OOPSLA Tutorial

Running Example(dg + ((x <num>) (y <num>) => <num>))

(dm + ((x <int>) (y <int>) => <int>) (%ib (%i+ (%iu x) (%iu y)))

(dm + ((x <flo>) (y <flo>) => <flo>) (%fb (%f+ (%fu x) (%fu y)))

(dm x2 ((x <num>) => <num>) (+ x x))(dm x2 ((x <int>) => <int>) (+ x x))

• Anatomy of Pure Proto Arithmetic– Dispatch– Boxing– Overflow checks– Actual instruction

• C Arithmetic– Actual instruction

Biggest Inefficiencies

• Method dispatch• Method calls• Boxing• Type checks • Overflow and range checks• Slot access• Object creation

Intraprocedural Type Inference

• Goal: determine concrete class(es) of each variable and expression

• Standard data flow analysis through control graph– Propagate bindings b -> { class … } – Sources are literals, isa expressions, results of some

primitives, and type declarations– Form unions of bindings at merge points– Narrow sets after typecases– Assumes closed world (or at least final classes)

Type Inference Example(set x (isa <tab> …)) ;; x in { <tab> }(set y (table-growth-factor x)) ;; y in { <int> <flo> }(set z (if t x y)) ;; z in { <tab> <int>

<flo> }

Narrowing Type Precision(if (isa? x <int>) (+ x 1) (+ x 37.0))

(if (isa? x <int>) (let (([x <int>] x)) (+ x 1)) (let (([x !<int>] x)) (+ x 37.0)))

Static Method Selection(set x (isa <tab> …)) ;; x in { <tab> }(set y (table-growth-factor x)) ;; y in { <int> <flo> }(print out y)

• If only one class is statically possible then can perform dispatch statically:(set y (<tab>:table-growth-factor x))

• If a couple classes are statically possible then can insert typecase:(sel (class-of y) ((<int>) (<int>:print y)) ((<flo>) (<flo>:print y)))

Type Check Removal

• Type inference can clearly be used to remove type checks and casts

(set x (isa <tab> …)) ;; x in { <tab> }(if (isa? x <tab>) (go) (stop))==>(set x (isa <tab> …)) ;; x in { <tab> }(go)

Intraprocedural Type Inference Critique

• Pros: – Simple– Fast – Fewer dependents

• Cons: – Limited type precision

• No result types• Incoming arg types• No slot types• Etc.

Specialization

• Q: How can we improve intraprocedural type inference precision?

• A: Specialization which is the cloning of methods with narrowed argument types

• Improves type precision of callee by contextualizing body:(dm sqr ((x <num>) (y <num>)) (* x y))==>(dm sqr ((x <int>) (y <int>)) (* x y))(dm sqr ((x <flo>) (y <flo>)) (* x y))

• Must make sure super calls still mean same thing

Specialization of Constructors

• Crucial to get object creation to be fast• Specialization can be used to build custom

constructors(def <thingy> (isa <any>)) (slot <thingy> thingy-x 0) (slot (t <thingy>) thingy-tracker (+ (thingy-x t) 1)) (slot <thingy> thingy-cache (fab <tab>))

(df thingy-isa (x tracker cache) (let ((thingy (clone <thingy>))) (unless (== x nul) (set (%slot-value thingy thingy-x) x)) (set (%slot-value thingy thingy-tracker) (if (== tracker nul) (+ (thingy-x p) 1) tracker)))) (set (%slot-value thingy thingy-cache) (if (== cache nul) (fab <tab>) cache))))

Inlining

• Q: Can we do better?• A: Inlining can improve specialization by

inserting specialized body• Improves type precision at call-site by

contextualizing body (includes result types):(dm f ((x <int>) (y <int>)) (+ (g x y) 1))(dm g (x y) (+ x y))==>(dm f ((x <int>) (y <int>)) (+ (+ x y) 1))

Synergy: Method Selection + Inlining

(df f ((x <int>) (y <int>)) (+ x y))

;; method selection(df f ((x <int>) (y <int>)) (<int>:+ x y))

;; inlining(df f ((x <int>) (y <int>)) (%ib (%i+ (%iu x) (%iu y))))

Pitfalls of Inlining and Specialization

• Must control inlining and specialization carefully to avoid code bloat

• Inlining can work merely using syntactic size trying never to increase size over original call

• Class-centric specialization usually works by copying down inherited methods tightening up self references (harder for multimethods)

• Can run inlining/specialization trials based on– Final static size– Performance feedback

Class Centric Specialization(def <point> (isa <any>)) (slot <point> (point-x <int>) 0)(dm point-move ((p <point>) (offset <num>)) (set (point-x p) (+ (point-x p) offset)))(def <color-point> (isa <point>))

==>

(dm point-move ((p <color-point>) (offset <num>)) (set (point-x p) (+ (point-x p) offset)))

Static Class Prediction

• Can improve type precision in cases where for a given generic a particular method is much more frequent

• Insert type check testing prediction– Can narrow type precision along then and else

branches• Especially useful in combination with

inlining

Static Class Prediction Example(df f (x) (let ((y (+ x 1))) (+ y 2)))

(df f (x) (let ((y (if (isa? x <int>) (+ x 1) (+ x 1)))) (if (isa? y <int>) (+ y 2) (+ y 2)))))

(df f (x) (let ((y (if (isa? x <int>) (<int>:+ x 1) (+ x 1)))) (if (isa? y <int>) (<int>:+ y 2) (+ y 2)))))

Synergy: Class Prediction + Method Selection + Inlining

(df f (x) (let ((y (if (isa? x <int>) (+ x 1) (+ x 1)))) (if (isa? y <int>) (+ y 2) (+ y 2)))))

;; method selection(df f (x) (let ((y (if (isa? x <int>) (<int>:+ x 1) (+ x 1)))) (if (isa? y <int>) (<int>:+ y 2) (+ y 2)))))

;; inlining(df f (x) (let ((y (if (isa? x <int>) (%ib (%i+ (%iu x) %1)) (+ x 1)))) (if (isa? y <int>) (%ib (%i+ (%iu y) (%iu 2))) (+ y 2)))))

Splitting

• Problem: Class prediction often leads to a bunch of redundant type tests

• Solution: Split off whole sections of graph specialized to particular class on variable– Can split off entire loops– Can specialize on other dataflow information

Splitting Example(df f (x) (let ((y (+ x 1))) (+ y 2)))

(df f (x) (if (isa? x <int>) (let ((y (+ x 1))) (+ y 2)) (let ((y (+ x 1))) (+ y 2))))

(df f (x) (if (isa? x <int>) (let ((y (<int>:+ x 1))) (<int>:+ y 2)) (let ((y (+ x 1))) (+ y 2))))

Splitting Downside

• Splitting can also lead to code bloat• Must be intelligent about what to split

– A priori knowledge (e.g., integers most frequent)

– Actual performance

Box / Unboxing(df + ((x <int>) (y <int>) => <int>) (%ib (%i+ (%iu x) (%iu y))))

(df f ((a <int>) (b <int>) => <int>) (+ (+ a b) a))

;; inlining +

(df f ((a <int>) (b <int>) => <int>) (%ib (%i+ (%iu (%ib (%i+ (%iu a) (%iu b)))) (%iu a))))

;; remove box/unbox pair

(df f ((a <int>) (b <int>) => <int>) (%ib (%i+ (%i+ (%iu a) (%iu b)) (%iu a))))

Synergy: Splitting + Method Selection + Inlining + Box/Unboxing

(df f (x) (if (isa? x <int>) (let ((y (+ x 1))) (+ y 2)) (let ((y (+ x 1))) (+ y 2))))

;; method selection(df f (x) (if (isa? x <int>) (let ((y (<int>:+ x 1))) (<int>:+ y 2)) (let ((y (+ x 1))) (+ y 2))))

(df f (x) (if (isa? x <int>) (<int>:+ (<int>:+ x 1) 2) (let ((y (+ x 1))) (+ y 2))));; inlining(df f (x) (if (isa? x <int>) (%ib (i+ (%iu (%ib (%i+ (%iu x) %1)))) %2)) (let ((y (+ x 1))) (+ y 2))));; box/unbox(df f (x) (if (isa? x <int>) (%ib (%i+ (%i+ (%iu x) %1)) %2)) (let ((y (+ x 1))) (+ y 2))))

Common Subexpression Elimination (CSE)

• Removes redundant computations– Constant slot or binding access– Stateless/side-effect-free function calls

• Examples(or (elt (cache x) ‘a) (elt (cache x) ‘b)) ==> (let ((t (cache x))) (or (elt t ‘a) (elt t ‘b))

(if (< i 0) (if (< i 0) (go) (putz)) (dance)) ==> (if (< i 0) (go) (dance))

Overflow and Bounds Checksaka “Moon Challenge”

• Goal: – Support mathematical integers and bounds checked collection

access– Eliminate bounds and overflow checks

• Strategy:– Assume most integer arithmetic and collection accesses occur

in restricted loop context where range can be readily inferred– Perform range analysis to remove checks

• Bound from above variables by size of collection• Bound from below variables by zero• Induction step is 1+

Range Check Example(rep (((sum <int>) 0) ((i <int>) 0)) (if (< i (len v)) (let ((e (elt v i))) (rep (+ sum e) (+ i 1))) sum))

;; inlining bounds checks(rep (((sum <int>) 0) ((i <int>) 0)) (if (< i (len v)) (let ((e (if (or (< i 0) (>= i (len v))) (sig ...) (vref v i)))) (rep (+ sum e) (+ i 1))) sum))

;; CSE(rep (((sum <int>) 0) ((i <int>) 0)) (if (< i (len v)) (let ((e (if (< i 0) (sig ...) (vref v i)))) (rep (+ sum e) (+ i 1))) sum))

;; range analysis(rep (((sum <int>) 0) ((i <int>) 0)) (if (< i (len v)) (let ((e (vref v i))) (rep (+ sum e) (+ i 1))) sum))

Overflow Check Removal aka “Moon Challenge” Critique

• Pros: – simple analysis

• Cons: – could miss a number of cases

• but then previous approaches (e.g., box/unbox) could be applied

Advanced topic:Representation Selection

• Embed objects in others to remove indirections

• Change object representation over time• Use minimum number of bits to represent

enums• Pack fields in objects

Advanced Topic:Algorithm Selection

• Goal: compiler determines that one algorithm is more appropriate for given data– Sorted data– Biased data

• Solution: – Embed statistics gathering in runtime– Add guards to code and split

Rule-based Compilation• First millennium compilers were based on special rules

for– Method selection– Pattern matching– Oft-used system functions like format

• Problems– Error prone– Don’t generalize to user code

• Challenge– Minimize number of rules– Competitive compiler speed– Produce competitive code

Partial Evaluation to the Rescue

• Holy grail idea:– Optimizations are manifest in code– Do previous optimizations with only p.e.

• Simplify compiler based on limited moves– Static eval and folding– Inlining

• Eliminate– Custom method selection– Custom constructor optimization– Etc.

Partial Eval Example(dm format (port msg (args …)) (rep nxt ((I 0) (ai 0)) (when (< I (len msg))) (let ((c (elt msg I))) (if (= c #\%) (seq (print port (elt args ai)) (nxt (+ I 1) (+ ai 1)))) (seq (write port c) (nxt (+ I 1) ai)))))))

(format out “%>? ” n)

• First millennium solution is to have a custom optimizer for format

(seq (print port n) (write port “> “))

• Second millennium solution with partial evaluation

(nxt 0 0)

(seq (print port n) (nxt 1 1))

(seq (print port n) (seq (write port #\>) (nxt 2 1)))

(seq (print port n) (seq (write port #\>) (seq (write port #\space))))

Partial Eval Challenge

• Inlining and static eval are slow– “Running” code through inlining– Need to compile oft-used optimizations

• Residual code is not necessarily efficient– Sometimes algorithmic change is necessary for optimal

efficiency• Example: method selection uses class numbering and decision

tree whereas straightforward code does naïve method sorting • Perhaps there is a middle ground

Open Problems

• Automatic inlining, splitting, and specialization• Efficient mathematical integers• Constant determination• Representation selection• Algorithmic selection• Efficient partial evaluation• Super compiler that runs for days

Reading List

• Chambers: “Efficient Implementation of Object-oriented Programming Languages” OOPSLA Tutorial

• Chambers and Ungar: SELF papers• Chambers et al.: Vortex papers