LECTURE 10:JUST-IN-TIME COMPILATION &PEEPHOLE OPTIMIZATIONS

CSC 313 – Advanced Programming Topics

Problem We Face

Want to optimize code, but face problems:

Programmers are stupid

Code awful: often ignores simple optimizations

Methods merge cases that optimized differently

Cannot tell system about likely values or cases

Problem We Face

Want to optimize code, but face problems:

Programmers are stupid

Code awful: often ignores simple optimizations

Methods merge cases that optimized differently

Cannot tell system about likely values or cases

Wait, why does this matter?

Data Centers & Server Farms Major expense for

companies Akamai at 95,000 servers

(12/2011) Dual core/dual processor in

each Each one had 16GB RAM 250+ TB of storage total in

system Powering all of this demanded

53% of Niagara Falls output

Data Centers & Server Farms At least 25 data centers for Google Centers held ~900,000 servers

Greater diversity in machines Generally < 2 years old 4 – 8 GB RAM each

MS had set goal of 800,000 servers (2011)

Optimization Matters

Lots of money at stake Estimates assume server counts double in

18 months At least 3 more centers being build by

Google Each center costs $450 - $600 million

1% improvement is huge for these companies For Google that is 4,500+ servers $72 million saved in power worldwide

(2005)

Traditional Approach

Entire program compiled and optimized once Target single processor (e.g., Xeon,

Power) Creates static machine language

executable Repeatedly analyze code during

optimization After each step reanalyze new result to

improve Does anything to get last 1%

improvement

Compilation Is Slow

Optimizer Limited

Compiler must examine code as it is written Cannot take advantage of many situationswhile (true) { String line = scanner.readLine(); int i; if (line.compareTo(“EXIT”) != 0) { i = 1; } else { i = 0; } // What is the value of i?

JIT to the Rescue

Add Just-In-Time compiler (JIT) As it runs, system will spy on methods Look for patterns that can yield

optimization When good pattern found, recompile

JIT to the Rescue

Add Just-In-Time compiler (JIT) As it runs, system will spy on methods Look for patterns that can yield

optimization When good pattern found, recompileExecution Time

Original 1.00

JIT to the Rescue

Add Just-In-Time compiler (JIT) As it runs, system will spy on methods Look for patterns that can yield

optimization When good pattern found, recompileExecution Time

Original 1.00 W/Speedup - 0.20

JIT to the Rescue

Add Just-In-Time compiler (JIT) As it runs, system will spy on methods Look for patterns that can yield

optimization When good pattern found, recompileExecution Time

Original 1.00 W/Speedup - 0.20

Total 1.50

JIT to the Rescue

Add Just-In-Time compiler (JIT) As it runs, system will spy on methods Look for patterns that can yield

optimization When good pattern found, recompileExecution Time

Original 1.00 W/Speedup - 0.20

Total 1.50

What Happened? (1)

Program must examine methods as they run

What Happened? (1)

Program must examine methods as they run

What Happened? (1)

Program must examine methods as they run Cannot just execute on computer at top

speed

What Happened? (1)

Program must examine methods as they run Cannot just execute on computer at top

speed

What Happened? (1)

Program must examine methods as they run Cannot just execute on computer at top

speed Will need to evaluate & interpret

instructions

What Happened? (1)

Program must examine methods as they run Cannot just execute on computer at top

speed Will need to evaluate & interpret

instructions

What Happened? (1)

Program must examine methods as they run Cannot just execute on computer at top

speed Will need to evaluate & interpret

instructions 10 times slower to interpret instructions

What Happened? (1)

Program must examine methods as they run Cannot just execute on computer at top

speed Will need to evaluate & interpret

instructions 10 times slower to interpret instructions

What Happened? (1)

Program must examine methods as they run Cannot just execute on computer at top

speed Will need to evaluate & interpret

instructions 10 times slower to interpret instructions

Analyze results to see if optimization found

What Happened? (2)

What Happened? (2)

Compiling done once, if we were not using JIT Will take forever (~10 minutes in coder

speak) Reducing even 1% still more than

employees salary As program runs, JIT compiles &

recompiles Each recompilation better & faster to run

program But compilation now added to total

execution time

What Went WrongYou're bound to be unhappy if you optimize everything.

What Went WrongYou're bound to be unhappy if you optimize everything.

Execution TimeOriginal 1.00 W/Overhead 0.70 W/Speedup - 0.20

Total 1.50

Can We Reduce Costs?

Not worth doing this for certain methods main() String.offsetByCodePoints() Most constructors Anything that your partner wrote

Ignore cold code (code not run often/long) Code not run enough to justify JIT costs Code not important enough to justify JIT

costs

Some Like It Hot

JIT examines only hottest methods Executed most often or for the longest

time Hot methods are where program's

time spent Need data to analyze; only these provide

it Also these methods only where impact

felt When profitable, recompile hot

methods So the JIT will do this only if costs < time

saved

Should We Recompile?

When we need to make this decision… Time to compile code must be known Need to find out how often method run Improvement percentage also needed Naturally, there is no way to know any of

this What can we do?

Do best, hope & pray that future resembles past

Make wild- scientific guess about how to do

Does It Work?

Dynamo runs programs 10% faster 45,000+ servers would not be needed by

Google Would have saved US $270 million in 2005

power bills

Does It Work?

Dynamo runs programs 10% faster 45,000+ servers would not be needed by

Google Would have saved US $270 million in 2005

power bills

Execution TimeOriginal 1.00 W/Overhead 0.10 W/Speedup - 0.20

Total 0.90

What Does It All Mean?

Optimizations can make a huge difference Without our knowledge this will already

be done Always check "optimizations" do no

harm Compilers optimize a lot, but only if can

prove safe Know compiler details if you want

fast code After all, this is only way to write

optimizable code

Instances of Class Class

Objects representing classes used by program What fields is has & how fields laid out in

memory Starting address for methods to call them

in code This is structure (literally) extended by

subclasses Superclass code expects members to

not move Overriden method has starting address

changed Add fields to end to save older field

definition But, we must ask, what about

optimizations?

Normal Method Call

public class Super {private void foo() {

System.err.print(“Foo”);}public void bar() {

foo();System.err.println(“bar”);

}}Super s = new Super();s.bar();

Class Superfoo() StartAddress1bar() StartAddress2

Polymorphic Method Call

public class Super {private void foo() { … }public void bar() {

foo();System.err.println(“bar”);

}}public class Sub extends Super {

private void foo() {System.err.print(“Not foo”);

}}Super s = new Sub();s.bar()

Class Superfoo() StartAddress1bar() StartAddress2

Class Subfoo() StartAddress3bar() StartAddress2

What Is Done For Us

Good news! JIT compilers are very smart If class has subclasses, will be recorded

& used Examines what the types of instances

used are If one method definition can ever be

used… JIT compiler can skip step to look up

address If more possible, but only one

actually used… Slightly slower, but guess address & add

failpath SOL if many used, since lookup is

necessary

Help Compiler Help Us

Use final keyword to let compiler know: Specify that a variable keeps initial value Method not overriden & starting address

constant Compiler can do as much as possible w/o

checks!

Compiler Helping Us

With final compiler can do many things Getter & setter method calls become

direct access Addresses hardcoded & lookups

eliminated Use keyhole optimizations relying on

actual values

final Method Calls

public class Super {private final void foo() {

System.err.print(“Foo”);}public void bar() {

foo();System.err.println(“bar”);

}}Super s = new Super();s.bar();

final Field Uses

public class GoodField {private int LENGTH = 8;public long recombine(byte[] arr){

long sum = 0;for (int i = 0; i < LENGTH; i++){ sum = (sum * LENGTH) + arr[i];}return sum;

}}

final Field Uses

public class GoodField {private final int LENGTH = 8;public long recombine(byte[] arr){

long sum = 0;for (int i = 0; i < LENGTH; i++){ sum = (sum * LENGTH) + arr[i];}return sum;

}}

final Field Uses

public class GoodField {private final int LENGTH = 8;public long recombine(byte[] arr){

long sum = 0;for (int i = 0; i < 8; i++){ sum = (sum * 8) + arr[i];}return sum;

}}

final Field Uses

public class GoodField {private final int LENGTH = 8;public long recombine(byte[] arr){

long sum = 0;for (int i = 0; i < 8; i++){ sum = (sum << 3) + arr[i];}return sum;

}}

Rules of Optimization

NO!Not Yet

For experts only

Problems With final

Remember that final is a negative option It removes options and limits future

possibilities Code becomes static and its reuse may

be hurt Each savings is small (remember

Amdahl's Law) Use when ABSOLUTELY

NECESSARY ONLY

Problems With final

Remember that final is a negative option It removes options and limits future

possibilities Code becomes static and its reuse may

be hurt Each savings is small (remember

Amdahl's Law) Use when ABSOLUTELY

NECESSARY ONLY

Isolate code that changes from code that stays the same

For Next Lecture

Read pages 79 – 94 in the book What is the DECORATOR PATTERN? Can we learn design from coffee shops?

They are LOUSY coders who mess up great systems

What do the following have in common: Dijkstra’s algorithm Household finance program E-mail program