LECTURE 10:JUST-IN-TIME COMPILATION &PEEPHOLE OPTIMIZATIONS
CSC 313 – Advanced Programming Topics
Problem We Face
Want to optimize code, but face problems:
Programmers are stupid
Code awful: often ignores simple optimizations
Methods merge cases that optimized differently
Cannot tell system about likely values or cases
Problem We Face
Want to optimize code, but face problems:
Programmers are stupid
Code awful: often ignores simple optimizations
Methods merge cases that optimized differently
Cannot tell system about likely values or cases
Wait, why does this matter?
Data Centers & Server Farms Major expense for
companies Akamai at 95,000 servers
(12/2011) Dual core/dual processor in
each Each one had 16GB RAM 250+ TB of storage total in
system Powering all of this demanded
53% of Niagara Falls output
Data Centers & Server Farms At least 25 data centers for Google Centers held ~900,000 servers
Greater diversity in machines Generally < 2 years old 4 – 8 GB RAM each
MS had set goal of 800,000 servers (2011)
Optimization Matters
Lots of money at stake Estimates assume server counts double in
18 months At least 3 more centers being build by
Google Each center costs $450 - $600 million
1% improvement is huge for these companies For Google that is 4,500+ servers $72 million saved in power worldwide
(2005)
Traditional Approach
Entire program compiled and optimized once Target single processor (e.g., Xeon,
Power) Creates static machine language
executable Repeatedly analyze code during
optimization After each step reanalyze new result to
improve Does anything to get last 1%
improvement
Compilation Is Slow
Optimizer Limited
Compiler must examine code as it is written Cannot take advantage of many situationswhile (true) { String line = scanner.readLine(); int i; if (line.compareTo(“EXIT”) != 0) { i = 1; } else { i = 0; } // What is the value of i?
JIT to the Rescue
Add Just-In-Time compiler (JIT) As it runs, system will spy on methods Look for patterns that can yield
optimization When good pattern found, recompile
JIT to the Rescue
Add Just-In-Time compiler (JIT) As it runs, system will spy on methods Look for patterns that can yield
optimization When good pattern found, recompileExecution Time
Original 1.00
JIT to the Rescue
Add Just-In-Time compiler (JIT) As it runs, system will spy on methods Look for patterns that can yield
optimization When good pattern found, recompileExecution Time
Original 1.00 W/Speedup - 0.20
JIT to the Rescue
Add Just-In-Time compiler (JIT) As it runs, system will spy on methods Look for patterns that can yield
optimization When good pattern found, recompileExecution Time
Original 1.00 W/Speedup - 0.20
Total 1.50
JIT to the Rescue
Add Just-In-Time compiler (JIT) As it runs, system will spy on methods Look for patterns that can yield
optimization When good pattern found, recompileExecution Time
Original 1.00 W/Speedup - 0.20
Total 1.50
What Happened? (1)
Program must examine methods as they run
What Happened? (1)
Program must examine methods as they run
What Happened? (1)
Program must examine methods as they run Cannot just execute on computer at top
speed
What Happened? (1)
Program must examine methods as they run Cannot just execute on computer at top
speed
What Happened? (1)
Program must examine methods as they run Cannot just execute on computer at top
speed Will need to evaluate & interpret
instructions
What Happened? (1)
Program must examine methods as they run Cannot just execute on computer at top
speed Will need to evaluate & interpret
instructions
What Happened? (1)
Program must examine methods as they run Cannot just execute on computer at top
speed Will need to evaluate & interpret
instructions 10 times slower to interpret instructions
What Happened? (1)
Program must examine methods as they run Cannot just execute on computer at top
speed Will need to evaluate & interpret
instructions 10 times slower to interpret instructions
What Happened? (1)
Program must examine methods as they run Cannot just execute on computer at top
speed Will need to evaluate & interpret
instructions 10 times slower to interpret instructions
Analyze results to see if optimization found
What Happened? (2)
What Happened? (2)
Compiling done once, if we were not using JIT Will take forever (~10 minutes in coder
speak) Reducing even 1% still more than
employees salary As program runs, JIT compiles &
recompiles Each recompilation better & faster to run
program But compilation now added to total
execution time
What Went WrongYou're bound to be unhappy if you optimize everything.
What Went WrongYou're bound to be unhappy if you optimize everything.
Execution TimeOriginal 1.00 W/Overhead 0.70 W/Speedup - 0.20
Total 1.50
Can We Reduce Costs?
Not worth doing this for certain methods main() String.offsetByCodePoints() Most constructors Anything that your partner wrote
Ignore cold code (code not run often/long) Code not run enough to justify JIT costs Code not important enough to justify JIT
costs
Some Like It Hot
JIT examines only hottest methods Executed most often or for the longest
time Hot methods are where program's
time spent Need data to analyze; only these provide
it Also these methods only where impact
felt When profitable, recompile hot
methods So the JIT will do this only if costs < time
saved
Should We Recompile?
When we need to make this decision… Time to compile code must be known Need to find out how often method run Improvement percentage also needed Naturally, there is no way to know any of
this What can we do?
Do best, hope & pray that future resembles past
Make wild- scientific guess about how to do
Does It Work?
Dynamo runs programs 10% faster 45,000+ servers would not be needed by
Google Would have saved US $270 million in 2005
power bills
Does It Work?
Dynamo runs programs 10% faster 45,000+ servers would not be needed by
Google Would have saved US $270 million in 2005
power bills
Execution TimeOriginal 1.00 W/Overhead 0.10 W/Speedup - 0.20
Total 0.90
What Does It All Mean?
Optimizations can make a huge difference Without our knowledge this will already
be done Always check "optimizations" do no
harm Compilers optimize a lot, but only if can
prove safe Know compiler details if you want
fast code After all, this is only way to write
optimizable code
Instances of Class Class
Objects representing classes used by program What fields is has & how fields laid out in
memory Starting address for methods to call them
in code This is structure (literally) extended by
subclasses Superclass code expects members to
not move Overriden method has starting address
changed Add fields to end to save older field
definition But, we must ask, what about
optimizations?
Normal Method Call
public class Super {private void foo() {
System.err.print(“Foo”);}public void bar() {
foo();System.err.println(“bar”);
}}Super s = new Super();s.bar();
Class Superfoo() StartAddress1bar() StartAddress2
Polymorphic Method Call
public class Super {private void foo() { … }public void bar() {
foo();System.err.println(“bar”);
}}public class Sub extends Super {
private void foo() {System.err.print(“Not foo”);
}}Super s = new Sub();s.bar()
Class Superfoo() StartAddress1bar() StartAddress2
Class Subfoo() StartAddress3bar() StartAddress2
What Is Done For Us
Good news! JIT compilers are very smart If class has subclasses, will be recorded
& used Examines what the types of instances
used are If one method definition can ever be
used… JIT compiler can skip step to look up
address If more possible, but only one
actually used… Slightly slower, but guess address & add
failpath SOL if many used, since lookup is
necessary
Help Compiler Help Us
Use final keyword to let compiler know: Specify that a variable keeps initial value Method not overriden & starting address
constant Compiler can do as much as possible w/o
checks!
Compiler Helping Us
With final compiler can do many things Getter & setter method calls become
direct access Addresses hardcoded & lookups
eliminated Use keyhole optimizations relying on
actual values
final Method Calls
public class Super {private final void foo() {
System.err.print(“Foo”);}public void bar() {
foo();System.err.println(“bar”);
}}Super s = new Super();s.bar();
final Field Uses
public class GoodField {private int LENGTH = 8;public long recombine(byte[] arr){
long sum = 0;for (int i = 0; i < LENGTH; i++){ sum = (sum * LENGTH) + arr[i];}return sum;
}}
final Field Uses
public class GoodField {private final int LENGTH = 8;public long recombine(byte[] arr){
long sum = 0;for (int i = 0; i < LENGTH; i++){ sum = (sum * LENGTH) + arr[i];}return sum;
}}
final Field Uses
public class GoodField {private final int LENGTH = 8;public long recombine(byte[] arr){
long sum = 0;for (int i = 0; i < 8; i++){ sum = (sum * 8) + arr[i];}return sum;
}}
final Field Uses
public class GoodField {private final int LENGTH = 8;public long recombine(byte[] arr){
long sum = 0;for (int i = 0; i < 8; i++){ sum = (sum << 3) + arr[i];}return sum;
}}
Rules of Optimization
NO!Not Yet
For experts only
Problems With final
Remember that final is a negative option It removes options and limits future
possibilities Code becomes static and its reuse may
be hurt Each savings is small (remember
Amdahl's Law) Use when ABSOLUTELY
NECESSARY ONLY
Problems With final
Remember that final is a negative option It removes options and limits future
possibilities Code becomes static and its reuse may
be hurt Each savings is small (remember
Amdahl's Law) Use when ABSOLUTELY
NECESSARY ONLY
Isolate code that changes from code that stays the same
For Next Lecture
Read pages 79 – 94 in the book What is the DECORATOR PATTERN? Can we learn design from coffee shops?
They are LOUSY coders who mess up great systems
What do the following have in common: Dijkstra’s algorithm Household finance program E-mail program