1

Chapter 4Assessing and Understanding Performance

2

• Measure, Report, and Summarize

• Make intelligent choices

• Key to understanding underlying organizational motivation

Why is some hardware better than others for different programs?

What factors of system performance are hardware related?(e.g., Do we need a new machine, or a new operating system?)

How does the machine's instruction set affect performance?

4.1 Introduction

3

Which of these airplanes has the best performance?

Airplane Passengers Range (mi) Speed (mph) pass * speed

Boeing 737-100 101 630598 60398Boeing 747 470 4150610 286700BAC/Sud Concorde 132 40001350 178200Douglas DC-8-50 146 8720544 79424

•How much faster is the Concorde compared to the 747?

•How much bigger is the 747 than the Douglas DC-8?

Defining Performance

4

Computer Performance: TIME, TIME, TIME

• Response Time (latency) also referred as execution time

— How long does it take for my job to run?

— How long does it take to execute a job?

— How long must I wait for the database query?

• Throughput

— How many jobs can the machine run at once?

— What is the average execution rate?

— How much work is getting done?

Question?

• If we upgrade a machine with a new processor what do we increase?

• If we add a new machine to the lab what do we increase?

Continue

5

Continue

• To maximize performance minimize response/execution time:

XX timeExecution

1ePerformanc

XY

YX

YX

timeExecution timeExecution

timeExecution

1

timeExecution

1

ePerformancPerformace

nY

X

ePerformanc

ePerformanc

• We will use phrase: “X is n times faster than Y” to mean:

• If X is n times faster than Y, then execution of Y is n longer than X:

nX

Y

Y

X

timeExecution

timeExecution

ePerformanc

ePerformanc

• If the performance of computer X is greater than Y, we have:

6

Example: Relative Performance

Computer A runs a program in 10 seconds and B runs it in 15 seconds, how much faster is A than B?

--------------------------------------------

A is n times faster than B if:

nA

B

B

A

timeExecution

timeExecution

ePerformanc

ePerformanc

Thus the performance ratio is:

and A is therefore 1.5 times faster than B

5.110

15

We could also say that computer B is 1.5 times slower than computer A.

7

Example: Relative Performance

Computer C’s performance is 4 times better than the performance of B, which runs a given application in 28 seconds. How long will computer C take to run that application?----------------------------------------

74

28

428

timeExecution

timeExecution 4

ePerformanc

ePerformanc

C

B

B

C

x

x

8

Measuring Performance

• Response time, or Elapsed time

– counts everything (disk and memory accesses, I/O , etc.)

– a useful number, but often not good for comparison purposes

However, a processor may work on several programs simultaneously, in such cases, the system may try to optimize throughput rather than elapsed. Thus use CPU time instead of Elapsed time.

• CPU time– doesn't count I/O or time spent running other programs– can be broken up into system time, and user time

• Our focus: user CPU time – time spent executing the lines of code that are "in" our program

• How fast the hardware?– clock cycles– clock period

9

4.2 CPU Performance and Its Factors

CPU execution time = CPU Clock cycles Clock cycle time

for a program for a program

Clock cycle and clock rate are inverses, thus:

CPU execution time =

• Improve performance by reducing either the length of the clock cycle or the number of clock cycles.

• However, there is a trade-off between the number of clock cycles needed and the length of each cycle.

rateClock

program afor cyclesClock CPU

10

Example: Improving Performance

• Our favorite program runs in 10 seconds on computer A, which has a 4 GHz. clock. We are trying to help a computer designer build a new machine B, that will run this program in 6 seconds. The designer can use new (or perhaps more expensive) technology to substantially increase the clock rate, but has informed us that this increase will affect the rest of the CPU design, causing machine B to require 1.2 times as many clock cycles as machine A for the same program. What clock rate should we tell the designer to target?"

11

Continue

GHz 8seconds

cycles 108

seconds 6

cycles 10401.2rateClock

rateClock

cycles 10401.2seconds 6

rateClock

cyclesclock CPU1.2 timeCPU

:equation thisusing found becan Bfor timeCPU

cycles1040second

cycles104seconds 10cyclesclock CPU

rateClock

cyclesclock CPU seconds 10

rateClock

cyclesclock CPU timeCPU

99

B

B

9

B

AB

99A

A

A

A

AA

12

Basic Performance Equation

CPU execution time = CPU Clock cycles Clock cycle time

for a program for a program

CPU Clock cycles = Instruction count (Ic) clock cycles per instruction (CPI)

CPU time = Ic CPI Clock cycle time

or

CPU time = rateClock

CPIIc

13

Example: Using the Performance Equation

We have two implementations:

Computer A: clock cycle time = 250 ps & CPI = 2.0

Computer B: clock cycle time = 500 ps & CPI = 1.2

Which one is faster, and by how much?

------------------------------------------------------------------------

For Computer A:

CPU timeA = CPU clock cyclesA Clock cycle timeA

= Ic 2.0 250 ps = 500 Ic ps

For Computer B:

CPU timeB = Ic 1.2 500 ps = 600 Ic ps

Clearly, computer A is faster, The amount faster is:

2.1I500

I600

timeExecution

timeExecution

eperformanc CPU

eperformanc CPU

c

c

A

B

B

A

14

Continue

• How can we determine the value of these factors:

CPU execution time, clock cycle time, Ic, and CPI

• It is possible to compute the CPU clock cycles by looking at the different class of instructions:

n

i 1ii ) C CPI ( cyclesclock CPU

15

Example: Comparing Code Segments

CPI for this instruction class

A B C

CPI 1 2 3

A compiler designer is trying to decide between two code sequences, given by the hardware designers the following facts:

Code sequenceInstructions counts for instruction class

A B C

1 2 1 2

2 4 1 1

The compiler writer is considering two code that require the following Ic:

Which code executes the most instruction? Which will be faster? What is CPI?

16

continue

Sequence 1 executes 2+1+2= 5 instructions, Sequence 2 execute 4+1+1= 6 instructions. So sequence 1 executes fewer instructions.

We can use the following equation to find the total number of clock cycles:

This yields

CPU clock cycles1 = (21)+(1 2)+(2 3)=2+2+6 = 10 cycles

CPU clock cycles2 = (41)+(1 2)+(1 3)=4+2+3 = 9 cycles

sequence 2 is faster.

The CPI values can be computed by:

n

i 1ii ) C CPI ( cyclesclock CPU

5.16

9

countn Instructio

cyclesclock CPUCPI

25

10

countn Instructio

cyclesclock CPUCPI

2

22

1

11

17

Clock Cycles

• Instead of reporting execution time in seconds, we often use cycles

• Clock “ticks” indicate when to start activities (one abstraction):

• cycle time = time between ticks = seconds per cycle

• clock rate (frequency) = cycles per second (1 Hz. = 1 cycle/sec)

A 4 Ghz. clock has a cycle time

time

seconds

program

cycles

program

seconds

cycle

(ps) spicosecond 2501210 9104

1

18

So, to improve performance (everything else being equal) you can either

(increase or decrease?)

________ the # of required cycles for a program, or

________ the clock cycle time or, said another way,

________ the clock rate.

How to Improve Performance

seconds

program

cycles

program

seconds

cycle

19

• Could assume that number of cycles equals number of instructions

This assumption is incorrect,

different instructions take different amounts of time on different machines.

Why? hint: remember that these are machine instructions, not lines of C code

time

1st

inst

ruct

ion

2nd

inst

ruct

ion

3rd

inst

ruct

ion

4th

5th

6th ...

How many cycles are required for a program?

20

• Multiplication takes more time than addition

• Floating point operations take longer than integer ones

• Accessing memory takes more time than accessing registers

• Important point: changing the cycle time often changes the number of cycles required for various instructions (more later)

time

Different numbers of cycles for different instructions

21

• Our favorite program runs in 10 seconds on computer A, which has a 4 GHz. clock. We are trying to help a computer designer build a new machine B, that will run this program in 6 seconds. The designer can use new (or perhaps more expensive) technology to substantially increase the clock rate, but has informed us that this increase will affect the rest of the CPU design, causing machine B to require 1.2 times as many clock cycles as machine A for the same program. What clock rate should we tell the designer to target?"

• Don't Panic, can easily work this out from basic principles

Example

22

• A given program will require

– some number of instructions (machine instructions)

– some number of cycles

– some number of seconds

• We have a vocabulary that relates these quantities:

– cycle time (seconds per cycle)

– clock rate (cycles per second)

– CPI (cycles per instruction)

a floating point intensive application might have a higher CPI

– MIPS (millions of instructions per second)

this would be higher for a program using simple instructions

Now that we understand cycles

23

Performance

• Performance is determined by execution time

• Do any of the other variables equal performance?

– # of cycles to execute program?

– # of instructions in program?

– # of cycles per second?

– average # of cycles per instruction?

– average # of instructions per second?

• Common pitfall: thinking one of the variables is indicative of performance when it really isn’t.

24

• Suppose we have two implementations of the same instruction set architecture (ISA).

For some program,

Machine A has a clock cycle time of 250 ps and a CPI of 2.0 Machine B has a clock cycle time of 500 ps and a CPI of 1.2

What machine is faster for this program, and by how much?

• If two machines have the same ISA which of our quantities (e.g., clock rate, CPI, execution time, # of instructions, MIPS) will always be identical?

CPI Example

25

• A compiler designer is trying to decide between two code sequences for a particular machine. Based on the hardware implementation, there are three different classes of instructions: Class A, Class B, and Class C, and they require one, two, and three cycles (respectively).

The first code sequence has 5 instructions: 2 of A, 1 of B, and 2 of CThe second sequence has 6 instructions: 4 of A, 1 of B, and 1 of C.

Which sequence will be faster? How much?What is the CPI for each sequence?

# of Instructions Example

26

• Two different compilers are being tested for a 4 GHz. machine with three different classes of instructions: Class A, Class B, and Class C, which require one, two, and three cycles (respectively). Both compilers are used to produce code for a large piece of software.

The first compiler's code uses 5 million Class A instructions, 1 million Class B instructions, and 1 million Class C instructions.

The second compiler's code uses 10 million Class A instructions, 1 million Class B instructions, and 1 million Class C instructions.

• Which sequence will be faster according to MIPS?

• Which sequence will be faster according to execution time?

MIPS example

27

• Performance best determined by running a real application

– Use programs typical of expected workload

– Or, typical of expected class of applicationse.g., compilers/editors, scientific applications, graphics,

etc.

• Small benchmarks

– nice for architects and designers

– easy to standardize

– can be abused

• SPEC (System Performance Evaluation Cooperative)

– companies have agreed on a set of real program and inputs

– valuable indicator of performance (and compiler technology)

– can still be abused

Benchmarks

28

Benchmark Games

• An embarrassed Intel Corp. acknowledged Friday that a bug in a software program known as a compiler had led the company to overstate the speed of its microprocessor chips on an industry benchmark by 10 percent. However, industry analysts said the coding error…was a sad commentary on a common industry practice of “cheating” on standardized performance tests…The error was pointed out to Intel two days ago by a competitor, Motorola …came in a test known as SPECint92…Intel acknowledged that it had “optimized” its compiler to improve its test scores. The company had also said that it did not like the practice but felt to compelled to make the optimizations because its competitors were doing the same thing…At the heart of Intel’s problem is the practice of “tuning” compiler programs to recognize certain computing problems in the test and then substituting special handwritten pieces of code…

Saturday, January 6, 1996 New York Times

29

SPEC ‘89

• Compiler “enhancements” and performance

0

100

200

300

400

500

600

700

800

tomcatvfppppmatrix300eqntottlinasa7doducspiceespressogcc

BenchmarkCompiler

Enhanced compiler

SP

EC

pe

rfo

rman

ce r

atio

30

SPEC CPU2000

31

SPEC 2000

Does doubling the clock rate double the performance?

Can a machine with a slower clock rate have better performance?

Clock rate in MHz

500 1000 1500 30002000 2500 35000

200

400

600

800

1000

1200

1400

Pentium III CINT2000

Pentium 4 CINT2000

Pentium III CFP2000

Pentium 4 CFP2000

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

SPECINT2000 SPECFP2000 SPECINT2000 SPECFP2000 SPECINT2000 SPECFP2000

Always on/maximum clock Laptop mode/adaptiveclock

Minimum power/minimumclock

Benchmark and power mode

Pentium M @ 1.6/0.6 GHz

Pentium 4-M @ 2.4/1.2 GHz

Pentium III-M @ 1.2/0.8 GHz

32

Experiment

• Phone a major computer retailer and tell them you are having trouble deciding between two different computers, specifically you are confused about the processors strengths and weaknesses

(e.g., Pentium 4 at 2Ghz vs. Celeron M at 1.4 Ghz )

• What kind of response are you likely to get?

• What kind of response could you give a friend with the same question?

33

Execution Time After Improvement =

Execution Time Unaffected +( Execution Time Affected / Amount of Improvement )

• Example:

"Suppose a program runs in 100 seconds on a machine, with multiply responsible for 80 seconds of this time. How much do we have to improve the speed of multiplication if we want the program to run 4 times faster?"

How about making it 5 times faster?

• Principle: Make the common case fast

Amdahl's Law

34

• Suppose we enhance a machine making all floating-point instructions run five times faster. If the execution time of some benchmark before the floating-point enhancement is 10 seconds, what will the speedup be if half of the 10 seconds is spent executing floating-point instructions?

• We are looking for a benchmark to show off the new floating-point unit described above, and want the overall benchmark to show a speedup of 3. One benchmark we are considering runs for 100 seconds with the old floating-point hardware. How much of the execution time would floating-point instructions have to account for in this program in order to yield our desired speedup on this benchmark?

Example

35

• Performance is specific to a particular program/s

– Total execution time is a consistent summary of performance

• For a given architecture performance increases come from:

– increases in clock rate (without adverse CPI affects)

– improvements in processor organization that lower CPI

– compiler enhancements that lower CPI and/or instruction count

– Algorithm/Language choices that affect instruction count

• Pitfall: expecting improvement in one aspect of a machine’s

performance to affect the total performance

Remember

36

Lets Build a Processor

• Almost ready to move into chapter 5 and start building a processor

• First, let’s review Boolean Logic and build the ALU we’ll need(Material from Appendix B)

32

32

32

operation

result

a

b

ALU

37

• Problem: Consider a logic function with three inputs: A, B, and C.

Output D is true if at least one input is trueOutput E is true if exactly two inputs are trueOutput F is true only if all three inputs are true

• Show the truth table for these three functions.

• Show the Boolean equations for these three functions.

• Show an implementation consisting of inverters, AND, and OR gates.

Review: Boolean Algebra & Gates

38

• Let's build an ALU to support the andi and ori instructions

– we'll just build a 1 bit ALU, and use 32 of them

• Possible Implementation (sum-of-products):

b

a

operation

result

op a b res

An ALU (arithmetic logic unit)

39

• Selects one of the inputs to be the output, based on a control input

• Lets build our ALU using a MUX:

S

CA

B0

1

Review: The Multiplexor

note: we call this a 2-input mux even though it has 3 inputs!

40

• Not easy to decide the “best” way to build something

– Don't want too many inputs to a single gate

– Don’t want to have to go through too many gates

– for our purposes, ease of comprehension is important

• Let's look at a 1-bit ALU for addition:

• How could we build a 1-bit ALU for add, and, and or?

• How could we build a 32-bit ALU?

Different Implementations

cout = a b + a cin + b cin

sum = a xor b xor cin

Sum

CarryIn

CarryOut

a

b

41

Building a 32 bit ALU

b

0

2

Result

Operation

a

1

CarryIn

CarryOut

Result31a31

b31

Result0

CarryIn

a0

b0

Result1a1

b1

Result2a2

b2

Operation

ALU0

CarryIn

CarryOut

ALU1

CarryIn

CarryOut

ALU2

CarryIn

CarryOut

ALU31

CarryIn

42

• Two's complement approach: just negate b and add.

• How do we negate?

• A very clever solution:

What about subtraction (a – b) ?

0

2

Result

Operation

a

1

CarryIn

CarryOut

0

1

Binvert

b

43

Adding a NOR function

• Can also choose to invert a. How do we get “a NOR b” ?

Binvert

a

b

CarryIn

CarryOut

Operation

1

0

2+

Result

1

0

Ainvert

1

0

44

• Need to support the set-on-less-than instruction (slt)

– remember: slt is an arithmetic instruction

– produces a 1 if rs < rt and 0 otherwise

– use subtraction: (a-b) < 0 implies a < b

• Need to support test for equality (beq $t5, $t6, $t7)

– use subtraction: (a-b) = 0 implies a = b

Tailoring the ALU to the MIPS

Supporting slt

• Can we figure out the idea?

Binvert

a

b

CarryIn

CarryOut

Operation

1

0

2+

Result

1

0

Ainvert

1

0

3Less

Binvert

a

b

CarryIn

Operation

1

0

2+

Result

1

0

3Less

Overflowdetection

Set

Overflow

Ainvert

1

0

Use this ALU for most significant bitall other bits

46

a0

Operation

CarryInALU0Less

CarryOut

b0

CarryIn

a1 CarryInALU1Less

CarryOut

b1

Result0

Result1

a2 CarryInALU2Less

CarryOut

b2

a31 CarryInALU31Less

b31

Result2

Result31

......

...

BinvertAinvert

0

0

0 Overflow

Set

CarryIn

Supporting slt

47

Test for equality

• Notice control lines:

0000 = and0001 = or0010 = add0110 = subtract0111 = slt1100 = NOR

•Note: zero is a 1 when the result is zero!

a0

Operation

CarryInALU0Less

CarryOut

b0

a1 CarryInALU1Less

CarryOut

b1

Result0

Result1

a2 CarryInALU2Less

CarryOut

b2

a31 CarryInALU31Less

b31

Result2

Result31

......

...

Bnegate

Ainvert

0

0

0 Overflow

Set

CarryIn...

...Zero

48

Conclusion

• We can build an ALU to support the MIPS instruction set

– key idea: use multiplexor to select the output we want

– we can efficiently perform subtraction using two’s complement

– we can replicate a 1-bit ALU to produce a 32-bit ALU

• Important points about hardware

– all of the gates are always working

– the speed of a gate is affected by the number of inputs to the gate

– the speed of a circuit is affected by the number of gates in series(on the “critical path” or the “deepest level of logic”)

• Our primary focus: comprehension, however,– Clever changes to organization can improve performance

(similar to using better algorithms in software)– We saw this in multiplication, let’s look at addition now

49

• Is a 32-bit ALU as fast as a 1-bit ALU?

• Is there more than one way to do addition?

– two extremes: ripple carry and sum-of-products

Can you see the ripple? How could you get rid of it?

c1 = b0c0 + a0c0 + a0b0

c2 = b1c1 + a1c1 + a1b1 c2 =

c3 = b2c2 + a2c2 + a2b2 c3 =

c4 = b3c3 + a3c3 + a3b3 c4 =

Not feasible! Why?

Problem: ripple carry adder is slow

50

• An approach in-between our two extremes

• Motivation:

– If we didn't know the value of carry-in, what could we do?

– When would we always generate a carry? gi = ai bi

– When would we propagate the carry? pi = ai + bi

• Did we get rid of the ripple?

c1 = g0 + p0c0

c2 = g1 + p1c1 c2 =

c3 = g2 + p2c2 c3 =

c4 = g3 + p3c3 c4 =

Feasible! Why?

Carry-lookahead adder

51

• Can’t build a 16 bit adder this way... (too big)• Could use ripple carry of 4-bit CLA adders• Better: use the CLA principle again!

Use principle to build bigger adders

a4 CarryIn

ALU1 P1 G1

b4a5b5a6b6a7b7

a0 CarryIn

ALU0 P0 G0

b0

Carry-lookahead unit

a1b1a2b2a3b3

CarryIn

Result0–3

pigi

ci + 1

pi + 1

gi + 1

C1

Result4–7

a8 CarryIn

ALU2 P2 G2

b8a9b9

a10b10a11b11

ci + 2

pi + 2

gi + 2

C2

Result8–11

a12 CarryIn

ALU3 P3 G3

b12a13b13a14b14a15b15

ci + 3

pi + 3

gi + 3

C3

Result12–15

ci + 4C4

CarryOut

52

ALU Summary

• We can build an ALU to support MIPS addition

• Our focus is on comprehension, not performance

• Real processors use more sophisticated techniques for arithmetic

• Where performance is not critical, hardware description languages allow designers to completely automate the creation of hardware!