1 cs 501 spring 2006 cs 501: software engineering lecture 22 performance of computer systems

30
1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

Upload: madlyn-hardy

Post on 01-Jan-2016

215 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

1 CS 501 Spring 2006

CS 501: Software Engineering

Lecture 22

Performance of Computer Systems

Page 2: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

2 CS 501 Spring 2006

Administration

Page 3: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

3 CS 501 Spring 2006

Performance of Computer Systems

In most computer systems

The cost of people is much greater than the cost of hardware

Yet performance is important

Future loads may be much greater than predicted

A single bottleneck can slow down an entire system

Page 4: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

4 CS 501 Spring 2006

Performance Challenges

Tasks

• Predict performance problems before a system is implemented

• Identify causes and fix problems after a system is implemented

Basic techniques

• Understand how the underlying hardware and networking components interact when executing the system

• Calculate the capacity and load on each component

• Identify components that are approaching peak capacity

Page 5: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

5 CS 501 Spring 2006

Understand the Interactions between Hardware and Software

Example: execution of http://www.cs.cornell.edu/

Client Servers

domain name service

TCP connection

HTTP get

Page 6: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

6 CS 501 Spring 2006

Understand the Interactions between Hardware and Software

:Thread :Toolkit :ComponentPeer target:HelloWorld

runrun callbackLoop

handleExpose

paint

Page 7: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

7 CS 501 Spring 2006

Decompress

Stream audioStream video

fork

join

start state

stop state

Understand Interactions between Hardware and Software

Page 8: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

8 CS 501 Spring 2006

Look for Bottlenecks

Possible areas of congestion

Network load

Database accesshow many joins to build a record?

Locks and sequential processing

CPU performance is rarely a factor, except in mathematical algorithms. More likely bottlenecks are:

Reading data from disk (including paging)

Moving data from memory to CPU

Page 9: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

9 CS 501 Spring 2006

Timescale

Operations per second

CPU instruction: 1,000,000,000

Disk latency: 60 read: 25,000,000 bytes

Network LAN: 10,000,000 bytesdial-up modem: 6,000 bytes

Page 10: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

10 CS 501 Spring 2006

Predicting System Performance

• Direct measurement on subsystem (benchmark)

• Mathematical models

• Simulation

• Rules of thumb

All require detailed understanding of the interaction between software and hardware systems.

Page 11: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

11 CS 501 Spring 2006

Look for Bottlenecks: Utilization

utilization =

mean service timemean inter-arrival time

When the utilization of any hardware component exceeds 30%, be prepared for congestion.

Peak loads and temporary increases in demand can be much greater than the average.

Utilization is the proportion of the capacity of a service that is used on average.

Page 12: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

12 CS 501 Spring 2006

Mathematical Models

Queueing theory

Good estimates of congestion can be made for single-server queues with:

• arrivals that are independent, random events (Poisson process)

• service times that follow families of distributions (e.g., negative exponential, gamma)

Many of the results can be extended to multi-server queues.

Page 13: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

13 CS 501 Spring 2006

Mathematical Models: Queues

arrive wait in line service depart

Single server queue

Page 14: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

14 CS 501 Spring 2006

Queues

arrive wait in line

service

depart

Multi-server queue

Page 15: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

15 CS 501 Spring 2006

Behavior of Queues: Utilization

meandelay

utilization10

Page 16: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

16 CS 501 Spring 2006

Measurements on Operational Systems

• Benchmarks: Run system on standard problem sets, sample inputs, or a simulated load on the system.

• Instrumentation: Clock specific events.

If you have any doubt about the performance of part

of a system, experiment with a simulated load.

Page 17: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

17 CS 501 Spring 2006

Example: Web Laboratory

Benchmark: Throughput v. number of CPUs

total MB/s

average / CPU

Page 18: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

18 CS 501 Spring 2006

Techniques: Simulation

Model the system as set of states and events

advance simulated time determine which events occurred update state and event listrepeat

Discrete time simulation: Time is advanced in fixed steps (e.g., 1 millisecond)

Next event simulation: Time is advanced to next event

Events can be simulated by random variables (e.g., arrival of next customer, completion of disk latency)

Page 19: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

19 CS 501 Spring 2006

Case Study: Performance of Disk Array

When many transaction use a disk array, each transaction must:

wait for specific disk platter

wait for I/O channel

signal to move heads on disk platter

wait for I/O channel

pause for disk rotation

read data

Close agreement between: results from queuing theory, simulation, and direct measurement (within 15%).

Page 20: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

20 CS 501 Spring 2006

Example: Web Laboratory

Balance of Resources

Ideal Realistic

Networking 500 Mbit/sec 100 Mbit/sec

Data online allfew crawls/year

Metadata online all all?

Disk 750 TB 240 TB

Tape archive all few crawls/year

Computers research sharedseparate with storage

Page 21: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

21 CS 501 Spring 2006

Fixing Bad Performance

If a system performs badly, begin by identifying the cause:

• Instrumentation. Add timers to the code. Often this will reveal that the delays are centered in one specific part of the system.

• Test loads. Run the system with varying loads, e.g., high transaction rates, large input files, many users, etc. This may reveal the characteristics of when the system runs badly.

• Design and code reviews. Have a team review the system design and suspect sections of code for performance problems. This may reveal an algorithm that is running very slowly, e.g., a sort, locking procedure, etc.

Fix the underlying cause or the problem will return!

Page 22: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

22 CS 501 Spring 2006

Techniques for Eliminating Bottlenecks

Serial and Parallel Processing

Single thread v. multi-thread

e.g., Unix fork

Granularity of locks on data

e.g., record locking

Network congestion

e.g., back-off algorithms

Page 23: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

23 CS 501 Spring 2006

Predicting Performance Change:Moore's Law

Original version:

The density of transistors in an integrated circuit will double every year. (Gordon Moore, Intel, 1965)

Current version:

Cost/performance of silicon chips doubles every 18 months.

Page 24: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

24 CS 501 Spring 2006

Moore's Law: Rules of Thumb

Planning assumptions:

Every year: cost/performance of silicon chips improves 25% cost/performance of magnetic media improves 30%

10 years = 100:120 years = 10,000:1

Page 25: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

25 CS 501 Spring 2006

Moore's Law and System Design

Design system: 2006

Production use: 2009

Withdrawn from production: 2019

Processor speeds: 1 1.9 28

Memory sizes: 1 1.9 28

Disk capacity: 1 2.2 51

System cost: 1 0.4 0.01

Page 26: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

26 CS 501 Spring 2006

Moore's Law Example

Will this be a typical personal computer?

2006 2019

Processor 2 GHz 50 GHz

Memory 512 MB 14 GB

Disc 50 GB 2 TB

Network 100 Mb/s 1 Gb/s

Surely there will be some fundamental changes in how this this power is packaged and used.

Page 27: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

27 CS 501 Spring 2006

Parkinson's Law

Original: Work expands to fill the time available. (C. Northcote Parkinson)

Planning assumptions:

(a) Demand will expand to use all the hardware available.

(b) Low prices will create new demands.

(c) Your software will be used on equipment that you have not envisioned.

Page 28: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

28 CS 501 Spring 2006

False Assumptions from the Past

Unix file system will never exceed 2 Gbytes (232 bytes).

AppleTalk networks will never have more than 256 hosts (28 bits).

GPS software will not last 1024 weeks.

Nobody at Dartmouth will ever earn more than $10,000 per month.

etc., etc., .....

Page 29: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

29 CS 501 Spring 2006

Moore's Law and the Long Term

1965

What level?

2005

Page 30: 1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems

30 CS 501 Spring 2006

Moore's Law and the Long Term

1965 When?

What level?

2006?

Within your working life?