performance evaluation: 30 years in review
DESCRIPTION
Performance Evaluation: 30 years in review. Martin Reiser GMD – German National Research Center for Information Technology [email protected] MMB’99, 22.–24.September 1999, Trier Presentation 23. September, 9:00. Host. Switch. Link. Memory. ALU. Channels. Very complex systems. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/1.jpg)
Performance Evaluation: 30 years in reviewMartin ReiserGMD – German National Research Center for Information Technology
MMB’99, 22.–24.September 1999, Trier
Presentation 23. September, 9:00
![Page 2: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/2.jpg)
Very complex systems… where are the limits?
… will it work?
… if so, how well?
… can it work better?
… what will I need tomorrow, in a month, year …?
but why bother anyway?
Host
Switch
Link
ALU
Memory
Channels
![Page 3: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/3.jpg)
30 years?m
any
subs
crib
ers
m tr
unk
line
s
?…
A. K. Erlang, The Post Office Electrical Engineers's Journal 1918
80!
E1,m (ρ) =
ρm
m!ρk
k!k=0m∑
![Page 4: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/4.jpg)
Cutting the Gordian knot
Kleinrock, 1964 Gaver, 1967; Buzen, 1971
Message Independence Central Server
![Page 5: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/5.jpg)
Big bang: BCMP
i
j
closed class s
G(•)
open class r r
i
j
class change pi,r; j ,s
Ns Jobs
Baskett, Chandy, Muntz and Palacios, 1975
LCFS/PRPSIS
E( )
G( )
FCFS
unlimited waiting roomqueue-dependent rates
![Page 6: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/6.jpg)
The PF solution is robust
P(n) C(nii
) Wi,rni ,r
r
station i, class r:
Routing details don’t matter, only mean number of visits does!
Distribution of service-times does not matter, only mean values enter into the solution!
long live the Central Server
![Page 7: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/7.jpg)
The classical models
Delay analysis of data networks (Kleinrock 64)
Time-sharing model (Scherr 67)
Sharing the CPU, i.e. PS (Kleinrock et. al. 64 +)
Central Server (Buzen 71)
Polling, token-ring (Konheim et. al. 74 +)
Aloha, CSMA …(Kleinrock, Lam, Tobagi 73 +)
![Page 8: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/8.jpg)
From trees
3274
3705
3274
370
3705
to meshed networks
![Page 9: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/9.jpg)
From window flow-control
Ti (W) i 1 ni (W 1)
W
Ti (W)ni(W) Ti(W)
to MVA
![Page 10: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/10.jpg)
From Poisson arrivalscall
to MAP…
1
silence talk spurt
2
MPX
![Page 11: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/11.jpg)
20
40
50
60
70
30
Product-Form Qing Theory State-space
Moline (PAP) Erlang MM∞/K
Tandem
Jackson
Gordon/Newell
Convolution
BCMP
Norton's theorem
MM1
Crommelin MDk
MG1LittleCohen
Cyclic Service
Conservation LawsPS
Simon Ando
Courtois
Methods
![Page 12: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/12.jpg)
80
90
MVA
BlockingExotic Qs
Waiting timesG-Networks
customized MVA
PF-Petri Nets
Matrix geom.
Complex tandem
MAP queues
Self-similar trafficLarge deviation th.
UniformizationPerformability
GSPN
Iterative Meth.
DSPNArnoldi
Algebraic meth.
Product-Form Qing Theory State-spaceMethods
![Page 13: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/13.jpg)
Architecture Communication Operation
40
50
60
70
80
90
Capacity planning(BEST1)
Manufacturing
Software Perf. Eng.
Workflow
Netw. mgmt.
Time-sharingVirtual memory
Central Server
Processor arch.Memory hierarchy
(caches)Multiprocessor
Parallel pgm's
Mem. consistencyPrefetching
multithreaded arch.
Telephony
Traffic theory
Delay analysis
Links (Polling)
Flow control
LAN
ATMBroadband
Interconn. netw's
Selfsimilar traffic
Internet, WWW
30
Problems
![Page 14: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/14.jpg)
40
50
60
70
80
90
Capacity planning(BEST1)
Manufacturing
Software Perf. Eng.
Workflow
Netw. mgmt.
Time-sharingVirtual memory
Central Server
Processor arch.Memory hierarchy
(caches)Multiprocessor
Parallel pgm's
Mem. consistencyPrefetching
multithreaded arch.
Telephony
Traffic theory
Delay analysis
Links (Polling)
Flow control
LAN
ATMBroadband
Interconn. netw's
Selfsimilar traffic
Internet, WWW
30
Architecture Communication OperationProblems
![Page 15: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/15.jpg)
PE specialists… work on PE problems cached in PE terms
… build tools
… are integral parts of development teams
… develop (and solve) models of real systems
really?
… use PE methodology in their daily work (planning, managing, tuning)
IS professionals
![Page 16: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/16.jpg)
Success stories
PE-based engineering and development
Network management
Capacity planning
Failures ?
Software performance engineering
![Page 17: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/17.jpg)
PE-basierte F+ESkilled PE specialist group
– is integrated in R&D team– is of sufficient size (5 – 10%)– is highly respected– and contributes to design!
All important design decisions are basedon PE results and analysis
Early prototype supportPE predictions
Case study: IBM Token Ring
![Page 18: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/18.jpg)
VLSI Design
Generally accepted specification process
Simulation data from specification • on several levels of detail
Automatic mask generation
Generation of test patterns
integrated processenvironment
![Page 19: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/19.jpg)
PE in standards
Success story LAN standard IEEE 802
Success story ATM
fix before build
Early attempts in SDLC/HDLC
![Page 20: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/20.jpg)
What is PE?
Is it a science
Is it used by engineers?
Is it used in IS planning/tuning?
Is it a business?
Is it regular part of university curriculum?
…………………………………………………yes
……………………………some
………………little
…………cottage industry
……not really
Quo vadis ?
![Page 21: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/21.jpg)
Stay where we are?
Reach out to main stream?Must make quantum leap in SPE
Must integrate tool into develop-ment processes • robust • Sprache des Benutzers
• accepted in EE, especially communications
• continue existence as niche science
![Page 22: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/22.jpg)
SPE roadmap
Rep.Usage
scenario editor
Workload data
Network capacityplanning
Analytic system structure model
System structure
editor
Capacityplanning
Path-lengthestimation
process
CASE me-thodology
In user’s language
![Page 23: Performance Evaluation: 30 years in review](https://reader035.vdocuments.net/reader035/viewer/2022062500/568150ee550346895dbf07a9/html5/thumbnails/23.jpg)
3 grand challenges
Understand self-similar traffic • do we need a new theory? • se we merely shifting loads?
Introduce analytical methods in computerarchitecture (“a gold mine” S. Adve)
to enter into the main stream ofthe CASE process and tools