using visualization to understand the behavior of computer systems robert p. bosch jr. stanford...
Post on 22-Dec-2015
218 views
TRANSCRIPT
Using Visualization to Understand
the Behavior of Computer Systems
Robert P. Bosch Jr.Stanford University
May 3, 2001
2
Motivation
• Explosion in complexity of computer systems
• Development of rich data collection tools– Complete Machine Simulation: SimOS– Software Monitoring: Performance Co-Pilot
(PCP)– Firmware Instrumentation: FlashPoint– Hardware Monitoring: DCPI
• Challenge: how do we fully exploit the large, detailed data sets these tools can generate?
3
Data Analysis Challenge
• How do we typically handle this data?– Visual inspection of huge log files– Summarize through statistics and aggregation– Focus on restricted data subsets
• Alternative approach: data visualization– Display large amounts of data at once– Enable interactive exploration of entire data set
• Overview, zoom and filter, details-on-demand
– Use human perception to discover patterns, trends, and interesting information
4
Computer Systems Visualization:Existing Work
• Pedagogical examples– Processors: DLXView, Pentium Pro Tutorial– Memory: Cache Visualization Tool (CVT)
• Parallel systems performance– AIMS, Pablo, Paradyn, ParaGraph, PARvis,
StormWatch, VAMPIR…
• Other examples– Network performance, file systems, etc.
5
Computer Systems Visualization:Existing Work
• Demonstrate potential of visualization• Limitations
– Focused on particular system components– Integrated with specific data collection tools– Limited to a fixed set of visual
representations
• Conclusion: rich, flexible data sources require an equally powerful and flexible visualization system
6
Outline
• Motivation• The Rivet visualization environment
– Architecture– Implementation
• Focused visualization systems– SUIF Explorer– Thor– PipeCleaner– Visible Computer
• Ad hoc analysis and visualization• Contributions
7
Rivet Architecture: Goals
• Learn once, apply to wide range of problems– Decouple visualization and data collection
• Interactive exploration of large data sets– Couple visualization and analysis
• Rapid prototyping of visualizations• Extensibility
– Allow users to add new components
8
Rivet Architecture: Approach
• Identify visualization “building blocks”• Mix and match to create visualizations• Users can add new components as
needed• Three basic object types:
– Data management: tuples, tables, and transforms
– Visual representation: primitives and metaphors
– Mapping from data to visual: encodings
9
Data Model
• Simplified relational model– Tuple: collection of data fields– Table: set of tuples with common data format– Familiar, homogeneous model
• Load data by parsing text files• Directly save/load tables in binary format
Procedure:PID:Page Faults:
Redraw()1717
129...
Tuple Tabl
e
10
.
.
.
Data Transforms
• Transforms enable users to operate on data• Can be composed to form data networks
– Active: data changes are propagated
• Rivet includes a set of standard transforms– Filter, sort, group, merge, join, aggregate, etc.
• Users may design and incorporate their own– Example: clustering algorithms
.
.
.......
GroupBy
Transform
11
Visual Objects and Data Mapping
• Primitives draw individual tuples• Metaphors draw entire tables
– Draws table attributes: axes, labels, etc.– For each tuple: compute bounding box, select primitive
• Encodings map tuple contents to visual properties– Metaphors use spatial encodings
• Map tuple fields to bounding box parameters
– Primitives use attribute encodings• Map tuple fields to retinal properties: color, fill pattern,
size…
• Encodings encapsulate datavisual mapping– Metaphors and primitives are data independent
12
Display
The Encoding Process: Example
X
Y
Graph Metaphor:
Spatial EncodingsProcedure:
PID:Page Faults:
Redraw()1717
129
Tuple
C
F
S
Rectangle Primitive: Attribute Encodings
0.5
13
Coordination and Interaction
• Implicit coordination: sharing of objects– Examples:
• Primitives share attribute encodings: brushing• Metaphors share primitives: common appearance• Metaphors share spatial encodings: common axes
– Listener mechanism
• Explicit coordination: events and bindings– Rivet objects can raise events– User can bind actions to these events– Example: details-on-demand
14
Rivet Implementation
• Goal: balance performance and flexibility– Interactive visualizations of large data sets– Rapid prototyping of visualizations
• C++ and OpenGL for performance– Also provides platform independence
• Scripting language interfaces for flexibility– Simplified Wrapper and Interface Generator (SWIG)– Create visualizations by writing scripts
• Interpreter is not in the main loop– Listener mechanism for high-frequency events– Event bindings for low-frequency user interaction
15
Sample Rivet Scriptset table [DataVector]set parser [CSVParser -args $table]rparse $parser poletops.csv Import data into
tableCreate point primitive
set primitive [GLPoint]
Create graph metaphor
rwindow .radiosrgeometry .radios W 500 H 675rglob Graph .radios.map.radios.map SetData $table
Encode Longitude as X
set gran [expr 1.0 / 3600.0]set lmin [$table GetMin Longitude]set lmax [$table GetMax Longitude]set long [QUniformRangeMap –args $lmin $lmax $gran].radios.map EncodeAsXPosition \ [QRangeEncoding -args $long Longitude]
Encode # Radios as Color
set hue [list 0.0 0.5]set sat [list 0.0 1.0]set val [list 0.6 1.0]set ramp [IsomorphicColorMap –args $hue $sat $val]set rmin [$table GetMin Radios]set rmax [$table GetMax Radios]set radios [QNumberMap -args $Log $rmin $rmax]$ramp SetDomainMap $radios$primitive EncodeAsColor \ [QColorEncoding -args $ramp Radios]
Encode Latitude as Y
set lmin [$table GetMin Latitude]set lmax [$table GetMax Latitude]set lat [QUniformRangeMap –args $lmin $lmax $gran].radios.map EncodeAsYPosition \ [QRangeEncoding -args $lat Latitude]
16
Outline
• Motivation• The Rivet visualization environment
– Architecture– Implementation
• Focused visualization systems– SUIF Explorer: Interactive user-directed
parallelization– Thor: Detailed memory profiling on FLASH– PipeCleaner: Superscalar processor pipelines– Visible Computer: System and cluster monitoring
• Ad hoc analysis and visualization• Contributions
18
SUIF Explorer: Background
• Goal: enable sequential codes to run fast on MPs
• Compiler parallelizes loops when possible• Otherwise:
– Compiler presents loop analysis results to user– User applies application knowledge to assist compiler
• Data source: SUIF dynamic analyzers– Performance metrics for all loops in program– Coverage, granularity, loop level
• Visualization– Displays data in context of program source code– Focuses on loops most deserving of user attention
19
SUIF Explorer: Discussion
• Benefits– Combines data with source, instead of loop
IDs– Allows user to filter uninteresting lines of
code– Facilitates comparisons between runs
• Limitations– Loosely coupled with rest of SUIF Explorer– Short loops less prominent than long loops
• Add sortable table of results as a linked view
21
Thor: Background
• Memory getting slower relative to CPU• High remote access latencies on NUMA systems• Memory system is often performance bottleneck• Data source: FlashPoint protocol on FLASH
– Collects all cache and TLB misses in firmware– Classified as local/remote, read/write– Attributed to CPU, procedure, data structure
• Visualization– Stacked bar charts showing miss counts– Collection of UI controls for configuring the display
22
Thor: Discussion
• Both post-mortem and real-time analysis– Live connection to FLASH via socket– Useful for multi-phase applications
• Simple, familiar visualization• Interactive filtering, sorting,
aggregation• Drill down from overview to data of
interest
24
PipeCleaner: Background
• High peak performance• Complex implementation techniques
– Speculation, multiple functional units, out-of-order execution
• Intended to be transparent to the programmer– True for correctness, not necessarily for performance
• Data sources: MXS and MMIX simulators– Detailed superscalar processor pipeline models
• Visualization: three linked views– Overview: occupancy strip charts– Detail: animated pipeline display– Context: program source code
25
PipeCleaner: Discussion
• Applications of PipeCleaner– Program development– Compiler optimizations– Hardware design– Education – Simulator development
• Possible extensions– Other computer pipelines: graphics
hardware– Physical pipelines: assembly lines, etc.
27
Visible Computer: Background
• Real-time analysis of system behavior– Observe behavior of system in its entirety– Explore interesting phenomena in detail
• Data sources: PCP, SimOS– Comprehensive data sources– Collect low-level hardware events
• Cache misses, disk requests, CPU utilization, etc.
– Classify using high-level structures• Process name, user/group, CPU mode, etc.
28
Visible Computer: Visualization
• Organizes data using nested physical hierarchy
• Provides overviews at each level of detail– Active icons representing system components– User defines data ranges of interest for each
component– Icons activate when components are in range
• Allows users to “remove the cover”, show next level
• Displays detailed charts on demand– Data filtered/colored using high-level classifiers
29
Visible Computer: Discussion
• Unified interface for computer systems data– Focus-plus-context view of system– Physical layout, virtual classification
• Provides an overview of system behavior
• Draws attention to potential problem areas
• Suggests targets for further exploration
30
Outline
• Motivation• The Rivet visualization environment
– Architecture– Implementation
• Focused visualization systems– SUIF Explorer– Thor– PipeCleaner– Visible Computer
• Ad hoc analysis and visualization• Contributions
31
Ad hoc Analysis and Visualization
• SimOS: Complete machine simulator– Full non-intrusive access to HW, OS, SW
state– Flexible data collection mechanism– Deterministic execution
• Combine with Rivet– Flexible data import mechanism– Rapid prototyping of visualizations
• Result: powerful analysis framework
32
Change HW model or SW
Simulation & Visualization Cycle
Configure simulated machine and
software
Perform simulation
Visualize results
Change annotations
Change visualization
33
Case Study: Argus
• Parallel, multithreaded graphics library– NURBS rendering application – Implemented using fork & shared memory
region
• Performance limitations on SGI Origin– Linear speedup up to 26 processors– Rapid performance falloff beyond 26 CPUs
• Imported into SimOS environment– Same performance characteristics observed
• Expectation: memory system is the problem
38
Memory Visualization
Large amounts of unexpected idle timeVery little memory stall time in process view
40
Process Scheduling & Kernel Locks
CPU schedulingkernel lock
If the lock is unavailable, the process is descheduledIf the lock is available, it is immediately granted
41
Process Scheduling & Kernel Locks
kernel lock is held
process is descheduled
Kernel lock is heavily contended
43
Results Using sproc
• Minimal kernel time• No idle time at all• Completes nearly
twice as fast as original version
• 95% parallel efficiency
44
Argus: Conclusion
• Five simulation & visualization iterations• Initial suspicions were totally incorrect• Flexibility of Rivet & SimOS enabled us
to follow leads• Visualizations led us to the bottleneck
– Single scheduling event out of over 50,000 events
45
Contributions
• Rivet computer systems visualization environment– Rapid prototyping support
• Quick ‘rough cut’ visualization of data• Incremental development of sophisticated data displays
– Modular architecture• Mix and match basic building blocks to create visualizations• Enables coordination through object sharing
– Data transforms as first-class citizens in viz environment• Unifies the analysis and visualization process
• Collection of computer systems visualizations– Emphasize interactive data exploration– Use relatively conventional visual representations– Provide extensive support for filtering, sorting, brushing
46
Acknowledgments
• Orals committee: Joel Ferziger, Mendel Rosenblum, Pat Hanrahan, Mark Horowitz, Monica Lam
• Mendel• Visualization collaborators
– PipeCleaner: Donald Knuth– Visible Computer: John Gerth– Argus: Gordon Stoll– Thor: Jeff Gibson– SUIF Explorer: Shih-Wei Liao
47
Acknowledgments
• Groupmates: Rivet, SimOS, FLASH, Graphics
• Studio 354– Steve Herrod and John Heinlein– The foosball table– Chris Stolte and Diane Tang
• Staff: – John, Charlie, Thoi, Kevin– Ada, Heather, Chris, …
• Funding agencies: ONR, [D?]ARPA, ASCI• Robert Bosch Corporation