Comparative Study of Parallel Comparative Study of Parallel Performance Visualization Performance Visualization

ToolsTools

By J. Ramphis CastroBy J. Ramphis Castro

December 4, 2002December 4, 2002

Presentation OverviewPresentation Overview

IntroductionIntroduction

Evaluation CriteriaEvaluation Criteria

AimsAims

ParaGraphParaGraph

VAmpirVAmpir

ConclusionsConclusions

IntroductionIntroduction

The reasons for poor performance of parallel The reasons for poor performance of parallel message-passing codes can be varied and message-passing codes can be varied and complex and users need to be able to complex and users need to be able to understand and correct performance problems. understand and correct performance problems.

Performance tools can help by monitoring a Performance tools can help by monitoring a program's execution and producing program's execution and producing performance data that can be analyzed to performance data that can be analyzed to locate and understand areas of poor locate and understand areas of poor performance. performance.

Introduction (cont)Introduction (cont)

There are many performance There are many performance visualization tools available to the visualization tools available to the parallel programming community. parallel programming community. The purpose of this review is to The purpose of this review is to analyze the distinctive features that analyze the distinctive features that separate them from each other.separate them from each other.

Evaluation CriteriaEvaluation Criteria

There must always be some There must always be some guidelines with which to separate guidelines with which to separate and gauge the value of a given piece and gauge the value of a given piece of software. Therefore, we used of software. Therefore, we used numerous review standards in order numerous review standards in order to accurately gauge the individual to accurately gauge the individual software’s value. software’s value.

UsabilityUsability

Intuitive easy-to-use interface Intuitive easy-to-use interface Adequate documentation and Adequate documentation and

supportsupport On-line help and man pagesOn-line help and man pages

PortabilityPortability

Amount of platforms it can be used Amount of platforms it can be used onon

Ease of building for a new hostEase of building for a new host MPI implementations and languages MPI implementations and languages

it could handleit could handle

VersatilityVersatility

Ability to analyze performance data Ability to analyze performance data in different ways and to display in different ways and to display performance information using performance information using different viewsdifferent views

Ability for new analysis methods and Ability for new analysis methods and views to be added easily views to be added easily

ScalabilityScalability

Ability to handle large numbers of Ability to handle large numbers of processes processes

Ability large or long-running Ability large or long-running programs programs

Acceptable overhead increasesAcceptable overhead increases

MaturityMaturity

Handles error conditions without Handles error conditions without crashingcrashing

Few or no penalties by incurring in Few or no penalties by incurring in mistakes while using the programmistakes while using the program

Level of support Level of support Size of user baseSize of user base

Automated Instrumentation Automated Instrumentation and Monitoring System and Monitoring System

(AIMS)(AIMS) AIMS is a software toolkit for AIMS is a software toolkit for

measurement and analysis of Fortran measurement and analysis of Fortran 77 and C message-passing programs 77 and C message-passing programs written using the NX, PVM, or MPI written using the NX, PVM, or MPI communication libraries. communication libraries.

UsabilityUsability

Software distribution included Software distribution included readme files and man pagesreadme files and man pages

User guide available from main web User guide available from main web pagepage

User interface was functional, not User interface was functional, not aesthetically pleasingaesthetically pleasing

PortabilityPortability

Platforms (HP, SGI, IBM, and SUN) Platforms (HP, SGI, IBM, and SUN) University of Tennessee is porting AIMS to University of Tennessee is porting AIMS to

DoD MSRC DoD MSRC An ASCI effort at LANL is porting AIMS to An ASCI effort at LANL is porting AIMS to

ASCIASCI Data files are in ASCII and thus portableData files are in ASCII and thus portable AIMS supporting HPF is being developed AIMS supporting HPF is being developed

jointly with the Portland Group jointly with the Portland Group Developers of AIMS are working on a port Developers of AIMS are working on a port

for the SGI Origin 2000 for the SGI Origin 2000

VersatilityVersatility

Instrumentation done automatically Instrumentation done automatically (xinstrument)(xinstrument)

There are a few display types: VK, There are a few display types: VK, tally, MK, and Xisk. (provide different tally, MK, and Xisk. (provide different display choices)display choices)

ScalabilityScalability

PC is the utility name that attempts PC is the utility name that attempts to remove overheadto remove overhead

No data available on the tool’s No data available on the tool’s impact on overhead impact on overhead

MaturityMaturity

From what information we could find From what information we could find about AIMS support and development about AIMS support and development we can conclude that it is still a we can conclude that it is still a relatively young program and counts relatively young program and counts on minimal developer supporton minimal developer support

There a numerous projects to port There a numerous projects to port AIMS to different platformsAIMS to different platforms

Verdict on AIMSVerdict on AIMS

Although there are a few research Although there are a few research groups trying to improve upon its groups trying to improve upon its availability to different platforms and availability to different platforms and its robustness, AIMS suffers from its robustness, AIMS suffers from reliability issues. reliability issues.

Has good features for all around Has good features for all around visualization.visualization.

ParaGraphParaGraph

This tool was created for the purpose This tool was created for the purpose of providing a detailed, dynamic, of providing a detailed, dynamic, graphical animation of the behavior graphical animation of the behavior of message-passing parallel of message-passing parallel programs and graphical summaries programs and graphical summaries of their performance.of their performance.

UsabilityUsability

Most features can be accessed Most features can be accessed through “Point and Click” operations through “Point and Click” operations

Easy manipulation of the trace file in Easy manipulation of the trace file in use use

Offers intuitive appeal and aesthetic Offers intuitive appeal and aesthetic quality of its displays quality of its displays

Vast amount of documentation Vast amount of documentation available from the developers available from the developers

PortabilityPortability

Based on X WindowsBased on X Windows Is available for most unix based Is available for most unix based

systemssystems Its availability to different platforms Its availability to different platforms

rivals even the most accomplished of rivals even the most accomplished of commercial tools commercial tools

VersatilityVersatility

Three categories - utilization, Three categories - utilization, communication, and task informationcommunication, and task information

25 different displays in its current 25 different displays in its current formform

Allows for users to create their own Allows for users to create their own visualization windowsvisualization windows

ScalabilityScalability

Due to its association and Due to its association and dependency with PICL, its overhead dependency with PICL, its overhead is basically a function of the is basically a function of the frequency and volume of frequency and volume of communication trafficcommunication traffic

Visualization still provides accurate Visualization still provides accurate and useful informationand useful information

MaturityMaturity

Fairly mature tool Fairly mature tool Minor bugs (“features”)Minor bugs (“features”) PG has a display limitation of how PG has a display limitation of how

many processors can be displayed at many processors can be displayed at one time (1024)one time (1024)

Verdict on PGVerdict on PG

Regardless of its few setbacks and Regardless of its few setbacks and limitations, PG is a full featured and powerful limitations, PG is a full featured and powerful visualization tool for small to medium sized visualization tool for small to medium sized clusters. Its ease of use, display features, clusters. Its ease of use, display features, developer support, open-source nature, and developer support, open-source nature, and its huge community support (which allows for its huge community support (which allows for numerous improvements over a shorter numerous improvements over a shorter period of time) it is the best visualization tool period of time) it is the best visualization tool available freely over the internet short of available freely over the internet short of developing one yourself.developing one yourself.

Visualization and Analysis Visualization and Analysis of MPI Programs (Vampir)of MPI Programs (Vampir)

VAMPIR is a graphical tool for VAMPIR is a graphical tool for analyzing the performance and analyzing the performance and message passing characteristics of message passing characteristics of parallel programs that use the MPI parallel programs that use the MPI and/or SHMEM communication and/or SHMEM communication libraries. It is the top commercial tool libraries. It is the top commercial tool available today and is developed and available today and is developed and supported by Pallas.supported by Pallas.

UsabilityUsability

On-the-fly segment zoom On-the-fly segment zoom Built-in snapshot capabilities Built-in snapshot capabilities Graph to source code linking Graph to source code linking Shortcuts to most of the commands Shortcuts to most of the commands

and displays and displays ““Point and Click” interfacePoint and Click” interface

PortabilityPortability

Architectures supported is simply Architectures supported is simply impressiveimpressive

Supported on T3E, IBM SP2, SGI Supported on T3E, IBM SP2, SGI PowerChallenge, and Sun Solaris PowerChallenge, and Sun Solaris (just to name a few)(just to name a few)

Only limited by its demand on any Only limited by its demand on any specific platform or research groupspecific platform or research group

VersatilityVersatility

The amount of displays available is The amount of displays available is just staggering (if you need it, they just staggering (if you need it, they probably have it or are in the process probably have it or are in the process of making it) of making it)

ScalabilityScalability

Causes the least amount of Causes the least amount of disturbance on the program behavior disturbance on the program behavior out of all the tools (its effects are out of all the tools (its effects are more noticeable on large clusters)more noticeable on large clusters)

Logistic curve overheadLogistic curve overhead

MaturityMaturity

Monthly fee, they can also provide 24/7 Monthly fee, they can also provide 24/7 customer service and support customer service and support

Has new releases or patches almost every Has new releases or patches almost every monthmonth

Robustness is one of its best qualities (i.e.-Robustness is one of its best qualities (i.e.-systems without a globally consistent systems without a globally consistent clock, VAMPIRtrace automatically corrects clock, VAMPIRtrace automatically corrects clock offset and skew so that timestamps clock offset and skew so that timestamps in the merged trace file are consistent)in the merged trace file are consistent)

Cannot benefit from their user base efforts Cannot benefit from their user base efforts to improve vampir at a faster rate to improve vampir at a faster rate

Verdict on VampirVerdict on Vampir

If you have the money required to purchase and If you have the money required to purchase and maintain such a product, vampir is the best maintain such a product, vampir is the best visualization tool available on the market today. visualization tool available on the market today. It has the most features, easiest and most It has the most features, easiest and most intuitive UI, and the largest support infrastructure intuitive UI, and the largest support infrastructure (funded by its customers).(funded by its customers).

Works best for very large cluster, where savings Works best for very large cluster, where savings in overhead are more noticeablein overhead are more noticeable

Displays contain a lot of information and legends Displays contain a lot of information and legends are very informativeare very informative

Visualization features are easy to use and Visualization features are easy to use and understandunderstand

Overall ConclusionOverall Conclusion

AIMS seemed like a very promising tool, AIMS seemed like a very promising tool, but its lack of support, available platforms but its lack of support, available platforms and features is very disappointing and features is very disappointing

Vampir has the most features, support and Vampir has the most features, support and platform availability out of all the tools. Its platform availability out of all the tools. Its display flexibility and the sheer volume of display flexibility and the sheer volume of data it can display accurately on screen at data it can display accurately on screen at any given time, make it the perfect tool for any given time, make it the perfect tool for very large cluster of computers. very large cluster of computers.

Overall Conclusion (cont)Overall Conclusion (cont)

ParaGraph is the clear winner. ParaGraph is the clear winner. Nothing more can be said of a FREE Nothing more can be said of a FREE tool that rivals Vampir in features, tool that rivals Vampir in features, robustness and versatility whose robustness and versatility whose open-source nature and design open-source nature and design structure allow it to be a dynamic structure allow it to be a dynamic and highly adaptive tool for today’s and highly adaptive tool for today’s fast paced research market. fast paced research market.