Benchmarking, Performance Evaluation, Modeling and Prediction Erich Strohmaier ICL Benchmarking Activities Linpack benchmark ParkBench LLCbench The Performance Database Server TOP500 Performance Analysis Tool for NAS Linpack Benchmark 3 main flavors 100x100 1000x1000 NxN R max from NxN used in the TOP500 ParkBench Collection of LowLevel benchmarks sequential MPI PVM Kernels Linear Algebra NAS PB Kernels Applications NAS PB (simulated applications) PSTSWM LLCbench Low-Level Tests for various hardware aspects: MPBench Core MPI contructs BLASBench BLAS routines CacheBench In and out of cache data movement Performance Database Server Searchable Interface to DB of benchmark results such as: Linpack SPEC (Dhrystone etc) Basis for analysing the HCP market - Quantification of observations - Detection of trends market, architecture, technology Listing of the 500 most powerful Computers in the World Yardstick: R max /LINPACK Updating halfyearly TOP500 Procedure TOP500 list - Data shown ManufacturerManufacturer or vendor Computer Type indicated by manufacturer or vendor Inst. SiteCustomer LocationLocation and country YearYear of installation/last major update Field of Appl.Academic,Research,Industry,Vendor,Class. # Proc.Number of processors R max Maxmimal LINPACK performance achieved R peak Theoretical peak performance N max Problemsize for achieving R max N 1/2 Problemsize for achieving half of R max N world Position within the TOP500 ranking TOP10 Performance Development Manufacturer Architectures Performance Development NAS Performance Analysis Tool To provide a tool to NAS users which allows Access to performance data of NAS reference codes Performing a variety of predefined performance tests on target systems in an easy and comparable ways Analyzing user performance measurements using advanced statistical methods Analyze scalability of his/her application Compare its efficiency to other codes Get a first prediction of attainable performance ranges on other NAS systems Necessary Functionality Broad variety of predefined performance tests Simple interface do define custom performance tests Reference measurement of NAS applications Generation and maintenance of performance database Necessary Functionality Black Box performance analysis for individual codes Cross-code and cross-platform performance analysis. Tool for performance prediction for NAS applications Detection and analysis of system and code performance signatures System Architecture Test suite setup file GUI: Benchmark Skeleton codes Test benchmarks Performance DB Connected by same keys Create script Test definition database - All predefined performance test with performance models Test results Execute script - Result commit to include in DB - Test suite setup - Test definition - Result display - Result analysis using various methods - User data entry Functions available for non-linar regression Input of measured data as: #proz time Data are NPB MG Class A on Steger Statistical output of analysis - best model Data and regression display GUI - Analysis Component