Ultra-Scale Visualization with

Open-Source SoftwareBerk GeveciKitware Inc.

Background

Kitware Is• A software company• creating open-source collaboration

platforms• which are used globally for

• research• teaching• commercial application.

• This software is created by• internationally recognized experts• in extended communities• using a rigorous, quality-inducing software

development process.

Technical Portfolio

Heart and Soul: Software Quality Process

In The Beginning There Was VTK

•From Ohloh: Very large, active development team: Over the

past twelve months, 66 developers contributed new code to VTK. This is one of the largest open-source teams in the world, and is in the top 2% of all project teams on Ohloh.

VTK Development Team

and many others...

Then Came ParaView

ParaView Architecture

Support for Large Displays

ParaView is VTK•ParaView is developed by the VTK team

•The ParaView team develops in the VTK repository : all development benefits the whole community

•ParaView leverages all features of VTK

Short Demo

Problem Statement

The Good

Ability to Compute

Ability to Communicate

Remote Access

The Bad

Ability toCompute>>Ability to

Transfer>> Ability toStore/Read

State of Large Data Vis

Parallel Visualization

Visualization Resources

•LANL Vis Cluster (2001): 128 dual-core nodes with NVIDIA Quadro FX 540

•Sandia Red Rose (2005): 264 dual-core nodes with NVIDIA Quadro FX 3400

•TACC Longhorn (2009): 256 8 core nodes with 2 NVIDIA Quadro FX 5800s

General Purpose Tools

• EnSight• ParaView• VisIt• FieldView• …

Specialized Tools

•VAPOR

•ViSUS

•...

1 billion cell asteroid detonation simulation (AMR)

½ billion element weather simulation (rectilinear)

150 million elementfire simulation (unstructured)

Where to Next?

Large Data, Small Bandwidth and

Small Vis Resources

• Vis on the supercomputer• Batch• Interactive

• Co-Processing• Multi-Resolution

Streaming

Ability to Compute

Vis on Supercomputer

Vis on Supercomputer

• Compiling• Parallel IO• Scalability• Software

Rendering• Compositing• Client/server• Resource allocation

ParaView on Supercomputers

•Blue Gene L and P

•Cray Xt 3, 4 and 5

•AIX

•Linux

Visualization of VPIC resultson Kraken (image courtesy of

Bill Daughton, LANL)

Co-Processing

1111

Co-Processing

Run-Time Visualization and

Setup

Library Architecture

• Based on VTK/ParaView• C, Fortran and Python bindings• Built-in Python interpreter (optional)• Full demand-driven pipeline• Distributed computing with MPI• Extract generation• Rendering (off-screen with Mesa)

Connection with the Simulation

simulation_initialize()

coprocessor_initialize()

for t in time_steps:

do_compute()

if coprocess_needed():

do_coprocess()

coprocessor_finalize()

simulation_finalize()

Simulations – the More the Merrier

• Overflow (overset curvilinear)• Phasta (unstructured)• Acusolve (unstructured)• Helios (unstructured - AMR hybrid)• CTH (AMR)• NPIC (structured)• ...

QuickTime™ and aPNG decompressor

are needed to see this picture.

Multi-Resolution Streaming

Data-Parallel Processing

Streaming

Prioritized Streaming

Adaptive Streaming

Collaboration and Web Visualization

Remote Access

Collaboration

Web Visualization

Architecture

Comparative Visualization

180

18

01

80

18

01

80

18

01

80

18

0

Data and Meta-Data Management

Loading Meta-Data in ParaView

Analysis and Subsetting of

Meta-Data

Aperture for Data-Analysis

Conclusions

Ability to Compute and Communicate

Bottlenecks

We need multiple solutions!

Interactive on Supercomputer

Co-Processing

Multi-Resolution Streaming

It’s About Collaboration

We need frameworks to

build specialized tools

General Purpose Tools

• EnSight• ParaView• VisIt• FieldView• …

Specialized Tools

•VAPOR

•ViSUS

•...

The End