pbs gridworks - efficient application scheduling in distributed environments

Copyright © 2008 Altair Engineering, Inc. All rights reserved.

PBS GridWorks -Efficient Application Scheduling in

Distributed Environments

Dr. Jochen Krebs

Business Development Manager Enterprise Computing

Altair Engineering GmbH


About

A global software and technology company focused on:

Product Development

Advanced Computing

Enterprise Analytics


Customers

3,800+ Customers Worldwide

Automotive Aerospace Heavy Equipment Government Life/Earth Sciences Consumer Goods Oil & Gas

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.


PBS GridWorks – Overview

• What is PBS GridWorks?

• Workload Manager PBS Professional

• Application Awareness AIF

• Web Service Interface AIF

• Historical & Near-Real Time Accounting GridWorks Analytics

• Job Submission, Monitoring, & Management GridWorks Portal & Desktop

• Administrator Control Center GridWorks Portal


PBS GridWorks – Overall Development Goals

• Easy to use

• Hard to break

• Do more with less

• Keep track & plan

• Open architecture


PBS GridWorks Suite/PBS Professional

• New/enhanced features May 2008

• Tunable Formula/Eligible time

• Standing Reservations

• Integration with SGI Propack 5 Altix ICE/XE

• Cray products will be able to use the latest version of PBS Professional (X2 and XT systems)


Tunable Formula/Eligible time

• Extend the scheduler's tunable formula to include additional mathematical operations Custom Resources currently are viewable and request able/modifiable by all users, operators, and managers.

• This Formula may be in in conjunction with the standard PBS Professional scheduler

• Example: Company A would like to create per-job coefficients in their formula which are set by system defaults and not able to be requested/modified or viewed by the user.

• For example A, B, C and D below would be these coefficient resources.

• A *(Queue Priority) + B*(Job Class Priority) + C*(CPUs) + D*(Queue Wait Time)


Standing Reservations

• Provides the ability to for administrator to setup a reoccurring requirement for resources


Easy to Use Portals Increase Productivity

I push enter and my jobs run;the results come back when they’re done.

--Major Automotive OEM


Open Architecture Best-in-Class Technology

• Commitment to standards• POSIX Batch Standard, key participant

• MPI-2 standard, editor

• Grid Forum, Board of Directors

• OGSA HPC Basic Profile, contributor

• Extensible architecture• Unified job and resource architecture

• Interfaces for 3rd party integrations (API, AIF, Hooks)

Since 1999


Introduction to GridWorks Analytics

• GridWorks Analytics is designed to:• Dynamically generate and present reports against any of the PBS Accounting

information • By using the technology of HiQube Business Intelligence Software as the core

technology.

• GridWorks Analytics will be able to:• Present PBS data from a historical and Near Real Time perspectives

• The GW Analytics• Will provide a web interface as the accounting and reporting user interface• Will deliver canned reports that can be configured “out of the box”• Will provide capability for customers to create own reports


Queues vs MonthsPBS Accounting System

5,000,000

10,000,000

15,000,000

20,000,000

25,000,000

exe_ia64exe_x86_64execqmediumprio_abaqusprio_lsdynaprio_radiossprioqshorttestwork_abaquswork_hyperworkswork_lsdynawork_nastranwork_radiossprio_hyperworksprio_nastranworkworkqundefined

Queues

2006 August2006 September2006 October2006 November2006 December2007 January2007 February2007 March

2007 April2007 May

2007 June

Days

Montly "Delta Drill" by QueuePBS Accounting System

Start End

2006 September2,585

2006 December2,884

Delta299

11.6%

work_hyperworks 49.0%

work_lsdyna 30.3%

work_abaqus 7.1%

execq 98.6%

+2,884

-2,585

+2,884execq: 2,844

prioq: 40workq: -1prio_hyperworks: -1prio_lsdyna: -1prio_nastran: -2prio_abaqus: -3prio_radioss: -9test: -17work: -79work_radioss: -119work_nastran: -122work_abaqus: -183work_lsdyna: -782work_hyperworks: -1,266

-2,585

(values in thousands)

Total Turnaround Time by Queue (khours)

PBS Accounting System

-100

0

100

200

300

400

500

600

700

800 workworkqundefinedwork_nastranwork_radiosswork_abaqusprio_radiossprio_nastranwork_lsdynawork_hyperworksprio_abaqustestprio_lsdynaprio_hyperworksexecqprioqmediumshortexe_ia64exe_x86_64

0 9

337

78

3 16

328

190

675

36

362

5

552

221157

2

132

2758

106137125

267

172

0

127163140

225215

Keep track and plan: GridWorks Analytics


GridWorks Analytics Reports

Top 15 usersBy number

of jobs

Total Average Time

Delta from November to December


Future Challenges

• Performance & Scalability

• Support Power Savings

• Support for new Hardware Platforms

• Interoperability

www.pbsgridworks.comTM

http://www.pbsgridworks.com/

pbs gridworks - efficient application scheduling in distributed environments

Documents