http://cyberaide.org 1

Experiment Management with Microsoft Project

Gregor von LaszewskiLeor E. Dilmanian

http://www.cyberaide.org

Acknowledgement: NSF NMI, CMMI, DDDAS

Outline• Introduction• Cyberaide.org• Management Requirements of e-Science• Microsoft Project for Managing Tasks• Extending Microsoft Project for e-Science• Conclusion• References• Acknowledgements

http://cyberaide.org 2

Introduction - Background

• E-Science is computationally or data intensive science, carried out in a distributed computing environment.

• Cyberinfrastructure is a research environment supporting advanced data acquisition and information processing services over the internet.

• Experiment Management is the management of a large number of experiments over Cyberinfrastructure

http://cyberaide.org 3

Introduction – Motivation • There is a high entry barrier into Grid computing.

• Experiment Management on the Grid or Cyberinfrastructure is a complicated affair.

• Issues include:– Application Design– Scheduling of Large Scale Resources– Orchestration of Activity– Monitoring Execution and Quality of Service– Data Acquisition

http://cyberaide.org 4

e-Science Experiment Management

http://cyberaide.org 5

QueuingSystems

Globus(Condor,Unicore, …)

Social NetworksClouds

Cyberaide

http://cyberaide.org 6

• is a framework used to simplify Cyberinfrastructure development.

e-Science Requirements

• Focus on Science– Well known, familiar user

interface– Easy to Navigate– Effectively manage large

volumes of information– Advanced features:

• Time based workflows with soft deadlines.

• Calendar & resource views, toolbars, forms & graphical indicators…

• Integrated User Experience– Design

• Easily specify a distributed application.

– Execution• Seamless scheduling and

execution on the Grid.– Monitoring

• Progress & Quality of Service.

http://cyberaide.org 7

Microsoft Project

• Is used for planning a project.

• Workflow or Project – a set of operations which contain the

following:

– Task – a unit of work.

– Dependency – among two tasks, used to specify ordering.

– Resource – people or machines used to carry out work.

– Assignment /Mapping – of resources to tasks, to execute the

workflow.

• A task becomes a job.

http://cyberaide.org 8

Microsoft ProjectMicrosoft Project is used to by project managers to manage resources and tasks. Shown above: a Finance and Accounting System Implementation project.

http://cyberaide.org 9

Source: Microsoft Project

Cyberaide Project• Cyberaide contains a prototype called Cyberaide Project.

– It uses the Microsoft Project software package as an elementary component.

– Based on the methodology of project management software.

– It is used to…• Design a high level application.• Execute the workflow on the Grid.• Track progress and Quality of Service during execution.

http://cyberaide.org 10

Cyberaide Project

http://cyberaide.org 11

Console WindowProxy Initialization

Task TableGantt Chart

Technologies• Visual Studio Professional 2008• C# and Visual Basic• Microsoft Project 2007 Professional• Microsoft Project Primary Interoperability Assembly• Java CoG Kit – Cyberaide

– in the near future, Cyberaide will the functionality the Java Cog Kit currently contains, but in additional programming languages.

• Apache CLI• Cyberaide Shell Mediator

– in the near future, this is how interaction with the Grid and Clusters will be working.

• WSI – web service interoperability

http://cyberaide.org 12

http://cyberaide.org 13

Cyberaide ProjectThe CyberaideProject Class• Written in C#• Console Application• Execution Shell• Runs user commands

The Command Interface• Specify a command• Register with the Shell• Uses Apache CLI for parsing• Manipulates instance of MSP

Registers/Executes

Console Application•Console Application•Launches Microsoft Project•Accepts Commands

Command Interface•Apache CLI•Manipulate MSP

http://cyberaide.org 14

Command Interface

http://cyberaide.org 15

CommandsA set of commands allow the user to manipulate, execute, and manage workflows.

Commands fall into the categories of Workflow Manipulation, Workflow Execution, and Utility Functions. Command Description Command Type

task Add a Task. Workflow Manipulationmap Map Task/Resource. Workflow Executiondel Delete a Task. Utility Functionedit Edit Task Information.dep Add a Task Dependency.rdep Remove a Task Dependency.res Add a Resource.run-all Run a Workflow.save Save a Workflow in JSON.load Load a Workflow in JSON.list List all Tasks with a valid name.find Find a Task by its name.

Cyberaide IntegrationAn easy client interface provides access through a mediator service to a variety of different services.

http://cyberaide.org 16

MS Project

MS Project

RubyRuby PythonPython JavaJava

Mediator & AbstractionsMediator & Abstractions

TeraGridTeraGrid AmazonCloud

AmazonCloud facebookfacebook

iGoogleiGoogle

SocialNetworks

SocialNetworks

GlobusGlobus

GPIR, NWS, …

GPIR, NWS, …

CloudCloudGridGrid

Clie

ntS

ervi

ces

JavaScriptJavaScriptC#C#

Loading

http://cyberaide.org 17

Design a Project

http://cyberaide.org 18

Execute & Monitor

http://cyberaide.org 19

Task Output

http://cyberaide.org 20

Project Completion

http://cyberaide.org 21

Summary

• Microsoft Project: Familiarity, Usability, Productivity– Task Table in Spreadsheet Format– Easy Navigation & Viewing

• Resource Mapping -> Jobs• Integration with TeraGrid

– through CoG Kit to TeraGrid– We have are developing a Grid mediator (e.g. a

Super Gram Service)

http://cyberaide.org 22

Conclusion• Cyberaide Project can be used to design applications in e-

Science– Supports the process of developing, executing and monitoring

workflows for the Grid

• Familiarity, simplicity and user friendliness are the major considerations– Useful for the casual Grid user familiar with or using Microsoft Project

• Spend less time learning software, and more time making discovery– Increases productivity

http://cyberaide.org 23

laszewski@gmail.com

References[1] von Laszewski, G., & Dilmanian, L. E-Science Project and Experiment Management with Microsoft Project. Accepted for publication at

GCE08 at SC08.[2] G. von Laszewski, J. Gawor, P. Lane, N. Rehn, M. Russell, and K. Jackson, “Features of the Java Commodity Grid Kit,” Concurrency and

Computation: Practice and Experience, vol. 14, pp. 1045–1055, 2002. [Online]. Available: http://www.mcs.anl.gov/gregor/papers/vonLaszewski--cog-features.pdf

[3] G. von Laszewski, B. Ruscic, K. Amin, P. Wagstrom, S. Krishnan, and S. Nijsure, “A Framework for Building Scientific Knowledge Grids Applied to Thermochemical Tables,” The International Journal of High Performance Computing Applications, vol. 17, no. 4, pp. 431–447, Winter 2003. [Online]. Available: http://www.mcs.anl.gov/gregor/papers/vonLaszewski--knowledge-grid.pdf

[4] G. von Laszewski, “Java CoG Kit Workflow Concepts,” Journal of Grid Computing, Jan. 2006, http://dx.doi.org/10.1007/s10723-005- 9013-5. [Online]. Available: http://www.mcs.anl.gov/gregor/papers/vonLaszewski-workflow-taylor-anl.pdf

[5] G. von Laszewski, A. J. Younge, X. He, and F. Wang, “GridShell: Interactive Task Management for Grid and Cluster Computing,” (im preparation for review), Nov. 2008.

http://cyberaide.org 24

laszewski@gmail.com

Acknowledgements

• NSF DDDAS• NSF CMMI 0540076• NSF SDCI NMI 0721656

http://cyberaide.org 25