April 19, 20231

GrenchMark:A workload generator for Grids

Demo at TU Delft

A. Iosup, D.H.J. EpemaPDS Group, ST/EWI, TU Delft

April 19, 20232

Evaluating Grid schedulers performance• Performance• Qualitative metrics

supported application types, advanced fault tolerance, advanced ...

• Quantitative metricsresource consumption, system performance, scheduler functionality

• Other metricscost, single number

• Needs• Applications, workloads, more

workloads…• Benefits • Configuration testing and tuning• Functionality testing• System design and procurement• Performance testing• Runtime estimation

April 19, 20233

GrenchMark

• A systematic approach to testing Grid schedulers• A set of metrics for comparing schedulers• A set of representative applications

• Both real and synthetic• Sequential tasks, MPI tasks, Java, IBIS/Satin tasks,

batch tasks, filters/streaming tasks, chains of tasks, DAGs, general graphs, ...(?)

• Easy tools to create synthetic workloads [TODAY]

• Testing, testing, testing… [EVERY DAY]

April 19, 20234

The GrenchMark process

April 19, 20235

Outline

• Introduction• Grenchmark• Using Grenchmark• Customizing Grenchmark• Extending Grenchmark• Conclusions

[done]

[done]

[here]

April 19, 20236

wl-desc.in

• Describes the workload to be generated; input file for wl-gen.py

• Fixed, non-fixed, semi-fixed execution sites• Applications inter-arrival time: Poisson, uniform, normal,

exp, hyperexp2, Weibull, log normal, gamma, …

• Over 35 supported applications, in 3 categories: sequential, MPI, Ibis

April 19, 20237

Grenchmark process

$ ./wl-gen.py wl-desc.in $ ./wl-submit.py out/wl-to-submit.wl

12

1

2

TODO

• Sample run:• 95 jobs / 411 components• 95 files / 32 directories• 300KB data

• Sample run:• multi-threaded – submission delay +/- 0.01s• 95 JDFs

April 19, 20238

Preliminary notions

• Job, workload, workload unit• Job = Set of components (co-allocation support)

[Job = one program execution / the basic scheduling unit / … ]

• Workload = Set of Jobs• Workload Unit = Set of jobs with the same

property, generated in the same step (definition useful only for workload generator)

• Other• JDF = Job description file• Inter arrival time = time between two consecutive

jobs arrive in the system

April 19, 20239

Describing the workload: Jobs wl-desc.in

ID Times UnitType SiteType TotalComps PerSiteDesc [I]ArrivalTimeDistr OtherInfo 0 1 sser single 1 *:? U(0,1000) -

• Generate Times x Jobs or Times x Sets of Jobs• One Job = k Components (here: 1 -- single)

April 19, 202311

Describing the workload: Jobs wl-desc.in

ID Times UnitType SiteType TotalComps PerSiteDesc [I]ArrivalTimeDistr OtherInfo 0 1 sser single 1 *:? U(0,1000) -

• Generate Times x Jobs or Times x Sets of Jobs• One Job = k Components (here: 1 -- single)

April 19, 202312

Describing the workload: Units wl-desc.in

ID Times UnitType SiteType TotalComps PerSiteDesc ArrivalTimeDistr OtherInfo? 1 wl:poisson - - - Poisson(500) File=wl.xin

wl.xinRandomWorkload=falseNComponents=4;8ComponentSize=8;16Submitter=drunnerApplication=pi-g2-gm

• Generate Times x Jobs or Times x Sets of Jobs• wl: prefix – workload unit generator [Hashim’s jobs]

April 19, 202313

Demo 4: Generating one workload unit

• Hashim’s sample workload:• Poisson-g2-gm (Globus MPICH-G2)• Components: 4 and 8• Component Sizes: 8 and 16• IAT Poisson(5s)• Run with drunner

• Total: 4 jobs, 24 components

April 19, 202314

Demo 5:Generating a full workload• Hashim’s full workload, 8 times:

• Poisson-g2-gm (Globus MPICH-G2) and findpi-gm• Components: 4 and 8• Component Sizes: 4/8 and 16• IAT Poisson(5s)• Run with drunner (poisson) / krunner (pi)• Submit time 1 day

• Generate:

• Submit:

• Total: 3200 jobs, 19200 components / 3k files, 4k dirs

• Timing: 30s generate / 86,400s submit (1 day, eh?)

$ wl-gen.py --duration=86400000 wl-desc.in

$ wl-submit.py --onefile out/wl-desc.in

April 19, 202315

Outline

• Introduction• Grenchmark• Using Grenchmark• Customizing Grenchmark• Extending Grenchmark• Conclusions

[done][done]

[here]

[done]

April 19, 202316

Customizing the workload generator Command-line parameters• wl-gen.py –help … displays a comprehensive help screen

• wl-gen.py –o outdir … creates the workload in directory outdir

• wl-gen.py –d 50000 … sets the total submission duration to 50s

• wl-gen.py –j my-jdf … uses my-jdf to write JDFs

• wl-gen.py –o out-1 wl-desc.in my favourite command

April 19, 202317

Customizing the submission tool Command-line parameters

• wl-submit.py –help … displays a comprehensive help screen

• wl-submit.py –t 10 … uses 10 threads to submit

• wl-submit.py --nosubmit … just prints the submission commands

• wl-submit.py --onefile … output job submit (runner) stderr and stdout to one file each (concatenated)

• wl-submit.py out/wl-to-submit.wl my favourite command

April 19, 202318

Outline

• Introduction• Grenchmark• Using Grenchmark• Customizing Grenchmark• Extending Grenchmark• Conclusions

[done][done]

[here]

[done]

[done]

April 19, 202319

Extending the workload generator (1) Write your own Job Generators

1. Create your-job-generator.py in apps/

2. Place a Generator attribute, organized as a Key=Value list, one per line, with at least Name=your-job-type defined. Example:Generator=""" Name=your-job-type Author=You R. Name Copyright=copyright © 2005 You R. Name. All rights reserved. ""“

3. Create a function generateWorkloadUnit, which takes in

UnitDir, UnitID, WLUnit, SubmitDurationMS, bGenerateRandom

4. Make generateWorkloadUnit return a dictionary with keys ‘info’ and ‘jobs’, with appropriate data each

5. Use the generator in wl-desc.in by putting your-job-type in the UnitType field

April 19, 202320

Extending the workload generator (2) Write your own Unit Generators

1. Create your-job-generator.py in apps/

2. Place a WLGenerator attribute, organized as a Key=Value list, one per line, with at least Name=your-job-type defined. Example:WLGenerator=""" Name=your-workload-type Author=You R. Name Copyright=copyright © 2005 You R. Name. All rights reserved. ""“

3. Create a function generateWorkload, which takes in UnitDir, UnitID, WLUnit, SubmitDurationMS, bGenerateRandom

4. Make generateWorkloadUnit return a dictionary with keys ‘info’ and ‘jobs’, with appropriate data each

5. Use the generator in wl-desc.in by putting your-job-type in the UnitType field

April 19, 202321

Extending the workload generator (2) Unit Generators in C, Ruby, Perl, ... ?1. Same as previous: generate your-job-generator.py in

apps/

2. Replace step 4 with:

1. Interface your Python code, so that it runs your C, C++, Ruby, Perl, … application (os.exec, os.popen, …)

2. Save your C, C++, Ruby, Perl, … application information to a text file, or use pipes to transfer info

3. Read the saved information.

April 19, 202322

Extending the workload generator (3) Write your own JDF Writers

1. Create your-jdf-writer.py in jdfprint/

2. Place a JDFGenerator attribute, organized as a Key=Value list, one per line, with at least Name=your-job-type defined. Example:JDFGenerator=""" Name=your-jdf-writer-type Author=You R. Name Copyright=copyright © 2005 You R. Name. All rights reserved. ""“

3. Create a function generateJDF, which takes in OutFileName, ListOfComponents

4. Make generateJDF write the components to the specified file, with appropriate data each

5. Use the generator by calling wl-gen.py with the –j parameter

April 19, 202323

Conclusions and future work

• Grenchmark easily generates diverse workloads of Grid applications

• KOALA developerssuccessfully used GrenchMark to test KOALA

• Future workdo you have specific Grid applications types that you would like to test? Try Grenchmark!

April 19, 202324

Thank you!

Questions? Remarks? Observations? All welcome!

Grenchmark http://grenchmark.st.ewi.tudelft.nl/ [10x Paulo]

Alexandru IOSUPTU Delft

A.Iosup@ewi.tudelft.nl

http://www.pds.ewi.tudelft.nl/~iosup/index.html [google: “iosup”]