enabling grids for e-science batch system integration jan just keijser nikhef amsterdam egee iii sa3...
DESCRIPTION
EGEE III SA3 Kickoff Meeting, May Targets & Milestones First milestone Take part in writing MSA2.3, 'Strategy and roadmap of the EGEE multi-platform support’ Near-Future targets Full support for 2 extra batch systems in gLite (apart from PBS and LSF) Both lcg-CE and CREAM-CE supported Develop a Batch System Integration Checklist (BaSIC ;-)) Write a 'cookbook' How to integrate Your Favourite Batch Systems in 6 Easy StepsTRANSCRIPT
Enabling Grids for E-sciencE
Batch System IntegrationJan Just Keijser
Nikhef AmsterdamEGEE III SA3 Kickoff Meeting, May 15-16 2008
EGEE III SA3 Kickoff Meeting, May 15-16 2008 2
Outline
• Targets/Milestones• Work done so far• Next steps• Issues• << Talk by Christan >>• Other Interesting System Integration Issues
EGEE III SA3 Kickoff Meeting, May 15-16 2008 3
Targets & Milestones
First milestone• Take part in writing MSA2.3,
'Strategy and roadmap of the EGEE multi-platform support’
Near-Future targets• Full support for 2 extra batch systems in gLite (apart from PBS and LSF)• Both lcg-CE and CREAM-CE supported• Develop a
Batch System Integration Checklist (BaSIC ;-))• Write a 'cookbook'
How to integrate Your Favourite Batch Systems in 6 Easy Steps
EGEE III SA3 Kickoff Meeting, May 15-16 2008 4
Work done so far...
• PIC:– Condor integration with lcg-CE: see Christian’s talk– CREAM-CE integration in the works
• CESGA: – Sun Grid Engine (SGE) integration with lcg-CE– SGE already certified by SA3
minor note: info-dynamic-scheduler ?
• Nikhef:– Inventorized existing batch system integration for Condor, SGE,
PBS– Set up testbeds for off-developer-site testing of BSI
CentOS 5.1 host OS, Xen 3.2, LVM Testbed consists of Batch System + WNs, CE, RB/WMS, MON, UI
EGEE III SA3 Kickoff Meeting, May 15-16 2008 5
Inventorizing...
• EGEE-UF03 in Clermont-Ferrand was very useful:
EGEE III SA3 Kickoff Meeting, May 15-16 2008 6
Next steps...
At Nikhef– we will set up a testbed with Batch System Of Choice– Next, we will use YOUR installation guidelines/Twiki pages on
how to integrate this batch system with gLite
What we need from you:– installation packages– installation guides/Twiki pages– Time & effort– your cooperation
EGEE III SA3 Kickoff Meeting, May 15-16 2008 7
Longer-term ideas
• Possible ARC integration– ARC-glite interface is at CE-level– Thus, ARC integration could be treated as
'Just Another Batch System'• Other batch systems
– ??• Other platforms
– 32bit Linux flavours, RPM-based, .deb-based– 64bit Linux flavours, RPM-based, .deb-based– Others??
EGEE III SA3 Kickoff Meeting, May 15-16 2008 8
Issues
• Which site will develop and support LSF integration?
• Interaction with JRA1 developers is limited thus far– Who will/should own the interface between a batch
system and the affected gLite components?– How to address issues like the current blahp
interface issue, where PBS+LSF are treated differently from Condor
• CREAM-CE is very much a “moving target” • Mismatch between FTEs available for doing
coordination and FTEs available for technical/development work
EGEE III SA3 Kickoff Meeting, May 15-16 2008 9
Concluding Remark
To make Batch System Integration a success it is vital that we have stable interfaces between the Batch System and the glite Middleware and that these are “owned” by SA3
DevelopmentJRA1
Batch system co-ordinationNikhef, SA3
OperationsSA1
SGECESGA, ..
.
LSF CondorPIC
Torque/MauiNikhef
Batch system integration