scheduler interfaces in the askalon grid environment
DESCRIPTION
Scheduler Interfaces in the ASKALON Grid Environment. Marek Wieczorek, GGF16 Athens, February 14th. Agenda. Scheduling in the ASKALON environment Scheduler interfaces Interesting scenarios Summary. ASKALON Grid Environment. ASKALON. AGWL (Workflow Representation). User Interface. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.vdocuments.net/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/1.jpg)
Scheduler Interfaces in the ASKALON Grid
Environment
Marek Wieczorek, GGF16 Athens, February 14th
![Page 2: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.vdocuments.net/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/2.jpg)
Agenda
•Scheduling in the ASKALON environment
•Scheduler interfaces
• Interesting scenarios
•Summary
![Page 3: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.vdocuments.net/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/3.jpg)
ASKALON Grid Environment
EnactmentEngine
ResourceManagement
PerformancePredictor
MonitoringService
UserInterface
AGWL(Workflow Representation)
Grid Environment (Globus Toolkit)
ASKALON
Scheduler
![Page 4: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.vdocuments.net/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/4.jpg)
Scheduling Model
• Scientific workflow scheduler
• Separate instance for each workflow submission
• Interacts with other ASKALON services
• Dynamic scheduling (rescheduling enabled)
![Page 5: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.vdocuments.net/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/5.jpg)
Scheduler Input
• Workflow specification created by the user
• A workflow composed of activity types and dependencies between them
• ASKALON workflow description language (AGWL)
Use
r In
terf
ace
(TEU
TA
)
Sch
edule
rworkflow specificationreceive: AGWL file (XML)protocol: GT4 WS interface
![Page 6: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.vdocuments.net/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/6.jpg)
Interface to the Resource Management (GridARM)
• Resource brokerage based on resource constraints (e.g., CPU count, memory)
• Logical resources considered, i.e. software components (executables Grid/Web services)
• Advance reservation support
• Billing/pricing
Sch
edule
r
Reso
urc
eM
anagem
en
t
resource queryreceive: resource listprotocol: GT4 WS interface
service queryreceive: service descriptionprotocol: GT4 WS interface
reservation requestreceive: reservation offersprotocol: GT4 WS interface
reservation confirmationreceive: reservation ticketprotocol: GT4 WS interface
![Page 7: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.vdocuments.net/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/7.jpg)
Interface to the Enactment Engine
• Enactment Engine – advanced Job Manager
• Communication protocol based on the internal workflow representation
• Current implementation based on the Java API
Sch
edule
r
Enact
ment
Eng
ine
workflow submissionsend: full workflow graphprotocol: GT4 WS interface
rescheduling requestreceive: executed workflowsend: rescheduled workflowprotocol: GT4 WS interface
![Page 8: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.vdocuments.net/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/8.jpg)
Interface to the Performance Predictor
• Auxiliary service
• Important for proper scheduling and for advance reservations
• Model: workflow-oriented activity predictions
Sch
edule
r
Perf
orm
ance
Pre
dic
tor
prediction requestsend: activity type, resource, input parametersreceive: predictionprotocol: Java API/GT4 WS
![Page 9: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.vdocuments.net/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/9.jpg)
Interface to the Monitoring Service
• Not implemented yet
• Monitoring of resources and activities
• Control over many aspects of execution (including scheduling)
• Possible execution problems reported
• May trigger workflow rescheduling
Sch
edule
r
Monit
ori
ng
Serv
ice
event notificationreceive: eventprotocol: GT4 WS interface
scheduling eventsend: scheduling/ reschedulingprotocol: GT4 WS interface
![Page 10: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.vdocuments.net/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/10.jpg)
Workflow Processing Lifecycle
Scheduler EnactmentEngine
PerformancePredictor
AGWL workflowspecification
ResourceManagement
resource/servicequery
MonitoringService
prediction request
negotiation/reservation
scheduled workflow
rescheduled workflow
rescheduling request
event notification
![Page 11: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.vdocuments.net/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/11.jpg)
Scheduling under Uncertainty
• Full-graph scheduling
• ASKALON workflow model (loops, conditional branches, etc.) allow different possible executions
• Runtime assumption made by the scheduler
• Assumption fulfilment controlled by the enactment engine
Scheduler EnactmentEngine
originalworkflow
predicted DAG
+assumptionviolation=
rescheduling
+assumptions=
rescheduled workflow
![Page 12: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.vdocuments.net/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/12.jpg)
Advance Reservation
•Negotiations between the scheduler and the Reservation Management– The scheduler acts on behalf of the user,– Resource Management optimizes the
profit of the Grid (resource usage, fair-sharing)
•Two-stage negotiation protocol:– reservation request,– reservation confirmation
![Page 13: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.vdocuments.net/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/13.jpg)
Event-driven Dynamic Scheduling
• Different sources of events
• Scheduling and rescheduling performed in a dynamic way.
• Two types of reasons for rescheduling:– execution model
(assumption violation)– unexpected behiavor
(execution problems)
Scheduler
Enactment Engine
ResourceManagement
MonitoringService
assumption violation,execution problem
SLA violation performancebreakdown
![Page 14: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.vdocuments.net/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/14.jpg)
Summary
• Importance of the scheduler in the ASKALON environment.
•Multitude of the scheduler interfaces
•Resource negotiation and event-driven scheduling
![Page 15: Scheduler Interfaces in the ASKALON Grid Environment](https://reader036.vdocuments.net/reader036/viewer/2022062409/568150a5550346895dbeacc4/html5/thumbnails/15.jpg)
ASKALON
http://dps.uibk.ac.at/projects/http://dps.uibk.ac.at/projects/askalon/askalon/