Principles of planning the

computing resources in cloud-based

problem solving environment

K. Borodulin and G. RadchenkoSouth Ural State University

The reported paper was supported by RFBR, research project No. 15-29-07959.

Problem definition• Modern problems of computational science are

characterised by high demands on provided computing resources and complex computing tasks structure, which can be defined as a workflow.

• Also, problems of this type are characterised by usage of multivariant calculations computing task is run hundreds or thousands of times with different variations of the input parameters. 

2

3

Example workflow for mixer simulation

Design Modeler

CFX-Mesh CFX-Pre

CFX-Solver

CFX-Post

Creation or correction of a

geometric model

Creation or correction of a mesh

Creation or correction of problem

definition

Problem Simulation in CFX-Solver

Visualization in CFX-Post

Values of the optimization

criteria are not satisfactory

The accuracy of the calculation is not satisfactory

4

Problem-solving environment

• Problem-solving environment is a program system that warps and provides a problem-oriented access to computational resources to solve a specific class e-Science problems

• This limitation would allow using a domain-specific information of task to predict a computational characteristics of the task in planning and scheduling workflow’s execution, inflating the efficiency of available computing resources’ consumption in cloud computing system.

03.10.16

5

Scheduler

Cloud Platform

DiVTB Web Interface

A Driver

Simulation Results

EngineerDistributed Virtual Test Bed(DiVTB ) includes an interface for a problem statement; a driver (a set of software tools

enabling the use of cloud resources for virtual experiment);

a set of services (a set of images of virtual machines)

a set of computing resources (a cloud computing environment)

Distributed Virtual Test Bed

6

Goal and tasksThe aim of the paper is to describe the principles of computing resources planning in Cloud-based Problem Solving Environment. To gain the aim of paper:• Analyze related solutions for the planning of

execution of problem-solving workflow’s. • Define a structure of cloud system for problem-

solving environment’s deployment. • Describe a scheme of an approach for the

computing resources planning in Cloud-based problem-solving environment.

03.10.16

7

Scheduling methods in workflow systems

• Pandey, S., Wu, L., Guru, S.M., Buyya, R.: A particle swarm optimization-based heuristic for scheduling workflow applications in cloud computing environments. In: Proceedings - International Conference on Advanced Information Networking and Applications, AINA. pp. 400407 (2010).

do not use information about previous executions.• Sokolinsky, L.B., Shamakina, A. V.: Methods of resource

management in problem-oriented computing environment. Program. Comput. Softw. 42, 17–26 (2016).

• Nepovinnykh, E.A., Radchenko, G.I.: Problem-Oriented Scheduling of Cloud Applications : PO-HEFT Algorithm Case Study. 2016 39th Int. Conv. Inf. Commun. Technol. Electron. Microelectron. MIPRO 2016 - Proc. 196–201 (2016).

scheduling the workflow actions’ execution - when the task has to run to provide a minimal makespan, for example.

03.10.16

8

Model of cloud-based problem-solving

environment• Components of Cloud-based

problem-solving environment• Input data for planning the

computing resources• Levels of planning the computing

resources in cloud-based problem solving environment

03.10.16

9

Components of cloud system

Base image• OS• Middleware

• Initialization service

• agent of remote task execution

• Agent of monitoring system

Software• Application

or software for task’s execution

• Service of software execution

Service

03.10.16

10

Virtual machine

Mem

CPU

Storage

Net

VMx

Service 1

NodeCPU Mem Ne

t

Local StorageVM1

VM2

Network Storage

03.10.16

11

Input data• Executable workflow• A set of the domain-

specific arguments’ values

• QoSo Makespano A number of the computing

resourceso Maximum of the cost

• Path of Input files• Path of result files

03.10.16

12

VM1

VM2

VM3

VM4

Vm6

VM5 VM7

T1T2

Node Node

VM1

Vm6

VM3VM2 VM4

VM7VM5T1

T2

Workflow level

Virtual machine Level

Computing nodes level

Servic

e

T1

T2

Service level

03.10.16

13

Workflow levelThe workflow layer implements the transformation of the abstract workflow into the executable job. • The data sources of the input

parameters are being connected with the certain tasks and sub-flows of the workflow during the transformation.

• The abstract workflow is executable if input arguments of any task are independent of the result of another task’s execution.

03.10.16

Creation or correction of a

geometric model

Creation or correction of a mesh

Creation or correction of problem

definition

Problem Simulation in CFX-Solver

Visualization in CFX-Post

Values of the optimization

criteria are not satisfactory

The accuracy of the calculation is not satisfactory

14

Service levelThe Service layer provides assignment of particular services to the required computing resources for any task in the workflow. The Workflow predictor sends to the Workflow planner the following prediction information: • time of the task execution (on the 1 computing core);• the amount of main memory, needed for the task

execution;• maximum task scaling (how much cores can be

provided to the task);• the amount of the result data;• prediction accuracy for each value.

03.10.16

15

Virtual machine levelThis layer performs the instances selection, using the prediction of computing resources required for the certain task execution, but • Workflow executor can allocate another set of

resources for QoS satisfaction, • If the prediction accuracy is low, i.e. most likely

prediction if false, then executor choose the type of virtual machine which is default for the certain service.

03.10.16

16

Computing node levelThe Computing node planning layer maps virtual machines’ onto computing nodes on the basis of a virtual machine computing resources and a volume of node’s local storage. At this layer, planner tends to place related virtual machine from the workflow (which on this layer are presented as Task-to-VM list on the same node to reduce the amount of data are transferred between the computing nodes.

03.10.16

17

Tasks that need to be addressed

Future work:• Development of workflow applications planning

algorithm with effective virtual machines allocation and possibility of dynamically adjustment of the execution plan of the application.

• Development of a database to support an estimation of execution characteristics of calculation tasks.

• Development of a model of workflow execution in cloud computing environment.

• Development of an experimental “Problem-oriented Scheduler” system

03.10.16

18

Thanks for your attention

03.10.16