ASeminar

onImplementation

Of

Exploiting Dynamic Resource Allocation

forEfficient Parallel Data Processing in the Cloud

Presented By: Hussain khan MdM.Tech(S.E) IIyear

10311D2506

Under The Guidance of

Mrs. Aruna Varanasi,H.O.D of C.S.E Dept,

Snist, Ghatkesar

Description:• In recent years ad-hoc parallel data processing has emerged to be one

of the killer applications for Infrastructure-as-a-Service (IaaS) clouds.

• Major Cloud computing companies have started to integrate frameworks for parallel data processing in their product portfolio, making it easy for customers to access these services and to deploy their programs.

• However, the processing frameworks which are currently used have been designed for static, homogeneous cluster setups and disregard the particular nature of a cloud.

• Consequently, the allocated compute resources may be inadequate for big parts of the submitted job and unnecessarily increase processing

time and cost.

• In this paper we discuss the opportunities and challenges for efficient parallel data processing in clouds and present our research project named NEPHELE.

• It is the first data processing framework to explicitly exploit the dynamic resource allocation offered by today’s IaaS clouds for both, task scheduling and execution.

• Particular tasks of a processing job can be assigned to different types of virtual machines which are automatically instantiated and terminated during the job execution.

• Based on this new framework, we perform extended evaluations of Map Reduce-inspired processing jobs on an IaaS cloud system and compare the results to the popular data processing framework Hadoop.

TECHNIQUE USED Nephele framework:  The actual execution of tasks which a Nephele job

consists of is carried out by a set of instances.

Each instance runs a so-called Task Manager (TM).

A Task Manager receives one or more tasks from the Job Manager at a time, executes them.

After that informs the Job Manager about their completion or possible errors.

Unless a job is submitted to the Job Manager, we expect the set of instances (and hence the set of Task Managers) to be empty.

Upon job reception the Job Manager then decides, depending on the job’s particular tasks, how many and what type of instances the job should be executed on, and when the respective instances must be allocated/ de-allocated to particular instance

It ensures a continuous but cost-efficient processing. Our current strategies for these decisions are

highlighted at the end of this section.

SYSTEM ARCHITECTURE

Description: Here the system architecture depicts that

Through client page we can give HTTP request to the servlet and retrieve resources from persistent storage by data access object.

From servlet job manager will get the HTTP response and handover the task to the task manager.

Task manager will process the task and again forward the response to the client page

DATA FLOW DIAGRAM

Description:From the data flow diagram it clearly shows that: User can connect with the cloud after checking the credentials through login.

Through the main page client can give request through main page then this job will redirect to job manager for scheduling and allocating.

Then proper task manager will perform particular task by getting the necessary resources from the persistent storages

Finally the result is being sent to the client

Design Work

List of Modules in the project Login Client request Job manager Resource storage Task manager Process the task

Implementation WorkHome Page

1)Login Page

Registration PageA new user registers details and gets authentication

Upon Successful Registration a message would be displayed

Then the User Logins with so created mail Id and Password

Module: 2)Client RequestThe client provides his request to job Manager by entering the data in the empty space provided

Sample Request created by the client

After creating the Request, by clicking upon the Submit button, the request would be sent directly to the Job Manager

Code for creating Client Request• protected void btnSubmit_Click(object sender, EventArgs e)

• {

•

• string Name = Session["username"].ToString();

• //ObjClientBO.Request = txtClient.Text;

• try

• {

• SqlConnection con = new SqlConnection(constring);

• con.Open();

• SqlCommand cmd = new SqlCommand("Insert into tblClientRequest(Name,Request)values('" + Name + "','" + txtClient.Text + "')", con);

• SqlDataAdapter da = new SqlDataAdapter(cmd);

• cmd.ExecuteNonQuery();

•

• ScriptManager.RegisterStartupScript(Panel1,Panel1.GetType(), "MyScript", "alert('Your Request has been sent successfully');", true);

• txtClient.Text = "";

• }

• catch (Exception ex)

• {

• ex.Message.ToString();

• }

• clear();

Module : 3) Job ManagerJob Manager logins to Cloud controller to view the client requests and process the requests.

A window opens displaying the “View Request” Tab, to view the requests sent by the client

Then the request sent by different would be displayed.

Then the job manager checks for the resources needed to process the tasks

A window opens displaying the services available by the job manager

If the resources are available, then the Job Manager selects the particular resources depending upon the request sent by the clients.

• Code for Job manager• public partial class JobManager : System.Web.UI.Page

• {

•  

• string constring = @"Data Source=HOME-PC;Initial Catalog=CloudComputing;Persist Security Info=True;User ID=sa;Password=srinivas";

• CloudBO.JobManager ObjJobBO = new CloudBO.JobManager();

• int id = 0;

• protected void Page_Load(object sender, EventArgs e)

• {

• if (!IsPostBack)

• {

• data();

• plat();

• }

• }

•  

• protected void btnViewRequest_Click(object sender, EventArgs e)

• {

• Panel4.Visible = true;

• Panel1.Visible = false;

• }

• protected void btnAvailableResourse_Click(object sender, EventArgs e)

• {

• Panel1.Visible = true;

• Panel4.Visible = false;

• lblmessage.Visible = true;

• lblmessage.Text = "IF RESOURCE IS UNAVAILABLE PLEASE REGISTER IN TO CLOUD REGISTRATION";

• lblmess.Visible = true;

• lblmess.Text = "IF RESOURCE IS UNAVAILABLE PLEASE REGISTER IN TO CLOUD REGISTRATION";

• }

• protected void btnsoftware_Click(object sender, EventArgs e)

• {

• ddlSW.Visible = true;

• btnCloudRegistration.Visible = true;

• }

•  

• protected void btnPlatform_Click(object sender, EventArgs e)

• {

• ddlPF.Visible = true;

• btnplatCloudReg.Visible = true;

• }

•  

• protected void btnSubmit_Click(object sender, EventArgs e)

• {

• // ObjJobBO.DdlSoftware = ddlSW.SelectedItem.Text;

• // ObjJobBO.DdlPlatform = ddlPF.SelectedItem.Text;

• // ObjJObBAL.InsertJobManager(ObjJobBO);

• try

• {

• id = Convert.ToInt32(lblrequestid.Text);

• SqlConnection con = new SqlConnection(constring);

• con.Open();

• SqlCommand cmd = new SqlCommand("Insert into tblJobManager(ddlSoftware,ddlPlatform,RequestID,Request,Name) values('" + ddlSW.SelectedItem.Text + "','" + ddlPF.SelectedItem.Text + "'," + id + ",'" + lblrequestmesg.Text + "','" + lblrequestname.Text + "')", con);

• cmd.ExecuteNonQuery();

• }

Module : 4) Resource storageIf the resources are unavailable, then the Job Manager gets additional resources from the Cloud Persistent storage by providing the authenticated details for accessing the storage

The job Manager provides his authenticated details for accessing the storage of the Cloud

Upon successful registration a message would be displayed

A window would be displayed for searching the needed resources

Then the Job Manager enters his needed resources and searches for availability

Job manager selects all the resources depending upon the necessity

Upon the successful selection the resource would be added to the existence resources of the job manager

Then the assigned resources for particular request would be displayed

Module : 5) Task ManagersDepending upon the number of request sent by a particular client the Job Manager selects the tasks and assigns the tasks to task managers.

Code for Task Managers• public partial class AssignedTask : System.Web.UI.Page

• {

• string constring = ConfigurationManager.ConnectionStrings["PharmaConnection"].ToString();

• //string constring = @"Data Source=HOME-PC;Initial Catalog=CloudComputing;Persist Security Info=True;User ID=sa;Password=srinivas";

• protected void Page_Load(object sender, EventArgs e)

• {

• }

• protected void ddljob_SelectedIndexChanged(object sender, EventArgs e)

• {

• if (ddljob.SelectedItem.Text == "--select--")

• {

• btnTask1.Visible = false;

• btnTask2.Visible = false;

• Button1.Visible = false;

• }

Module: 6) Processing the TasksBy clicking upon the individual task managers three windows would be opened parallel in the

browser.• Task 1: To create a Student Registration page for

client as first request .

• Task2 : To create a Student Profile form for client as second request.

• Task3: To display the so entered details by using database

Data Base Tables

Sample Data base For Job Manager

ADVANTAGES

• Reduce the workload on cloud.

• Fast processing.

• Easy to maintain and troubleshoot.

• Easy to design as the infrastructure we need.

Thank You