a case for economy grid architecture for service oriented grid computing authors: rajkumar buyya,...
Post on 13-Dec-2015
216 Views
Preview:
TRANSCRIPT
A Case for Economy Grid A Case for Economy Grid Architecture for Service Architecture for Service
Oriented Grid ComputingOriented Grid Computing
Authors: Authors: Rajkumar Buyya, David Abramson & Jonathan GiddyRajkumar Buyya, David Abramson & Jonathan Giddy
Presenter: Diego LopezPresenter: Diego Lopez
Agnostic: Djuradj BabicAgnostic: Djuradj Babic
June 12, 2006June 12, 2006 22
OutlineOutline
IntroductionIntroduction Grid Economy and Resource Management Grid Economy and Resource Management
IssuesIssues Economy Models and Related WorkEconomy Models and Related Work GRACEGRACE Resource Trading and Scheduling Resource Trading and Scheduling
ExperimentationExperimentation Conclusion and Future WorkConclusion and Future Work
June 12, 2006June 12, 2006 Source: Buyya, R http://www.buyya.comSource: Buyya, R http://www.buyya.com 33
1. Introduction1. Introduction“ “ We expect that an economy driven We expect that an economy driven
approach to resource management and approach to resource management and scheduling will make a great impact on the scheduling will make a great impact on the
eventual success and widespread eventual success and widespread adoption of the Grid in day-to-day adoption of the Grid in day-to-day
computational activities.” computational activities.”
Dr. BuyyaProf. Abramson
Researcher Giddy
June 12, 2006June 12, 2006 44
1. Introduction1. Introduction
Grid environment is complex ($$$)Grid environment is complex ($$$) Different access cost modelsDifferent access cost models Dynamically varying loads and availability Dynamically varying loads and availability
conditionsconditions Use of economic models in the Grid to Use of economic models in the Grid to
encourage participation and wide-scale encourage participation and wide-scale adoptionadoption
Proposal of computational economy Proposal of computational economy framework that leverage existing Grid sitesframework that leverage existing Grid sites
June 12, 2006June 12, 2006 Source: Buyya, R A Case for Economy Grid ArchitecSource: Buyya, R A Case for Economy Grid Architecture for Service Oriented Grid Computing (Pg. 2)ture for Service Oriented Grid Computing (Pg. 2)
55
View of Economic GridView of Economic Grid
June 12, 2006June 12, 2006 66
2. Grid Economy and Resource 2. Grid Economy and Resource Management IssuesManagement Issues
Establish policies that promote Grid Establish policies that promote Grid resource sharingresource sharing
2 key players in Grid economy2 key players in Grid economy Resource providers (GSP)Resource providers (GSP) Resource consumers (GRB)Resource consumers (GRB)
Consumers interact with brokers to Consumers interact with brokers to express their express their budgetbudget and and deadlinedeadline requirements from the requirements from the GridGrid
June 12, 2006June 12, 2006 77
Proposal of GRACEProposal of GRACE Grid Architecture for Computational Grid Architecture for Computational
EconomyEconomy Leverage of existing infrastructures:Leverage of existing infrastructures:
Globus/LegionGlobus/Legion Condor/GCondor/G
Provide an infrastructure that allows for:Provide an infrastructure that allows for: Info/Market directory for publicizing entitiesInfo/Market directory for publicizing entities Model for determining value of resourcesModel for determining value of resources Resource pricing schemesResource pricing schemes Accounting, Billing and Payment mechanismsAccounting, Billing and Payment mechanisms
June 12, 2006June 12, 2006 88
3. Economy Models and Related 3. Economy Models and Related WorkWork
Possible economic models for resource trading Possible economic models for resource trading and pricing strategiesand pricing strategies
Commodity MarketCommodity Market Posted PricePosted Price BargainingBargaining Tendering/Contract-NetTendering/Contract-Net AuctionAuction Bid-based Proportional Resource SharingBid-based Proportional Resource Sharing Community/Coalition/Bartering Community/Coalition/Bartering
June 12, 2006June 12, 2006 99
Examples of Computational Examples of Computational Economy SystemsEconomy Systems
SYS_NAMESYS_NAME ECO_MODELECO_MODEL PLATFORMPLATFORM
Mariposa (’96)Mariposa (’96) BiddingBidding Dist.DbaseDist.Dbase
Mungi (’02)Mungi (’02) CommodityCommodity Storage serversStorage servers
Popcorn (’98)Popcorn (’98) AuctionAuction Web browserWeb browser
Mojo NationMojo Nation Credit-based or Credit-based or barteringbartering
Network Network storagestorage
Java MarketJava Market QoS based QoS based computationalcomputational
Web browser Web browser (applets)(applets)
June 12, 2006June 12, 2006 1010
4. GRACE4. GRACE
Use of well-adopted Grid technologies, Use of well-adopted Grid technologies, Globus/CondorGlobus/Condor
Development of middleware services for Development of middleware services for resource trading using different economic resource trading using different economic modelsmodels
Development of advanced user-centric Development of advanced user-centric Grid resource brokersGrid resource brokers
June 12, 2006June 12, 2006 http://ipdps.cc.gatech.edu/2000/papers/Abramson.pdhttp://ipdps.cc.gatech.edu/2000/papers/Abramson.pdff
1111
4.1 Grid Resource Broker (GRB)4.1 Grid Resource Broker (GRB)
Mediator between user and grid resourcesMediator between user and grid resources *Nimrod – parametric modeling language *Nimrod – parametric modeling language Use of Nimrod/G broker (superscheduler)Use of Nimrod/G broker (superscheduler)
Job Control AgentJob Control Agent Schedule AdvisorSchedule Advisor Grid ExplorerGrid Explorer Trade ManagerTrade Manager Deployment AgentDeployment Agent
June 12, 2006June 12, 2006 Src: http://www.buyya.com/ecogrid/Src: http://www.buyya.com/ecogrid/ 1212
Nimrod/G BrokerNimrod/G Broker
Grid Middleware
Nimrod/G ClientNimrod/G ClientNimrod/G Client
Grid Information Server(s)
Schedule Advisor
Trading Manager
Nimrod/G Engine
GridStore
Grid Explorer
GE GISTM TS
RM & TS
Grid Dispatcher
RM: Local Resource Manager, TS: Trade Server
Globus, Legion, Condor, etc.
G
G
CL
Globus enabled node.Legion enabled node.
LCondor enabled node.
RM & TSRM & TS
June 12, 2006June 12, 2006 1313
4.2 Economy Grid Middleware in 4.2 Economy Grid Middleware in Globus ContextGlobus Context
Trade Server (TS) – maximize the Trade Server (TS) – maximize the resource utility and profit for its ownerresource utility and profit for its owner
Pricing Policies – define prices for Pricing Policies – define prices for resources based on economic models resources based on economic models previously mentionedpreviously mentioned
Resource Accounting and charging – Resource Accounting and charging – tracking resource usage for billing and tracking resource usage for billing and auditing purposesauditing purposes
June 12, 2006June 12, 2006 1414
4.3 Grid Open Trading Protocols 4.3 Grid Open Trading Protocols and Deal Templateand Deal Template
Establish rules and format for exchanging Establish rules and format for exchanging commands between a GRACE client commands between a GRACE client (Trade Manager) and Trade Server(Trade Manager) and Trade Server
Deal Template (DT) containsDeal Template (DT) contains CPU timeCPU time Storage requirementsStorage requirements Initial offerInitial offer
This trading overhead can be reduced if This trading overhead can be reduced if prices are announced via GISprices are announced via GIS
June 12, 2006June 12, 2006 1515
4.4 4.4 PricingPricing, Accounting, and , Accounting, and Payment MechanismsPayment Mechanisms
N-ways to determine resource pricingN-ways to determine resource pricing Fixed price model (no QoS like today’s www)Fixed price model (no QoS like today’s www) Usage timing (peak, off-peak)Usage timing (peak, off-peak) Bulk purchaseBulk purchase Demand and supplyDemand and supply Loyalty of customers (i.e. frequent flyer miles)Loyalty of customers (i.e. frequent flyer miles) Calendar basedCalendar based
June 12, 2006June 12, 2006 1616
4.4 Pricing, 4.4 Pricing, AccountingAccounting, and , and Payment MechanismsPayment Mechanisms
Service items to be chargedService items to be charged CPU timeCPU time MemoryMemory Storage usedStorage used Software and Libraries accessed (ASP)Software and Libraries accessed (ASP)
Access to these services can be chargedAccess to these services can be charged IndividuallyIndividually Combination (costing matrix)Combination (costing matrix)
June 12, 2006June 12, 2006 *not incorporated into GRACE described in this pap*not incorporated into GRACE described in this paperer
1717
4.4 Pricing, Accounting, and 4.4 Pricing, Accounting, and Payment MechanismsPayment Mechanisms
Prepaid – purchase credits from GSP or Prepaid – purchase credits from GSP or Grid BankGrid Bank
Use and pay later (like electricity)Use and pay later (like electricity) Pay as you go (wireless calling cards)Pay as you go (wireless calling cards) Grants basedGrants based *Billing services handled by 3*Billing services handled by 3rdrd party: party:
NetChequeNetCheque PaypalPaypal
June 12, 2006June 12, 2006 1818
4.5 System Prototype & Demo 4.5 System Prototype & Demo ExperiencesExperiences
Prototype of the Nimrod/G resource Prototype of the Nimrod/G resource brokering demo held during HPDC 2000brokering demo held during HPDC 2000
Parameter study experiment performed Parameter study experiment performed over Grid resources located in both over Grid resources located in both Australia and the USAustralia and the US
Ability to change deadline and budget to Ability to change deadline and budget to trade-off cost vs. timeframe to illustrate trade-off cost vs. timeframe to illustrate Grid marketplace dynamicsGrid marketplace dynamics
June 12, 2006June 12, 2006 1919
5. Resource Trading and 5. Resource Trading and Scheduling ExperimentationScheduling Experimentation
Experiment to test operation of Grid Trade Experiment to test operation of Grid Trade Server across 5 systems (165 jobs)Server across 5 systems (165 jobs)
Use of Posted Price Market Model for the Use of Posted Price Market Model for the Nimrod/G brokeringNimrod/G brokering
Runs during peak time vs. off-peak timeRuns during peak time vs. off-peak time Access price expressed in Grid units per Access price expressed in Grid units per
CPU second (G$)CPU second (G$) Resource/service price provided by Resource/service price provided by
GRACE frameworkGRACE framework
June 12, 2006June 12, 2006 2020
Economy Grid ResultsEconomy Grid Results Cost-Optimization algorithm successfullyCost-Optimization algorithm successfully
Minimized artificial access cost per resourceMinimized artificial access cost per resource Completed within one-hour deadlineCompleted within one-hour deadline
Initial calibration phase ensures Initial calibration phase ensures completion within budget/time constraintscompletion within budget/time constraints
Scheduler excluded usage of resources Scheduler excluded usage of resources during peak timeduring peak time
Scheduler predictions met deadline using Scheduler predictions met deadline using least-expensive resources availableleast-expensive resources available
June 12, 2006June 12, 2006 2121
6. Conclusion and Future Work6. Conclusion and Future Work
GRACE leverages existing middleware GRACE leverages existing middleware systems (Condor/Legion/Globus)systems (Condor/Legion/Globus)
Nimrod/G can discover best resource Nimrod/G can discover best resource providers based on user’s requirementsproviders based on user’s requirements
Nimrod/G does not support dynamic prices Nimrod/G does not support dynamic prices once initial scheduling is proposedonce initial scheduling is proposed
Nimrod/G Nimrod/G PortalPortal available … available …
top related