programming hybrid services in the cloud
DESCRIPTION
For solving complex problems, we advocate for constructing “social computers” which combine software and human services. However, to date, hu- man capabilities cannot be easily programmed into applications in a similar way like software capabilities. Existing approaches exploiting human capabilities via crowds do not support well on-demand, proactive, team-based human computa- tion. In this paper, we explore a new way to virtualize, provision and to program human capabilities using cloud computing concepts and service delivery mod- els. We propose novel methods for modeling clouds of human-based services and combine human-based services with software-based services to establish clouds of hybrid services. In our model, we present common APIs, similar to APIs for software services, to access individual and team-based compute units in clouds of human-based services. Based on that, we propose frameworks and programming primitives for hybrid services. We illustrate our concepts via some examples of using our cloud APIs and existing cloud APIs for software.TRANSCRIPT
Programming Hybrid Services in the Cloud
Hong-Linh Truong*, Schahram Dustdar*, Kamal Bhattacharya§
*Distributed Systems Group,
Vienna University of Technology
§ IBM Research Africa
http://www.infosys.tuwien.ac.at/research/viecom
1 ICSOC 2012, Shanghai, China,
13 Nov 2012
2
Outline
Motivation
Our approach
Main contributions
Model for clouds of human-based services and hybrid
serices provisioning
Framework for solving complex problems using
clouds of hybrid services
Programming primitives for hybrid services
Conclusions and future work
ICSOC 2012, Shanghai, China,
13 Nov 2012
Motivation (1)
ICSOC 2012, Shanghai, China,
13 Nov 2012 3
We need to actively finding and binding human
capabilities into the application!
Humans and software for
solving complex problems
Motivation (2)
4 ICSOC 2012, Shanghai, China,
13 Nov 2012
But how to program human-based services and
software-based services together?
Motivation (3)
ICSOC 2012, Shanghai, China,
13 Nov 2012 5
Existing technologies are not adequate
Our approach -- incorporate humans
into a programming paradigm
• Abstracting human compute units as program elements
• Extending programming languages to support human compute units
• Data/control flows via extensible APIs
Programming languages
• Shared memory (e.g., human –software – human), message passing (human-to-human), artifact-centric, etc., via APIs working atop the compute unit abstraction layer
Multiple programming
models
• Computing capability /profile management: human computing power, reputation and incentive models
• Monitoring and enforcing incentives/rewards, quality of results, availability
• Communication between human-middleware, among Individual Compute Units (ICU)/Social Compute Units (SCU) for exchanging artifacts and comprehensing l tasks
Execution environment
6 ICSOC 2012, Shanghai, China,
13 Nov 2012
Our approach -- incorporate humans
into a programming paradigm
Volunteers Professionals Team Individual
Service-based Middleware
Communication Monitoring Capability/Profile
Management
Provisioning/Negotiation/Execution API
Abstraction of Human-based Compute Units
SCU ICU ICU ICU SCU
Program languages and programming models
Program
elements
Software
Compute
Units
program human actions
and dependencies
program incentive condition
and rewarding action
program result evaluation
method
Human-to-middleware
interfaces:
•visualization of collective tasks
•embedding of common forms
•mobile app
7 ICSOC 2012, Shanghai, China,
13 Nov 2012
Clouds of hybrid services
ICSOC 2012, Shanghai, China,
13 Nov 2012 8
A cloud of hybrid services includes software-
based services and human-based services
that can be provisioned, deployed and utilized
on-demand based on different pricing models
HBS Communication Interface
Based on well known technologies for
integrating human requests/responses into SBS
Emails, web interfaces, Web services, etc.
Some special characteristics
The same protocol can be used for multiple layers
(middleware, application, OS) -- unlike in SBS
Typically asyschronous communication and unknown
upper-bound delay
Can support message passing and shared
memory models
hbsi hbsj or hbsi sbs hbsj
9 ICSOC 2012, Shanghai, China,
13 Nov 2012
Human Power Unit (HPU)
Humans are determined via skills/skill levels
A particular HBS cloud can define its own sets of
skills (CS) and skill levels (SK)
Similar to Amazon defines its own EC unit
Different ways to make sure skill and skill levels
declared in a cloud consistent (e.g., via testing and
monitoring).
10
HPU Definition: HPU is a value describing the computing power of an HBS measured in an abstract unit. A cloud of HBS has a pre-defined basic power unit, hpuθ, corresponding to the baseline skill bsθ of the cloud.”
ICSOC 2012, Shanghai, China,
13 Nov 2012
Human Power Unit (HPU) – our
starting point
Given an HBS, its theoretical HPU can be larger
or smaller than its real HPU – unlike SBS 11
HPU for a particular (skill,
skill level)
HPU for a set of (skill, skill
level), no weighted factors
Decomposition/compositi
on of HPU for a shared
HBS
ICSOC 2012, Shanghai, China,
13 Nov 2012
Human Power Unit (HPU) – our
starting point (2)
ICSOC 2012, Shanghai, China,
13 Nov 2012 12
HPU can be defined for different „archtypes“
An „archtype“ characterizes the problem domain (e.g., bones simulation) that the ICU/SCU can solve (the type of tasks)
HBS Instances Provisioning
13
• Individual Compute Unit (ICU)
• Social Compute Unit (SCU
Types of services
• iICU(CS, HPU, archtype, price, utilization,location, APIs)
• iSCU(CS,HPU, archtype, price, utilization,connectedness, location, APIs)
• Other (traditional) NFPs
Instances Descriptions
• utilization
• offering communication APIs
• connectedness
Pricing factors
ICSOC 2012, Shanghai, China,
13 Nov 2012
Cloud APIs for Provisioning Hybrid
Services
14
APIs hide low-level platforms and utilize low level HBS
communication interfaces
ICSOC 2012, Shanghai, China,
13 Nov 2012
APIs for HBS information and management
• listSkills();listSkillLevels();
• listICU();listSCU()
• negotiateHBS()
• startHBS()
• suspendHBS ()
• resumeHBS ()
• stopHBS()
• reduceHBS()
• expandHBS()
APIs for HBS execution and communication
• runRequestOnHBS ()
• receiveResultFromHBS()
• sendMessageToHBS()
• receiveMessageFromHBS()
Prototype (simulated environment)
15
Combined with Jcloud/boto for real SBS
ICSOC 2012, Shanghai, China,
13 Nov 2012
Framework for utilizing hybrid
services
16
IT Management
Smart City Management
ICSOC 2012, Shanghai, China,
13 Nov 2012
Modeling HPU-aware task
dependency graphs
Link management skills and compliance constraints to tasks
required HBS
17
Can be specified in TOSCA!
Modeled in dependency graphs
ICSOC 2012, Shanghai, China,
13 Nov 2012
Programming a combination of
HBS and SBS
18
e.g., preparing/managing inputs/outputs for HBS using SBS
ICSOC 2012, Shanghai, China,
13 Nov 2012
Forming iSCUs
Done by consumers or cloud providers
19
SCU Formation
Algorithms
Business As
Usual
Corrective
Action
HBS
Constraints
ICSOC 2012, Shanghai, China,
13 Nov 2012
Configuring iSCU
Establish „connectedness“ based on compliance
constraints and network topology
Addional cost might occur!
Program SBS and HBS for the iSCU to have a
complete working environment.
Different connectedness
E.g., ring-based, star-based, and master-slave
topologies
20 ICSOC 2012, Shanghai, China,
13 Nov 2012
Selecting HBS: Some algorithms
21
Several algorithms can be built based on existing team
formation algorithms which do not consider dependency
graphs
Different weighted factors can be considered ICSOC 2012, Shanghai, China,
13 Nov 2012
Forming iSCU by minimizing cost
and considering no direction
22 ICSOC 2012, Shanghai, China,
13 Nov 2012
Example of star-based iSCU using
Dropbox as a communication hub
23 ICSOC 2012, Shanghai, China,
13 Nov 2012
Change model for task graph’s
Human Power Unit
24 ICSOC 2012, Shanghai, China,
13 Nov 2012
Conclusions and Future Work (1)
Novel models and APIs are needed for integrating
humans into program paradigms
We introduce
Methods for modeling clouds of human-based services
Techniques for combining human-based services with
software-based services
Future works
Programming elements/constructs/patterns for hybrid
services
Hybrid service life-cycle management
25 ICSOC 2012, Shanghai, China,
13 Nov 2012
Conclusions and Future Work (2)
26
Mapping from a coordination language to hybrid service
APIs
Statelet: a declarative coordination language for collaboration
processes
Vitaliy Liptchinsky, Roman Khazankin, Hong Linh Truong, Schahram
Dustdar: Statelets: Coordination of Social Collaboration Processes.
COORDINATION 2012
Provisioning/Negotiation/Execution API
Abstraction of Human-based Compute Units
SCU ICU ICU ICU SCU
ICSOC 2012, Shanghai, China,
13 Nov 2012
27
Thanks for your attention
Hong-Linh Truong
Distributed Systems Group
Vienna University of Technology
http://www.infosys.tuwien.ac.at/staff/truong
ICSOC 2012, Shanghai, China,
13 Nov 2012