Research ArticleHierarchical Brokering with Feedback Control Framework inMobile Device-Centric Clouds

Chao-Lieh Chen1 Chao-Chun Chen2 and Chun-Ting Chen1

1Department of Electronic Engineering National Kaohsiung First University of Science and Technology (First Tech)No 1 University Road Yanchao District Kaohsiung 824 Taiwan2Department of Computer Science and Information Engineering and Institute of Manufacturing Information and SystemsNational Cheng Kung University Tainan Taiwan

Correspondence should be addressed to Chao-Lieh Chen fredericieeeorg

Received 11 April 2016 Accepted 13 June 2016

Academic Editor Young-June Choi

Copyright copy 2016 Chao-Lieh Chen et al This is an open access article distributed under the Creative Commons AttributionLicense which permits unrestricted use distribution and reproduction in any medium provided the original work is properlycited

We propose a hierarchical brokering architecture (HiBA) and Mobile Multicloud Networking (MMCN) feedback controlframework for mobile device-centric cloud (MDC2) computing Exploiting the MMCN framework and RESTful web-basedinterconnection each tier broker probes resource state of its federation for control and management Real-time and seamlessservices were developed Case studies including intrafederation energy-aware balancing based on fuzzy feedback control andhigher tier load balancing are further demonstrated to show how HiBA with MMCN relieves the embedding of algorithms whendeveloping servicesTheoretical performance model and real-world experiments both show that anMDC2 based onHiBA featuresbetter quality in terms of resource availability and network latency if it federates devices with enough resources distributed inlower tier hierarchy The proposed HiBA realizes a development platform for MDC2 computing which is a feasible solution toUser-Centric Networks (UCNs)

1 Introduction

A user possesses many mobile devices nearby to enjoyproactively provided services Wearable and portable devicestogether with proximity and remote servers process largeamount of requests to complete a service Therefore aninstance of cloud computing methodology to federate thesedevices and servers is desired especially to the 5th-generation(5G) era that cooperation among massive devices is oneof technology focuses Moreover the latest surveys [1ndash5]on User-Centric Network (UCN) advocate self-organizingautonomic networks in which users cooperate through shar-ing of network services and resources The self-organizingautonomic architecture no matter whether ad hoc or infra-structure-based federates nominal low cost devices andmakes users a new type of resource provider and stakeholderin addition to content consumer or producer Based onsociological relations mobile devices proactively join theUCN and become network elements (such as router orgateway) to share bandwidth among themselves and to obtain

more reliable network connection allowing free roamingTheuser-provided networks (UPNs) proposed in [2] are one ofthe examplesThe contribution [3] based on software-definednetworking (SDN) explores cooperation among wirelessmobile devices to share network transmission bandwidthRouting opportunistic relaying and sensing of hybrid typesof information [5] through spontaneous networks [4] areprojection of user behaviors in their social networks

The UCNs emphasize three main key techniques [1] (1)understanding user context (2) profiling and predicting userinterests and (3) personalizing content delivery Thereforethe state observation of UCN underlying a specific serviceis essential for constructing control and management planesHowever depending on resource distribution among attend-ing mobile devices and data centers the state space can betoo large to be observed Then design of control and man-agement algorithms becomes complex and challenging

In this paper we propose HiBA architecture with hier-archical brokering and feedback control framework MMCN

Hindawi Publishing CorporationMobile Information SystemsVolume 2016 Article ID 3057397 11 pageshttpdxdoiorg10115520163057397

2 Mobile Information Systems

for both control and management planes in UCNs Theidea comes from the hierarchical fuzzy control [6] where acomplex control problem with huge state space is conqueredIn a hierarchical control system the cross product of statespaces of all control hierarchies remains huge while eachhierarchy adopts only a few fuzzy rules to perform simplecontrol tasks

A series of contributions adopting MDC2 for improvingload balance [7] user-centric security [8] access control[9] in the cloud as well as MDC2 applications of real-timeseamless video sharing [8] and sociological ontology-basedhybrid recommendation [10] were proposed In these contri-butions we see that MDC2 is a feasible solution to provideservices in UCN In this paper HiBA with feedback controlframework generalizes the brokering and state observationAvailability andnetwork latency are analyzed and experimen-tal HiBA comprising real-world mobile devices and serversare realized We prove that a UCN with huge state spaceis observable and hence controllable through HiBA cloudnetworking In this paper we further extend our previouspresentation [11] to demonstrate algorithm embedding usingthe MMCN development platform in both lower and highertiers of HiBA We study energy-aware balancing in thecloudlet tierwhere densemobile devices federate In additionwe study the load balancing in bigger data centers where largeamount of requests usually arrives in a short time Throughboth theoretical analysis and real-world experiments weconclude that a HiBA federation with MMCN is a platformfor developing distributed algorithms in future 5Grsquos massivemachine-to-machine (M2M) communications and UCNs

This paper is organized as follows Section 2 depictsthe HiBA architecture and the feedback control frameworkMathematical analysis on availability and network latencyis provided in Section 3 Section 4 demonstrates real-worldexperiments in which the HiBA federates heterogeneousmobile and fixed devices in three tiers using different networkinterfaces Energy-aware balancing for a cloudlet and loadbalancing for a bigger data center are both conductedWe alsopresent conclusions at the end of Section 5

2 System Architecture

Unlike traditional data centers that span trees of VMs ina top-down manner from a primary computing domainusually called domain zero in a physical machine (PM)the proposed HiBA architecture is initiated from physicalmobile devices at the bottom tier called the cloudlet tierEach broker dynamically connects and adaptively controlslower tier PMs or VMs and thus can avoid drawbacks suchas the degradation of the aggregate available bandwidth andcircuitous communication path of VM spanning trees [12]The system includes resources in the cloudlets then associatescloudlets with private and public clouds to further scale upthe cloud federation On the management aspect we exploitadaptive feedback control framework to tackle the uncertaindynamics of mobile ad hoc networks in order to adapt tothe dynamics such as dynamic joining and leaving a brokerrsquosdomain caused by user mobility device failure or physicalnetwork handoff [8]

User request initiator Service provider

Cloudling tier

Cloud rack tier

Cloud bank tier

Cloudlet tier

Public cloud broker Cloudling broker

User cloudlet brokerUsersrsquo resource instances

Public clouds

private cloudsMobile

Figure 1 Hierarchical brokering architecture (HiBA) for mobiledevice-centric cloud computing

21 Hierarchical Brokering Figure 1 shows the HiBA archi-tecture which is comprised of four tiers Tier 0 called thecloudlet tier contains groups of mobile user devices Eachcloudlet is managed by a cloudlet broker maintaining net-work connectivity andmember association and also perform-ing QoS control A cloudlet is dynamically organized by thebroker according to resource instant status including CPUmemory network capability and energy consumption Theresources underlying a tier is abstracted as a JSON or XMLlist in its brokerrsquos database accessible to federation membersthrough RESTful API Thus each entry of the list includesa URL containing available RESTful command or a URL offilename and file attributes Fields of security configurationsNAT port and GPS coordinates are optionally included ineach entry [8] A tier-0 device does not have a VM memberexcept itself and it aggregates its resources to be shared withothermembers in the resource list Each broker hierarchicallyaggregates the resource lists from members and uploadsupdates to its upper tier broker

We continue using the cloudlet federation procedurein our previous research [8] A cloudlet broker is eitherdynamically elected among user devices according to theresources status or selected by its upper tier (tier 1) brokercalled cloudling broker A cloudling broker is a small datacenter or private cloud proximate to a cluster of cloudlets thatassociate with the cloudling It is similar to the small datacenter proposed in [13 14] owned by a smaller business unitsuch as a coffee shop or a clinic We differ from [13 14] inthat our cloudlets are autonomously grouped by user deviceswhich also share resources with others A tier-119899 brokerassociates with a tier-(119899 + 1) broker to scale the federation indepth or alternatively to include more tier-(119899 minus 1) federationsto scale in width In the proposed architecture each brokeronly recognizes next lower tier brokers while further lowertiers are transparent to it Furthermore each broker has thesame feedback control framework for managing lower tierdevicesrsquo network attachments and resource associations aswell as for QoS control on requesting and executing services

Mobile Information Systems 3

instanceUpper tier

CIScloud

inspection

CCScloud

controlTask assignment

Queryresource

Control

Forwardedrequest

Internal state update

Lower tier state update

Observedstate

State update

Forwarded request

SQLdatabase

Local request

Lower tierinstances

CEScloud

execution

Task assignments

Tier (n minus 1)

tier nInstance in of

Tier (n + 1) inminus1

in+1

Figure 2 The feedback control framework MMCN in a HiBA broker

22 Feedback Control Framework Figure 2 shows the feed-back control framework MMCN of each broker in the HiBAvirtualized network The feedback loop is used to adaptivelymanage and control the cloud federation governed by thebroker This management includes network attachments andresource associations caused by usermobility andVMmigra-tion [8] In future application the framework is ready fordeveloping hierarchical control algorithms such as requestdifferentiation and task scheduling as well as real-timeQoS control and job tracking once a service is started Afeedback control loop comprises three subsystems includingCloud Inspection Subsystem (CIS) CloudControl Subsystem(CCS) and Cloud Execution Subsystem (CES) which areanalogous to observer controller and plant respectively in aclassic feedback control system When performing real-timecontrol for QoS multiple lower tier instances of the feedbackloop are allocated A tier-119899 CCS determines the number oftier-(119899 minus 1) instances to be allocated then the tier-119899 CESinvokes the required instances each of which is constitutedby the other set of CIS CCS and CES at the tier-(119899 minus 1)

A CCS processes tasks assigned by its upper tier brokerand processes requests from lower tiers Then according tothe database and states tracked by the CIS subsystem itdetermines the controls including admission of memberjoining allocating member federations for task executionschedules of tasks and service level agreements (SLAs) oftasks to be assigned tomember federations A number of CESinstances physically accomplish the network attachment forthe admission control as well as the recruiting of membersassociated with corresponding SLAs to complete currentschedule A CCS produces these determinations as controlsto CES subsystems while the executions are left to CESs andassociated member federations The CIS observes the stateupdates from lower tiers for CCSrsquos reference when deter-mining these controls

23 Algorithm Embedding It is obvious that this frameworkis feasible for future development of hierarchical controland management algorithms including admission controlrequest differentiation task partition adaptive resource allo-cation and dynamic task scheduling regarding SLAs assignedby upper tier broker For example on receiving a new requesta priority queue of requests is adapted with new set ofprioritiesTheCCS checks with the CIS to determinewhetherto forward the request to upper tier broker or to reduce therequest into smaller tasks such that the CESs can furtherassign the task partitions to lower tier members just as theprocessing of task assignments from upper tier brokerrsquos CESTheCESrsquosmakes lower tiers transparent to theCCS and uppertiers as if it is the single plant of the CCS controller Sinceit is the CESs and associated member federations executingtasks rather than the CCS the CCS is able to process themanagement and control algorithms without waiting for theend of the current job executionThis is easy to be realized byprogramming multiple threads each of which realizes CCSCESs and CIS subsystems

In summary the CCS of a tier-119899 broker forwards requeststo tier-(119899 + 1) if it is not able to process them accordingto the CIS database The tier-(119899 + 1) broker assigns jobs toother tier-119899 brokers through CES if the requested resourcesare sharable in these tier-119899 members and below Thereforethe sharing becomes intercloud service of tier-119899 as well asintracloud service of tier-(119899 + 1)

3 Performance Analysis

Thebaseline performance for benchmarking is to evaluate theHiBA architecture and feedback control framework withoutoptimization on specific management and control That isthe queues are simple FIFOs without priority adaptationand all devices in all tiers randomly generate requests The

4 Mobile Information Systems

120582n0

tn

tn

tn

Dn0

B120582nminus1120582nminus1120582nminus1

tnminus1 tnminus1 tnminus1 tnminus1 tnminus1 tnminus1

Cnminus10

Dnminus10

Cnminus12

Dnminus12

Cnminus1B

Dnminus1B

middot middot middot

middot middot middotmiddot middot middotmiddot middot middot

t1 t1 t1 t1 t1 t1

12058200 1205820B 1205820B+1 1205820j 1205820k 1205820m

C00 C0B C0B+1 C0j C0k C0m

D00 D0BD0B+1 D0j D0k D0m

Cn0

Figure 3 Performance model of the HiBA architecture

performance model of the virtualized network is shown inFigure 3 Suppose that themaximumnumber ofmembers of aHiBA broker is 119861 a broker 119895 in tier 119894 has computing capability119862119894119895 which is also the minimum length of the Task Queue

The capability 119862119894119895also represents the maximum number of

tasks that can be processed by a broker in tier 119894 under thecondition that no task is droppedThe resources are assumedto be uniformly distributed in each tier In tier 119894 the averageamount of contents and resources is 119863

119894119895 A tier-119894 broker has

its amount of local requests in a Poisson distribution withmean 120582

119894119895 This simulates the reinitiated requests caused by

request partitions and task handoffs 119905119899denotes the mean

initial latency caused by network delay to forward requestsand the waiting time that requests stay in the Request Queueuntil being partitioned into smaller tasks In this paper wederive baseline performances on availability and latency

31 Resource Availability We define availability 119886119894119895 as the

ratio of computing capability119862119894119895over the number of accepted

requests 119877119894119895to a broker 119895 in tier 119894 That is

119886119894119895= min(

119862119894119895

119877119894119895

1) (1)

In a HiBA architecture of 119899 tiers the expected total resourceamount summing from the records in the CIS databases isapproximated as

119863total = 119863119899+

119899

sum

119894=1

(

119894minus1

prod

119896=0

119861119899minus119896

)119863119899minus119894

=

119899

sum

119894=0

119861119894119863119899minus119894

(2)

supposing that119863119894= 119864[119863

119894119895]Thus the expected total resource

amount of the subtree rooted at a tier-119894 node is

119863119894total =

119894

sum

119896=0

119861119896119863119899minus119896

(3)

For any request the probability that the requested resource 119903119894119895

is available in local node 119894119895 is approximated by

119875 (119903119894119895) = 119875 119903

119894119895isin D119894119895 =

119863119894119895

119863total (4)

where D119894119895is the set of resources in node 119894119895 Suppose that

the sample space is the union of D119894119895in the whole HiBA

architecture Then the worst case of availability occurs whennetwork latency is much smaller than the mean time interval1120582119894119895between two consequent requestsThat is requests from

other cloud federations including those in lower and highertiers arrive in current tier instantly with negligible networkdelayThus the maximal number of requests arriving node 119894119895is

119877119894119895= 119875 (119903

119894119895)

119899

sum

119896=0

119861119896120582119899minus119896119895

(5)

Then we obtain the worst case availability as

119886119894119895= min(

119862119894119895

119875 (119903119894119895)sum119899

119896=0119861119896120582119899minus119896119895

1)

= min(119862119894119895

119863119894119895

sdot

119863totalsum119899

119896=0119861119896120582119899minus119896119895

1)

(6)

Mobile Information Systems 5

From (6) we see that when 119899 is smaller resources (includingcontents) are more centralized with larger 119863

119894119895and if 119863total

remains large this causes lower availability Thus there arevarious means to increase availability One is to increasecapability 119862

119894119895 though this will increase costs Alternatively

branch amount 119861 of each tier can be increased though thisalso will increase computing capability and costs Third byadopting HiBA we put user devices in tier 0 or tier 1 (oncethe user device is elected cloudlet broker) However thecapabilities 119862

0119895and 119862

1119895of thin devices may be small and

so 1198630119895and 119863

1119895are also expected to be small This indicates

that availability remains close to 1 if resource distribution119863119894119895

is proportional to the capability 119862119894119895with the ratio of total

number of requests over total resource amountTherefore theoptimal resource distribution to low tier brokers and mobiledevices both offloads data centersrsquo computing overhead andincreases user satisfaction

32 Latency Social networks can cause locality such thatmany requests do not travel to their destinations over longdistances in terms of either network relay counts or theterrestrial radio barriers Thus initial latency of a serviceis reduced Lower tier brokers have smaller resource gran-ules while locality based on social network provides accessproximity and thus results in shorter latency We estimatethe latency of a request traversing the HiBA architecture asfollows Suppose that a request initiated from a tier-119894 nodearrives at its destination possessing the required resources intier 119897 119894 lt 119897 The expected latency of the request is

119879119894119897= 119875 (119903

119894lt119897)

119897

sum

119896=119894

(119905119873119896

+ 119905119863119896

+ 119905119862119896

) (7)

where 119875(119903119894lt119897) is the probability that the requested resource

is not found in tiers lower than 119897 and 119905119873119896

119905119863119896

119905119862119896

arelatencies caused respectively by network communicationrequest queuing plus database querying and computing forthe brokering at each tier-119896 broker on the path to tier 119897Supposing that request arrival is independent of resourcedistribution we have

119875 (119903119894lt119897) =

120582119894

120582total(1 minus

119863119897minus1total

119863total) (8)

Thus the estimated latency is

119879119894119897=

120582119894

120582total(1 minus

119863119897minus1total

119863total)(

119897

sum

119896=119894

119905119873119896

+ 119905119863119896

+ 119905119862119896

) (9)

If a request is originated from a lower tier user device thatis 119894 = 0 or 1 the effective way to reduce expected latency1198790119897or 1198791119897is to reduce 119875(119903

119894lt119897) This means increasing119863

119897minus1totalby distributing resources to lower tiers that is increasing119863

119896

for 119896 lt 119897 However this also increases both database search-ing time 119905

119863119896and computation time 119905

119862119896 Considering network

delay 119905119873119896

we can expect that in the virtualized network thephysical distance of a cloud server is increasing as 119896 is gettinglarger A high tier link in the virtualized HiBA networkactually contains many more physical relaying hops than a

lower tier link If request is sent using TCP protocol thenetwork latency is increasing muchmore than that caused bydatabase searching and brokerage computing as 119896 increasesFrom (9) we still effectively decrease the latency by increasing119863119897minus1total This can be expected especially for multimedia

content sharing since according to social network relationscontents are stored in proximate cloudlets or cloudlingswhich are lower tiers (small 119897)

33 Development Platform for MDC2 Control and Manage-ment From the availability and latency analysis we see thata HiBA with feedback framework is a development platformIt is easy to observe the performance when developingalgorithms for admission control request differentiation taskpartition adaptive resource allocation and dynamic taskscheduling regarding SLAs assigned by upper tier broker Forexample the admission control in federating a tier affectsresource distribution 119863

119894119895and according to the number of

network hops between a broker and a member the networklatency 119905

119873119896in (9) also differs The request differentiation

and queuing mechanism directly affect 119879119894119897since 119905

119863119896includes

the queue delay that a request stays in the Request QueueTask partitioning resource allocation and scheduling furtheraffect the efficiency processing requests and consequentlythey further affect 119905

119863119896and resultant 119879

119894119897 Globally they also

affect arrival rate 120582119894at tier 119894 Exploiting hierarchical feedback

control it is easy to ensure availability by performing propermanagement while tracking the latency by tuning controlregarding the deadline specified in the SLA of each task

4 Case Studies

To demonstrate algorithm embedding using the MMCNdevelopment platform in both lower and higher tiers westudy energy-aware balancing among mobile devices in thecloudlet tier as well as the load balancing in bigger datacenterswhere large amount of requestswould arrive in a shorttime that is large 120582

119894119895 The energy-aware balancing is critical

in crowd-sensing applications especially when the sensingdata amount is large such as using cameras for image datacollectingThe load balancing is essential for bigger data cen-ters especially when database access frequency is high Bothof the balancing algorithms are effective in the Industry 40era when crowd-sensing and big data analytics are deployed

41 Energy-Aware Balancing The energy-aware balancing isbased on unsupervised fuzzy feedback control where thereference command is also adapted according to the feedbackstate of the federation self-organized by mobile devices Theprincipal idea comes from the energy proportional routing[15] that the lifetime of a clustering-based sensor network isprolonged ifmember nodesrsquo proportions of consumed energyin the remaining are close to the clusterrsquos We exploit theenergy proportion of the cloudlet federation as the adaptivereference to be tracked by the fuzzy feedback control systemTherefore the energy sharing is unsupervised because of nogiven objective of the control a priori

6 Mobile Information Systems

0

1

0

1

120583

120583

N P

e

N998400 P998400

e998400

minus256 0 256

minus512 512

Figure 4 Membership functions for fuzzy sets119873 1198751198731015840 and 1198751015840

Suppose that in a cloudlet tier of119873member nodes datatransmission is observed by the cloudlet broker in each round119905 (discrete time) We first define symbols as follows

(i) 119896 member node identification 1 le 119896 le 119873

(ii) 119863119896(119905) data transmission amount being assigned to

node 119896 at round 119905

(iii) 119864119896(119905) remaining energy of node 119896 at 119905

(iv) 119883119896(119905) = 119863

119896(119905)119864119896(119905) representing the ratio of over-

head to the remaining energy (current capability)This is the indication of how node 119896 is loaded withrespect to the remaining energy

(v) 119879ℎ(119905) = sum119873

119896=1119863119896(119905)sum

119873

119896=1119864119896(119905) representing how the

whole cloudlet federation is loaded

(vi) 119890119896(119905) = 119883

119896(119905) minus 119879ℎ(119905) representing the difference of

loading between node 119896 and the whole federation

(vii) 1198901015840119896(119905) = 119890

119896(119905)minus119890119896(119905minus1) representing how the difference

changes

(viii) 119903119896(119905) the ratio of data amount assigned to node 119896 at 119905

We define fuzzy rules as follows

(R1) If 119890119896(119905) is 119875 and 119890

1015840

119896(119905) is 1198751015840 then 119903

119896(119905 + 1) is 119878

119896(119905)

(R2) If 119890119896(119905) is 119875 and 119890

1015840

119896(119905) is1198731015840 then 119903

119896(119905 + 1) is119872

119896(119905)

(R3) If 119890119896(119905) is119873 and 119890

1015840

119896(119905) is 1198751015840 then 119903

119896(119905 + 1) is119872

119896(119905)

(R4) If 119890119896(119905) is119873 and 119890

1015840

119896(119905) is1198731015840 then 119903

119896(119905 + 1) is 119871

119896(119905)

Themembership functions for fuzzy sets119873 1198751198731015840 and 1198751015840 areconfigured in Figure 4

We realize the ldquoandrdquo operator with 119905-norm ldquominimumrdquoThat is the matching degrees 120583

1 1205832 1205833 and 120583

4of premise

part of rules (R1) to (R4) respectively are

1205831 (119905) = min (120583

119875 (119890 (119905)) 120583119875

1015840 (1198901015840(119905)))

1205832 (119905) = min (120583

119875 (119890 (119905)) 120583119873

1015840 (1198901015840(119905)))

1205833 (119905) = min (120583

119873 (119890 (119905)) 120583119875

1015840 (1198901015840(119905)))

1205834 (119905) = min (120583

119873 (119890 (119905)) 120583119873

1015840 (1198901015840(119905)))

(10)

Obtaining the matching degrees 1205831 1205832 1205833 and 120583

4of the

four fuzzy rules respectively we perform the defuzzificationequivalently applying the Takagi-Sugeno inference methodThe inference result adequate ratio of data amount totransmit is

119903119896 (119905 + 1)

=

1205831 (119905) 119878 (119905) + 120583

2 (119905)119872 (119905) + 120583

3 (119905)119872 (119905) + 120583

4 (119905) 119871 (119905)

1205831 (119905) + 120583

2 (119905) + 120583

3 (119905) + 120583

4 (119905)

(11)

by configuring the conclusion part membership functions asdynamic singletons as follows

119871 (119905) = 119903119896 (119905 minus 1) + Δ

119872(119905) = 119903119896 (119905 minus 1)

119878 (119905) = 119903119896 (119905 minus 1) minus Δ

(12)

where Δ is the ratio tuning amount for each round Finallythe data amount assigned to node 119896 is determined as

119863119896 (119905 + 1) =

119903119896 (119905)

sum119873

119894=1119903119894 (119905)

D (119905) (13)

where D(119905) is the total data amount to be transmitted fromthis cloudlet at time 119905 In the above fuzzy inference algorithmeach member node tracks the dynamic control goal 119879ℎ(119905)and the proportion of energy to be consumed in the remain-ing energy approaches the proportion regarding the wholefederation Therefore the intrafederation energy sharing isachieved by offloading transmission jobs using the proposedalgorithm When each node 119896 is a lower tier federation theoffloading is hierarchically extended through the MMCNnetworking where the data amount 119863

119896(119905) is determined and

assigned by CCS and the energy status updates 119864119896(119905)rsquos are

received and aggregated by CIS This shows the scalability ofthe HiBA architecture

42 Load Balancing The load balancing is required in abigger federation organized in a higher tier of HiBA becausea higher federation always has larger amount of requestarrivals As shown in Figure 5 the framework of the proposedload-balanced cloud service interface consists of three com-ponents which realizes CES CCS and CIS respectively Thedetails of components are presented as follows

(1) Cloud Service Interface Node (CSIN) it is a virtualmachine that provides cloud service interface and is

Mobile Information Systems 7

Service module 1

Service module 2

Service

CSIallocation

table

Clouddatabase

HiBA networking

Cloud bank tier

(1)(3)

(4)

(5)

(6)

(7)(8)

(2)CSI

manager CSIN 1(9)

CSIN n

module m

middot middot middotmiddot middot middot

middot middot middot

Figure 5 Framework of load-balanced cloud service interface

denoted as CSINi in Figure 5 A CSIN can receivemembersrsquo requests and obtain results by interactingwith requested cloud service module In our designeach cloud service interface in CSIN is implementedas a RESTful API for universally communicating withdifferent types of mobile devices A CSIN is then therealization of CES in the MMCN framework

(2) CSImanager it is designed tomanage the usage statusof CSINs andmatchesmobile users to CSINsThe effi-ciency of the load balance depends on the matchingmethod of the CSI Manager A CSI manager is thenthe realization of CCS in the MMCN framework

(3) CSI allocation table it maintains the serving statusbetween mobile users and CSINs and is used by theCSI manager for assisting the matching decision forthe new-arrived mobile users This table is stored inthe cloud database A CSI allocation table is then therealization of CIS in the MMCN framework

The processing flow of using a cloud service for a mobile useris designed for achieving the load balance The idea of com-mitting requests requires three phases (1) service registration(Steps (1)-(2)) (2) service execution (Steps (3)ndash(7)) and (3)service deregistration (Steps (8)-(9)) The detailed steps alsoillustrated in Figure 5 are presented as follows

(1) Mobile user MU119894sends a request to CSI manager for

allocating a CSI address(2) CSI manager searches for a CSI machine with least

number of serving users say CSIN119899 in the CSI allo-

cation table and then registers (MU119894 CSIN

119899) into the

CSI allocation table(3) CSI manager informs MU

119894that CSIN

119899can serve his

her cloud service requests in the following period(4) MU

119894configures the cloud service interface by replac-

ing the IP attribute in the service template with the IPaddress of CSIN

119899

(5) MU119894connects to CSIN

119899through the configured URL

of the cloud service interface and sends a request(6) CSIN

119899delivers the request to the associated cloud

service which may access the cloud databases ifrequired After processing the results will be sentback to CSIN

119899

(7) CSIN119899sends the results to MU

119894 (If more requests are

required to processed Steps (3)ndash(7) are repeated)(8) MU

119894deregisters CSIN

119899to CSI manager

(9) CSI manager removes the registration record of MU119894

from the CSI allocation table

Notice that the matching rule for a new mobile user andCSINs in the current design uses least-user-first basis asshown in Step (2) The matching principle can be modifiedaccording to different demands In the experiments wewill show that the current design already obtains splendidperformance More study on customizing matching methodsis one of our future research directions That is althoughfive machines are used in our CSI (more than the single-machine CSI) the improvement is by a factor of one hundredindicating the superior performance of our proposed load-balanced CSI mechanism under the HiBA architecture

5 Real-World Experiments

51 HiBA Baseline Performance We conduct an experimentdemonstrating performance difference in terms of availabil-ity and latency when distributions of virtualized resourceinstances (VRIs) differ We leave development of enhancingalgorithms for specific purposes in the CCS as future workwhich will seek performance in terms of various metricsA total of 50000 VRIs are distributed in a 3-tier HiBAfederation comprising Android smart phones tablets andVMs in desktop PCs Figure 6 is the HiBA topology andrelated specifications of the machines are shown in Table 1Each device is labeled (119894 119895) if it is the 119895th node at tier 119894 Device(1 3) connects to (2 0) through a 3G access point with VPNtunneling Device (0 11) connects to (1 0) with USB andshares Internet connection from (1 0) These machines havedifferent computing ability though request and task queuesbeing of the same size of 20 to preserve the differences ofcomputing ability Requests are randomly generated in eachdevice Mean request arrivals (120582) are 3 15 and 1 whichequivalently mean that request intervals (1120582) are 333ms666ms and 1 s respectively The resource distributions havefour casesThe first case is that the tier-2 device possess all the50000 VRIs while 16000 VRIs are duplicated to each tier-1 federations In the remaining three cases we respectivelyduplicate 28000 32000 and 36000 VRIs to each tier-1federation The experiment is conducted in a heterogeneousnetworking environment connecting machines by differentcommunication technologies shown in Table 1

The results of the experiment are shown in Figures 7 and8 Both Figures 7 and 8 reveal that distributing resources inlower tiers provides better performance Figures 8(a) and 8(b)show the latency differences caused by the resource distri-butions that 16000 32000 and 40000 VRIs are duplicatedin each cloudling tier The results before all brokers fullyloaded are also given with respective zoom-in charts Whendevices all continuously issue requests at a high frequency (3requests per second) the queue delays are obviously high andwe see a few requests that are dropped Requests from tier-1devices have lower loss rate since the resources requested are

8 Mobile Information Systems

Table 1 Machines in the offloading experiments using heterogeneous communication technologies

(Tier device) Specifications Physical network linkTier 2(2 0) Desktop PC Intel Core i5-3470 CPU 32GHz 4G RAM

EthernetTier 1(1 0) (1 1) Desktop PC Intel Core i7-4770 CPU 34GHz 8G RAMTier 1(1 2)Tier 1(1 3) Laptop Intel Core i7-2820QM CPU 23GHz 8G RAM Down WiFi UP 3G VPNTier 0(0 0sim3)

Desktop PC Intel Pentium D CPU 34GHz 1 G RAM EthernetTier 0(0 4sim6)Tier 0(0 7)Tier 0(0 8) Tablet ASUS Fonepad Intel Atom Z2420 12 GHz

WiFiTier 0(0 9) Cell Phone S3 SAMUNG Exynos 4412 14 GHzTier 0(0 10) Tablet ASUS Nexus7 NVIDIA Tegra 3 4-coreTier 0(0 11) Tablet WS-170 Qualcomm QCTMSM8x60 surf15 GHz duo-core USBlarrrarr PClarrrarr EthernetTier 0(0 12sim13) Virtual Machines (VirtualBox) EthernetTier 0(0 14) Tablet Samsung Galaxy CPU GT-I8260 WiFi

(011)

LAN LAN WiFi 3G

Tier 2

Tier 1

Tier 0

VPN

LAN LAN WiFi WiFi

(0 14)

(0 8)

(0 5)

(0 12)

(0 13) (0 9)

(0 7)(0 4) (0 6)(0 3)

(0 1)(0 10)

(0 2)(0 0)

(1 3)

(2 0)

(1 1)(1 0) (1 2)

Figure 6 Real-world HiBA topology for offloading performancemeasurement Gray nodes are local VMs of respective tier-1 brokersDiverse physical network links are exploited in tiers of federations

obtained in a shorter time For each physical machine theCPU utilization of theMMCNbrokering itself is smaller than2 depending on the number of cores The mean brokeringcomputation and database querying time is estimated tobe about 500ms while the WLAN delay time is smallerthan 5ms In the heterogeneous network case the latencydifference will be larger if upper tiers are physically at a longdistance from the cloudlings and cloudlets If the requestinterval is higher than the processing time the latency tendsto be smaller When more VRIs are duplicated in lower tierdevices we also see that the latencies are smaller

In this paper we leave algorithms such as request par-tition network embedding and federation for specific per-formance metrics as future work Instead we implementreal-world HiBA to prove that with feedback framework it isa new design paradigm and development platform for thesealgorithms

52 Energy-Aware Balancing In the energy-aware balancingexperiment we exploit four mobile devices of different typesfrom different manufacturers The cloudlet is self-organizedaccording to the broker election protocol in [8] The electedbroker is ASUS-Fonepad The other hierarchical federations

are in Figure 6 Each device including the broker itselfin each round updates its energy status to the CIS of thebroker (ASUS-Fonepad) Each mobile device in the cloudletis recharged to 100of respective battery energy capacityThedata and total data amount to be transmitted in each roundare randomly generated with mean of 24GB prior to theexperiment For each round 119905 total data amount is the samefor both balancing and nonbalancing cases In nonbalancingcase the data amounts are evenly assigned to members Inbalancing case the data amounts are determined by the fuzzycontroller in Section 41 The data amount assignment is ofunit 100MB When any device has remaining energy loweror equal to 40 the experiment is terminated The initialsingletons are configured as 119878(0) = 3 119872(0) = 5 and 119871(0) =

7 (Δ = 2) The result of energy-aware balancing is shownin Figure 9 The respective control systemsrsquo behaviors are inFigure 10 The fuzzy controllers of mobile devices all trackthe federationrsquos data amount to energy proportion (119879ℎ) Wesee that no matter how the performance of respective devicevaries a device with better transmission capability will shareits energy with others Tablet PCs are usually with highercapacity batteries Without energy sharing the remainingenergy of a tablet drops more slowly than a mobile phoneWith energy sharing tablets undertake more transmissionjobs and becomewith higher energy drop rate and themobilephones become with lower energy drop rate However wesee that the whole cloudlet federation has longer lifetimeTheexperiment can also be applied to energy consuming tasks inaddition to data transmission

53 Load Balancing The load-balanced cloud service inter-face (CSI) in the form of RESTful APIs is implementedby using Jersey and is deployed on Microsoft WindowsAzure public cloud platform We conduct an experiment forcomparing the proposed CSI to that of single machine Inour experiment five CSI machines are used to share therequests from users In the experiment the data producingrate for each user is 1 request per second The performance

Mobile Information Systems 9

000 000000 000000 000000 000

000100200300

Homo Hetero16000 075 00028000 000 00032000 015 00036000 018 000

()

()

()

000100200300

Homo Hetero16000280003200036000

000100200300

Homo Hetero16000280003200036000

000 000000 012015 000000 000

Tier-1 1120582 = 666Tier-1 1120582 = 333

1600028000

3200036000

1600028000

3200036000

1600028000

3200036000

Tier-11120582 = 999

(a)

110 340050 230082 180080 170

030 330016 160000 130016 030

030 020000 000000 010000 000

1600028000

3200036000

000100200300

Homo Hetero16000280003200036000

()

000100200300

Homo Hetero16000280003200036000

()

000100200300

Homo Hetero16000280003200036000

()

Tier-0 1120582 = 666Tier-0 1120582 = 333

1600028000

3200036000

1600028000

3200036000

Tier-0 1120582 = 999

(b)

Figure 7 Mean losses of tier 0 (b) and tier 1 (a) in cases of different resource distributions when 16000 28000 32000 and 36000 VRIs areduplicated to each tier-1 federation Request intervals 1120582 are in milliseconds

Case3_tier0

0

20000

40000

1 6 11 16 21 26 31 36

050000

100000150000200000250000300000350000

Late

ncy

(ms)

898513 8121 25 7733 37 736949 6157 93 975345412917 655 91

Request

1120582 = 333

1120582 = 666

1120582 = 999

(a) 16000 VRIs in each tier-1 federation

Case3_tier0

0

20000

40000

1 6 11 16 21 26 31 36 41

050000

100000150000200000250000300000350000

Late

ncy

(ms)

898513 8121 25 7733 37 736949 6157 93 975345412917 655 91

Request

1120582 = 333

1120582 = 666

1120582 = 999

(b) 40000 VRIs in each tier-1 federation

Figure 8 Mean latencies in cases of different resource distributions

metric is the missing rate defined as (119879 minus 119878)119879 where 119879

is the number of total requests and 119878 is the number ofsuccessful requests Table 2 shows the experimental resultsunder different amount of users from 100 to 300 We can seethat when the number of users is low (100 users) both CSIdesigns can successfully process their data requests As thenumber of users increases (eg 300 users) our proposed CSIcan almost process it with only 0 27missing rate howeverover 27 requests are failed to deliver data to cloud databasein the CSI of single machine

6 Conclusion

We have proposed and implemented a mobile device-centriccloud computing architecture HiBA based on feedback con-trol framework The proposed hierarchical brokering archi-tecture is self-organized featuring scalability and hierarchicalautonomy and is easy to embed management and controlalgorithms to develop a federation with better performancein terms of availability and latency Mobile devices and smallservers federate into cloudlets and cloudlings of hierarchical

10 Mobile Information Systems

Table 2 Experimental comparison of different CSI designs

Users Single Load-balanced schemeTotal requests Successful requests Missing requests Total requests Successful requests Missing requests

100 52395 52395 0 53370 53370 0200 97457 80947 1691 104244 104244 0300 113624 81883 2794 131742 131388 027

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 251

No control-ASUS-FonepadNo control-ASUS-NEXUSNo control-HTC-M7No control-SAMSUNG-GTI8260With control-ASUS-FonepadWith control-ASUS-NEXUSWith control-HTC-M7With control-SAMSUNG-GTI8260

0102030405060708090

100

()

Figure 9 Energy-aware balancing result 119909-axis round 119910-axisremaining energy

02468

101214161820

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 241

ASUS-FonepadASUS-NEXUSHTC-M7

SAMSUNG-GTI8260Th

Figure 10 Data amount to energy proportions of individual devices(119883119894) and the cloudlet federation (119879ℎ) 119909-axis round 119910-axis dataamount to energy proportions

tiers such that mobile devices not only request servicesbut also provide resources required by diverse servicesThe implemented HiBA architecture with feedback controlframework proves improved performancewhen resources areadequately distributed to lower tier federations rather thanbeing centralized in a remote cloud server Through two casestudies of cloudlet energy-aware balancing and bigger data

center load balancing we show that the HiBA is actually adevelopment platform for mobile cloud computing and pro-vides a feasible solution to UCN services Future worksinclude optimization algorithms embedding and big dataanalytics applicationswith crowd sensing are to be continued

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

Thanks are due to the Ministry of Science and Tech-nology (MOST which was the National Science Coun-cil) in Taiwan for sponsoring this research under con-tinuous funded Projects NSC101-2221-E-327-020- NSC101-2221-E-327-022- NSC102-2218-E-327-002- MOST103-2221-E-327-048- MOST105-2221-E-327-027- andMOST105-2221-E-006-141-

References

[1] Technicolor ldquoUser-Centric Networkingrdquo httpusercentricnet-workingeuabout-ucn

[2] G Iosifidis L Gao J Huang and L Tassiulas ldquoIncentive mech-anisms for user-provided networksrdquo IEEE CommunicationsMagazine vol 52 no 9 pp 20ndash27 2014

[3] B A A Nunes M A S SAntos B T De OliveirA C B MArgiK ObrAczkA and T Turletti ldquoSoftware-defined-networking-enabled capacity sharing in user-centric networksrdquo IEEE Com-munications Magazine vol 52 no 9 pp 28ndash36 2014

[4] G Aloi M D Felice V Loscrı P Pace and G Ruggeri ldquoSpon-taneous smartphone networks as a user-centric solution for thefuture internetrdquo IEEECommunicationsMagazine vol 52 no 12pp 26ndash33 2014

[5] F Hao M Jiao G Min and L T Yang ldquoA trajectory-basedrecruitment strategy of social sensors for participatory sensingrdquoIEEE Communications Magazine vol 52 no 12 pp 41ndash47 2014

[6] M-L LeeH-YChung andF-MYu ldquoModeling of hierarchicalfuzzy systemsrdquo Fuzzy Sets and Systems vol 138 no 2 pp 343ndash361 2003

[7] T C Lin M-Y Pai C-L Chen and C-C Chen ldquoLoad-balanced cloud service interface for the HiBA mobile cloudenvironmentrdquo in Proceedings of the IEEE International Confer-ence onConsumer ElectronicsmdashTaiwan (ICCE-TW rsquo15) pp 360ndash361 Taipei Taiwan June 2015

[8] C-L Chen S-C Chen C Chang and C Lin ldquoScalable andautonomous mobile device-centric cloud for secured D2Dsharingrdquo in Information Security Applications vol 8909 ofLecture Notes in Computer Science pp 177ndash189 2015

Mobile Information Systems 11

[9] W-T Su W Liu C-L Chen and T-P Chen ldquoCloud accesscontrol in multi-layer cloud networksrdquo in Proceedings ofthe IEEE International Conference on Consumer ElectronicsmdashTaiwan (ICCE-TW rsquo15) pp 364ndash365 IEEE Taipei Taiwan June2015

[10] H Huang H Yang and E H Lu ldquoA fuzzy-rough set basedontology for hybrid recommendationrdquo in Proceedings of theIEEE International Conference on Consumer ElectronicsmdashTaiwan (ICCE-TW rsquo12) pp 358ndash359 Taipei Taiwan June 2015

[11] C-L Chen and C-T Chen ldquoHierarchical Brokering with feed-back state observation in Mobile Device-Centric Cloudsrdquo inProceedings of the 7th International Conference on Ubiquitousand Future Networks (ICUFN rsquo15) pp 410ndash415 July 2015

[12] C J S Decusatis A Carranza and C M Decusatis ldquoCom-munication within clouds open standards and proprietaryprotocols for data center networkingrdquo IEEE CommunicationsMagazine vol 50 no 9 pp 26ndash33 2012

[13] M Satyanarayanan P Bahl R Caceres andNDavies ldquoThe casefor VM-based cloudlets in mobile computingrdquo IEEE PervasiveComputing vol 8 no 4 pp 14ndash23 2009

[14] S Mahadev ldquoMobile computing the next decaderdquo in Proceed-ings of the the 8th International Conference on Mobile SystemsApplications and Services (MobiSysrsquo10) pp 1ndash6 San FranciscoCalif USA June 2010

[15] C-L Chen J-W Lee W-T Su M-F Horng and Y-HKuo ldquoNoise-referred energy-proportional routing with packetlength adaptation for clustered sensor networksrdquo InternationalJournal of Ad Hoc and Ubiquitous Computing vol 3 no 4 pp224ndash235 2008

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

2 Mobile Information Systems

for both control and management planes in UCNs Theidea comes from the hierarchical fuzzy control [6] where acomplex control problem with huge state space is conqueredIn a hierarchical control system the cross product of statespaces of all control hierarchies remains huge while eachhierarchy adopts only a few fuzzy rules to perform simplecontrol tasks

A series of contributions adopting MDC2 for improvingload balance [7] user-centric security [8] access control[9] in the cloud as well as MDC2 applications of real-timeseamless video sharing [8] and sociological ontology-basedhybrid recommendation [10] were proposed In these contri-butions we see that MDC2 is a feasible solution to provideservices in UCN In this paper HiBA with feedback controlframework generalizes the brokering and state observationAvailability andnetwork latency are analyzed and experimen-tal HiBA comprising real-world mobile devices and serversare realized We prove that a UCN with huge state spaceis observable and hence controllable through HiBA cloudnetworking In this paper we further extend our previouspresentation [11] to demonstrate algorithm embedding usingthe MMCN development platform in both lower and highertiers of HiBA We study energy-aware balancing in thecloudlet tierwhere densemobile devices federate In additionwe study the load balancing in bigger data centers where largeamount of requests usually arrives in a short time Throughboth theoretical analysis and real-world experiments weconclude that a HiBA federation with MMCN is a platformfor developing distributed algorithms in future 5Grsquos massivemachine-to-machine (M2M) communications and UCNs

This paper is organized as follows Section 2 depictsthe HiBA architecture and the feedback control frameworkMathematical analysis on availability and network latencyis provided in Section 3 Section 4 demonstrates real-worldexperiments in which the HiBA federates heterogeneousmobile and fixed devices in three tiers using different networkinterfaces Energy-aware balancing for a cloudlet and loadbalancing for a bigger data center are both conductedWe alsopresent conclusions at the end of Section 5

2 System Architecture

Unlike traditional data centers that span trees of VMs ina top-down manner from a primary computing domainusually called domain zero in a physical machine (PM)the proposed HiBA architecture is initiated from physicalmobile devices at the bottom tier called the cloudlet tierEach broker dynamically connects and adaptively controlslower tier PMs or VMs and thus can avoid drawbacks suchas the degradation of the aggregate available bandwidth andcircuitous communication path of VM spanning trees [12]The system includes resources in the cloudlets then associatescloudlets with private and public clouds to further scale upthe cloud federation On the management aspect we exploitadaptive feedback control framework to tackle the uncertaindynamics of mobile ad hoc networks in order to adapt tothe dynamics such as dynamic joining and leaving a brokerrsquosdomain caused by user mobility device failure or physicalnetwork handoff [8]

User request initiator Service provider

Cloudling tier

Cloud rack tier

Cloud bank tier

Cloudlet tier

Public cloud broker Cloudling broker

User cloudlet brokerUsersrsquo resource instances

Public clouds

private cloudsMobile

Figure 1 Hierarchical brokering architecture (HiBA) for mobiledevice-centric cloud computing

21 Hierarchical Brokering Figure 1 shows the HiBA archi-tecture which is comprised of four tiers Tier 0 called thecloudlet tier contains groups of mobile user devices Eachcloudlet is managed by a cloudlet broker maintaining net-work connectivity andmember association and also perform-ing QoS control A cloudlet is dynamically organized by thebroker according to resource instant status including CPUmemory network capability and energy consumption Theresources underlying a tier is abstracted as a JSON or XMLlist in its brokerrsquos database accessible to federation membersthrough RESTful API Thus each entry of the list includesa URL containing available RESTful command or a URL offilename and file attributes Fields of security configurationsNAT port and GPS coordinates are optionally included ineach entry [8] A tier-0 device does not have a VM memberexcept itself and it aggregates its resources to be shared withothermembers in the resource list Each broker hierarchicallyaggregates the resource lists from members and uploadsupdates to its upper tier broker

We continue using the cloudlet federation procedurein our previous research [8] A cloudlet broker is eitherdynamically elected among user devices according to theresources status or selected by its upper tier (tier 1) brokercalled cloudling broker A cloudling broker is a small datacenter or private cloud proximate to a cluster of cloudlets thatassociate with the cloudling It is similar to the small datacenter proposed in [13 14] owned by a smaller business unitsuch as a coffee shop or a clinic We differ from [13 14] inthat our cloudlets are autonomously grouped by user deviceswhich also share resources with others A tier-119899 brokerassociates with a tier-(119899 + 1) broker to scale the federation indepth or alternatively to include more tier-(119899 minus 1) federationsto scale in width In the proposed architecture each brokeronly recognizes next lower tier brokers while further lowertiers are transparent to it Furthermore each broker has thesame feedback control framework for managing lower tierdevicesrsquo network attachments and resource associations aswell as for QoS control on requesting and executing services

Mobile Information Systems 3

instanceUpper tier

CIScloud

inspection

CCScloud

controlTask assignment

Queryresource

Control

Forwardedrequest

Internal state update

Lower tier state update

Observedstate

State update

Forwarded request

SQLdatabase

Local request

Lower tierinstances

CEScloud

execution

Task assignments

Tier (n minus 1)

tier nInstance in of

Tier (n + 1) inminus1

in+1

Figure 2 The feedback control framework MMCN in a HiBA broker

22 Feedback Control Framework Figure 2 shows the feed-back control framework MMCN of each broker in the HiBAvirtualized network The feedback loop is used to adaptivelymanage and control the cloud federation governed by thebroker This management includes network attachments andresource associations caused by usermobility andVMmigra-tion [8] In future application the framework is ready fordeveloping hierarchical control algorithms such as requestdifferentiation and task scheduling as well as real-timeQoS control and job tracking once a service is started Afeedback control loop comprises three subsystems includingCloud Inspection Subsystem (CIS) CloudControl Subsystem(CCS) and Cloud Execution Subsystem (CES) which areanalogous to observer controller and plant respectively in aclassic feedback control system When performing real-timecontrol for QoS multiple lower tier instances of the feedbackloop are allocated A tier-119899 CCS determines the number oftier-(119899 minus 1) instances to be allocated then the tier-119899 CESinvokes the required instances each of which is constitutedby the other set of CIS CCS and CES at the tier-(119899 minus 1)

A CCS processes tasks assigned by its upper tier brokerand processes requests from lower tiers Then according tothe database and states tracked by the CIS subsystem itdetermines the controls including admission of memberjoining allocating member federations for task executionschedules of tasks and service level agreements (SLAs) oftasks to be assigned tomember federations A number of CESinstances physically accomplish the network attachment forthe admission control as well as the recruiting of membersassociated with corresponding SLAs to complete currentschedule A CCS produces these determinations as controlsto CES subsystems while the executions are left to CESs andassociated member federations The CIS observes the stateupdates from lower tiers for CCSrsquos reference when deter-mining these controls

23 Algorithm Embedding It is obvious that this frameworkis feasible for future development of hierarchical controland management algorithms including admission controlrequest differentiation task partition adaptive resource allo-cation and dynamic task scheduling regarding SLAs assignedby upper tier broker For example on receiving a new requesta priority queue of requests is adapted with new set ofprioritiesTheCCS checks with the CIS to determinewhetherto forward the request to upper tier broker or to reduce therequest into smaller tasks such that the CESs can furtherassign the task partitions to lower tier members just as theprocessing of task assignments from upper tier brokerrsquos CESTheCESrsquosmakes lower tiers transparent to theCCS and uppertiers as if it is the single plant of the CCS controller Sinceit is the CESs and associated member federations executingtasks rather than the CCS the CCS is able to process themanagement and control algorithms without waiting for theend of the current job executionThis is easy to be realized byprogramming multiple threads each of which realizes CCSCESs and CIS subsystems

In summary the CCS of a tier-119899 broker forwards requeststo tier-(119899 + 1) if it is not able to process them accordingto the CIS database The tier-(119899 + 1) broker assigns jobs toother tier-119899 brokers through CES if the requested resourcesare sharable in these tier-119899 members and below Thereforethe sharing becomes intercloud service of tier-119899 as well asintracloud service of tier-(119899 + 1)

3 Performance Analysis

Thebaseline performance for benchmarking is to evaluate theHiBA architecture and feedback control framework withoutoptimization on specific management and control That isthe queues are simple FIFOs without priority adaptationand all devices in all tiers randomly generate requests The

4 Mobile Information Systems

120582n0

tn

tn

tn

Dn0

B120582nminus1120582nminus1120582nminus1

tnminus1 tnminus1 tnminus1 tnminus1 tnminus1 tnminus1

Cnminus10

Dnminus10

Cnminus12

Dnminus12

Cnminus1B

Dnminus1B

middot middot middot

middot middot middotmiddot middot middotmiddot middot middot

t1 t1 t1 t1 t1 t1

12058200 1205820B 1205820B+1 1205820j 1205820k 1205820m

C00 C0B C0B+1 C0j C0k C0m

D00 D0BD0B+1 D0j D0k D0m

Cn0

Figure 3 Performance model of the HiBA architecture

performance model of the virtualized network is shown inFigure 3 Suppose that themaximumnumber ofmembers of aHiBA broker is 119861 a broker 119895 in tier 119894 has computing capability119862119894119895 which is also the minimum length of the Task Queue

The capability 119862119894119895also represents the maximum number of

tasks that can be processed by a broker in tier 119894 under thecondition that no task is droppedThe resources are assumedto be uniformly distributed in each tier In tier 119894 the averageamount of contents and resources is 119863

119894119895 A tier-119894 broker has

its amount of local requests in a Poisson distribution withmean 120582

119894119895 This simulates the reinitiated requests caused by

request partitions and task handoffs 119905119899denotes the mean

initial latency caused by network delay to forward requestsand the waiting time that requests stay in the Request Queueuntil being partitioned into smaller tasks In this paper wederive baseline performances on availability and latency

31 Resource Availability We define availability 119886119894119895 as the

ratio of computing capability119862119894119895over the number of accepted

requests 119877119894119895to a broker 119895 in tier 119894 That is

119886119894119895= min(

119862119894119895

119877119894119895

1) (1)

In a HiBA architecture of 119899 tiers the expected total resourceamount summing from the records in the CIS databases isapproximated as

119863total = 119863119899+

119899

sum

119894=1

(

119894minus1

prod

119896=0

119861119899minus119896

)119863119899minus119894

=

119899

sum

119894=0

119861119894119863119899minus119894

(2)

supposing that119863119894= 119864[119863

119894119895]Thus the expected total resource

amount of the subtree rooted at a tier-119894 node is

119863119894total =

119894

sum

119896=0

119861119896119863119899minus119896

(3)

For any request the probability that the requested resource 119903119894119895

is available in local node 119894119895 is approximated by

119875 (119903119894119895) = 119875 119903

119894119895isin D119894119895 =

119863119894119895

119863total (4)

where D119894119895is the set of resources in node 119894119895 Suppose that

the sample space is the union of D119894119895in the whole HiBA

architecture Then the worst case of availability occurs whennetwork latency is much smaller than the mean time interval1120582119894119895between two consequent requestsThat is requests from

other cloud federations including those in lower and highertiers arrive in current tier instantly with negligible networkdelayThus the maximal number of requests arriving node 119894119895is

119877119894119895= 119875 (119903

119894119895)

119899

sum

119896=0

119861119896120582119899minus119896119895

(5)

Then we obtain the worst case availability as

119886119894119895= min(

119862119894119895

119875 (119903119894119895)sum119899

119896=0119861119896120582119899minus119896119895

1)

= min(119862119894119895

119863119894119895

sdot

119863totalsum119899

119896=0119861119896120582119899minus119896119895

1)

(6)

Mobile Information Systems 5

From (6) we see that when 119899 is smaller resources (includingcontents) are more centralized with larger 119863

119894119895and if 119863total

remains large this causes lower availability Thus there arevarious means to increase availability One is to increasecapability 119862

119894119895 though this will increase costs Alternatively

branch amount 119861 of each tier can be increased though thisalso will increase computing capability and costs Third byadopting HiBA we put user devices in tier 0 or tier 1 (oncethe user device is elected cloudlet broker) However thecapabilities 119862

0119895and 119862

1119895of thin devices may be small and

so 1198630119895and 119863

1119895are also expected to be small This indicates

that availability remains close to 1 if resource distribution119863119894119895

is proportional to the capability 119862119894119895with the ratio of total

number of requests over total resource amountTherefore theoptimal resource distribution to low tier brokers and mobiledevices both offloads data centersrsquo computing overhead andincreases user satisfaction

32 Latency Social networks can cause locality such thatmany requests do not travel to their destinations over longdistances in terms of either network relay counts or theterrestrial radio barriers Thus initial latency of a serviceis reduced Lower tier brokers have smaller resource gran-ules while locality based on social network provides accessproximity and thus results in shorter latency We estimatethe latency of a request traversing the HiBA architecture asfollows Suppose that a request initiated from a tier-119894 nodearrives at its destination possessing the required resources intier 119897 119894 lt 119897 The expected latency of the request is

119879119894119897= 119875 (119903

119894lt119897)

119897

sum

119896=119894

(119905119873119896

+ 119905119863119896

+ 119905119862119896

) (7)

where 119875(119903119894lt119897) is the probability that the requested resource

is not found in tiers lower than 119897 and 119905119873119896

119905119863119896

119905119862119896

arelatencies caused respectively by network communicationrequest queuing plus database querying and computing forthe brokering at each tier-119896 broker on the path to tier 119897Supposing that request arrival is independent of resourcedistribution we have

119875 (119903119894lt119897) =

120582119894

120582total(1 minus

119863119897minus1total

119863total) (8)

Thus the estimated latency is

119879119894119897=

120582119894

120582total(1 minus

119863119897minus1total

119863total)(

119897

sum

119896=119894

119905119873119896

+ 119905119863119896

+ 119905119862119896

) (9)

If a request is originated from a lower tier user device thatis 119894 = 0 or 1 the effective way to reduce expected latency1198790119897or 1198791119897is to reduce 119875(119903

119894lt119897) This means increasing119863

119897minus1totalby distributing resources to lower tiers that is increasing119863

119896

for 119896 lt 119897 However this also increases both database search-ing time 119905

119863119896and computation time 119905

119862119896 Considering network

delay 119905119873119896

we can expect that in the virtualized network thephysical distance of a cloud server is increasing as 119896 is gettinglarger A high tier link in the virtualized HiBA networkactually contains many more physical relaying hops than a

lower tier link If request is sent using TCP protocol thenetwork latency is increasing muchmore than that caused bydatabase searching and brokerage computing as 119896 increasesFrom (9) we still effectively decrease the latency by increasing119863119897minus1total This can be expected especially for multimedia

content sharing since according to social network relationscontents are stored in proximate cloudlets or cloudlingswhich are lower tiers (small 119897)

33 Development Platform for MDC2 Control and Manage-ment From the availability and latency analysis we see thata HiBA with feedback framework is a development platformIt is easy to observe the performance when developingalgorithms for admission control request differentiation taskpartition adaptive resource allocation and dynamic taskscheduling regarding SLAs assigned by upper tier broker Forexample the admission control in federating a tier affectsresource distribution 119863

119894119895and according to the number of

network hops between a broker and a member the networklatency 119905

119873119896in (9) also differs The request differentiation

and queuing mechanism directly affect 119879119894119897since 119905

119863119896includes

the queue delay that a request stays in the Request QueueTask partitioning resource allocation and scheduling furtheraffect the efficiency processing requests and consequentlythey further affect 119905

119863119896and resultant 119879

119894119897 Globally they also

affect arrival rate 120582119894at tier 119894 Exploiting hierarchical feedback

control it is easy to ensure availability by performing propermanagement while tracking the latency by tuning controlregarding the deadline specified in the SLA of each task

4 Case Studies

To demonstrate algorithm embedding using the MMCNdevelopment platform in both lower and higher tiers westudy energy-aware balancing among mobile devices in thecloudlet tier as well as the load balancing in bigger datacenterswhere large amount of requestswould arrive in a shorttime that is large 120582

119894119895 The energy-aware balancing is critical

in crowd-sensing applications especially when the sensingdata amount is large such as using cameras for image datacollectingThe load balancing is essential for bigger data cen-ters especially when database access frequency is high Bothof the balancing algorithms are effective in the Industry 40era when crowd-sensing and big data analytics are deployed

41 Energy-Aware Balancing The energy-aware balancing isbased on unsupervised fuzzy feedback control where thereference command is also adapted according to the feedbackstate of the federation self-organized by mobile devices Theprincipal idea comes from the energy proportional routing[15] that the lifetime of a clustering-based sensor network isprolonged ifmember nodesrsquo proportions of consumed energyin the remaining are close to the clusterrsquos We exploit theenergy proportion of the cloudlet federation as the adaptivereference to be tracked by the fuzzy feedback control systemTherefore the energy sharing is unsupervised because of nogiven objective of the control a priori

6 Mobile Information Systems

0

1

0

1

120583

120583

N P

e

N998400 P998400

e998400

minus256 0 256

minus512 512

Figure 4 Membership functions for fuzzy sets119873 1198751198731015840 and 1198751015840

Suppose that in a cloudlet tier of119873member nodes datatransmission is observed by the cloudlet broker in each round119905 (discrete time) We first define symbols as follows

(i) 119896 member node identification 1 le 119896 le 119873

(ii) 119863119896(119905) data transmission amount being assigned to

node 119896 at round 119905

(iii) 119864119896(119905) remaining energy of node 119896 at 119905

(iv) 119883119896(119905) = 119863

119896(119905)119864119896(119905) representing the ratio of over-

head to the remaining energy (current capability)This is the indication of how node 119896 is loaded withrespect to the remaining energy

(v) 119879ℎ(119905) = sum119873

119896=1119863119896(119905)sum

119873

119896=1119864119896(119905) representing how the

whole cloudlet federation is loaded

(vi) 119890119896(119905) = 119883

119896(119905) minus 119879ℎ(119905) representing the difference of

loading between node 119896 and the whole federation

(vii) 1198901015840119896(119905) = 119890

119896(119905)minus119890119896(119905minus1) representing how the difference

changes

(viii) 119903119896(119905) the ratio of data amount assigned to node 119896 at 119905

We define fuzzy rules as follows

(R1) If 119890119896(119905) is 119875 and 119890

1015840

119896(119905) is 1198751015840 then 119903

119896(119905 + 1) is 119878

119896(119905)

(R2) If 119890119896(119905) is 119875 and 119890

1015840

119896(119905) is1198731015840 then 119903

119896(119905 + 1) is119872

119896(119905)

(R3) If 119890119896(119905) is119873 and 119890

1015840

119896(119905) is 1198751015840 then 119903

119896(119905 + 1) is119872

119896(119905)

(R4) If 119890119896(119905) is119873 and 119890

1015840

119896(119905) is1198731015840 then 119903

119896(119905 + 1) is 119871

119896(119905)

Themembership functions for fuzzy sets119873 1198751198731015840 and 1198751015840 areconfigured in Figure 4

We realize the ldquoandrdquo operator with 119905-norm ldquominimumrdquoThat is the matching degrees 120583

1 1205832 1205833 and 120583

4of premise

part of rules (R1) to (R4) respectively are

1205831 (119905) = min (120583

119875 (119890 (119905)) 120583119875

1015840 (1198901015840(119905)))

1205832 (119905) = min (120583

119875 (119890 (119905)) 120583119873

1015840 (1198901015840(119905)))

1205833 (119905) = min (120583

119873 (119890 (119905)) 120583119875

1015840 (1198901015840(119905)))

1205834 (119905) = min (120583

119873 (119890 (119905)) 120583119873

1015840 (1198901015840(119905)))

(10)

Obtaining the matching degrees 1205831 1205832 1205833 and 120583

4of the

four fuzzy rules respectively we perform the defuzzificationequivalently applying the Takagi-Sugeno inference methodThe inference result adequate ratio of data amount totransmit is

119903119896 (119905 + 1)

=

1205831 (119905) 119878 (119905) + 120583

2 (119905)119872 (119905) + 120583

3 (119905)119872 (119905) + 120583

4 (119905) 119871 (119905)

1205831 (119905) + 120583

2 (119905) + 120583

3 (119905) + 120583

4 (119905)

(11)

by configuring the conclusion part membership functions asdynamic singletons as follows

119871 (119905) = 119903119896 (119905 minus 1) + Δ

119872(119905) = 119903119896 (119905 minus 1)

119878 (119905) = 119903119896 (119905 minus 1) minus Δ

(12)

where Δ is the ratio tuning amount for each round Finallythe data amount assigned to node 119896 is determined as

119863119896 (119905 + 1) =

119903119896 (119905)

sum119873

119894=1119903119894 (119905)

D (119905) (13)

where D(119905) is the total data amount to be transmitted fromthis cloudlet at time 119905 In the above fuzzy inference algorithmeach member node tracks the dynamic control goal 119879ℎ(119905)and the proportion of energy to be consumed in the remain-ing energy approaches the proportion regarding the wholefederation Therefore the intrafederation energy sharing isachieved by offloading transmission jobs using the proposedalgorithm When each node 119896 is a lower tier federation theoffloading is hierarchically extended through the MMCNnetworking where the data amount 119863

119896(119905) is determined and

assigned by CCS and the energy status updates 119864119896(119905)rsquos are

received and aggregated by CIS This shows the scalability ofthe HiBA architecture

42 Load Balancing The load balancing is required in abigger federation organized in a higher tier of HiBA becausea higher federation always has larger amount of requestarrivals As shown in Figure 5 the framework of the proposedload-balanced cloud service interface consists of three com-ponents which realizes CES CCS and CIS respectively Thedetails of components are presented as follows

(1) Cloud Service Interface Node (CSIN) it is a virtualmachine that provides cloud service interface and is

Mobile Information Systems 7

Service module 1

Service module 2

Service

CSIallocation

table

Clouddatabase

HiBA networking

Cloud bank tier

(1)(3)

(4)

(5)

(6)

(7)(8)

(2)CSI

manager CSIN 1(9)

CSIN n

module m

middot middot middotmiddot middot middot

middot middot middot

Figure 5 Framework of load-balanced cloud service interface

denoted as CSINi in Figure 5 A CSIN can receivemembersrsquo requests and obtain results by interactingwith requested cloud service module In our designeach cloud service interface in CSIN is implementedas a RESTful API for universally communicating withdifferent types of mobile devices A CSIN is then therealization of CES in the MMCN framework

(2) CSImanager it is designed tomanage the usage statusof CSINs andmatchesmobile users to CSINsThe effi-ciency of the load balance depends on the matchingmethod of the CSI Manager A CSI manager is thenthe realization of CCS in the MMCN framework

(3) CSI allocation table it maintains the serving statusbetween mobile users and CSINs and is used by theCSI manager for assisting the matching decision forthe new-arrived mobile users This table is stored inthe cloud database A CSI allocation table is then therealization of CIS in the MMCN framework

The processing flow of using a cloud service for a mobile useris designed for achieving the load balance The idea of com-mitting requests requires three phases (1) service registration(Steps (1)-(2)) (2) service execution (Steps (3)ndash(7)) and (3)service deregistration (Steps (8)-(9)) The detailed steps alsoillustrated in Figure 5 are presented as follows

(1) Mobile user MU119894sends a request to CSI manager for

allocating a CSI address(2) CSI manager searches for a CSI machine with least

number of serving users say CSIN119899 in the CSI allo-

cation table and then registers (MU119894 CSIN

119899) into the

CSI allocation table(3) CSI manager informs MU

119894that CSIN

119899can serve his

her cloud service requests in the following period(4) MU

119894configures the cloud service interface by replac-

ing the IP attribute in the service template with the IPaddress of CSIN

119899

(5) MU119894connects to CSIN

119899through the configured URL

of the cloud service interface and sends a request(6) CSIN

119899delivers the request to the associated cloud

service which may access the cloud databases ifrequired After processing the results will be sentback to CSIN

119899

(7) CSIN119899sends the results to MU

119894 (If more requests are

required to processed Steps (3)ndash(7) are repeated)(8) MU

119894deregisters CSIN

119899to CSI manager

(9) CSI manager removes the registration record of MU119894

from the CSI allocation table

Notice that the matching rule for a new mobile user andCSINs in the current design uses least-user-first basis asshown in Step (2) The matching principle can be modifiedaccording to different demands In the experiments wewill show that the current design already obtains splendidperformance More study on customizing matching methodsis one of our future research directions That is althoughfive machines are used in our CSI (more than the single-machine CSI) the improvement is by a factor of one hundredindicating the superior performance of our proposed load-balanced CSI mechanism under the HiBA architecture

5 Real-World Experiments

51 HiBA Baseline Performance We conduct an experimentdemonstrating performance difference in terms of availabil-ity and latency when distributions of virtualized resourceinstances (VRIs) differ We leave development of enhancingalgorithms for specific purposes in the CCS as future workwhich will seek performance in terms of various metricsA total of 50000 VRIs are distributed in a 3-tier HiBAfederation comprising Android smart phones tablets andVMs in desktop PCs Figure 6 is the HiBA topology andrelated specifications of the machines are shown in Table 1Each device is labeled (119894 119895) if it is the 119895th node at tier 119894 Device(1 3) connects to (2 0) through a 3G access point with VPNtunneling Device (0 11) connects to (1 0) with USB andshares Internet connection from (1 0) These machines havedifferent computing ability though request and task queuesbeing of the same size of 20 to preserve the differences ofcomputing ability Requests are randomly generated in eachdevice Mean request arrivals (120582) are 3 15 and 1 whichequivalently mean that request intervals (1120582) are 333ms666ms and 1 s respectively The resource distributions havefour casesThe first case is that the tier-2 device possess all the50000 VRIs while 16000 VRIs are duplicated to each tier-1 federations In the remaining three cases we respectivelyduplicate 28000 32000 and 36000 VRIs to each tier-1federation The experiment is conducted in a heterogeneousnetworking environment connecting machines by differentcommunication technologies shown in Table 1

The results of the experiment are shown in Figures 7 and8 Both Figures 7 and 8 reveal that distributing resources inlower tiers provides better performance Figures 8(a) and 8(b)show the latency differences caused by the resource distri-butions that 16000 32000 and 40000 VRIs are duplicatedin each cloudling tier The results before all brokers fullyloaded are also given with respective zoom-in charts Whendevices all continuously issue requests at a high frequency (3requests per second) the queue delays are obviously high andwe see a few requests that are dropped Requests from tier-1devices have lower loss rate since the resources requested are

8 Mobile Information Systems

Table 1 Machines in the offloading experiments using heterogeneous communication technologies

(Tier device) Specifications Physical network linkTier 2(2 0) Desktop PC Intel Core i5-3470 CPU 32GHz 4G RAM

EthernetTier 1(1 0) (1 1) Desktop PC Intel Core i7-4770 CPU 34GHz 8G RAMTier 1(1 2)Tier 1(1 3) Laptop Intel Core i7-2820QM CPU 23GHz 8G RAM Down WiFi UP 3G VPNTier 0(0 0sim3)

Desktop PC Intel Pentium D CPU 34GHz 1 G RAM EthernetTier 0(0 4sim6)Tier 0(0 7)Tier 0(0 8) Tablet ASUS Fonepad Intel Atom Z2420 12 GHz

WiFiTier 0(0 9) Cell Phone S3 SAMUNG Exynos 4412 14 GHzTier 0(0 10) Tablet ASUS Nexus7 NVIDIA Tegra 3 4-coreTier 0(0 11) Tablet WS-170 Qualcomm QCTMSM8x60 surf15 GHz duo-core USBlarrrarr PClarrrarr EthernetTier 0(0 12sim13) Virtual Machines (VirtualBox) EthernetTier 0(0 14) Tablet Samsung Galaxy CPU GT-I8260 WiFi

(011)

LAN LAN WiFi 3G

Tier 2

Tier 1

Tier 0

VPN

LAN LAN WiFi WiFi

(0 14)

(0 8)

(0 5)

(0 12)

(0 13) (0 9)

(0 7)(0 4) (0 6)(0 3)

(0 1)(0 10)

(0 2)(0 0)

(1 3)

(2 0)

(1 1)(1 0) (1 2)

Figure 6 Real-world HiBA topology for offloading performancemeasurement Gray nodes are local VMs of respective tier-1 brokersDiverse physical network links are exploited in tiers of federations

obtained in a shorter time For each physical machine theCPU utilization of theMMCNbrokering itself is smaller than2 depending on the number of cores The mean brokeringcomputation and database querying time is estimated tobe about 500ms while the WLAN delay time is smallerthan 5ms In the heterogeneous network case the latencydifference will be larger if upper tiers are physically at a longdistance from the cloudlings and cloudlets If the requestinterval is higher than the processing time the latency tendsto be smaller When more VRIs are duplicated in lower tierdevices we also see that the latencies are smaller

In this paper we leave algorithms such as request par-tition network embedding and federation for specific per-formance metrics as future work Instead we implementreal-world HiBA to prove that with feedback framework it isa new design paradigm and development platform for thesealgorithms

52 Energy-Aware Balancing In the energy-aware balancingexperiment we exploit four mobile devices of different typesfrom different manufacturers The cloudlet is self-organizedaccording to the broker election protocol in [8] The electedbroker is ASUS-Fonepad The other hierarchical federations

are in Figure 6 Each device including the broker itselfin each round updates its energy status to the CIS of thebroker (ASUS-Fonepad) Each mobile device in the cloudletis recharged to 100of respective battery energy capacityThedata and total data amount to be transmitted in each roundare randomly generated with mean of 24GB prior to theexperiment For each round 119905 total data amount is the samefor both balancing and nonbalancing cases In nonbalancingcase the data amounts are evenly assigned to members Inbalancing case the data amounts are determined by the fuzzycontroller in Section 41 The data amount assignment is ofunit 100MB When any device has remaining energy loweror equal to 40 the experiment is terminated The initialsingletons are configured as 119878(0) = 3 119872(0) = 5 and 119871(0) =

7 (Δ = 2) The result of energy-aware balancing is shownin Figure 9 The respective control systemsrsquo behaviors are inFigure 10 The fuzzy controllers of mobile devices all trackthe federationrsquos data amount to energy proportion (119879ℎ) Wesee that no matter how the performance of respective devicevaries a device with better transmission capability will shareits energy with others Tablet PCs are usually with highercapacity batteries Without energy sharing the remainingenergy of a tablet drops more slowly than a mobile phoneWith energy sharing tablets undertake more transmissionjobs and becomewith higher energy drop rate and themobilephones become with lower energy drop rate However wesee that the whole cloudlet federation has longer lifetimeTheexperiment can also be applied to energy consuming tasks inaddition to data transmission

53 Load Balancing The load-balanced cloud service inter-face (CSI) in the form of RESTful APIs is implementedby using Jersey and is deployed on Microsoft WindowsAzure public cloud platform We conduct an experiment forcomparing the proposed CSI to that of single machine Inour experiment five CSI machines are used to share therequests from users In the experiment the data producingrate for each user is 1 request per second The performance

Mobile Information Systems 9

000 000000 000000 000000 000

000100200300

Homo Hetero16000 075 00028000 000 00032000 015 00036000 018 000

()

()

()

000100200300

Homo Hetero16000280003200036000

000100200300

Homo Hetero16000280003200036000

000 000000 012015 000000 000

Tier-1 1120582 = 666Tier-1 1120582 = 333

1600028000

3200036000

1600028000

3200036000

1600028000

3200036000

Tier-11120582 = 999

(a)

110 340050 230082 180080 170

030 330016 160000 130016 030

030 020000 000000 010000 000

1600028000

3200036000

000100200300

Homo Hetero16000280003200036000

()

000100200300

Homo Hetero16000280003200036000

()

000100200300

Homo Hetero16000280003200036000

()

Tier-0 1120582 = 666Tier-0 1120582 = 333

1600028000

3200036000

1600028000

3200036000

Tier-0 1120582 = 999

(b)

Figure 7 Mean losses of tier 0 (b) and tier 1 (a) in cases of different resource distributions when 16000 28000 32000 and 36000 VRIs areduplicated to each tier-1 federation Request intervals 1120582 are in milliseconds

Case3_tier0

0

20000

40000

1 6 11 16 21 26 31 36

050000

100000150000200000250000300000350000

Late

ncy

(ms)

898513 8121 25 7733 37 736949 6157 93 975345412917 655 91

Request

1120582 = 333

1120582 = 666

1120582 = 999

(a) 16000 VRIs in each tier-1 federation

Case3_tier0

0

20000

40000

1 6 11 16 21 26 31 36 41

050000

100000150000200000250000300000350000

Late

ncy

(ms)

898513 8121 25 7733 37 736949 6157 93 975345412917 655 91

Request

1120582 = 333

1120582 = 666

1120582 = 999

(b) 40000 VRIs in each tier-1 federation

Figure 8 Mean latencies in cases of different resource distributions

metric is the missing rate defined as (119879 minus 119878)119879 where 119879

is the number of total requests and 119878 is the number ofsuccessful requests Table 2 shows the experimental resultsunder different amount of users from 100 to 300 We can seethat when the number of users is low (100 users) both CSIdesigns can successfully process their data requests As thenumber of users increases (eg 300 users) our proposed CSIcan almost process it with only 0 27missing rate howeverover 27 requests are failed to deliver data to cloud databasein the CSI of single machine

6 Conclusion

We have proposed and implemented a mobile device-centriccloud computing architecture HiBA based on feedback con-trol framework The proposed hierarchical brokering archi-tecture is self-organized featuring scalability and hierarchicalautonomy and is easy to embed management and controlalgorithms to develop a federation with better performancein terms of availability and latency Mobile devices and smallservers federate into cloudlets and cloudlings of hierarchical

10 Mobile Information Systems

Table 2 Experimental comparison of different CSI designs

Users Single Load-balanced schemeTotal requests Successful requests Missing requests Total requests Successful requests Missing requests

100 52395 52395 0 53370 53370 0200 97457 80947 1691 104244 104244 0300 113624 81883 2794 131742 131388 027

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 251

No control-ASUS-FonepadNo control-ASUS-NEXUSNo control-HTC-M7No control-SAMSUNG-GTI8260With control-ASUS-FonepadWith control-ASUS-NEXUSWith control-HTC-M7With control-SAMSUNG-GTI8260

0102030405060708090

100

()

Figure 9 Energy-aware balancing result 119909-axis round 119910-axisremaining energy

02468

101214161820

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 241

ASUS-FonepadASUS-NEXUSHTC-M7

SAMSUNG-GTI8260Th

Figure 10 Data amount to energy proportions of individual devices(119883119894) and the cloudlet federation (119879ℎ) 119909-axis round 119910-axis dataamount to energy proportions

tiers such that mobile devices not only request servicesbut also provide resources required by diverse servicesThe implemented HiBA architecture with feedback controlframework proves improved performancewhen resources areadequately distributed to lower tier federations rather thanbeing centralized in a remote cloud server Through two casestudies of cloudlet energy-aware balancing and bigger data

center load balancing we show that the HiBA is actually adevelopment platform for mobile cloud computing and pro-vides a feasible solution to UCN services Future worksinclude optimization algorithms embedding and big dataanalytics applicationswith crowd sensing are to be continued

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

Thanks are due to the Ministry of Science and Tech-nology (MOST which was the National Science Coun-cil) in Taiwan for sponsoring this research under con-tinuous funded Projects NSC101-2221-E-327-020- NSC101-2221-E-327-022- NSC102-2218-E-327-002- MOST103-2221-E-327-048- MOST105-2221-E-327-027- andMOST105-2221-E-006-141-

References

[1] Technicolor ldquoUser-Centric Networkingrdquo httpusercentricnet-workingeuabout-ucn

[2] G Iosifidis L Gao J Huang and L Tassiulas ldquoIncentive mech-anisms for user-provided networksrdquo IEEE CommunicationsMagazine vol 52 no 9 pp 20ndash27 2014

[3] B A A Nunes M A S SAntos B T De OliveirA C B MArgiK ObrAczkA and T Turletti ldquoSoftware-defined-networking-enabled capacity sharing in user-centric networksrdquo IEEE Com-munications Magazine vol 52 no 9 pp 28ndash36 2014

[4] G Aloi M D Felice V Loscrı P Pace and G Ruggeri ldquoSpon-taneous smartphone networks as a user-centric solution for thefuture internetrdquo IEEECommunicationsMagazine vol 52 no 12pp 26ndash33 2014

[5] F Hao M Jiao G Min and L T Yang ldquoA trajectory-basedrecruitment strategy of social sensors for participatory sensingrdquoIEEE Communications Magazine vol 52 no 12 pp 41ndash47 2014

[6] M-L LeeH-YChung andF-MYu ldquoModeling of hierarchicalfuzzy systemsrdquo Fuzzy Sets and Systems vol 138 no 2 pp 343ndash361 2003

[7] T C Lin M-Y Pai C-L Chen and C-C Chen ldquoLoad-balanced cloud service interface for the HiBA mobile cloudenvironmentrdquo in Proceedings of the IEEE International Confer-ence onConsumer ElectronicsmdashTaiwan (ICCE-TW rsquo15) pp 360ndash361 Taipei Taiwan June 2015

[8] C-L Chen S-C Chen C Chang and C Lin ldquoScalable andautonomous mobile device-centric cloud for secured D2Dsharingrdquo in Information Security Applications vol 8909 ofLecture Notes in Computer Science pp 177ndash189 2015

Mobile Information Systems 11

[9] W-T Su W Liu C-L Chen and T-P Chen ldquoCloud accesscontrol in multi-layer cloud networksrdquo in Proceedings ofthe IEEE International Conference on Consumer ElectronicsmdashTaiwan (ICCE-TW rsquo15) pp 364ndash365 IEEE Taipei Taiwan June2015

[10] H Huang H Yang and E H Lu ldquoA fuzzy-rough set basedontology for hybrid recommendationrdquo in Proceedings of theIEEE International Conference on Consumer ElectronicsmdashTaiwan (ICCE-TW rsquo12) pp 358ndash359 Taipei Taiwan June 2015

[11] C-L Chen and C-T Chen ldquoHierarchical Brokering with feed-back state observation in Mobile Device-Centric Cloudsrdquo inProceedings of the 7th International Conference on Ubiquitousand Future Networks (ICUFN rsquo15) pp 410ndash415 July 2015

[12] C J S Decusatis A Carranza and C M Decusatis ldquoCom-munication within clouds open standards and proprietaryprotocols for data center networkingrdquo IEEE CommunicationsMagazine vol 50 no 9 pp 26ndash33 2012

[13] M Satyanarayanan P Bahl R Caceres andNDavies ldquoThe casefor VM-based cloudlets in mobile computingrdquo IEEE PervasiveComputing vol 8 no 4 pp 14ndash23 2009

[14] S Mahadev ldquoMobile computing the next decaderdquo in Proceed-ings of the the 8th International Conference on Mobile SystemsApplications and Services (MobiSysrsquo10) pp 1ndash6 San FranciscoCalif USA June 2010

[15] C-L Chen J-W Lee W-T Su M-F Horng and Y-HKuo ldquoNoise-referred energy-proportional routing with packetlength adaptation for clustered sensor networksrdquo InternationalJournal of Ad Hoc and Ubiquitous Computing vol 3 no 4 pp224ndash235 2008

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mobile Information Systems 3

instanceUpper tier

CIScloud

inspection

CCScloud

controlTask assignment

Queryresource

Control

Forwardedrequest

Internal state update

Lower tier state update

Observedstate

State update

Forwarded request

SQLdatabase

Local request

Lower tierinstances

CEScloud

execution

Task assignments

Tier (n minus 1)

tier nInstance in of

Tier (n + 1) inminus1

in+1

Figure 2 The feedback control framework MMCN in a HiBA broker

22 Feedback Control Framework Figure 2 shows the feed-back control framework MMCN of each broker in the HiBAvirtualized network The feedback loop is used to adaptivelymanage and control the cloud federation governed by thebroker This management includes network attachments andresource associations caused by usermobility andVMmigra-tion [8] In future application the framework is ready fordeveloping hierarchical control algorithms such as requestdifferentiation and task scheduling as well as real-timeQoS control and job tracking once a service is started Afeedback control loop comprises three subsystems includingCloud Inspection Subsystem (CIS) CloudControl Subsystem(CCS) and Cloud Execution Subsystem (CES) which areanalogous to observer controller and plant respectively in aclassic feedback control system When performing real-timecontrol for QoS multiple lower tier instances of the feedbackloop are allocated A tier-119899 CCS determines the number oftier-(119899 minus 1) instances to be allocated then the tier-119899 CESinvokes the required instances each of which is constitutedby the other set of CIS CCS and CES at the tier-(119899 minus 1)

A CCS processes tasks assigned by its upper tier brokerand processes requests from lower tiers Then according tothe database and states tracked by the CIS subsystem itdetermines the controls including admission of memberjoining allocating member federations for task executionschedules of tasks and service level agreements (SLAs) oftasks to be assigned tomember federations A number of CESinstances physically accomplish the network attachment forthe admission control as well as the recruiting of membersassociated with corresponding SLAs to complete currentschedule A CCS produces these determinations as controlsto CES subsystems while the executions are left to CESs andassociated member federations The CIS observes the stateupdates from lower tiers for CCSrsquos reference when deter-mining these controls

23 Algorithm Embedding It is obvious that this frameworkis feasible for future development of hierarchical controland management algorithms including admission controlrequest differentiation task partition adaptive resource allo-cation and dynamic task scheduling regarding SLAs assignedby upper tier broker For example on receiving a new requesta priority queue of requests is adapted with new set ofprioritiesTheCCS checks with the CIS to determinewhetherto forward the request to upper tier broker or to reduce therequest into smaller tasks such that the CESs can furtherassign the task partitions to lower tier members just as theprocessing of task assignments from upper tier brokerrsquos CESTheCESrsquosmakes lower tiers transparent to theCCS and uppertiers as if it is the single plant of the CCS controller Sinceit is the CESs and associated member federations executingtasks rather than the CCS the CCS is able to process themanagement and control algorithms without waiting for theend of the current job executionThis is easy to be realized byprogramming multiple threads each of which realizes CCSCESs and CIS subsystems

In summary the CCS of a tier-119899 broker forwards requeststo tier-(119899 + 1) if it is not able to process them accordingto the CIS database The tier-(119899 + 1) broker assigns jobs toother tier-119899 brokers through CES if the requested resourcesare sharable in these tier-119899 members and below Thereforethe sharing becomes intercloud service of tier-119899 as well asintracloud service of tier-(119899 + 1)

3 Performance Analysis

Thebaseline performance for benchmarking is to evaluate theHiBA architecture and feedback control framework withoutoptimization on specific management and control That isthe queues are simple FIFOs without priority adaptationand all devices in all tiers randomly generate requests The

4 Mobile Information Systems

120582n0

tn

tn

tn

Dn0

B120582nminus1120582nminus1120582nminus1

tnminus1 tnminus1 tnminus1 tnminus1 tnminus1 tnminus1

Cnminus10

Dnminus10

Cnminus12

Dnminus12

Cnminus1B

Dnminus1B

middot middot middot

middot middot middotmiddot middot middotmiddot middot middot

t1 t1 t1 t1 t1 t1

12058200 1205820B 1205820B+1 1205820j 1205820k 1205820m

C00 C0B C0B+1 C0j C0k C0m

D00 D0BD0B+1 D0j D0k D0m

Cn0

Figure 3 Performance model of the HiBA architecture

performance model of the virtualized network is shown inFigure 3 Suppose that themaximumnumber ofmembers of aHiBA broker is 119861 a broker 119895 in tier 119894 has computing capability119862119894119895 which is also the minimum length of the Task Queue

The capability 119862119894119895also represents the maximum number of

tasks that can be processed by a broker in tier 119894 under thecondition that no task is droppedThe resources are assumedto be uniformly distributed in each tier In tier 119894 the averageamount of contents and resources is 119863

119894119895 A tier-119894 broker has

its amount of local requests in a Poisson distribution withmean 120582

119894119895 This simulates the reinitiated requests caused by

request partitions and task handoffs 119905119899denotes the mean

initial latency caused by network delay to forward requestsand the waiting time that requests stay in the Request Queueuntil being partitioned into smaller tasks In this paper wederive baseline performances on availability and latency

31 Resource Availability We define availability 119886119894119895 as the

ratio of computing capability119862119894119895over the number of accepted

requests 119877119894119895to a broker 119895 in tier 119894 That is

119886119894119895= min(

119862119894119895

119877119894119895

1) (1)

In a HiBA architecture of 119899 tiers the expected total resourceamount summing from the records in the CIS databases isapproximated as

119863total = 119863119899+

119899

sum

119894=1

(

119894minus1

prod

119896=0

119861119899minus119896

)119863119899minus119894

=

119899

sum

119894=0

119861119894119863119899minus119894

(2)

supposing that119863119894= 119864[119863

119894119895]Thus the expected total resource

amount of the subtree rooted at a tier-119894 node is

119863119894total =

119894

sum

119896=0

119861119896119863119899minus119896

(3)

For any request the probability that the requested resource 119903119894119895

is available in local node 119894119895 is approximated by

119875 (119903119894119895) = 119875 119903

119894119895isin D119894119895 =

119863119894119895

119863total (4)

where D119894119895is the set of resources in node 119894119895 Suppose that

the sample space is the union of D119894119895in the whole HiBA

architecture Then the worst case of availability occurs whennetwork latency is much smaller than the mean time interval1120582119894119895between two consequent requestsThat is requests from

other cloud federations including those in lower and highertiers arrive in current tier instantly with negligible networkdelayThus the maximal number of requests arriving node 119894119895is

119877119894119895= 119875 (119903

119894119895)

119899

sum

119896=0

119861119896120582119899minus119896119895

(5)

Then we obtain the worst case availability as

119886119894119895= min(

119862119894119895

119875 (119903119894119895)sum119899

119896=0119861119896120582119899minus119896119895

1)

= min(119862119894119895

119863119894119895

sdot

119863totalsum119899

119896=0119861119896120582119899minus119896119895

1)

(6)

Mobile Information Systems 5

From (6) we see that when 119899 is smaller resources (includingcontents) are more centralized with larger 119863

119894119895and if 119863total

remains large this causes lower availability Thus there arevarious means to increase availability One is to increasecapability 119862

119894119895 though this will increase costs Alternatively

branch amount 119861 of each tier can be increased though thisalso will increase computing capability and costs Third byadopting HiBA we put user devices in tier 0 or tier 1 (oncethe user device is elected cloudlet broker) However thecapabilities 119862

0119895and 119862

1119895of thin devices may be small and

so 1198630119895and 119863

1119895are also expected to be small This indicates

that availability remains close to 1 if resource distribution119863119894119895

is proportional to the capability 119862119894119895with the ratio of total

number of requests over total resource amountTherefore theoptimal resource distribution to low tier brokers and mobiledevices both offloads data centersrsquo computing overhead andincreases user satisfaction

32 Latency Social networks can cause locality such thatmany requests do not travel to their destinations over longdistances in terms of either network relay counts or theterrestrial radio barriers Thus initial latency of a serviceis reduced Lower tier brokers have smaller resource gran-ules while locality based on social network provides accessproximity and thus results in shorter latency We estimatethe latency of a request traversing the HiBA architecture asfollows Suppose that a request initiated from a tier-119894 nodearrives at its destination possessing the required resources intier 119897 119894 lt 119897 The expected latency of the request is

119879119894119897= 119875 (119903

119894lt119897)

119897

sum

119896=119894

(119905119873119896

+ 119905119863119896

+ 119905119862119896

) (7)

where 119875(119903119894lt119897) is the probability that the requested resource

is not found in tiers lower than 119897 and 119905119873119896

119905119863119896

119905119862119896

arelatencies caused respectively by network communicationrequest queuing plus database querying and computing forthe brokering at each tier-119896 broker on the path to tier 119897Supposing that request arrival is independent of resourcedistribution we have

119875 (119903119894lt119897) =

120582119894

120582total(1 minus

119863119897minus1total

119863total) (8)

Thus the estimated latency is

119879119894119897=

120582119894

120582total(1 minus

119863119897minus1total

119863total)(

119897

sum

119896=119894

119905119873119896

+ 119905119863119896

+ 119905119862119896

) (9)

If a request is originated from a lower tier user device thatis 119894 = 0 or 1 the effective way to reduce expected latency1198790119897or 1198791119897is to reduce 119875(119903

119894lt119897) This means increasing119863

119897minus1totalby distributing resources to lower tiers that is increasing119863

119896

for 119896 lt 119897 However this also increases both database search-ing time 119905

119863119896and computation time 119905

119862119896 Considering network

delay 119905119873119896

we can expect that in the virtualized network thephysical distance of a cloud server is increasing as 119896 is gettinglarger A high tier link in the virtualized HiBA networkactually contains many more physical relaying hops than a

lower tier link If request is sent using TCP protocol thenetwork latency is increasing muchmore than that caused bydatabase searching and brokerage computing as 119896 increasesFrom (9) we still effectively decrease the latency by increasing119863119897minus1total This can be expected especially for multimedia

content sharing since according to social network relationscontents are stored in proximate cloudlets or cloudlingswhich are lower tiers (small 119897)

33 Development Platform for MDC2 Control and Manage-ment From the availability and latency analysis we see thata HiBA with feedback framework is a development platformIt is easy to observe the performance when developingalgorithms for admission control request differentiation taskpartition adaptive resource allocation and dynamic taskscheduling regarding SLAs assigned by upper tier broker Forexample the admission control in federating a tier affectsresource distribution 119863

119894119895and according to the number of

network hops between a broker and a member the networklatency 119905

119873119896in (9) also differs The request differentiation

and queuing mechanism directly affect 119879119894119897since 119905

119863119896includes

the queue delay that a request stays in the Request QueueTask partitioning resource allocation and scheduling furtheraffect the efficiency processing requests and consequentlythey further affect 119905

119863119896and resultant 119879

119894119897 Globally they also

affect arrival rate 120582119894at tier 119894 Exploiting hierarchical feedback

control it is easy to ensure availability by performing propermanagement while tracking the latency by tuning controlregarding the deadline specified in the SLA of each task

4 Case Studies

To demonstrate algorithm embedding using the MMCNdevelopment platform in both lower and higher tiers westudy energy-aware balancing among mobile devices in thecloudlet tier as well as the load balancing in bigger datacenterswhere large amount of requestswould arrive in a shorttime that is large 120582

119894119895 The energy-aware balancing is critical

in crowd-sensing applications especially when the sensingdata amount is large such as using cameras for image datacollectingThe load balancing is essential for bigger data cen-ters especially when database access frequency is high Bothof the balancing algorithms are effective in the Industry 40era when crowd-sensing and big data analytics are deployed

41 Energy-Aware Balancing The energy-aware balancing isbased on unsupervised fuzzy feedback control where thereference command is also adapted according to the feedbackstate of the federation self-organized by mobile devices Theprincipal idea comes from the energy proportional routing[15] that the lifetime of a clustering-based sensor network isprolonged ifmember nodesrsquo proportions of consumed energyin the remaining are close to the clusterrsquos We exploit theenergy proportion of the cloudlet federation as the adaptivereference to be tracked by the fuzzy feedback control systemTherefore the energy sharing is unsupervised because of nogiven objective of the control a priori

6 Mobile Information Systems

0

1

0

1

120583

120583

N P

e

N998400 P998400

e998400

minus256 0 256

minus512 512

Figure 4 Membership functions for fuzzy sets119873 1198751198731015840 and 1198751015840

Suppose that in a cloudlet tier of119873member nodes datatransmission is observed by the cloudlet broker in each round119905 (discrete time) We first define symbols as follows

(i) 119896 member node identification 1 le 119896 le 119873

(ii) 119863119896(119905) data transmission amount being assigned to

node 119896 at round 119905

(iii) 119864119896(119905) remaining energy of node 119896 at 119905

(iv) 119883119896(119905) = 119863

119896(119905)119864119896(119905) representing the ratio of over-

head to the remaining energy (current capability)This is the indication of how node 119896 is loaded withrespect to the remaining energy

(v) 119879ℎ(119905) = sum119873

119896=1119863119896(119905)sum

119873

119896=1119864119896(119905) representing how the

whole cloudlet federation is loaded

(vi) 119890119896(119905) = 119883

119896(119905) minus 119879ℎ(119905) representing the difference of

loading between node 119896 and the whole federation

(vii) 1198901015840119896(119905) = 119890

119896(119905)minus119890119896(119905minus1) representing how the difference

changes

(viii) 119903119896(119905) the ratio of data amount assigned to node 119896 at 119905

We define fuzzy rules as follows

(R1) If 119890119896(119905) is 119875 and 119890

1015840

119896(119905) is 1198751015840 then 119903

119896(119905 + 1) is 119878

119896(119905)

(R2) If 119890119896(119905) is 119875 and 119890

1015840

119896(119905) is1198731015840 then 119903

119896(119905 + 1) is119872

119896(119905)

(R3) If 119890119896(119905) is119873 and 119890

1015840

119896(119905) is 1198751015840 then 119903

119896(119905 + 1) is119872

119896(119905)

(R4) If 119890119896(119905) is119873 and 119890

1015840

119896(119905) is1198731015840 then 119903

119896(119905 + 1) is 119871

119896(119905)

Themembership functions for fuzzy sets119873 1198751198731015840 and 1198751015840 areconfigured in Figure 4

We realize the ldquoandrdquo operator with 119905-norm ldquominimumrdquoThat is the matching degrees 120583

1 1205832 1205833 and 120583

4of premise

part of rules (R1) to (R4) respectively are

1205831 (119905) = min (120583

119875 (119890 (119905)) 120583119875

1015840 (1198901015840(119905)))

1205832 (119905) = min (120583

119875 (119890 (119905)) 120583119873

1015840 (1198901015840(119905)))

1205833 (119905) = min (120583

119873 (119890 (119905)) 120583119875

1015840 (1198901015840(119905)))

1205834 (119905) = min (120583

119873 (119890 (119905)) 120583119873

1015840 (1198901015840(119905)))

(10)

Obtaining the matching degrees 1205831 1205832 1205833 and 120583

4of the

four fuzzy rules respectively we perform the defuzzificationequivalently applying the Takagi-Sugeno inference methodThe inference result adequate ratio of data amount totransmit is

119903119896 (119905 + 1)

=

1205831 (119905) 119878 (119905) + 120583

2 (119905)119872 (119905) + 120583

3 (119905)119872 (119905) + 120583

4 (119905) 119871 (119905)

1205831 (119905) + 120583

2 (119905) + 120583

3 (119905) + 120583

4 (119905)

(11)

by configuring the conclusion part membership functions asdynamic singletons as follows

119871 (119905) = 119903119896 (119905 minus 1) + Δ

119872(119905) = 119903119896 (119905 minus 1)

119878 (119905) = 119903119896 (119905 minus 1) minus Δ

(12)

where Δ is the ratio tuning amount for each round Finallythe data amount assigned to node 119896 is determined as

119863119896 (119905 + 1) =

119903119896 (119905)

sum119873

119894=1119903119894 (119905)

D (119905) (13)

where D(119905) is the total data amount to be transmitted fromthis cloudlet at time 119905 In the above fuzzy inference algorithmeach member node tracks the dynamic control goal 119879ℎ(119905)and the proportion of energy to be consumed in the remain-ing energy approaches the proportion regarding the wholefederation Therefore the intrafederation energy sharing isachieved by offloading transmission jobs using the proposedalgorithm When each node 119896 is a lower tier federation theoffloading is hierarchically extended through the MMCNnetworking where the data amount 119863

119896(119905) is determined and

assigned by CCS and the energy status updates 119864119896(119905)rsquos are

received and aggregated by CIS This shows the scalability ofthe HiBA architecture

42 Load Balancing The load balancing is required in abigger federation organized in a higher tier of HiBA becausea higher federation always has larger amount of requestarrivals As shown in Figure 5 the framework of the proposedload-balanced cloud service interface consists of three com-ponents which realizes CES CCS and CIS respectively Thedetails of components are presented as follows

(1) Cloud Service Interface Node (CSIN) it is a virtualmachine that provides cloud service interface and is

Mobile Information Systems 7

Service module 1

Service module 2

Service

CSIallocation

table

Clouddatabase

HiBA networking

Cloud bank tier

(1)(3)

(4)

(5)

(6)

(7)(8)

(2)CSI

manager CSIN 1(9)

CSIN n

module m

middot middot middotmiddot middot middot

middot middot middot

Figure 5 Framework of load-balanced cloud service interface

denoted as CSINi in Figure 5 A CSIN can receivemembersrsquo requests and obtain results by interactingwith requested cloud service module In our designeach cloud service interface in CSIN is implementedas a RESTful API for universally communicating withdifferent types of mobile devices A CSIN is then therealization of CES in the MMCN framework

(2) CSImanager it is designed tomanage the usage statusof CSINs andmatchesmobile users to CSINsThe effi-ciency of the load balance depends on the matchingmethod of the CSI Manager A CSI manager is thenthe realization of CCS in the MMCN framework

(3) CSI allocation table it maintains the serving statusbetween mobile users and CSINs and is used by theCSI manager for assisting the matching decision forthe new-arrived mobile users This table is stored inthe cloud database A CSI allocation table is then therealization of CIS in the MMCN framework

The processing flow of using a cloud service for a mobile useris designed for achieving the load balance The idea of com-mitting requests requires three phases (1) service registration(Steps (1)-(2)) (2) service execution (Steps (3)ndash(7)) and (3)service deregistration (Steps (8)-(9)) The detailed steps alsoillustrated in Figure 5 are presented as follows

(1) Mobile user MU119894sends a request to CSI manager for

allocating a CSI address(2) CSI manager searches for a CSI machine with least

number of serving users say CSIN119899 in the CSI allo-

cation table and then registers (MU119894 CSIN

119899) into the

CSI allocation table(3) CSI manager informs MU

119894that CSIN

119899can serve his

her cloud service requests in the following period(4) MU

119894configures the cloud service interface by replac-

ing the IP attribute in the service template with the IPaddress of CSIN

119899

(5) MU119894connects to CSIN

119899through the configured URL

of the cloud service interface and sends a request(6) CSIN

119899delivers the request to the associated cloud

service which may access the cloud databases ifrequired After processing the results will be sentback to CSIN

119899

(7) CSIN119899sends the results to MU

119894 (If more requests are

required to processed Steps (3)ndash(7) are repeated)(8) MU

119894deregisters CSIN

119899to CSI manager

(9) CSI manager removes the registration record of MU119894

from the CSI allocation table

Notice that the matching rule for a new mobile user andCSINs in the current design uses least-user-first basis asshown in Step (2) The matching principle can be modifiedaccording to different demands In the experiments wewill show that the current design already obtains splendidperformance More study on customizing matching methodsis one of our future research directions That is althoughfive machines are used in our CSI (more than the single-machine CSI) the improvement is by a factor of one hundredindicating the superior performance of our proposed load-balanced CSI mechanism under the HiBA architecture

5 Real-World Experiments

51 HiBA Baseline Performance We conduct an experimentdemonstrating performance difference in terms of availabil-ity and latency when distributions of virtualized resourceinstances (VRIs) differ We leave development of enhancingalgorithms for specific purposes in the CCS as future workwhich will seek performance in terms of various metricsA total of 50000 VRIs are distributed in a 3-tier HiBAfederation comprising Android smart phones tablets andVMs in desktop PCs Figure 6 is the HiBA topology andrelated specifications of the machines are shown in Table 1Each device is labeled (119894 119895) if it is the 119895th node at tier 119894 Device(1 3) connects to (2 0) through a 3G access point with VPNtunneling Device (0 11) connects to (1 0) with USB andshares Internet connection from (1 0) These machines havedifferent computing ability though request and task queuesbeing of the same size of 20 to preserve the differences ofcomputing ability Requests are randomly generated in eachdevice Mean request arrivals (120582) are 3 15 and 1 whichequivalently mean that request intervals (1120582) are 333ms666ms and 1 s respectively The resource distributions havefour casesThe first case is that the tier-2 device possess all the50000 VRIs while 16000 VRIs are duplicated to each tier-1 federations In the remaining three cases we respectivelyduplicate 28000 32000 and 36000 VRIs to each tier-1federation The experiment is conducted in a heterogeneousnetworking environment connecting machines by differentcommunication technologies shown in Table 1

The results of the experiment are shown in Figures 7 and8 Both Figures 7 and 8 reveal that distributing resources inlower tiers provides better performance Figures 8(a) and 8(b)show the latency differences caused by the resource distri-butions that 16000 32000 and 40000 VRIs are duplicatedin each cloudling tier The results before all brokers fullyloaded are also given with respective zoom-in charts Whendevices all continuously issue requests at a high frequency (3requests per second) the queue delays are obviously high andwe see a few requests that are dropped Requests from tier-1devices have lower loss rate since the resources requested are

8 Mobile Information Systems

Table 1 Machines in the offloading experiments using heterogeneous communication technologies

(Tier device) Specifications Physical network linkTier 2(2 0) Desktop PC Intel Core i5-3470 CPU 32GHz 4G RAM

EthernetTier 1(1 0) (1 1) Desktop PC Intel Core i7-4770 CPU 34GHz 8G RAMTier 1(1 2)Tier 1(1 3) Laptop Intel Core i7-2820QM CPU 23GHz 8G RAM Down WiFi UP 3G VPNTier 0(0 0sim3)

Desktop PC Intel Pentium D CPU 34GHz 1 G RAM EthernetTier 0(0 4sim6)Tier 0(0 7)Tier 0(0 8) Tablet ASUS Fonepad Intel Atom Z2420 12 GHz

WiFiTier 0(0 9) Cell Phone S3 SAMUNG Exynos 4412 14 GHzTier 0(0 10) Tablet ASUS Nexus7 NVIDIA Tegra 3 4-coreTier 0(0 11) Tablet WS-170 Qualcomm QCTMSM8x60 surf15 GHz duo-core USBlarrrarr PClarrrarr EthernetTier 0(0 12sim13) Virtual Machines (VirtualBox) EthernetTier 0(0 14) Tablet Samsung Galaxy CPU GT-I8260 WiFi

(011)

LAN LAN WiFi 3G

Tier 2

Tier 1

Tier 0

VPN

LAN LAN WiFi WiFi

(0 14)

(0 8)

(0 5)

(0 12)

(0 13) (0 9)

(0 7)(0 4) (0 6)(0 3)

(0 1)(0 10)

(0 2)(0 0)

(1 3)

(2 0)

(1 1)(1 0) (1 2)

Figure 6 Real-world HiBA topology for offloading performancemeasurement Gray nodes are local VMs of respective tier-1 brokersDiverse physical network links are exploited in tiers of federations

obtained in a shorter time For each physical machine theCPU utilization of theMMCNbrokering itself is smaller than2 depending on the number of cores The mean brokeringcomputation and database querying time is estimated tobe about 500ms while the WLAN delay time is smallerthan 5ms In the heterogeneous network case the latencydifference will be larger if upper tiers are physically at a longdistance from the cloudlings and cloudlets If the requestinterval is higher than the processing time the latency tendsto be smaller When more VRIs are duplicated in lower tierdevices we also see that the latencies are smaller

In this paper we leave algorithms such as request par-tition network embedding and federation for specific per-formance metrics as future work Instead we implementreal-world HiBA to prove that with feedback framework it isa new design paradigm and development platform for thesealgorithms

52 Energy-Aware Balancing In the energy-aware balancingexperiment we exploit four mobile devices of different typesfrom different manufacturers The cloudlet is self-organizedaccording to the broker election protocol in [8] The electedbroker is ASUS-Fonepad The other hierarchical federations

are in Figure 6 Each device including the broker itselfin each round updates its energy status to the CIS of thebroker (ASUS-Fonepad) Each mobile device in the cloudletis recharged to 100of respective battery energy capacityThedata and total data amount to be transmitted in each roundare randomly generated with mean of 24GB prior to theexperiment For each round 119905 total data amount is the samefor both balancing and nonbalancing cases In nonbalancingcase the data amounts are evenly assigned to members Inbalancing case the data amounts are determined by the fuzzycontroller in Section 41 The data amount assignment is ofunit 100MB When any device has remaining energy loweror equal to 40 the experiment is terminated The initialsingletons are configured as 119878(0) = 3 119872(0) = 5 and 119871(0) =

7 (Δ = 2) The result of energy-aware balancing is shownin Figure 9 The respective control systemsrsquo behaviors are inFigure 10 The fuzzy controllers of mobile devices all trackthe federationrsquos data amount to energy proportion (119879ℎ) Wesee that no matter how the performance of respective devicevaries a device with better transmission capability will shareits energy with others Tablet PCs are usually with highercapacity batteries Without energy sharing the remainingenergy of a tablet drops more slowly than a mobile phoneWith energy sharing tablets undertake more transmissionjobs and becomewith higher energy drop rate and themobilephones become with lower energy drop rate However wesee that the whole cloudlet federation has longer lifetimeTheexperiment can also be applied to energy consuming tasks inaddition to data transmission

53 Load Balancing The load-balanced cloud service inter-face (CSI) in the form of RESTful APIs is implementedby using Jersey and is deployed on Microsoft WindowsAzure public cloud platform We conduct an experiment forcomparing the proposed CSI to that of single machine Inour experiment five CSI machines are used to share therequests from users In the experiment the data producingrate for each user is 1 request per second The performance

Mobile Information Systems 9

000 000000 000000 000000 000

000100200300

Homo Hetero16000 075 00028000 000 00032000 015 00036000 018 000

()

()

()

000100200300

Homo Hetero16000280003200036000

000100200300

Homo Hetero16000280003200036000

000 000000 012015 000000 000

Tier-1 1120582 = 666Tier-1 1120582 = 333

1600028000

3200036000

1600028000

3200036000

1600028000

3200036000

Tier-11120582 = 999

(a)

110 340050 230082 180080 170

030 330016 160000 130016 030

030 020000 000000 010000 000

1600028000

3200036000

000100200300

Homo Hetero16000280003200036000

()

000100200300

Homo Hetero16000280003200036000

()

000100200300

Homo Hetero16000280003200036000

()

Tier-0 1120582 = 666Tier-0 1120582 = 333

1600028000

3200036000

1600028000

3200036000

Tier-0 1120582 = 999

(b)

Figure 7 Mean losses of tier 0 (b) and tier 1 (a) in cases of different resource distributions when 16000 28000 32000 and 36000 VRIs areduplicated to each tier-1 federation Request intervals 1120582 are in milliseconds

Case3_tier0

0

20000

40000

1 6 11 16 21 26 31 36

050000

100000150000200000250000300000350000

Late

ncy

(ms)

898513 8121 25 7733 37 736949 6157 93 975345412917 655 91

Request

1120582 = 333

1120582 = 666

1120582 = 999

(a) 16000 VRIs in each tier-1 federation

Case3_tier0

0

20000

40000

1 6 11 16 21 26 31 36 41

050000

100000150000200000250000300000350000

Late

ncy

(ms)

898513 8121 25 7733 37 736949 6157 93 975345412917 655 91

Request

1120582 = 333

1120582 = 666

1120582 = 999

(b) 40000 VRIs in each tier-1 federation

Figure 8 Mean latencies in cases of different resource distributions

metric is the missing rate defined as (119879 minus 119878)119879 where 119879

is the number of total requests and 119878 is the number ofsuccessful requests Table 2 shows the experimental resultsunder different amount of users from 100 to 300 We can seethat when the number of users is low (100 users) both CSIdesigns can successfully process their data requests As thenumber of users increases (eg 300 users) our proposed CSIcan almost process it with only 0 27missing rate howeverover 27 requests are failed to deliver data to cloud databasein the CSI of single machine

6 Conclusion

We have proposed and implemented a mobile device-centriccloud computing architecture HiBA based on feedback con-trol framework The proposed hierarchical brokering archi-tecture is self-organized featuring scalability and hierarchicalautonomy and is easy to embed management and controlalgorithms to develop a federation with better performancein terms of availability and latency Mobile devices and smallservers federate into cloudlets and cloudlings of hierarchical

10 Mobile Information Systems

Table 2 Experimental comparison of different CSI designs

Users Single Load-balanced schemeTotal requests Successful requests Missing requests Total requests Successful requests Missing requests

100 52395 52395 0 53370 53370 0200 97457 80947 1691 104244 104244 0300 113624 81883 2794 131742 131388 027

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 251

No control-ASUS-FonepadNo control-ASUS-NEXUSNo control-HTC-M7No control-SAMSUNG-GTI8260With control-ASUS-FonepadWith control-ASUS-NEXUSWith control-HTC-M7With control-SAMSUNG-GTI8260

0102030405060708090

100

()

Figure 9 Energy-aware balancing result 119909-axis round 119910-axisremaining energy

02468

101214161820

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 241

ASUS-FonepadASUS-NEXUSHTC-M7

SAMSUNG-GTI8260Th

Figure 10 Data amount to energy proportions of individual devices(119883119894) and the cloudlet federation (119879ℎ) 119909-axis round 119910-axis dataamount to energy proportions

tiers such that mobile devices not only request servicesbut also provide resources required by diverse servicesThe implemented HiBA architecture with feedback controlframework proves improved performancewhen resources areadequately distributed to lower tier federations rather thanbeing centralized in a remote cloud server Through two casestudies of cloudlet energy-aware balancing and bigger data

center load balancing we show that the HiBA is actually adevelopment platform for mobile cloud computing and pro-vides a feasible solution to UCN services Future worksinclude optimization algorithms embedding and big dataanalytics applicationswith crowd sensing are to be continued

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

Thanks are due to the Ministry of Science and Tech-nology (MOST which was the National Science Coun-cil) in Taiwan for sponsoring this research under con-tinuous funded Projects NSC101-2221-E-327-020- NSC101-2221-E-327-022- NSC102-2218-E-327-002- MOST103-2221-E-327-048- MOST105-2221-E-327-027- andMOST105-2221-E-006-141-

References

[1] Technicolor ldquoUser-Centric Networkingrdquo httpusercentricnet-workingeuabout-ucn

[2] G Iosifidis L Gao J Huang and L Tassiulas ldquoIncentive mech-anisms for user-provided networksrdquo IEEE CommunicationsMagazine vol 52 no 9 pp 20ndash27 2014

[3] B A A Nunes M A S SAntos B T De OliveirA C B MArgiK ObrAczkA and T Turletti ldquoSoftware-defined-networking-enabled capacity sharing in user-centric networksrdquo IEEE Com-munications Magazine vol 52 no 9 pp 28ndash36 2014

[4] G Aloi M D Felice V Loscrı P Pace and G Ruggeri ldquoSpon-taneous smartphone networks as a user-centric solution for thefuture internetrdquo IEEECommunicationsMagazine vol 52 no 12pp 26ndash33 2014

[5] F Hao M Jiao G Min and L T Yang ldquoA trajectory-basedrecruitment strategy of social sensors for participatory sensingrdquoIEEE Communications Magazine vol 52 no 12 pp 41ndash47 2014

[6] M-L LeeH-YChung andF-MYu ldquoModeling of hierarchicalfuzzy systemsrdquo Fuzzy Sets and Systems vol 138 no 2 pp 343ndash361 2003

[7] T C Lin M-Y Pai C-L Chen and C-C Chen ldquoLoad-balanced cloud service interface for the HiBA mobile cloudenvironmentrdquo in Proceedings of the IEEE International Confer-ence onConsumer ElectronicsmdashTaiwan (ICCE-TW rsquo15) pp 360ndash361 Taipei Taiwan June 2015

[8] C-L Chen S-C Chen C Chang and C Lin ldquoScalable andautonomous mobile device-centric cloud for secured D2Dsharingrdquo in Information Security Applications vol 8909 ofLecture Notes in Computer Science pp 177ndash189 2015

Mobile Information Systems 11

[9] W-T Su W Liu C-L Chen and T-P Chen ldquoCloud accesscontrol in multi-layer cloud networksrdquo in Proceedings ofthe IEEE International Conference on Consumer ElectronicsmdashTaiwan (ICCE-TW rsquo15) pp 364ndash365 IEEE Taipei Taiwan June2015

[10] H Huang H Yang and E H Lu ldquoA fuzzy-rough set basedontology for hybrid recommendationrdquo in Proceedings of theIEEE International Conference on Consumer ElectronicsmdashTaiwan (ICCE-TW rsquo12) pp 358ndash359 Taipei Taiwan June 2015

[11] C-L Chen and C-T Chen ldquoHierarchical Brokering with feed-back state observation in Mobile Device-Centric Cloudsrdquo inProceedings of the 7th International Conference on Ubiquitousand Future Networks (ICUFN rsquo15) pp 410ndash415 July 2015

[12] C J S Decusatis A Carranza and C M Decusatis ldquoCom-munication within clouds open standards and proprietaryprotocols for data center networkingrdquo IEEE CommunicationsMagazine vol 50 no 9 pp 26ndash33 2012

[13] M Satyanarayanan P Bahl R Caceres andNDavies ldquoThe casefor VM-based cloudlets in mobile computingrdquo IEEE PervasiveComputing vol 8 no 4 pp 14ndash23 2009

[14] S Mahadev ldquoMobile computing the next decaderdquo in Proceed-ings of the the 8th International Conference on Mobile SystemsApplications and Services (MobiSysrsquo10) pp 1ndash6 San FranciscoCalif USA June 2010

[15] C-L Chen J-W Lee W-T Su M-F Horng and Y-HKuo ldquoNoise-referred energy-proportional routing with packetlength adaptation for clustered sensor networksrdquo InternationalJournal of Ad Hoc and Ubiquitous Computing vol 3 no 4 pp224ndash235 2008

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

4 Mobile Information Systems

120582n0

tn

tn

tn

Dn0

B120582nminus1120582nminus1120582nminus1

tnminus1 tnminus1 tnminus1 tnminus1 tnminus1 tnminus1

Cnminus10

Dnminus10

Cnminus12

Dnminus12

Cnminus1B

Dnminus1B

middot middot middot

middot middot middotmiddot middot middotmiddot middot middot

t1 t1 t1 t1 t1 t1

12058200 1205820B 1205820B+1 1205820j 1205820k 1205820m

C00 C0B C0B+1 C0j C0k C0m

D00 D0BD0B+1 D0j D0k D0m

Cn0

Figure 3 Performance model of the HiBA architecture

performance model of the virtualized network is shown inFigure 3 Suppose that themaximumnumber ofmembers of aHiBA broker is 119861 a broker 119895 in tier 119894 has computing capability119862119894119895 which is also the minimum length of the Task Queue

The capability 119862119894119895also represents the maximum number of

tasks that can be processed by a broker in tier 119894 under thecondition that no task is droppedThe resources are assumedto be uniformly distributed in each tier In tier 119894 the averageamount of contents and resources is 119863

119894119895 A tier-119894 broker has

its amount of local requests in a Poisson distribution withmean 120582

119894119895 This simulates the reinitiated requests caused by

request partitions and task handoffs 119905119899denotes the mean

initial latency caused by network delay to forward requestsand the waiting time that requests stay in the Request Queueuntil being partitioned into smaller tasks In this paper wederive baseline performances on availability and latency

31 Resource Availability We define availability 119886119894119895 as the

ratio of computing capability119862119894119895over the number of accepted

requests 119877119894119895to a broker 119895 in tier 119894 That is

119886119894119895= min(

119862119894119895

119877119894119895

1) (1)

In a HiBA architecture of 119899 tiers the expected total resourceamount summing from the records in the CIS databases isapproximated as

119863total = 119863119899+

119899

sum

119894=1

(

119894minus1

prod

119896=0

119861119899minus119896

)119863119899minus119894

=

119899

sum

119894=0

119861119894119863119899minus119894

(2)

supposing that119863119894= 119864[119863

119894119895]Thus the expected total resource

amount of the subtree rooted at a tier-119894 node is

119863119894total =

119894

sum

119896=0

119861119896119863119899minus119896

(3)

For any request the probability that the requested resource 119903119894119895

is available in local node 119894119895 is approximated by

119875 (119903119894119895) = 119875 119903

119894119895isin D119894119895 =

119863119894119895

119863total (4)

where D119894119895is the set of resources in node 119894119895 Suppose that

the sample space is the union of D119894119895in the whole HiBA

architecture Then the worst case of availability occurs whennetwork latency is much smaller than the mean time interval1120582119894119895between two consequent requestsThat is requests from

other cloud federations including those in lower and highertiers arrive in current tier instantly with negligible networkdelayThus the maximal number of requests arriving node 119894119895is

119877119894119895= 119875 (119903

119894119895)

119899

sum

119896=0

119861119896120582119899minus119896119895

(5)

Then we obtain the worst case availability as

119886119894119895= min(

119862119894119895

119875 (119903119894119895)sum119899

119896=0119861119896120582119899minus119896119895

1)

= min(119862119894119895

119863119894119895

sdot

119863totalsum119899

119896=0119861119896120582119899minus119896119895

1)

(6)

Mobile Information Systems 5

From (6) we see that when 119899 is smaller resources (includingcontents) are more centralized with larger 119863

119894119895and if 119863total

remains large this causes lower availability Thus there arevarious means to increase availability One is to increasecapability 119862

119894119895 though this will increase costs Alternatively

branch amount 119861 of each tier can be increased though thisalso will increase computing capability and costs Third byadopting HiBA we put user devices in tier 0 or tier 1 (oncethe user device is elected cloudlet broker) However thecapabilities 119862

0119895and 119862

1119895of thin devices may be small and

so 1198630119895and 119863

1119895are also expected to be small This indicates

that availability remains close to 1 if resource distribution119863119894119895

is proportional to the capability 119862119894119895with the ratio of total

number of requests over total resource amountTherefore theoptimal resource distribution to low tier brokers and mobiledevices both offloads data centersrsquo computing overhead andincreases user satisfaction

32 Latency Social networks can cause locality such thatmany requests do not travel to their destinations over longdistances in terms of either network relay counts or theterrestrial radio barriers Thus initial latency of a serviceis reduced Lower tier brokers have smaller resource gran-ules while locality based on social network provides accessproximity and thus results in shorter latency We estimatethe latency of a request traversing the HiBA architecture asfollows Suppose that a request initiated from a tier-119894 nodearrives at its destination possessing the required resources intier 119897 119894 lt 119897 The expected latency of the request is

119879119894119897= 119875 (119903

119894lt119897)

119897

sum

119896=119894

(119905119873119896

+ 119905119863119896

+ 119905119862119896

) (7)

where 119875(119903119894lt119897) is the probability that the requested resource

is not found in tiers lower than 119897 and 119905119873119896

119905119863119896

119905119862119896

arelatencies caused respectively by network communicationrequest queuing plus database querying and computing forthe brokering at each tier-119896 broker on the path to tier 119897Supposing that request arrival is independent of resourcedistribution we have

119875 (119903119894lt119897) =

120582119894

120582total(1 minus

119863119897minus1total

119863total) (8)

Thus the estimated latency is

119879119894119897=

120582119894

120582total(1 minus

119863119897minus1total

119863total)(

119897

sum

119896=119894

119905119873119896

+ 119905119863119896

+ 119905119862119896

) (9)

If a request is originated from a lower tier user device thatis 119894 = 0 or 1 the effective way to reduce expected latency1198790119897or 1198791119897is to reduce 119875(119903

119894lt119897) This means increasing119863

119897minus1totalby distributing resources to lower tiers that is increasing119863

119896

for 119896 lt 119897 However this also increases both database search-ing time 119905

119863119896and computation time 119905

119862119896 Considering network

delay 119905119873119896

we can expect that in the virtualized network thephysical distance of a cloud server is increasing as 119896 is gettinglarger A high tier link in the virtualized HiBA networkactually contains many more physical relaying hops than a

lower tier link If request is sent using TCP protocol thenetwork latency is increasing muchmore than that caused bydatabase searching and brokerage computing as 119896 increasesFrom (9) we still effectively decrease the latency by increasing119863119897minus1total This can be expected especially for multimedia

content sharing since according to social network relationscontents are stored in proximate cloudlets or cloudlingswhich are lower tiers (small 119897)

33 Development Platform for MDC2 Control and Manage-ment From the availability and latency analysis we see thata HiBA with feedback framework is a development platformIt is easy to observe the performance when developingalgorithms for admission control request differentiation taskpartition adaptive resource allocation and dynamic taskscheduling regarding SLAs assigned by upper tier broker Forexample the admission control in federating a tier affectsresource distribution 119863

119894119895and according to the number of

network hops between a broker and a member the networklatency 119905

119873119896in (9) also differs The request differentiation

and queuing mechanism directly affect 119879119894119897since 119905

119863119896includes

the queue delay that a request stays in the Request QueueTask partitioning resource allocation and scheduling furtheraffect the efficiency processing requests and consequentlythey further affect 119905

119863119896and resultant 119879

119894119897 Globally they also

affect arrival rate 120582119894at tier 119894 Exploiting hierarchical feedback

control it is easy to ensure availability by performing propermanagement while tracking the latency by tuning controlregarding the deadline specified in the SLA of each task

4 Case Studies

To demonstrate algorithm embedding using the MMCNdevelopment platform in both lower and higher tiers westudy energy-aware balancing among mobile devices in thecloudlet tier as well as the load balancing in bigger datacenterswhere large amount of requestswould arrive in a shorttime that is large 120582

119894119895 The energy-aware balancing is critical

in crowd-sensing applications especially when the sensingdata amount is large such as using cameras for image datacollectingThe load balancing is essential for bigger data cen-ters especially when database access frequency is high Bothof the balancing algorithms are effective in the Industry 40era when crowd-sensing and big data analytics are deployed

41 Energy-Aware Balancing The energy-aware balancing isbased on unsupervised fuzzy feedback control where thereference command is also adapted according to the feedbackstate of the federation self-organized by mobile devices Theprincipal idea comes from the energy proportional routing[15] that the lifetime of a clustering-based sensor network isprolonged ifmember nodesrsquo proportions of consumed energyin the remaining are close to the clusterrsquos We exploit theenergy proportion of the cloudlet federation as the adaptivereference to be tracked by the fuzzy feedback control systemTherefore the energy sharing is unsupervised because of nogiven objective of the control a priori

6 Mobile Information Systems

0

1

0

1

120583

120583

N P

e

N998400 P998400

e998400

minus256 0 256

minus512 512

Figure 4 Membership functions for fuzzy sets119873 1198751198731015840 and 1198751015840

Suppose that in a cloudlet tier of119873member nodes datatransmission is observed by the cloudlet broker in each round119905 (discrete time) We first define symbols as follows

(i) 119896 member node identification 1 le 119896 le 119873

(ii) 119863119896(119905) data transmission amount being assigned to

node 119896 at round 119905

(iii) 119864119896(119905) remaining energy of node 119896 at 119905

(iv) 119883119896(119905) = 119863

119896(119905)119864119896(119905) representing the ratio of over-

head to the remaining energy (current capability)This is the indication of how node 119896 is loaded withrespect to the remaining energy

(v) 119879ℎ(119905) = sum119873

119896=1119863119896(119905)sum

119873

119896=1119864119896(119905) representing how the

whole cloudlet federation is loaded

(vi) 119890119896(119905) = 119883

119896(119905) minus 119879ℎ(119905) representing the difference of

loading between node 119896 and the whole federation

(vii) 1198901015840119896(119905) = 119890

119896(119905)minus119890119896(119905minus1) representing how the difference

changes

(viii) 119903119896(119905) the ratio of data amount assigned to node 119896 at 119905

We define fuzzy rules as follows

(R1) If 119890119896(119905) is 119875 and 119890

1015840

119896(119905) is 1198751015840 then 119903

119896(119905 + 1) is 119878

119896(119905)

(R2) If 119890119896(119905) is 119875 and 119890

1015840

119896(119905) is1198731015840 then 119903

119896(119905 + 1) is119872

119896(119905)

(R3) If 119890119896(119905) is119873 and 119890

1015840

119896(119905) is 1198751015840 then 119903

119896(119905 + 1) is119872

119896(119905)

(R4) If 119890119896(119905) is119873 and 119890

1015840

119896(119905) is1198731015840 then 119903

119896(119905 + 1) is 119871

119896(119905)

Themembership functions for fuzzy sets119873 1198751198731015840 and 1198751015840 areconfigured in Figure 4

We realize the ldquoandrdquo operator with 119905-norm ldquominimumrdquoThat is the matching degrees 120583

1 1205832 1205833 and 120583

4of premise

part of rules (R1) to (R4) respectively are

1205831 (119905) = min (120583

119875 (119890 (119905)) 120583119875

1015840 (1198901015840(119905)))

1205832 (119905) = min (120583

119875 (119890 (119905)) 120583119873

1015840 (1198901015840(119905)))

1205833 (119905) = min (120583

119873 (119890 (119905)) 120583119875

1015840 (1198901015840(119905)))

1205834 (119905) = min (120583

119873 (119890 (119905)) 120583119873

1015840 (1198901015840(119905)))

(10)

Obtaining the matching degrees 1205831 1205832 1205833 and 120583

4of the

four fuzzy rules respectively we perform the defuzzificationequivalently applying the Takagi-Sugeno inference methodThe inference result adequate ratio of data amount totransmit is

119903119896 (119905 + 1)

=

1205831 (119905) 119878 (119905) + 120583

2 (119905)119872 (119905) + 120583

3 (119905)119872 (119905) + 120583

4 (119905) 119871 (119905)

1205831 (119905) + 120583

2 (119905) + 120583

3 (119905) + 120583

4 (119905)

(11)

by configuring the conclusion part membership functions asdynamic singletons as follows

119871 (119905) = 119903119896 (119905 minus 1) + Δ

119872(119905) = 119903119896 (119905 minus 1)

119878 (119905) = 119903119896 (119905 minus 1) minus Δ

(12)

where Δ is the ratio tuning amount for each round Finallythe data amount assigned to node 119896 is determined as

119863119896 (119905 + 1) =

119903119896 (119905)

sum119873

119894=1119903119894 (119905)

D (119905) (13)

where D(119905) is the total data amount to be transmitted fromthis cloudlet at time 119905 In the above fuzzy inference algorithmeach member node tracks the dynamic control goal 119879ℎ(119905)and the proportion of energy to be consumed in the remain-ing energy approaches the proportion regarding the wholefederation Therefore the intrafederation energy sharing isachieved by offloading transmission jobs using the proposedalgorithm When each node 119896 is a lower tier federation theoffloading is hierarchically extended through the MMCNnetworking where the data amount 119863

119896(119905) is determined and

assigned by CCS and the energy status updates 119864119896(119905)rsquos are

received and aggregated by CIS This shows the scalability ofthe HiBA architecture

42 Load Balancing The load balancing is required in abigger federation organized in a higher tier of HiBA becausea higher federation always has larger amount of requestarrivals As shown in Figure 5 the framework of the proposedload-balanced cloud service interface consists of three com-ponents which realizes CES CCS and CIS respectively Thedetails of components are presented as follows

(1) Cloud Service Interface Node (CSIN) it is a virtualmachine that provides cloud service interface and is

Mobile Information Systems 7

Service module 1

Service module 2

Service

CSIallocation

table

Clouddatabase

HiBA networking

Cloud bank tier

(1)(3)

(4)

(5)

(6)

(7)(8)

(2)CSI

manager CSIN 1(9)

CSIN n

module m

middot middot middotmiddot middot middot

middot middot middot

Figure 5 Framework of load-balanced cloud service interface

denoted as CSINi in Figure 5 A CSIN can receivemembersrsquo requests and obtain results by interactingwith requested cloud service module In our designeach cloud service interface in CSIN is implementedas a RESTful API for universally communicating withdifferent types of mobile devices A CSIN is then therealization of CES in the MMCN framework

(2) CSImanager it is designed tomanage the usage statusof CSINs andmatchesmobile users to CSINsThe effi-ciency of the load balance depends on the matchingmethod of the CSI Manager A CSI manager is thenthe realization of CCS in the MMCN framework

(3) CSI allocation table it maintains the serving statusbetween mobile users and CSINs and is used by theCSI manager for assisting the matching decision forthe new-arrived mobile users This table is stored inthe cloud database A CSI allocation table is then therealization of CIS in the MMCN framework

The processing flow of using a cloud service for a mobile useris designed for achieving the load balance The idea of com-mitting requests requires three phases (1) service registration(Steps (1)-(2)) (2) service execution (Steps (3)ndash(7)) and (3)service deregistration (Steps (8)-(9)) The detailed steps alsoillustrated in Figure 5 are presented as follows

(1) Mobile user MU119894sends a request to CSI manager for

allocating a CSI address(2) CSI manager searches for a CSI machine with least

number of serving users say CSIN119899 in the CSI allo-

cation table and then registers (MU119894 CSIN

119899) into the

CSI allocation table(3) CSI manager informs MU

119894that CSIN

119899can serve his

her cloud service requests in the following period(4) MU

119894configures the cloud service interface by replac-

ing the IP attribute in the service template with the IPaddress of CSIN

119899

(5) MU119894connects to CSIN

119899through the configured URL

of the cloud service interface and sends a request(6) CSIN

119899delivers the request to the associated cloud

service which may access the cloud databases ifrequired After processing the results will be sentback to CSIN

119899

(7) CSIN119899sends the results to MU

119894 (If more requests are

required to processed Steps (3)ndash(7) are repeated)(8) MU

119894deregisters CSIN

119899to CSI manager

(9) CSI manager removes the registration record of MU119894

from the CSI allocation table

Notice that the matching rule for a new mobile user andCSINs in the current design uses least-user-first basis asshown in Step (2) The matching principle can be modifiedaccording to different demands In the experiments wewill show that the current design already obtains splendidperformance More study on customizing matching methodsis one of our future research directions That is althoughfive machines are used in our CSI (more than the single-machine CSI) the improvement is by a factor of one hundredindicating the superior performance of our proposed load-balanced CSI mechanism under the HiBA architecture

5 Real-World Experiments

51 HiBA Baseline Performance We conduct an experimentdemonstrating performance difference in terms of availabil-ity and latency when distributions of virtualized resourceinstances (VRIs) differ We leave development of enhancingalgorithms for specific purposes in the CCS as future workwhich will seek performance in terms of various metricsA total of 50000 VRIs are distributed in a 3-tier HiBAfederation comprising Android smart phones tablets andVMs in desktop PCs Figure 6 is the HiBA topology andrelated specifications of the machines are shown in Table 1Each device is labeled (119894 119895) if it is the 119895th node at tier 119894 Device(1 3) connects to (2 0) through a 3G access point with VPNtunneling Device (0 11) connects to (1 0) with USB andshares Internet connection from (1 0) These machines havedifferent computing ability though request and task queuesbeing of the same size of 20 to preserve the differences ofcomputing ability Requests are randomly generated in eachdevice Mean request arrivals (120582) are 3 15 and 1 whichequivalently mean that request intervals (1120582) are 333ms666ms and 1 s respectively The resource distributions havefour casesThe first case is that the tier-2 device possess all the50000 VRIs while 16000 VRIs are duplicated to each tier-1 federations In the remaining three cases we respectivelyduplicate 28000 32000 and 36000 VRIs to each tier-1federation The experiment is conducted in a heterogeneousnetworking environment connecting machines by differentcommunication technologies shown in Table 1

The results of the experiment are shown in Figures 7 and8 Both Figures 7 and 8 reveal that distributing resources inlower tiers provides better performance Figures 8(a) and 8(b)show the latency differences caused by the resource distri-butions that 16000 32000 and 40000 VRIs are duplicatedin each cloudling tier The results before all brokers fullyloaded are also given with respective zoom-in charts Whendevices all continuously issue requests at a high frequency (3requests per second) the queue delays are obviously high andwe see a few requests that are dropped Requests from tier-1devices have lower loss rate since the resources requested are

8 Mobile Information Systems

Table 1 Machines in the offloading experiments using heterogeneous communication technologies

(Tier device) Specifications Physical network linkTier 2(2 0) Desktop PC Intel Core i5-3470 CPU 32GHz 4G RAM

EthernetTier 1(1 0) (1 1) Desktop PC Intel Core i7-4770 CPU 34GHz 8G RAMTier 1(1 2)Tier 1(1 3) Laptop Intel Core i7-2820QM CPU 23GHz 8G RAM Down WiFi UP 3G VPNTier 0(0 0sim3)

Desktop PC Intel Pentium D CPU 34GHz 1 G RAM EthernetTier 0(0 4sim6)Tier 0(0 7)Tier 0(0 8) Tablet ASUS Fonepad Intel Atom Z2420 12 GHz

WiFiTier 0(0 9) Cell Phone S3 SAMUNG Exynos 4412 14 GHzTier 0(0 10) Tablet ASUS Nexus7 NVIDIA Tegra 3 4-coreTier 0(0 11) Tablet WS-170 Qualcomm QCTMSM8x60 surf15 GHz duo-core USBlarrrarr PClarrrarr EthernetTier 0(0 12sim13) Virtual Machines (VirtualBox) EthernetTier 0(0 14) Tablet Samsung Galaxy CPU GT-I8260 WiFi

(011)

LAN LAN WiFi 3G

Tier 2

Tier 1

Tier 0

VPN

LAN LAN WiFi WiFi

(0 14)

(0 8)

(0 5)

(0 12)

(0 13) (0 9)

(0 7)(0 4) (0 6)(0 3)

(0 1)(0 10)

(0 2)(0 0)

(1 3)

(2 0)

(1 1)(1 0) (1 2)

Figure 6 Real-world HiBA topology for offloading performancemeasurement Gray nodes are local VMs of respective tier-1 brokersDiverse physical network links are exploited in tiers of federations

obtained in a shorter time For each physical machine theCPU utilization of theMMCNbrokering itself is smaller than2 depending on the number of cores The mean brokeringcomputation and database querying time is estimated tobe about 500ms while the WLAN delay time is smallerthan 5ms In the heterogeneous network case the latencydifference will be larger if upper tiers are physically at a longdistance from the cloudlings and cloudlets If the requestinterval is higher than the processing time the latency tendsto be smaller When more VRIs are duplicated in lower tierdevices we also see that the latencies are smaller

In this paper we leave algorithms such as request par-tition network embedding and federation for specific per-formance metrics as future work Instead we implementreal-world HiBA to prove that with feedback framework it isa new design paradigm and development platform for thesealgorithms

52 Energy-Aware Balancing In the energy-aware balancingexperiment we exploit four mobile devices of different typesfrom different manufacturers The cloudlet is self-organizedaccording to the broker election protocol in [8] The electedbroker is ASUS-Fonepad The other hierarchical federations

are in Figure 6 Each device including the broker itselfin each round updates its energy status to the CIS of thebroker (ASUS-Fonepad) Each mobile device in the cloudletis recharged to 100of respective battery energy capacityThedata and total data amount to be transmitted in each roundare randomly generated with mean of 24GB prior to theexperiment For each round 119905 total data amount is the samefor both balancing and nonbalancing cases In nonbalancingcase the data amounts are evenly assigned to members Inbalancing case the data amounts are determined by the fuzzycontroller in Section 41 The data amount assignment is ofunit 100MB When any device has remaining energy loweror equal to 40 the experiment is terminated The initialsingletons are configured as 119878(0) = 3 119872(0) = 5 and 119871(0) =

7 (Δ = 2) The result of energy-aware balancing is shownin Figure 9 The respective control systemsrsquo behaviors are inFigure 10 The fuzzy controllers of mobile devices all trackthe federationrsquos data amount to energy proportion (119879ℎ) Wesee that no matter how the performance of respective devicevaries a device with better transmission capability will shareits energy with others Tablet PCs are usually with highercapacity batteries Without energy sharing the remainingenergy of a tablet drops more slowly than a mobile phoneWith energy sharing tablets undertake more transmissionjobs and becomewith higher energy drop rate and themobilephones become with lower energy drop rate However wesee that the whole cloudlet federation has longer lifetimeTheexperiment can also be applied to energy consuming tasks inaddition to data transmission

53 Load Balancing The load-balanced cloud service inter-face (CSI) in the form of RESTful APIs is implementedby using Jersey and is deployed on Microsoft WindowsAzure public cloud platform We conduct an experiment forcomparing the proposed CSI to that of single machine Inour experiment five CSI machines are used to share therequests from users In the experiment the data producingrate for each user is 1 request per second The performance

Mobile Information Systems 9

000 000000 000000 000000 000

000100200300

Homo Hetero16000 075 00028000 000 00032000 015 00036000 018 000

()

()

()

000100200300

Homo Hetero16000280003200036000

000100200300

Homo Hetero16000280003200036000

000 000000 012015 000000 000

Tier-1 1120582 = 666Tier-1 1120582 = 333

1600028000

3200036000

1600028000

3200036000

1600028000

3200036000

Tier-11120582 = 999

(a)

110 340050 230082 180080 170

030 330016 160000 130016 030

030 020000 000000 010000 000

1600028000

3200036000

000100200300

Homo Hetero16000280003200036000

()

000100200300

Homo Hetero16000280003200036000

()

000100200300

Homo Hetero16000280003200036000

()

Tier-0 1120582 = 666Tier-0 1120582 = 333

1600028000

3200036000

1600028000

3200036000

Tier-0 1120582 = 999

(b)

Figure 7 Mean losses of tier 0 (b) and tier 1 (a) in cases of different resource distributions when 16000 28000 32000 and 36000 VRIs areduplicated to each tier-1 federation Request intervals 1120582 are in milliseconds

Case3_tier0

0

20000

40000

1 6 11 16 21 26 31 36

050000

100000150000200000250000300000350000

Late

ncy

(ms)

898513 8121 25 7733 37 736949 6157 93 975345412917 655 91

Request

1120582 = 333

1120582 = 666

1120582 = 999

(a) 16000 VRIs in each tier-1 federation

Case3_tier0

0

20000

40000

1 6 11 16 21 26 31 36 41

050000

100000150000200000250000300000350000

Late

ncy

(ms)

898513 8121 25 7733 37 736949 6157 93 975345412917 655 91

Request

1120582 = 333

1120582 = 666

1120582 = 999

(b) 40000 VRIs in each tier-1 federation

Figure 8 Mean latencies in cases of different resource distributions

metric is the missing rate defined as (119879 minus 119878)119879 where 119879

is the number of total requests and 119878 is the number ofsuccessful requests Table 2 shows the experimental resultsunder different amount of users from 100 to 300 We can seethat when the number of users is low (100 users) both CSIdesigns can successfully process their data requests As thenumber of users increases (eg 300 users) our proposed CSIcan almost process it with only 0 27missing rate howeverover 27 requests are failed to deliver data to cloud databasein the CSI of single machine

6 Conclusion

We have proposed and implemented a mobile device-centriccloud computing architecture HiBA based on feedback con-trol framework The proposed hierarchical brokering archi-tecture is self-organized featuring scalability and hierarchicalautonomy and is easy to embed management and controlalgorithms to develop a federation with better performancein terms of availability and latency Mobile devices and smallservers federate into cloudlets and cloudlings of hierarchical

10 Mobile Information Systems

Table 2 Experimental comparison of different CSI designs

Users Single Load-balanced schemeTotal requests Successful requests Missing requests Total requests Successful requests Missing requests

100 52395 52395 0 53370 53370 0200 97457 80947 1691 104244 104244 0300 113624 81883 2794 131742 131388 027

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 251

No control-ASUS-FonepadNo control-ASUS-NEXUSNo control-HTC-M7No control-SAMSUNG-GTI8260With control-ASUS-FonepadWith control-ASUS-NEXUSWith control-HTC-M7With control-SAMSUNG-GTI8260

0102030405060708090

100

()

Figure 9 Energy-aware balancing result 119909-axis round 119910-axisremaining energy

02468

101214161820

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 241

ASUS-FonepadASUS-NEXUSHTC-M7

SAMSUNG-GTI8260Th

Figure 10 Data amount to energy proportions of individual devices(119883119894) and the cloudlet federation (119879ℎ) 119909-axis round 119910-axis dataamount to energy proportions

tiers such that mobile devices not only request servicesbut also provide resources required by diverse servicesThe implemented HiBA architecture with feedback controlframework proves improved performancewhen resources areadequately distributed to lower tier federations rather thanbeing centralized in a remote cloud server Through two casestudies of cloudlet energy-aware balancing and bigger data

center load balancing we show that the HiBA is actually adevelopment platform for mobile cloud computing and pro-vides a feasible solution to UCN services Future worksinclude optimization algorithms embedding and big dataanalytics applicationswith crowd sensing are to be continued

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

Thanks are due to the Ministry of Science and Tech-nology (MOST which was the National Science Coun-cil) in Taiwan for sponsoring this research under con-tinuous funded Projects NSC101-2221-E-327-020- NSC101-2221-E-327-022- NSC102-2218-E-327-002- MOST103-2221-E-327-048- MOST105-2221-E-327-027- andMOST105-2221-E-006-141-

References

[1] Technicolor ldquoUser-Centric Networkingrdquo httpusercentricnet-workingeuabout-ucn

[2] G Iosifidis L Gao J Huang and L Tassiulas ldquoIncentive mech-anisms for user-provided networksrdquo IEEE CommunicationsMagazine vol 52 no 9 pp 20ndash27 2014

[3] B A A Nunes M A S SAntos B T De OliveirA C B MArgiK ObrAczkA and T Turletti ldquoSoftware-defined-networking-enabled capacity sharing in user-centric networksrdquo IEEE Com-munications Magazine vol 52 no 9 pp 28ndash36 2014

[4] G Aloi M D Felice V Loscrı P Pace and G Ruggeri ldquoSpon-taneous smartphone networks as a user-centric solution for thefuture internetrdquo IEEECommunicationsMagazine vol 52 no 12pp 26ndash33 2014

[5] F Hao M Jiao G Min and L T Yang ldquoA trajectory-basedrecruitment strategy of social sensors for participatory sensingrdquoIEEE Communications Magazine vol 52 no 12 pp 41ndash47 2014

[6] M-L LeeH-YChung andF-MYu ldquoModeling of hierarchicalfuzzy systemsrdquo Fuzzy Sets and Systems vol 138 no 2 pp 343ndash361 2003

[7] T C Lin M-Y Pai C-L Chen and C-C Chen ldquoLoad-balanced cloud service interface for the HiBA mobile cloudenvironmentrdquo in Proceedings of the IEEE International Confer-ence onConsumer ElectronicsmdashTaiwan (ICCE-TW rsquo15) pp 360ndash361 Taipei Taiwan June 2015

[8] C-L Chen S-C Chen C Chang and C Lin ldquoScalable andautonomous mobile device-centric cloud for secured D2Dsharingrdquo in Information Security Applications vol 8909 ofLecture Notes in Computer Science pp 177ndash189 2015

Mobile Information Systems 11

[9] W-T Su W Liu C-L Chen and T-P Chen ldquoCloud accesscontrol in multi-layer cloud networksrdquo in Proceedings ofthe IEEE International Conference on Consumer ElectronicsmdashTaiwan (ICCE-TW rsquo15) pp 364ndash365 IEEE Taipei Taiwan June2015

[10] H Huang H Yang and E H Lu ldquoA fuzzy-rough set basedontology for hybrid recommendationrdquo in Proceedings of theIEEE International Conference on Consumer ElectronicsmdashTaiwan (ICCE-TW rsquo12) pp 358ndash359 Taipei Taiwan June 2015

[11] C-L Chen and C-T Chen ldquoHierarchical Brokering with feed-back state observation in Mobile Device-Centric Cloudsrdquo inProceedings of the 7th International Conference on Ubiquitousand Future Networks (ICUFN rsquo15) pp 410ndash415 July 2015

[12] C J S Decusatis A Carranza and C M Decusatis ldquoCom-munication within clouds open standards and proprietaryprotocols for data center networkingrdquo IEEE CommunicationsMagazine vol 50 no 9 pp 26ndash33 2012

[13] M Satyanarayanan P Bahl R Caceres andNDavies ldquoThe casefor VM-based cloudlets in mobile computingrdquo IEEE PervasiveComputing vol 8 no 4 pp 14ndash23 2009

[14] S Mahadev ldquoMobile computing the next decaderdquo in Proceed-ings of the the 8th International Conference on Mobile SystemsApplications and Services (MobiSysrsquo10) pp 1ndash6 San FranciscoCalif USA June 2010

[15] C-L Chen J-W Lee W-T Su M-F Horng and Y-HKuo ldquoNoise-referred energy-proportional routing with packetlength adaptation for clustered sensor networksrdquo InternationalJournal of Ad Hoc and Ubiquitous Computing vol 3 no 4 pp224ndash235 2008

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mobile Information Systems 5

From (6) we see that when 119899 is smaller resources (includingcontents) are more centralized with larger 119863

119894119895and if 119863total

remains large this causes lower availability Thus there arevarious means to increase availability One is to increasecapability 119862

119894119895 though this will increase costs Alternatively

branch amount 119861 of each tier can be increased though thisalso will increase computing capability and costs Third byadopting HiBA we put user devices in tier 0 or tier 1 (oncethe user device is elected cloudlet broker) However thecapabilities 119862

0119895and 119862

1119895of thin devices may be small and

so 1198630119895and 119863

1119895are also expected to be small This indicates

that availability remains close to 1 if resource distribution119863119894119895

is proportional to the capability 119862119894119895with the ratio of total

number of requests over total resource amountTherefore theoptimal resource distribution to low tier brokers and mobiledevices both offloads data centersrsquo computing overhead andincreases user satisfaction

32 Latency Social networks can cause locality such thatmany requests do not travel to their destinations over longdistances in terms of either network relay counts or theterrestrial radio barriers Thus initial latency of a serviceis reduced Lower tier brokers have smaller resource gran-ules while locality based on social network provides accessproximity and thus results in shorter latency We estimatethe latency of a request traversing the HiBA architecture asfollows Suppose that a request initiated from a tier-119894 nodearrives at its destination possessing the required resources intier 119897 119894 lt 119897 The expected latency of the request is

119879119894119897= 119875 (119903

119894lt119897)

119897

sum

119896=119894

(119905119873119896

+ 119905119863119896

+ 119905119862119896

) (7)

where 119875(119903119894lt119897) is the probability that the requested resource

is not found in tiers lower than 119897 and 119905119873119896

119905119863119896

119905119862119896

arelatencies caused respectively by network communicationrequest queuing plus database querying and computing forthe brokering at each tier-119896 broker on the path to tier 119897Supposing that request arrival is independent of resourcedistribution we have

119875 (119903119894lt119897) =

120582119894

120582total(1 minus

119863119897minus1total

119863total) (8)

Thus the estimated latency is

119879119894119897=

120582119894

120582total(1 minus

119863119897minus1total

119863total)(

119897

sum

119896=119894

119905119873119896

+ 119905119863119896

+ 119905119862119896

) (9)

If a request is originated from a lower tier user device thatis 119894 = 0 or 1 the effective way to reduce expected latency1198790119897or 1198791119897is to reduce 119875(119903

119894lt119897) This means increasing119863

119897minus1totalby distributing resources to lower tiers that is increasing119863

119896

for 119896 lt 119897 However this also increases both database search-ing time 119905

119863119896and computation time 119905

119862119896 Considering network

delay 119905119873119896

we can expect that in the virtualized network thephysical distance of a cloud server is increasing as 119896 is gettinglarger A high tier link in the virtualized HiBA networkactually contains many more physical relaying hops than a

lower tier link If request is sent using TCP protocol thenetwork latency is increasing muchmore than that caused bydatabase searching and brokerage computing as 119896 increasesFrom (9) we still effectively decrease the latency by increasing119863119897minus1total This can be expected especially for multimedia

content sharing since according to social network relationscontents are stored in proximate cloudlets or cloudlingswhich are lower tiers (small 119897)

33 Development Platform for MDC2 Control and Manage-ment From the availability and latency analysis we see thata HiBA with feedback framework is a development platformIt is easy to observe the performance when developingalgorithms for admission control request differentiation taskpartition adaptive resource allocation and dynamic taskscheduling regarding SLAs assigned by upper tier broker Forexample the admission control in federating a tier affectsresource distribution 119863

119894119895and according to the number of

network hops between a broker and a member the networklatency 119905

119873119896in (9) also differs The request differentiation

and queuing mechanism directly affect 119879119894119897since 119905

119863119896includes

the queue delay that a request stays in the Request QueueTask partitioning resource allocation and scheduling furtheraffect the efficiency processing requests and consequentlythey further affect 119905

119863119896and resultant 119879

119894119897 Globally they also

affect arrival rate 120582119894at tier 119894 Exploiting hierarchical feedback

control it is easy to ensure availability by performing propermanagement while tracking the latency by tuning controlregarding the deadline specified in the SLA of each task

4 Case Studies

To demonstrate algorithm embedding using the MMCNdevelopment platform in both lower and higher tiers westudy energy-aware balancing among mobile devices in thecloudlet tier as well as the load balancing in bigger datacenterswhere large amount of requestswould arrive in a shorttime that is large 120582

119894119895 The energy-aware balancing is critical

in crowd-sensing applications especially when the sensingdata amount is large such as using cameras for image datacollectingThe load balancing is essential for bigger data cen-ters especially when database access frequency is high Bothof the balancing algorithms are effective in the Industry 40era when crowd-sensing and big data analytics are deployed

41 Energy-Aware Balancing The energy-aware balancing isbased on unsupervised fuzzy feedback control where thereference command is also adapted according to the feedbackstate of the federation self-organized by mobile devices Theprincipal idea comes from the energy proportional routing[15] that the lifetime of a clustering-based sensor network isprolonged ifmember nodesrsquo proportions of consumed energyin the remaining are close to the clusterrsquos We exploit theenergy proportion of the cloudlet federation as the adaptivereference to be tracked by the fuzzy feedback control systemTherefore the energy sharing is unsupervised because of nogiven objective of the control a priori

6 Mobile Information Systems

0

1

0

1

120583

120583

N P

e

N998400 P998400

e998400

minus256 0 256

minus512 512

Figure 4 Membership functions for fuzzy sets119873 1198751198731015840 and 1198751015840

Suppose that in a cloudlet tier of119873member nodes datatransmission is observed by the cloudlet broker in each round119905 (discrete time) We first define symbols as follows

(i) 119896 member node identification 1 le 119896 le 119873

(ii) 119863119896(119905) data transmission amount being assigned to

node 119896 at round 119905

(iii) 119864119896(119905) remaining energy of node 119896 at 119905

(iv) 119883119896(119905) = 119863

119896(119905)119864119896(119905) representing the ratio of over-

head to the remaining energy (current capability)This is the indication of how node 119896 is loaded withrespect to the remaining energy

(v) 119879ℎ(119905) = sum119873

119896=1119863119896(119905)sum

119873

119896=1119864119896(119905) representing how the

whole cloudlet federation is loaded

(vi) 119890119896(119905) = 119883

119896(119905) minus 119879ℎ(119905) representing the difference of

loading between node 119896 and the whole federation

(vii) 1198901015840119896(119905) = 119890

119896(119905)minus119890119896(119905minus1) representing how the difference

changes

(viii) 119903119896(119905) the ratio of data amount assigned to node 119896 at 119905

We define fuzzy rules as follows

(R1) If 119890119896(119905) is 119875 and 119890

1015840

119896(119905) is 1198751015840 then 119903

119896(119905 + 1) is 119878

119896(119905)

(R2) If 119890119896(119905) is 119875 and 119890

1015840

119896(119905) is1198731015840 then 119903

119896(119905 + 1) is119872

119896(119905)

(R3) If 119890119896(119905) is119873 and 119890

1015840

119896(119905) is 1198751015840 then 119903

119896(119905 + 1) is119872

119896(119905)

(R4) If 119890119896(119905) is119873 and 119890

1015840

119896(119905) is1198731015840 then 119903

119896(119905 + 1) is 119871

119896(119905)

Themembership functions for fuzzy sets119873 1198751198731015840 and 1198751015840 areconfigured in Figure 4

We realize the ldquoandrdquo operator with 119905-norm ldquominimumrdquoThat is the matching degrees 120583

1 1205832 1205833 and 120583

4of premise

part of rules (R1) to (R4) respectively are

1205831 (119905) = min (120583

119875 (119890 (119905)) 120583119875

1015840 (1198901015840(119905)))

1205832 (119905) = min (120583

119875 (119890 (119905)) 120583119873

1015840 (1198901015840(119905)))

1205833 (119905) = min (120583

119873 (119890 (119905)) 120583119875

1015840 (1198901015840(119905)))

1205834 (119905) = min (120583

119873 (119890 (119905)) 120583119873

1015840 (1198901015840(119905)))

(10)

Obtaining the matching degrees 1205831 1205832 1205833 and 120583

4of the

four fuzzy rules respectively we perform the defuzzificationequivalently applying the Takagi-Sugeno inference methodThe inference result adequate ratio of data amount totransmit is

119903119896 (119905 + 1)

=

1205831 (119905) 119878 (119905) + 120583

2 (119905)119872 (119905) + 120583

3 (119905)119872 (119905) + 120583

4 (119905) 119871 (119905)

1205831 (119905) + 120583

2 (119905) + 120583

3 (119905) + 120583

4 (119905)

(11)

by configuring the conclusion part membership functions asdynamic singletons as follows

119871 (119905) = 119903119896 (119905 minus 1) + Δ

119872(119905) = 119903119896 (119905 minus 1)

119878 (119905) = 119903119896 (119905 minus 1) minus Δ

(12)

where Δ is the ratio tuning amount for each round Finallythe data amount assigned to node 119896 is determined as

119863119896 (119905 + 1) =

119903119896 (119905)

sum119873

119894=1119903119894 (119905)

D (119905) (13)

where D(119905) is the total data amount to be transmitted fromthis cloudlet at time 119905 In the above fuzzy inference algorithmeach member node tracks the dynamic control goal 119879ℎ(119905)and the proportion of energy to be consumed in the remain-ing energy approaches the proportion regarding the wholefederation Therefore the intrafederation energy sharing isachieved by offloading transmission jobs using the proposedalgorithm When each node 119896 is a lower tier federation theoffloading is hierarchically extended through the MMCNnetworking where the data amount 119863

119896(119905) is determined and

assigned by CCS and the energy status updates 119864119896(119905)rsquos are

received and aggregated by CIS This shows the scalability ofthe HiBA architecture

42 Load Balancing The load balancing is required in abigger federation organized in a higher tier of HiBA becausea higher federation always has larger amount of requestarrivals As shown in Figure 5 the framework of the proposedload-balanced cloud service interface consists of three com-ponents which realizes CES CCS and CIS respectively Thedetails of components are presented as follows

(1) Cloud Service Interface Node (CSIN) it is a virtualmachine that provides cloud service interface and is

Mobile Information Systems 7

Service module 1

Service module 2

Service

CSIallocation

table

Clouddatabase

HiBA networking

Cloud bank tier

(1)(3)

(4)

(5)

(6)

(7)(8)

(2)CSI

manager CSIN 1(9)

CSIN n

module m

middot middot middotmiddot middot middot

middot middot middot

Figure 5 Framework of load-balanced cloud service interface

denoted as CSINi in Figure 5 A CSIN can receivemembersrsquo requests and obtain results by interactingwith requested cloud service module In our designeach cloud service interface in CSIN is implementedas a RESTful API for universally communicating withdifferent types of mobile devices A CSIN is then therealization of CES in the MMCN framework

(2) CSImanager it is designed tomanage the usage statusof CSINs andmatchesmobile users to CSINsThe effi-ciency of the load balance depends on the matchingmethod of the CSI Manager A CSI manager is thenthe realization of CCS in the MMCN framework

(3) CSI allocation table it maintains the serving statusbetween mobile users and CSINs and is used by theCSI manager for assisting the matching decision forthe new-arrived mobile users This table is stored inthe cloud database A CSI allocation table is then therealization of CIS in the MMCN framework

The processing flow of using a cloud service for a mobile useris designed for achieving the load balance The idea of com-mitting requests requires three phases (1) service registration(Steps (1)-(2)) (2) service execution (Steps (3)ndash(7)) and (3)service deregistration (Steps (8)-(9)) The detailed steps alsoillustrated in Figure 5 are presented as follows

(1) Mobile user MU119894sends a request to CSI manager for

allocating a CSI address(2) CSI manager searches for a CSI machine with least

number of serving users say CSIN119899 in the CSI allo-

cation table and then registers (MU119894 CSIN

119899) into the

CSI allocation table(3) CSI manager informs MU

119894that CSIN

119899can serve his

her cloud service requests in the following period(4) MU

119894configures the cloud service interface by replac-

ing the IP attribute in the service template with the IPaddress of CSIN

119899

(5) MU119894connects to CSIN

119899through the configured URL

of the cloud service interface and sends a request(6) CSIN

119899delivers the request to the associated cloud

service which may access the cloud databases ifrequired After processing the results will be sentback to CSIN

119899

(7) CSIN119899sends the results to MU

119894 (If more requests are

required to processed Steps (3)ndash(7) are repeated)(8) MU

119894deregisters CSIN

119899to CSI manager

(9) CSI manager removes the registration record of MU119894

from the CSI allocation table

Notice that the matching rule for a new mobile user andCSINs in the current design uses least-user-first basis asshown in Step (2) The matching principle can be modifiedaccording to different demands In the experiments wewill show that the current design already obtains splendidperformance More study on customizing matching methodsis one of our future research directions That is althoughfive machines are used in our CSI (more than the single-machine CSI) the improvement is by a factor of one hundredindicating the superior performance of our proposed load-balanced CSI mechanism under the HiBA architecture

5 Real-World Experiments

51 HiBA Baseline Performance We conduct an experimentdemonstrating performance difference in terms of availabil-ity and latency when distributions of virtualized resourceinstances (VRIs) differ We leave development of enhancingalgorithms for specific purposes in the CCS as future workwhich will seek performance in terms of various metricsA total of 50000 VRIs are distributed in a 3-tier HiBAfederation comprising Android smart phones tablets andVMs in desktop PCs Figure 6 is the HiBA topology andrelated specifications of the machines are shown in Table 1Each device is labeled (119894 119895) if it is the 119895th node at tier 119894 Device(1 3) connects to (2 0) through a 3G access point with VPNtunneling Device (0 11) connects to (1 0) with USB andshares Internet connection from (1 0) These machines havedifferent computing ability though request and task queuesbeing of the same size of 20 to preserve the differences ofcomputing ability Requests are randomly generated in eachdevice Mean request arrivals (120582) are 3 15 and 1 whichequivalently mean that request intervals (1120582) are 333ms666ms and 1 s respectively The resource distributions havefour casesThe first case is that the tier-2 device possess all the50000 VRIs while 16000 VRIs are duplicated to each tier-1 federations In the remaining three cases we respectivelyduplicate 28000 32000 and 36000 VRIs to each tier-1federation The experiment is conducted in a heterogeneousnetworking environment connecting machines by differentcommunication technologies shown in Table 1

The results of the experiment are shown in Figures 7 and8 Both Figures 7 and 8 reveal that distributing resources inlower tiers provides better performance Figures 8(a) and 8(b)show the latency differences caused by the resource distri-butions that 16000 32000 and 40000 VRIs are duplicatedin each cloudling tier The results before all brokers fullyloaded are also given with respective zoom-in charts Whendevices all continuously issue requests at a high frequency (3requests per second) the queue delays are obviously high andwe see a few requests that are dropped Requests from tier-1devices have lower loss rate since the resources requested are

8 Mobile Information Systems

Table 1 Machines in the offloading experiments using heterogeneous communication technologies

(Tier device) Specifications Physical network linkTier 2(2 0) Desktop PC Intel Core i5-3470 CPU 32GHz 4G RAM

EthernetTier 1(1 0) (1 1) Desktop PC Intel Core i7-4770 CPU 34GHz 8G RAMTier 1(1 2)Tier 1(1 3) Laptop Intel Core i7-2820QM CPU 23GHz 8G RAM Down WiFi UP 3G VPNTier 0(0 0sim3)

Desktop PC Intel Pentium D CPU 34GHz 1 G RAM EthernetTier 0(0 4sim6)Tier 0(0 7)Tier 0(0 8) Tablet ASUS Fonepad Intel Atom Z2420 12 GHz

WiFiTier 0(0 9) Cell Phone S3 SAMUNG Exynos 4412 14 GHzTier 0(0 10) Tablet ASUS Nexus7 NVIDIA Tegra 3 4-coreTier 0(0 11) Tablet WS-170 Qualcomm QCTMSM8x60 surf15 GHz duo-core USBlarrrarr PClarrrarr EthernetTier 0(0 12sim13) Virtual Machines (VirtualBox) EthernetTier 0(0 14) Tablet Samsung Galaxy CPU GT-I8260 WiFi

(011)

LAN LAN WiFi 3G

Tier 2

Tier 1

Tier 0

VPN

LAN LAN WiFi WiFi

(0 14)

(0 8)

(0 5)

(0 12)

(0 13) (0 9)

(0 7)(0 4) (0 6)(0 3)

(0 1)(0 10)

(0 2)(0 0)

(1 3)

(2 0)

(1 1)(1 0) (1 2)

Figure 6 Real-world HiBA topology for offloading performancemeasurement Gray nodes are local VMs of respective tier-1 brokersDiverse physical network links are exploited in tiers of federations

obtained in a shorter time For each physical machine theCPU utilization of theMMCNbrokering itself is smaller than2 depending on the number of cores The mean brokeringcomputation and database querying time is estimated tobe about 500ms while the WLAN delay time is smallerthan 5ms In the heterogeneous network case the latencydifference will be larger if upper tiers are physically at a longdistance from the cloudlings and cloudlets If the requestinterval is higher than the processing time the latency tendsto be smaller When more VRIs are duplicated in lower tierdevices we also see that the latencies are smaller

In this paper we leave algorithms such as request par-tition network embedding and federation for specific per-formance metrics as future work Instead we implementreal-world HiBA to prove that with feedback framework it isa new design paradigm and development platform for thesealgorithms

52 Energy-Aware Balancing In the energy-aware balancingexperiment we exploit four mobile devices of different typesfrom different manufacturers The cloudlet is self-organizedaccording to the broker election protocol in [8] The electedbroker is ASUS-Fonepad The other hierarchical federations

are in Figure 6 Each device including the broker itselfin each round updates its energy status to the CIS of thebroker (ASUS-Fonepad) Each mobile device in the cloudletis recharged to 100of respective battery energy capacityThedata and total data amount to be transmitted in each roundare randomly generated with mean of 24GB prior to theexperiment For each round 119905 total data amount is the samefor both balancing and nonbalancing cases In nonbalancingcase the data amounts are evenly assigned to members Inbalancing case the data amounts are determined by the fuzzycontroller in Section 41 The data amount assignment is ofunit 100MB When any device has remaining energy loweror equal to 40 the experiment is terminated The initialsingletons are configured as 119878(0) = 3 119872(0) = 5 and 119871(0) =

7 (Δ = 2) The result of energy-aware balancing is shownin Figure 9 The respective control systemsrsquo behaviors are inFigure 10 The fuzzy controllers of mobile devices all trackthe federationrsquos data amount to energy proportion (119879ℎ) Wesee that no matter how the performance of respective devicevaries a device with better transmission capability will shareits energy with others Tablet PCs are usually with highercapacity batteries Without energy sharing the remainingenergy of a tablet drops more slowly than a mobile phoneWith energy sharing tablets undertake more transmissionjobs and becomewith higher energy drop rate and themobilephones become with lower energy drop rate However wesee that the whole cloudlet federation has longer lifetimeTheexperiment can also be applied to energy consuming tasks inaddition to data transmission

53 Load Balancing The load-balanced cloud service inter-face (CSI) in the form of RESTful APIs is implementedby using Jersey and is deployed on Microsoft WindowsAzure public cloud platform We conduct an experiment forcomparing the proposed CSI to that of single machine Inour experiment five CSI machines are used to share therequests from users In the experiment the data producingrate for each user is 1 request per second The performance

Mobile Information Systems 9

000 000000 000000 000000 000

000100200300

Homo Hetero16000 075 00028000 000 00032000 015 00036000 018 000

()

()

()

000100200300

Homo Hetero16000280003200036000

000100200300

Homo Hetero16000280003200036000

000 000000 012015 000000 000

Tier-1 1120582 = 666Tier-1 1120582 = 333

1600028000

3200036000

1600028000

3200036000

1600028000

3200036000

Tier-11120582 = 999

(a)

110 340050 230082 180080 170

030 330016 160000 130016 030

030 020000 000000 010000 000

1600028000

3200036000

000100200300

Homo Hetero16000280003200036000

()

000100200300

Homo Hetero16000280003200036000

()

000100200300

Homo Hetero16000280003200036000

()

Tier-0 1120582 = 666Tier-0 1120582 = 333

1600028000

3200036000

1600028000

3200036000

Tier-0 1120582 = 999

(b)

Figure 7 Mean losses of tier 0 (b) and tier 1 (a) in cases of different resource distributions when 16000 28000 32000 and 36000 VRIs areduplicated to each tier-1 federation Request intervals 1120582 are in milliseconds

Case3_tier0

0

20000

40000

1 6 11 16 21 26 31 36

050000

100000150000200000250000300000350000

Late

ncy

(ms)

898513 8121 25 7733 37 736949 6157 93 975345412917 655 91

Request

1120582 = 333

1120582 = 666

1120582 = 999

(a) 16000 VRIs in each tier-1 federation

Case3_tier0

0

20000

40000

1 6 11 16 21 26 31 36 41

050000

100000150000200000250000300000350000

Late

ncy

(ms)

898513 8121 25 7733 37 736949 6157 93 975345412917 655 91

Request

1120582 = 333

1120582 = 666

1120582 = 999

(b) 40000 VRIs in each tier-1 federation

Figure 8 Mean latencies in cases of different resource distributions

metric is the missing rate defined as (119879 minus 119878)119879 where 119879

is the number of total requests and 119878 is the number ofsuccessful requests Table 2 shows the experimental resultsunder different amount of users from 100 to 300 We can seethat when the number of users is low (100 users) both CSIdesigns can successfully process their data requests As thenumber of users increases (eg 300 users) our proposed CSIcan almost process it with only 0 27missing rate howeverover 27 requests are failed to deliver data to cloud databasein the CSI of single machine

6 Conclusion

We have proposed and implemented a mobile device-centriccloud computing architecture HiBA based on feedback con-trol framework The proposed hierarchical brokering archi-tecture is self-organized featuring scalability and hierarchicalautonomy and is easy to embed management and controlalgorithms to develop a federation with better performancein terms of availability and latency Mobile devices and smallservers federate into cloudlets and cloudlings of hierarchical

10 Mobile Information Systems

Table 2 Experimental comparison of different CSI designs

Users Single Load-balanced schemeTotal requests Successful requests Missing requests Total requests Successful requests Missing requests

100 52395 52395 0 53370 53370 0200 97457 80947 1691 104244 104244 0300 113624 81883 2794 131742 131388 027

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 251

No control-ASUS-FonepadNo control-ASUS-NEXUSNo control-HTC-M7No control-SAMSUNG-GTI8260With control-ASUS-FonepadWith control-ASUS-NEXUSWith control-HTC-M7With control-SAMSUNG-GTI8260

0102030405060708090

100

()

Figure 9 Energy-aware balancing result 119909-axis round 119910-axisremaining energy

02468

101214161820

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 241

ASUS-FonepadASUS-NEXUSHTC-M7

SAMSUNG-GTI8260Th

Figure 10 Data amount to energy proportions of individual devices(119883119894) and the cloudlet federation (119879ℎ) 119909-axis round 119910-axis dataamount to energy proportions

tiers such that mobile devices not only request servicesbut also provide resources required by diverse servicesThe implemented HiBA architecture with feedback controlframework proves improved performancewhen resources areadequately distributed to lower tier federations rather thanbeing centralized in a remote cloud server Through two casestudies of cloudlet energy-aware balancing and bigger data

center load balancing we show that the HiBA is actually adevelopment platform for mobile cloud computing and pro-vides a feasible solution to UCN services Future worksinclude optimization algorithms embedding and big dataanalytics applicationswith crowd sensing are to be continued

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

Thanks are due to the Ministry of Science and Tech-nology (MOST which was the National Science Coun-cil) in Taiwan for sponsoring this research under con-tinuous funded Projects NSC101-2221-E-327-020- NSC101-2221-E-327-022- NSC102-2218-E-327-002- MOST103-2221-E-327-048- MOST105-2221-E-327-027- andMOST105-2221-E-006-141-

References

[1] Technicolor ldquoUser-Centric Networkingrdquo httpusercentricnet-workingeuabout-ucn

[2] G Iosifidis L Gao J Huang and L Tassiulas ldquoIncentive mech-anisms for user-provided networksrdquo IEEE CommunicationsMagazine vol 52 no 9 pp 20ndash27 2014

[3] B A A Nunes M A S SAntos B T De OliveirA C B MArgiK ObrAczkA and T Turletti ldquoSoftware-defined-networking-enabled capacity sharing in user-centric networksrdquo IEEE Com-munications Magazine vol 52 no 9 pp 28ndash36 2014

[4] G Aloi M D Felice V Loscrı P Pace and G Ruggeri ldquoSpon-taneous smartphone networks as a user-centric solution for thefuture internetrdquo IEEECommunicationsMagazine vol 52 no 12pp 26ndash33 2014

[5] F Hao M Jiao G Min and L T Yang ldquoA trajectory-basedrecruitment strategy of social sensors for participatory sensingrdquoIEEE Communications Magazine vol 52 no 12 pp 41ndash47 2014

[6] M-L LeeH-YChung andF-MYu ldquoModeling of hierarchicalfuzzy systemsrdquo Fuzzy Sets and Systems vol 138 no 2 pp 343ndash361 2003

[7] T C Lin M-Y Pai C-L Chen and C-C Chen ldquoLoad-balanced cloud service interface for the HiBA mobile cloudenvironmentrdquo in Proceedings of the IEEE International Confer-ence onConsumer ElectronicsmdashTaiwan (ICCE-TW rsquo15) pp 360ndash361 Taipei Taiwan June 2015

[8] C-L Chen S-C Chen C Chang and C Lin ldquoScalable andautonomous mobile device-centric cloud for secured D2Dsharingrdquo in Information Security Applications vol 8909 ofLecture Notes in Computer Science pp 177ndash189 2015

Mobile Information Systems 11

[9] W-T Su W Liu C-L Chen and T-P Chen ldquoCloud accesscontrol in multi-layer cloud networksrdquo in Proceedings ofthe IEEE International Conference on Consumer ElectronicsmdashTaiwan (ICCE-TW rsquo15) pp 364ndash365 IEEE Taipei Taiwan June2015

[10] H Huang H Yang and E H Lu ldquoA fuzzy-rough set basedontology for hybrid recommendationrdquo in Proceedings of theIEEE International Conference on Consumer ElectronicsmdashTaiwan (ICCE-TW rsquo12) pp 358ndash359 Taipei Taiwan June 2015

[11] C-L Chen and C-T Chen ldquoHierarchical Brokering with feed-back state observation in Mobile Device-Centric Cloudsrdquo inProceedings of the 7th International Conference on Ubiquitousand Future Networks (ICUFN rsquo15) pp 410ndash415 July 2015

[12] C J S Decusatis A Carranza and C M Decusatis ldquoCom-munication within clouds open standards and proprietaryprotocols for data center networkingrdquo IEEE CommunicationsMagazine vol 50 no 9 pp 26ndash33 2012

[13] M Satyanarayanan P Bahl R Caceres andNDavies ldquoThe casefor VM-based cloudlets in mobile computingrdquo IEEE PervasiveComputing vol 8 no 4 pp 14ndash23 2009

[14] S Mahadev ldquoMobile computing the next decaderdquo in Proceed-ings of the the 8th International Conference on Mobile SystemsApplications and Services (MobiSysrsquo10) pp 1ndash6 San FranciscoCalif USA June 2010

[15] C-L Chen J-W Lee W-T Su M-F Horng and Y-HKuo ldquoNoise-referred energy-proportional routing with packetlength adaptation for clustered sensor networksrdquo InternationalJournal of Ad Hoc and Ubiquitous Computing vol 3 no 4 pp224ndash235 2008

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

6 Mobile Information Systems

0

1

0

1

120583

120583

N P

e

N998400 P998400

e998400

minus256 0 256

minus512 512

Figure 4 Membership functions for fuzzy sets119873 1198751198731015840 and 1198751015840

Suppose that in a cloudlet tier of119873member nodes datatransmission is observed by the cloudlet broker in each round119905 (discrete time) We first define symbols as follows

(i) 119896 member node identification 1 le 119896 le 119873

(ii) 119863119896(119905) data transmission amount being assigned to

node 119896 at round 119905

(iii) 119864119896(119905) remaining energy of node 119896 at 119905

(iv) 119883119896(119905) = 119863

119896(119905)119864119896(119905) representing the ratio of over-

head to the remaining energy (current capability)This is the indication of how node 119896 is loaded withrespect to the remaining energy

(v) 119879ℎ(119905) = sum119873

119896=1119863119896(119905)sum

119873

119896=1119864119896(119905) representing how the

whole cloudlet federation is loaded

(vi) 119890119896(119905) = 119883

119896(119905) minus 119879ℎ(119905) representing the difference of

loading between node 119896 and the whole federation

(vii) 1198901015840119896(119905) = 119890

119896(119905)minus119890119896(119905minus1) representing how the difference

changes

(viii) 119903119896(119905) the ratio of data amount assigned to node 119896 at 119905

We define fuzzy rules as follows

(R1) If 119890119896(119905) is 119875 and 119890

1015840

119896(119905) is 1198751015840 then 119903

119896(119905 + 1) is 119878

119896(119905)

(R2) If 119890119896(119905) is 119875 and 119890

1015840

119896(119905) is1198731015840 then 119903

119896(119905 + 1) is119872

119896(119905)

(R3) If 119890119896(119905) is119873 and 119890

1015840

119896(119905) is 1198751015840 then 119903

119896(119905 + 1) is119872

119896(119905)

(R4) If 119890119896(119905) is119873 and 119890

1015840

119896(119905) is1198731015840 then 119903

119896(119905 + 1) is 119871

119896(119905)

Themembership functions for fuzzy sets119873 1198751198731015840 and 1198751015840 areconfigured in Figure 4

We realize the ldquoandrdquo operator with 119905-norm ldquominimumrdquoThat is the matching degrees 120583

1 1205832 1205833 and 120583

4of premise

part of rules (R1) to (R4) respectively are

1205831 (119905) = min (120583

119875 (119890 (119905)) 120583119875

1015840 (1198901015840(119905)))

1205832 (119905) = min (120583

119875 (119890 (119905)) 120583119873

1015840 (1198901015840(119905)))

1205833 (119905) = min (120583

119873 (119890 (119905)) 120583119875

1015840 (1198901015840(119905)))

1205834 (119905) = min (120583

119873 (119890 (119905)) 120583119873

1015840 (1198901015840(119905)))

(10)

Obtaining the matching degrees 1205831 1205832 1205833 and 120583

4of the

four fuzzy rules respectively we perform the defuzzificationequivalently applying the Takagi-Sugeno inference methodThe inference result adequate ratio of data amount totransmit is

119903119896 (119905 + 1)

=

1205831 (119905) 119878 (119905) + 120583

2 (119905)119872 (119905) + 120583

3 (119905)119872 (119905) + 120583

4 (119905) 119871 (119905)

1205831 (119905) + 120583

2 (119905) + 120583

3 (119905) + 120583

4 (119905)

(11)

by configuring the conclusion part membership functions asdynamic singletons as follows

119871 (119905) = 119903119896 (119905 minus 1) + Δ

119872(119905) = 119903119896 (119905 minus 1)

119878 (119905) = 119903119896 (119905 minus 1) minus Δ

(12)

where Δ is the ratio tuning amount for each round Finallythe data amount assigned to node 119896 is determined as

119863119896 (119905 + 1) =

119903119896 (119905)

sum119873

119894=1119903119894 (119905)

D (119905) (13)

where D(119905) is the total data amount to be transmitted fromthis cloudlet at time 119905 In the above fuzzy inference algorithmeach member node tracks the dynamic control goal 119879ℎ(119905)and the proportion of energy to be consumed in the remain-ing energy approaches the proportion regarding the wholefederation Therefore the intrafederation energy sharing isachieved by offloading transmission jobs using the proposedalgorithm When each node 119896 is a lower tier federation theoffloading is hierarchically extended through the MMCNnetworking where the data amount 119863

119896(119905) is determined and

assigned by CCS and the energy status updates 119864119896(119905)rsquos are

received and aggregated by CIS This shows the scalability ofthe HiBA architecture

42 Load Balancing The load balancing is required in abigger federation organized in a higher tier of HiBA becausea higher federation always has larger amount of requestarrivals As shown in Figure 5 the framework of the proposedload-balanced cloud service interface consists of three com-ponents which realizes CES CCS and CIS respectively Thedetails of components are presented as follows

(1) Cloud Service Interface Node (CSIN) it is a virtualmachine that provides cloud service interface and is

Mobile Information Systems 7

Service module 1

Service module 2

Service

CSIallocation

table

Clouddatabase

HiBA networking

Cloud bank tier

(1)(3)

(4)

(5)

(6)

(7)(8)

(2)CSI

manager CSIN 1(9)

CSIN n

module m

middot middot middotmiddot middot middot

middot middot middot

Figure 5 Framework of load-balanced cloud service interface

denoted as CSINi in Figure 5 A CSIN can receivemembersrsquo requests and obtain results by interactingwith requested cloud service module In our designeach cloud service interface in CSIN is implementedas a RESTful API for universally communicating withdifferent types of mobile devices A CSIN is then therealization of CES in the MMCN framework

(2) CSImanager it is designed tomanage the usage statusof CSINs andmatchesmobile users to CSINsThe effi-ciency of the load balance depends on the matchingmethod of the CSI Manager A CSI manager is thenthe realization of CCS in the MMCN framework

(3) CSI allocation table it maintains the serving statusbetween mobile users and CSINs and is used by theCSI manager for assisting the matching decision forthe new-arrived mobile users This table is stored inthe cloud database A CSI allocation table is then therealization of CIS in the MMCN framework

The processing flow of using a cloud service for a mobile useris designed for achieving the load balance The idea of com-mitting requests requires three phases (1) service registration(Steps (1)-(2)) (2) service execution (Steps (3)ndash(7)) and (3)service deregistration (Steps (8)-(9)) The detailed steps alsoillustrated in Figure 5 are presented as follows

(1) Mobile user MU119894sends a request to CSI manager for

allocating a CSI address(2) CSI manager searches for a CSI machine with least

number of serving users say CSIN119899 in the CSI allo-

cation table and then registers (MU119894 CSIN

119899) into the

CSI allocation table(3) CSI manager informs MU

119894that CSIN

119899can serve his

her cloud service requests in the following period(4) MU

119894configures the cloud service interface by replac-

ing the IP attribute in the service template with the IPaddress of CSIN

119899

(5) MU119894connects to CSIN

119899through the configured URL

of the cloud service interface and sends a request(6) CSIN

119899delivers the request to the associated cloud

service which may access the cloud databases ifrequired After processing the results will be sentback to CSIN

119899

(7) CSIN119899sends the results to MU

119894 (If more requests are

required to processed Steps (3)ndash(7) are repeated)(8) MU

119894deregisters CSIN

119899to CSI manager

(9) CSI manager removes the registration record of MU119894

from the CSI allocation table

Notice that the matching rule for a new mobile user andCSINs in the current design uses least-user-first basis asshown in Step (2) The matching principle can be modifiedaccording to different demands In the experiments wewill show that the current design already obtains splendidperformance More study on customizing matching methodsis one of our future research directions That is althoughfive machines are used in our CSI (more than the single-machine CSI) the improvement is by a factor of one hundredindicating the superior performance of our proposed load-balanced CSI mechanism under the HiBA architecture

5 Real-World Experiments

51 HiBA Baseline Performance We conduct an experimentdemonstrating performance difference in terms of availabil-ity and latency when distributions of virtualized resourceinstances (VRIs) differ We leave development of enhancingalgorithms for specific purposes in the CCS as future workwhich will seek performance in terms of various metricsA total of 50000 VRIs are distributed in a 3-tier HiBAfederation comprising Android smart phones tablets andVMs in desktop PCs Figure 6 is the HiBA topology andrelated specifications of the machines are shown in Table 1Each device is labeled (119894 119895) if it is the 119895th node at tier 119894 Device(1 3) connects to (2 0) through a 3G access point with VPNtunneling Device (0 11) connects to (1 0) with USB andshares Internet connection from (1 0) These machines havedifferent computing ability though request and task queuesbeing of the same size of 20 to preserve the differences ofcomputing ability Requests are randomly generated in eachdevice Mean request arrivals (120582) are 3 15 and 1 whichequivalently mean that request intervals (1120582) are 333ms666ms and 1 s respectively The resource distributions havefour casesThe first case is that the tier-2 device possess all the50000 VRIs while 16000 VRIs are duplicated to each tier-1 federations In the remaining three cases we respectivelyduplicate 28000 32000 and 36000 VRIs to each tier-1federation The experiment is conducted in a heterogeneousnetworking environment connecting machines by differentcommunication technologies shown in Table 1

The results of the experiment are shown in Figures 7 and8 Both Figures 7 and 8 reveal that distributing resources inlower tiers provides better performance Figures 8(a) and 8(b)show the latency differences caused by the resource distri-butions that 16000 32000 and 40000 VRIs are duplicatedin each cloudling tier The results before all brokers fullyloaded are also given with respective zoom-in charts Whendevices all continuously issue requests at a high frequency (3requests per second) the queue delays are obviously high andwe see a few requests that are dropped Requests from tier-1devices have lower loss rate since the resources requested are

8 Mobile Information Systems

Table 1 Machines in the offloading experiments using heterogeneous communication technologies

(Tier device) Specifications Physical network linkTier 2(2 0) Desktop PC Intel Core i5-3470 CPU 32GHz 4G RAM

EthernetTier 1(1 0) (1 1) Desktop PC Intel Core i7-4770 CPU 34GHz 8G RAMTier 1(1 2)Tier 1(1 3) Laptop Intel Core i7-2820QM CPU 23GHz 8G RAM Down WiFi UP 3G VPNTier 0(0 0sim3)

Desktop PC Intel Pentium D CPU 34GHz 1 G RAM EthernetTier 0(0 4sim6)Tier 0(0 7)Tier 0(0 8) Tablet ASUS Fonepad Intel Atom Z2420 12 GHz

WiFiTier 0(0 9) Cell Phone S3 SAMUNG Exynos 4412 14 GHzTier 0(0 10) Tablet ASUS Nexus7 NVIDIA Tegra 3 4-coreTier 0(0 11) Tablet WS-170 Qualcomm QCTMSM8x60 surf15 GHz duo-core USBlarrrarr PClarrrarr EthernetTier 0(0 12sim13) Virtual Machines (VirtualBox) EthernetTier 0(0 14) Tablet Samsung Galaxy CPU GT-I8260 WiFi

(011)

LAN LAN WiFi 3G

Tier 2

Tier 1

Tier 0

VPN

LAN LAN WiFi WiFi

(0 14)

(0 8)

(0 5)

(0 12)

(0 13) (0 9)

(0 7)(0 4) (0 6)(0 3)

(0 1)(0 10)

(0 2)(0 0)

(1 3)

(2 0)

(1 1)(1 0) (1 2)

Figure 6 Real-world HiBA topology for offloading performancemeasurement Gray nodes are local VMs of respective tier-1 brokersDiverse physical network links are exploited in tiers of federations

obtained in a shorter time For each physical machine theCPU utilization of theMMCNbrokering itself is smaller than2 depending on the number of cores The mean brokeringcomputation and database querying time is estimated tobe about 500ms while the WLAN delay time is smallerthan 5ms In the heterogeneous network case the latencydifference will be larger if upper tiers are physically at a longdistance from the cloudlings and cloudlets If the requestinterval is higher than the processing time the latency tendsto be smaller When more VRIs are duplicated in lower tierdevices we also see that the latencies are smaller

In this paper we leave algorithms such as request par-tition network embedding and federation for specific per-formance metrics as future work Instead we implementreal-world HiBA to prove that with feedback framework it isa new design paradigm and development platform for thesealgorithms

52 Energy-Aware Balancing In the energy-aware balancingexperiment we exploit four mobile devices of different typesfrom different manufacturers The cloudlet is self-organizedaccording to the broker election protocol in [8] The electedbroker is ASUS-Fonepad The other hierarchical federations

are in Figure 6 Each device including the broker itselfin each round updates its energy status to the CIS of thebroker (ASUS-Fonepad) Each mobile device in the cloudletis recharged to 100of respective battery energy capacityThedata and total data amount to be transmitted in each roundare randomly generated with mean of 24GB prior to theexperiment For each round 119905 total data amount is the samefor both balancing and nonbalancing cases In nonbalancingcase the data amounts are evenly assigned to members Inbalancing case the data amounts are determined by the fuzzycontroller in Section 41 The data amount assignment is ofunit 100MB When any device has remaining energy loweror equal to 40 the experiment is terminated The initialsingletons are configured as 119878(0) = 3 119872(0) = 5 and 119871(0) =

7 (Δ = 2) The result of energy-aware balancing is shownin Figure 9 The respective control systemsrsquo behaviors are inFigure 10 The fuzzy controllers of mobile devices all trackthe federationrsquos data amount to energy proportion (119879ℎ) Wesee that no matter how the performance of respective devicevaries a device with better transmission capability will shareits energy with others Tablet PCs are usually with highercapacity batteries Without energy sharing the remainingenergy of a tablet drops more slowly than a mobile phoneWith energy sharing tablets undertake more transmissionjobs and becomewith higher energy drop rate and themobilephones become with lower energy drop rate However wesee that the whole cloudlet federation has longer lifetimeTheexperiment can also be applied to energy consuming tasks inaddition to data transmission

53 Load Balancing The load-balanced cloud service inter-face (CSI) in the form of RESTful APIs is implementedby using Jersey and is deployed on Microsoft WindowsAzure public cloud platform We conduct an experiment forcomparing the proposed CSI to that of single machine Inour experiment five CSI machines are used to share therequests from users In the experiment the data producingrate for each user is 1 request per second The performance

Mobile Information Systems 9

000 000000 000000 000000 000

000100200300

Homo Hetero16000 075 00028000 000 00032000 015 00036000 018 000

()

()

()

000100200300

Homo Hetero16000280003200036000

000100200300

Homo Hetero16000280003200036000

000 000000 012015 000000 000

Tier-1 1120582 = 666Tier-1 1120582 = 333

1600028000

3200036000

1600028000

3200036000

1600028000

3200036000

Tier-11120582 = 999

(a)

110 340050 230082 180080 170

030 330016 160000 130016 030

030 020000 000000 010000 000

1600028000

3200036000

000100200300

Homo Hetero16000280003200036000

()

000100200300

Homo Hetero16000280003200036000

()

000100200300

Homo Hetero16000280003200036000

()

Tier-0 1120582 = 666Tier-0 1120582 = 333

1600028000

3200036000

1600028000

3200036000

Tier-0 1120582 = 999

(b)

Figure 7 Mean losses of tier 0 (b) and tier 1 (a) in cases of different resource distributions when 16000 28000 32000 and 36000 VRIs areduplicated to each tier-1 federation Request intervals 1120582 are in milliseconds

Case3_tier0

0

20000

40000

1 6 11 16 21 26 31 36

050000

100000150000200000250000300000350000

Late

ncy

(ms)

898513 8121 25 7733 37 736949 6157 93 975345412917 655 91

Request

1120582 = 333

1120582 = 666

1120582 = 999

(a) 16000 VRIs in each tier-1 federation

Case3_tier0

0

20000

40000

1 6 11 16 21 26 31 36 41

050000

100000150000200000250000300000350000

Late

ncy

(ms)

898513 8121 25 7733 37 736949 6157 93 975345412917 655 91

Request

1120582 = 333

1120582 = 666

1120582 = 999

(b) 40000 VRIs in each tier-1 federation

Figure 8 Mean latencies in cases of different resource distributions

metric is the missing rate defined as (119879 minus 119878)119879 where 119879

is the number of total requests and 119878 is the number ofsuccessful requests Table 2 shows the experimental resultsunder different amount of users from 100 to 300 We can seethat when the number of users is low (100 users) both CSIdesigns can successfully process their data requests As thenumber of users increases (eg 300 users) our proposed CSIcan almost process it with only 0 27missing rate howeverover 27 requests are failed to deliver data to cloud databasein the CSI of single machine

6 Conclusion

We have proposed and implemented a mobile device-centriccloud computing architecture HiBA based on feedback con-trol framework The proposed hierarchical brokering archi-tecture is self-organized featuring scalability and hierarchicalautonomy and is easy to embed management and controlalgorithms to develop a federation with better performancein terms of availability and latency Mobile devices and smallservers federate into cloudlets and cloudlings of hierarchical

10 Mobile Information Systems

Table 2 Experimental comparison of different CSI designs

Users Single Load-balanced schemeTotal requests Successful requests Missing requests Total requests Successful requests Missing requests

100 52395 52395 0 53370 53370 0200 97457 80947 1691 104244 104244 0300 113624 81883 2794 131742 131388 027

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 251

No control-ASUS-FonepadNo control-ASUS-NEXUSNo control-HTC-M7No control-SAMSUNG-GTI8260With control-ASUS-FonepadWith control-ASUS-NEXUSWith control-HTC-M7With control-SAMSUNG-GTI8260

0102030405060708090

100

()

Figure 9 Energy-aware balancing result 119909-axis round 119910-axisremaining energy

02468

101214161820

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 241

ASUS-FonepadASUS-NEXUSHTC-M7

SAMSUNG-GTI8260Th

Figure 10 Data amount to energy proportions of individual devices(119883119894) and the cloudlet federation (119879ℎ) 119909-axis round 119910-axis dataamount to energy proportions

tiers such that mobile devices not only request servicesbut also provide resources required by diverse servicesThe implemented HiBA architecture with feedback controlframework proves improved performancewhen resources areadequately distributed to lower tier federations rather thanbeing centralized in a remote cloud server Through two casestudies of cloudlet energy-aware balancing and bigger data

center load balancing we show that the HiBA is actually adevelopment platform for mobile cloud computing and pro-vides a feasible solution to UCN services Future worksinclude optimization algorithms embedding and big dataanalytics applicationswith crowd sensing are to be continued

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

Thanks are due to the Ministry of Science and Tech-nology (MOST which was the National Science Coun-cil) in Taiwan for sponsoring this research under con-tinuous funded Projects NSC101-2221-E-327-020- NSC101-2221-E-327-022- NSC102-2218-E-327-002- MOST103-2221-E-327-048- MOST105-2221-E-327-027- andMOST105-2221-E-006-141-

References

[1] Technicolor ldquoUser-Centric Networkingrdquo httpusercentricnet-workingeuabout-ucn

[2] G Iosifidis L Gao J Huang and L Tassiulas ldquoIncentive mech-anisms for user-provided networksrdquo IEEE CommunicationsMagazine vol 52 no 9 pp 20ndash27 2014

[3] B A A Nunes M A S SAntos B T De OliveirA C B MArgiK ObrAczkA and T Turletti ldquoSoftware-defined-networking-enabled capacity sharing in user-centric networksrdquo IEEE Com-munications Magazine vol 52 no 9 pp 28ndash36 2014

[4] G Aloi M D Felice V Loscrı P Pace and G Ruggeri ldquoSpon-taneous smartphone networks as a user-centric solution for thefuture internetrdquo IEEECommunicationsMagazine vol 52 no 12pp 26ndash33 2014

[5] F Hao M Jiao G Min and L T Yang ldquoA trajectory-basedrecruitment strategy of social sensors for participatory sensingrdquoIEEE Communications Magazine vol 52 no 12 pp 41ndash47 2014

[6] M-L LeeH-YChung andF-MYu ldquoModeling of hierarchicalfuzzy systemsrdquo Fuzzy Sets and Systems vol 138 no 2 pp 343ndash361 2003

[7] T C Lin M-Y Pai C-L Chen and C-C Chen ldquoLoad-balanced cloud service interface for the HiBA mobile cloudenvironmentrdquo in Proceedings of the IEEE International Confer-ence onConsumer ElectronicsmdashTaiwan (ICCE-TW rsquo15) pp 360ndash361 Taipei Taiwan June 2015

[8] C-L Chen S-C Chen C Chang and C Lin ldquoScalable andautonomous mobile device-centric cloud for secured D2Dsharingrdquo in Information Security Applications vol 8909 ofLecture Notes in Computer Science pp 177ndash189 2015

Mobile Information Systems 11

[9] W-T Su W Liu C-L Chen and T-P Chen ldquoCloud accesscontrol in multi-layer cloud networksrdquo in Proceedings ofthe IEEE International Conference on Consumer ElectronicsmdashTaiwan (ICCE-TW rsquo15) pp 364ndash365 IEEE Taipei Taiwan June2015

[10] H Huang H Yang and E H Lu ldquoA fuzzy-rough set basedontology for hybrid recommendationrdquo in Proceedings of theIEEE International Conference on Consumer ElectronicsmdashTaiwan (ICCE-TW rsquo12) pp 358ndash359 Taipei Taiwan June 2015

[11] C-L Chen and C-T Chen ldquoHierarchical Brokering with feed-back state observation in Mobile Device-Centric Cloudsrdquo inProceedings of the 7th International Conference on Ubiquitousand Future Networks (ICUFN rsquo15) pp 410ndash415 July 2015

[12] C J S Decusatis A Carranza and C M Decusatis ldquoCom-munication within clouds open standards and proprietaryprotocols for data center networkingrdquo IEEE CommunicationsMagazine vol 50 no 9 pp 26ndash33 2012

[13] M Satyanarayanan P Bahl R Caceres andNDavies ldquoThe casefor VM-based cloudlets in mobile computingrdquo IEEE PervasiveComputing vol 8 no 4 pp 14ndash23 2009

[14] S Mahadev ldquoMobile computing the next decaderdquo in Proceed-ings of the the 8th International Conference on Mobile SystemsApplications and Services (MobiSysrsquo10) pp 1ndash6 San FranciscoCalif USA June 2010

[15] C-L Chen J-W Lee W-T Su M-F Horng and Y-HKuo ldquoNoise-referred energy-proportional routing with packetlength adaptation for clustered sensor networksrdquo InternationalJournal of Ad Hoc and Ubiquitous Computing vol 3 no 4 pp224ndash235 2008

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mobile Information Systems 7

Service module 1

Service module 2

Service

CSIallocation

table

Clouddatabase

HiBA networking

Cloud bank tier

(1)(3)

(4)

(5)

(6)

(7)(8)

(2)CSI

manager CSIN 1(9)

CSIN n

module m

middot middot middotmiddot middot middot

middot middot middot

Figure 5 Framework of load-balanced cloud service interface

denoted as CSINi in Figure 5 A CSIN can receivemembersrsquo requests and obtain results by interactingwith requested cloud service module In our designeach cloud service interface in CSIN is implementedas a RESTful API for universally communicating withdifferent types of mobile devices A CSIN is then therealization of CES in the MMCN framework

(2) CSImanager it is designed tomanage the usage statusof CSINs andmatchesmobile users to CSINsThe effi-ciency of the load balance depends on the matchingmethod of the CSI Manager A CSI manager is thenthe realization of CCS in the MMCN framework

(3) CSI allocation table it maintains the serving statusbetween mobile users and CSINs and is used by theCSI manager for assisting the matching decision forthe new-arrived mobile users This table is stored inthe cloud database A CSI allocation table is then therealization of CIS in the MMCN framework

The processing flow of using a cloud service for a mobile useris designed for achieving the load balance The idea of com-mitting requests requires three phases (1) service registration(Steps (1)-(2)) (2) service execution (Steps (3)ndash(7)) and (3)service deregistration (Steps (8)-(9)) The detailed steps alsoillustrated in Figure 5 are presented as follows

(1) Mobile user MU119894sends a request to CSI manager for

allocating a CSI address(2) CSI manager searches for a CSI machine with least

number of serving users say CSIN119899 in the CSI allo-

cation table and then registers (MU119894 CSIN

119899) into the

CSI allocation table(3) CSI manager informs MU

119894that CSIN

119899can serve his

her cloud service requests in the following period(4) MU

119894configures the cloud service interface by replac-

ing the IP attribute in the service template with the IPaddress of CSIN

119899

(5) MU119894connects to CSIN

119899through the configured URL

of the cloud service interface and sends a request(6) CSIN

119899delivers the request to the associated cloud

service which may access the cloud databases ifrequired After processing the results will be sentback to CSIN

119899

(7) CSIN119899sends the results to MU

119894 (If more requests are

required to processed Steps (3)ndash(7) are repeated)(8) MU

119894deregisters CSIN

119899to CSI manager

(9) CSI manager removes the registration record of MU119894

from the CSI allocation table

Notice that the matching rule for a new mobile user andCSINs in the current design uses least-user-first basis asshown in Step (2) The matching principle can be modifiedaccording to different demands In the experiments wewill show that the current design already obtains splendidperformance More study on customizing matching methodsis one of our future research directions That is althoughfive machines are used in our CSI (more than the single-machine CSI) the improvement is by a factor of one hundredindicating the superior performance of our proposed load-balanced CSI mechanism under the HiBA architecture

5 Real-World Experiments

51 HiBA Baseline Performance We conduct an experimentdemonstrating performance difference in terms of availabil-ity and latency when distributions of virtualized resourceinstances (VRIs) differ We leave development of enhancingalgorithms for specific purposes in the CCS as future workwhich will seek performance in terms of various metricsA total of 50000 VRIs are distributed in a 3-tier HiBAfederation comprising Android smart phones tablets andVMs in desktop PCs Figure 6 is the HiBA topology andrelated specifications of the machines are shown in Table 1Each device is labeled (119894 119895) if it is the 119895th node at tier 119894 Device(1 3) connects to (2 0) through a 3G access point with VPNtunneling Device (0 11) connects to (1 0) with USB andshares Internet connection from (1 0) These machines havedifferent computing ability though request and task queuesbeing of the same size of 20 to preserve the differences ofcomputing ability Requests are randomly generated in eachdevice Mean request arrivals (120582) are 3 15 and 1 whichequivalently mean that request intervals (1120582) are 333ms666ms and 1 s respectively The resource distributions havefour casesThe first case is that the tier-2 device possess all the50000 VRIs while 16000 VRIs are duplicated to each tier-1 federations In the remaining three cases we respectivelyduplicate 28000 32000 and 36000 VRIs to each tier-1federation The experiment is conducted in a heterogeneousnetworking environment connecting machines by differentcommunication technologies shown in Table 1

The results of the experiment are shown in Figures 7 and8 Both Figures 7 and 8 reveal that distributing resources inlower tiers provides better performance Figures 8(a) and 8(b)show the latency differences caused by the resource distri-butions that 16000 32000 and 40000 VRIs are duplicatedin each cloudling tier The results before all brokers fullyloaded are also given with respective zoom-in charts Whendevices all continuously issue requests at a high frequency (3requests per second) the queue delays are obviously high andwe see a few requests that are dropped Requests from tier-1devices have lower loss rate since the resources requested are

8 Mobile Information Systems

Table 1 Machines in the offloading experiments using heterogeneous communication technologies

(Tier device) Specifications Physical network linkTier 2(2 0) Desktop PC Intel Core i5-3470 CPU 32GHz 4G RAM

EthernetTier 1(1 0) (1 1) Desktop PC Intel Core i7-4770 CPU 34GHz 8G RAMTier 1(1 2)Tier 1(1 3) Laptop Intel Core i7-2820QM CPU 23GHz 8G RAM Down WiFi UP 3G VPNTier 0(0 0sim3)

Desktop PC Intel Pentium D CPU 34GHz 1 G RAM EthernetTier 0(0 4sim6)Tier 0(0 7)Tier 0(0 8) Tablet ASUS Fonepad Intel Atom Z2420 12 GHz

WiFiTier 0(0 9) Cell Phone S3 SAMUNG Exynos 4412 14 GHzTier 0(0 10) Tablet ASUS Nexus7 NVIDIA Tegra 3 4-coreTier 0(0 11) Tablet WS-170 Qualcomm QCTMSM8x60 surf15 GHz duo-core USBlarrrarr PClarrrarr EthernetTier 0(0 12sim13) Virtual Machines (VirtualBox) EthernetTier 0(0 14) Tablet Samsung Galaxy CPU GT-I8260 WiFi

(011)

LAN LAN WiFi 3G

Tier 2

Tier 1

Tier 0

VPN

LAN LAN WiFi WiFi

(0 14)

(0 8)

(0 5)

(0 12)

(0 13) (0 9)

(0 7)(0 4) (0 6)(0 3)

(0 1)(0 10)

(0 2)(0 0)

(1 3)

(2 0)

(1 1)(1 0) (1 2)

Figure 6 Real-world HiBA topology for offloading performancemeasurement Gray nodes are local VMs of respective tier-1 brokersDiverse physical network links are exploited in tiers of federations

obtained in a shorter time For each physical machine theCPU utilization of theMMCNbrokering itself is smaller than2 depending on the number of cores The mean brokeringcomputation and database querying time is estimated tobe about 500ms while the WLAN delay time is smallerthan 5ms In the heterogeneous network case the latencydifference will be larger if upper tiers are physically at a longdistance from the cloudlings and cloudlets If the requestinterval is higher than the processing time the latency tendsto be smaller When more VRIs are duplicated in lower tierdevices we also see that the latencies are smaller

In this paper we leave algorithms such as request par-tition network embedding and federation for specific per-formance metrics as future work Instead we implementreal-world HiBA to prove that with feedback framework it isa new design paradigm and development platform for thesealgorithms

52 Energy-Aware Balancing In the energy-aware balancingexperiment we exploit four mobile devices of different typesfrom different manufacturers The cloudlet is self-organizedaccording to the broker election protocol in [8] The electedbroker is ASUS-Fonepad The other hierarchical federations

are in Figure 6 Each device including the broker itselfin each round updates its energy status to the CIS of thebroker (ASUS-Fonepad) Each mobile device in the cloudletis recharged to 100of respective battery energy capacityThedata and total data amount to be transmitted in each roundare randomly generated with mean of 24GB prior to theexperiment For each round 119905 total data amount is the samefor both balancing and nonbalancing cases In nonbalancingcase the data amounts are evenly assigned to members Inbalancing case the data amounts are determined by the fuzzycontroller in Section 41 The data amount assignment is ofunit 100MB When any device has remaining energy loweror equal to 40 the experiment is terminated The initialsingletons are configured as 119878(0) = 3 119872(0) = 5 and 119871(0) =

7 (Δ = 2) The result of energy-aware balancing is shownin Figure 9 The respective control systemsrsquo behaviors are inFigure 10 The fuzzy controllers of mobile devices all trackthe federationrsquos data amount to energy proportion (119879ℎ) Wesee that no matter how the performance of respective devicevaries a device with better transmission capability will shareits energy with others Tablet PCs are usually with highercapacity batteries Without energy sharing the remainingenergy of a tablet drops more slowly than a mobile phoneWith energy sharing tablets undertake more transmissionjobs and becomewith higher energy drop rate and themobilephones become with lower energy drop rate However wesee that the whole cloudlet federation has longer lifetimeTheexperiment can also be applied to energy consuming tasks inaddition to data transmission

53 Load Balancing The load-balanced cloud service inter-face (CSI) in the form of RESTful APIs is implementedby using Jersey and is deployed on Microsoft WindowsAzure public cloud platform We conduct an experiment forcomparing the proposed CSI to that of single machine Inour experiment five CSI machines are used to share therequests from users In the experiment the data producingrate for each user is 1 request per second The performance

Mobile Information Systems 9

000 000000 000000 000000 000

000100200300

Homo Hetero16000 075 00028000 000 00032000 015 00036000 018 000

()

()

()

000100200300

Homo Hetero16000280003200036000

000100200300

Homo Hetero16000280003200036000

000 000000 012015 000000 000

Tier-1 1120582 = 666Tier-1 1120582 = 333

1600028000

3200036000

1600028000

3200036000

1600028000

3200036000

Tier-11120582 = 999

(a)

110 340050 230082 180080 170

030 330016 160000 130016 030

030 020000 000000 010000 000

1600028000

3200036000

000100200300

Homo Hetero16000280003200036000

()

000100200300

Homo Hetero16000280003200036000

()

000100200300

Homo Hetero16000280003200036000

()

Tier-0 1120582 = 666Tier-0 1120582 = 333

1600028000

3200036000

1600028000

3200036000

Tier-0 1120582 = 999

(b)

Figure 7 Mean losses of tier 0 (b) and tier 1 (a) in cases of different resource distributions when 16000 28000 32000 and 36000 VRIs areduplicated to each tier-1 federation Request intervals 1120582 are in milliseconds

Case3_tier0

0

20000

40000

1 6 11 16 21 26 31 36

050000

100000150000200000250000300000350000

Late

ncy

(ms)

898513 8121 25 7733 37 736949 6157 93 975345412917 655 91

Request

1120582 = 333

1120582 = 666

1120582 = 999

(a) 16000 VRIs in each tier-1 federation

Case3_tier0

0

20000

40000

1 6 11 16 21 26 31 36 41

050000

100000150000200000250000300000350000

Late

ncy

(ms)

898513 8121 25 7733 37 736949 6157 93 975345412917 655 91

Request

1120582 = 333

1120582 = 666

1120582 = 999

(b) 40000 VRIs in each tier-1 federation

Figure 8 Mean latencies in cases of different resource distributions

metric is the missing rate defined as (119879 minus 119878)119879 where 119879

is the number of total requests and 119878 is the number ofsuccessful requests Table 2 shows the experimental resultsunder different amount of users from 100 to 300 We can seethat when the number of users is low (100 users) both CSIdesigns can successfully process their data requests As thenumber of users increases (eg 300 users) our proposed CSIcan almost process it with only 0 27missing rate howeverover 27 requests are failed to deliver data to cloud databasein the CSI of single machine

6 Conclusion

We have proposed and implemented a mobile device-centriccloud computing architecture HiBA based on feedback con-trol framework The proposed hierarchical brokering archi-tecture is self-organized featuring scalability and hierarchicalautonomy and is easy to embed management and controlalgorithms to develop a federation with better performancein terms of availability and latency Mobile devices and smallservers federate into cloudlets and cloudlings of hierarchical

10 Mobile Information Systems

Table 2 Experimental comparison of different CSI designs

Users Single Load-balanced schemeTotal requests Successful requests Missing requests Total requests Successful requests Missing requests

100 52395 52395 0 53370 53370 0200 97457 80947 1691 104244 104244 0300 113624 81883 2794 131742 131388 027

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 251

No control-ASUS-FonepadNo control-ASUS-NEXUSNo control-HTC-M7No control-SAMSUNG-GTI8260With control-ASUS-FonepadWith control-ASUS-NEXUSWith control-HTC-M7With control-SAMSUNG-GTI8260

0102030405060708090

100

()

Figure 9 Energy-aware balancing result 119909-axis round 119910-axisremaining energy

02468

101214161820

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 241

ASUS-FonepadASUS-NEXUSHTC-M7

SAMSUNG-GTI8260Th

Figure 10 Data amount to energy proportions of individual devices(119883119894) and the cloudlet federation (119879ℎ) 119909-axis round 119910-axis dataamount to energy proportions

tiers such that mobile devices not only request servicesbut also provide resources required by diverse servicesThe implemented HiBA architecture with feedback controlframework proves improved performancewhen resources areadequately distributed to lower tier federations rather thanbeing centralized in a remote cloud server Through two casestudies of cloudlet energy-aware balancing and bigger data

center load balancing we show that the HiBA is actually adevelopment platform for mobile cloud computing and pro-vides a feasible solution to UCN services Future worksinclude optimization algorithms embedding and big dataanalytics applicationswith crowd sensing are to be continued

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

Thanks are due to the Ministry of Science and Tech-nology (MOST which was the National Science Coun-cil) in Taiwan for sponsoring this research under con-tinuous funded Projects NSC101-2221-E-327-020- NSC101-2221-E-327-022- NSC102-2218-E-327-002- MOST103-2221-E-327-048- MOST105-2221-E-327-027- andMOST105-2221-E-006-141-

References

[1] Technicolor ldquoUser-Centric Networkingrdquo httpusercentricnet-workingeuabout-ucn

[2] G Iosifidis L Gao J Huang and L Tassiulas ldquoIncentive mech-anisms for user-provided networksrdquo IEEE CommunicationsMagazine vol 52 no 9 pp 20ndash27 2014

[3] B A A Nunes M A S SAntos B T De OliveirA C B MArgiK ObrAczkA and T Turletti ldquoSoftware-defined-networking-enabled capacity sharing in user-centric networksrdquo IEEE Com-munications Magazine vol 52 no 9 pp 28ndash36 2014

[4] G Aloi M D Felice V Loscrı P Pace and G Ruggeri ldquoSpon-taneous smartphone networks as a user-centric solution for thefuture internetrdquo IEEECommunicationsMagazine vol 52 no 12pp 26ndash33 2014

[5] F Hao M Jiao G Min and L T Yang ldquoA trajectory-basedrecruitment strategy of social sensors for participatory sensingrdquoIEEE Communications Magazine vol 52 no 12 pp 41ndash47 2014

[6] M-L LeeH-YChung andF-MYu ldquoModeling of hierarchicalfuzzy systemsrdquo Fuzzy Sets and Systems vol 138 no 2 pp 343ndash361 2003

[7] T C Lin M-Y Pai C-L Chen and C-C Chen ldquoLoad-balanced cloud service interface for the HiBA mobile cloudenvironmentrdquo in Proceedings of the IEEE International Confer-ence onConsumer ElectronicsmdashTaiwan (ICCE-TW rsquo15) pp 360ndash361 Taipei Taiwan June 2015

[8] C-L Chen S-C Chen C Chang and C Lin ldquoScalable andautonomous mobile device-centric cloud for secured D2Dsharingrdquo in Information Security Applications vol 8909 ofLecture Notes in Computer Science pp 177ndash189 2015

Mobile Information Systems 11

[9] W-T Su W Liu C-L Chen and T-P Chen ldquoCloud accesscontrol in multi-layer cloud networksrdquo in Proceedings ofthe IEEE International Conference on Consumer ElectronicsmdashTaiwan (ICCE-TW rsquo15) pp 364ndash365 IEEE Taipei Taiwan June2015

[10] H Huang H Yang and E H Lu ldquoA fuzzy-rough set basedontology for hybrid recommendationrdquo in Proceedings of theIEEE International Conference on Consumer ElectronicsmdashTaiwan (ICCE-TW rsquo12) pp 358ndash359 Taipei Taiwan June 2015

[11] C-L Chen and C-T Chen ldquoHierarchical Brokering with feed-back state observation in Mobile Device-Centric Cloudsrdquo inProceedings of the 7th International Conference on Ubiquitousand Future Networks (ICUFN rsquo15) pp 410ndash415 July 2015

[12] C J S Decusatis A Carranza and C M Decusatis ldquoCom-munication within clouds open standards and proprietaryprotocols for data center networkingrdquo IEEE CommunicationsMagazine vol 50 no 9 pp 26ndash33 2012

[13] M Satyanarayanan P Bahl R Caceres andNDavies ldquoThe casefor VM-based cloudlets in mobile computingrdquo IEEE PervasiveComputing vol 8 no 4 pp 14ndash23 2009

[14] S Mahadev ldquoMobile computing the next decaderdquo in Proceed-ings of the the 8th International Conference on Mobile SystemsApplications and Services (MobiSysrsquo10) pp 1ndash6 San FranciscoCalif USA June 2010

[15] C-L Chen J-W Lee W-T Su M-F Horng and Y-HKuo ldquoNoise-referred energy-proportional routing with packetlength adaptation for clustered sensor networksrdquo InternationalJournal of Ad Hoc and Ubiquitous Computing vol 3 no 4 pp224ndash235 2008

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

8 Mobile Information Systems

Table 1 Machines in the offloading experiments using heterogeneous communication technologies

(Tier device) Specifications Physical network linkTier 2(2 0) Desktop PC Intel Core i5-3470 CPU 32GHz 4G RAM

EthernetTier 1(1 0) (1 1) Desktop PC Intel Core i7-4770 CPU 34GHz 8G RAMTier 1(1 2)Tier 1(1 3) Laptop Intel Core i7-2820QM CPU 23GHz 8G RAM Down WiFi UP 3G VPNTier 0(0 0sim3)

Desktop PC Intel Pentium D CPU 34GHz 1 G RAM EthernetTier 0(0 4sim6)Tier 0(0 7)Tier 0(0 8) Tablet ASUS Fonepad Intel Atom Z2420 12 GHz

WiFiTier 0(0 9) Cell Phone S3 SAMUNG Exynos 4412 14 GHzTier 0(0 10) Tablet ASUS Nexus7 NVIDIA Tegra 3 4-coreTier 0(0 11) Tablet WS-170 Qualcomm QCTMSM8x60 surf15 GHz duo-core USBlarrrarr PClarrrarr EthernetTier 0(0 12sim13) Virtual Machines (VirtualBox) EthernetTier 0(0 14) Tablet Samsung Galaxy CPU GT-I8260 WiFi

(011)

LAN LAN WiFi 3G

Tier 2

Tier 1

Tier 0

VPN

LAN LAN WiFi WiFi

(0 14)

(0 8)

(0 5)

(0 12)

(0 13) (0 9)

(0 7)(0 4) (0 6)(0 3)

(0 1)(0 10)

(0 2)(0 0)

(1 3)

(2 0)

(1 1)(1 0) (1 2)

Figure 6 Real-world HiBA topology for offloading performancemeasurement Gray nodes are local VMs of respective tier-1 brokersDiverse physical network links are exploited in tiers of federations

obtained in a shorter time For each physical machine theCPU utilization of theMMCNbrokering itself is smaller than2 depending on the number of cores The mean brokeringcomputation and database querying time is estimated tobe about 500ms while the WLAN delay time is smallerthan 5ms In the heterogeneous network case the latencydifference will be larger if upper tiers are physically at a longdistance from the cloudlings and cloudlets If the requestinterval is higher than the processing time the latency tendsto be smaller When more VRIs are duplicated in lower tierdevices we also see that the latencies are smaller

In this paper we leave algorithms such as request par-tition network embedding and federation for specific per-formance metrics as future work Instead we implementreal-world HiBA to prove that with feedback framework it isa new design paradigm and development platform for thesealgorithms

52 Energy-Aware Balancing In the energy-aware balancingexperiment we exploit four mobile devices of different typesfrom different manufacturers The cloudlet is self-organizedaccording to the broker election protocol in [8] The electedbroker is ASUS-Fonepad The other hierarchical federations

are in Figure 6 Each device including the broker itselfin each round updates its energy status to the CIS of thebroker (ASUS-Fonepad) Each mobile device in the cloudletis recharged to 100of respective battery energy capacityThedata and total data amount to be transmitted in each roundare randomly generated with mean of 24GB prior to theexperiment For each round 119905 total data amount is the samefor both balancing and nonbalancing cases In nonbalancingcase the data amounts are evenly assigned to members Inbalancing case the data amounts are determined by the fuzzycontroller in Section 41 The data amount assignment is ofunit 100MB When any device has remaining energy loweror equal to 40 the experiment is terminated The initialsingletons are configured as 119878(0) = 3 119872(0) = 5 and 119871(0) =

7 (Δ = 2) The result of energy-aware balancing is shownin Figure 9 The respective control systemsrsquo behaviors are inFigure 10 The fuzzy controllers of mobile devices all trackthe federationrsquos data amount to energy proportion (119879ℎ) Wesee that no matter how the performance of respective devicevaries a device with better transmission capability will shareits energy with others Tablet PCs are usually with highercapacity batteries Without energy sharing the remainingenergy of a tablet drops more slowly than a mobile phoneWith energy sharing tablets undertake more transmissionjobs and becomewith higher energy drop rate and themobilephones become with lower energy drop rate However wesee that the whole cloudlet federation has longer lifetimeTheexperiment can also be applied to energy consuming tasks inaddition to data transmission

53 Load Balancing The load-balanced cloud service inter-face (CSI) in the form of RESTful APIs is implementedby using Jersey and is deployed on Microsoft WindowsAzure public cloud platform We conduct an experiment forcomparing the proposed CSI to that of single machine Inour experiment five CSI machines are used to share therequests from users In the experiment the data producingrate for each user is 1 request per second The performance

Mobile Information Systems 9

000 000000 000000 000000 000

000100200300

Homo Hetero16000 075 00028000 000 00032000 015 00036000 018 000

()

()

()

000100200300

Homo Hetero16000280003200036000

000100200300

Homo Hetero16000280003200036000

000 000000 012015 000000 000

Tier-1 1120582 = 666Tier-1 1120582 = 333

1600028000

3200036000

1600028000

3200036000

1600028000

3200036000

Tier-11120582 = 999

(a)

110 340050 230082 180080 170

030 330016 160000 130016 030

030 020000 000000 010000 000

1600028000

3200036000

000100200300

Homo Hetero16000280003200036000

()

000100200300

Homo Hetero16000280003200036000

()

000100200300

Homo Hetero16000280003200036000

()

Tier-0 1120582 = 666Tier-0 1120582 = 333

1600028000

3200036000

1600028000

3200036000

Tier-0 1120582 = 999

(b)

Figure 7 Mean losses of tier 0 (b) and tier 1 (a) in cases of different resource distributions when 16000 28000 32000 and 36000 VRIs areduplicated to each tier-1 federation Request intervals 1120582 are in milliseconds

Case3_tier0

0

20000

40000

1 6 11 16 21 26 31 36

050000

100000150000200000250000300000350000

Late

ncy

(ms)

898513 8121 25 7733 37 736949 6157 93 975345412917 655 91

Request

1120582 = 333

1120582 = 666

1120582 = 999

(a) 16000 VRIs in each tier-1 federation

Case3_tier0

0

20000

40000

1 6 11 16 21 26 31 36 41

050000

100000150000200000250000300000350000

Late

ncy

(ms)

898513 8121 25 7733 37 736949 6157 93 975345412917 655 91

Request

1120582 = 333

1120582 = 666

1120582 = 999

(b) 40000 VRIs in each tier-1 federation

Figure 8 Mean latencies in cases of different resource distributions

metric is the missing rate defined as (119879 minus 119878)119879 where 119879

is the number of total requests and 119878 is the number ofsuccessful requests Table 2 shows the experimental resultsunder different amount of users from 100 to 300 We can seethat when the number of users is low (100 users) both CSIdesigns can successfully process their data requests As thenumber of users increases (eg 300 users) our proposed CSIcan almost process it with only 0 27missing rate howeverover 27 requests are failed to deliver data to cloud databasein the CSI of single machine

6 Conclusion

We have proposed and implemented a mobile device-centriccloud computing architecture HiBA based on feedback con-trol framework The proposed hierarchical brokering archi-tecture is self-organized featuring scalability and hierarchicalautonomy and is easy to embed management and controlalgorithms to develop a federation with better performancein terms of availability and latency Mobile devices and smallservers federate into cloudlets and cloudlings of hierarchical

10 Mobile Information Systems

Table 2 Experimental comparison of different CSI designs

Users Single Load-balanced schemeTotal requests Successful requests Missing requests Total requests Successful requests Missing requests

100 52395 52395 0 53370 53370 0200 97457 80947 1691 104244 104244 0300 113624 81883 2794 131742 131388 027

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 251

No control-ASUS-FonepadNo control-ASUS-NEXUSNo control-HTC-M7No control-SAMSUNG-GTI8260With control-ASUS-FonepadWith control-ASUS-NEXUSWith control-HTC-M7With control-SAMSUNG-GTI8260

0102030405060708090

100

()

Figure 9 Energy-aware balancing result 119909-axis round 119910-axisremaining energy

02468

101214161820

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 241

ASUS-FonepadASUS-NEXUSHTC-M7

SAMSUNG-GTI8260Th

Figure 10 Data amount to energy proportions of individual devices(119883119894) and the cloudlet federation (119879ℎ) 119909-axis round 119910-axis dataamount to energy proportions

tiers such that mobile devices not only request servicesbut also provide resources required by diverse servicesThe implemented HiBA architecture with feedback controlframework proves improved performancewhen resources areadequately distributed to lower tier federations rather thanbeing centralized in a remote cloud server Through two casestudies of cloudlet energy-aware balancing and bigger data

center load balancing we show that the HiBA is actually adevelopment platform for mobile cloud computing and pro-vides a feasible solution to UCN services Future worksinclude optimization algorithms embedding and big dataanalytics applicationswith crowd sensing are to be continued

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

Thanks are due to the Ministry of Science and Tech-nology (MOST which was the National Science Coun-cil) in Taiwan for sponsoring this research under con-tinuous funded Projects NSC101-2221-E-327-020- NSC101-2221-E-327-022- NSC102-2218-E-327-002- MOST103-2221-E-327-048- MOST105-2221-E-327-027- andMOST105-2221-E-006-141-

References

[1] Technicolor ldquoUser-Centric Networkingrdquo httpusercentricnet-workingeuabout-ucn

[2] G Iosifidis L Gao J Huang and L Tassiulas ldquoIncentive mech-anisms for user-provided networksrdquo IEEE CommunicationsMagazine vol 52 no 9 pp 20ndash27 2014

[3] B A A Nunes M A S SAntos B T De OliveirA C B MArgiK ObrAczkA and T Turletti ldquoSoftware-defined-networking-enabled capacity sharing in user-centric networksrdquo IEEE Com-munications Magazine vol 52 no 9 pp 28ndash36 2014

[4] G Aloi M D Felice V Loscrı P Pace and G Ruggeri ldquoSpon-taneous smartphone networks as a user-centric solution for thefuture internetrdquo IEEECommunicationsMagazine vol 52 no 12pp 26ndash33 2014

[5] F Hao M Jiao G Min and L T Yang ldquoA trajectory-basedrecruitment strategy of social sensors for participatory sensingrdquoIEEE Communications Magazine vol 52 no 12 pp 41ndash47 2014

[6] M-L LeeH-YChung andF-MYu ldquoModeling of hierarchicalfuzzy systemsrdquo Fuzzy Sets and Systems vol 138 no 2 pp 343ndash361 2003

[7] T C Lin M-Y Pai C-L Chen and C-C Chen ldquoLoad-balanced cloud service interface for the HiBA mobile cloudenvironmentrdquo in Proceedings of the IEEE International Confer-ence onConsumer ElectronicsmdashTaiwan (ICCE-TW rsquo15) pp 360ndash361 Taipei Taiwan June 2015

[8] C-L Chen S-C Chen C Chang and C Lin ldquoScalable andautonomous mobile device-centric cloud for secured D2Dsharingrdquo in Information Security Applications vol 8909 ofLecture Notes in Computer Science pp 177ndash189 2015

Mobile Information Systems 11

[9] W-T Su W Liu C-L Chen and T-P Chen ldquoCloud accesscontrol in multi-layer cloud networksrdquo in Proceedings ofthe IEEE International Conference on Consumer ElectronicsmdashTaiwan (ICCE-TW rsquo15) pp 364ndash365 IEEE Taipei Taiwan June2015

[10] H Huang H Yang and E H Lu ldquoA fuzzy-rough set basedontology for hybrid recommendationrdquo in Proceedings of theIEEE International Conference on Consumer ElectronicsmdashTaiwan (ICCE-TW rsquo12) pp 358ndash359 Taipei Taiwan June 2015

[11] C-L Chen and C-T Chen ldquoHierarchical Brokering with feed-back state observation in Mobile Device-Centric Cloudsrdquo inProceedings of the 7th International Conference on Ubiquitousand Future Networks (ICUFN rsquo15) pp 410ndash415 July 2015

[12] C J S Decusatis A Carranza and C M Decusatis ldquoCom-munication within clouds open standards and proprietaryprotocols for data center networkingrdquo IEEE CommunicationsMagazine vol 50 no 9 pp 26ndash33 2012

[13] M Satyanarayanan P Bahl R Caceres andNDavies ldquoThe casefor VM-based cloudlets in mobile computingrdquo IEEE PervasiveComputing vol 8 no 4 pp 14ndash23 2009

[14] S Mahadev ldquoMobile computing the next decaderdquo in Proceed-ings of the the 8th International Conference on Mobile SystemsApplications and Services (MobiSysrsquo10) pp 1ndash6 San FranciscoCalif USA June 2010

[15] C-L Chen J-W Lee W-T Su M-F Horng and Y-HKuo ldquoNoise-referred energy-proportional routing with packetlength adaptation for clustered sensor networksrdquo InternationalJournal of Ad Hoc and Ubiquitous Computing vol 3 no 4 pp224ndash235 2008

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mobile Information Systems 9

000 000000 000000 000000 000

000100200300

Homo Hetero16000 075 00028000 000 00032000 015 00036000 018 000

()

()

()

000100200300

Homo Hetero16000280003200036000

000100200300

Homo Hetero16000280003200036000

000 000000 012015 000000 000

Tier-1 1120582 = 666Tier-1 1120582 = 333

1600028000

3200036000

1600028000

3200036000

1600028000

3200036000

Tier-11120582 = 999

(a)

110 340050 230082 180080 170

030 330016 160000 130016 030

030 020000 000000 010000 000

1600028000

3200036000

000100200300

Homo Hetero16000280003200036000

()

000100200300

Homo Hetero16000280003200036000

()

000100200300

Homo Hetero16000280003200036000

()

Tier-0 1120582 = 666Tier-0 1120582 = 333

1600028000

3200036000

1600028000

3200036000

Tier-0 1120582 = 999

(b)

Figure 7 Mean losses of tier 0 (b) and tier 1 (a) in cases of different resource distributions when 16000 28000 32000 and 36000 VRIs areduplicated to each tier-1 federation Request intervals 1120582 are in milliseconds

Case3_tier0

0

20000

40000

1 6 11 16 21 26 31 36

050000

100000150000200000250000300000350000

Late

ncy

(ms)

898513 8121 25 7733 37 736949 6157 93 975345412917 655 91

Request

1120582 = 333

1120582 = 666

1120582 = 999

(a) 16000 VRIs in each tier-1 federation

Case3_tier0

0

20000

40000

1 6 11 16 21 26 31 36 41

050000

100000150000200000250000300000350000

Late

ncy

(ms)

898513 8121 25 7733 37 736949 6157 93 975345412917 655 91

Request

1120582 = 333

1120582 = 666

1120582 = 999

(b) 40000 VRIs in each tier-1 federation

Figure 8 Mean latencies in cases of different resource distributions

metric is the missing rate defined as (119879 minus 119878)119879 where 119879

is the number of total requests and 119878 is the number ofsuccessful requests Table 2 shows the experimental resultsunder different amount of users from 100 to 300 We can seethat when the number of users is low (100 users) both CSIdesigns can successfully process their data requests As thenumber of users increases (eg 300 users) our proposed CSIcan almost process it with only 0 27missing rate howeverover 27 requests are failed to deliver data to cloud databasein the CSI of single machine

6 Conclusion

We have proposed and implemented a mobile device-centriccloud computing architecture HiBA based on feedback con-trol framework The proposed hierarchical brokering archi-tecture is self-organized featuring scalability and hierarchicalautonomy and is easy to embed management and controlalgorithms to develop a federation with better performancein terms of availability and latency Mobile devices and smallservers federate into cloudlets and cloudlings of hierarchical

10 Mobile Information Systems

Table 2 Experimental comparison of different CSI designs

Users Single Load-balanced schemeTotal requests Successful requests Missing requests Total requests Successful requests Missing requests

100 52395 52395 0 53370 53370 0200 97457 80947 1691 104244 104244 0300 113624 81883 2794 131742 131388 027

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 251

No control-ASUS-FonepadNo control-ASUS-NEXUSNo control-HTC-M7No control-SAMSUNG-GTI8260With control-ASUS-FonepadWith control-ASUS-NEXUSWith control-HTC-M7With control-SAMSUNG-GTI8260

0102030405060708090

100

()

Figure 9 Energy-aware balancing result 119909-axis round 119910-axisremaining energy

02468

101214161820

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 241

ASUS-FonepadASUS-NEXUSHTC-M7

SAMSUNG-GTI8260Th

Figure 10 Data amount to energy proportions of individual devices(119883119894) and the cloudlet federation (119879ℎ) 119909-axis round 119910-axis dataamount to energy proportions

tiers such that mobile devices not only request servicesbut also provide resources required by diverse servicesThe implemented HiBA architecture with feedback controlframework proves improved performancewhen resources areadequately distributed to lower tier federations rather thanbeing centralized in a remote cloud server Through two casestudies of cloudlet energy-aware balancing and bigger data

center load balancing we show that the HiBA is actually adevelopment platform for mobile cloud computing and pro-vides a feasible solution to UCN services Future worksinclude optimization algorithms embedding and big dataanalytics applicationswith crowd sensing are to be continued

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

Thanks are due to the Ministry of Science and Tech-nology (MOST which was the National Science Coun-cil) in Taiwan for sponsoring this research under con-tinuous funded Projects NSC101-2221-E-327-020- NSC101-2221-E-327-022- NSC102-2218-E-327-002- MOST103-2221-E-327-048- MOST105-2221-E-327-027- andMOST105-2221-E-006-141-

References

[1] Technicolor ldquoUser-Centric Networkingrdquo httpusercentricnet-workingeuabout-ucn

[2] G Iosifidis L Gao J Huang and L Tassiulas ldquoIncentive mech-anisms for user-provided networksrdquo IEEE CommunicationsMagazine vol 52 no 9 pp 20ndash27 2014

[3] B A A Nunes M A S SAntos B T De OliveirA C B MArgiK ObrAczkA and T Turletti ldquoSoftware-defined-networking-enabled capacity sharing in user-centric networksrdquo IEEE Com-munications Magazine vol 52 no 9 pp 28ndash36 2014

[4] G Aloi M D Felice V Loscrı P Pace and G Ruggeri ldquoSpon-taneous smartphone networks as a user-centric solution for thefuture internetrdquo IEEECommunicationsMagazine vol 52 no 12pp 26ndash33 2014

[5] F Hao M Jiao G Min and L T Yang ldquoA trajectory-basedrecruitment strategy of social sensors for participatory sensingrdquoIEEE Communications Magazine vol 52 no 12 pp 41ndash47 2014

[6] M-L LeeH-YChung andF-MYu ldquoModeling of hierarchicalfuzzy systemsrdquo Fuzzy Sets and Systems vol 138 no 2 pp 343ndash361 2003

[7] T C Lin M-Y Pai C-L Chen and C-C Chen ldquoLoad-balanced cloud service interface for the HiBA mobile cloudenvironmentrdquo in Proceedings of the IEEE International Confer-ence onConsumer ElectronicsmdashTaiwan (ICCE-TW rsquo15) pp 360ndash361 Taipei Taiwan June 2015

[8] C-L Chen S-C Chen C Chang and C Lin ldquoScalable andautonomous mobile device-centric cloud for secured D2Dsharingrdquo in Information Security Applications vol 8909 ofLecture Notes in Computer Science pp 177ndash189 2015

Mobile Information Systems 11

[9] W-T Su W Liu C-L Chen and T-P Chen ldquoCloud accesscontrol in multi-layer cloud networksrdquo in Proceedings ofthe IEEE International Conference on Consumer ElectronicsmdashTaiwan (ICCE-TW rsquo15) pp 364ndash365 IEEE Taipei Taiwan June2015

[10] H Huang H Yang and E H Lu ldquoA fuzzy-rough set basedontology for hybrid recommendationrdquo in Proceedings of theIEEE International Conference on Consumer ElectronicsmdashTaiwan (ICCE-TW rsquo12) pp 358ndash359 Taipei Taiwan June 2015

[11] C-L Chen and C-T Chen ldquoHierarchical Brokering with feed-back state observation in Mobile Device-Centric Cloudsrdquo inProceedings of the 7th International Conference on Ubiquitousand Future Networks (ICUFN rsquo15) pp 410ndash415 July 2015

[12] C J S Decusatis A Carranza and C M Decusatis ldquoCom-munication within clouds open standards and proprietaryprotocols for data center networkingrdquo IEEE CommunicationsMagazine vol 50 no 9 pp 26ndash33 2012

[13] M Satyanarayanan P Bahl R Caceres andNDavies ldquoThe casefor VM-based cloudlets in mobile computingrdquo IEEE PervasiveComputing vol 8 no 4 pp 14ndash23 2009

[14] S Mahadev ldquoMobile computing the next decaderdquo in Proceed-ings of the the 8th International Conference on Mobile SystemsApplications and Services (MobiSysrsquo10) pp 1ndash6 San FranciscoCalif USA June 2010

[15] C-L Chen J-W Lee W-T Su M-F Horng and Y-HKuo ldquoNoise-referred energy-proportional routing with packetlength adaptation for clustered sensor networksrdquo InternationalJournal of Ad Hoc and Ubiquitous Computing vol 3 no 4 pp224ndash235 2008

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

10 Mobile Information Systems

Table 2 Experimental comparison of different CSI designs

Users Single Load-balanced schemeTotal requests Successful requests Missing requests Total requests Successful requests Missing requests

100 52395 52395 0 53370 53370 0200 97457 80947 1691 104244 104244 0300 113624 81883 2794 131742 131388 027

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 251

No control-ASUS-FonepadNo control-ASUS-NEXUSNo control-HTC-M7No control-SAMSUNG-GTI8260With control-ASUS-FonepadWith control-ASUS-NEXUSWith control-HTC-M7With control-SAMSUNG-GTI8260

0102030405060708090

100

()

Figure 9 Energy-aware balancing result 119909-axis round 119910-axisremaining energy

02468

101214161820

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 241

ASUS-FonepadASUS-NEXUSHTC-M7

SAMSUNG-GTI8260Th

Figure 10 Data amount to energy proportions of individual devices(119883119894) and the cloudlet federation (119879ℎ) 119909-axis round 119910-axis dataamount to energy proportions

tiers such that mobile devices not only request servicesbut also provide resources required by diverse servicesThe implemented HiBA architecture with feedback controlframework proves improved performancewhen resources areadequately distributed to lower tier federations rather thanbeing centralized in a remote cloud server Through two casestudies of cloudlet energy-aware balancing and bigger data

center load balancing we show that the HiBA is actually adevelopment platform for mobile cloud computing and pro-vides a feasible solution to UCN services Future worksinclude optimization algorithms embedding and big dataanalytics applicationswith crowd sensing are to be continued

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

Thanks are due to the Ministry of Science and Tech-nology (MOST which was the National Science Coun-cil) in Taiwan for sponsoring this research under con-tinuous funded Projects NSC101-2221-E-327-020- NSC101-2221-E-327-022- NSC102-2218-E-327-002- MOST103-2221-E-327-048- MOST105-2221-E-327-027- andMOST105-2221-E-006-141-

References

[1] Technicolor ldquoUser-Centric Networkingrdquo httpusercentricnet-workingeuabout-ucn

[2] G Iosifidis L Gao J Huang and L Tassiulas ldquoIncentive mech-anisms for user-provided networksrdquo IEEE CommunicationsMagazine vol 52 no 9 pp 20ndash27 2014

[3] B A A Nunes M A S SAntos B T De OliveirA C B MArgiK ObrAczkA and T Turletti ldquoSoftware-defined-networking-enabled capacity sharing in user-centric networksrdquo IEEE Com-munications Magazine vol 52 no 9 pp 28ndash36 2014

[4] G Aloi M D Felice V Loscrı P Pace and G Ruggeri ldquoSpon-taneous smartphone networks as a user-centric solution for thefuture internetrdquo IEEECommunicationsMagazine vol 52 no 12pp 26ndash33 2014

[5] F Hao M Jiao G Min and L T Yang ldquoA trajectory-basedrecruitment strategy of social sensors for participatory sensingrdquoIEEE Communications Magazine vol 52 no 12 pp 41ndash47 2014

[6] M-L LeeH-YChung andF-MYu ldquoModeling of hierarchicalfuzzy systemsrdquo Fuzzy Sets and Systems vol 138 no 2 pp 343ndash361 2003

[7] T C Lin M-Y Pai C-L Chen and C-C Chen ldquoLoad-balanced cloud service interface for the HiBA mobile cloudenvironmentrdquo in Proceedings of the IEEE International Confer-ence onConsumer ElectronicsmdashTaiwan (ICCE-TW rsquo15) pp 360ndash361 Taipei Taiwan June 2015

[8] C-L Chen S-C Chen C Chang and C Lin ldquoScalable andautonomous mobile device-centric cloud for secured D2Dsharingrdquo in Information Security Applications vol 8909 ofLecture Notes in Computer Science pp 177ndash189 2015

Mobile Information Systems 11

[9] W-T Su W Liu C-L Chen and T-P Chen ldquoCloud accesscontrol in multi-layer cloud networksrdquo in Proceedings ofthe IEEE International Conference on Consumer ElectronicsmdashTaiwan (ICCE-TW rsquo15) pp 364ndash365 IEEE Taipei Taiwan June2015

[10] H Huang H Yang and E H Lu ldquoA fuzzy-rough set basedontology for hybrid recommendationrdquo in Proceedings of theIEEE International Conference on Consumer ElectronicsmdashTaiwan (ICCE-TW rsquo12) pp 358ndash359 Taipei Taiwan June 2015

[11] C-L Chen and C-T Chen ldquoHierarchical Brokering with feed-back state observation in Mobile Device-Centric Cloudsrdquo inProceedings of the 7th International Conference on Ubiquitousand Future Networks (ICUFN rsquo15) pp 410ndash415 July 2015

[12] C J S Decusatis A Carranza and C M Decusatis ldquoCom-munication within clouds open standards and proprietaryprotocols for data center networkingrdquo IEEE CommunicationsMagazine vol 50 no 9 pp 26ndash33 2012

[13] M Satyanarayanan P Bahl R Caceres andNDavies ldquoThe casefor VM-based cloudlets in mobile computingrdquo IEEE PervasiveComputing vol 8 no 4 pp 14ndash23 2009

[14] S Mahadev ldquoMobile computing the next decaderdquo in Proceed-ings of the the 8th International Conference on Mobile SystemsApplications and Services (MobiSysrsquo10) pp 1ndash6 San FranciscoCalif USA June 2010

[15] C-L Chen J-W Lee W-T Su M-F Horng and Y-HKuo ldquoNoise-referred energy-proportional routing with packetlength adaptation for clustered sensor networksrdquo InternationalJournal of Ad Hoc and Ubiquitous Computing vol 3 no 4 pp224ndash235 2008

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mobile Information Systems 11

[9] W-T Su W Liu C-L Chen and T-P Chen ldquoCloud accesscontrol in multi-layer cloud networksrdquo in Proceedings ofthe IEEE International Conference on Consumer ElectronicsmdashTaiwan (ICCE-TW rsquo15) pp 364ndash365 IEEE Taipei Taiwan June2015

[10] H Huang H Yang and E H Lu ldquoA fuzzy-rough set basedontology for hybrid recommendationrdquo in Proceedings of theIEEE International Conference on Consumer ElectronicsmdashTaiwan (ICCE-TW rsquo12) pp 358ndash359 Taipei Taiwan June 2015

[11] C-L Chen and C-T Chen ldquoHierarchical Brokering with feed-back state observation in Mobile Device-Centric Cloudsrdquo inProceedings of the 7th International Conference on Ubiquitousand Future Networks (ICUFN rsquo15) pp 410ndash415 July 2015

[12] C J S Decusatis A Carranza and C M Decusatis ldquoCom-munication within clouds open standards and proprietaryprotocols for data center networkingrdquo IEEE CommunicationsMagazine vol 50 no 9 pp 26ndash33 2012

[13] M Satyanarayanan P Bahl R Caceres andNDavies ldquoThe casefor VM-based cloudlets in mobile computingrdquo IEEE PervasiveComputing vol 8 no 4 pp 14ndash23 2009

[14] S Mahadev ldquoMobile computing the next decaderdquo in Proceed-ings of the the 8th International Conference on Mobile SystemsApplications and Services (MobiSysrsquo10) pp 1ndash6 San FranciscoCalif USA June 2010

[15] C-L Chen J-W Lee W-T Su M-F Horng and Y-HKuo ldquoNoise-referred energy-proportional routing with packetlength adaptation for clustered sensor networksrdquo InternationalJournal of Ad Hoc and Ubiquitous Computing vol 3 no 4 pp224ndash235 2008

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014