1

Ontology-based Semantic Priority Scheduling forMulti-domain Active Measurements1Prasad Calyam, 2Lakshmi Kumarasamy, 3Chang-Gun Lee, 2Fusun Ozguner

1Dept. of Computer Science, University of Missouri-Columbia, USA; Email: calyamp@missouri.edu2Dept. of Electrical and Computer Engineering, The Ohio State University, USA; Email: kumarasamy.2, ozguner.2@osu.edu

3School of Computer Science and Engineering, Seoul National University, South Korea; Email: cglee@snu.ac.kr

Abstract

Network control and management techniques (e.g., dynamic path switching and on-demand bandwidth provisioning) rely onactive measurements of the end-to-end network status. The measurements are needed to meet network monitoring objectives suchas network weather forecasting, anomaly detection, and fault-diagnosis. Recent widespread deployment of openly accessible multi-domain active measurement frameworks, such as perfSONAR, has resulted in users competing for system and network measurementresources. Hence, there is a need to prioritize measurement requests of users before they are scheduled on measurement resources.In this paper, we present a novel ontology-based semantic priority scheduling (SPS) algorithm that handles resource contentionwhile servicing measurement requests for meeting network monitoring objectives. We adopt ontologies to formalize semanticdefinitions and develop an inference engine to dynamically prioritize measurement requests. The prioritization is based upon userroles, user sampling preferences, resource policies, and oversampling mitigation factors. Performance evaluation results demonstratethat our SPS algorithm outperforms existing deterministic and heuristic algorithms in terms of user ‘satisfaction ratio’ and ‘averagestretch’ among serviced measurement requests. Further, by sampling experiments on real-network perfSONAR measurement datasets, we show that our SPS algorithm successfully mitigates oversampling and further improves the satisfaction ratio. Our SPSscheme and evaluation results are vital to manage large-scale measurement infrastructures used for meeting monitoring objectivesin the next-generation applications and networks.

I. INTRODUCTION

The next-generation of networks being developed are critical for supporting many cutting-edge end-applications used inbusiness and research environments. The business end-applications include desktop/immersive videoconferencing and data-backup for disaster recovery. The research end-applications include distributed data-intensive computation and visualization.To ensure next-generation networks provide adequate Quality of Service (QoS) to end-applications, researchers are developingsophisticated network control and management techniques. These techniques serve complex functions that enable networkequipment and end-applications to monitor, diagnose, and adapt to the network dynamics, so as to deliver optimum end-userperceived experience. The complex functions include dynamic path switching [1], on-demand bandwidth provisioning [2], andcall admission control [3].

The network status information is obtained through end-to-end active measurements that are collected and analyzed to meetnetwork monitoring objectives such as: (a) intra-domain/inter-domain paths status-checking for ensuring Service AgreementLevel (SLA) compliance, (b) network weather forecasting, (c) anomaly event detection, and (d) fault-diagnosis. The activemeasurement tools that include Ping, Traceroute, Iperf [4], and Pathrate [5] inject a series of test packets into a network pathand analyze their behavior to report network status in terms of metrics such as availability, delay, jitter, loss, route changes,and bottleneck-bandwidth.

In pursuit of developing sophisticated network control and management techniques, researchers employ different samplingtime-interval patterns as shown in Figure 1. These include periodic, random, stratified random, and adaptive patterns. Periodicsampling involves selection of sampling time-intervals according to a deterministic function and is required for e.g., to produceaccurate network weather forecasts [6] [9]. Whereas, random sampling involves selection of sampling time-intervals accordingto a random process, and is commonly used in network flow monitoring [10]. Stratified random sampling involves dividingparent population into subsets and drawing sampling time-intervals randomly from each group. That is shown to be usefulin validating SLA compliance [11] [12]. Finally, depending upon the application nature or diurnal characteristics, and also toreduce the amount of active measurement probing and measurement data storage, adaptive sampling is chosen by frameworkssuch as [13] (adapts sampling based on traffic rate) and [14] (adapts sampling for rapid anomaly detection). In any case, thesampling time-interval pattern chosen depends on the user’s monitoring accuracy objectives i.e., the chosen pattern shouldproduce the least variance between actual and estimated [15].

Corresponding Author is Prasad Calyam

This material is based upon work supported by the Department of Energy under Award Numbers: DE-SC0001331 and DE-SC0007531, and in part by the ITR&D program of MKE/KEIT [10035243]. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United StatesGovernment or any agency thereof.

2

Fig. 1. Network Status Sampling Patterns

Anticipating the above demands for network status measurements from researchers and also to serve their own routine moni-toring purposes, ISPs have instrumented networks with measurement frameworks such as NLANR AMP [16], perfSONAR [17],Network Weather Service (NWS) [6], and ActiveMon [9] [7]. The measurement frameworks allow sampling of measurementsat strategic points along the inter-domain and/or intra-domain, i.e., multi-domain network paths. The measurement samplingis performed using measurement schedulers that invoke active measurement tools in a manner that there are no “measurementconflicts” or violation of “measurement level agreements” (MLAs) [9]. Measurement conflict is a phenomenon that occurs whenmultiple concurrent active measurements are initiated over the same measurement servers or along the same network paths.Due to the CPU and channel resource contention, the results from such concurrent measurements produce misleading reportsof network performance. The MLAs are specified to regulate the active measurements from consuming network bandwidth,which might be required by actual application traffic. The amount of regulation is formally specified in terms of a certainpercentage (1 - 5)% or only a certain number of bits per second (1 - 2) Mbps of the network path bandwidth that can be usedfor measurement traffic.

Different measurement scheduling algorithms have been adopted in measurement frameworks. NLANR AMP uses round-robin scheduling, perfSONAR uses resource-broker scheduling, and NWS uses token-passing scheduling. In comparision,ActiveMon uses both earliest-deadline-first as well as heuristic-bin-packing scheduling with concurrent execution of non-conflicting measurements that have been shown to be more scalable than round-robin, resource-broker, and token-passingscheduling algorithms. Building upon ActiveMon’s scheduling algorithms with concurrent execution, variants have been recentlyproposed by [18] [19] and [20].

Given the rising trend in recent times among ISPs to deploy openly accessible measurement frameworks, a need has arisento develop measurement schedulers that handle semantic priorities. ISPs are using these frameworks to create “measurementfederations” that facilitate measurements across multiple domains for reaping the mutual benefits of performing end-to-end pathmeasurements. The most widely adopted is the perfSONAR framework that has been adopted by over a 100 user communitiesthat include major regional, national, and international ISPs in academia and universities [21]. Given that measurement resources(i.e., tool servers, network bandwidth) are limited, it might not be possible for a measurement scheduler to accommodate alluser requests, i.e., generate completely feasible schedules under high measurement request loads. Consequently, measurementrequests that could not be scheduled might adversely affect monitoring accuracy needed in critical resource adaptation decisions.Moreover, lack of semantic priorities might block intra-domain measurement requests that are more important than inter-domainmeasurement requests from an ISP’s perspective. In such cases, there is a need to prioritize measurement requests by usingsemantic priorities based on user and resource policies. Semantic priorities of measurement requests can indicate cases ofurgency to a measurement scheduler, which can then generate measurement schedules in a manner that supersedes typicalscheduling priorities, i.e., period, laxity, execution time. Our work in this paper is motivated by the fact that none of theexisting measurement scheduling algorithms have the ability to handle semantic priorities that allow preferential treatment ofmeasurement requests based on semantic definitions.

In this paper, we present a novel ontology-based semantic priority scheduling (SPS) algorithm that handles resourcecontention while servicing active measurement requests for meeting network monitoring objectives within multi-domain measure-ment federations. The SPS algorithm is conflict free and adheres to MLA specifications input by the users. The measurementloads that can be handled include diverse sampling pattern preferences, i.e., periodic, random, stratified random, adaptiveover measurement topologies, i.e., full-mesh, tree, hybrid with a varying number of tools and servers. We adopt ontologiesto formalize semantic definitions and develop an inference engine to dynamically prioritize measurement requests. Offlineprioritization is based upon semantic definitions of user roles, user sampling preferences, and resource policies. Onlineprioritization: (a) increases the priority of measurement requests whose monitoring accuracy drops below tolerable bounds,and (b) correspondingly decreases the priority of measurement requests that are within tolerable bounds and are foundto be oversampling in terms of their associated monitoring objective. Performance evaluation results demonstrate that ourSPS algorithm outperforms existing deterministic and heuristic algorithms in terms of user ‘satisfaction ratio’ and ‘averagestretch,’ i.e., fairness amongst serviced measurement requests. Further, using sampling experiments on real-network perfSONARmeasurement data sets, we show that our SPS algorithm successfully mitigates oversampling, and further improves satisfactionratio.

The remainder of this paper is organized as follows: Section II presents the problem background and solution methodology.Section III provides details of the semantic priority scheduling of measurement requests. Section IV presents performanceevaluation results of our proposed algorithm using simulations and real-network datasets. Section V concludes the paper.

3

Fig. 2. Measurement federation setup and measurement requests considered as a case study

II. SOLUTION METHODOLOGY

In this section, we first describe a case study setup for presentation of the solution overview and to define relevantmeasurement request terminology. Next, we explain ontology trees to capture user roles, user sampling preferences, and resourcepolicies. Following this, we present an overview of existing priority-based scheduling algorithms and mention how they differfrom our proposed SPS algorithm. Finally, we explain how online oversampling detection in measurement frameworks can bea factor to improve our SPS algorithm’s schedulability, i.e., ability to satisfy a greater number of user measurement requests.

A. Case Study SetupFigure 2 shows a case study setup of a measurement federation comprised of example measurement requests spanning two

autonomous domains. We consider a scenario where three intra-domain and three external federation users submit measurementrequests to their respective domain policy authority systems. A domain policy authority system is comprised of (i) a userdatabase to capture user information, (ii) a resource policy database to store policy settings and rules, and (iii) a web-serverthat allows requesting and querying of measurements via web-services. We assume that the authentication of users to initiatemeasurements, query performance results, as well as for the domains to exchange measurement topologies are performedthrough a federated authentication system. Authenticated measurement requests are sent to the respective domain’s centralizedschedulers, which generate the measurement schedules based on their knowledge of each other’s measurement topology.

B. Solution OverviewFigure 3 shows our proposed ontology-based semantic priority scheduling scheme that is part of a central scheduler. The

priority calculator receives measurement requests, i.e., tasks to be scheduled as explained in Section II-C from end-users anduses the inference gained from ontology trees and an oversampling detector to dynamically calculate the priority of eachtask. To capture the relative importance of the tasks, we propose “offline” and “online” priority calculations. The offlinepriority calculation is based on the user and policy ontologies described in Section II-D. As shown in Figure 3, “user rolebased priority”, “resource policy based priority”, and “sampling preference based priority” account for offline prioritization.The scheduler module takes these offline prioritized measurement requests and generates a schedule table that is sent to themeasurement framework, which in turn initiates tools to sample performance data. Online prioritization is influenced by theoversampling detector that analyzes collected network measurements and identifies scheduled measurement requests that aremeeting users’ monitoring objectives but are oversampling as explained in Section II-F. These requests are penalized with an“oversampling based priority” that is less than their offline calculated priority. A new task priority is calculated as the weighteddifference between offline and online priority calculations. Subsequently, the schedule is altered by the scheduler based on thenewly computed priority and the schedule output is sent to the measurement framework and the process repeats. We remark

4

Fig. 3. Ontology-based Semantic Priority Scheduling Scheme

Fig. 4. Example Task Set and Conflict Graph

that the context of semantics in measurement requests is leveraged only in the Priority Calculator module in Figure 3 forprioritization purposes and not in the actual scheduling of the prioritized measurement requests.

C. Measurement TasksAs shown in Figure 2, the network paths to be measured are specified by a measurement topology, which can be formally

represented by a graph G = (N,E), where N is the set of measurement servers and E is the set of edges between a pair ofmeasurement servers as shown in Figure 4. On top of the measurement topology, a set of on-going measurement requests isspecified. Using real-time systems terminology, an on-going measurement request can be treated as an “offline task” τi. The τispecification contains the source server srci, destination server dsti, active measurement tool tooli, and the inter-sampling timebetween the consecutive measurement instances based on the desired sampling pattern shown in Figure 1, and the executiontime ei of a single measurement instance. Thus, a task τi with a fixed inter-sampling time or period (pi) can be expressed asfollows:

τi = (srci, dsti, tooli, pi, ei).

For non-periodic tasks such as τ2, τ3 and τ4 shown in Figure 4(b), the inter-sampling time pi follows a function f(s)corresponding to their sampling pattern. We define the deadline of a task di,j in terms of the tasks inter-sampling timepi and stretch s. Stretch is formally defined as the time a task spends in a system, normalized by its processing time [36].For periodic tasks, the deadline is equal to the inter-sampling time pi × s. In the case of random sampling, the frequencyof sampling is not as strict as periodic and thus, we can assume a deadline for random tasks that is more relaxed. However,there should not be major gaps in inter-sampling times that might affect performance estimation analysis. As a rule-of-thumb,

5

Fig. 5. User Ontology Tree

we consider the deadline for random tasks to be equal to twice the inter-sampling time pi × s. We denote the j-th instance(or job) of τi as τij . Further, the time when the j-th job τij is released to be scheduled is called the release time rij andsimply given by (j − 1)pi. The tool conflict matrix Figure 4(c) will have an entry of 1 if any tool is empirically found tobe a CPU or channel intensive, and if it causes measurement conflict with another tool that has similar resource consumptioncharacteristics. The task conflict graph Figure 4(d) contains edges between conflicting tasks that have a common measurementserver or measurement path. In general, the task conflict graph can be stored as a matrix with entries of 1 to indicate the edges.However, if there are a large number of requests, the task conflict matrix results in a sparse matrix that consumes space andadds delays in lookup during scheduling. Consequently, we can maintain a hash table with task as key and conflicting tasks asvalue - to efficiently store and lookup a task conflict graph. We define hyperperiod for task set ζ as the least common multipleof all the task periods in the set. The schedule for all the input tasks is generated over a hyperperiod duration.

D. Ontology TreesOntology is a form of knowledge representation that is being increasingly used to capture (in a flexible and scalable manner),

any formal descriptions of (a) “concepts” in a domain of discourse, (b) “properties” of each concept describing various featuresand attributes of the concept, and (c) “role restrictions” on properties. Ontologies are represented by trees whose nodes areconcepts and whose arcs connecting the nodes represent the relationships/associations among these concepts [23]. In thispaper, we use ontology trees to store the semantic definitions of measurement requests, and employ the Perl Inference Engine(PIE) [24], an expert system to process the definitions in order to dynamically prioritize measurement requests.

Ontology trees allow consolidation of multi-domain policies and enable conflict-resolution, sharing and interoperability withinmeasurement federations. More specifically, they allow for creation of standard information models that help in knowledgerepresentation of different domain policies for underlying web services of network management systems. Coupled with thedefinition of rule bases for mapping using Semantic Web Rule Language (SWRL) [26] that are domain independent, i.e., rulescan be used to logically reason simultaneously by mapping ontology trees from different domains. They allow semantic queriesabout capabilities or configurations such as a measurement task priority calculation relevant in our work. SWRL captures rulesas an implication between an antecedent (body) and consequent (head) whose meaning can be read as “whenever conditionsspecified in the antecedent (body) are true, then conditions specified in consequent (head) must also hold true”.

We categorize semantic definitions in a measurement federation involving users from multiple measurement domains usingtwo ontologies: (i) user ontology, and (ii) resource policy ontology. We develop the user and resource policy ontologies usingthe open-source protege-OWL [25]. The protege-OWL editor supports Web Ontology Language (OWL) and a knowledge-basedframework. OWL is one of the widely used ontology languages in the semantic web world to precisely describe the relationamong concepts. It is endorsed by the World Wide Web Consortium (W3C). In addition, we use SWRL to manage the rule-basein the inference engine based on the semantic definitions in the ontology trees.

Other recent works in the area of policy-based network management such as [27] - [29] have adopted OWL as the preferredsemantic language that can be used to translate high-level abstract policy specifications to a low-level concrete implementationthat is flexible, context-aware, and easily extensible. The authors in [30] provide a survey of several network managementresearch projects in areas such as security and monitoring, where ontology-based approaches (similar to our work) were applied.They also provide insights into the main advantages and drawbacks found in the implementation of the approaches. The workin [31] presents a domain-ontology driven multi-agent based scheme that uses an ontology-based approach, and the [32] workuses a similar approach to form a knowledge plane that can help troubleshoot alarms from multiple actors using a common

6

Fig. 6. Resource Policy Ontology Tree

semantic model and an associated rule base. The work in [33] solves a semantic interoperability problem using ontologymappings in the co-operative management of network router configurations. Finally, the authors in [34] discuss the benefits ofautomation of network management systems when ontology-based and policy-based approaches are integrated using commonmodels.

1) User Ontology: User ontology trees are used to capture detailed information of each user. The information includesthe user role within the internal domain and in the measurement federation. It also includes the user name and associatedauthentication credentials that are sent along with a user’s measurement request. A role is assigned to the users dependingupon their authorized activities. Based on our study of roles in perfSONAR-like measurement federations, we assume roleshave instances such as a regular user, power user, network operator, and external federation user. Note that a power user canbe, e.g., a data-intensive science researcher demanding robust end-to-end performance between his/her user site and a remotehigh-performance computing facility or a user who pays a premium to use the network. Federation, i.e., external-domain userscan similarly be a network operator or power user or regular user requesting inter-domain measurements within the federationas shown in the case study scenario in Figure 2. We assume that any one user can possibly have multiple roles. The userexpresses sampling pattern preferences and their associated monitoring objectives that are actively recorded in this tree. Figure 5shows a user ontology tree that has the User class as a super class, and four disjoint sub-classes, viz., user role, samplingpattern preference, name and authentication credentials. The super class and sub-classes are linked by properties viz., hasRole,hasPreference, hasName, and hasCredentials.

2) Resource Policy Ontology: Resource policy ontology trees are used to capture detailed information of the internaldomain’s resource policies, as well as the common resource policies in the federation for mutual co-operation. In the internaldomain’s policies, information instances correspond with policies to handle measurement requests when users are requestingmeasurements only between resources owned by the domain, i.e., intra-domain or between resources across multiple domains,i.e., inter-domain. In the case of the common resource policies in the federation, information instances correspond to user andresource policies of each of the domains. Figure 6 shows a resource policy ontology tree that has Policy class as a super class,and two disjoint sub-classes, viz., internal resource policy and federation resource policy. The super class and sub-classes arelinked by properties, viz., hasInternalPolicy and hasFederationPolicy.

E. Measurement Scheduling AlgorithmsMeasurement schedulers employ different algorithms in measurement frameworks for orchestrating network-wide measure-

ments. They generate schedule tables that determine the times when measurement jobs should start. At the start of measurementjobs, active measurement tools are invoked at corresponding measurement servers in a manner that there are no “measurementconflicts” or violations of any “measurement level agreements” (MLAs). The round-robin scheduling used in NLANR AMP [16],the resource-broker scheduling used in perfSONAR [17], and the token-passing scheduling used in NWS [6] employs a simpleapproach where measurement servers take turns such that only one tool executes at a time. Therefore, they cannot leveragethe concurrent execution of multiple measurement jobs along non-conflicting paths and hence, limit the schedulability ofthe measurement framework. In comparison, the scheduling algorithms in ActiveMon [9], as well as in [18] [19] and [20]use different earliest-deadline-first approaches that basically maintain task conflict graphs to resolve measurement conflicts,and enforce MLA constraints by limiting the amount of concurrent execution when generating measurement schedules. Aninteresting modification to the earliest-deadline-first approach is the ascending order of the sum of the clique number and

7

Fig. 7. Measurement Scheduling Algorithms Flowchart

degree of tasks (ACD) algorithm proposed in [19]. ACD scheduling, which is based on graph theory, orders the measurementjobs in ascending order of job color. The job color depends on the job’s deadline and conflict parameter. [19] also presents adescending order of a sub-vertices degree (DSD) scheme for comparison with the ACD scheme. The DSD is similar to theACD except that it orders the jobs in descending order. Further, in [7], a heuristic-bin-packing approach similarly maintainstask conflict graphs to resolve measurement conflicts and enforces MLA constraints.

Assuming a measurement job is characterized by its arrival or start time sti,j , finish time fti,j , deadline di,j , and prioritypri,j , authors in [22] differentiate scheduling algorithms in today’s literature based on the mechanism by which the next jobis selected for scheduling:

1) Earliest Deadline First (EDF) based on deadline di,j2) Smallest-residual Laxity First (SLF) based on laxity defined as equal to di,j - fti,j - sti,j3) Scheduling based on Priority and Deadline (PDS)

di,j + w * f(pri,j); where w is weight4) Packing Heuristics, e.g., heuristic-bin-packing (HBP) based on execution time ei,jFigure 7 shows the flowchart of how each of the above broadly defined scheduling algorithms can be used in a measurement

scheduler. When task set ζ and corresponding task conflict graph are input to the measurement scheduler, the first step involvesordering the jobs depending upon the scheduling priority. In the case of the EDF, the scheduling priority is based on theearliest deadline, and hence, the jobs are ordered by release times. In the case of the SLF, the scheduling priority is givento jobs with smaller residual laxity. In the case of the PDS, a priority due to urgency factors of the application, supercedesscheduling priority based on earliest deadline, and hence, the jobs are ordered by release time and priority. In the case ofHBP, the scheduling priority is given to jobs with the largest execution times that are needed by principles such as “first-fitdecreasing” [8]. Finally, in the case of the ACD and DSD, the scheduling priority is based on a deadline similar to the EDF.

While scheduling the jobs, all the algorithms use the “concurrent execution where possible (CE)” idea and check compliancewith any specified MLA constraints. Only the jobs that have been determined to meet the algorithm’s scheduling criteriaand MLA constraints are scheduled. The output measurement schedule has the start time stij and finish time ftij of eachmeasurement job τij in a hyperperiod. We remark that our proposed semantic priority scheduling (SPS) algorithm is basicallya variant of the PDS algorithm. The priorities of measurement jobs are determined by using semantic priorities based on userand resource policies. Also, we remark that our proposed SPS algorithm builds upon our preliminary work in [35], where wemotivated the need for a semantic priority scheduling in openly-accessible measurement frameworks.

F. Online Oversampling DetectionA limitation in using the semantic priority scheduling approach is that higher priority tasks may unfairly block relatively

lower priority tasks when available measurement resources are heavily utilized. The unfairness can be attributed to cases wherehigh priority tasks of users are meeting their monitoring objectives but are oversampling. If the oversampling of such tasks wasdetected online and mitigated by reducing the sampling frequency, few of the blocked lower priority tasks can be scheduled.Thus, a greater number of user measurement requests overall can be satisfied by the measurement scheduler without disruptingthe monitoring objectives of higher priority tasks.

We have found evidence of oversampling in perfSONAR traces in our experiments with 2 monitoring objectives: (a) networkweather forecasting, and (b) SLA compliance monitoring. For our experiments, we chose 8 exemplar perfSONAR traces thatwe queried (via web-services) from openly accessible measurement archives comprised of several months of measurement datasets along diverse network paths. Descriptions of these measurement traces are shown in Table I.

8

For the first monitoring objective, we use the “dynamic winning-predictor selection” based forecasting technique used inNWS [6] that is not computationally-intensive and is suitable for real-time measurement predictions. In this technique, forforecasting a future value, a variety of models are applied on historical data and a cumulative error measure is determined.The model that generates the lowest prediction mean square error (mse), i.e., the winner is used to forecast the future value.Figure 8 shows that in all the traces, nearly 15% of the time the last value predictor is the winning predictor. This oversamplingevidence in the traces indicates that the network status can still be accurately estimated by tuning the sampling frequncy toavoid redundant samples, especially at high measurement request loads. For the second monitoring objective, we use the “best-fit sampling pattern” - based SLA compliance monitoring technique used in [11]. In this technique, for determining the bestsampling strategy for ongoing SLA validation, the absolute error values of all historical data points are plotted for a variety ofsampling patterns, i.e., periodic, random, stratified random. The sampling pattern that has the highest percentage of absoluteerrors (pae) < 0.01 is used for SLA compliance monitoring. Figure 9 shows the oversampling evidence in, e.g., trace 6, forwhich we apriori identified stratified random sampling as the “best-fit”. We can see that we achieve successful SLA compliancemonitoring with an 80% of pae < 0.01 with 1

10

th reduced sampling rate. Moreover, it can be noted from table II that not onlyin trace 6, but in all the traces using stratified random sampling, nearly 80% of the estimation errors (pae) lie within the givenboundaries, i.e., pae < 0.01.

We leverage the above two observations in our SPS algorithm to detect oversampling and develop a mitigation scheme inSection III-A2 that reduces the sampling frequency of oversampling tasks but keeps the monitoring error of these tasks withintolerable thresholds. Also, we can observe that the oversampling detection algorithms are based on online estimation algorithmsand simple heuristics based on expert knowledge of estimation techniques. Hence, they are not computationally intensive andare suitable for use in our oversampling mitigation scheme.

TABLE ITRACES DESCRIPTION

TraceID

Source ↔ Destination Time Range (Start -End)

1 psmsu02.aglt2.org ↔psum02.aglt2.org

2009-10-9 15:03:19 -2010-4-7 17:28:05

2 bwctl.ucsc.edu ↔bwctl.atla.net.internet2.edu

2010-1-16 06:51:22 -2010-4-7 20:36:05

3 bwctl.ucsc.edu ↔bwctl.wash.net.internet2.edu

2010-1-16 08:50:36 -2010-4-7 20:37:43

4 wtg248.otctest.psu.edu ↔perfsonar.dragon.maxgigapop.net

2010-2-8 14:08:31 -2010-4-7 21:25:57

5 chic-pt1.es.net ↔anl-pt1.es.net

2009-7-2 20:04:41 -2010-1-9 12:32:48

6 nersc-pt1.es.net ↔wash-pt1.es.net

2009-5-18 22:48:13 -2010-1-9 16:46:47

7 hous-pt1.es.net ↔pnwg-pt1.es.net

2009-5-19 04:05:12 -2010-4-7 13:39:31

8 nettest.boulder.noaa.gov ↔wtg248.otctest.psu.edu

2009-10-6 20:41:22 -2010-4-7 21:27:05

TABLE IITRACES - P VALUE CHART

Trace -1 Trace -2 Trace -3 Trace -4Percentage of Error pae < 0.01 82.3% 83.9 % 82.1 % 81.8 %

Trace -5 Trace -6 Trace -7 Trace -883.2 % 82.8% 83.5% 83.4%

III. SCHEDULING TASKS WITH SEMANTIC PRIORITY

In this section, we first present details of the priority calculator module of the ontology-based SPS scheme shown in Figure 3.Then we present the semantic priority scheduling algorithm to schedule measurement requests from the end-users.

A. Priority CalculatorAs we can see from Figure 3, the priority calculator receives measurement requests from the end-users, prioritizes the

measurement requests based on the semantics and sends the prioritized measurement input to the scheduler module. In orderto prioritize measurement requests, the priority calculator uses the inference lookup, i.e., rule base that is based on informationprovided in ontology trees to compute the priority of each measurement request and a subsequent oversampling detector toupdate the priority. We consider two kinds of priority calculations, namely, offline and online priority calculation. The prioritycalculator uses the inference lookup to perform offline priority calculations and the oversampling detector for online prioritycalculations. We consider the priority values to be normalized in 0-to-1 range.

9

Fig. 8. Oversampling evidence in perfSONAR data for network weather forecasting

0 200 400 600 800 1000 1200 14000

20

40

60

80

100

Number of Samples

pae

(%)

< 0

.01

Evidence of

Oversampling

Fig. 9. Oversampling evidence in perfSONAR data for SLA compliance monitoring

The user role based priority, resource policy-based priority, and sampling preference-based priority account for offlineprioritization. The user role based priority is calculated depending on the user requesting the measurement. The user roleis captured in the user ontology tree and the rule-based inference engine reads the ontology trees and generates a user rolepriority rule, which can be invoked by the priority calculator to generate the user role based priority. Similarly, the resourcepolicy based priority is calculated based on the resource policies captured in the resource policy ontology tree, and samplingpreference-based priority is calculated based on the sampling pattern preference set by the users in the user ontology tree.Offline priority calculation generates offline prioritized measurement requests that are sorted in decreasing priority order andare sent to the semantic scheduler. The semantic scheduler takes these offline prioritized tasks and generates a schedule tablethat is sent to the measurement framework to initiate measurement tools to collect the performance data. For online prioritycalculation, the priority calculator uses the oversampling detector to analyze the performance data collected by the measurementframework. It identifies the oversampled high priority tasks and penalizes them to provide a schedule slack for low semanticpriority tasks. Detailed explanation of the offline and online prioritization is provided in the subsequent subsections.

1) Offline Prioritization: In this subsection, we first present a general approach for offline priority calculation, and subse-quently present an illustration example that shows how the approach applies for the case study scenario in Figure 2.

For the general approach, we consider the offline priority (profflineij ) for a job ij to be a combination of three factors: (i)User role priority (URP) score, (ii) Resource policy (RP) score, and (iii) User sampling preference (USP) score. Given thatthere can be any number of user-specified instances for each of these three factors, we can treat each factor as a separatevector. The length of a factor vector represents its magnitude of influence on the overall profflineij score for a job. Hence, thecombined influence of the three factors can be analytically represented as an additive summation of their vectors as shownbelow in Equation 1.

−→profflineij =−→RP +

−−−→URP +

−−−→USP (1)

In terms of the RP factor that most likely will have two instances, i.e., intra-domain, inter-domain, the general rule in assigningscore levels (higher scores represent a higher priority) can be expected to be such that RPintra−domain > RPinter−domain, i.e.,internal measurement requests pertaining to intra-domain resources have a higher priority than external measurement requestspertaining to inter-domain resources. In the same line of tendency, the RP score can also be expected to have a higher influencethan the URP or USP scores. To accommodate such a user preference, it is desirable to have the scale of the RP be greaterthan the individual scales of the URP and USP. In addition, the scores for instances within the USP and URP can vary, andthe influence of the priority values selected with the USP and URP can be normalized to have values in the 0-to-1 range,and represented as USP* and URP*, respectively. Following this, the overall normalized offline priority proffline∗ij for a job

10

can be analytically represented as an additive summation of its RP, USP*, URP*, or equivalently as a ratio shown below inEquation 2, where profflinehighest corresponds to the highest priority job.

−→proffline∗ij =−→profflineij

−→profflinehighest

(2)

As an illustration example of offline priority calculation, we consider three rules, namely, a rule for calculating the RP score,a rule for calculating the URP score, and a rule for calculating the USP score. Figure 10 shows the example of the SWRL rulesthat we consider for calculating the offline priority. We remark that an intra-domain or federation administrator can modify therules and score ranges to suit measurement needs of the users within the domain or as agreed upon in the Measurement LevelAgreements in the measurement federation. The scale for the scores in our illustration example are based on typical practiceassumptions followed in the perfSONAR community in context of intra-domain and inter-domain measurement requests.

The resource policy based priority is calculated based on the resource policies captured in the resource policy ontologytree. As shown in Figure 10(a), the rule base inference engine checks the internal domain resource policy set by the networkadministrator in the antecedent (body). To show that priority levels or scores can be assigned based on tendencies in userrequirements, we have set the RP score on a scale of 0-to-2 to provide more preference to the RP score compared to the URPand USP scores, which are on a scale of 0-to-1; e.g., if the internal domain resource policy is set as “intra-domain”, then“User1”, who initiates an intra-domain measurement request, gets a resource policy score of 2. Similarly, if the internal domainresource policy is set as “inter-domain”, then “User1”, who initiates a intra-domain measurement request, gets a resource policyscore of 0.

In computing the user role based priority, we consider three user priority scores namely 1, 2, and 3, with 3 representingthe highest priority score as an illustration example. Network operators who control the network traffic by obtaining detailednetwork status information are more important than a power user or a regular user. Since the network operators control thenetwork, if the network operator’s measurement requests are not scheduled, then it may eventually lead to failure of the entirenetwork. Hence, we assign network operators with a user priority score 3, followed by power users with score 2, and regularusers with score 1. The rule base inference engine, checks the user role in the antecedent (body) as shown in Figure 10(b);e.g., if the user role is “PowerUser” in the antecedent, then in the consequent (head), we assign a maximum user role priorityscore of 3 and a priority score corresponding to the power user of 2. So the normalized value of the user role priority scorecorresponding to “User1” is 0.667, i.e., 2 / 3.

Similarly, the user sampling priority score is computed based on the sampling pattern preference set by the users in theuser ontology tree based on their monitoring objective; e.g., if the user’s monitoring objective is network forecasting, they canset periodic sampling as their sampling preference by assigning a high sampling preference score to periodic sampling. Recallthat periodic sampling is required to produce accurate network weather forecasts [6] [9]. Users can choose their samplingpattern preference from: periodic, stratified random, adaptive, and random. Since we have assumed 4 sampling patterns in thispaper, we consider the maximum score for sampling pattern preference of 4. The rule base inference engine, checks the usersampling pattern preference in the antecedent (body) as shown in Figure 10(c); e.g., if the user sampling pattern preference is“Periodic” in the antecedent, then in the consequent (head) we assign a maximum user role priority score as 4 and a priorityscore corresponding to the periodic as 4. So the normalized value of the user sampling priority score corresponding to “User1”is 1, i.e., 4 / 4.

To illustrate offline priority profflineij calculation, we present the following example. Let us assume that resource policyontology has captured the internal domain resource policy as “intra-domain”. We also assume six users, namely, a networkoperator, power user, regular user, external federation network operator, external federation power user, and external federation(EF) regular user submit measurement requests (see Table III based on the case study scenario in Figure 2). For simplicity,let us assume that all users have set their sampling pattern preference score as follows: a periodic sampling of tasks gets thehighest score of 4, a stratified random sampling gets a score of 3, an adaptive sampling gets a score of 2, followed by arandom sampling that gets a score of 1. It is typical for network operators to want periodically sampled measurements to havea higher priority than other sampling for monitoring objectives relating to a continuous routine checking of the network status.Regardless, the score selection is configurable to suit measurement needs of the users within the domain or as agreed upon inthe Measurement Level Agreements in the measurement federation. Let us also assume that each user submits two requests.Then, the offline priority score computed for the example task set is shown in Table IV. We can see that the resource policyscore gets a higher preference since its on scale 0-to-2. Hence, if the resource policy is set as intra-domain, the inter-domaintasks have a very low offline priority score compared to the intra-domain tasks. In Table IV, RP denotes Resource PolicyScore, URP* indicates User Role Priority Score (Normalized), USP* indicates User Sampling Preference Score (Normalized)and profflineij * indicates Offline Priority Score (Normalized).

We remark that it might be possible to have scenarios where the score selections assumed in the above illustration examplemay need to change adaptively based on dynamism in the context of monitoring objectives of incoming measurement requests.Developing related adaptation schemes is beyond the scope of this paper and is part of future work.

2) Online Prioritization: A limitation with any priority based scheduling algorithm is that higher priority tasks may unfairlyblock relatively lower priority tasks when the available resources are heavily utilized. The unfairness is predominant when: (i)low priority tasks which are blocked by high priority tasks, are not meeting their monitoring objective and (ii) high priority

11

(a) Resource Policy Rule-base (b) User Role Priority Rule-base (c) User Sampling Preference Rule-base

Fig. 10. Example Rule-bases in SWRLs

TABLE IIIEXAMPLE MEASUREMENT REQUESTS FOR PRIORITY CALCULATION

Task Domain User Sampling Patternτ1 intra-domain network operator periodicτ2 intra-domain power user adaptive periodicτ3 intra-domain regular user randomτ4 inter-domain EF network operator randomτ5 inter-domain EF network operator periodicτ6 inter-domain EF power user periodicτ7 intra-domain network operator adaptive periodicτ8 intra-domain power user stratified randomτ9 intra-domain regular user randomτ10 inter-domain EF power user randomτ11 inter-domain EF regular user randomτ12 inter-domain EF regular user adaptive periodic

tasks, in turn, are oversampling while meeting their monitoring objectives. In order to overcome such a situation, we use theDETECT OV ERSAMPLING FORECAST or DETECT OV ERSAMPLING SLA−MON subroutines (shown in AlgorithmsIII.1 and III.2) to (a) identify the measurement requests that are found to be oversampling in terms of their associated monitoringobjective; and (b) penalize them by reducing their semantic priority, thus allowing few of the blocked low priority tasks to bescheduled.

Online prioritization calculates the final priority prfinalij as a weighted difference between the offline priority profflineij scoreand the oversampling penalty score. The weight (wΩ) signifies the amount of the oversampling penalty, and its value is set to1 for newly arriving jobs of detected oversampling tasks, and 0 for all other jobs. The final priority prfinalij value is equal tothe offline priority profflineij and depends on the oversampling penalty score as shown below in Equation 3.

prfinalij = profflineij - wΩ * Ω (3)

The method for identifying an oversampling task varies depending on the monitoring objective. If the monitoring objective isforecasting, then in the DETECT OV ERSAMPLING FORECAST subroutine, shown in Algorithm III.1, we check the mse orprediction error that indicates how much the actual performance data of a task τ1 collected by the measurement infrastructuredeviates from the predicted performance data calculated by the “winning predictor”. The oversampling penalty score associated

TABLE IVOFFLINE PRIORITY CALCULATION - EXAMPLE

Task RP URP * USP * profflineij *

τ1 2 1 1 1τ2 2 0.66 0.5 0.79τ3 2 0.33 0.25 0.645τ4 0 1 0.25 0.3125τ5 0 1 1 0.5τ6 0 0.66 1 0.415τ7 2 1 0.5 0.875τ8 2 0.66 0.75 0.8525τ9 2 0.33 0.25 0.645τ10 0 0.66 0.25 0.2275τ11 0 0.33 0.25 0.145τ12 0 0.33 0.5 0.2075

12

with τ1 depends on the mse error value and is calculated as shown in Equation 4.

Ω ∝ 1mse

(4)

Algorithm III.1: DETECT OVERSAMPLING FORECAST( )

Input: Schedule time table STT (ζ), History data H(ζ)

Output:prfinalij of each scheduled task

1.for each scheduled task τij

do

2. Generate prediction for each scheduled task using NWS models3. Compute cumulative mse for each prediction method4. Choose the prediction method with the lowest prediction error as the winning predictor5. Use the winning predictor to predict the next measurement performance value6. Compute oversampling score based on mse7. Compute prfinal

ij of τij

If the deviation or mse is considerably less, then it is an indication that even though some jobs of task τ1 did not getscheduled, their performance values can be predicted and hence their monitoring objectives are not affected to a large extent.We associate a high oversampling penalty to newly arriving jobs of task τ1 and hence, the final semantic priority (shown inEquation 3) for the next job τ12 of task τ1 released at the current schedule time is lowered. We again check for mse whenτ12 and subsequent jobs are released and associate an oversampling penalty based on the most current mse error value.

Algorithm III.2: DETECT OVERSAMPLING SLA-MON( )

Input: Schedule time table STT (ζ),History data H(ζ)

Output:prfinalij of each scheduled task

1.for each scheduled task τij

do

2.Generate a sampled measurement trace using sampling patterns i.e., periodic, random, stratified random patterns3. Compute percentage of absolute errors pae < 0.01 for each sampling pattern4. Choose the sampling pattern with the highest pae < 0.015. Use the best-fit sampling pattern to sample the original measurement performance value6. Compute the oversampling score based on pae7. Compute prfinal

ij of τij

Algorithm III.3: JOBS PRIORITIZER( )

Input: Jobs in pending − job− queue pqOutput: Sorted prioritized schedule-list slInitialize Ω of job to 01. Set the resource policy preference2. Set the user preference3. Compute Offline priority proffline

ij of each job τij4. Sort the jobs in sl by descending order of prfinal

ij such that relative order of input jobs is maintained

If the monitoring objective is SLA validation related, then in the DETECT OV ERSAMPLING SLA −MON subroutineshown in Algorithm III.2, we use the best-fit sampling pattern technique for determining the best sampling strategy for theongoing SLA validation. The best-fit sampling pattern is the pattern that has the highest percentage of absolute errors (pae)< 0.01. Then we check the absolute error between the actual measurement value and the sampled measurement value, i.e.,measurement value collected by best-fit sampling pattern of the actual measurement traces and compute the oversamplingpenalty score Ω for SLA monitoring as shown in Equation 5.

Ω ∝ 1pae

(5)

Similar to the above example, if the absolute error value of a task τ1 is less, then we associate a high oversampling penalty tothe next job τ12 and subsequent jobs to be released. This results in the final semantic priority of job τ12 and other penalizedjobs being lowered and hence, their probability of getting scheduled before their deadlines is decreased. We again check forpae when τ12 and subsequent jobs are released and associate an oversampling penalty based on the most current pae errorvalue.

Once the final priority is computed, the jobs are ordered by decreasing the final priority by the JOBS PRIORITIZER sub-routine. In case of a tie or rule conflict between the tasks, the sorting algorithm JOBS PRIORITIZER shown in Algorithm

13

III.3 prioritizes the tasks by considering their user role priority; e.g., if two tasks have the same priority, in order to determinetheir position in the sorted list, the sorting algorithm checks their user role priority score. The task with the higher role prioritygets the higher semantic priority. If the user role priority of the tasks is also similar, then we randomly select one task to havea higher semantic priority over the other. The sorted prioritized tasks are then sent to the measurement scheduler.

B. SPS AlgorithmThe semantic scheduler shown in Figure 3 tries to find a schedule of all measurement jobs so that (a) monitoring objectives

of the tasks are met, (b) measurement conflicts are prevented, (c) concurrent execution of measurement requests occurs whenpossible while MLA constraint is also being adhered, and (d) different sampling patterns are accommodated. In addition, theSPS algorithm tries to satisfy two additional properties namely: (i) deadline, and (ii) stretch.

The first step in schedule construction is to make the task conflict graph by combining the measurement topology G and thetask set ζ as shown in Figure 4(d). For the given task set ζ and task conflict graph, SPS algorithm tries to find the measurementschedule during a hyperperiod as shown in Algorithm III.4.

The SPS algorithm maintains an ordered list of release times rt− list and an ordered list of finish times ft− list. Line 1initializes rt− list with all release times in a hyperperiod. Line 2 initializes ft− list as empty since no job is yet scheduled.We also maintain a pending − job − queue that holds all jobs that are released but not scheduled. We initialize it as emptyin Line 3. We initialize the initial stretch to 1 (minimum value) in line 4. Since, the stretch of a task is associated with itsdeadline calculation, stretch cannot be relaxed for higher semantic priority tasks. However, the stretch can be relaxed for lowsemantic priority tasks if they are not scheduled before their deadline. Relaxing the stretch is performed by increasing thestretch by 0.1. Hence, higher semantic priority tasks have minimum stretch compared to lower priority tasks.

The repeat-until loop from lines 5 to 17 progresses the virtual time variable time till hyperperiod and also determines theschedule at each schedule time point ts. Note that the only time points (when we need to make a scheduling decision) areeither when a new job is released or when a current executing job is finished. Line 5 calculates the minimum of the times inrt− list and ft− list, which is the next schedule time point and moves ts to the next schedule time value. Line 6 adds allnewly released jobs into the pending−job−queue and we call the JOBS PRIORITIZER subroutine to prioritize the releasedtasks in Line 7. This subroutine computes prfinalij based on the inference from the knowledge gained from the ontology trees,and sorts the jobs. The foreach loop from lines 8 to 17 examines the sorted jobs in the schedule− list to check whether theycan start at time without conflicting with the already scheduled jobs and without violating the MLA constraint.

Algorithm III.4: SPS( )

Input: ζ, task conflict graphOutput: Schedule timetables for servers1.Initialize rt− list with the ordered list of all release times in a hyperperiod2.Initialize ft− list= 3.Initialize pq= 4.Initialize stretch s = 1looprepeat

5.ts = min(next ts from rt− list, ftij from ft− list)6.add all newly released job in pq7.sl = JOBS PRIORITIZER(pq)

8.for each τij in sl

do

9.if τij does not conflict with scheduled jobs and scheduling τij does not violate MLA

then

10.stij = ts11.ftij = ts + ei12.if di,j later than ftij

then

13. remove τij from sl14. add ftij to ft− list in order

else

15. Relax s /*if it is feasible*/16. discard τij /*if it is not feasible*/

else

17.do nothing/*τij will be considered again at the next ts*/

until ts = hyperperiod18.if satisfaction ratio ≤ 0.5 and monitoring objective eq FORECAST

then 19.DETECT OVERSAMPLING FORECAST(STT(ζ),H(ζ))else if satisfaction ratio ≤ 0.5 and monitoring objective eq SLA-MONthen 20.DETECT OVERSAMPLING SLA-MON(STT(ζ),H(ζ))

end loop

If the jobs can start at time, job τij ’s start time stij is determined as time and its finish time ftij is determined as time+eiin lines 10 and 11. Next we check the deadline dij of the job in Line 12. If the deadline dij is later than the finish time of

14

the job ftij , we remove the job τij from the scheduling − list and move it to the finish list ft − list in Lines 13 and 14.This way, ftij can be considered as a new scheduling time point ts in the outer repeat-until loop. However, if the deadlinedij of the job is earlier than the finish time ftij , then the algorithm checks if it is feasible to relax the stretch. If feasible, thenew deadline is computed, and the task will be considered again at the next scheduling time point ts. If it is not feasible torelax the stretch s, then it implies that the task cannot be scheduled before its deadline and hence, the algorithm discards thejob - resulting in an infeasible task set. If job τij cannot be scheduled at ts due to a conflict with an already scheduled job,then it is kept in the scheduling − list and is considered again in the next scheduling time point in the outer repeat-until theloop in Line 17.

Once we find the output schedule, we can convert it into a measurement schedule table for each server, considering thesource server of each job. These schedule tables are transferred to corresponding servers in the measurement framework so thatthey can start and stop the measurement jobs. Once the schedule table is generated, in Line 18, we check for the satisfactionratio and the monitoring objective. Based on the monitoring objective, we call the DETECT OV ERSAMPLING FORECAST

or the DETECT OV ERSAMPLING SLA − MON subroutine only if a majority, i.e., more than 50% of the users are notsatisfied. This ensures that we penalize the high priority tasks only when: (i) the measurement resources are over-utilized, and(ii) more than 50% of the users’ tasks are not meeting their monitoring objectives. The outer loop then orders the jobs in thepending − job− queue (in a non-increasing order) to correspond to their final priority prfinalij , and the cycle repeats.

IV. PERFORMANCE EVALUATION

In this section, we investigate the performance of our proposed SPS algorithm with both synthetic tasks and with the realperfSONAR data traces shown in Table I. We first describe the simulation setup in terms of the task set generation, schemesbeing compared, and performance evaluation metrics. Following this, we present both the offline and online performanceevaluation results of our proposed SPS algorithm.

A. Simulation Setup1) Task Set Generation: Our synthetic task set is similar to the example task set shown in Figure 4. We assume that the

task set is spread across a fairly large number of servers in order to check the scalability of our proposed SPS algorithm.Given the fact that there are several hundred measurement servers that are active in perfSONAR deployments at any givenpoint of time, we choose a range of 1 to 2500 servers in our simulations. The period pi and execution time ei of each taskτi is randomly selected from [1000 sec, 10000 sec] and [100 sec, 999 sec], respectively. We decided upon the periodicity andexecution time range of the tools mentioned above based on the respective considerations in widely-deployed frameworks suchas perfSONAR, NLANR AMP, ActiveMon, and the NWS mentioned in Section I.

The type of sampling pattern associated with each task input is chosen randomly from the set of sampling patterns comprisedof periodic, stratified random, random, and adaptive patterns. For generating measurement input with periodic sampling, wegenerate jobs at a constant time interval, i.e., period, while extending jobs till the hyperperiod. For random sampling, wegenerate jobs randomly using a common, e.g., poisson, statistical distribution. To generate measurement input with adaptivesampling, we use a heuristic to adjust the sampling rate, i.e., we increase the sampling frequency (generate more jobs whileextending the jobs till hyperperiod) in the initial cycles and then decrease the sampling frequency in the later cycles. Toperform stratified random sampling, we divide the jobs into subsets according to a given characteristic called “strata” and thenrandomly select jobs from each strata.

The task conflict graph is randomly created using a parameter called “conflict factor” as described in [9]. The conflict factorrepresents the probability that there is a conflict edge between any two tasks. When the conflict factor is 1, the task conflictgraph is fully connected; if the conflict factor is 0, there is no edge between tasks.

2) Schemes for Comparison: To study the performance of our proposed SPS algorithm, we compare it with other existingscheduling algorithms described earlier in Section II-E:

• Earliest Deadline First (EDF) - This concurrently schedules the measurement jobs (when possible) using the EDFalgorithm.

• Heuristic Bin Packing (HBP) - This concurrently schedules the measurement jobs (when possible) using the heuristicprinciple based on largest execution time.

• Descending order of sub-vertices degree (DSD) - This schedules the measurement jobs concurrently when possible byordering them in descending order of the degree of the sub-vertices mapped to the jobs.

• Ascending order of the sum of the clique number and degree of tasks (ACD) - This concurrently schedules themeasurement jobs (when possible) using a coloring scheme by ordering the tasks in ascending order of the sum of cliquenumber and conflict task degree in a conflict graph.

• Round Robin (RR) - This sequentially schedules the jobs without any concurrent execution of jobs (when possible) in thetime frame and without considering the priority associated with the jobs. Recall from Section 1 that existing measurementframeworks such as NLANR AMP, perfSONAR, and NWS use variants of RR; hence, it is relevant to include RR in theperformance comparisons.

15

0 200 400 600 800 1000 12000

500

1000

1500

2000

Number of Tasks

Cy

cle

Tim

e (M

inu

tes)

MLA=2

MLA=4

MLA=8

Fig. 11. Cycle time trends for increasing MLA constraint values and number of tasks

3) Metrics: As defined in [7], Cycle Time is the duration over which a complete round of all scheduled jobs are executedi.e., it is the time within which one unique set of all measurement jobs are scheduled to complete between all the measurementservers. Cycle time is an important metric that describes the efficiency of the scheduling algorithm. The lower the cycle timeis, the higher the frequency, with which we can obtain network status information that aids in better understanding networkperformance characteristics. Hence, the lower the cycle time, the higher the efficiency of the scheduling algorithm.

Satisfaction Ratio is a novel metric that we developed in this paper. It is the ratio of satisfied users to the total numberof users requesting measurements. We remark that satisfied users are those whose monitoring objectives are successfully metby a measurement scheduling algorithm, assuming that (a) each task is input by a unique user, and (b) all the jobs of atask are scheduled prior to their deadlines. Each user has different semantic priority and also, in turn, has prioritized theirpreferences. According to the priority of each user and priority of their preferences i.e., the corresponding monitoring objective,a measurement scheduler computes the satisfied number of preferences for each user. The satisfaction ratio calculation can bedone as shown in Equation 6.

Satisfaction ratio of users =

n∑i=1

m∑j=1

((prij)2)(wij) (6)

wij=Weight of the user preference, prij = Semantic priority, n = Total number of users, m = Total number of satisfiedpreferences. We have squared the semantic priority while calculating the satisfaction ratio in order to reward a scheme thatschedules high priority tasks. Hence, the higher the satisfaction ratio is, the higher is the chance that high priority tasks arescheduled.

Average Stretch is defined as the average of stretch of all jobs in a task set. Stretch of a job in a system is the time a jobspends in the system normalized by its execution time, as defined earlier in Section II-C. Average stretch calculation can bedone as shown in Equation 7.

Average stretch = avg(∑

(ftij - rij

ei)) (7)

Average stretch is similar to the average turnaround time that the scheduling metric used in [19], which uses the differencebetween finish time and release time of jobs. However, average stretch also considers ‘execution time’ of the jobs whilecalculating the stretch in order to ensure fairness to all jobs in the task set. Minimizing the stretch results in an efficientschedule of the tasks. We consider the average stretch to be normalized.

We remark that for each task set and corresponding task conflict graph, we average the results for each of the above metricsby running the simulation over 500 iterations to obtain the data points shown in the evaluation graphs below.

B. Offline Prioritization Results1) Impact of MLA: Figure 11 shows the effect of MLAs on cycle time trends for an increasing number of input tasks. We

can see that as the MLA constraint value (given by the maximum amount of concurrent execution jobs permitted network-wide)increases, it allows for a larger amount of concurrent job executions, and hence, the cycle time decreases. We observe that theMLA constraint is a bottleneck in generating a highly efficient semantic schedule. We can also see that - as the number oftasks increases, the cycle time also increases correspondingly. In the graphs shown in the subsequent subsections, we assumeMLA = 4, i.e., ≈10 Mbps bandwidth consumption is permitted for measurements unless otherwise mentioned. Such an MLAassumption in the analysis shown in subsequent subsections avoids any MLA bottlenecks that might arise while finding efficientsemantic schedules for different input task sets.

16

0 500 1000 1500 2000 25000

0.2

0.4

0.6

0.8

1

1.2

1.4

Number of Tasks

Sati

sfa

cti

on

Rati

o

SPS

EDF

ACD

HBP

DSD

RR

Fig. 12. Satisfaction Ratio comparison for increasing number of tasks

0 0.2 0.4 0.6 0.8 10.4

0.5

0.6

0.7

0.8

0.9

1

Conflict Factor

Sati

sfa

cti

on

Rati

o

SPS

EDF

ACD

HBP

DSD

Fig. 13. Satisfaction Ratio comparison for increasing conflict factor

2) Satisfaction Ratio Comparison: We compare the satisfaction ratio provided by other schemes described in Section IV-A2,with our proposed SPS scheme with these two parameters: (i) an increasing number of tasks and (ii) an increasing probabilityof conflict among jobs. Figure 12 shows the satisfaction ratio comparison for the different schemes with an increasing numberof tasks. We can see that our proposed SPS scheme has the highest satisfaction ratio. The reason for a high satisfaction ratio ofthe SPS scheme, compared to other schemes, is that it orders the tasks based on the decreasing order of the semantic priority,which results in scheduling of higher priority tasks first. Hence, the monitoring objective of the high priority tasks is metresulting in a higher satisfaction ratio. The EDF and ACD schemes almost provide a similar satisfaction ratio since both theschemes order the jobs based on the deadline of the jobs. The EDF and ACD schemes do not consider the semantic prioritiesassociated with the task while generating the schedule. Hence, the EDF and ACD schemes result in a lower satisfaction ratiocompared to SPS scheme. The HBP and DSD schemes provide lower satisfaction than the EDF and ACD schemes becausethey select jobs with a long execution time first, and hence, the remaining tasks may miss their deadline resulting in a lowersatisfaction ratio. The low satisfaction ratio of RR is expected as it schedules jobs without concurrent execution, and hence alarge number of jobs have to wait for extended time durations, resulting in deadline misses and a lower satisfaction ratio.

Next, Figure 13 shows the satisfaction ratio comparison of different schemes with an increasing conflict factor. We cansee that as the conflict factor increases, the satisfaction ratio of the users decreases. In low conflict factor cases, all thescheduling algorithms behave similarly and deliver satisfaction ratio levels close to 1. However, as the conflict factor increases,the satisfaction ratio of the users is again higher with our proposed SPS scheme compared to others. We can also see that theEDF and ACD provide higher satisfaction than the HBP and DSD, similar to the previous case.

3) Average Stretch Comparison: We perform an average stretch comparison by first considering a task set comprised of allpriority jobs. Following this, we compare with a task set comprised of only high priority jobs and point out the difference inthe performance with our proposed SPS algorithm in the context of high priority jobs. Figure 14(a) shows the average stretchfor the different schemes with an increasing number of tasks comprised of jobs featuring all priority levels. We can see thatthe SPS, EDF, and ACD schemes have a similar stretch. We can also see that the SPS, EDF, and ACD schemes have lowerstretch compared to HBP and DSD, and RR has the highest stretch. The reason for HBP and DSD schemes to have higheraverage stretch compared to SPS, EDF and ACD schemes is that they select jobs with long execution times first, and hence,the smaller execution time tasks suffer large delays. The higher average normalized stretch of RR is expected as it schedulesjobs without concurrent execution. As a result, a large number of jobs have to wait for long durations before getting scheduledand thus, yield the highest average stretch. In contrast, as shown in Figure 14(b), the average stretch of our proposed SPSscheme is much lower than all the other schemes when particularly compared with high priority tasks.

C. Online Prioritization ResultsLeveraging the evidence of oversampling in perfSONAR data traces shown in Table I, we performed experiments on these

traces to show how our SPS algorithm can online mitigate the oversampling to further improve the satisfaction ratio of usersunder high measurement request loads, i.e., how we serve low priority measurement requests that were offline blocked due tooversampling of higher priority measurement requests. We compare both prioritization and scheduling performance of offlineSPS and online SPS using a satisfaction gain metric. Satisfaction gain signifies the % improvement in the satisfaction ratiowhile using the online SPS scheme. We analyze the differences in the satisfaction ratio and satisfaction gain for three constraintscenarios: (i) MLA, (ii) Arrival Rate (Arrivals/Hour), and (iii) Number of Servers.

In the first scenario, we performed experiments by varying the MLA while keeping the number of servers and arrival rateas constant constraints. This ensures that the MLA is the major constraint in generating the schedules. We assume the numberof servers to be 8 and the arrival rate of jobs to be 30 time units, which are the median values of the respective metrics in thesecond and third scenario experiments below. From Figure 15, we can see that the online SPS further improves satisfactionratio compared to offline SPS, more notably for smaller MLA values. The reason for the improved satisfaction ratio of online

17

0 500 1000 1500 2000 25000

0.2

0.4

0.6

0.8

1

1.2

1.4

Number of Tasks

Aver

age

Str

etch

SPS

EDF

ACD

HBP

DSD

RR

(a) Considering all priority tasks

0 500 1000 1500 2000 25000

0.2

0.4

0.6

0.8

1

1.2

1.4

Number of Tasks

Aver

age

Str

etch

SPS

EDF

ACD

HBP

DSD

RR

LowerStretch forhighprioritytasks

(b) Considering high priority tasks

Fig. 14. Average Stretch comparison for increasing number of tasks

SPS is that it identifies oversampled tasks online and penalizes them resulting in a greater number of jobs getting scheduledcompared to the offline SPS.

In the second scenario, we performed experiments by varying the arrival rate of jobs while keeping the MLA, and thenumber of servers as constant constraints. We assume the MLA to be 4 and the number of servers to be 8. Figure 16 showsthe satisfaction ratio comparison between the offline SPS and online SPS with the arrival rate of jobs as a major constraintwhile generating schedules. We can see that the online SPS results in a further improved satisfaction ratio compared to theoffline SPS, particularly when the arrival rate of jobs is high.

0 2 4 6 8 100

0.2

0.4

0.6

0.8

1

MLA

Sati

sfa

cti

on

Rati

o

Offline SPS

Online SPS

Fig. 15. Satisfaction Ratio comparison of offline and online SPS withincreasing MLA constraint values

0 50 100 150 2000

0.2

0.4

0.6

0.8

1

Arrival Rate (Arrivals/Hour)

Sati

sfacti

on

Rati

o

Offline SPS

Online SPS

Fig. 16. Satisfaction Ratio comparison of offline and online SPS withincreasing Arrival Rate

Finally, in the third scenario, we perform experiments by varying the number of servers while keeping the MLA, and arrivalrate of jobs as constant constraints. We assume the MLA to be 4 and the arrival rate of jobs to be 30 time units. Figure 17shows that the online SPS produces a higher satisfaction ratio among users compared to the offline SPS, particularly when themeasurement servers are limited.

Figures 18, 19, and 20 show the satisfaction gain in % using the online SPS prioritization and oversampling mitigationscheme. We can see that the online SPS produces satisfaction gains that are more than 100% when the measurement loadis high or when the measurement constraints are strict, i.e., when (i) the MLA setting is low, (ii) the arrival rate of jobs ishigh, and (iii) the number of measurement servers available for scheduling are limited. Hence, by performing online SPSoversampling mitigation, we can generate more efficient schedules that can handle a higher number of measurement requestsand achieve further improvement in satisfaction ratio among users.

V. CONCLUSIONS

In this paper, we address the problem of measurement request prioritization and scheduling to handle measurement resourcecontention in widely-deployed and openly-accessible active measurement frameworks such as perfSONAR. We propose a novelontology-based semantic priority scheduling (SPS) scheme that handles the resource contention by dynamically prioritizingmeasurement requests based on user roles, user sampling preferences, and resource policies. Ultimately, it aims to efficientlysatisfy network monitoring objectives such as network weather forecasting, anomaly detection and fault-diagnosis across multi-domain measurement federations. To the best of our knowledge, our SPS scheme is the first to apply the ontology concept andan inference engine rule base to offline prioritize measurement requests and generate schedules. In addition, our SPS scheme

18

0 5 10 15 200

0.2

0.4

0.6

0.8

1

Number of Servers

Sati

sfa

cti

on

Rati

o

Offline SPS

Online SPS

Fig. 17. Satisfaction Ratio comparison of offline and online SPS withincreasing number of servers

0 2 4 6 8 100

100

200

300

400

500

MLA

Sati

sfacti

on

Gain

(%

)

Fig. 18. Satisfaction Gain from using online SPS with increasingMLA constraint values

0 50 100 150 2000

100

200

300

400

500

600

Arrival Rate (Arrivals / Hour)

Sati

sfacti

on

Gain

(%

)

Fig. 19. Satisfaction Gain from using online SPS with increasingArrival Rate

0 5 10 15 200

100

200

300

400

500

Number of Servers

Sati

sfacti

on

Gain

(%

)

Fig. 20. Satisfaction Gain from using online SPS with increasingnumber of servers

also has the ability to online detect and mitigate oversampling in measurement requests to further improve the measurementschedulability under high measurement loads.

Our performance evaluations that use real-world measurement request parameters and multi-domain considerations demon-strate that the SPS scheme outperforms existing measurement scheduling algorithms such as EDF, HBP, DSD, and ACD thatare based on typical scheduling priorities, e.g., period, laxity and execution time. More specifically, we showed how the SPSalgorithm can improve the satisfaction ratio among users and also how the SPS algorithm reduces the average stretch to ensurefairness in handling measurement requests. Thus, our SPS scheme and evaluation results foster the deployment and managementof large-scale multi-domain measurement infrastructures used for meeting monitoring objectives in support of next-generationapplications and networks.

REFERENCES

[1] S. Tao, K. Xu, A. Estepa, et. al., “Improving VoIP Quality through Path Switching”, Proceedings of the 24th Annual Joint Conference of the IEEEComputer and Communications Societies (IEEE INFOCOM), Vol. 4, pp. 2268-2278, March 2005.

[2] M. Yang, Y. Huang, J. Kim, et.al., “An End-to-End QoS Framework with On-Demand Bandwidth Reconfiguration”, Elsevier Computer Communications,Vol. 28, No. 18, pp. 2034-2046, November 2005.

[3] N. C. David, M. Grossglauser, “Measurement-Based Call Admission Control: Analysis and Simulation”, Proceedings of the 16th Annual Joint Conferenceof the IEEE Computer and Communications Societies (IEEE INFOCOM), Vol. 3, pp. 981-989, April 1997.

[4] A. Tirumala, L. Cottrell, T. Dunigan, “Measuring End-To-End Bandwidth with Iperf Using Web100”, Proceedings of Passive and Active MeasurementWorkshop (PAM), San Diego, CA, April 2003.

[5] C. Dovrolis, P. Ramanathan, D. Morre, “Packet Dispersion Techniques and Capacity Estimation”, IEEE/ACM Transactions on Networking, Vol. 12, No.6, pp. 963-977, December 2004.

[6] B. Gaidioz, R. Wolski, B. Tourancheau, “Synchronizing Network Probes to avoid Measurement Intrusiveness with the Network Weather Service”,Proceedings of the 9th International Symposium on High-performance Distributed Computing Conference, pp. 147-154, Pittsburgh, PA, August 2000.

[7] P. Calyam, C.-G. Lee, P. K. Arava, D. Krymskiy, D. Lee, “OnTimeMeasure: A Scalable Framework for Scheduling Active Measurements”, Proceedingsof End-to-End Monitoring Techniques and Services (IEEE E2EMON), pp. 86-100, May 2005.

[8] G. Ausiello, P. Crescenzi, V. Kann, et. al., “Complexity and Approximation: Combinatorial Optimization Problems and their Approximability Properties”,Springer Publication ISBN:3540654313, 1998.

[9] P. Calyam, C.-G. Lee, E. Ekici, M. Haffner, N. Howes, “Orchestrating Network-wide Active Measurements for Supporting Distributed ComputingApplications”, IEEE Transactions on Computers, Vol. 56, No. 12, pp. 1629-1642, December 2007.

[10] N. Duffield, “Sampling for Passive Internet Measurement: A Review”, Statistical Science, Vol. 19, No. 3, pp. 472-498, 2004.[11] T. Zseby, “Deployment of Sampling Methods for SLA Validation with Non-intrusive Measurements”, Proeedings of Passive and Active Measurement

Workshop (PAM), Fort Collins, CO, March 2002.

19

[12] T. Zseby, “Stratification Strategies for Sampling-based Non-intrusive Measurements of One-way Delay”, Proceedings of Passive and Active MeasurementWorkshop (PAM), San Diego, CA, April 2003.

[13] W. Ma, J. Yan, C. Huang, “Adaptive Sampling Methods for Network Performance Measurement under Voice Traffic”, Proceedings of InternationalConference on Communications (IEEE ICC), Vol. 2, pp 1129-1134, June 2004.

[14] P. Calyam, J. Pu, W. Mandrawa, A. Krishnamurthy, “OnTimeDetect: Dynamic Network Anomaly Notification in perfSONAR Deployments”, Proceedingsof International Symphosium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (IEEE/ACM MASCOTS), pp. 328-337,Miami Beach, FL, August 2010.

[15] K. Claffy, H. Braun, G. Polyzos, “Application of Sampling Methodologies to Network Traffic Characterization”, Proceedings of ACM SIGCOMM, SanFrancisco, CA, September 1993.

[16] A. McGregor, H-W. Braoun, “Automated Event Detection for Active Measurement Systems”, Proceedings of Passive and Active Measurement Workshop(PAM), Amsterdam, The Netherlands, April 2001.

[17] A. Hanemann, J. Boote, E. Boyd, J. Durand, L. Kudarimoti, R. Lapacz, M. Swany, S. Trocha, J. Zurawski, “PerfSONAR: A Service Oriented Architecturefor Multi-Domain Network Monitoring”, Proceedings of 3rd International Conference on Service Oriented Computing (ICSOC), Springer Verlag, LNCS3826 , pp. 241-254, Amsterdam, The Netherlands, December 2005. (http://www.perfsonar.net)

[18] E. Blanton, S. Fahmy, S. Banerjee, “Resource Management in an Active Measurement Service”, Proceedings of IEEE Global Internet Symposium, pp.1-6, April 2008.

[19] Z. Qin, R. Rojas-Cessa, N. Ansari, “Task-execution Scheduling Schemes for Network Measurement and Monitoring”, Elsevier Computer Communications,Vol. 33, No. 2, pp. 124-135, 2010.

[20] M. Fraiwan, G. Manimaran, “Scheduling Algorithms for Conducting Conflict-free Measurements in Overlay Networks”, Elesvier Computer Networks,Vol. 52, No. 15, pp. 2819-2830, 2008.

[21] J. Zurawski, “perfSONAR Tutorial”, First Workshop on the perfSONAR Network Measurement Infrastructure, Arlington, VA, July 2010(http://www.internet2.edu/workshops/perfSONAR).

[22] J.Etkin, J.Fridman, “An Algorithm for Scheduling Prioritized Tasks in a Hard Real-Time Environment”, Proceedings of EUROMICRO, pp. 69-76,September 1996.

[23] M. Huhns, L. Stephens, “Personal Ontologies”, IEEE Internet Computing, Vol. 3, No. 5, pp. 85-87, September 1999.[24] Perl Inference Engine - http://www.pre-emptive.net/doco/pie-perl-inference-engine[25] Protege OWL - http://protege.stanford.edu[26] Semantic Web Rule Language (SWRL) - http://www.w3.org/Submission/SWRL[27] A. Uszok, J. Bradshaw, L. Lott, et.al., “Toward a Flexible Ontology-based Policy Approach for Network Operations using the KAoS Framework”,

Proceedings of Military Communications Conference (MILCOM), 2011.[28] A. Rana, B. Jennings, M. Foghlu, S. van der Meer, “Autonomic Policy-based HAN Traffic Classification using Augmented Meta Model for Policy

Translation”, Proceedings of Wireless and Optical Communications Networks (WOCN), 2011.[29] J. Keeney, O. Conlan, V. Holub, W. Miao, L. Chapel, M. Serrano, S. van der Meer, “A Semantic Monitoring and Management Framework for End-to-end

Services”, Proceedings of IFIP/IEEE Integrated Network Management (IM), 2011.[30] J. Vergara, V. Villagra, A. Guerrero, J. Berrocal, “Ontology-based Network Management: Study Cases and Lessons Learned”, Springer Journal of

Network and Systems Management (JNSM), Vol. 17, No. 3, pp. 234 - 254, May 2009.[31] S. Abar, Y. Iwaya, T. Abe, T. Kinoshita, “Exploiting Domain Ontologies and Intelligent Agents: An Automated Network Management Support Paradigm”,

Springer Lecture Notes in Computer Science (LNCS), Vol. 3961, pp. 823 - 832, 2006.[32] A. Castro, J. Lozano, B. Fuentes, B. Costales, V. Villagra, “Multi-domain Fault Management Architecture based on a Shared Ontology-based Knowledge

Plane”, Proceedings of IEEE Conference on Network and Service Management (CNSM), pp. 493 - 498, 2010.[33] A. Wong, P. Ray, N. Parameswaran, J. Strassner, “Ontology Mapping for the Interoperability Problem in Network Management”, IEEE Journal on

Selected Areas in Communication (JSAC), Vol. 23, No. 10, pp. 2058 - 2068, October 2005.[34] D. Xiao, H. Xu, “An Integration of Ontology-based and Policy-based Network Management for Automation”, Proceedings of International Conference

on Intelligent Agents, Web Technologies and Internet Commerce (IAWTIC), 2006.[35] P. Calyam, L. Kumarasamy, F. Ozguner, “Semantic Scheduling of Active Measurements for meeting Network Monitoring Objectives”, Proceedings of

International Conference on Network and Service Management (IEEE CNSM), pp. 435-438, Niagara Falls, Canada, October 2010.[36] E. Saule, D. Bozdag, U. Catalyurek, “A Moldable Online Scheduling Algorithm and its Application to Parallel Short Sequence Mapping”, Proceedings

of 15th International Conference on Job Scheduling Strategies for Parallel Processing (JSSPP), Lecture Notes in Computer Science, Springer-Verlag, Vol.6253, pp. 93-109, Atlanta, April 2010.

PLACEPHOTOHERE

Prasad Calyam received his M.S. and Ph.D. degrees from the Department of Electrical and Computer Engineering at The OhioState University in 2002 and 2007, respectively. He is currently an Assistant Professor in the Department of Computer Scienceat University of Missouri-Columbia. His research and development areas of interest include: Distributed and Cloud Computing,Computer Networking, Networked-Multimedia Applications, and Cyber Security.

PLACEPHOTOHERE

Lakshmi Kumarasamy received her M.S. degree in Electrical and Computer Engineering from The Ohio State University in 2011.While pursuing her M.S. degree, she was involved in several research projects at the Ohio Supercomputer Center. One of the projectsshe contributed relates to a network performance monitoring research project, salient findings from which are presented in this paper.Currently, she is employed at Capital One Financial Corporation. Her research interests include network measurements, data miningand data analytics.

20

PLACEPHOTOHERE

Chang-Gun Lee received the B.S., M.S., and Ph.D. degrees in computer engineering from Seoul National University, Korea, in1991, 1993, and 1998, respectively. He is currently a professor in the School of Computer Science and Engineering, Seoul NationalUniversity, Korea. His current research interests include realtime embedded systems, cyber-physical systems, ubiquitous systems,QoS management, wireless ad hoc networks, and flash memory systems.

PLACEPHOTOHERE

Fusun Ozguner received an M.S. degree in electrical engineering from the Istanbul Technical University in 1972, and the Ph.D.degree in electrical engineering from the University of Illinois, Urbana-Champaign, in 1975. She is currently a Professor of Electricaland Computer Engineering at The Ohio State University, Department of Electrical and Computer Engineering. Her current researchinterests are parallel and fault-tolerant architectures, multicore chips, heterogeneous distributed computing, real-time parallel computingand communication, cybersecurity in intelligent vehicles and mobile networks, and cyberphysical systems.