university of illinois at urbana-champaign real-time capacity of networked data fusion university of...
TRANSCRIPT
![Page 1: University of Illinois at Urbana-Champaign Real-Time Capacity of Networked Data Fusion University of Illinois at Urbana-Champaign Forrest Iandola (University](https://reader030.vdocuments.net/reader030/viewer/2022020117/56649eb55503460f94bbd35f/html5/thumbnails/1.jpg)
Real-Time Capacity of Networked Data Fusion University of Illinois at Urbana-ChampaignUniversity of Illinois at Urbana-Champaign
Forrest Iandola (University of Illinois)Fatemeh Saremi (University of Illinois)Tarek Abdelzaher (University of Illinois)Praveen Jayachandran (IBM Research)
Aylin Yener (Pennsylvania State University)
![Page 2: University of Illinois at Urbana-Champaign Real-Time Capacity of Networked Data Fusion University of Illinois at Urbana-Champaign Forrest Iandola (University](https://reader030.vdocuments.net/reader030/viewer/2022020117/56649eb55503460f94bbd35f/html5/thumbnails/2.jpg)
Motivation and Goals Develop a theoretical bound for the
capacity of data fusion systems Enable data fusion systems to run at
full capacity without missing deadlines
Forrest IandolaIllustration of a data fusion system with merging
![Page 3: University of Illinois at Urbana-Champaign Real-Time Capacity of Networked Data Fusion University of Illinois at Urbana-Champaign Forrest Iandola (University](https://reader030.vdocuments.net/reader030/viewer/2022020117/56649eb55503460f94bbd35f/html5/thumbnails/3.jpg)
Outline Introduce data fusion system model Scheduling theory background: Feasible
Region Calculus Derive a capacity utilization bound for
data fusion pipelines Extend this bound to capture merging
pipelines Performance evaluation
Forrest Iandola
![Page 4: University of Illinois at Urbana-Champaign Real-Time Capacity of Networked Data Fusion University of Illinois at Urbana-Champaign Forrest Iandola (University](https://reader030.vdocuments.net/reader030/viewer/2022020117/56649eb55503460f94bbd35f/html5/thumbnails/4.jpg)
“Data Fusion System” refers to… Distributed sensor networks Control systems that receive one or
more data feeds “Real-Time Capacity” = data
packets transmitted within time constraints
Forrest Iandola
Data Fusion System Model (1/3)
![Page 5: University of Illinois at Urbana-Champaign Real-Time Capacity of Networked Data Fusion University of Illinois at Urbana-Champaign Forrest Iandola (University](https://reader030.vdocuments.net/reader030/viewer/2022020117/56649eb55503460f94bbd35f/html5/thumbnails/5.jpg)
Data Fusion System Model (2/3) Workflow i is denoted as Fi
Invocation of Fi is a “job” q Di = deadline of Fi
Pi = period of Fi
Ri = 1/Pi = “Rate” Ci,j = computation of Fi on stage j
Forrest Iandola
![Page 6: University of Illinois at Urbana-Champaign Real-Time Capacity of Networked Data Fusion University of Illinois at Urbana-Champaign Forrest Iandola (University](https://reader030.vdocuments.net/reader030/viewer/2022020117/56649eb55503460f94bbd35f/html5/thumbnails/6.jpg)
Data Fusion System Model (3/3) System constraints reflect a
realistic data fusion system Non-preemptive earliest deadline first
(EDF) scheduling Workflows are periodic. Di >> Pi (in other words, many
invocations of Fi may be active simultaneously.)
Forrest Iandola
![Page 7: University of Illinois at Urbana-Champaign Real-Time Capacity of Networked Data Fusion University of Illinois at Urbana-Champaign Forrest Iandola (University](https://reader030.vdocuments.net/reader030/viewer/2022020117/56649eb55503460f94bbd35f/html5/thumbnails/7.jpg)
Scheduling Theory Background: Feasible Region Calculus (FRC) A pipeline task set can be reduced
to a uniprocessor equivalent: Assume qN is the lowest-priority
workflow
Forrest Iandola
![Page 8: University of Illinois at Urbana-Champaign Real-Time Capacity of Networked Data Fusion University of Illinois at Urbana-Champaign Forrest Iandola (University](https://reader030.vdocuments.net/reader030/viewer/2022020117/56649eb55503460f94bbd35f/html5/thumbnails/8.jpg)
For simplicity, let us refer to the “modified” equivalent of the lowest-priority task as q
Forrest Iandola
Scheduling Theory Background: Feasible Region Calculus (FRC)
![Page 9: University of Illinois at Urbana-Champaign Real-Time Capacity of Networked Data Fusion University of Illinois at Urbana-Champaign Forrest Iandola (University](https://reader030.vdocuments.net/reader030/viewer/2022020117/56649eb55503460f94bbd35f/html5/thumbnails/9.jpg)
Deriving Capacity Bound from FRC Testing schedulability of equivalent
uniprocessor from as defined by FRC Remember: we assume non-
preemptive EDF scheduling
Forrest Iandola
![Page 10: University of Illinois at Urbana-Champaign Real-Time Capacity of Networked Data Fusion University of Illinois at Urbana-Champaign Forrest Iandola (University](https://reader030.vdocuments.net/reader030/viewer/2022020117/56649eb55503460f94bbd35f/html5/thumbnails/10.jpg)
Testing schedulability of equivalent uniprocessor from as defined by FRC Remember: we assume non-
preemptive EDF scheduling
Forrest Iandola
Deriving Capacity Bound from FRC
Basic utilization formula:
Combining utilization formula with FRC definitions:
To avoid deadline misses,utilization must be less than 1.
![Page 11: University of Illinois at Urbana-Champaign Real-Time Capacity of Networked Data Fusion University of Illinois at Urbana-Champaign Forrest Iandola (University](https://reader030.vdocuments.net/reader030/viewer/2022020117/56649eb55503460f94bbd35f/html5/thumbnails/11.jpg)
Simplifying the Capacity Bound to Reduce Computation Overhead
Stage-additive component is very small when Di >> Pi
Can approximate the utilization even if we ignore stage-additive component
Forrest Iandola
Reduce computation time bydropping lowest-priority invocation:
Replace ceiling function with (DiRi+1):
![Page 12: University of Illinois at Urbana-Champaign Real-Time Capacity of Networked Data Fusion University of Illinois at Urbana-Champaign Forrest Iandola (University](https://reader030.vdocuments.net/reader030/viewer/2022020117/56649eb55503460f94bbd35f/html5/thumbnails/12.jpg)
Handling Merging Flows
Forrest Iandola
![Page 13: University of Illinois at Urbana-Champaign Real-Time Capacity of Networked Data Fusion University of Illinois at Urbana-Champaign Forrest Iandola (University](https://reader030.vdocuments.net/reader030/viewer/2022020117/56649eb55503460f94bbd35f/html5/thumbnails/13.jpg)
Forrest Iandola
Handling Merging Flows
![Page 14: University of Illinois at Urbana-Champaign Real-Time Capacity of Networked Data Fusion University of Illinois at Urbana-Champaign Forrest Iandola (University](https://reader030.vdocuments.net/reader030/viewer/2022020117/56649eb55503460f94bbd35f/html5/thumbnails/14.jpg)
Let’s discuss the intuition behind this.
Step 1: Reduce child pipelines to equivalent uniprocessor workflow sets
Step 2: Obtain two-stage pipeline Ignore all but the largest equivalent
pipeline per workflow Step 3: Calculate equivalent
uniprocessor for two-stage pipelineForrest Iandola
Handling Merging Flows
![Page 15: University of Illinois at Urbana-Champaign Real-Time Capacity of Networked Data Fusion University of Illinois at Urbana-Champaign Forrest Iandola (University](https://reader030.vdocuments.net/reader030/viewer/2022020117/56649eb55503460f94bbd35f/html5/thumbnails/15.jpg)
Fundamental Results
Forrest Iandola
![Page 16: University of Illinois at Urbana-Champaign Real-Time Capacity of Networked Data Fusion University of Illinois at Urbana-Champaign Forrest Iandola (University](https://reader030.vdocuments.net/reader030/viewer/2022020117/56649eb55503460f94bbd35f/html5/thumbnails/16.jpg)
Evaluation of Capacity Bound Comparing predicted useful work of a data fusion tree to actual useful
work (just before onset of deadline misses) Note: Utilization due to jobs/flows that miss deadlines is not counted as useful
work.
Forrest Iandola
Observations: Capacity bound
accurately predicts ability to do useful work
![Page 17: University of Illinois at Urbana-Champaign Real-Time Capacity of Networked Data Fusion University of Illinois at Urbana-Champaign Forrest Iandola (University](https://reader030.vdocuments.net/reader030/viewer/2022020117/56649eb55503460f94bbd35f/html5/thumbnails/17.jpg)
Evaluation of Overload Behavior
Comparing overload behavior of a data fusion tree with admission control (based on new capacity result) to one without Note: Utilization due to jobs/flows that miss deadlines is not counted as useful
work.
Forrest Iandola
Observations: Capacity bound
accurately predicts ability to do useful work
At high load, significant degradation is observed in the absence of admission control due to excessive deadline misses
![Page 18: University of Illinois at Urbana-Champaign Real-Time Capacity of Networked Data Fusion University of Illinois at Urbana-Champaign Forrest Iandola (University](https://reader030.vdocuments.net/reader030/viewer/2022020117/56649eb55503460f94bbd35f/html5/thumbnails/18.jpg)
Conclusions Derived a capacity utilization bound
for data fusion systems Simplified the bound into an easy-
to-use approximation Extended this result for merging
workflows Evaluation demonstrates accuracy
of bound
Forrest Iandola