cedar counter-estimation decoupling for approximate rates
DESCRIPTION
CEDAR Counter-Estimation Decoupling for Approximate Rates. Erez Tsidon ( Technion , Israel). Joint work with Iddo Hanniel and Isaac Keslassy ( Technion ). Network Flow Counters Usage. Network management applications require per-flow counters, for example: Congestion Control - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/1.jpg)
CEDARCounter-Estimation Decoupling for
Approximate Rates
Erez Tsidon (Technion, Israel)
Joint work with Iddo Hanniel and Isaac Keslassy (Technion)
1
![Page 2: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/2.jpg)
Network Flow Counters Usage
Network management applications require per-flow counters, for example: Congestion Control Detection of Denial of Service Attacks Detection of Traffic Anomalies
Counter types: Packet counting Byte counting Rate measurement
2
![Page 3: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/3.jpg)
Switch Example
40MB memory 32nsec for single counter access
DRAM is too slowSRAM is too expensive
3
SWITCH106 flows
Total Packet CountTotal Byte CountPacket RateCount event ACount event B
per-flow counters
64-bit width
Min packet size = 40B
Link Rate 10Gbps
![Page 4: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/4.jpg)
Suggested Solutions
Hybrid SRAM-DRAM counters [Shah, Iyer, Prabhakar and McKeown ’02] Cannot support fast reading
Counter Braids – compress counters into small SRAM [Y. Lu et al ’08] Cannot decompress in real time
Heavy Hitters – store only high counters [Estan and Varghese ’03] No records of small counter values
4
![Page 5: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/5.jpg)
Counter Estimation Solutions
Intuitively we want counters to be as precise as possible, unbiased whenever possible, and scalable
SAC – R. Stanojevic, “Small Active Counters”, 2007 Exponent-Magnitude representation Unbiased estimation ScalableRestricted to specific representation which
prevents error optimization DISCO – C. Hu et al, “DISCO: Memory Efficient and Accurate Flow
Statistics for Network Measurement”, 2010 Convex conversion function that reduces increment values Unbiased estimationRestricted to a close function representation. No
scaling
5
![Page 6: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/6.jpg)
Our Contributions
CEDAR – decoupling counters from estimators
Optimal estimators for the min-max relative error
Dynamic up-scale algorithm Exponential-averaged rate estimation
6
![Page 7: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/7.jpg)
Counter-Estimators Decoupling
7
995,784
1.2
1,000,000
1.2
Counter estimates
FN-1
FN-2
F1
F0
1,000,000
995,784
1.2
0
p(L-2)
p(1)
p(L-1)
p(1)
AL-1
AL-2
A1
A0
3.7 A2
Flow pointers
Shared estimators
log2 L
q
FN-1
FN-2
F1
F0
![Page 8: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/8.jpg)
CEDAR Structure
N flows L estimators Flow array: Estimation array:
Estimator differences:
,0jF j N
0,0 , 0iA i L A
1 , 1i i i iD A A D 8
1,000,000995,784
1.20
p(L-2)p(1)
p(L-1)p(1)
AL-1
AL-2
A1
A0
3.7 A2
Flow pointers
Shared estimators
log2 L
q
FN-1
FN-2
F1
F0
![Page 9: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/9.jpg)
01
4 4
CEDAR Increment Algorithm
941
A3
A2
A1
0A0
21113254.711
A7
A6
A5
A4
9410
21113254.711
9410
21113254.711
9410
21113254.711
time
p=1
p=1/3p=1/5
t=0 t=1 t=2 t=3
9
Upon packet arrival: with probability1j jF F 1
jF
pD
![Page 10: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/10.jpg)
Performance Measures
Traffic Amount – - a random variable that represents
the number of real counter increments in order for to hit estimator
Relative error – Coefficient of Variation:
( )T a
X̂ a
10
2
( ) ( )( )
( ) ( )Var T a T a
T aE T a E T a
![Page 11: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/11.jpg)
Min-Max Relative Error
Problem: assuming L and AL-1 are given, find an estimation array that minimizes the guaranteed relative error δ such that
Solution: equal relative error
Estimation values
2
( ), ( )
( )l
ll
Var T Al T A
E T A
11
12
01 2
1 2
1
l
ii
l l
DA A
, ( )ll T A
![Page 12: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/12.jpg)
Equal Relative Error Example
12Estimation Values
Relative Error
δ
δ
A1 A2 A3
A1 A2 A3
δ
A1 A2 A3
![Page 13: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/13.jpg)
Capacity Region of Static CEDAR
13
Example:• 10-bit counters• Max value 10^6min-max relative error 8%
![Page 14: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/14.jpg)
4.5
1
Up-Scale Procedure
3
1
A3
A2
A1
0A0
211
132
54
11
A7
A6
A5
A4
24
5
2
0
517
314
156
93
54
11
p=0.5 0
p=0.43=(54-24)/(93-24)
93
14
A’ A’’
upscalethreshold
Initial relative error δ0
Increase the relative error
δ0+ δstep
![Page 15: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/15.jpg)
CEDAR Unbiasedness
15
![Page 16: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/16.jpg)
CEDAR Equal Error
16
![Page 17: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/17.jpg)
CEDAR Vs. SAC & DISCO 12-bit
17
4096 estima
tors
![Page 18: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/18.jpg)
CEDAR Vs. SAC & DISCO 8-bit
18
256estima
tors
![Page 19: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/19.jpg)
CEDAR Error Adjustment 12-bit
19
![Page 20: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/20.jpg)
CEDAR Implementation on FPGA
20
5.4 Gbps
12K gates
![Page 21: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/21.jpg)
Exponential-Average Rate Estimation
A3
A2
A1
A0
A7
A6
A5
A4
time
x0.98 down-scale
33.316.60
99.983.366.649.9
119.5
5134170
1221028568
0
P = 0.02x100/(85-68)
Incoming packet
513417
1221028568
t0 t0 + 1 t0 + 2
x0.98 down-scale
32.616.30
97.981.665.348.9
117.1
6885 83.3 81.683.383.3
21
![Page 22: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/22.jpg)
Exponential-Average Rate Estimation
A3
A2
A1
A0
A7
A6
A5
A4
time
t0 + 10 t0 + 2
32.616.30
97.981.665.348.9
117.1
41.627.7
0
99.683.369.455.5
13.8
up-scaling
0
513417
1221028568P =
(85-69.4)/(85-68)
After 8 more down-scaling cycles
81.6 69.468
99.6
22
![Page 23: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/23.jpg)
EXP-CEDAR 7-bit
23
Real exp-average with 64 bits
For 10^(-2) precision – 14 bits are required
![Page 24: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/24.jpg)
EXP-CEDAR 9-bit
24
![Page 25: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/25.jpg)
CEDAR Summary
Decoupling - flexible estimators Unbiased and scalable estimation Attains the min-max relative error FPGA supports link rate of 5.4Gbps and may
increase to tens of Gbps on ASIC Exponential-average rate estimation
25
![Page 26: CEDAR Counter-Estimation Decoupling for Approximate Rates](https://reader035.vdocuments.net/reader035/viewer/2022070422/56816498550346895dd66e1a/html5/thumbnails/26.jpg)
Thank you.
26