![Page 1: Unsupervised Learning for Detection of Leakage from the ... · Unsupervised Learning for Detection of Leakage from the HFC Network 26-28 November Santa Fe, Argentina Emilia Gibellini](https://reader030.vdocuments.net/reader030/viewer/2022041014/5ec48df024424118e95a545d/html5/thumbnails/1.jpg)
Unsupervised Learning for
Detection of Leakage from the
HFC Network
26-28 November
Santa Fe, Argentina
Emilia Gibellini
Telecom Argentina
![Page 2: Unsupervised Learning for Detection of Leakage from the ... · Unsupervised Learning for Detection of Leakage from the HFC Network 26-28 November Santa Fe, Argentina Emilia Gibellini](https://reader030.vdocuments.net/reader030/viewer/2022041014/5ec48df024424118e95a545d/html5/thumbnails/2.jpg)
26-28 November
Santa Fe, Argentina
IntroCable operators & MSO’s delivery architecture: Hybrid Fiber/Coax
(HFC)
Transmissions can be distorted when folds, breaks, corrosion of
connectors, etc. occur.
We need to detect them PNM.
![Page 3: Unsupervised Learning for Detection of Leakage from the ... · Unsupervised Learning for Detection of Leakage from the HFC Network 26-28 November Santa Fe, Argentina Emilia Gibellini](https://reader030.vdocuments.net/reader030/viewer/2022041014/5ec48df024424118e95a545d/html5/thumbnails/3.jpg)
26-28 November
Santa Fe, Argentina
Intro
Current HFC Spectrum
50 MHz 1 GHz
Radio Digital public TVLTE 3G
Extended spectrum(after 2020)
![Page 4: Unsupervised Learning for Detection of Leakage from the ... · Unsupervised Learning for Detection of Leakage from the HFC Network 26-28 November Santa Fe, Argentina Emilia Gibellini](https://reader030.vdocuments.net/reader030/viewer/2022041014/5ec48df024424118e95a545d/html5/thumbnails/4.jpg)
26-28 November
Santa Fe, Argentina
MeasurementFull-Band Capture:
• 24,000 measurements per cable modem
• 45 MHz to 1 GHz
![Page 5: Unsupervised Learning for Detection of Leakage from the ... · Unsupervised Learning for Detection of Leakage from the HFC Network 26-28 November Santa Fe, Argentina Emilia Gibellini](https://reader030.vdocuments.net/reader030/viewer/2022041014/5ec48df024424118e95a545d/html5/thumbnails/5.jpg)
26-28 November
Santa Fe, Argentina
• Goal: find similarities &
differences among
observations
• Many algorithms
• Well-known technique:
k-means
Methodology: cluster analysis
![Page 6: Unsupervised Learning for Detection of Leakage from the ... · Unsupervised Learning for Detection of Leakage from the HFC Network 26-28 November Santa Fe, Argentina Emilia Gibellini](https://reader030.vdocuments.net/reader030/viewer/2022041014/5ec48df024424118e95a545d/html5/thumbnails/6.jpg)
26-28 November
Santa Fe, Argentina
Methodology: k-means
1. Set K points as centers.
2. Calculate distances to
centers.
3. Assign observations.
4. Evaluate & recalculate
centers.
(repeat 2 - 5)
![Page 7: Unsupervised Learning for Detection of Leakage from the ... · Unsupervised Learning for Detection of Leakage from the HFC Network 26-28 November Santa Fe, Argentina Emilia Gibellini](https://reader030.vdocuments.net/reader030/viewer/2022041014/5ec48df024424118e95a545d/html5/thumbnails/7.jpg)
26-28 November
Santa Fe, Argentina
Challenge: define meaningful dimensions for the analysis.
Methodology
We can think of…
• Each frequency as a new dimension of analysis.
• Measurements on each modem as a vector.
![Page 8: Unsupervised Learning for Detection of Leakage from the ... · Unsupervised Learning for Detection of Leakage from the HFC Network 26-28 November Santa Fe, Argentina Emilia Gibellini](https://reader030.vdocuments.net/reader030/viewer/2022041014/5ec48df024424118e95a545d/html5/thumbnails/8.jpg)
26-28 November
Santa Fe, Argentina
• Algorithm: k-means
• K=6 (based on empirical methods)
• Distance: Euclidean
• 4,458 measurements per cable modem (frequencies where ingress
may occur)
• Vectors of the form: X: {X1; X2; …;X4,458}
Methodology: to sum up…
![Page 9: Unsupervised Learning for Detection of Leakage from the ... · Unsupervised Learning for Detection of Leakage from the HFC Network 26-28 November Santa Fe, Argentina Emilia Gibellini](https://reader030.vdocuments.net/reader030/viewer/2022041014/5ec48df024424118e95a545d/html5/thumbnails/9.jpg)
26-28 November
Santa Fe, Argentina
Examples:
Input
One modem All modems
![Page 10: Unsupervised Learning for Detection of Leakage from the ... · Unsupervised Learning for Detection of Leakage from the HFC Network 26-28 November Santa Fe, Argentina Emilia Gibellini](https://reader030.vdocuments.net/reader030/viewer/2022041014/5ec48df024424118e95a545d/html5/thumbnails/10.jpg)
26-28 November
Santa Fe, Argentina
Results
88 MHz – 108 MHz:
• 3 & 6: acceptable noise.
• 1, 2, 4 & 5: “peaks”
• Peaks indicate ingress from
FM radios.
![Page 11: Unsupervised Learning for Detection of Leakage from the ... · Unsupervised Learning for Detection of Leakage from the HFC Network 26-28 November Santa Fe, Argentina Emilia Gibellini](https://reader030.vdocuments.net/reader030/viewer/2022041014/5ec48df024424118e95a545d/html5/thumbnails/11.jpg)
26-28 November
Santa Fe, Argentina
Results
869 MHz - 894 MHz:
• 2 & 6: internal noise only.
• 1, 4 & 5: “squared” shapes.
• 3: ~ “squared”, less serious.
• Squared signals indicate 3G
ingress.
![Page 12: Unsupervised Learning for Detection of Leakage from the ... · Unsupervised Learning for Detection of Leakage from the HFC Network 26-28 November Santa Fe, Argentina Emilia Gibellini](https://reader030.vdocuments.net/reader030/viewer/2022041014/5ec48df024424118e95a545d/html5/thumbnails/12.jpg)
26-28 November
Santa Fe, Argentina
About the project
Find clusters
AnalyzeCheck w/
PNM team
Training setSupervised algorithm
![Page 13: Unsupervised Learning for Detection of Leakage from the ... · Unsupervised Learning for Detection of Leakage from the HFC Network 26-28 November Santa Fe, Argentina Emilia Gibellini](https://reader030.vdocuments.net/reader030/viewer/2022041014/5ec48df024424118e95a545d/html5/thumbnails/13.jpg)
26-28 November
Santa Fe, Argentina
Latest resultsLTE ingress on another (greater) sample:
![Page 14: Unsupervised Learning for Detection of Leakage from the ... · Unsupervised Learning for Detection of Leakage from the HFC Network 26-28 November Santa Fe, Argentina Emilia Gibellini](https://reader030.vdocuments.net/reader030/viewer/2022041014/5ec48df024424118e95a545d/html5/thumbnails/14.jpg)
26-28 November
Santa Fe, Argentina
Latest resultsFM radio ingress on another (greater) sample:
![Page 15: Unsupervised Learning for Detection of Leakage from the ... · Unsupervised Learning for Detection of Leakage from the HFC Network 26-28 November Santa Fe, Argentina Emilia Gibellini](https://reader030.vdocuments.net/reader030/viewer/2022041014/5ec48df024424118e95a545d/html5/thumbnails/15.jpg)
26-28 November
Santa Fe, Argentina
Latest resultsFM radio ingress in another sample:
Severity
1
2
3
4
![Page 16: Unsupervised Learning for Detection of Leakage from the ... · Unsupervised Learning for Detection of Leakage from the HFC Network 26-28 November Santa Fe, Argentina Emilia Gibellini](https://reader030.vdocuments.net/reader030/viewer/2022041014/5ec48df024424118e95a545d/html5/thumbnails/16.jpg)
26-28 November
Santa Fe, Argentina
Latest results
• Simpler way to
characterize signals with
impairments.
• Train anomaly detection
(supervised) algorithm.
• Is it possible to
extrapolate?
O.K.
Ingress
![Page 17: Unsupervised Learning for Detection of Leakage from the ... · Unsupervised Learning for Detection of Leakage from the HFC Network 26-28 November Santa Fe, Argentina Emilia Gibellini](https://reader030.vdocuments.net/reader030/viewer/2022041014/5ec48df024424118e95a545d/html5/thumbnails/17.jpg)
26-28 November
Santa Fe, Argentina
Conclusions
The clustering method is useful for pattern detection.
Easy to share and replicate.
It provides a successful example of ML application.
However…
• Early stage - needs further analysis.
• Minimum requirements?
• Characterize sample to use as training set.
• Train supervised algorithm.
• Share - is it generalizable?
![Page 18: Unsupervised Learning for Detection of Leakage from the ... · Unsupervised Learning for Detection of Leakage from the HFC Network 26-28 November Santa Fe, Argentina Emilia Gibellini](https://reader030.vdocuments.net/reader030/viewer/2022041014/5ec48df024424118e95a545d/html5/thumbnails/18.jpg)
Thank you
![Page 19: Unsupervised Learning for Detection of Leakage from the ... · Unsupervised Learning for Detection of Leakage from the HFC Network 26-28 November Santa Fe, Argentina Emilia Gibellini](https://reader030.vdocuments.net/reader030/viewer/2022041014/5ec48df024424118e95a545d/html5/thumbnails/19.jpg)
26-28 November
Santa Fe, Argentina
Elbow rule
Methodology: find K
Stable classes