machine learning for autonomic network management in a ...€¦ · network management is becoming...
TRANSCRIPT
![Page 1: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/1.jpg)
1 1
Machine Learning for Autonomic Network Management in a Connected Cars Scenario
Gorka Vélez, Marco Quartulli, Ángel Martín, Oihana Otaegui (Vicomtech-IK4)
Haytham Assem
(IBM Ireland)
![Page 2: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/2.jpg)
2 2
Index
• Introduction
• Proposed solution
– Architecture
– Implementation details
– Evaluation
• Conclusions
• Future work
![Page 3: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/3.jpg)
3 3
Introduction
• Due to the vast increase in the number of connected devices, network management is becoming more complicated.
• 4G is approaching its limits.
• 5G will need to specially take into account IoT challenges in relation to network management.
• Apart from maximising spectrum and data rates, it will need to largely manage itself.
![Page 4: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/4.jpg)
4 4
Introduction
• The transportation sector is one of the key sectors that will be benefited from 5G.
• Autonomous and semi-autonomous cars will depend heavily on connectivity.
• So the network must ensure high data rates of big volumes of data at potentially high travelling speeds, what supposes a great challenge.
![Page 5: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/5.jpg)
5 5
Introduction
• Current trend is to decouple hardware from software and move Network Functions (NF) towards software: Network Function Virtualization (NFV).
• A Virtual Network Function (VNF), consists of one or more virtual machines that implements a network function (e.g., router, firewall, etc)
• A policy manager and orchestator is needed to organize all the VNF instances: MANO (proposed by ETSI)
![Page 6: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/6.jpg)
6 6
Introduction
![Page 7: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/7.jpg)
7 7
Introduction
• We believe that Machine Learning will be one of the key technologies that will enable Autonomic Network Management.
• Machine Learning technologies can learn from historical data, and make predictions or decisions.
• Instead of following static programming instructions, it can dynamically adapt to new situations learning from new data.
• For example, it can be used to forecast resource demand requirements, detect security threats and error conditions, and react correctly to them.
![Page 8: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/8.jpg)
8 8
Introduction
• Proposal:
Combine virtualisation and Machine Learning in network management to overcome the main challenges in connected cars scenarios and advance towards Autonomic Network Management.
![Page 9: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/9.jpg)
9 9
Proposed solution: architecture
System Z System Y
Data Collector Data Storage
Data Analyser (ML)
ML Algorithm X
ML Algorithm Y
ML Algorithm Z
Policy Manager
Policy for System X
Policy for System Y
Policy for System Z
System X System/VNF
analysed/optimized
Outgoing API
Incoming API
(feedback)
High-level architecture: Smart Engine
![Page 10: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/10.jpg)
10 10
Proposed solution: architecture
Data Collector
(Near) Real-time Processing Engine
Batch Processing Engine
Data Pre-processing
User
inputs
Data Normalisation,
Transformation
Adaptors
Feature Selection and
Extraction
OptionalMandatory
Distributed File System
Measurements (Network
and Infrastructure)
Automated Model Selection
Dataset
Analysis
Comparative
Analysis
Model
Selection
Data Storage
Frozen
Data
Stream
Generator
(Semi) Supervised & Unsupervised ML
Distortion
Evaluation
Model
Training
Online Learning and Scoring or Score using model
from Batch Layer
Model
Training
Pull ModelModel
Scoring
Model
Scoring
Data Cleaning,
Filtering, Feature
Extraction
Hardware
AnalysisUser
inputs
User
inputs
E2E
inputs
Smart Engine:
![Page 11: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/11.jpg)
11 11
Proposed solution: architecture
System Z System Y
Data Collector Data Storage
Data Analyser (ML)
ML Algorithm X
ML Algorithm Y
ML Algorithm Z
Policy Manager
Policy for System X
Policy for System Y
Policy for System Z
System X System/VNF
analysed/optimized
Outgoing API
Incoming API
(feedback)
High-level architecture:
![Page 12: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/12.jpg)
12 12
Proposed solution: architecture
Policy Manager • Machine Learning cannot directly operate in the network
infrastructure. • An additional component is needed for translating the
output from the Smart Engine to the policies that can be directly understood by the related components in the NFV Infrastructure and the orchestrator.
• In same cases human interaction might also be needed.
![Page 13: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/13.jpg)
13 13
Proposed solution: implementation
System Z System Y
Data Collector Data Storage
Data Analyser (ML)
ML Algorithm X
ML Algorithm Y
ML Algorithm Z
Policy Manager
Policy for System X
Policy for System Y
Policy for System Z
System X System/VNF
analysed/optimized
Outgoing API
Incoming API
(feedback)
Virtual Infrastructure
SDN controller
NFV Management and Orchestration
![Page 14: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/14.jpg)
14 14
Proposed solution: implementation
System Z System Y
Data Collector Data Storage
Data Analyser (ML)
ML Algorithm X
ML Algorithm Y
ML Algorithm Z
Policy Manager
Policy for System X
Policy for System Y
Policy for System Z
System X System/VNF
analysed/optimized
Outgoing API
Incoming API
(feedback)
monitoring solution integrated with OpenStack
![Page 15: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/15.jpg)
15 15
Proposed solution: implementation
System Z System Y
Data Collector Data Storage
Data Analyser (ML)
ML Algorithm X
ML Algorithm Y
ML Algorithm Z
Policy Manager
Policy for System X
Policy for System Y
Policy for System Z
System X System/VNF
analysed/optimized
Outgoing API
Incoming API
(feedback)
Database compatible with Monasca
![Page 16: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/16.jpg)
16 16
Proposed solution: implementation
System Z System Y
Data Collector Data Storage
Data Analyser (ML)
ML Algorithm X
ML Algorithm Y
ML Algorithm Z
Policy Manager
Policy for System X
Policy for System Y
Policy for System Z
System X System/VNF
analysed/optimized
Outgoing API
Incoming API
(feedback)
MLlib
![Page 17: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/17.jpg)
17 17
Proposed solution: implementation
System Z System Y
Data Collector Data Storage
Data Analyser (ML)
ML Algorithm X
ML Algorithm Y
ML Algorithm Z
Policy Manager
Policy for System X
Policy for System Y
Policy for System Z
System X System/VNF
analysed/optimized
Outgoing API
Incoming API
(feedback)
![Page 18: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/18.jpg)
18 18
Evaluation of proposed solution
• Evaluation based on KPIs. • Two configurations of the connected cars scenario:
• Application based on low-level data: e.g., raw data from camera sensors (video streaming) • High data rate (up to 10-20Mbits/s) • Medium tolerance on errors ( 10-2 )
• Application based on high-level data: e.g., objects or events detected • Medium data rate (up to 1Mbit/s) • Very low tolerance on errors (10-5)
![Page 19: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/19.jpg)
19 19
Evaluation of proposed solution
• Other requirements of connected cars scenario: • End-to-end latency < 5ms for message sizes of about
1600 bytes. • Data sent either event-driven or periodically at 10 Hz. • Minimum throughput: 3 Mbit/s
![Page 20: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/20.jpg)
20 20
Conclusions
• The softwarisation and virtualisation of common network functions can raise the level of abstraction, simplifying and making more flexible the network configuration process.
• If we combine virtualisation and Machine Learning we can move towards Autonomic Network Management.
• This new paradigm can also impact on ITS, specially with the adoption of 5G
• A software-based self-managing 5G network could more easily fulfil the requirements of vehicle communications (V2X).
![Page 21: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/21.jpg)
21 21
Conclusions
• It does not exist a generic machine learning mechanism suitable for all use cases.
• So instead of proposing a specific approach, we propose a generic architecture that can accommodate different machine learning strategies.
![Page 22: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/22.jpg)
22 22
Future work
• The deployment and validation of our solution require further and intensive research and development work.
• Currently developing a simulator to generate network traffic with different vehicle traffic conditions.
• We will create a service demand prediction using Machine Learning and based on vehicle traffic flow.
![Page 23: Machine Learning for Autonomic Network Management in a ...€¦ · network management is becoming more complicated. • 4G is approaching its limits. • 5G will need to specially](https://reader036.vdocuments.net/reader036/viewer/2022071003/5fbfc36de11ec11ff550406a/html5/thumbnails/23.jpg)
23 23
Machine Learning for Autonomic Network Management in a Connected Cars Scenario
Gorka Vélez, Marco Quartulli, Ángel Martín, Oihana Otaegui (Vicomtech-IK4)
Haytham Assem
(IBM Ireland)