seba operations and fcaps - open networking foundation · how is seba doing monitoring, telemetry...

21
SEBA Operations and FCAPS Session 3: 1:30pm - 3pm 56

Upload: others

Post on 22-May-2020

19 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: SEBA Operations and FCAPS - Open Networking Foundation · How is SEBA doing monitoring, telemetry and FCAPS with ELK ... SDN Controller - ONOS Network Edge Mediator (NEM) SEBA POD

SEBA Operations and FCAPS

Session 3: 1:30pm - 3pm

56

Page 2: SEBA Operations and FCAPS - Open Networking Foundation · How is SEBA doing monitoring, telemetry and FCAPS with ELK ... SDN Controller - ONOS Network Edge Mediator (NEM) SEBA POD

In this session

● How are operator workflows implemented in XOS, and what

are its internal models and services?

● How is SEBA doing monitoring, telemetry and FCAPS with ELK

stack, Prometheus and Grafana?

● How do we troubleshoot the system?

● How is the lifecycle of the pod managed?

57

Page 3: SEBA Operations and FCAPS - Open Networking Foundation · How is SEBA doing monitoring, telemetry and FCAPS with ELK ... SDN Controller - ONOS Network Edge Mediator (NEM) SEBA POD

VOLTHA apps

VOLTHA

SDN Controller - ONOS

Network Edge Mediator (NEM)

SEBA POD

Trellis apps

ONAP Op X OSS/BSS

Abstract OLT

DockerK8s

Helm

OLTONUExternal BNGSWITCH

SERVER

SEBASDN EnabledBroadband Access

(Embedded BNG)

Page 4: SEBA Operations and FCAPS - Open Networking Foundation · How is SEBA doing monitoring, telemetry and FCAPS with ELK ... SDN Controller - ONOS Network Edge Mediator (NEM) SEBA POD

VOLTHA

ONOS Cluster

vOLT dhcp mcast

OF

TPTPTP

NNI

OLT

ONU

UNI

Compute

AGG Switch

AGG Switch

SR FPM T3

OF

NEM

RedfishRedfish

SEBA POD

NEM Orchestration & SEBA Workflows

BNG is a workflow option• External physical BNG• vBNG in compute• BNG in Agg switches• BNG in OLT boxes

Different operators == Different workflows

Authentication is a workflow option• 802.1x based• PPPOE based• DHCP based

Subscriber services are workflow options• HSIA• Voice• IPTV• Business services• Technology profiles• Speed profiles

XOS GUI

XOS Core

Chameleon

Synchronizers

Tosca

Database

Page 5: SEBA Operations and FCAPS - Open Networking Foundation · How is SEBA doing monitoring, telemetry and FCAPS with ELK ... SDN Controller - ONOS Network Edge Mediator (NEM) SEBA POD

Why do we need a WorkflowDriver?

Most services are common to many use-cases.

The business logic is different operator by operator.

The workflow driver lets us re-use the same components to

achieve different solutions

60

Page 6: SEBA Operations and FCAPS - Open Networking Foundation · How is SEBA doing monitoring, telemetry and FCAPS with ELK ... SDN Controller - ONOS Network Edge Mediator (NEM) SEBA POD

ATTWorkflowDriver Modeling

● White List○ Specifies which ONUs are allowed, and on which PONs

● *WorkflowDriverServiceInstance○ Identifies ONU○ Identifies Subscriber○ Contains dynamic state, typically learned from events

■ administrative■ authentication■ dhcp

61

Page 7: SEBA Operations and FCAPS - Open Networking Foundation · How is SEBA doing monitoring, telemetry and FCAPS with ELK ... SDN Controller - ONOS Network Edge Mediator (NEM) SEBA POD

ATTWorkflowDriver: State Machine

62

Page 8: SEBA Operations and FCAPS - Open Networking Foundation · How is SEBA doing monitoring, telemetry and FCAPS with ELK ... SDN Controller - ONOS Network Edge Mediator (NEM) SEBA POD

Workflow kickoff

63

VOLTService

RCORD Service

Fabric XC Service

RCORDSubscriber

WorkflowDriver

ONU DISCOVER

WorkflowServiceInstance

OSS

ONU validation(whitelist)

Page 9: SEBA Operations and FCAPS - Open Networking Foundation · How is SEBA doing monitoring, telemetry and FCAPS with ELK ... SDN Controller - ONOS Network Edge Mediator (NEM) SEBA POD

Sample effects on the Service Chain

64

VOLTService

VOLTServiceInstance

RCORD Service

Fabric XC Service

Fabric XC ServiceInstance

RCORDSubscriber

WorkflowDriver

AUTH SUCCESS

Update subscriber status

add subscriber call in ONOS

create x-connect in ONOS

WorkflowServiceInstance

Page 10: SEBA Operations and FCAPS - Open Networking Foundation · How is SEBA doing monitoring, telemetry and FCAPS with ELK ... SDN Controller - ONOS Network Edge Mediator (NEM) SEBA POD

RG

VOLTHA

ONOS

NNI

OLT

ONUUNI

Compute

AGG

BNG

NEMKAFKA

PROMETHEUS ELK STACK

GRAFANA KIBANA

XOS

Operator OSS

NEM Monitoring

Infrastructure

Alarms, events & stats

Logs

Page 11: SEBA Operations and FCAPS - Open Networking Foundation · How is SEBA doing monitoring, telemetry and FCAPS with ELK ... SDN Controller - ONOS Network Edge Mediator (NEM) SEBA POD

Data flow for Monitoring and Logging

66

Page 12: SEBA Operations and FCAPS - Open Networking Foundation · How is SEBA doing monitoring, telemetry and FCAPS with ELK ... SDN Controller - ONOS Network Edge Mediator (NEM) SEBA POD

Kafka Topics within SEBA

Kafka allows many components to publish and subscribe to

messages, and has topics for:

- *.log* - log messages

- *.kpis - key performance indicators

- *.events - events created by ONOS or VOLTHA

Current convention is to encode messages as JSON, in the future

Protobuf may be used.

67

Page 13: SEBA Operations and FCAPS - Open Networking Foundation · How is SEBA doing monitoring, telemetry and FCAPS with ELK ... SDN Controller - ONOS Network Edge Mediator (NEM) SEBA POD

NEM Dashboards: XOS GUIRuntime service instantiation, Inventory, Workflow status

Page 14: SEBA Operations and FCAPS - Open Networking Foundation · How is SEBA doing monitoring, telemetry and FCAPS with ELK ... SDN Controller - ONOS Network Edge Mediator (NEM) SEBA POD

NEM Dashboards: GrafanaStatistics (FCAPS)

Page 15: SEBA Operations and FCAPS - Open Networking Foundation · How is SEBA doing monitoring, telemetry and FCAPS with ELK ... SDN Controller - ONOS Network Edge Mediator (NEM) SEBA POD

NEM Dashboards: KibanaLogs (FCAPS)

Page 16: SEBA Operations and FCAPS - Open Networking Foundation · How is SEBA doing monitoring, telemetry and FCAPS with ELK ... SDN Controller - ONOS Network Edge Mediator (NEM) SEBA POD

Backup

The NEM database (Postgres) can be backed up and restored using

the cordctl CLI tool, or via gRPC API.

Logging/monitoring services can be configured to use persistent

storage within the pod, and have API's for export/storage.

71

Page 17: SEBA Operations and FCAPS - Open Networking Foundation · How is SEBA doing monitoring, telemetry and FCAPS with ELK ... SDN Controller - ONOS Network Edge Mediator (NEM) SEBA POD

SEBA Lifecycle and Upgrades

In-service software upgrade (ISSU) is supported within NEM

- XOS

- 3rd party components (monitoring/logging/storage)

Other components (ONOS, VOLTHA) don't have persistent state

and are configured by XOS, so can be upgraded or restarted with

minimal service interruption.

72

Page 18: SEBA Operations and FCAPS - Open Networking Foundation · How is SEBA doing monitoring, telemetry and FCAPS with ELK ... SDN Controller - ONOS Network Edge Mediator (NEM) SEBA POD

Troubleshooting a SEBA deployment

● Check the cabling

● Check that traffic is flowing○ Packet counters are aggregated in Prometheus and Grafana○ tcpdump and similar

● Check for errors in the logs

○ Logs are aggregated across components in ELK Stack

● Follow the troubleshooting guide○ https://guide.opencord.org/profiles/seba/

73

Page 19: SEBA Operations and FCAPS - Open Networking Foundation · How is SEBA doing monitoring, telemetry and FCAPS with ELK ... SDN Controller - ONOS Network Edge Mediator (NEM) SEBA POD

Most common issues

● Configuration differs from physical setup○ How to spot: port state (up/down) in ONOS and XOS○ Errors in the ONOS log

● Serial Numbers mismatch○ How to spot: AttWorkflowDriver, sadis command in ONOS

74

Page 20: SEBA Operations and FCAPS - Open Networking Foundation · How is SEBA doing monitoring, telemetry and FCAPS with ELK ... SDN Controller - ONOS Network Edge Mediator (NEM) SEBA POD

Most common issues

● OLT Agent incompatible version○ How to spot: Device is not active in VOLTHA, or no ONUs are

discovered

● BNG or Radius configuration○ How to spot: EAPOL, DHCP or Ping not working,

ONOS/AttWorkflowDriver logs

75

Page 21: SEBA Operations and FCAPS - Open Networking Foundation · How is SEBA doing monitoring, telemetry and FCAPS with ELK ... SDN Controller - ONOS Network Edge Mediator (NEM) SEBA POD

Q & A?

76