commissioning, managing & troubleshooting industrial networks

Tips for Commissioning,

Managing, and Troubleshooting

your Industrial Network

Moxa Technology Webinar Series

Richard Wood

Networking Infrastructure Manager

Agenda

Industrial Network Challenges

Network Configuration & Commissioning

Managing Industrial Networks

Troubleshooting to Minimize Downtime

Tips for Commissioning, Managing & Troubleshooting Your Industrial Network

Industrial Network Challenges

• Harsh operating

environments

• Network availability

requirements are much

higher than enterprise IT

• Cost of downtime is

extremely high

• Interoperability of industrial

devices/networks

• Limited networking

expertise

Typical challenges

Source:

http://www.strategiccompanies.com/pdfs/Assessing%20t

he%20Financial%20Impact%20of%20Downtime.pdf

Network Configuration &

CommissioningTips, Tricks & Tools

Network Configuration & Commisioning

Installation Configuration Troubleshooting Testing Commissioning

Typical steps

Unmanaged VS. Managed

HARDWARE

SOFTWARE

APPLICATIONSmall Scale Network

P2P Communication

Mid to Large Scale Network

Mission Critical Network with

Remote Monitoring

Packet Switching:

• Entry Level Switch ASIC

Packet Switching + Network

Management:

• Advanced Switch ASIC +

• CPU + Flash / RAM

Simple Data Switching Powerful Performance for

Network ManagementPOSITION

Plug and Play

No Configuration Required

Web / CLI Setting

• Network Security

• Network Redundancy

• Network Management

• Traffic Prioritization

Unmanaged

Switch

Managed

Switch

Network TopologyTypical Enterprise Star Topology

• Single point of failure

• Long, costly wire/fiber runs

Network ConfigurationSelecting the Right Topology for Your Needs

Redundant

Technology

Type Mesh STP RSTP Ring/Chain HSR/PRP

Feature

• Every node

connects to

each other

• IEEE

802.1D

• Loop-free

tree shape

topology

• IEEE 802.1w

• Loop-free

tree shape

topology

• Proprietary

technology

• Ring/Chain

Topology

• IEC 61850

• Dual Network (PRP)

• Dual Path (HSR)

Pros

• Highly

reliable

• Self-healing

• Open

Protocol

• Self-healing

• Open

Protocol

• Faster

recovery time:

~1 sec

• Low cost

• Self-healing

• Faster recovery

time (<20 ms)

• Open protocol

• Self-healing

• Zero recovery time

(0 ms)

Cons

• Too costly for

large network

deployment

• Recovery

time:

~15 sec

• Recovery

time not fast

enough

• Vendor specific

technology

• Prohibitively

expensive unless

absolutely needed

Backup Link

Root

Network TopologyTypical Industrial Ring Topology

• No single point of failure

• Reduced wiring costs

Industrial Protocols

• SCADA control / monitor PLC and field

devices via industrial protocols

Integration of SCADA & PLC Networks

Drive

I/O PLC

Ethernet

Switch

HMI

Network Configuration & Commissioning

• Two different methodologies for configuration of

network devices

• Many users from the industrial side prefer web

GUI

• Most users for commercial/enterprise side will

favor CLI

– Used by Cisco

Web Interface vs CLI

Device ConfigurationCommand Line Interface (CLI)

Device ConfigurationGraphical User Interface

• Visual confirmation of current settings

• Menu based configuration

• Standard web browser interface

Network Management Tools

Easy Configuration @ Installation Stage

Efficient Monitoring @ Operation Stage

Easy Backup/recovery @ Maintenance Stage

Quick Troubleshooting @ Diagnostics Stage

Mass Configuration Tools

Up to 10X Productivity Boost

One by One Setting by Web Batch Configuration by MXconfig

Multiple Devices Wiring

in Series

Broadcast Search

Group IP

Configuration

Group Redundancy

Configuration

Finish

400

sec

20

sec

200

sec

100

sec

Total

12 min

Single Power Supply

Single Device Wiring

IP Configuration

Redundancy Configuration

Repeat

100 times

Finish

10

sec

30

sec

35

sec

Total

125 min

Fast Group ConfigurationNetwork (IP address) Setting

Confidential

IP address setting for

mass devices

Fast Group Configuration802.1Q VLAN Setting

Confidential

Quick Add Panel

for cloning setting

*Mass 802.1Q VLAN Setting only for devices with the same model name

Fast Configuration DeploymentCopy Configuration

Confidential

Quick configuration copy

from one specific setting

to mass devices

Support mass IP

address setting

*Copy Configuration only for devices with the same model name

Configuration CheckStatus Overview

Confidential

Redundancy Setting

Overview802.1Q VLAN Setting

Overview

Startup Troubleshooting

Confidential21

Compare a Single Device with Whole Network

VLAN

1: Access, PVID=1, Forb=200


3: Trunk, PVID=100, Tag=1,2


VLAN





VLAN





VLAN





VLAN





VLAN





VLAN





VLAN





VLAN





VLAN





VLAN





VLAN





VLAN





VLAN





VLAN





Comparison

Sample

Benefit

Reduce Manual Setting Errors

DocumentationExport Configuration

Confidential

Export mass

configurations by

preference name

Network Management & Maintenance

Best Practices

Network Management & Maintenance

• Industrial NMS– Auto topology visualization

– Remote device management

– Real-time event management

– Comprehensive performance

reporting

Network Management Software

Confidential

Network Management & MaintenanceEfficient Visual Monitoring

Virtual Device Panel

Real-time Event

VLAN/IGMP

Visualization

CONFIGURATION CENTER

1-click for mass configuration backup and

firmware upgrade

Job scheduling for nightly configuration backup

Configuration change history

Network Management & MaintenanceSchedule Automatic Backups

• One-click Backup

– Only trigger ‘Reset’ button on switch to copy configuration and log

files to ABC-02-USB

• Files Import & Backup

– Configuration import & backup

– Firmware upgrade

– System log backup

Confidential

Rotate blinking under backup

Network Management & MaintenanceEasy Field Backup & Recovery

Potential Cyber Security Threats in Automation

• Operations disrupted by huge number of nuisance messages on network, slowing or blocking legitimate network traffic

Denial of service

• Causes computer to run attacker’s programStorage modification

• Replaces pieces of running program with attacker’s programMemory modification /Memory

Injection / SQL injection

• Attacker impersonates trusted computer, inserting itself as a middleman between trusted partner computers, modifying the messages between them to accomplish the attacker’s goals

Man-in-the-Middle

• Watches messages between computers to gain information about systemNetwork monitoring

• Gives attacker administrative privileges on systemEscalation of privilege

• Convincing users to unknowingly install malware by clicking on links, bypassing outward-directed firewallsPhishing attacks

• Attackers exploit trusting, helpful impulses of plant personnel to gain information used to bypass defenses and physical modification or sabotage of control equipment

Social engineering

Past Control

network security

• Physical perimeter security

• Air-gapping

• Security through obscurity

Maximize system

availability

• Remote access portals were added by plant engineering and vendor personnel

• Often without the acknowledge or approval by IT people

The security threat

environment has

substantially changed

• Nearly all systems are directly or indirectly connected to public networks

• Attackers are now aware of the possibilities of attacking control systems

Cyber Security Trend of Automation Network

Ref: Best practices in automation security by Murray McKay, Principal Application Engineer, Siemens Industry, Inc.

Create a Defense-in-Depth

Network Security Environment

Defense in Multiple Places

• Defend the Networks and Infrastructure (encryption and traffic flow security measures to resist passive monitoring)

• Defend the Enclave Boundaries (deploy Firewalls and Intrusion Detection to resist active network attacks)

• Defend the Computing Environment

Layered Defenses

• Each of these mechanisms must present unique obstacles to the adversary.

• Further, each should include both “protection” and “detection” measures

Confidential

The Best Countermeasure against Cyber Threats

Layered Cyber Security Solution for Automation

Security Site

• High-performance

• 500 Mbps

Security Zone

• Best Cost/Performance

• 300Mbps

Security Cell

• Best Integration

• 110 Mbps

Firmware updates

• FW updates are critical to ensuring your devices

are always up to date with the latest technology

– Includes both technology and security updates

• Many manufacturers offer free FW upgrades to

ensure their customers have longevity with the

products they have purhcased

Network Troubleshooting

Minimizing Downtime

Alerts on Unmanaged Switches

• While unmanaged switches

generally cannot communicate

status over the network, they

can be simply configured to

provide relay outputs for

alarms such as:

– Power Supply Failure

– Port Break Alarms

Monitoring System Changes

Alerts & Event Logs

Monitoring System Changes

Predictive Monitoring & AlertsComprehensive Fiber Status Monitoring and Warnings

Fiber Status Monitoring – Fiber

Temperature, Working Voltage,

Tx /Rx Powers

Auto Event Warning – SNMP

trap, Relay, Email, Event log

(DDM: Digital Diagnostics Monitoring)

SC ST SFP

All Fiber should be monitored

for fault prevention

Troubleshooting ToolsNetwork “Snapshot” Comparison Tools

• Quickly Collect Switch Info

(Take Network Snapshot)

• Quickly Compare Switch Info

(Compare Network Snapshots)

Troubleshooting ToolsEvent Playback

EVENT PLAYBACK

Record network status in 30 days

Network playback on any time/any event

Play at 1x, 2x, or 4x speed

Troubleshooting Tools

• Speed up on-site device finding to quickly diagnosis

Switch Finder

Confidential

Troubleshooting ToolsNetwork Protocol Analyzer

Thank You