![Page 1: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/1.jpg)
Operational Excellence Data Center Solutions Day, 5 July 2013, Martin Hogberg
Increasing Uptime in Data Centers by
Addressing Human Factors
![Page 2: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/2.jpg)
Human Factors It’s not About Pointing Fingers
Mike, are you
racking the new
server?
Yea, some guy
installed more
power last night.
Ok put it in. Right, here
we go!
![Page 3: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/3.jpg)
Common Issues in the NOC
![Page 4: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/4.jpg)
Why is it important to focus on the operator?
1. Rick Schuknecht, The 451 Group Uptime Institute, June 2011
Availability/Downtime 75% of downtime due to human error1
• Over the total life cycle, most important decisions are made by the
operator.
• The amount of information from the process has increased dramatically
with risk for stress in critical situations.
• Experience and operator skills are invaluable assets.
• It is difficult and expensive to recruit and train new operators among the
young generation.
![Page 5: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/5.jpg)
Traditional NOC environments Issues we are seeing across every industry
• A complex environment with a multitude of different systems and a low level of
integration.
• Very difficult to get an overview of what’s going on in the data center and what’s
about to happen.
• Lack of attention to human factors result in an unhealthy working environment.
![Page 6: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/6.jpg)
Traditional NOC environments Issues that degrade reliability and profitability
• Poorly integrated CCTV and telecom equipment.
• Too many keyboards and other input devices.
• None ergonomic operator desks with built-in computers that generate heat,
noise and are difficult to keep clean.
![Page 7: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/7.jpg)
Newer control rooms with traditional approach Issues that degrade safety, quality and profitability
• A newer centralized control
room built according to
traditional principles.
• Very low level of
integration results in total
screen and keyboard
overload.
• Overview information
screens very difficult to
view.
![Page 8: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/8.jpg)
Operations Environment
![Page 9: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/9.jpg)
University evaluating the positive alertness effect by working with Modern Concepts In the spring of
2012 Chalmers
has been testing
the
effectiveness of
the EOW, and
compared it to
the traditional
way of working
with control
room design. To
facilitate the test
Chalmers has I
corporation with
ABB created a
simulation of a
“color factory”
State of the art ergonomic and high
level of integration environment
![Page 10: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/10.jpg)
Research Trials Traditional Operations
Basic level of integration that fulfill
standard traditional requirements
![Page 11: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/11.jpg)
How to measure the alertness effect ?
Chalmers testning the Eow Concept
The tests has been conducted but using subjective and
Objective measuring tools as well as interviews
The subjective Tools measure
the perceived comfort and stress
level of the operator.
The SAM form one of several different
forms that use to evaluate the result.
The objective tools measure
the reaction time, cursor path
and number of problems
resolved
The specially developed facility
![Page 12: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/12.jpg)
How to measure the alertness effect?
Chalmers evaluating the effect of the EOW Concept
Basic level of integration
that fulfill standard and
traditional requirements
Subjective Objective Interviews
State of the art
ergonomic and high
level of integration
environment
Subjective Objective Interviews Analysis and evaluations of
operators
environment and
technology solutions
resulting in expected
findings of
5-10 % increased operator
effectiveness
![Page 13: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/13.jpg)
Integrated operations High information density and ergonomic workplaces
Interactive, close large overview ”owned” by the NOC operators
Easy to use individual ergonomics
Built-in communication tools simplifies collaboration
Saves up to 30% floor space compared to traditional solutions
![Page 14: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/14.jpg)
Adaptive NOC Environment
Extended Operator Workplace as a centerpiece
of control room.
Cover all operator needs from one
console.
Futuristic NOC equipment.
Desk raises up on critical alarms so
Operator has to stand to deal with
situation.
Blows cold air to maintain Operator
alertness.
Sound shower directs operator through
audible alarms.
Can link to lights & window-blinds.
![Page 15: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/15.jpg)
Going beyond the operator workplace
Overall control room environment is critical
Dedicated space for control, removes distractions and focus operators
on task of controlling the facility.
Visitor area to keep non-essential personnel out of the control area
Collaboration space. Meetings, troubleshooting, problem solving with
A/V tied to control center visualization.
Relaxation area for extended shift recharging, separate eating area to
avoid noise, segregated printers to remove noise
![Page 16: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/16.jpg)
Helping the Teams and Communication - Integration
![Page 17: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/17.jpg)
Data Centers – Management Today
Isolated Silos with Individual Planning,
Applications and Reporting
System and Network Management, Servers, Network and Storage
Building, Power and Environmental Management
Change Management, Help Desks and Trouble
Tickets
Integrated Reporting: Monthly Spreadsheets and Slides
IT Facility Operations
Utility Services
![Page 18: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/18.jpg)
ABB Decathlon™ System
Security
Switchgear RDU CRAC VESDA/NOVEC
Chiller
Control
Fire
Protection
![Page 19: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/19.jpg)
Integration Command Center – Single Pane of Glass
Alarm List
Trend Display Documentation
Faceplate
Navigation
Display Operator Note
One display management system
Consistent methods for navigation and display access
Supports power, cooling, and IT
Provides local and remote display access
Improves operational efficiency with better visibility
Creating a Collaboration Culture
CCTV
![Page 20: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/20.jpg)
IT, planning the moves adds
changes.
Utility & Facility, verifying the
necessary space, power and
cooling is available.
Possible though an integrated
system.
Integration of Facility and IT Work Faster and Avoid the Mistakes
Creating a Collaboration Culture
![Page 21: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/21.jpg)
Effective Software which Supports the Operator
![Page 22: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/22.jpg)
Reduce the Isolated Systems Information Kept With the Asset
Data center based information
access
One Click to all information
Navigation based on job function
Real-time decisions and action
Control
Faceplates
Trend
Graphics
Video
Reports
Documentation ERP/ CMMS
Asset
Health
![Page 23: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/23.jpg)
Integrated Information – CRAC Example
Trends
Alarm Instructions
Documentation
Operator Note
![Page 24: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/24.jpg)
Overview of Effective Data Center HMI
High Performance Design
User Interface - The human performance improvement is
supported by selection of color scheme, shape of elements
and icons and tab based navigation.
Addressing Alarm Flooding - The effectiveness of new
operator interface concepts for improving operators’ ability
to handle data center disturbances that generate alarm
floods.
![Page 25: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/25.jpg)
Integrated Alarm Management Alarm Management - A key concern for data centers
Too Many Alarms! Operators acknowledge/silence
alarms without looking at or
acting on them.
Incidents or near-incidents
where operators missed alarms.
Acoustic alarms turned off.
Operators don't know what
particular alarms mean.
Alarms disabled/suppressed for
long periods without review.
© ABB / PA / Control Systems
July 8, 2013 | Slide 25
![Page 26: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/26.jpg)
Integrated Alarm Management Mission-Critical performance
View alarms from cooling, power, and IT in a
single list master list as well as individual lists
sorted chronologically, by priority, or by equipment
and filtered by location, functionality, or network
connectivity
Embedded analysis tools for determining the most
frequent offenders, the ones that take the longest
to remedy, or those that result in the greatest
downtime
![Page 27: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/27.jpg)
Integrated Alarm Management
Enables a starting point for alarm rationalization
Analysis of ”worst cases”
Allows for continuous analysis and optimization
View alarms from cooling, power, and IT in a single list
sorted chronologically, by priority, or by equipment and
filtered by location, functionality, or network connectivity
![Page 28: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/28.jpg)
Consistent alarm-response navigation
Fast navigation from an alarm reveals detailed information
needed to handle it correctly.
© ABB / PA / Control Technologies July 8, 2013 | Slide 28 3BSE068040 en A
![Page 29: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/29.jpg)
Alarm grouping replaces long lists
Damage limitation no longer depends on an individual’s
ability to ‘piece together’ information from several
simultaneous alarms
© ABB / PA / Control Technologies July 8, 2013 | Slide 29 3BSE068040 en A
![Page 30: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/30.jpg)
Condition Based Maintenance
The System Guides the User
Common format to determine faults in real time including Description of fault Possible cause Suggested action Priority / Severity of fault
SMS / e-mail messaging
![Page 31: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/31.jpg)
Online snapshot reports for better change management Reduces time for shift handovers and maintenance stops
Reduced time to get right information Thanks to the integration capabilities of the system
the ”snapshot” covers all DDC, PLC’s etc regardless of brand
Enables real-time shift reports for example Navigation with the context menu that’s accessible from the
report
![Page 32: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/32.jpg)
Decathlon Users Future Operator
![Page 33: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/33.jpg)
Future Operator - Possibilities
Augmented reality
3D visualization
Gestures
Social media
Sensor technologies
Mobile devices
Touch
Depth cameras
Eye-tracking
© ABB Inc
3BSE074371 en.
July 8, 2013 | Slide 33
![Page 34: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/34.jpg)
Future Operator – Visualizing in 3D When it makes sense
Before After
![Page 35: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/35.jpg)
Future Operator - Controlling by Eye-Tracking Gesture Interaction
![Page 36: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/36.jpg)
Future Maintenance Engineer Augmented Reality
A coffee cup tag shows
information related to the
coffee cup
An ABB logo shows the
latest news related to
Decathlon
A factory logo brings up a
3D plant
![Page 37: Common Issues in the NOC - ABB GroupFILE/ATTBK4D0.pdf/05.… · Fire Protection . Integration Command Center – Single Pane of Glass Alarm List Trend Display Documentation Faceplate](https://reader033.vdocuments.net/reader033/viewer/2022051719/5a7337be7f8b9aa7538e5c61/html5/thumbnails/37.jpg)