avoiding being overwhelmed
TRANSCRIPT
-
8/9/2019 Avoiding Being Overwhelmed
1/35
Avoiding Being Overwhelmed By
Connectivity Complexity
David Cuthbertson
-
8/9/2019 Avoiding Being Overwhelmed
2/35
Overview Understand the Problem(s)
Standards can help or hinder New technolo challen es
Organisational challenges
Case Study(s)
-
8/9/2019 Avoiding Being Overwhelmed
3/35
Personal Back roundPersonal Experience Industry Groups and Frameworks
Network
Troubleshooting
Infrastructure Management
Installations
Managed Services
am ngLabelling
VisualizationCommunication
ase n ng
Change Process
SkillsAwareness Toolsets
-
8/9/2019 Avoiding Being Overwhelmed
4/35
Chan e is ConstantInformation requirements force technical change
Information and Data
Applications
Virtual
Hardware
Physical Infrastructure
Infrastructure responds to change
-
8/9/2019 Avoiding Being Overwhelmed
5/35
Chan e Affects Mana ementGovernance & control force organizational change
RegulatorsSecurityOperationsProjects Management
Do it faster Consolidate
/ Optimize
Change
Records
No
Downtime!
Data Loss
costs Reporting
Partitioning Use of
Partners
Prevention
Physical Infrastructure Management
Practice and processes are impacted
-
8/9/2019 Avoiding Being Overwhelmed
6/35
How Do We Co e? Adopt industry best practices where they exist
Ensure clarity on roles and practices
Make the complexity easier to manage
Clear naming, labelling and documentation
Reduce the number of data sets needed for governance and control
-
8/9/2019 Avoiding Being Overwhelmed
7/35
The Case StudTwo data centers (co-lo)
- approx 400 rac s
- 36200 fiber/copper connections
- 5800 patches
- 600 servers
4 spreadsheets inventory + connectivity per DC
Im just over loaded and can t get others to understand
(>1.8 million cells) + >6000 other spreadsheets
Maintained by a data center manager no onsite staff
-
8/9/2019 Avoiding Being Overwhelmed
8/35
1. Is every device in the data centers or equipment rooms currently documented,
along with how it is connected in a system that supports both planning and
operations information needs? NO
2. Does existing connectivity documentation for data centers and shared equipment
rooms use consistent naming conventions across all locations, devices and
connections? DONT KNOW
. re up ates to nventory an connect v ty ata up cate n sprea s eets,
diagrams and databases in a consistent way? PARTIALLY
When planning choices are confirmed
When devices are added / removed
With changes in backbone cabling or patching
4. Can detailed work packages be created for internal or external engineers, without
,
-
8/9/2019 Avoiding Being Overwhelmed
9/35
The Im act On O erations Increasing project costs and delays
ver oa ng o ey eng neers n v ua s
Reverse engineering with workshops, site surveys, etc
The human factor (guess, assume, worry, forget, etc.)
Strategic initiatives often limited in scope and benefit
Asset mana ement doesnt reall hel DC mana ement
Localised attempts to retain/manage knowledge of
confi urations s readsheets, dia rams
-
8/9/2019 Avoiding Being Overwhelmed
10/35
Information SetsDocuments
Project plans
Design and specifications
Commissioning documents / contractsManuals and procedures
Lists
Inventory actual and planned
Connectivity paths, end points
Capacity space, power, connectivity
Support warranty, maintenance
Backbone cabling - physical, logical
Topology Network, SAN, voice, power
Layouts - Floor plans, racks, cable tray
or ow anges, n er aces
-
8/9/2019 Avoiding Being Overwhelmed
11/35
Re uestin a Connection -1SW-BHAM-09
Device DeviceConnection
Port Port23
CAT6A CopperNIC 1
Device Port Connection Port Device
SW-BHAM-09 23 To be filled in by DC team NIC 1 UK_BIRM_BLADE-03
This was the initial spreadsheet used to capture connectivity requests
-
8/9/2019 Avoiding Being Overwhelmed
12/35
Re uestin a Connection -2SW-BHAM-09
Device DeviceConnection
Port Port23
CAT6A CopperNIC 1
Device Port Connection Port Device
SW-BHAM-09 23 To be filled in by DC team NIC 1 UK_BIRM_BLADE-03
Room RoomPort Type Colour Port Type
PositionMake
Model
Asset Number
PositionMake
Model
Asset Number
VLAN
Connection typeLength
VLAN
The spreadsheet justIP A ress IP A ressw u w
-
8/9/2019 Avoiding Being Overwhelmed
13/35
Namin & Labellin
-
8/9/2019 Avoiding Being Overwhelmed
14/35
Ports and ConnectionsEquipment use the physical label?
Port name
- 1 or 01 or 001?
- 2/1 2\1 2/01 SL2/1 Port 2/1 Gig 2/1 Fe2/1 Slot 2/09
- Mgmt MGT Con Console ILO Net Mgmt-
Cable Labels
1) port 2) local devices 3) end devices4) full path 5) cable unique id 6) path unique ID
-
8/9/2019 Avoiding Being Overwhelmed
15/35
And Which Cablin To olo ?Top Of Rack
1 to 5 patches
-
8/9/2019 Avoiding Being Overwhelmed
16/35
Multi- ath O tionsNetwork
N01
Zone A
N02
Zone B
N03
Zone C
N04
Zone D
ac s
- - - -- - - -
01-06 07-12 13-18 19-24 01-06 07-12 13-18 19-24
RACK
A01
RACK
A02
Server Racks
-
8/9/2019 Avoiding Being Overwhelmed
17/35
Ver Multi- ath
One rack @ 6 Units = 288 modules = 1728 ports
-
8/9/2019 Avoiding Being Overwhelmed
18/35
So Increase The S readsheet SizeDevice Port Patch PPA
Port
Patch
Panel
Back
to
Patch
Panel
PPB
Port
Patch Port Device
A Back B
A 23 05 AA AC 05 NIC 1 B
Room
Rack
Room
Rack
Port Type
S eed
Colour
Cable Label
Port Type
S eed
Port Type
S eed
Port Type
S eed
Type
Len th
Colour
Cable Label
Room
Rack
Room
Rack
Position
Make
Model
Asset Number
IP Address
Position
Make
Model
Asset Number
IP Address
VLAN Type
Length
VLANType
Length
Position
Make
Model
Position
Make
Model
Contact
Purpose
Request date
Work number
Path reference
Scheduled date
The spreadsheet data has now
u u
Project Code / Ref
Path length
grown to up to 60 columns
wide
-
8/9/2019 Avoiding Being Overwhelmed
19/35
But With Structured Wirin4. Services3. Service2. Zone and1. Cabinet
PresentationPatchingBackbone CablingPatching
Network
Patch PatchPatch
Patch
Panel
Panel PanelPanel
Copper
Storage
KVMPatch
Panel
Patch
Panel Fiber
It makes sense to present services in patch panels
-
8/9/2019 Avoiding Being Overwhelmed
20/35
Expand The Spreadsheet(s) a Bit More
P A BTo
B P C BTo
D P E Bto
A 23 05 AA AB 05 13 25 B3/04 3/04 B
Switch Ports
Presented
In Patch Panel
As common practice is to present Core
infrastructure services at cross
connects.
So you may be connecting to a service
,
-
8/9/2019 Avoiding Being Overwhelmed
21/35
-
8/9/2019 Avoiding Being Overwhelmed
22/35
Define Active Device NaminDevice Name
1) Logical Name US-SW-DC02-03P
FANSTATUS
1
WS-X6K-SUP1A-2GE
SUPERVISOR1
STATUS
SYSTEM
ACTIVE
PWRMGMT
RESET
CONSOLE
DTE/DCE
SWITCH LOAD
100%
1%
PORT1
P CM CI A E JE CT
PORT 2
LINK
LINK
SW-BHAM-CORE1.Card1
2) Type of device / location Cisco 6509 DC02-B03-U2
3) Asset Number Asset HW0078732
2
3
4
5
6
SW-BHAM-CORE1.Card3
STATUS
WS-X6748-GE-TX47
48
37
38
35
36
25
26
23
24
13
14
11
12
1
2
48 P O R T
12119 107 85 63 41 2 24232
1 2219 2017 1815 1613 14 363533 3431 3229 3027 2825 26 48474
5 4643 4441 4239 4037 38
10/100/1000
GEMOD
FABRICENABLED
RJ45
SW-BHAM-CORE1.Card4
STATUS
WS-X6748-GE-TX47
48
37
38
35
36
25
26
23
24
13
14
11
12
1
2
48 P O R T
12119 1 07 85 63 41 2 242
321 2219 2017 1815 1613 14 363533 3431 3229 3027 2825 26 484
745 4643 4441 4239 4037 38
10/100/1000
GEMOD
FABRICENABLEDRJ45
SW-BHAM-CORE1.Card5
Device component Switch CardUS-SW-DC02-03P Card3
Cisco 6509 DC02-B03-U2.Slot03
C6509-E-FAN
7
8
9
WS-C6509-E
INPUT1100-240V~16A60/50Hz
INPUT1100-240V~16A60/50Hz
SW-BHAM-CORE1.Card9
Asset HW0078737
1
FANOK
OUTPUTFAIL
2
INPUTOK
SWITCH MUSTBE IN OFF O POSITION TO INSTALL/
REMOVE POWER SUPPLY. FASTENER MUST BE FULLY
ENGAGED PRIOR TO OPERATING POWER SUPPLY.
CISCOSYSTEMS,INC.
INPUT2100-240V~16A60/50Hz
INSTAL
L
RUN
1
FANOK
OUTPUTFAIL
2
INPUTOK
SWITCH MUST BE IN OFF O POSITION TO INSTALL/
REMOVE POWER SUPPLY. FASTENER MUST BE FULLY
ENGAGED PRIOR TO OPERATING POWER SUPPLY.
CISCOSYSTEMS,INC.
INPUT2100-240V~16A60/50Hz
INSTAL
L
RUN
SW-BHAM-CORE1
UX-NY0445-PROD
HP BL685C BLNY05-DC02-B03-U2.Slot03
Asset HW0078143
-
8/9/2019 Avoiding Being Overwhelmed
23/35
Just Make the S readsheet Bi er!P A B
ToB P C B
ToD P E B
to
A 23 05 AA AB 05 13 25 3
/04
3/
04
B
1
2
3
4
5
6
7
8
17
18
19
20
9
10
11
12
13
14
15
16
1
2
3
4
5
6
7
8
17
18
19
20
9
10
11
12
13
14
15
16
1
2
3
4
5
6
7
8
17
18
19
20
9
10
11
12
13
14
15
16
1
2
3
4
5
6
7
8
17
18
19
20
9
10
11
12
13
14
15
16
21
22
23
24
21
22
23
24
21
22
23
24
21
22
23
24
The spreadsheet(s) become un-manageablea) Too many blank cells
b) Too big to easily comprehend and checkc) Updating is a nightmare to ensure consistency and accuracy
d) Lots of manual cutting/pasting to other spreadsheets
-
8/9/2019 Avoiding Being Overwhelmed
24/35
And To Mana e The Re uestData Center New
Equipment Equipment
Placement
Equipment Positioning
Y
New
Infrastructure
?
Assess
Space /Power
Connectivity
N
N
Allocate
location
Create
Work Order
New
Infrastructure
Build
Design
Install
Allocate
location
Path(s)
Allocate
PP ports
Create
Work Order
Install
& Test
Patch
& Test Path
Allocate
Resources
Patch to
Equipment
Installed
And Working
ew n ras ruc ure
Connectivity
Test
-
8/9/2019 Avoiding Being Overwhelmed
25/35
Define Infrastructure Namin1 2 3 4 5 6 7 8 17 18 19 20 21 22 23 249 10 1 1 1 2 13 1 4 1 5 1 61 2 3 4 5 6 7 8 17 18 19 20 21 22 23 249 10 1 1 1 2 13 1 4 1 5 1 6
AB
B03-ABB03-5
US-NY-DC05-H1-B03-508
09
10
11
12
08
09
10
11
12
AB/
AB/
AB/
AB/
AB/
BC/
BC/
BC/
BC/
BC/
CD/
CD/
CD/
CD/
CD/
24 Port Copper Patch Panel
PP B03-AB-U5
PP B03-AB-U5 to H07-AC-U2
PP B03-AB-U5 to H07-AC-U2/H06-AG-U9
PPC B03-AB-U5 to H07-AC-U2:Ports 01 to 24
01
02
03
04
05
06
01
02
03
04
05
06
AB/
AB/
AB/
AB/
AB/
AB/
AB/
BC/
BC/
BC/
BC/
BC/
BC/
BC/
CD/
CD/
CD/
CD/
CD/
CD/
CD/
288 Port Fibre Panel
or e ec onAB-A Ports 1-6H07-AC-B Ports 7-12
PCI AB-A Ports 1-6 to H07-AC-B Port 7-126 Copper RJ45 Ports
AC-03B Ports 1-6
PFI B03-AB-03B Ports 1-6 to H07-AC-12A Ports 1-6
6 Fiber LC Ports
-
8/9/2019 Avoiding Being Overwhelmed
26/35
Infrastructure Data Issues1. Names and labelling of devices, ports, patch panels are often
inconsistent. Keep them as simple as possible.
Use external standards if helpful TIA606B, TIA942
As equipment vendors use various names for ports, decide on your standard
Labels and administration identifiers may be different!
2. Working practices will dictate when an update is carried out
Re uest desi n issue desi n docs make chan e com lete
A spreadsheet will be locked by one user, so look at a database option
Capacity reporting, device views
Diagrams Rack layout, backbone architectures
-
8/9/2019 Avoiding Being Overwhelmed
27/35
In PracticeIn an estate of about 400 racks
32, 000 copper / 22, 000 fiber backbone connections
9,000 paths
,
Master spreadsheet(s) with connection paths 9,000 rowsMaster spreadsheet(s) for backbone 54,000 rows
Approx. 4000 change spreadsheets identified 300,000 rows
Switch ort allocations a tab er switch /card 16,000 rows
San switch port allocations a tab per SAN switch
Plus asset list(s), 800 rack layouts, backbone diagrams
-
8/9/2019 Avoiding Being Overwhelmed
28/35
Some Reasons To Chan e
Faster implementation time demanded
Improve switch resource utilisation Service provider / partner labor costs
Individual overload / resignation
Failed audit / compliance
Fewer accidents mistakes
Upgrades / refurbishment
Physical connectivity data highlights the current understanding of inventory
and dependency. An easy target for auditors!
-
8/9/2019 Avoiding Being Overwhelmed
29/35
In Hindsi ht
1. Maybe a spreadsheet wasnt the best way to start
- It gives structure, but isnt best suited to data center connectivity
- Loading a database is 3 spreadsheets - 1)racks 2)devices 3)connections
2. One day you will have to establish a baseline inventory and
maintain it before you can manage connectivity- Inaccuracies in data / naming are often present in other systems
- Without resolving naming issues, audits/ data capture may be wasted
3. Managing connectivity has to cope with the real world
- Complex device build switches, servers, cabling distribution, pre-
term nate cassettes, , own re, verse pat s, etc.
-
8/9/2019 Avoiding Being Overwhelmed
30/35
The Bi er Picture Think Forward
Physical locations
Physical connectivity Logical connectivity
Rac capacity
Port capacity
Asset lists Project plans
Work orders
Monitoring toolsets
-
8/9/2019 Avoiding Being Overwhelmed
31/35
Reducin Data Sets Tomorrow
Excel Visio
Floor box list Floor plan
Patch panel list Backbone cabling diagram
Inventory Network diagram
Inventory Rack diagram
Inventory Server connectivity diagram
-
8/9/2019 Avoiding Being Overwhelmed
32/35
Exam le Link Visio to Data
Embed device / link data
Standard Visio Feature!
-
8/9/2019 Avoiding Being Overwhelmed
33/35
A More Strate ic A roach
Infrastructure
Database(s)
&
Reporting
-
8/9/2019 Avoiding Being Overwhelmed
34/35
In Summary - When?
1. Every device in the data center and how it is connected is currently documented
2. Updates to inventory and connectivity data are duplicated in spreadsheets,
When devices are added / removed
With changes in backbone cabling or patching
3. Detai e wor pac ages or ata center c anges can e create or interna or
external engineers, without visiting site beforehand or being there during
changes.
4. Current connectivity documentation for data centers and shared equipment
rooms uses consistent naming conventions across all locations, devices and
connections
-
8/9/2019 Avoiding Being Overwhelmed
35/35
Additional Material
www.tiaonline.org ANSI/TIA568C, ANSI/TIA606B, ANSI/TIA942
www.bicsi.org ANSI/BICSI-002
www.squaremilesystems.comFree SMS Visio utilities Downloads and videos
Webinars/videos Visio automation, documenting cabling, etc.
www.assetgen.comEvaluation software Free DCIM/CMS evaluation version
,