Cognitive SystemsCognitive Systems
Session ID:
Agenda Key:
External Storage for IBM iKris Whitney
13AD
Systems Management
@i_Virtualize
Cognitive Systems
© Copyright IBM Corporation 2017.
Agenda
• What’s different about IBM i, and why should I care?• Why move to external storage?• Performance • High Availability and Disaster Recovery• Virtualization• Systems Management• Encryption• IBM Disk Product Portfolio for IBM i• Summary
Cognitive Systems
© Copyright IBM Corporation 2017.
IBM i Storage Architecture
3
• Built on DB2 with heavy exploitation of Data in Memory
• Built originally on disk block size of 520 bytes
• Data is spread across all disks in disk pool
• Designed for ease of use and self-management
• Why does this matter when choosing a storage system?• Little skew • Random access patterns• Cache ‘unfriendly’• All-or-nothing copies (full system
copy)
Cognitive Systems
© Copyright IBM Corporation 2017.
IBM i Single Page (4K) I/O with different Sector Sizes
5128
8512
5128
8512
5128
8512
5128
8512
Header
512
512
512
512
512
512
512
512
512
64
4096
64
Partially Compressed4096
8 x 520 Byte
Sectors 9 x 512 Byte
Sectors
1 x 4160 Byte Sector
1 x 4096 Byte Sector
IBM i 7.2 and later
4
Cognitive Systems
© Copyright IBM Corporation 2017.
Storage attach terminology
• Native attach – consists of one of two methods• Direct attach – switch or fabric device is not required between the adapter and the
storage device • Fabric attach – switch or fabric device is required between the adapter and the storage
device
• Virtual attach – consists of one of two methods• Virtual SCSI – A virtual I/O server (VIOS or IBM i) provides a virtual SCSI adapter and
virtual SCSI storage devices to a client operating system (IBM i, AIX, or Linux for VIOS; IBM i, AIX, Linux, or Windows for IBM i)
• N_Port ID Virtualization (NPIV) – aka virtual fibre channel (VFC) – A virtual I/O server (VIOS) provides a virtual fibre channel adapter to the client operating system (IBM i, AIX, or Linux)
Cognitive Systems
© Copyright IBM Corporation 2017.
XIV
DS8000
Native
VIOS (virtual) Any supported storage
IBM i Native
VIOS
Power System
Storwize V7000and SVC
NPIV
NPIV
vSCSI
IBM FlashSystem
NPIV
Storage Options
Cognitive Systems
© Copyright IBM Corporation 2017.
7
Why external disk?
High AvailabilityDisaster Recovery
Consolidation
Encryption Centralized Storage
Management
VirtualizationPerformance
and Flexibility
Cognitive Systems
8
Performance
Performance
Cognitive Systems
© Copyright IBM Corporation 2017.
Performance Objectives
• Provide predictable and consistent performance at the most cost effective point• Easy performance management/minimize tuning• Easy performance reporting
Cognitive Systems
© Copyright IBM Corporation 2017.
IBM Disk Magic - Performance planning Integration
IBM/BP delivered custom performance and capacity sizing for your workloads
IBM i Performance Tools Report Data
POWER Workload Estimator
Manual Input
Modeled Workload
Output
Cognitive Systems
© Copyright IBM Corporation 2017.
Memory and IO Hierachy
DRAM
Flash
Enterprise Disk
Nearline Disk
Cost
SpeedNano seconds
Micro seconds
Milliseconds
Cognitive Systems
© Copyright IBM Corporation 2017.
Source: Worldwide External Enterprise Storage Systems Forecast, 2016–2020, Publication Date: IDC Jul , 2016
Future view of the All-flash and Hybrid-flash market …
All Hard Disk Drive
Hybrid Flash
All Flash
5.64% 11.20% 16.28% 20.38% 24.05% 27.38% 29.28%
42.11% 41.70% 42.64% 43.85% 44.66% 45.63% 45.25%
IBM & IBM Business Partner Use Only
Cognitive Systems
© Copyright IBM Corporation 2017.
IBM System Storage Easy Tier
– Places data on the correct hardware tier• Optimize your hardware investment • Maximize application performance
– Automated and granular: • Automated movement of data based on actual workload
performance
– Virtually no administrative effort: • Easy Tier optimizes storage tiering without laborious manual
tuning
– Supported on DS8000, Spectrum Virtualize, and Storwize family
• Slightly different implementations
New generation of smart storage that intelligently adapts to application workloads
Cognitive Systems
© Copyright IBM Corporation 2017.
IO pattern
§ Easy Tier learning of application IO patterns –Heat Map
§ Migration plan§ 24 hour cycle for
inter-tier migration§ 6 hour cycle for
intra-tier migrations
§ Extent migration –background task
Heatmap and I/O Density Report
Solid-state
Enterprise - FC / SAS
Nearline - SAS / SATA
Migration Plan
Easy Tier - Workload Learning
Cognitive Systems
© Copyright IBM Corporation 2017.
Easy Tier Performance with IBM i
DS8700 600GB 10K rpm disks plus SSD managed by Easy Tier
Cognitive Systems
© Copyright IBM Corporation 2017.
IBM i Exploitation of SSD with DS8000
• IBM i can recognize SSD-backed LUNs on DS8000• Manually – ASP placement• DB2 Media Preference• ASP Balancer • BRMS Migration
• Easy Tier• Transparent to server• Minimal implementation effort
Best option
Best option
Cognitive Systems
© Copyright IBM Corporation 2017.
Disk Performance on IBM i
• Understand your workload• Use the sizing tools (Work with IBM or Business Partner)
• Disk Magic• Sue Baker’s IOA sizing spreadsheet
• Plan logical configuration as well as hardware configuration• No measureable difference in performance between native, NPIV, and VIOS
attachments• Isolation or sharing?
• Balance IO activity across all resources• Stripe and share everything
• Isolate IBM i workloads for performance protection• Easy Tier can work well with IBM i
• Use minimum 20% Flash capacity guideline
Cognitive Systems
18
High Availability and Disaster Recovery
High AvailabilityDisaster Recovery
Cognitive Systems
© Copyright IBM Corporation 2017.
High Availability and Disaster Recovery
• Increase system up-time• Reduce backup times• Reduce planned outages
• Maintenance windows, Backup windows, Software upgrades• Reduce time to recover in the event of a Disaster (RTO)• Reduce RPO in the event of a disaster• Increase backup frequency• Automate Recovery• Reduce overheads of Disaster Recovery provision
• Combined HA/DR, Reduce costs, Reduce manpower
Cognitive Systems
© Copyright IBM Corporation 2017.
Copy Services
• Point in Time copy – within the same storage system• FlashCopy• Snapshot
• Remote Replication – between storage systems• Synchronous (for campus/metro distances)• Asyncronous (for unlimited distance)
• Local Replication • Spectrum Virtualize Stretch Cluster• vDisk Mirroring• Hyperswap
Cognitive Systems
© Copyright IBM Corporation 2017.
Copy Services Architectures for IBM i
21
Full Copy resiliency– Full system copy – Required IPL – ‘crash’ consistency– Manual operation or toolkit
management
Network
PROD (source) HA/DR (target)
LPAR-1
Ext. Storage
LPAR-2
Ext. Storage
iASP based replication– PowerHA and copy services– PowerHA LUN Level Switching– Clustering and E2E automation– IASPs eliminate need for IPL – HA, DR and Backups
PowerHA
Network
PROD (source) HA/DR (target)
LPAR-1
Ext. Storage Ext. Storage
LPAR-2
IASP IASP
Cognitive Systems
© Copyright IBM Corporation 2017.
Point-in-Time Copy
Tape System
ProdLUNs
FlashCopy
BRMS
Thin provisioned copies, multiple targets, flash-back capability
Production System/LPAR Backup System/LPAR
Automation available from IBM lab services for DS8000, Spectrum Virtualize, and XIV
Cognitive Systems
© Copyright IBM Corporation 2017.
Flashcopy for backup timeline
Production processing
Production processing
Quiescesystem
Quiescesystem
IPL resumeProduction processing
IPL resumeProduction processing
Backup to tape
Backup to tapeIPL Backupsystem
Flashcopy
Traditional Backup
Backup with FlashCopy
Cognitive Systems
© Copyright IBM Corporation 2017.
Full System Copy
24
Tape System
FullSystem
Full SystemTarget
Local site
Remote Copy
Metro Mirror (or Global Mirror)
Full System
Flash
BRMS
Backup System i will require an abnormal IPL for unplanned outages.
Planned outages require an IPL of the Backup LPAR
Production Site Backup Site
Automation available from IBM lab services
Cognitive Systems
© Copyright IBM Corporation 2017.
Power HA with Storage Replication
Admin domain
IASPSysbase Sysbase
Production LPAR
HA/DRLPAR
Replication
• Only the IASP is replicated/switchable• Replication target is offline until failover when it is varied online to the HA/DR system• Each IBM i LPAR is running it’s own version of the System (SYSBAS)
Cognitive Systems
© Copyright IBM Corporation 2017.
ü Comprehensive HA/DR solutions for multi-site or data centerü Shared storage clustering technologyü Provides automation for ease of use ü Implemented in IBM i, SLIC, and storage management ü Data replication is provided by IBM storage ü FlashCopy enables off line tape backupü Designed for regular role swap operations and automated failoversü Developed and supported by IBM
PowerHA for IBM i
Cognitive Systems
© Copyright IBM Corporation 2017.
PowerHA Tools for IBM i
§ Complement and extend PowerHA and IBM Storage capabilities for HA/DR
§ Helps reduce business risk and improve resiliency for critical applications
§ Simplifies set up and automation of HA/DR and backup solutions
§ Reduces cost of maintaining and regular testing of an HA/DR environment
§ Facilitates flexible deployment options for single or multi-site protection
§ Assures consistent deployment using best practices and experienced consultants
PowerHA Tools for IBM i is a service offering from IBM Systems Lab Services
Cognitive Systems
© Copyright IBM Corporation 2017.
HyperSwap – IBM i 7.2
Metro Mirror Connection
Active Path
The IBM i OS stays up during the switch.
The first stage is with the entire OS
28
Cognitive Systems
© Copyright IBM Corporation 2017.
HyperSwap with LPM – IBM i 7.2
Metro Mirror Connection
Active Path Inactive Path
After LPM, the affinity to the storage box might be non-optimal
Active Path
After the HyperSwap the storage affinity can be maintained
29
Cognitive Systems
© Copyright IBM Corporation 2017.
Copy services selection
RPO
Bandwidth efficiency
DS8000 GM
Metro Mirror
Spectrum VirtualizeGM
Spectrum VirtualizeGM with Change Volumes
DS8000 GM
Spectrum VirtualizeGM with Change Volumes
Spectrum VirtualizeGM
Metro Mirror
Cognitive Systems
© Copyright IBM Corporation 2017.
Copy services selection
DS8000
3-site replication
Spectrum Virtualize
Virtualizeexternal disk
Power HAMulti-target
Toolkit Support
Hyperswap
Multi-session
Cognitive Systems
© Copyright IBM Corporation 2017.
Replication Options
DS8000 Storwize family XIV
Point in time copy Yes Yes Yes
Async Remote Copy Yes Yes Yes
Metro Mirroring Yes Yes Yes
Power HA Support Yes Yes No
Full System Toolkit Yes Yes Yes
Hyper-Swap Yes Yes No
RPO less than 3-5 seconds
Yes No No
RPO Less than 1 minute
Yes ‘Classic’ GM with unconstrained bandwidth only
Yes
RPO Less than 10 minutes
Yes Yes (GM with Change Vols) Yes
3 site Replication Yes No Yes
Multi-target PPRC Yes No Yes
Cognitive Systems
33
Virtualization
Virtualization
Cognitive Systems
© Copyright IBM Corporation 2017.
Virtualized architecture
Power Server
Disks/Logical Units(LUNs)
SSD/HPFE/Flash
300GB HDD (disk drive)
600GB HDD
IOA
Extent/Grain
Storage Pool /Extent pool
Storage system
Cognitive Systems
© Copyright IBM Corporation 2017.
Spectrum Virtualize
SAN
VirtualDisk
VirtualDisk
VirtualDisk
VirtualDisk
Storage PoolHDS EMC NetApp
IBM HPFlash
Oracle
Internal disk
Manage the storage pool
from a central point
Combine the capacity from
multiple arrays into pools of
storage
Make changes to the storage
without disrupting host applications
StorwizeVirtualization
Apply common services across diverse storage
Enable Real Time
Compression for all storage
Automated use of FlashSystem/SSD through tiering with Easy Tier
Enable encryption for all
storage
Cognitive Systems
© Copyright IBM Corporation 2017.
IBM i File System
Used Space Free Space
1. Object deleted2. Space released to ASP pool (only the first 4KB page zeroed out)3. A new object is created4. Space utilization reported by DS8000 remains the same5. Space utilization reported by IBM i is reduced
Cognitive Systems
© Copyright IBM Corporation 2017.
IBM i File System
Used Space Free Space
6. Another object is created7. Space utilization reported by the DS8000 increases as new extent added
to thin provisioned volume8. Space utilization in ASP as reported by IBM i increases
Used Space Free Space
Cognitive Systems
© Copyright IBM Corporation 2017.
IBM i Exploitation of Thin Provisioning
• IBM i supports thin provisioning on all IBM storage platforms• The IBM i Single Level Storage architecture spreads I/O across the entire
storage pool (ASP)• When space is released within the IBM i storage pool the space is not released
to the storage system• Released space from deleted objects is not zeroes, only the first 4 KB page is
zeroed• Space released from delete luns only once they have been reinitialized from the
storage system and the IBM i system IPLed
Cognitive Systems
© Copyright IBM Corporation 2017.
Thin Provisioning Guidelines for IBM i
• Thin provisioning (ESE) is not typically recommended for production volumes due to the requirement for continual monitoring of real capacity utilization
• Thin provisioning is a reasonable option for FlashCopy targets, especially where these are retained for a short duration and can be an excellent replacement for Track Space Efficient FlashCopy• Small extents are recommended• Recommendation for thin provisioned FlashCopy targets is
50%• Global Mirror journal volumes can also be used to exploit thin
provisioning and recommended to be sized at 50%• When FlashCopy targets are thin provisioned ensure that
there are adequate hardware resources to sustain the performance requirements of the application• Ideally the target luns should be in the same extent pool as the
source luns
Cognitive Systems
© Copyright IBM Corporation 2017.
Real-time Compression – What is it?
• A compressed Volume is a third type of Volume• Regular (fully allocated)• Thin Provisioned• Compressed
• A compressed Volume is “a kind of” Thin-Provisioning• Only allocates and uses physical storage to store the compressed data for the Volume
• Supported on Storwize and XIV• Compression has a latency and throughput impact
Cognitive Systems
© Copyright IBM Corporation 2017.
Impacts of Compression
Cognitive Systems
© Copyright IBM Corporation 2017.
Thin Provisioning and Compression
Thin Provisoning
Compression
DS8000 Yes No
FS900 No No
Spectrum VirtualizeV7000V9000
Yes Yes
XIV Yes Yes
Cognitive Systems
43
Systems management
Consolidation
and Flexibility
Centralized Storage
Management
Cognitive Systems
© Copyright IBM Corporation 2017.
Systems Management
• Consolidation and Flexibility• Can move disk between servers and between platforms• Non-disruptive re-provisioning • Non-disruptive capacity addition
• Centralized storage management• Command line interface hosted on IBM i
• DS8000• Spectrum Virtualize Software
• Intuitive easy-to-use GUI across all IBM storage systems
Cognitive Systems
© Copyright IBM Corporation 2017.
Performance Monitoring Integration
• Focus on end-to-end performance monitoring and investigation for an IBM i and DS8000 environment• IBM i 7.1 adds a new category to IBM i Collection Services
• *EXTSTG – new collection performance metrics from DS8000 • Requires DS8000 R4 or later firmware
• Data can be presented in graphs using iDoctor
POWER IBM i DS8000Cache
Ranks
LinksLUN
Mem
CPU
Other
PerformanceStatistics
IBM i Collection Services
SAN Infrastructure
Performance monitoring integration and ease of use
Cognitive Systems
46
Encryption
Encryption
Cognitive Systems
© Copyright IBM Corporation 2017.
Encryption Options
Encryption Performance Impact
Key Management
Encrypt external storage
DS8000 AES 256 None TLKM/SKLM No
FS900 AES-XTS 256
None USB Key Yes
Spectrum VirtualizeV7000V9000
AES 256-XTS
None USB Key or SKLM
Yes
XIV AES 256 None TLKM/SKLMKMIP
No
Cognitive Systems
48
Best Practices
Cognitive Systems
© Copyright IBM Corporation 2017.
SAN Zone
• The basic premise of a zone is to control who can see what in a SAN.
• The IBM i fiber channel IOA port is considered an initiator. The DS8000 host adapter port can be either an initiator or target/endpoint. SAN zones are defined by pairing initiators and targets.
• Hard zoning vs. soft zoning:
• In a hard zone, sometimes referred to as a port zone, zone members are specified by physical port number. Hard zones are position-dependent, that is, a device is identified by the physical port to which it is connected.
• In a soft zone, zone members are specified logically by World Wide Port Name (WWPN). Soft zones are generally more flexible in that new devices can be attached without regard to physical port location.
Cognitive Systems
© Copyright IBM Corporation 2017.
IBM i in a SAN best practices
• Basic rules and recommendations:
• Keep IBM i in dedicated zones. Do not intermix with other platforms.
• Do not zone IBM i IOAs and non-IBM i host bus adapters (HBAs) to the same DS8000 host adapter (HA) port. Always try to keep the IBM i IOAs isolated.
• Zone one IBM i IOP-less (Smart) IOA to one DS8000 host adapter unless the workload is light. At V5R4 zone two or three IBM i IOP-based IOAs to a single DS8000 host adapter depending on the workload characteristics.
• Zone one IBM i initiator to one DS8000 target so it sees a single endpoint.
• If zones are not defined, a DS8000 host attachment can specify a port mask that controls which DS8000 I/O port an HBA is allowed to log in to.
• When implementing multi-path use redundant SAN switches to eliminate a single point of failure.
• While standards are evolving, there is no consistent or single means of documenting a SAN. In a complex environment request the network administrator to break out the IBM i components.
Cognitive Systems
© Copyright IBM Corporation 2017.
SVC and Storwize Terminology
Control Enclosure
• Hardware until that includes chassis, node canisters, drives, power supplies and batteries
• Contains 2 Node Canisters
Node Canister ( Node )
• Hardware unit that includes fiber ports, service interfaces and SAS expansion ports
• 4 fiber ports per node
Cognitive Systems
© Copyright IBM Corporation 2017.
1 2
3 4
4 3
2 1Note:secondnodecanisterisupsidedown
Eachnodecanisterhas4portsTotal8portsperV7K
Storwize V7000 Gen1 Fiber Ports
Cognitive Systems
© Copyright IBM Corporation 2017.
4 3
2 1
1 2
3 4
VIOS2
NPIVconnection(shownwith1LUNownedbynode1)
• EachVIOSHBAhaspathtobothcontrollers
• Nocommoncomponents(I.e nosinglepointoffailure)
• V7KwillautomaticallyfailtheLUNovertonode2ineventofnode0failure.
• IBMi willautomaticallysensethenewactivepathstoLUN
• IBMi supports1activeand1passivepathperIBMi Hostport
• IBMi willnotlogintothesamecontrollerWWPNonmorethan1port.
Active path 1
Active path 2
Passive path 1
Passive path 2
VIOS1
Storwize V7000 Virtualization with NPIV - best practice
Note: V7K ports are zoned to the Virtual Fiber wwpn
Node 2
Node 1
SAN switch 1 SAN switch 2
Cognitive Systems
© Copyright IBM Corporation 2017.
4 3
2 1
1 2
3 4
VIOS2
Active path 1
Active path 2
Passive path 1
Passive path 2
VIOS1
Storwize V7000 Virtualization with NPIV - best practice
Note: V7K ports are zoned to the Virtual Fiber wwpn
Node 2
Node 1
SAN switch 1 SAN switch 2
Zone ExampleSAN Switch 1:
VFC1 WWPNaV7K_Node1_Port1V7K_Node2_Port1
SAN Switch 2:VFC2 WWPNaV7K_Node1_Port3V7K_Node2_Port3
Cognitive Systems
© Copyright IBM Corporation 2017.
Storwize V7000 Virtualization - more information
Note: V7K ports are zoned to the Virtual Fiber wwpn
4 32 1
1 23 4
VIOS2VIOS1
Node 1
Node 0
SAN switch 1 SAN switch 2
• Max config for LUN = 8 active paths (I.e 8 separate ports from IBM I partition to Storwize V7000
• Also have 8 passive paths for redundancy
• Multipath driver in IBM i uses round robin across all active paths
• LUNS are distributed across nodes by default. For a normal host connection, 50 % of the LUNs will be using node 1 for the active paths and 50% will be using node 2
• It is possible to change the active node of a LUN with the V7K.
• To view current node ownership from the V7K GUI, add the preferred node column in the volumes display
• The next set of ports from IBM i should be mapped to ports 2 & 4 of the V7K nodes in this example to balance usage of the V7K ports
Cognitive Systems
© Copyright IBM Corporation 2017.
4 3
2 1
1 2
3 4
VIOS
WhatistheresultofextrapathsbetweentheswitchandtheV7K?
IBMi willnotusetheextrapath!ThedriverinIBMi willonlylogintoasvc/v7knodeonceperportintheIBMi partition.
ToprovideextrapathstoaLUN,connectanotherportintheIBMi partitiontotheswitch
Active path 1
Active path 2
Passive path 1
Passive path 2vfiber vfiber
VIOS
Storwize V7000 Virtualization with NPIV
Unused path
Not best practice
Cognitive Systems
© Copyright IBM Corporation 2017.
4 3
2 1
1 2
3 4• SameconnectionstructureasNPIVexceptzonedtorealwwpn ofadapterinIBMi insteadofvirtualwwpn
• BestpracticesthesameascoveredforNPIV
Active path 1
Active path 2
Passive path 1
Passive path 2
SVC and Storwize Native Attachment via SAN switch
Cognitive Systems
© Copyright IBM Corporation 2017.
4 3
2 1
1 2
3 4• OneportinIBMi to1portin
Storwize v7000• Usesecondportonadapterto
providepathtopassivenode
• Thisconfig providesfullredundancyforanycomponentinthepathtofail
• Idealforsmallenvironmentswhere1partitionwillbemappedto1V7K
• Consumes4portsontheV7K(I.e 2partitionsmaxtoasingleV7K)
Active path 1
Active path 2
Passive path 1
Passive path 2
Native attachment - Direct Connect – Full protection
Cognitive Systems
© Copyright IBM Corporation 2017.
Display Disk Path Status
Serial Resource Path ASP Unit Number Type Model Name Status
1 1 Y9F93F0001C4 2145 050 DMP112 ActiveDMP111 PassiveDMP119 ActiveDMP120 Passive
1 2 Y9F93F0001C1 2145 050 DMP105 Active DMP106 Passive DMP113 Active DMP114 Passive
1 3 Y9F93F0001C2 2145 050 DMP107 Passive DMP108 Active DMP115 Active DMP116 Passive
• View paths with SST 3, 1, 9 - Display Disk Path Status• For each disk, at least 2 active and 2 passive paths should be available -
more are possible by configuring more paths
Note: All other Display Disk Configuration screens in SST only show 1 of the paths.
IBM i View of Disk
Cognitive Systems
© Copyright IBM Corporation 2017.
Display Disk Unit Details
Type option, press Enter. 5=Display hardware resource information details
Serial Sys Sys Sys I/O I/OOPT ASP Unit Number Bus Card Board Adapter Bus Ctl Dev
1 1 Y9F93F0001C4 255 53 128 0 0 0 255 53 128 0 0 5 255 50 128 0 0 6 255 50 128 0 0 7
1 2 Y9F93F0001C1 255 53 128 0 0 1 255 53 128 0 0 4 255 50 128 0 0 0 255 50 128 0 0 1
• Use SST 3, 1, 9, F9 - Display Disk Unit Details• For each port, there should be 2 connections to each disk.• Sys Card = virtual client fiber #
IBM i View of Disk
Cognitive Systems
© Copyright IBM Corporation 2017.
Storwize and SVC Zoning Summary
• Multiple paths between the switch and the V7K node from a single host port does provide a level of redundancy - if the path in use fails, IBM i will automatically start using the other path
• The downside to having multiple paths configured from the same host port is there is no way to configure IBM i to use specific target port
• When multiple partitions/adapters are all configured using multiple paths between the switch and V7K, the result is typically that all I/O from all partitions go through 1 port on V7K, while the other ports are left idle.
• Use multipath with multiple VIOS servers and assigning specific zones to balance the ports on the storage side.
Cognitive Systems
© Copyright IBM Corporation 2017.
4 3
2 1
1 2
3 4
VIOS2VIOS1
Storwize V7000 Virtualization with NPIV - LPM
Node 2
Node 1
SAN switch 1 SAN switch 2
Zone ExampleSAN Switch 1:
VFC1 WWPNaVFC1 WWPNbV7K_Node1_Port1V7K _Node2_Port1
SAN Switch 2:VFC2 WWPNaVFC2 WWPNbV7K_Node1_Port3V7K_Node2_Port3
VIOS2VIOS1
Each Virtual Fibre Channel adapter has 2 WWPNs that need to be zoned and configure at the host connection for LPM
Source
Target
Cognitive Systems
© Copyright IBM Corporation 2017.
Zone Configuration for Live Partition Mobility
If you configure your host connection with both WWPNs, one will be active and the other offline
With SVC and Storwize this will show the connection as degraded
Functionally this works but if this bothers you, create 1 host connection for each VFC WWPN. One will then be online and the other offline.
Cognitive Systems
© Copyright IBM Corporation 2017.
Physical HW Layout for LPM
VIOS2VIOS1
VIOS2VIOS1
Source System A
VIOS2VIOS1
VIOS2VIOS1
Target System B
IO Domain 1
IO Domain 2
IO Domain 1
IO Domain 2
Option 1:Strip your VIOS I/O Adapters across physical hardware boundaries
Cognitive Systems
© Copyright IBM Corporation 2017.
Physical HW Layout for LPM
VIOS2VIOS1
VIOS2VIOS1
Source System A
IO Domain 1
IO Domain 2
VIOS2VIOS1
VIOS2VIOS1
Target System B
IO Domain 1
IO Domain 2
The expectation that after you do an LPM action that the client lpar is still redundant at the hardware level may not always be true.
Cognitive Systems
© Copyright IBM Corporation 2017.
Physical HW Layout for LPM
VIOS2VIOS1
VIOS2VIOS1
Source System A
VIOS2VIOS1
VIOS2VIOS1
Target System B
IO Domain 1
IO Domain 2
IO Domain 1
IO Domain 2
Example of automatic placement that doesn’t do what you might want
Cognitive Systems
© Copyright IBM Corporation 2017.
Physical HW Layout for LPM
VIOS2VIOS1
VIOS2VIOS1
Source System A
VIOS2VIOS1
VIOS2VIOS1
Target System B
IO Domain 1
IO Domain 2
IO Domain 1
IO Domain 2
The VIOS may choose FC ports that are in the small I/O Domain. There is capabilities to manually specify the FC ports you want to use.
Cognitive Systems
© Copyright IBM Corporation 2017.
Physical HW Layout for LPM
VIOS1VIOS1
VIOS2VIOS2
Source System A
VIOS1VIOS1
VIOS2VIOS2
Target System B
IO Domain 1
IO Domain 2
IO Domain 1
IO Domain 2
Option 2:Keep your VIOS I/O Adapters in the same physical hardware boundaries
Cognitive Systems
© Copyright IBM Corporation 2017.
Physical HW Layout for LPM
VIOS2VIOS1
VIOS2VIOS1
Source System A
IO Domain 1
IO Domain 2
VIOS1VIOS1
VIOS2VIOS2
Target System B
IO Domain 1
IO Domain 2
As long as you select 2 different VIOSes you’ll be redundant at the hardware level
Cognitive Systems
© Copyright IBM Corporation 2017.
IBM i Fibre Channel Adapter Limits Summary
Adapter Speed # of Ports
LUNs per port
Direct Attach Support IBM inative support
6B25 (VFC)
8 or 16 1 64 active64 passive
No N/A
5774 / 5276
4 2 64 active64 passive
Yes - (DS8K, SVC, Storwize)
Yes
5735 / 5273
8 2 64 active64 passive
Yes – DS8K Yes
5729 8 4 VIOS only, see 6B25
VSCSI only No
EN0Y 8 4 VIOS only,see 6B25
VSCSI only No
EN0A / EN0B
16 2 64 active64 passive
No Yes
Cognitive Systems
© Copyright IBM Corporation 2017.
VIOS Settings
Consider changing settings on the VIOS if you see errors in the fcstat command;
• FIBRE CHANNEL STATISTICS REPORT:fcs12FC SCSI Adapter Driver Information
No DMA Resource Count: 5 No Adapter Elements Count: 84 No Command Resource Count: 438200232
Cognitive Systems
© Copyright IBM Corporation 2017.
Am I going to run over the number of command elements?
Rough Estimate of Command Elements for a port = # of DS8000 LUNs X 6+ # of SVC/Storwize/FlashSystem LUNs X 16 + # of VSCSI LUNs X 32
Assuming the workload can drive enough I/O for all LUNs to have deferred wait time at the same time.
Cognitive Systems
© Copyright IBM Corporation 2017.
Typical VIOS Defaults
$ lsdev -dev fcs0 -attr
attribute value description user_settable
DIF_enabled no DIF (T10 protection) enabled True
bus_intr_lvl Bus interrupt level False
bus_mem_addr 0xffe76000 Bus memory address False
bus_mem_addr2 0xffe78000 Bus memory address False
init_link auto INIT Link flags False
intr_msi_1 259584 Bus interrupt level False
intr_priority 3 Interrupt priority False
lg_term_dma 0x800000 Long term DMA True
max_xfer_size 0x100000 Maximum Transfer Size True
num_cmd_elems 500 Maximum number of COMMANDS to queue to the adapter True
pref_alpa 0x1 Preferred AL_PA True
sw_fc_class 2 FC Class for Fabric True
tme no Target Mode Enabled True
Cognitive Systems
Typical VIOS (depending on SAN configuration):
FSCSIchdev –dev fsci# -attr fc_err_recov=fast_failchdev –dev fscsi# -attr dyntrk=yes
FCS
max_xfer_size=0x200000* Maximum Transfer Size, basically any number higher than default bumps up the DMA space to 128MB
num_cmd_elems=4096* Maximum number of COMMANDS to queue to the adapter. [2048, 4096 typical max depending on the adapter]
lg_term_dma=0x2000000 [32MB ] Buffers used to map terminal control structures. Resources needed on per transaction basis. Default is typically ok here.
Hex conversion
x100000=1MB x400000=4MBx800000=8MBx1000000=16MB x2000000=32MB
Cognitive Systems
© Copyright IBM Corporation 2017.
Switch settings
• If you have an 8Gb Brocade Switch and connect to 8Gb host ports make sure the fillword is set to ‘3’. If you do not you will get a large number of CRC errors on the switch and may exhibit strange SAN errors to the IBM i host adapter.• portcfgfillword <port #> 3
• If you have Cisco FC switch and want to use NPIV, enable NPIV on the switch to start with. Changing the setting will require a switch restart.
Cognitive Systems
© Copyright IBM Corporation 2017.
Does LUN Size Matter?
Cognitive Systems
© Copyright IBM Corporation 2017.
Storage Management Styles – IBM i Compared To …
Unix Linux Windows
Boot & OS
'C'
PGMs
'D''E'
'F'
DB
'G''H'
'I''J'
'K''L'
DAS, SAN attached, or virtualSystem ASP
Page
Cognitive Systems
It’s not really the size, but the number!
Even though the external storage array has many drive, IBM i still needs multiple LUNs to perform well.
• Database optimizes to the number of LUNs IBM i can see.
• IBM i Storage Management is optimized to scale with the number of LUNs
• Journal can be impacted by the number of LUNs
Don’t mix capacities in the same ASP
Take caution when increasing LUN size and dramatically reducing the quantity of LUNs.
Dynamically increasing the LUN size is not supported on IBM i !
“What do you mean you need that many LUNs? There’s hundreds of arms backing that LUN.”
2 TB
Unamed Storage Admin
Cognitive Systems
© Copyright IBM Corporation 2017.
Review of Load Source Sizes
7.1 7.2 7.2 TR1 7.3Minimum LS Size
17 GB 70 GB 35 GB 35 GB
Max Device size
2 TB 2 TB 2 TB 2 TB
Max LS Size
1.2 TB 2 TB 2 TB 2TB
PTFs are required before upgrading to a 7.2 TR2 35 GB Load Source
Cognitive Systems
80
IBM Storage Product Portfolio for IBM i
Cognitive Systems
© Copyright IBM Corporation 2017.
DeepFlash150
DS8888F
Big Data
• High capacity• High density
FlashSystem 900 Application acceleration
IBM FlashCore™ Technology Optimized
FlashSystem A9000
FlashSystem A9000R
• Full time data reduction
• Workloads: Cloud, VDI, VMware
Large deployments
FlashSystem V9000
Virtualizing the DC
Cloud service providers
• Full time data reduction
• Workloads: Mixed and cloud
StorwizeV7000F
Mid-Range
Storwize V5030F
Entry / Mid-Range
Enhanced data storage functions, economics and flexibility with sophisticated virtualization
SVC
Simplified managementFlexible consumption model
Virtualized, enterprise-class, flash-optimized, modular storageEnterprise class heterogeneous data services and selectable
data reduction
DS8884F
Business class
DS8886F
Enterprise class
Analytic class with superior performance
Business critical, deepest integration with z Systems, superior performance, highest
availability, Three-site/Four-site replication and industry-leading reliability
IBMPowerSystems
IBMSpectrum Scale
IBM z Systems
Heterogenous flash storage
IBM Systems Flash Storage Offerings Portfolio
IBM & IBM Business Partner Use Only
Note1: FlashSystem A9000 requires an SCORE/RPQ for IBM i attachment (as of 5/7/2017)Note2: FlashSystem 900 may perform better behind an SVC for IBM i workloads
Cognitive Systems
© Copyright IBM Corporation 2017.
DS8880 Family Highlights
• Performance• Built on Power 8• High-Performance Flash Enclosure• Five generations of Easy Tier® and automated QoS management
• Storage synergy• Integration with IBM i for an infrastructure optimized for critical workloads and high performance
• Reliability and availability• Designed for 99.9999% availability to keep critical applications “online all the time”• The gold standard in business continuity solutions with deep integration with high-end server HA/clustering
• Economic Impact• Self-optimization and QoS management • Efficient architecture keeps hardware resources balanced and capacity utilization high
• Data Protection and Security• Flexible replication services to address backup and business-continuity needs. • Government-grade security with stronger encryption, advanced access control, and
self-encrypting drives• Ease of Use
• Self optimization features to redirect IT staff to more important initiatives• Advanced GUI keeps administrators productive and happy
Cognitive Systems
© Copyright IBM Corporation 2017.
IBM & IBM Business Partner Use Only
• Lowest entry cost for midrange enterprises• Built with IBM Power Systems S822 • 6-core POWER8 processor per S822• Up to 256 GB Cache (DRAM)• Up to 32 Fibre channel/FICON ports• From 6.4TB to 154 TB of flash capacity
• High performance for large enterprises• Built with IBM Power Systems S824• From 8 to 24-core POWER8 processor per
S824• Up to 2 TB Cache (DRAM)• Up to 128 Fibre channel/FICON ports• From 6.4TB to 614.4 TB of flash capacity
• Highest performance for faster insights • Built with IBM Power Systems E850• From 24 to 48-core POWER8 processor per
E850• Up to 2 TB Cache (DRAM)• Up to 128 Fibre channel/FICON ports• From 6.4TB to 1.22 PB of flash capacity
DS8884FBusiness class
Efficiently designed to meet a wide range of business needs, from midrange to large enterprises
DS8886FEnterprise class
DS8888FAnalytics class
DS8880F all-flash family
Cognitive Systems
© Copyright IBM Corporation 2017.
DS8886 Performance – IBM i Client Data
Cognitive Systems
© Copyright IBM Corporation 2017.
Powered by IBM FlashCore™ technology
Extreme Performance with IBM MicroLatency™: Delivers 100 microsecond response times
Macro Efficiency: Lowest latency offering with >40% greater capacity
Enterprise Reliability: IBM enhanced Micron flash technology which contains MLC offering with Flash Wear Guarantee
Scalable Performance: Grow capacity and performance with up to 2.2 PB scaling capability
Enduring Economics: Next generation flash media for less than the cost of HDD
Agile Integration: Fully integrated system management to simplify management and improve workforce productivity under a single name space
IBM FlashSystem family
IBM FlashSystem 900IBM FlashSystem V9000
Cognitive Systems
© Copyright IBM Corporation 2017.
IBM i client Performance SVC/FlashSystem
0
20000
40000
60000
80000
100000
120000
140000
12:0
6 12
:56
13:46
14:36
15:26
16:16
17:06
17:56
18:46
19:36
20:2
621
:16
22:0
622
:56
23:46
00:36
01:2
602
:16
03:06
03:56
04:46
05:3
606
:26
07:16
08:0
608
:56
09:46
10:36
11:2
6
IO/sec
0500
100015002000250030003500
12:0
6 13
:01
13:56
14:51
15:46
16:41
17:36
18:31
19:26
20:2
121
:16
22:11
23:0
600
:01
00:56
01:51
02:4
603
:41
04:3
605
:31
06:2
607
:21
08:16
09:1
110
:06
11:01
11:56
MB/sec
00.10.20.30.40.50.60.70.80.9
1
12:0
6 12
:41
13:16
13:51
14:2
615
:01
15:36
16:11
16:46
17:21
17:56
18:31
19:06
19:41
20:1
620
:51
21:2
622
:01
22:36
23:11
23:46
00:2
100
:56
01:31
02:0
602
:41
03:16
03:51
04:2
605
:01
05:3
606
:11
06:46
07:2
107
:56
08:31
09:0
609
:41
10:16
10:51
11:2
612
:01
Read, write, total latency in SVC
Read latency (ms) Write latency(ms) Total latency (ms)
Cognitive Systems
© Copyright IBM Corporation 2017.
IBM Spectrum Accelerate EcoSystem
Intelligent, feature rich and field proven storage services
87
…On premisesStorage Rich
ServersClient choice of x86 hardware
XIV Gen3FlashSystem
A9000/A9000R
CAPACITY OPTIMIZED
PERFORMANCE OPTIMIZED
SOFTWARE DEFINED
• Data services• Snapshot• Sync/Async, 3 site replication• Thin-Provisioning• Hyper-Scale Manager, Mobility
and Consistency• Encryption• Multi-tenancy• QoS per host, domain, pool and
volume• Data Migration
• Integrations• VMWare• Microsoft (Hyper-V)• OpenStack
• New for A9000/A9000R• Data Reduction: Pattern Removal,
Deduplication and Compression• Cross platform management GUI
Note: Not all features available in each deployment model
Cloud providerStorage Rich
Servers
Note1: FlashSystem A9000 requires an SCORE/RPQ for IBM i attachment (as of 5/7/2017)Note2: FlashSystem A9000 requires VIOS to attach to IBM i
Cognitive Systems
© Copyright IBM Corporation 2017.
Systems Built With IBM Spectrum Virtualize
• IBM Spectrum Virtualize software has been available only pre-installed in IBM storage systems• IBM SAN Volume Controller• IBM Storwize family• IBM FlashSystem V9000
• Fully integrated solutions designed, tested, and supported by IBM
• Great choice for clients who prefer the simplicity of preconfigured storage appliances from midrange to enterprise
|
88
Cognitive Systems
© Copyright IBM Corporation 2017.
IBM/BusinessPartnerUseOnly
MIDRANGE ENTERPRISEENTRY
IBM All-Flash Solutions Built with IBM Spectrum Virtualize Software
Cognitive Systems
© Copyright IBM Corporation 2017.
Storwize V5010
Storwize V5020
Storwize V5030
Storwize V7000
Scale-out Yes, upgradable
Yes, upgradable
2-node to 4-nodeClustering
Plus External Storage
2-node to 8-nodeClustering
Plus External Storage
Data Reduction Compression Compression Compression Compression
Drives 504Clustering – 1,008
504Clustering – 1,056
Memory 16GB 32GB Base - 64GBClustering - 128GB
Base - 64GBClustering - 256GB
CPU/CoresBroadwell2 x 6-core, 2 thread
Broadwell2 x 6-core, 4 thread
Broadwell4 x 6-core, 4 thread
Broadwell8 x 8-core
PLUS Compression HW
FE Ports iSCSI, FCP, SAS iSCSI, FCP, SAS iSCSI, FCP, SAS iSCSI, FCP
BE Ports 12Gb/s 12Gb/s 12Gb/s 12Gb/s
RAID Options 0,1,5,6,DRAID 0,1,5,6,DRAID 0,1,5,6,DRAID 0,1,5,6,DRAID
IBM Storwize Offers Superior Upgradability and Flexibility!90
Storwize Hardware Platform
Cognitive Systems
© Copyright IBM Corporation 2017.
Storage System Comparisons
DS8000 Storwize family XIV FlashSystem
900
Native device support
Yes Yes No Yes
NPIV support Yes Yes No Yes
Power HA Support Yes Yes No No
Full System Toolkit Yes Yes Yes No
Hyper-Swap Yes Yes No No
IBM i hosted CLI Yes Yes Yes (Service Offering)
No
RAID Options 5, 6, &10 0, 1, 5, 6, & 10 (DRAID 5 & 6)
Grid 5
Automatic load balancing
3-tier Easy Tier 3-tier Easy Tier Yes No
Flash/SSD Exploitation
Automatic and IBM i managed
Automatic Automatic All-Flash
Thin provisioning Yes Yes Automatic No
Compression No Yes Yes No
Point in time copy Yes Yes Yes No
Async remote copy Yes Yes Yes No
Encryption Yes Yes Yes Yes
Cognitive Systems
© Copyright IBM Corporation 2017.
92
Why external disk?
High AvailabilityDisaster Recovery
Consolidation
Encryption Centralized Storage
Management
VirtualizationPerformance
and Flexibility
Cognitive Systems
© Copyright IBM Corporation 2017.
Session summary
• One size does not fit all• Flash exploitation across the board• Integration with IBM i and Power HA is key • IBM storage meets the requirements of IBM i customers better than other
vendors
Cognitive Systems
© Copyright IBM Corporation 2017.
Accelerate with IBM Storage Webinars
The Free IBM Storage Technical Webinar Series Continues in 2016... IBM Technical Experts cover a variety of Storage topics Audience: Clients who are either currently have IBM Storage products or considering acquiring IBM Storage products. Business Partners and IBMers are also welcome.
How to sign up? To automatically receive announcements of the Accelerate with ATS Storage webinar series, Clients, Business Partners or IBMers can send an email to [email protected]
Information, schedules, and archives: Located in the Accelerate with ATS Blog:https://www.ibm.com/developerworks/mydeveloperworks/blogs/accelerate/?lang=en
Cognitive Systems
https://facebook.com/IBMPowerSystems
https://twitter.com/IBMPowerSystems
https://www.linkedin.com/company/ibm-power-systems
IBM Power Systems Official Channels:
http://www.youtube.com/c/ibmpowersystems
https://www.ibm.com/blogs/systems/topics/servers/power-systems/
Power Systems Social Media
Cognitive Systems
96
More to Follow:
Blogs to Follow More to Follow Hashtags To Use
• IBM Systems Magazine You and I (Steve Will)• IBM Systems Magazine i-Can (Dawn May)• IBM Systems Magazine: iDevelop (Jon Paris
and Susan Gantner)• IBM Systems Magazine: iTalk with Tuohy• IBM DB2 for i (Mike Cain)• IBM DB2 Web Query for i (Doug Mack)
@IBMSystems@COMMONug
@IBMChampions@IBMSystemsISVs
@LinuxIBMMag@OpenPOWERorg
@AIXMag@IBMiMag
@SAPonIBMi@SiDforIBMi
@IBMAIXeSupp@IBMAIXdoc
#PowerSystems#IBMi
#IBMAIX#POWER8
#LinuxonPower#OpenPOWER#HANAonPower#ITinfrastructure#OpenSource#HybridCloud
#BigData
Cognitive Systems
97
Special notices
This document was developed for IBM offerings in the United States as of the date of publication. IBM may not make these offerings available in other countries, and the information is subject to change without notice. Consult your local IBM business contact for information on the IBM offerings available in your area.Information in this document concerning non-IBM products was obtained from the suppliers of these products or other public sources. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.IBM may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents. Send license inquires, in writing, to IBM Director of Licensing, IBM Corporation, New Castle Drive, Armonk, NY 10504-1785 USA. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. The information contained in this document has not been submitted to any formal IBM test and is provided "AS IS" with no warranties or guarantees either expressed or implied.All examples cited or described in this document are presented as illustrations of the manner in which some IBM products can be used and the results that may be achieved. Actual environmental costs and performance characteristics will vary depending on individual client configurations and conditions.IBM Global Financing offerings are provided through IBM Credit Corporation in the United States and other IBM subsidiaries and divisions worldwide to qualified commercial and government clients. Rates are based on a client's credit rating, financing terms, offering type, equipment type and options, and may vary by country. Other restrictions may apply. Rates and offerings are subject to change, extension or withdrawal without notice.IBM is not responsible for printing errors in this document that result in pricing or information inaccuracies.All prices shown are IBM's United States suggested list prices and are subject to change without notice; reseller prices may vary.IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.Any performance data contained in this document was determined in a controlled environment. Actual results may vary significantly and are dependent on many factors including system hardware configuration and software design and configuration. Some measurements quoted in this document may have been made on development-level systems. There is no guarantee these measurements will be the same on generally-available systems. Some measurements quoted in this document may have been estimated through extrapolation. Users of this document should verify the applicable data for their specific environment.
Revised September 26, 2006
Cognitive Systems
98
Special notices (cont.)
IBM, the IBM logo, ibm.com AIX, AIX (logo), AIX 5L, AIX 6 (logo), AS/400, BladeCenter, Blue Gene, ClusterProven, DB2, ESCON, i5/OS, i5/OS (logo), IBM Business Partner (logo), IntelliStation, LoadLeveler, Lotus, Lotus Notes, Notes, Operating System/400, OS/400, PartnerLink, PartnerWorld, PowerPC, pSeries, Rational, RISC System/6000, RS/6000, THINK, Tivoli, Tivoli (logo), Tivoli Management Environment, WebSphere, xSeries, z/OS, zSeries, Active Memory, Balanced Warehouse, CacheFlow, Cool Blue, IBM Systems Director VMControl, pureScale, TurboCore, Chiphopper, Cloudscape, DB2 Universal Database, DS4000, DS6000, DS8000, EnergyScale, Enterprise Workload Manager, General Parallel File System, , GPFS, HACMP, HACMP/6000, HASM, IBM Systems Director Active Energy Manager, iSeries, Micro-Partitioning, POWER, PowerExecutive, PowerVM, PowerVM (logo), PowerHA, Power Architecture, Power Everywhere, Power Family, POWER Hypervisor, Power Systems, Power Systems (logo), Power Systems Software, Power Systems Software (logo), POWER2, POWER3, POWER4, POWER4+, POWER5, POWER5+, POWER6, POWER6+, POWER7, System i, System p, System p5, System Storage, System z, TME 10, Workload Partitions Manager and X-Architecture are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries.
A full list of U.S. trademarks owned by IBM may be found at: http://www.ibm.com/legal/copytrade.shtml.
Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.AltiVec is a trademark of Freescale Semiconductor, Inc.AMD Opteron is a trademark of Advanced Micro Devices, Inc.InfiniBand, InfiniBand Trade Association and the InfiniBand design marks are trademarks and/or service marks of the InfiniBand Trade Association. Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency which is now part of the Office of Government Commerce.Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates.Linear Tape-Open, LTO, the LTO Logo, Ultrium, and the Ultrium logo are trademarks of HP, IBM Corp. and Quantum in the U.S. and other countries.Linux is a registered trademark of Linus Torvalds in the United States, other countries or both.Microsoft, Windows and the Windows logo are registered trademarks of Microsoft Corporation in the United States, other countries or both.NetBench is a registered trademark of Ziff Davis Media in the United States, other countries or both.SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPEC OMP, SPECviewperf, SPECapc, SPEChpc, SPECjvm, SPECmail, SPECimap and SPECsfs are trademarks of the Standard Performance Evaluation Corp (SPEC).The Power Architecture and Power.org wordmarks and the Power and Power.org logos and related marks are trademarks and service marks licensed by Power.org.TPC-C and TPC-H are trademarks of the Transaction Performance Processing Council (TPPC).UNIX is a registered trademark of The Open Group in the United States, other countries or both.
Other company, product and service names may be trademarks or service marks of others.
Revised December 2, 2010
Cognitive Systems
© Copyright IBM Corporation 2017.
IBM, the IBM logo, ibm.com AIX, AIX (logo), AIX 5L, AIX 6 (logo), AS/400, BladeCenter, Blue Gene, ClusterProven, DB2, ESCON, i5/OS, i5/OS (logo), IBM Business Partner (logo), IntelliStation, LoadLeveler, Lotus, Lotus Notes, Notes, Operating System/400, OS/400, PartnerLink, PartnerWorld, PowerPC, pSeries, Rational, RISC System/6000, RS/6000, THINK, Tivoli, Tivoli (logo), Tivoli Management Environment, WebSphere, xSeries, z/OS, zSeries, Active Memory, Balanced Warehouse, CacheFlow, Cool Blue, IBM Systems Director VMControl, pureScale, TurboCore, Chiphopper, Cloudscape, DB2 Universal Database, DS4000, DS6000, DS8000, EnergyScale, Enterprise Workload Manager, General Parallel File System, , GPFS, HACMP, HACMP/6000, HASM, IBM Systems Director Active Energy Manager, iSeries, Micro-Partitioning, POWER, PowerExecutive, PowerVM, PowerVM (logo), PowerHA, Power Architecture, Power Everywhere, Power Family, POWER Hypervisor, Power Systems, Power Systems (logo), Power Systems Software, Power Systems Software (logo), POWER2, POWER3, POWER4, POWER4+, POWER5, POWER5+, POWER6, POWER6+, POWER7, System i, System p, System p5, System Storage, System z, TME 10, Workload Partitions Manager and X-Architecture are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries.
A full list of U.S. trademarks owned by IBM may be found at: http://www.ibm.com/legal/copytrade.shtml.
Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.AltiVec is a trademark of Freescale Semiconductor, Inc.AMD Opteron is a trademark of Advanced Micro Devices, Inc.InfiniBand, InfiniBand Trade Association and the InfiniBand design marks are trademarks and/or service marks of the InfiniBand Trade Association. Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency which is now part of the Office of Government Commerce.Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates.Linear Tape-Open, LTO, the LTO Logo, Ultrium, and the Ultrium logo are trademarks of HP, IBM Corp. and Quantum in the U.S. and other countries.Linux is a registered trademark of Linus Torvalds in the United States, other countries or both.Microsoft, Windows and the Windows logo are registered trademarks of Microsoft Corporation in the United States, other countries or both.NetBench is a registered trademark of Ziff Davis Media in the United States, other countries or both.SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPEC OMP, SPECviewperf, SPECapc, SPEChpc, SPECjvm, SPECmail, SPECimap and SPECsfs are trademarks of the Standard Performance Evaluation Corp (SPEC).The Power Architecture and Power.org wordmarks and the Power and Power.org logos and related marks are trademarks and service marks licensed by Power.org.TPC-C and TPC-H are trademarks of the Transaction Performance Processing Council (TPPC).UNIX is a registered trademark of The Open Group in the United States, other countries or both.
Other company, product and service names may be trademarks or service marks of others. Revised December 2, 2010
Special notices (cont.)