security, integrity and choices for nvme over fabrics · security, integrity and choices for nvme...
TRANSCRIPT
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 1
Security, Integrity and Choices for NVMe over Fabrics
Nishant LodhaMarvell
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 22019 Storage Developer Conference. © Marvell. All Rights Reserved.
Agenda
§ NVMe-oF®, the choices and the confusion§ Use Cases by Fabric§ Securing NVMe-oF§ Key Takeaways
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 3
NVMe-oF
3
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 4
NVMe Server Software
Server Transport Abstraction
FibreChannel Infiniband FCoERoCEv2 iWARP
Storage Transport Abstraction
NVMe SSDs
TCP
Scaling our NVMe Requires a (Real) Network
§ Many options, plenty of confusion§ Fibre Channel is the transport for the vast
majority of today’s all flash arraysFC-NVMe Standardized in Mid-2017
§ RoCEv2, iWARP and InfiniBand are RDMA-based but not compatible with each other
NVMe-oF RDMA Standardized in 2016§ FCoE fabric is an option§ NVMe/TCP – is here! Standardized in
NOV2018
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 5
RDMA Use Cases by Application
RoCE
RoCEv2iWARPRDMA
NIC withDisaggregated Storage - SMB
HyperConvergedInfrastructure
NFS over RDMA
VM Migration Disaggregated Storage - iSER
Disaggregated Storage – NVMe-oF
RDMA Accelerated CEPHS
Low Latency VMs
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 6
NVMe-oF™ RDMA – potential challenges
RNIC Upgrade Required
RDMA Camps
Creates IslandsBackward Compatibility
Infrastructure and Skillset change?
Not Automatic
Not Precise
Not for everyoneCongestion
Keeping the network ‘lossless’
RDMA/OEFDexpertise
Skillset Requirements
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 7
Relationship Status: Microsoft and RoCE
See the Microsoft Blog – comparing the RDMA typeshttps://blogs.technet.microsoft.com/filecab/2017/09/21/storage-spaces-direct-with-cavium-fastlinq-41000/
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 8
NVMe Transport Performance Comparisons
Local NVMe iSCSI NVMe-oF 25GbEIO
Ope
rato
ns P
er S
econ
d (IO
PS)
NVMe-oF IOPS Comparisons32KB Random Reads 8 Threads and 32 IO Depth
Local NVMe Software iSCSI NVMe-oF RoCE
Late
ncy
(us)
NVMe-oF Latency Comparisons4KB Random Reads Single Thead and IO Depth
iSCSI adds 82% more latency, Delivers fewer IOPS
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 9
FC-NVMe!
Low Latency, High ThroughputIncreased Virtualization Density
More Content Video, Big Data
Greater OPEX Efficiency
Transport NVMe Natively over Fibre Channel
FC-NVMe T11 Committee
Leverage Existing Investments in Fibre Channel
“NVMe” Over Fibre Channel
FC-NVMe
Low Latency
Reliable, Secure, Available
Ecosystem Ready
Fabric
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 10
FCP vs. FC-NVMe
1 2 4 8 16 32 64 128 256 512 1024
IOPS
OUTSTANDING IOS
FCP vs. FC-NVMe: 4KB RD & 4 Jobs / DP to 1 LUN/NS per port
FC-NVMe Scales in performance
FC-NVMe
FCP
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 11
Use Cases by Fabric
NVMe/TCP (Ethernet)
No one size fits all!
NVMe/RDMA (Ethernet) FC-NVMe (Fibre Channel)
DAS, HPC, AI/ML Enterprise Applications All Applications
Performance at the cost of complexity
Leverage existing infrastructure. Reliability is
key
Simplicity is key. Balance of performance and cost
Logos are indicative of workload characteristics only.
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 12
NVMe/TCP
12
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 13
NVMe-oF: NVMe/TCP§ What: Defines a TCP Transport
Binding layer for NVMe-oF§ Promoted by Facebook,
Google, Intel, Marvell etc.§ Not RDMA-based, Standardized
on 15NOV18§ Why:
§ Enables adoption of NVMe-oF into existing datacenter IP network environments that are not RDMA-enabled
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 14
Block-mq
nvmet
nvmet APInvmet APInvmet API
qedn(Target mode)
nvmet-tcp(sw nvme-tcp)
qedrqede
nvmet-rdma
FW
qed
nvme
nvme APIblk-mq API
nvme APIblk-mq API
nvme APIblk-mq API
qedn(Host mode)
nvme-tcp(sw nvme-tcp)
qedrqede
nvme-rdma
FW
qed
NVMe-oF Driver Stack
Target Initiator
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 15
IO stackNVMe Host (nvme)
Send Queue
Completion Queue
Read/Write Data Buffer
NVMe Host Transport (nvme-tcp)
Marvell NICL2 Firmware
PDU PayloadNVMeTCP PDU
Header
Host
Linu
x st
ack
HW
/FW
TCP/IP Stack
L2 Driver (qede)
TCP Payload
TCP Heade
r
TCP Payload
TCP Heade
r
Offloading NVMe/TCP
Marvell NICNVMeTCP Target Firmware
+TCP Offload
NVMeTarget (nvmet)
Send Queue
Completion Queue
Read/Write Data Buffer
Target
HW
/FW
Linu
x st
ack
NVMeTarget Transport (qedn)
PDU PayloadNVMeTCP PDU Header
TCP PayloadTCP
HeaderTCP Payload
TCP Header
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 16
Accelerating NVMe/TCP
0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% 100.00%
4K Read IO - 1 pending latency [usec]
NVMe/TCP (software) NVMe/RoCEv2 NVMe/TCP - Offloaded
0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% 100.00%
4K Write IO - 1 pending latency [usec]
NVMe/TCP (software) NVMe/RoCEv2 NVMe/TCP - Offloaded
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 17
11%32%
0%
20%
40%
60%
80%
100%
NVMeTCP offload NVMeTCP SW
128K Read IOs - 100Gbps - CPU Utilization
11%
62%
0%
20%
40%
60%
80%
100%
NVMeTCP offload NVMeTCP SW
128K Write IOs - 100Gbps - CPU Utilization
Significant CPU Savings with NVMe/TCP Offload
Cost of I/O – NVMe/TCP
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 18
Security FC-NVMe
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 19
Drivers for FC-NVMe SecuritySecurity and Privacy Sensitive Verticals
Healthcare Financial Government Defense
Privacy / Regulatory HIPAAGDPR ISO270001
Security/Regulatory EU Payment Services DirectiveEU Cyber Security Directive for Infrastructure
Malicious InsidersHIPAA
New Deployment Use Cases DR
Secondary Cloud StorageMulti-tenant Data Centers
Hybrid Cloud
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 20
Cost of a data breach and Recent events
Source: IBM Security
sources: databreaches.net, IDTheftCentre and media reports
None of these breaches have been directly attributed to
Fibre Channel
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 21
Isn’t FC Secure Already?
• Data Centers are physically securedPhysical Security
• Fibre Channel SANs are segregated networksSegregation
• FC Zoning ensures fabric partitioningPartitioning
• LUN masking restricts access to specific LUNsMasking
• Out-of-Band Management (IP) is secure, OS ControlsManagement
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 22
Yes, But…§ New Data Center Architectures bring new threats
§ Distributed data centers - Remote replication and DR backups may be accessed by different users over Fabrics that span several sites
§ Multi Tenant data centers – Need to segregate and protect data traversing the same wire
§ Increasing scale of FC SANs§ Networks can be misconfigured§ Fabric configuration databases are shared, have WKAs
§ Existing mechanisms may not be enough§ Switches are the sole entity that grant/deny access
§ Authorization based§ “Segmentation” tools being used to implement “Security”
§ Soft zoning, LUN Masking
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 23
Mitigated by Fibre Channel SAN Security
Potential DC Storage Security Threats
Sniffing Storage Traffic
Data Corruption
Storage Masquerading
Session Hijacking
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 24
FC-SP-2: What and Why?§ Why? : Need to transition SANs from Authorization and
segmentation based FC security to authentication and encryption based security!
§ What? FC-SP-2 is a ANSI/INCITS standard (2012) that defines protocols to –§ Authenticate Fibre Channel entities§ Setup session encryption keys§ Negotiate parameters to ensure per frame integrity and confidentiality§ Define and distribute security policies over FC
§ Designed to protect against several classes of threats
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 25
Fabric Security Architecture
Authentication Infrastructure
Secret, certificate, password and pre-
shared key based
architecture
Authentication
Protocol to assure identify
of communicatin
g entities, negotiation of
security requirement and protocol
Security Associations
Protocol to establish
Shared key between
communicating entities, Based on
IKEv2 (RFC4595)
Crypto Integrity Confidentiality
Frame by frame
encryption, replay
protection, origin
authentication, ESP_Header
or CT_Authentic
ation
Authorization
Fabric policies that control
which entities can connect with each
other, management access to the
fabric
Components of FC-SP-2 Security Architecture
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 26
FC-SP-2 ESP_header§ ESP_header (optional) is a layer 2 security protocol that provides
§ Origin authentication§ Integrity§ Anti-replay protection§ Confidentially
§ Encapsulating Security Payload (ESP) is defined in RFC 4303§ FC-FS-3 defines optional headers for Fibre Channel, FC-SP defines how
to use ESP in Fibre Channel§ Similar protections exist for CT_Authentication
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 27
Silicon Root of TrustProtecting the Integrity of Fibre Channel Firmware
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 28
Key Takeaway
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 29
Making the right “fabric” choice!
Not “just” about “fabrics” performance Use Cases and SecurityCulture and Install Base
2019 Storage Developer Conference. © Marvell. All Rights Reserved. 30
That’s it!