intel® solutions for lustre* software sc16 presentation · tests document performance of...
TRANSCRIPT
![Page 1: Intel® Solutions for Lustre* Software SC16 Presentation · Tests document performance of components on a particular test, in specific systems. Differences in hardware, software,](https://reader031.vdocuments.net/reader031/viewer/2022011904/5f1e26ba9ca30c5ecb5e1516/html5/thumbnails/1.jpg)
![Page 2: Intel® Solutions for Lustre* Software SC16 Presentation · Tests document performance of components on a particular test, in specific systems. Differences in hardware, software,](https://reader031.vdocuments.net/reader031/viewer/2022011904/5f1e26ba9ca30c5ecb5e1516/html5/thumbnails/2.jpg)
Craig Hendren Global OEM Alliances Manager for Lustre and HPC OrchestratorHigh Performance Data Division, Intel Corporation
![Page 3: Intel® Solutions for Lustre* Software SC16 Presentation · Tests document performance of components on a particular test, in specific systems. Differences in hardware, software,](https://reader031.vdocuments.net/reader031/viewer/2022011904/5f1e26ba9ca30c5ecb5e1516/html5/thumbnails/3.jpg)
Legal informationTests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. Consult other sources of information to evaluate performance as you consider your purchase. For more complete information about performance and benchmark results, visit http://www.intel.com/performance.
Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at http://www.intel.com/content/www/us/en/software/intel-solutions-for-lustre-software.html.
You may not use or facilitate the use of this document in connection with any infringement or other legal analysis concerning Intel products described herein. You agree to grant Intel a non-exclusive, royalty-free license to any patent claim thereafter drafted which includes subject matter disclosed herein.
No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.
Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.
This document contains information on products, services and/or processes in development. All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps.
The products described may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request.
A "Mission Critical Application" is any application in which failure of the Intel Product could result, directly or indirectly, in personal injury or death. SHOULD YOU PURCHASE OR USE INTEL'S PRODUCTS FOR ANY SUCH MISSION CRITICAL APPLICATION, YOU SHALL INDEMNIFY AND HOLD INTEL AND ITS SUBSIDIARIES, SUBCONTRACTORS AND AFFILIATES, AND THE DIRECTORS, OFFICERS, AND EMPLOYEES OF EACH, HARMLESS AGAINST ALL CLAIMS COSTS, DAMAGES, AND EXPENSES AND REASONABLE ATTORNEYS' FEES ARISING OUT OF, DIRECTLY OR INDIRECTLY, ANY CLAIM OF PRODUCT LIABILITY, PERSONAL INJURY, OR DEATH ARISING IN ANY WAY OUT OF SUCH MISSION CRITICAL APPLICATION, WHETHER OR NOT INTEL OR ITS SUBCONTRACTOR WAS NEGLIGENT IN THE DESIGN, MANUFACTURE, OR WARNING OF THE INTEL PRODUCT OR ANY OF ITS PARTS.
Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of any features or instructions marked "reserved" or "undefined". Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. The information here is subject to change without notice. Do not finalize a design with this information.
3D Xpoint, Intel, the Intel logo, Intel Core, Intel Xeon Phi, Optane and Xeon are trademarks of Intel Corporation in the U.S. and/or other countries.
* Other names and brands may be claimed as the property of others.
© 2016 Intel Corporation
3
![Page 4: Intel® Solutions for Lustre* Software SC16 Presentation · Tests document performance of components on a particular test, in specific systems. Differences in hardware, software,](https://reader031.vdocuments.net/reader031/viewer/2022011904/5f1e26ba9ca30c5ecb5e1516/html5/thumbnails/4.jpg)
December 2015: Intel’s Analysis of Top 100 Systems (top100.org)
9 of Top10 Sites
73% of Top100
Most Adopted PFS
Most Scalable PFS
Open Source GPL v2
Commercial Packaging
Vibrant Community
4
73%
16%4% 7%
Lustre GPFS other unknown
![Page 5: Intel® Solutions for Lustre* Software SC16 Presentation · Tests document performance of components on a particular test, in specific systems. Differences in hardware, software,](https://reader031.vdocuments.net/reader031/viewer/2022011904/5f1e26ba9ca30c5ecb5e1516/html5/thumbnails/5.jpg)
* Other names and brands may be claimed as the property of others.
1999 2011 2013
20152001
2003
2007
2009
2010 2012
2016
1.0
1.6
1.8
2.0
2.1
2.3
2.5
2.7
2.8
5
![Page 6: Intel® Solutions for Lustre* Software SC16 Presentation · Tests document performance of components on a particular test, in specific systems. Differences in hardware, software,](https://reader031.vdocuments.net/reader031/viewer/2022011904/5f1e26ba9ca30c5ecb5e1516/html5/thumbnails/6.jpg)
1 Source: Chris Morrone, Lead of OpenSFS Lustre Working Group, April 2016
Intel65%
8%
6%
6%
3%3%
2%2%
1%2%
Intel ORNL* Seagate* Cray* DDN*
Atos* LLNL* CEA* IU Other
Intel65%
18%
4%2%
2%2%1%1%1%
Intel ORNL Cray Atos Seagate
DDN IU CEA Other
Commit per Organization Lines of codes per organization
6
![Page 7: Intel® Solutions for Lustre* Software SC16 Presentation · Tests document performance of components on a particular test, in specific systems. Differences in hardware, software,](https://reader031.vdocuments.net/reader031/viewer/2022011904/5f1e26ba9ca30c5ecb5e1516/html5/thumbnails/7.jpg)
Intel® Scalable System FrameworkA Holistic Solution for All HPC Needs
Small Clusters Through Supercomputers
Compute and Data-Centric Computing
Standards-Based Programmability
On-Premise and Cloud-Based
Intel® Xeon® Processors
Intel® Xeon Phi™ Processors
Intel® FPGAs and Server Solutions
Intel® Solutions for Lustre*
Intel® Optane™ Technology
3D XPoint™ Technology
Intel® SSDs
Intel® Omni-Path Architecture
Intel® Silicon Photonics
Intel® Ethernet
Intel® HPC Orchestrator
Intel® Software Tools
Intel® Cluster Ready Program
Intel Supported SDVis
ComputeFabric
Memory / Storage
Software
7
![Page 8: Intel® Solutions for Lustre* Software SC16 Presentation · Tests document performance of components on a particular test, in specific systems. Differences in hardware, software,](https://reader031.vdocuments.net/reader031/viewer/2022011904/5f1e26ba9ca30c5ecb5e1516/html5/thumbnails/8.jpg)
* Other names and brands may be claimed as the property of others.
Bioscience Government research and defense Large-scale manufacturingMechanical, computer-aided design & computer-aided engineering systems
Genomic data analysis, modeling and simulations
Weather and climate Energy FinanceFraud detection, Monte Carlo simulations,
risk management analysisHighly complex CGI rendering Seismic processing, reservoir modeling /
characterization, sensor data analysis
8
Government funded research. Surveillance, Signal Processing, encryption etc.
![Page 9: Intel® Solutions for Lustre* Software SC16 Presentation · Tests document performance of components on a particular test, in specific systems. Differences in hardware, software,](https://reader031.vdocuments.net/reader031/viewer/2022011904/5f1e26ba9ca30c5ecb5e1516/html5/thumbnails/9.jpg)
Intel® Scalable System Framework for HPC
* Other names and brands may be claimed as the property of others.
Intel® FOUNDATION Edition for Lustre* software
Delivers the latest functions and features, fully supported by Intel
Ideal for organizations that prefer to design and deploy their own open
source configurations
Intel® ENTERPRISE Edition for Lustre* software
Maximum performance with minimal complexity and cost for multi-
petabyte file system. Management with Intel® Manager for Lustre*
software
Intel® CLOUD Edition for Lustre* software
Cost-effective access to parallel storage on Amazon Web Services*
(AWS) and Microsoft Azure* to boost cloud-computing
9
![Page 10: Intel® Solutions for Lustre* Software SC16 Presentation · Tests document performance of components on a particular test, in specific systems. Differences in hardware, software,](https://reader031.vdocuments.net/reader031/viewer/2022011904/5f1e26ba9ca30c5ecb5e1516/html5/thumbnails/10.jpg)
10
![Page 11: Intel® Solutions for Lustre* Software SC16 Presentation · Tests document performance of components on a particular test, in specific systems. Differences in hardware, software,](https://reader031.vdocuments.net/reader031/viewer/2022011904/5f1e26ba9ca30c5ecb5e1516/html5/thumbnails/11.jpg)
Proven Experience of #1 Lustre Provider
Industry’s Fastest Storage – SFA
DDN + Intel – Unique Features, Support
Versatile – Embedded / External / Custom
End2End Data Management
11
DDN
![Page 12: Intel® Solutions for Lustre* Software SC16 Presentation · Tests document performance of components on a particular test, in specific systems. Differences in hardware, software,](https://reader031.vdocuments.net/reader031/viewer/2022011904/5f1e26ba9ca30c5ecb5e1516/html5/thumbnails/12.jpg)
Select DDN EXAScaler & Lustre Customers
12
![Page 13: Intel® Solutions for Lustre* Software SC16 Presentation · Tests document performance of components on a particular test, in specific systems. Differences in hardware, software,](https://reader031.vdocuments.net/reader031/viewer/2022011904/5f1e26ba9ca30c5ecb5e1516/html5/thumbnails/13.jpg)
13
![Page 14: Intel® Solutions for Lustre* Software SC16 Presentation · Tests document performance of components on a particular test, in specific systems. Differences in hardware, software,](https://reader031.vdocuments.net/reader031/viewer/2022011904/5f1e26ba9ca30c5ecb5e1516/html5/thumbnails/14.jpg)
* Other names and brands may be claimed as the property of others.
14
![Page 15: Intel® Solutions for Lustre* Software SC16 Presentation · Tests document performance of components on a particular test, in specific systems. Differences in hardware, software,](https://reader031.vdocuments.net/reader031/viewer/2022011904/5f1e26ba9ca30c5ecb5e1516/html5/thumbnails/15.jpg)
* Other names and brands may be claimed as the property of others.
Software Core for Intel® EE for Lustre* software. Bulk IO
Performance, Subdirectory Mounts
Management OpenZFSstorage. Jobstats Monitoring. LFSCK, Dynamic Lnet; Intel®
Manager for Lustre*
Latest updates and security patches to customers.
Kerberos* Network Authentication and
Encryption. SELinux* Lustre Client Support
Hadoop* Adapter for Lustre and HPC Adapter for MapReduce.
Lustre 2.7
Intel® Omni-Path Architecture support.
Metadata Performance Improvements. Snapshots
15
New in v3.1
![Page 16: Intel® Solutions for Lustre* Software SC16 Presentation · Tests document performance of components on a particular test, in specific systems. Differences in hardware, software,](https://reader031.vdocuments.net/reader031/viewer/2022011904/5f1e26ba9ca30c5ecb5e1516/html5/thumbnails/16.jpg)
Product placement not representative of final launch date within the specified quarter* Other names and brands may be claimed as the property of others
Q4 ’16 Q1 ’17 Q2’17 Q3 ’17 Q4 ’17
EnterpriseHPC, commercial technical computing, and analytics Client m’data perf.
OpenZFS perf.SnapshotsSELinux, KerberosOPA supportRHEL 7OpenZFS 0.6.5.3
EE 3.0
Lustre* 2.7.x
DNE 2Client m’data perf.LFSCK perf.SELinux, Kerberos
FE 2.8
Lustre 2.8.x
CloudHPC, commercial technical computing and analytics delivered on cloud services
FoundationCommunity Lustresoftware coupled with Intel support
Expanded OpenZFS support in IMLIML graphical UI enhancementsBulk IO Performance (16MB IO)OpenZFS metadata performance Subdirectory mounts
Enterprise Edition 3.1
Lustre 2.7.x
DNE 2Multi-rail LNETIML asset mgmt. ZED updates
EE 4.0
Lustre <TBA>
RHEL 7 Server SupportLustre release update
Cloud Edition 1.3
Lustre 2.8.x
SW updatesNew instances
CE 1.5
Lustre <TBA>
Updated Lustre and base OS buildsNew instance support
Cloud Edition 1.4
Lustre 2.9.x
UID/GID MappingShared key cryptoLarge Block IOSubdirectory mounts
Foundation Edition 2.9
Lustre 2.9.x
OpenZFS SnapshotsMulti-rail LNETProject quotas
Foundation Edition 2.10
Lustre 2.10.x
Shipping
Development
Planning
16
![Page 17: Intel® Solutions for Lustre* Software SC16 Presentation · Tests document performance of components on a particular test, in specific systems. Differences in hardware, software,](https://reader031.vdocuments.net/reader031/viewer/2022011904/5f1e26ba9ca30c5ecb5e1516/html5/thumbnails/17.jpg)
Intel Manager for Lustre
* Other names and brands may be claimed as the property of others.
ManagementNetwork
High Performance Data Network(Infiniband*, 10GbE)
MetadataServers(1-10s)
Object Storage Servers
(10s-1000s)
Lustre Clients (1 – 100,000+)
Object StorageTargets (OSTs)
Object StorageTargets (OSTs)MetadataTarget (MDT)
ManagementTarget (MGT)
Native Lustre* Client for Intel® Xeon Phi™ processorIntel® Omni-Path Support
Robin HoodOpenZFS, RAIDz
Hadoop* Adapters
17
HSM
![Page 18: Intel® Solutions for Lustre* Software SC16 Presentation · Tests document performance of components on a particular test, in specific systems. Differences in hardware, software,](https://reader031.vdocuments.net/reader031/viewer/2022011904/5f1e26ba9ca30c5ecb5e1516/html5/thumbnails/18.jpg)
* Other names and brands may be claimed as the property of others.
Community LustreSystem integrators selling systems
based on community Lustre* do not receive maintenance releases from Intel
Reliant on open source community for support
Patches, when available, are supplied as source code only
Integration and testing left to SI or end user to verify
Deploying solutions based on community edition carries risk for end user
18
Intel support globally—24/7/365
Regular cadence of maintenance and patch releases
Scheduled proven features and value-add major releases
Enterprise Edition is bundled with features (IML, HSM, DSS, Hadoop*, ZFS, etc.)
Optimizations for Intel® architecture –Intel® Scalable System Framework
Online training and testing for end users, channel partners and OEM partners
www.intel.com/lustre-training & www.intel.com/lustre-reseller
![Page 19: Intel® Solutions for Lustre* Software SC16 Presentation · Tests document performance of components on a particular test, in specific systems. Differences in hardware, software,](https://reader031.vdocuments.net/reader031/viewer/2022011904/5f1e26ba9ca30c5ecb5e1516/html5/thumbnails/19.jpg)