openpower roadmap toward coral

27

Upload: inside-bigdatacom

Post on 13-Apr-2017

685 views

Category:

Technology


1 download

TRANSCRIPT

Page 1: OpenPOWER Roadmap Toward CORAL
Page 2: OpenPOWER Roadmap Toward CORAL

FundamentalforcesareacceleratingchangeinourindustryProcessortechnologyalonecannolongerprovidethepriceperformancegainsnecessary tosustain

Moore’sLaw

Scale-out & Hyperscale Data Centers

Hybrid Cloud

Open Solutions

ITconsumptionmodelsareexpandingto:

Abetterprocessorandfullsystemstackopeninnovationrequired

Price

/Perform

ance

Moore’sLaw

Processor

Technology

2000 2020

Firmware/OS

Accelerators

Software

Storage

Network

2

Page 3: OpenPOWER Roadmap Toward CORAL

• Data&infrastructuresilos

• Datainducedbottlenecks

• Blindlyaddingcapacity

• Dataduplicationandmoresilos

• Adhoccloudusage

Complexity

Cost

Availability

Traditionalcompute&storageapproacheswon’tsuffice…

3

Page 4: OpenPOWER Roadmap Toward CORAL

Price

/Perform

ance Semiconductor

Technologyand

Processors

2000 2020

IBMHPCstrategy

POWER8era

ShiftbacktowardstheMoore’sLawpredictionthrough:1. IBMHPCInnovation(processorarchitectureenhancement,scalablefilesystems,workflowmanagement)2. Acceleration throughpartnerecosystem(e.g.NVIDIAGPUsdeliver2Xperf/wattand~3Xperformance)

POWER9 era

IBMHPCInnovation

Acceleration

Moore’sLaw,

Client

Expectations

4

Page 5: OpenPOWER Roadmap Toward CORAL

IBMVision:Data-CentricComputing

• Hardwareandsoftwaretosupport&enablecomputeindata• AllowworkloadstorunwheretheyrunbestMinimizeDataMotion

• Introduce“active”systemelementsincludingnetwork,memory,storage

• Globalaccesstoshareddata

EnableComputeEverywhere

• Balanced,composable architecture• Modularandupgradeabledesignscalableupto100sofracksModularity

• Userealworkloads/workflowstodrivedesignpoints• Co-designforcustomervalue

Application-drivendesign

5

Page 6: OpenPOWER Roadmap Toward CORAL

IBMHPCInnovationsandInvestments:OurCommitmenttoHPC

ProcessorDesign• POWER8,POWER9,14nmProcessorDesign

ProcessorInterfaces• CAPI,POWERwithNVLinkTechnology

Filesystems&StorageSolutions• IBMSpectrumScale• ElasticStorageServer• HPSS• IBMFlashSystem

Compilers,Development,Libraries• IBMXL,Parallel Environment,ESSL

MPIandClusterProgramming• IBMPlatformMPI,OpenMP 3.1and4.0Development

Workload&ClusterManagement• IBMPlatformClusterManager,xCAT

• IBM PlatformLSFPowerandData-AwareScheduling

6

Page 7: OpenPOWER Roadmap Toward CORAL

Highlights:IBMHPCaroundtheWorld2014-2015

IBM,Mellanox,andNVIDIAawarded$325MU.S.DepartmentofEnergy’sCORALSupercomputers

GENCIannouncescollaboration withIBMon

speedingthepathtoexascale computing.

DESYcollaborateswithIBMResearchselectsSoftwareDefinedStorageSDI,IBM

Powerforparticleacceleratordata

UKGovernmentInvests£115Million inBigData&CognitiveComputing

ResearchwithSTFCandIBM

LeadingoilandgasfirmselectsIBMforworkload

management

LSUselectsacomprehensiveIBMHPCsolutionforgenomicsresearch

IntegratedPortfolio:

IBMPowerSystemsSoftwareDefinedInfrastructureIBMStorageIBMResearch

Withmany,manymorearoundtheworld… 7

Page 8: OpenPOWER Roadmap Toward CORAL

OpenPOWER:OpenArchitectureforHPC &BigData

ProcessorIPLicensing

OpenInterfaces

Systems&Software

LicensingIPtoenablesemiconductorpartnerslikeSuzhouPowercore tobuildPOWERchips

TightintegrationusingCAPI&NVLink withAccelerators(NVIDIA,Xilinx,Altera),Networking(Mellanox),Storage(CAPIFlash)

EnablingInnovativePOWER-basedserversfromPartners&OpenCompute andSharingOpenSourceSoftwareincludingFirmware&Hypervisor

8

Page 9: OpenPOWER Roadmap Toward CORAL

MajorWinsEndorsetheStrategyandEstablishthePathForward

IBM,Mellanox,andNVIDIAawarded$325M

U.S.DepartmentofEnergy’sCORALSupercomputers

IBM&UK’sSTFC PartnerforBigData&CognitiveComputing Researchin£313MPartnership

CORAL: Leadership Class Supercomputers 5X – 10X HIGHER APP PERF THAN CURRENT SYSTEMS

US,UK,GermanScientificResearchCommunitiesSelectOpenPOWERorCommittoOpenPOWERCollaboration

PADCOpeningOctober12th,2015

Jülich Supercomputing CentrePOWERAcceleration

andDesignCenter(PADC) RoadmapTowardCORAL

Page 10: OpenPOWER Roadmap Toward CORAL

190+ Open Power Foundation Members

10

Page 11: OpenPOWER Roadmap Toward CORAL

1900+ApplicationsRunonPOWERBigData&

MachineLearningCloud MobileEnterprise

Major Linux Distros

HPC

miniDFTCTHBLASTBowtieBWAFASTAHMMERGATKSOAP3STAC-A2SHOC

Graph500Ilog

CHARMMGROMACSNAMDAMBERRTM

GAMESSWRF

HYCOMHOMME

LESMiniGhostAMG2013OpenFOAM

11

Page 12: OpenPOWER Roadmap Toward CORAL

POWER8:ProcessorPerformanceLeadership

MemoryBuffer

DRAMChips

POWER8

12Cores,96Threads

4LevelLargeCaches

Upto1TBpersocket

Upto230GB/ssustained

FasterCores8ThreadsPerCore

LargerCachesDirectAccelerator

Interconnect

3xHigherMemoryBandwidth,1TB

MemoryperSocket

12

Page 13: OpenPOWER Roadmap Toward CORAL

HPCApplicationPerformance

0,0

0,5

1,0

1,5

2,0

2,5

3,0

RelativePerformance

HaswellBased POWER8– S822LC

POWER8:Upto2.5xFasteronApplicationsoverIntelHaswellx86CPUs

13

Page 14: OpenPOWER Roadmap Toward CORAL

SharedSystemMemoryviaCoherentInterface 2.5xFasterCPU-GPUConnectionviaNVLink

OpenPOWER:TightintegrationusingCAPI&NVLink

POWER

CPUs

Memory

FPGA

CAPICoherentAcceleratorProcessorInterface

14

Page 15: OpenPOWER Roadmap Toward CORAL

IBMOpenPOWER-basedHPC Roadmap

2015 2016 2017

POWER8POWER8+

POWER9

OpenPowerCAPI Interface

NVLink

EnhancedCAPI &NVLink

Connect-IBFDRInfinibandPCIe Gen3

ConnectX-4EDR Infiniband

CAPI overPCIe Gen3

ConnectX-5Next-GenInfiniband

EnhancedCAPI overPCIe Gen4

Mellanox

Interconnect

Technology

IBMCPUs

NVIDIAGPUsKepler

PCIe Gen3Volta

EnhancedNVLinkPascalNVLink

IBMNodes

15

Page 16: OpenPOWER Roadmap Toward CORAL

NewIBM“LC”PowerSystemsforLinux

812LCCPU

Memory

1xPOWER8CPU10cores,2.9-3.3GHz

Upto1TeraByteMemory115GB/sMemoryBandwith14Drives(84TB,HDD,SSD)

OptimizedforHadoop,Spark,

In-MemoryAnalytics

822LCCPU

Memory

OptimizedforDatabasesand

CloudWorkloads

CPU

2xPOWER8CPU10coreseach,2.9-3.3GHzUpto1TeraByteMemory

230GB/sMemoryBandwith2Drives(2TB,HDD,SSD)

822LCHPC

CPU

Memory

BuiltforHPC andDeep

Learning

CPU

2xPOWER8CPU10coreseach,2.9-3.3GHzUpto1TeraByteMemory

230GB/sMemoryBandwith2xNVIDIATeslaK80GPUs

GPU GPU

16

Page 17: OpenPOWER Roadmap Toward CORAL

IBMSoftwareDefinedInfrastructurefordiverseworkloads

IBMPlatformComputing

BigData/

Hadoop

High

Performance

Analytics

HighPerf

Computing

Makelotsofcomputers&storagelooklikeoneresourcepool

ScaleUpandOut

Prioritizedmatchingofsupplywithdemand

Benefits

ü Efficientutilizationü HighAvailabilityü HighThroughputü HighPerformanceü SLAPrioritizationü Reducedcost

IBMSpectrumStorage

x86 Linuxonz

IBMFlashSystem

IBMHybridStorage IBMTapeStorage

On-premise,On-cloud,HybridInfrastructure

(heterogeneous distributedcomputing andstorageenvironment)

Other

Commercial

Other

NewGen

App

Framework

Other

17

Page 18: OpenPOWER Roadmap Toward CORAL

IBMPlatformLSF10.1(Q22016)

• Productivity-centric, enterprise-class workload management for HPC environments

• Intelligent policy-based data management

• Optimized for both high throughput and traditional HPC applications

12xlesswork

150xlesswork

6xlesswork

LSF9.1.2(2013) LSF9.1.3.3(2015)5.8million jobs/hr

Highthroughputsyntheticbenchmarkexternallyauditedandverified

Superiorrepeatableperformance=betterbusinessoutcomes

IBMConfidentialuntilannounced

EaseofUse

•Understandwhyworkloadisnotrunning

•Understandwhenworkwillrun

HPCAccessibility

•Webenabledworkflows•Simplifymigrationtothecloud

Performance

atScale

• Delivering nextevolution ofperformanceandscalability

18

Page 19: OpenPOWER Roadmap Toward CORAL

Wellcome TrustSangerInstituteUsinganalyticstooptimizehighperformancecomputingcapacityacross15,000cores

ProblemThe Wellcome Trust Sanger Institute, a non-profit research organization based in Hinxton, England, wanted to accelerate results to meet scientific journal publication deadlines and secure funding for future projects.

Solution• Platform LSF® policy-based scheduler• IBM® Platform™ Analytics with Custom dashboards showing cluster

utilization according to criteria such as job, job submitter & memory use• IBM® Platform™ Process Manager for workflow design and executionOutcome• Each job runs on the optimal compute node increasing throughput across

the HPC environment• Researchers understand the factors that reduce processing efficiency,

and take action to improve performance• A clear record of efficient compute utilization enables a

strong business case for future growth investments

Industry: ResearchProducts: IBMPlatform™LSF®

IBMPlatformAnalyticsIBMPlatformProcessManager

Overview

Profile

“AnalyticsisoneofthemostsignificantITinvestmentsthatwehavemade.WeneededtheIBMsolution todelivertheHPCutilizationrequiredtohelpourresearchteamsgenerateresultsrapidly,meettheirpublicationdeadlinesand,ultimately,securenewfunding.”

Dr.PeterClaphamPrincipalSystemsAdministrator,InformaticsSystemsGroup,Wellcome TrustSangerInstitute

19

Page 20: OpenPOWER Roadmap Toward CORAL

IBMSpectrumScale4.2andIBMElasticStorageServer

• Reducecapitalandoperatingexpenseforstoragesupportingverylargescaledata• Enableacommonpoolofstorageformodernworkloadswithunifiedfileandobjectstorage• Policy-drivencompressionandQualitiesofService

• Accelerateanalyticsapplications• EliminateneedtomovedatabackandforthfromHadoop/HDFSstoragesiloswithintegratedHDFS

• AutomateBigDatadeploymentwithApacheAmbari support

• Speedandsimplifydeployment

• NewSpectrumScaleGUIconsistentwithSpectrumStoragefamily• ExpandmanagementoptionswithSpectrumControlintegration• Fullyfunctionalvirtualmachinedemonstrationlowersbarrierstoadoption

AvailablesinceNovember20(IBMSpectrumScale)November6(IBMElasticStorageServer)

20

Page 21: OpenPOWER Roadmap Toward CORAL

DESY- ResearchinstituteinGermanyAcceleratingexperimentaloutcomeswithIBMSoftwareDefinedInfrastructure

Solution components

The Challenge Deutsches Elektronen-Synchrotron needed a solution that could keep up with one of the most powerful x-rays in the world, rapidly delivering experimental results to researchers

The SolutionA high-performance IBM storage and compute solution to support increasingly sophisticated experiments and offers analysis-as-a-service, allowing scientists to tap into DESY’s particle accelerator facilities and access the data from anywhere in the world.

The Results§ Accelerates research with data available for analysis in

4 minutes, down from 2-3 days§ Accommodates more demanding projects, handling 20GB/second§ Scales up for continuing innovation

• IBM® Elastic Storage™ Server• IBM Spectrum Scale• IBM Platform HPC• IBM Power Systems™ running Linux• IBM Advanced Business Partner:

MCS Moorbek Computer Systeme GmbH

"The ability to provide scientists with data on very short time scales has changed the way experiments are done at the beam line.”--Steve Aplin, senior scientist 21

Page 22: OpenPOWER Roadmap Toward CORAL

ActionableIntelligencewithOptimizedPerformanceIBMFlashSystemandSpectrumScale

IBMFlashSystem poweredbyFlashCore™Technology

isoptimizedtodeliverextremeperformance,reliability,andlowlatencyinthemostdemandingenvironments

IBM FlashSystem

All active files

Eliminate costly performance bottlenecks

IBMFlashSystem

Hot Files

HDD Storage All other files

Data Metadata

IBM FlashSystemSpectrum Scale

Primary storage

1MillionIOPS

ResponseTimesunder0.5millisecond

Over6GB/ssequential

readthroughput

Asacachedevice Asastoragetier Asametadatastoragedevice

22

Page 23: OpenPOWER Roadmap Toward CORAL

LeadingObjectStorageInnovation:SpectrumScaleandCleversafe

IBMofferscomplementaryObjectstorageofferings- eachdeliveringindustryleadinginnovationsfordifferentworkloads/usecases

• Cleversafe isdesignedtoenablesecure,largescaleactivearchivesandcloud-basedcontentrepositorieswherefocusisonlowestcostatscale• Erasurecodingdeliverscost-efficientreliability,requiringlessspacethantriplecopyapproaches• UniquekeylessencryptiontechnologyprovidesstrongsecurityforObjectarchives

• SpectrumScaleofferslargescaleFile& Objectstoragethatsupportshighperformanceprocessingandbigdataanalytics• Supportsmultiplearchitectures:storagerichservers;NASclusters;globallydistributedgrids• POSIXsupportforscale-outOLTPdatabasesanddatawarehouses• IntegratedlifecyclemanagementacrossFlash,DiskandTapetiers

23

Page 24: OpenPOWER Roadmap Toward CORAL

HighPerformanceStorageSystem(HPSS)

ImprovedstorageeconomicsanddataavailabilityOpenStack Swift

• OakRidgeNationalLaboratorycutredundanttapecostsby75%with4+PHPSSRAITandupto872MB/sfile-transfers

• Enterprisetaperecommendedaccessorder(RAO)forfastertaperecalls• HPSSTrashCan• Filesystemandtraditionaldiskstoragetiers• ProductionreadyonPower8• OpenStack “SwiftOnHPSS”providesobjectinterfaceforfilesharing• Stopcorruptedfilesfrommakingittotapewithend-to-enddataintegrity• 219MB/spertapedriveusing2.6MBfiles

24

Page 25: OpenPOWER Roadmap Toward CORAL

IBMishelpingourclients…

ü Expandfromtraditionalcompute-centrictoadata-centricHPCmodelü Efficientlymanagethelifecycleofalldataincludingfilesandobjectsü Achievenewinsightsfromalldatawhileimprovingdataeconomicsü Store,analyzeandprotectdatawithahyperscale convergedinfrastructure

MaximizePerformance

EnableNewWorkloads

OptimizeEfficiency

25

Page 26: OpenPOWER Roadmap Toward CORAL

MoreInformationFormoreinformation:

• Highperformancecomputing(HPC)andtechnicalcomputingsolutions• http://www-03.ibm.com/systems/power/solutions/high-performance-computing/

• http://www-03.ibm.com/systems/technicalcomputing/

• FindIT:yourgatewaytofindandusethetopassetsyouneedforanygivensalessituation• https://findit.cloud.dst.ibm.com

• Redbook:ImplementinganIBMHigh-PerformanceComputingSolutionon

IBMPOWER8• http://www.redbooks.ibm.com/abstracts/sg248263.html?Open=

26

Page 27: OpenPOWER Roadmap Toward CORAL

Questions?

27