EnergyEfficientHPCWorkingGroup

NatalieBates8thAnnualWorkshop

AGENDA:•  Teamreports–•  EEHPCWGNextStepsandDirecEon–Wheretonext•  SCPrimer–whatelsetodo…BoF,Panels•  IndustryLinks,AcEviEes,andCollaboraEon

–  ASHRAETC9.9,TheGreenGrid,Top500,Green500,PRACE

OBJECTIVE:•  EncourageparEcipaEoninEEHPCWG

TEAMREPORTS:•  DavidGrant,OakRidgeNaEonalLaboratory•  DaleSartor,LawrenceBerkeleyNaEonalLaboratory•  DavidMarEnez,SandiaNaEonalLaboratory•  AnnaMariaBaily,LawrenceLivermoreNaEonalLaboratory•  MichaelPaXerson,IntelCorporaEon•  SuzanneRivoire,SonomaStateUniversity•  TorstenWilde,LeibnizSupercompuEngCenter•  KevinPedre[,SandiaNaEonalLaboratory•  SteveMarEn,CrayInc.

•  Impactsofthework•  Increaseeaseofdeploymentforliquidcoolingcontrols•  Improveenergyefficiencyofthecoolingsystemandreducecostsofcooling

•  Deliverables•  “EEHPCWGLiquidCoolingControlsTeamWhitepaper”;hXps://eehpcwg.llnl.gov/pages/infra_ctrls.htm;2017.•  StateofthepracEcecasestudiesforliquidcoolingcontrolsystems

•  CurrentacEviEes•  DiscussionswithASHRAE,RedfishandPowerAPIonincorporaEngthesedatainputsintheirrecommendaEons

•  Nextsteps•  InclusionofdatainputsinASHRAE,RedfishandPowerAPI&EEHPCWGProcurementConsideraEonsDocument2017•  TotalCostofOwnershipandHPCSystemProcurementBoF;Tuesday12:15•  Redfish,PowerAPIandGEOPMBoF;Tuesday17:15

•  Helpneeded•  AdvocatesforEEHPCWGtoworkonRedfishandPowerAPICommiXees•  Contributecasestudiesandlessonslearned

LiquidCoolingControlsDavidGrant,ORNL

–  Impactsofthework•  Encourageliquid-cooledsoluEonsthatdonotrequirecompressors•  Increaseeaseofdeploymentby“standardizing”facilityandHPCequipment•  SetthebarformoreopportuniEestoreusewasteheat

–  Deliverables•  Wxtemperatureclassesdeveloped,presented,andpublished(e.g.SC11),andincludedinASHRAETC9.9Liquid

CoolingGuidelinesforDatacomEquipmentCenters.

–  CurrentacEviEes•  EEHPCWGmembersprovidinginputandexperEsetodevelop“open”specificaEonforwarmwaterliquid

cooledrackwithmajorinternetcompanies

–  Nextsteps•  ConEnuetocommunicateandclarifyWxrecommendaEons•  ProvideinputtoliquidcooledrackspecificaEonharmonizingU.S.andChinesestandards(e.g.,OCPandScorpio)

–  Helpneeded•  InputondralspecificaEonsincludingfluidandconnectorsaswellasoperaEngcondiEons(e.g.temperatures

andpressure)

LiquidCooling“Standards”DaleSartor,LBNL

•  Impactsofthework•  EncouragedecreasedcostsandimproveenergyefficiencywitheffecEveliquidcoolingcommissioning

•  Deliverables•  “SystemaEcapproachforcommissioningliquidcoolinginfrastructuretosupportliquidcooledHPC

systems”;hXps://eehpcwg.llnl.gov/pages/infra_lccs.htm.2015.•  StateofthepracEcecasestudiesforliquidcoolingcommissioning

•  CurrentacEviEes•  IncluderecommendaEonsinEEHPCWGProcurementConsideraEonsDocument2017

•  Nextsteps•  TotalCostofOwnershipandHPCSystemProcurementBoF;Tuesday12:15•  ASHRAETC9.9toincorporateliquidcoolingcommissioningincommissioningguideline

•  Helpneeded•  Technicalexpertw/strongtechnicalwriEngskillstofinalizewhitepaperwithASHRAETC9.9CommiXee•  Contributecasestudiesandlessonslearned

LiquidCoolingCommissioningDaveMarEnez,SandiaNL

•  Impactsofthework•  IncreaseenergyandoperaEonalefficiencybyimprovingReliabilityAvailabilityServiceability(RAS)and

MaintainabilitybeyondtheHPCsystemtofacilityinfrastructure•  Deliverables

•  QuesEonnairecreatedandsenttomajorUSsupercompuEngsites•  (4)responsesindicatereliability,availability,serviceability(maintainability)extendsbeyondthesystems

•  CurrentacEviEes•  SoliciEngfeedbackonteamcreaEonfrommajorUSsupercompuEngsites•  DowecreateateamonHPCfacilitymaintainabilityandreliabilityasitrelatestoenergyefficiencyand

availability?•  Nextsteps

•  WaiEngforresponses•  Helpneeded

•  CompletethequesEonnaire•  SharebestpracEces/lessonslearned

RASandMaintainabilityAnnaMariaBailey,LLNL

•  Impactsofthework•  CombineswithPUEtoprovideaTOTALviewofwheretheinefficienciesare.Addsa“serverPUE”.Precludes

miscounEngpowerandcoolinglossesonthewrongsideoftheequaEon.•  Deliverables

•  “TUE,anewenergy-efficiencymetricappliedatORNL'sJaguar”;GaussBestPaperAward;ISC13InternaEonalSupercompuEngConference;Leipzig,Germany;2013.

•  StateofthepracEcecasestudiesforTUEandiTUE•  RecommendedcapabilityinEEHPCWGProcurementConsideraEonsdocument

•  CurrentacEviEes•  EvangelizingITUE/TUEintalksandconferences

•  Nextsteps•  DevelopanITUEcasestudiessessionfornextyearsworkshop

•  Helpneeded•  EngageTheGreenGridtopromoteiTUEandTUE•  ExploreyourabilitytomeasureoresEmateyouriTUEandTUE•  Contributecasestudies/lessonslearned

iTUEandTUEMikePaXerson,Intel

•  Impactsofthework•  ProvidesthefuncEonswhichcanmonitorandrecordpowerconsumpEonofenEresysteminrealEme•  MoreaccurateHPCsystemarchitecturaltrenddatafortheHPCCommunity

•  Deliverables•  StateofthepracEcecasestudiesonpowermeasurementmethodology•  “EnergyEfficientHighPerformanceCompuEngPowerMeasurementMethodology(version1.0)”;2012•  “Apower-measurementmethodologyforlargescale,highperformancecompuEng”.ACM/SPECInternaEonalConferenceonPerformanceEngineering;2014.•  “NodeVariabilityinLarge-ScalePowerMeasurements:PerspecEvesfromtheGreen500,Top500&EEHPCWG”.SC15;2015.•  “EnergyEfficientHighPerformanceCompuEngPowerMeasurementMethodology(version2.0RC1.0)”;

hXps://www.top500.org/green500/resources/eehpc-wg-power-measurement-methology/;2015.•  “SubmissionsOpenforNewlyMergedTOP500andGreen500”;May6,2016;RichBrueckner;InsideHPC.

•  CurrentacEviEes•  Solicitandunderstandfeedbackonsystem-levelworkloadpowermeasurementmethodology•  EncourageL2/L3measurementsubmissionstoGreen500/Top500

•  Nextsteps•  Top500/Green500ListL2/L3MeasurementfeedbackpresentedatSC17Green500BoF,Wednesday17:15

•  Helpneeded•  MakeL2/L3measurementsubmissionstoTop500/Green500List•  EncourageextensionofL2/L3measurementsubmissionstootherbenchmarks,e.g.,GreenGraph500

PowerMeasurementMethodologySuzanneRivoire,SonomaState

•  Impactsofthework•  RaisingawarenessofevolvingrelaEonshipbetweenSCsandtheirElectricityServiceProviders–  Thelandscapeischanging-gettoknowyourESPandtheirpartners.–  ImplementconEngencyplanningforpowermanagementwhileminimizingimpacttousers.

•  Deliverables•  "TheElectricalGridandSupercompuEngCenters:AnInvesEgaEveAnalysisofEmergingOpportuniEesandChallenges”;

EnergiinformaEk;Zurich,Switerland;2014.•  "SupercompuEngCentersandElectricityServiceProviders:AGeographicallyDistributedPerspecEveonDemandManagement

inEuropeandtheUnitedStates”;ISC16InternaEonalSupercompuEngConference;Frankfurt,Germany;2016.

•  CurrentacEviEes•  WriEngapaperthatexamineselectricityservicecontractsinmajorSupercompuEngCenters(SC)

•  Nextsteps•  Finalizeandpublishnextpaper

•  Helpneeded•  Contributecasestudies/lessonslearned

ElectricGridIntegraEonTorstenWilde,LRZ

•  Impactsofthework•  SharebestpracEcesofEnergyandPowerAwareJobSchedulingandResourceManagement(EPAJSRM)and

learnfromeachother•  IdenEfyopportuniEesforinfluencingproductdevelopment

•  Deliverables•  Interviewresultsfrom9sitesthathavedeployedoraredoingtechnologydevelopment(TD)withtheintent

todeploylargescaleEPAJSRMcapabilityinaproducEonenvironment•  CurrentacEviEes

•  AnalyzingdatafromsurveyoflargescaleEPAJSRMdeployments•  Nextsteps

•  EPAJSRMPoster,EPAJSRMBoF;Wednesday12:15,SolwareandEnergyEfficiencyPanel;Friday10:30.•  Whitepaper•  Technicalpaper

•  Helpneeded•  JointheteamandhelpwithdatacollecEon,analysisandwriEngthepaper•  IdenEfyotherpotenEalsiteswithlargescaleEPAJSRMinproducEonofTDwithanintenttodeploy

EPAJSRMKevinPedre[,SandiaNL

•  Impactsofthework•  InfluenceproductdevelopmenttodriveenergyefficientHPCsystems

•  Deliverables•  StateofthepracEcecasestudiesforenergyefficiencyconsideraEonsinprocurement•  ComponentandSystemIntegratorresponsestoEEHPCWGProcurementConsideraEons•  EnergyEfficiencyConsideraEonsforHPCProcurementDocuments:2014andEnergyEfficiencyConsideraEonsforHPC

ProcurementDocuments:2013hXps://eehpcwg.llnl.gov/pages/compsys_pro.htm•  CurrentacEviEes

•  UpdaEngprocurementconsideraEonsdocumentfor2017withveXedmaterial(e.g.,liquidcoolingcontrols)•  CollaboraEngwithPRACE(PartnershipforAdvancedCompuEnginEurope)onTCOandProcurement

•  Nextsteps•  PublishEnergyEfficiencyConsideraEonsforHPCProcurementDocuments:2017•  StartworkingonEnergyEfficiencyConsideraEonsforHPCProcurementDocuments:2018•  TotalCostofOwnershipandHPCSystemProcurementBoF;Tuesday12:15

•  Helpneeded•  ParEcipateontheProcurementConsideraEonsTeamandhelpwritedocuments•  SharebestpracEceprocurementdocuments

ProcurementConsideraEonsSteveMarEn,CrayInc.

•  Impactsofthework•  StriveforconsensusonHPCcenterdashboardenergyefficiencyelementsandmetrics

•  Deliverables•  "Re-examiningHPCEnergyEfficiencyDashboardElements”;12thWorkshoponHighPerformance

PowerAwareCompuEng•  “GeneralRecommendaEonsforHighPerformanceCompuEngDataCenterEnergyManagement

DashboardDisplay”;9thWorkshoponHigh-PerformancePower-AwareCompuEngConference

•  CurrentacEviEes•  QuesEonnaire-currentuseofdashboardsatmajorsupercompuEngcentersinUSA,Europe&Japan

•  Nextsteps•  AnalyzequesEonnaireresultsandwrite/publishpaper

•  Helpneeded•  ParEcipateintheDashboardTeamandhelpwithanalyzingdataandwriEngpaper

DashboardsNatalieBates,EEHPCWG

•  PotenEalNewTeams•  Maintainability•  PowerAPI•  Warmwatercooling–rackstandard•  FacilityHighVoltageandDirectCurrent•  EnergyRe-use•  OtherSolwareTeams

•  ExpandpresencefromUSandEuropetoJapan

EEHPCWGNextStepsandDirecEon–Wheretonext?

IndustryLinksandCollaboraEons

LinkstorelevantBoFDocuments

SC17EEHPCWGPrimer

•  WorkshopLunch–  LiquidCooling–  Maintainability–  SWandEnergyEfficiency–  SystemPowerMeasurement

•  Booth#286–  LiquidCooling:TuesdayandWednesday10:30amtoNoon–  ReliabilityAvailabilityServiceabilityandMaintainability:Tuesday2:00pm-4:00pm–  SolwareandEnergyEfficiency:Wednesday11:00amtoNoon,3:00pm–4:00pm

DiscussionGroups

Wevalueyourfeedback!

hXps://www.surveymonkey.com/r/Y9F3MCN

Thank you!!h"p://eehpcwg.llnl.govnatalie.jean.bates@gmail.com