第七章 分布共享存储多处理机

Click here to load reader

Upload: cecile

Post on 22-Feb-2016

128 views

Category:

Documents


0 download

DESCRIPTION

第七章 分布共享存储多处理机. 第一节 分布存储与可扩展性. 可扩展通信网络. 一、分布 存储的系统结构. ···. 节点. CA. NIC. 1 、分布 存储的系统 结构 有共享存储、非 共享 存储 2 种. C. M. Shell. P. LD. *NUMA/NORMA 系统与 UMA 系统的 区别: 存储系统 — 一致性模型的实现不同 ( 复杂 ) 通信模型 — 编程 模型、通信辅助部件 ( CA) 的实现不同 同步操作 — 实现方法不同 ( 复杂 ). 回下页. 2 、系统结构相关设计内容. - PowerPoint PPT Presentation

TRANSCRIPT

PowerPoint

7431 (TTCAT)2 3 / 4 76662(WC) // P0P1R1W2S1W1 S3W3 R3W2 S2 R2W1 S2 SCSC 32(4) (5)(1) *Cache/ */(2) *() *CA(3) () -- CA4 *I/O *1 *() *IN2 t(n)=t+t+n/B +t */(p) IN22INt=1sB=64MBpst0=125ns/p641024128B *4t64(128)=1+0.125*6+128/64=3.75st1024(128)=4.25s13%-t64(128)=1+(0.125+128/64)*6=13.75st024(128)=22.25s62%(8B)t64(128)=1+(0.125+8/64)*6+128/64=4.5st1024(128)=5.5s22%

53 C(p,m)=C+C(p,m)pmMEM * IN2264CPUMEMIN(33.3%)1024IN *IN 4 3 * *5nIN/=(n)=log2nIN/=C*log2n1024CPU*0.33+MEM*0.33+IN*0.33*10/6=1.22IN33%45%6 * / / *() CA//(NIC)(IN) 671 I/O CA7821()()()()2 ()(/) () ()()89 (/) (-/) () 910 ()+ -- ()1 -/

Load r[] Store r[]//10+ +

112 Cache-- - , Cache. -- ()1112 () [][] () -- () ()12131 Psrc/

Send(Pdest,local,VA,len) Recv(Psrc,local,VA,len) 13

142 /

/ Send(Pdest,local,VA,len)Recv(Psrc,local,VA,len) 1416 PEPE() PEPE () NIC 16171 () ()182 () (-1) ()(NACK)()284819 CA () ()()/ CA () (NIC)1920DMACA 1CA CADMA DMAREGREG()P,CmdDMAAddrLengthStatusNICDMACAP,CmdDMAAddrLengthStatusNICDMACA21 OSOSDMA DMA I/ODMAREG2 23CA 1CA NIC/CACA/P,/P, OSFIFO()23OS

25CA (CP)1CA CPPCPNICMEMCPPCAMEMCPPCAMEMCPPCA PMEMCPMEMMEM CPMEMMEMP CP262Intel Paragon MEMP CPL1$i860XPL1$i860XPDMANIC2KB FIFO175Mbps64bit400MbpsCacheMem bus CANIC(/)REGDMAFIFO/Cache DMADMAPDMAPCacheCA(FIFO)DMA26NIREGFIFOFIFOFIFO27CA ()1CA CADestSrcAddrTagSrcTagRrspMEM$MMUPMEM$MMUPLD RAddr MEMMEM27 CACA MMU CA LD LD/ST swaperandfetch&addbarrier282CRAY T3DPEDEC Alpha(64) 8KBI-Cache+8KBD-Cache 43DTB32() [21PE#+FC()]DRAMSwaperandFetch&addbarrier$MMUPRespondReqReqRespondDMADTB3264MB52732 PE --5(DTB)+27(MEM) -PE(27) DTB--0#REGPEMMULL-SCREG28 5DTBAnnex0LL-SCDTBREG ADTBBDTBCDTBLD/ST PE29 /---PE#DTB(DTB)LD/ST CA CA5iDTB[i] NICCA --PE() --DMA64KB() REG303CADMA/31 Cache -- -- Cache MESIMESI Cache ---31IN()321 Cache Cache(M) Cache MESI MESI33 CacheRdRd1234a4bNICP$Mem/dirNICP$Mem/dirNICP$Mem/dirRdX123a4b3b4aNICP$Mem/dirNICP$Mem/dirNICP$Mem/dirNICP$Mem/dir MESI(M) 3334 Cache() n(PE)1(ONON)NICP1$NICPn$ ()Cache()363 373 ()1 37382 () =[(N+1)/8](M/K)N NMMEMK() O(MN 2) 393 i()i() =[(ilog2N+1)/8](M/K)N O(MNlog2N) ()()() i404 null0 $1 $2 $null =[(s2log2N+1)/8](M/K)N s O(MNlog2N) 5840414 --()LHR1:2:3:4a:4b:LHR1:4:2:3:a)-LHR1:2:3a:3b:b)c)LHR 5 41-443

4211 2 CachePL2$PL2$PL2$PL2$NICL2$PNICPPP PE42PPEP

432 1 PE FIFOPE ()() NACK4345SGIOrigin1Origin20001SysAD(1~4G)PL2$(1~4M)PL2$(1~4M)I/O HubCache(PEPE)4546 Hub(CA)1644()MEM/DIRNICI/OPE2 SPIDERCray512(1024PE)47 SPIDER-- 6162 66LLPSSD/SSRLLPSSD/SSRSSD/SSRLLPSSD/SSRLLPSSD/SSRLLPLLPSSD/SSR SSD SSR LLP Cray-- 87SPIDER4748 () 32SPIDER16328SPIDER4 32Cray Cray64512492Cache MESI1 (I)(S)(E/M) (RdBusy)(RdXBusy)(NCRdBusy) P(TLB)2 3 (Rd)(RdX)(WB) - +NACK57504LHRHRHRHRHRHRISEMBusyRdIa(11)Ia(12)b(1)ESb(2)MSe(1)RdXIa(21)Ic(1)S Id(1)EId(2)MIe(2)Sc(2)S Ie(2)Ec(3)Ie(3)MWBSfS S/EgXS/E/MEfIgIE/MMfIgIE/M H(HCache)R/LCache a(11)a(12)a(21) RR 51 LH1:Rd/RdX2:/(a)I/SRdIRdX(b)E/MRdLHR1:Rd2b:3b:3a:/2a:/LHR1:RdX/Upgr2a:2b:3:/(c)S/ERdX(d)MRd/RdXLHR1:RdX2b:3b:3a:/2a:/LH1:Rd/RdX /Upgr2:NACK(e)BusyLH1:WB2:(f)S/E/MWBLHR1:2b:2a:WB3b:3a:2c:(g)BusyWB523 ()(FIFO) --PEHubHubFIFO --Hub()PE -- NACK52552 SCI SCIIQ-PlusSCIQuadQuadIQ-PlusQuadQuad3IQ-Link IQ-LinkMEMPE(SCLIC)(OBIC)$$$SCI NICSCISCI56 OBIC--CacheMEMPE SCLIC--SCI Cache32MB4P6 SCLICOBICDRAMSRAM 2 26SCI() CacheCacheCache 4 76SCI()56572Cache PE L2$$MESI SCI IQ-Link$L2$ L2$-- $--L2$L1$L2$L1$L2$$OBIC$SCLICSCI NIC$$OBIC$SCLICSCI NIC$L1$L2$SCIMESI57581 L1$L2$MESI HOME(I)FRESH(E/S)GONE(M) $729 + ONLYHEADTAILMID DIRTYCLEANFRESHCOPY OD(ONLY-DIRTY)HDMDTDOFHFMFTF 2 58593 () () ()() - (a)HnullR1R2L(b)(c)HnullR1R2R1HR1nullR2HR2nullR1HnullR1R2LHnullR1R2R1 (Rd)(RdX)(WB) 59 Rd RdX61 HDnullTDHDnullMDTD ODTDHDMD RdxOD*F/*D()FRESH/GONE *FWBLHHOME(H)/FRESH(F) ~ODWBLOD623 () NACK(PENDING)FIFOPENDINGTF/TDPENDINGPENDINGAXBCABCpBackpBackABC 6263 --PEIQ-LinkFIFO --FIFOPE (pNext)- 64 PCWCRC 1(PC) ()65P0P1 R1W1W2W3 S S R3W2 R2W4 R1MEMFIFO WWRRRWWRR TSOWRIntelSPARC V8 PSOWRWWSPARC V9 673(RC) // / ()P0 R3P1SA1SA2SR1SR2W2 R1 R2W2 R1W1 R3 69 1 PE2 70Cache Cache Cache() SCPERCPE Cache / / / /71 1 T(n)=Tcomput+Tlocaldata+Tcomm+Tsynch+Tpara Tcomm=(T+TCA+T+T+T-T) T-- 72 ()787321 /PE 2 Tcomp=T+TTcomm=TCA+T+T S=(Tcomp+Tcomm)/Tcomp=1+Tcomm/Tcomp1.00.5(a) TcompTcommTcomm(b) TcompTcommTcommTcompTcomm S275 (CC-NUMA) 1 DSMPE DSM(SVM)Cache -- PE/ -- 762 (put) DMACache L2$L1$771 (Cache)CC-NUMA() Cache()CC-NUMA78 ()() --Cache1 a.Cache b.PC() 792 Cache a. b.3 (Cache) ()802 1 CTEXTCvAvBpApBAN2N1BN3N1C=B-A=vB-vAvA=rolad pAvB=rolad pB(a)CTEXTCvAvBpApBAN2N1BN3N1C=B-A=vB-vAABABN1(b)ready1ready2 Cache812 PEPEPEREG()CPUCache(a)PCPSWREGs PEPECPUCache(b)PCPSWREGsPCPSWREGs82 PPEPEPP()CPUCache(c)PPCPSWREGsPCPSWREGsCPUCache PEP833 REGREG1234nFPFP//I-CachePCI-FIFOPCI-FIFOPCI-FIFOD-CacheREGs()REGs()REGs()84 Cache 1~2 E1=R/(R+L)R L --Esat=R/(R+C)CC