overview of ddr4 memory in hpe proliant gen9 servers … · improvement over the 1866 mt/s memory...

Overview of DDR4 memory in HPE ProLiant Gen9 Servers with Intel Xeon E5-2600 v3 Best Practice Guidelines

Technical white paper


Contents Introduction ................................................................................................................................................................................................................................................................................................................................................... 3 Overview of DDR4 memory technology ............................................................................................................................................................................................................................................................................. 3

Basics of DIMMs ................................................................................................................................................................................................................................................................................................................................. 3 Introduction to DDR4 memory ............................................................................................................................................................................................................................................................................................. 4 DDR4 DIMM types ........................................................................................................................................................................................................................................................................................................................... 4 HPE ProLiant Gen9 memory resilience with DDR4 SmartMemory .................................................................................................................................................................................................... 5 HPE SmartMemory .......................................................................................................................................................................................................................................................................................................................... 5 HPE Advanced Memory Error Detection .................................................................................................................................................................................................................................................................... 5 DDR4 versus DDR3 memory performance ............................................................................................................................................................................................................................................................... 6

ProLiant Gen9 memory architecture for servers with Intel Xeon E5-2600 v3 processors ................................................................................................................................................. 6 Overview .................................................................................................................................................................................................................................................................................................................................................... 6 ProLiant Gen9 servers using the Intel Xeon processor E5-2600 v3 product family ......................................................................................................................................................... 7 ProLiant Gen9 Intel Xeon processor E5-2600 v3 product family ....................................................................................................................................................................................................... 8

DDR4 DIMMs for ProLiant Gen9 servers with ES-2600 v3 processors ............................................................................................................................................................................................... 9 Populating DDR4 memory in ProLiant Gen9 servers ........................................................................................................................................................................................................................................ 10

Population rules for ProLiant Gen9 servers .......................................................................................................................................................................................................................................................... 10 DIMM Population Order .......................................................................................................................................................................................................................................................................................................... 10

Optimizing memory configurations ..................................................................................................................................................................................................................................................................................... 12 Optimizing for capacity ............................................................................................................................................................................................................................................................................................................ 12 Optimizing for performance................................................................................................................................................................................................................................................................................................. 12 Optimizing DDR4 performance using Quick Path Interconnect (QPI) Snoop Modes .................................................................................................................................................... 16 Optimizing for lowest power consumption ............................................................................................................................................................................................................................................................ 19 Optimizing for Resiliency ........................................................................................................................................................................................................................................................................................................ 21

Understanding unbalanced memory configurations ........................................................................................................................................................................................................................................... 21 Memory configurations that are unbalanced across channels ............................................................................................................................................................................................................ 21 Memory configurations that are unbalanced across processors ....................................................................................................................................................................................................... 22

Settings for memory operation ............................................................................................................................................................................................................................................................................................... 22 Controlling memory speed ................................................................................................................................................................................................................................................................................................... 23 Setting memory interleave .................................................................................................................................................................................................................................................................................................... 23

Appendix A—Sample Configurations for 2P ProLiant Gen9 servers .................................................................................................................................................................................................. 24 24 DIMM slot servers using the Intel Xeon processor E5-2600 v3 product family ........................................................................................................................................................ 24 16 DIMM Slot Servers using the Intel Xeon processor E5-2600 v3 product family ....................................................................................................................................................... 25


Introduction This paper provides an overview of the new DDR4 Smart Memory and its use in the 2-socket Hewlett Packard Enterprise ProLiant Gen9 servers using the Intel® Xeon® processor Gen9 product family. HPE ProLiant Gen9 servers with Intel Xeon E5-2600 v3 processors support DDR4 memory—with its faster data rates, lower latencies and greater power efficiency than the DDR3 memory used in the three previous generations of HPE ProLiant servers. HPE ProLiant Gen9 servers with DDR4 memory also support HPE SmartMemory, which provides superior performance over 3rd party memory in certain configurations.

The 2-socket HPE ProLiant Gen9 servers feature similar memory architecture to that introduced with Gen8 servers. HPE ProLiant Gen9 servers using the Intel Xeon processor E5-2600 v3 product family support 4 separate memory channels per CPU and up to 24 DIMM slots—allowing large memory configurations and delivering improved memory performance. They also incorporate HPE Advanced Memory Protection technology, which improves the prediction of critical memory error conditions.

In addition to describing these improvements, this paper reviews the rules, best practices, and optimization strategies that should be used when installing DDR4 memory on HPE ProLiant Gen9 servers.

Overview of DDR4 memory technology Basics of DIMMs Before exploring the new technologies in DDR4 DIMMs for ProLiant Gen9 servers, let’s quickly review some of the basics of DIMM technology.

DRAM technology DIMMs are made up of DRAM chips that are grouped together to form one or more ranks. Each DRAM chip contains arrays of individual bit storage locations. A DRAM chip with one billion storage locations is called 1 Gigabit (1 Gb) technology. Note the lower case b in Gb. Eight 1 Gb chips ganged together will provide 1 Gigabyte (1 GB) of memory. Note the upper case B in GB.

DDR4 DIMMs currently use 4 Gb and 8 Gb DRAM chips. It is not possible to mix DRAM technologies on the same DIMM.

A DRAM chip may have 4 data I/O signals or 8 data I/O signals. These are called x4 or x8, pronounced “by four” or “by eight” respectively.

Ranks A rank is a group of DRAM chips that are grouped together to provide 64 bits (8 Bytes) of data on the memory bus. All chips in a rank are controlled simultaneously by the same Chip Select, Address and Command signals. DDR4 DIMMs are available in single-, dual- and quad-ranks (1, 2, and 4 ranks respectively.)

Eight x8 DRAM chips or 16 x4 chips form a rank. All DIMMs for HPE ProLiant servers with 8 bits of Error Correction Code (ECC) use nine x8 chips and 18 x4 chips for each rank.

Speed Speed refers to the frequency of the memory clock. The memory subsystem uses a different clock than the processor cores, and the memory controllers use this clock to coordinate data transfers between the memory controller and the DIMMs. The actual speed at which this clock operates in a particular server depends on five factors:

• Rated memory speed of the processor. Each Intel Xeon processor model supports a specific maximum memory speed.

• Rated memory speed of the DIMM. DDR4 DIMMs can run at different speeds, or frequencies. For ProLiant Gen9 servers with E5-2600 v3 processors, HPE offers two native speeds of DDR3 memory: DDR3-1866 and DDR3-1600.

• Number of ranks on the DIMM. Each rank on a memory channel adds one electrical load. As the electrical loads increase, the signal integrity degrades. To maintain the signal integrity the memory channel may run at a lower speed.

• Number of DIMMs populated on a channel. The number of DIMMs attached to a memory controller also affects the loading and signal integrity of the controller’s circuits. In order to maintain signal integrity, the memory controller may operate DIMMs at lower than their rated speed. In general, the more DIMMs that are populated, the lower the operational speed for the DIMMs.

• BIOS settings. Enabling certain BIOS features can affect memory speed. For example, the ROM Based Setup Utility (RBSU) in HPE ProLiant servers includes a user-selectable setting to force memory to run at a slower speed than the normally configured speed in order to save on power consumption. See the section on BIOS settings for details.


Introduction to DDR4 memory DDR4, the fourth-generation of DDR SDRAM technology, uses refinements in memory technologies to deliver improvements in both bandwidth and power consumption over the DDR3 memory used in the previous three generations of HPE ProLiant servers.

DDR4 Memory Technology DDR4 memory incorporates several key enhancements designed to increase performance, reduce power consumption and increase reliability compared to DDR3 memory. The DDR4 connector uses 288 pins (compared to 240 pins on DDR3). These extra signals are crucial to delivering the improved performance of DDR4 memory over DDR3. The notch on the DDR4 DIMM connector is also located in a different position relative to the DDR3 connector. This prevents the accidental insertion of a DDR3 DIMM into a DDR4 system. Key technology improvements for DDR4 include all of the following:

• 1.2 volt operation. All DDR4 memory operates at 1.2 volts, compared to 1.35 or 1.5 volt operation of DDR3 memory. This delivers significant system power savings, particularly in larger memory configurations.

• 16 banks of memory per rank. Internally, the DRAMs used in DIMMs are organized into arrays of cells defined by banks, rows and columns. DDR4 memory has 16 banks of memory in a DRAM chip compared to the 8 banks in DDR3. This allows an increase in the number of memory requests that can be queued up by the memory controller. It is one of the contributors to the lower latency of DDR4 memory.

• Encoded Rank Selection. DDR4 eliminates the work-around known as rank multiplication that DDR3 employed to enable 4 ranks of memory on LRDIMMs using the traditional chip select lines. When there are eight or fewer total ranks installed on a memory channel, DDR4 uses the traditional direct chip select mode to address the correct rank. When more than eight ranks are installed, DDR4 uses a 4-bit encoded chip select value for rank selection. This encoded value is interpreted by the registers on the DIMMs to determine the correct rank to enable for the memory operation. This new encoded chip select scheme allows DDR4 memory to theoretically address up to 24 memory ranks on a memory channel.

• Retry on error. DDR4 memory and the new memory controllers will retry a memory request whenever a memory error or address parity error occurs. This reduces the number of system halts that may have occurred due to transient errors in previous generations of memory subsystems.

DDR4 Speeds The DDR4 specification defines eventual data rates of up to 3200 Mega transfers per second (MT/s), more than 70% faster than the 1866 MT/s of the last iteration of DDR3 memory speed. The DDR4 memory for HPE ProLiant Gen9 2 socket servers will operate at data rates of up to 2133 MT/s—depending on the memory configuration and the specific version of the E5-2600 v3 processor in the server. This is a 14% improvement over the 1866 MT/s memory speed supported in the last generation of servers with DDR3 memory.

DDR4 DIMM types ProLiant Gen9 servers with Intel Xeon E5-2600 v3 processors support two DIMM types—Registered Memory (RDIMMs), and Load Reduced Memory (LRDIMMs). Although Unbuffered DIMMs (UDIMMs) are defined for the DDR4 standard, they no longer offer any performance advantage (in terms of lower latencies) over RDIMMs and LRDIMMs. For that reason, the E5-2600 v3 family of processors do not support UDIMMs.

Registered DIMMs Registered DIMMs (RDIMMs) improve signal integrity by having a register on the DIMM to buffer the address and command signals between the DRAMs and the memory controller. This allows each memory channel to support up to three dual-rank DIMMs, increasing the amount of memory that a server can support. With RDIMMs, the partial buffering slightly increases both power consumption and memory latency.

Load Reduced DIMMs DDR3 Load Reduced DIMMs were introduced on ProLiant Gen8 servers and are continuing to evolve with DDR4 memory. LRDIMMs use memory buffers to consolidate the electrical loads of the ranks on the LRDIMM to a single electrical load, allowing them to have up to 8 ranks on a single DIMM module. Using LRDIMMs you can configure systems with the largest possible memory footprints. However, LRDIMMs also use more power and have longer latencies compared to the lower capacity RDIMMs.


Comparing DIMM Types Table 1 provides a quick comparison of RDIMMs and LRDIMMs for ProLiant Gen9 servers using the 2P Intel architecture.

Table 1. Comparison of RDIMMS and LRDIMMs for ProLiant Gen9 servers with E5-2600 v3 processors

Feature RDIMM LRDIMM

DIMM Sizes Available 4 GB, 8 GB, 16 GB, 32 GB 16 GB, 32 GB, 64 GB

Advanced ECC support Yes Yes

Address parity Yes Yes

Rank Sparing Yes Yes

Lock-Step Mode Yes Yes

Maximum capacity on a server with 16 DIMM slots 512 GB 1024 GB

Maximum capacity on a server with 16 DIMM slots 768 GB 1536 GB

HPE ProLiant Gen9 memory resilience with DDR4 SmartMemory HPE ProLiant Gen9 servers with E5-2600 v3 processors support internal retries on memory operations that return an address parity error, and uncorrectable ECC memory error or a correctable ECC memory error. In some instances, these errors can be caused by rare but transient conditions and may be corrected by a retry of the operation.

Gen8 severs using the E5-2600 v2 processors and DDR3 memory were able to manage correctable errors, but did not perform retries on address parity errors or uncorrectable ECC errors. These would cause immediate system shutdowns.

In order to avoid data corruption, Gen9 servers will still perform system shutdowns if retries for these operations fail—indicating a non-transient memory error. However, testing indicates that internal retries on address parity errors and uncorrectable ECC errors can reduce the number of system shutdowns due to memory errors by up to 70 percent without sacrificing memory integrity.

HPE SmartMemory ProLiant Gen9 servers with E5-2600 v3 processors support HPE SmartMemory technology for DDR4 memory. HPE SmartMemory enables authentication of installed memory. This verifies whether DIMMs have passed our qualification and testing processes and determines if the memory has been optimized to run on HPE ProLiant Gen9 servers. Using HPE SmartMemory DIMMs enables extended performance and manageability features for the 2P ProLiant Gen9 servers. HPE SmartMemory supports extended performance compared to third party memory for several DIMM types and configurations. Table 2 summarizes these performance extensions.

Table 2. Extended performance for HPE SmartMemory DDR4 DIMMs in 2P ProLiant Gen9 servers with E5-2600 v3 processors

DIMM Type 1 DIMM per channel 2 DIMMs per channel 3 DIMMs per channel

2133 MT/s RDIMMs 2133 MT/s (SmartMemory) 2133 MT/s (3rd Party)

2133 @ 1.2V (SmartMemory) 1866 @ 1.2V (3rd Party)

1600 MT/s (SmartMemory) 1600 MT/s (3rd Party)

2133 MT/s LRDIMMs 2133 @ 12V (SmartMemory) 2133 @ 1.2V (3rd Party)



HPE Advanced Memory Error Detection Over the past five years, the average size of server memory configurations has increased by more than 500%. With these increased memory capacities, increases in memory errors are unavoidable. Fortunately, most memory errors are both transient and correctable. Current memory subsystems can correct up to a 4 bit memory error in the 64 bits of data that are transferred in each memory cycle.

HPE Advanced Memory Error Detection technology introduces refinements to error detection technology. Instead of simply counting each correctable memory error, this new technology analyzes all correctable errors to determine which ones have a higher probability of leading to uncorrectable errors in the future. Using this advanced approach, HPE Advanced Memory Error Detection is able to better monitor the memory subsystem and increase the effectiveness of the Pre-Failure Alert notification. All ProLiant Gen9 servers feature HPE Advanced Memory Error Detection.


DDR4 versus DDR3 memory performance As we noted earlier, DDR4 memory operates at a 14% faster clock rate and 20% lower voltage than DDR3 memory. This—combined with other architectural improvements in DDR4 memory—delivers significant improvements in memory latency, throughput, and power efficiency over DDR3.

Latency

Table 3. Comparison of Idle Latency—DDR4 versus DDR3 DIMMs. (8 GB RDIMMs installed at 1 DIMM per channel)

DDR3 DDR4 Difference

Idle Latency

(8 GB 1Rx4 RDIMMs)

61.3 ns 52.7 ns 14% decrease

Idle Latency

(8 GB 2Rx4 RDIMMs)

57.9 53.2 ns 8% decrease

Throughput and Power

Table 4. Comparison of throughput and power—DDR4 versus DDR3 DIMMs. (Eight 16 GB 2Rx4 RDIMMs installed at 1 DIMM per channel)

DDR3 (1866 MT/s) DDR4 (2133 MT/s) Difference

Throughput (Reads) 103.9 GB/s 123.0 GB/s 18% increase in throughput

Idle Power 6.4 watts 2.4 watts 63% decrease in idle power

Loaded Power 45.6 watts 44.0 watts 4% decrease in power

ProLiant Gen9 memory architecture for servers with Intel Xeon E5-2600 v3 processors Overview The DDR4 memory architecture for 2 socket ProLiant Gen9 servers is very similar to that of ProLiant Gen8 servers. 2 socket HPE ProLiant Gen9 servers using E5-2600 v3 processors feature 4 DDR4 memory channels per processor and 16 or 24 total memory slots. In addition to the performance advantages the DDR4 memory provides, 2 socket ProLiant Gen9 servers also feature advances in processor/memory controller architecture that can provide additional performance and reliability over previous generations. These include:

• Built-in memory resilience capability that decreases system shutdowns due to memory errors by up to 70 percent.

• Support for using one of three possible QPI Snoop Modes for performing memory snoops across processors.


Figure 1. A block diagram of this memory architecture

ProLiant Gen9 servers using the Intel Xeon processor E5-2600 v3 product family There are several models of 2P ProLiant Gen9 servers that use the Intel Xeon E5-2600 v3 processors. These are shown in Table 5.

Table 5. 2P HPE ProLiant Gen9 servers using E5-2600 v3 processors

HPE ProLiant server model Number of DIMM slots Maximum Memory

DL380 Gen9 24 1.5 TB

DL180 Gen9 16 1 TB

DL360 Gen9 24 1.5 TB

DL160 Gen9 16 1 TB

ML150 Gen9 16 1 TB

ML350 Gen9 24 1.5 TB

BL460c Gen9 16 1 TB

WS460c Gen9 Graphics Server 16 1 TB


ProLiant Gen9 Intel Xeon processor E5-2600 v3 product family There are a number of processor models of the Intel Xeon processor E5-2600 v3 product family. Processor models differ in their number of cores, maximum processor frequency, amount of cache memory, and features supported (such as Intel Hyper-Threading Technology). Different processor models also support different maximum memory speeds. This affects the maximum performance and the power consumption of the memory subsystem and of the server in general.

Table 6. Intel Xeon processor E5-2600 v3 product family for HPE ProLiant Gen9 servers

Processor Model Number

CPU Frequency Cores Level 3 Cache Size Power QPI Speed Maximum Memory Speed

E5-2699v3 2.3 GHz 18 45 MB 145 W 9.6 GT/s 2133 MT/s E5-2698v3 2.3 GHz 16 40 MB 135 W 9.6 GT/s 2133 MT/s E5-2697v3 2.6 GHz 14 35 MB 145 W 9.6GT/s 2133 MT/s E5-2695v3 2.3 GHz 14 35 MB 120 W 9.6GT/s 2133 MT/s E5-2690v3 2.6 GHz 12 30 MB 135 W 9.6 GT/s 2133 MT/s E5-2687Wv3 3.1 GHz 10 25 MB 160 W 9.6 GT/s 2133 MT/s E5-2683v3 2.0 GHz 14 35 MB 120 W 9.6 GT/s 2133 MT/s E5-2680v3 2.5 GHz 12 30 MB 120 W 9.6 GT/s 2133 MT/s E5-2670v3 2.3 GHz 12 30 MB 120 W 9.6 GT/s 2133 MT/s E5-2667v3 3.2 GHz 8 20 MB 135 W 9.6 GT/s 2133 MT/s E5-2660v3 2.6 GHz 10 25 MB 105 W 9.6 GT/s 2133 MT/s E5-2650v3 2.3 GHz 10 25 MB 105 W 9.6 GT/s 2133 MT/s E5-2650Lv3 1.8 GHz 12 30 MB 65 W 9.6 GT/s 2133 MT/s E5-2643v3 3.4 GHz 6 20 MB 135 W 9.6 GT/s 2133 MT/s E5-2640v3 2.6 GHz 8 20 MB 90 W 8.0 GT/s 1866 MT/s E5-2637v3 3.5 GHz 4 15 MB 135 W 9.6 GT/s 2133 MT/s E5-2630v3 2.4 Ghz 8 20 MB 85 W 8.0 GT/s 1866 MT/s

E5-2630Lv3 1.8 GHz 8 20 MB 55 W 8.0 GT/s 1866 MT/s

E5-2623v3 3.0 GHz 4 10 MB 105 W 8.0 GT/s 1866 MT/s

E5-2620v3 2.4 GHz 6 15 MB 85 W 8.0 GT/s 1866 MT/s

E5-2609v3 1.9 GHz 6 15 MB 85 W 6.4 GT/s 1600 MT/s

E5-2603v3 1.6 GHz 6 15 MB 85 W 6.4 GT/s 1600 MT/s


DDR4 DIMMs for ProLiant Gen9 servers with ES-2600 v3 processors HPE ProLiant Gen9 servers with the Intel Xeon E5-2600 v3 processors support the use of DDR4 DIMMs specified at a maximum speed of 2133 MT/s. Table 7 lists the DDR4 DIMMs that are qualified for use in these servers.

Table 7. HPE DDR4 DIMMs for ProLiant Gen9 servers with Intel Xeon E5-2600 v3 processors

Registered DIMMs (RDIMM) HPE Part Number

HPE 4 GB (1x4 GB) Single Rank x8 DDR4-2133 CAS-15-15-15 Registered Memory Kit 726717-B21

HPE 8 GB (1x8 GB) Single Rank x4 DDR4-2133 CAS-15-15-15 Registered Memory Kit 726718-B21 726718-B21

HPE 8 GB (1x8 GB) Dual Rank x8 DDR4-2133 CAS-15-15-15 Registered Memory Kit 759934-B21 759934-B21



Load Reduced DIMMs (LRDIMMs)

HPE 16 GB (1x16 GB) Dual Rank x4 DDR4-2133 CAS-15-15-15 Load Reduced Memory Kit 726720-B21

HPE 32 GB (1x32 GB) Quad Rank x4 DDR4-2133 CAS-15-15-15 Load Reduced Memory Kit 726722-B21

HPE 64 GB (1x64 GB) Quad Rank x4 DDR4-2133 CAS-15-15-15 Load Reduced Memory Kit 726724-B21

HPE part descriptions use codes from the JEDEC standard for specifying DIMM types and speeds. For DDR4 memory, some of these description codes have changed in both format and position compared to DDR3. DDR4 memory has the following changes to DIMM labelling from DDR3 memory:

• DIMM speed is referenced by the bit rate (e.g. 2133 for 2133 MT/s) rather than the DIMM module bandwidth (e.g. 17000 for 17 GB/s bandwidth)

• CAS latency now comes before DIMM type and is referenced by a letter rather than the number of clock cycles.

Figure 2 shows a breakdown of the DIMM label nomenclature for DDR4 DIMMs.

Figure 2. HPE DDR4 Memory Part Number Decoder


Populating DDR4 memory in ProLiant Gen9 servers The high-level memory system architecture for HPE ProLiant Gen9 2-socket servers with E5-2600 v3/v4 processors is in most ways the same as that of Gen8 servers. As with Gen8 servers, ProLiant Gen9 2-socket servers have either 24 or 16 memory slot configurations. As a result, the population rules for these servers are similar. HPE recommends populating all memory channels whenever possible. This ensures the best memory performance.

Population rules for ProLiant Gen9 servers For optimal performance and functionality, you should obey the following rules when populating HPE ProLiant Gen9 2-socket servers with DDR4 memory. Violating these rules may result in reduced memory capacity or error messages during boot. Table 9 summarizes the overall DIMM population rules for HPE ProLiant Gen9 2-socket servers.

Table 8. DIMM population rules for HPE ProLiant Gen9 2-socket servers

Category Population Guidelines

Processors and DIMM Slots Install DIMMs only if the corresponding processor is installed

White DIMM slots denote the first slot to be populated in a channel

Black DIMM slots denote the second slot to be populated in a channel

Blue DIMM slots denote the last slot to be populated in a channel

Place the DIMMs with the highest number of ranks in the white slot when mixing DIMMs of different ranks on the same channel

Performance Balance the total memory capacity across all installed processors and load the channels similarly whenever possible

DIMM Types and Capacities RDIMMs and LRDIMMs can’t be mixed in the same system

128 GB LRDIMMs can’t be mixed with other capacity DIMMs

DIMM Speeds DIMMs of different speeds may be mixed in any order

The server will select the lowest common speed of the DIMMs/CPU

DIMM Population Order Figure 3 shows the memory slot configuration for the 24-slot ProLiant DL380 Gen9 2-socket server. In this drawing, the first memory slots for each channel on each processor are the white memory slots (A, B, C, and D). The second memory slots for each channel on each processor are the black memory slots (E, F, G, and H). The third memory slot for each channel on each processor are the blue memory slots (I, J, L, and K).

Figure 3. DIMM slots and population order for 24 slot ProLiant Gen9 2-socket servers


In general, memory population order follows the same logic for all ProLiant servers—although the processors’ physical arrangement may vary from server to server. To populate the server memory in the correct order, you should use the following rules:

• When a single processor is installed in the system, install DIMMs in sequential alphabetical order—A, B, C, D, and so on.

• When two processors are installed in the server, install DIMMs in sequential alphabetical order—P1-A, P2-A, P1-B, P2-B, and so on.

• Within a given channel, you should populate DIMMs from the heaviest electrical load (dual-rank) to the lightest load (single-rank).

For more information, consult the User Guide for your particular HPE ProLiant Gen9 server model.

Figure 4 shows the memory slot configuration for 16-slot ProLiant Gen9 2-socket servers. The configuration is similar to the 24-slot servers, but the 16-slot servers have two DIMM slots per channel instead of three. Again, the first memory slots for each channel on each processor are the white memory slots (A, B, C, and D). You should populate the memory for 16-slot servers using the same rules as those for 24-slot servers.

Figure 4. DIMM slots and population order for 16 slot 2-socket ProLiant Gen9 servers

Populating systems using DDR4 DIMMs and NVDIMMs HPE 8 GB nonvolatile DIMMs (NVDIMMs) represent a new hybrid memory/storage technology known as Persistent Memory. Persistent Memory is a high-speed storage medium with the general performance of memory and the persistence of traditional storage. NVDIMMs use the existing DIMM slots in an HPE ProLiant Gen9 server and will co-exist with installed DIMMs—but they are not part of the server’s general memory. Instead, they are seen by operating systems that support them as block-level or byte-level storage devices. When populating a system with both DIMMs and NVDIMMs, the following rules apply:

• NVDIMMs are supported only on HPE ProLiant DL360 Gen9 and HPE ProLiant DL380 Gen9 servers featuring Intel E5-2600v4 processors.

• Only RDIMMs can be mixed with HPE NVDIMMs. You cannot use LRDIMMs with NVDIMMs.

• When installing NVDIMM(s) on the same memory channel as RDIMM(s), populate the RDIMM(s) first and farthest from the processor, then populate the NVDIMM(s) last and closest to the processor.

• When NVDIMMs exist in the system, there must be a minimum of one RDIMM installed in a DIMM slot corresponding to the first CPU socket.

• A single processor may have more than eight NVDIMMs, but the total number of NVDIMMs among the two processors may not exceed 16. Balanced memory configuration between the two processors and between memory channels is still recommended to maximize performance.

• NVDIMMs follow the general memory population rules and guidance.


Optimizing memory configurations By taking advantage of the different DIMM types and sizes available for HPE ProLiant Gen9 servers, you can optimize server memory configuration to meet different application or datacenter requirements.

Optimizing for capacity You can maximize memory capacity on ProLiant Gen9 servers using 128 GB LRDIMMs. With LRDIMMs, you can install up to three octal-ranked DIMMs in a memory channel. On 24-slot servers, you can configure the system with up to 3072 GB of total memory.

Table 9 shows the maximum memory capacities for ProLiant Gen9 servers using each of the two DIMM types.

Table 9. Maximum memory capacities for HPE ProLiant Gen9 2-socket servers using different DDR4 DIMM types

Number of DIMM Slots DIMM Type Maximum Capacity Configuration

24 RDIMM 768 GB 24 x 32 GB 2R

LRDIMM 3072 GB 24 x 128 GB 8R

16 RDIMM 512 GB 16 x 32 GB 2R

LRDIMM 2048 GB 16 x 128 GB 8R

Optimizing for performance The two primary measurements of memory subsystem performance are throughput and latency. Latency is a measure of the time it takes for the memory subsystem to deliver data to the processor core after the processor makes a request. Throughput measures the total amount of data that the memory subsystem can transfer to the system processor(s) during a given period—usually one second.

Factors influencing latency Unloaded and loaded latencies are a measure of the efficiency of the memory subsection in a server. Memory latency in servers is usually measured from the time of a read request in the core of a processor until the data is supplied to that core. This is also called load-to-use. Unloaded latency measures the latency when the system is idle and represents the lowest latency that the system can achieve for memory requests for a given processor/memory combination. Loaded latency is the latency when the memory subsystem is saturated with memory requests. Loaded latency will always be greater than unloaded latency.

There are a number of factors that influence memory latency in a system.

• DIMM Speed. Faster DIMM speeds deliver lower latency, particularly loaded latency. Under loaded conditions, the primary contributor to latency is the time that memory requests spend in a queue waiting to be executed. The faster the DIMM speed, the more quickly the memory controller can process the queued commands. For example, memory running at 2400 Megatransfers per second (MT/s) has about 5% lower loaded latency than memory running at 2133 MT/s.

• Ranks. For the same DDR4 memory speed and DIMM type, more ranks will tend to slightly increase the loaded latency. While more ranks on the channel give the memory controller a greater capability to parallelize the processing of memory requests and reduce the size of request queues, it also requires the controller to issue more refresh commands. The benefits of greater parallelizing outweighs the penalty of the additional refresh cycles up to 4 ranks. The net result is a slight reduction in loaded latencies for 2 to 4 ranks on a channel. With more than 4 ranks on a channel there is a slight increase in loaded latency.

• CAS latency. CAS (Column Address Strobe) latency represents the basic DRAM response time. It is specified as the number of clock cycles (e.g., 13, 15, 17) that the controller must wait after issuing the Column Address before data is available on the bus. CAS latency is a constant in both loaded and unloaded latency measurements.

• Utilization. Increased memory bus utilization does not change the low-level read latency on the memory bus. Individual read and write commands are always completed in the same amount of time regardless of the amount of traffic on the bus. However, increased utilization causes increased memory system latency due to latencies accumulating in the queues within the memory controller.


Figure 5 shows both unloaded and loaded latency numbers for various DDR4 DIMMs when using one DIMMs per channel (DPC) configurations. Because the primary component of idle latency is the memory system overhead of performing the basic memory operations, it is almost the same for every DIMM type, capacity, and speed. LRDIMMs have slightly higher idle latencies than RDIMMs due to the data buffer on the LRDIMM.

Figure 5. Idle and Loaded Latencies for various DDR4-2400 DIMMs on 2P HPE ProLiant Gen9 servers with E5-2600 v4 processors

Factors influencing memory throughput Factors affecting memory throughput include the number of memory channels populated, number of ranks on the channel (rank interleaving), channel interleaving, and the speed at which the memory is running. The DIMM operating speed is determined by the capability of the DIMM, the capability of the CPU, and the number of DIMMs on a channel.

Number of memory channels and throughput The largest impact on throughput is the number of memory channels populated. By interleaving memory access across multiple memory channels, the integrated memory controllers are able to increase memory throughput significantly. Optimal throughput and latencies are achieved when all channels of each installed CPU are populated identically.


As Figure 6 shows, adding a second DIMM to the system (and thus populating the second memory channel) essentially doubles system read throughput. Gains in throughput for each additional DIMM installed increase linearly until all eight memory channels are populated.

Figure 6. System throughput with one, two, four, six and eight channels populated in a 2-socket system

Memory speed and throughput Higher memory speeds increase throughput. Figure 7 shows that maximum system memory read throughput at 2400 MT/s is about 14% greater than at 2133 MT/s when using a one DPC configuration,. Throughput at 1600 MT/s memory speed is only about 70% of the throughput at 2400 MT/s. Lowering the memory speed will save power, but it has a significant negative effect on maximum throughput.

Figure 7 also illustrates how the number of active cores affects memory throughput. More cores deliver better throughput, but only up to a point. Even when we configure server memory for a highly-demanding application, the use of 20 cores where every core is accessing memory as quickly as it can, would be somewhat of a strenuous situation. When more than 20 cores are active, maximum throughput levels out due to the saturation of the activity in the memory subsystem itself. Unlike previous generations of servers, however, throughput does not start to decline as more than 20 cores are activated. Improvements in processor and system architecture in ProLiant Gen9 servers keep the memory subsystem from becoming oversaturated when a larger number of cores are activated and causing a reduction in throughput.

Figure 7. Memory Throughput as a function of data rate and number of active CPU cores using eight 16 GB 2Rx4 RDIMMs at one DPC


Figure 8 shows the measured memory throughput for one, two, and three DPC configurations using several different DIMM types. Maximum throughput increases slightly when a second RDIMM is added to each channel, but decreases slightly when a second LRDIMM is added to the channel. Decreases in throughput with two LRDIMMs installed are due to increased overhead of the LRDIMM buffers. Using three DPC causes a significant drop in maximum throughput. This is primarily due to the lower memory speeds for most three DPC configurations as well as increased refresh overhead.

Figure 8. Throughput of DDR4 memory by DIMM type at one, two, and three DPC

If you are optimizing a system for maximum loaded throughput, configurations of one or two DPC deliver the best results due to their ability to maintain 2400 MT/s memory speed.

Throughput benefits of extended performance at 2400 MT/s HPE has engineered ProLiant Gen9 servers using HPE SmartMemory to operate at higher memory speeds than comparable third-party memory in several key configurations. For ProLiant Gen9 servers with E5 2600 v4 processors, HPE SmartMemory supports 2400 MT/s operation for configurations with two dual-rank RDIMMs per channel. The standard third-party RDIMMs only supports two DPC configurations operating at the slower 2133 MT/s memory speed. Enabling 2400 MT/s operation for two HPE SmartMemory RDIMMs per channel delivers higher throughputs and decreases in loaded latency. For a configuration using two 16 GB RDIMMS per channel, this results in an 11.4% increase in throughput and a 10.5% decrease in loaded latency (Table 10).

Table 10. Increased throughput with two 16 GB DDR4-2400 dual-ranked RDIMMs per channel at 2400 MT/s versus 2133 MT/s

2 DIMMS per channel 2133 MT/S 2 DIMMS per channel 2400 MT/S % Change

Throughput (GB/s) 124.9 139.2 11.4% higher

Idle Latency (ns) 56.64 53.8 5% lower

Loaded Latency (ns) 186.2 166.6 10.5% lower

50

60

70

80

90

100

110

120

130

140

150

8GB 1Rx8 RDIMM

16GB 1Rx4 RDIMM

16GB 2Rx4 RDIMM

32GB 2Rx4 RDIMM

32GB 2Rx4 LRDIMM

64GB 4Rx4 LRDIMM

128GB 8Rx4 LRDIMM

Thro

ughp

ut (G

B/s

)

DDR4-2400 Throughput by DIMM Type 1, 2 AND 3 DIMMs per channel

1 DIMM

2 DIMMs

3 DIMMs


HPE DDR4-2400 LRDIMMs are capable of operating at 2400 MT/s in configurations of three DPC in the HPE ProLiant DL380/DL360 Gen9 servers—two full speed steps higher than the reference standard. Table 11 shows the additional throughput and latency advantages that these configurations can deliver.

Table 11. Increased throughput with three 32 GB DDR4-2400 LRDIMMs per channel at 2400 MT/s versus 1866 MT/s

3 DIMMS per channel 1866 MT/S 3 DIMMS per channel 2400 MT/S % Change

Throughput (GB/s) 102.1 129.8 27% higher

Idle Latency (ns) 60.9 60.9 0%

Loaded Latency (ns) 121.4 120.8 5% lower

HPE ProLiant Gen9 servers using DDR4 SmartMemory show similar throughput and latency advantages for all memory configurations where HPE SmartMemory operates at a higher memory speed than the corresponding third-party DIMMs.

Mixing DIMM sizes There are no performance implications for mixing sets of different capacity DIMMs at the same operating speed. For example, latency and throughput will not be negatively impacted by installing eight 16 GB dual-rank DDR4-2400 DIMMs (one per channel), plus eight 32 GB dual-rank DDR4-2400 DIMMs (one per channel).

General guidelines For optimal throughput and latency, populate all four channels of each installed CPU identically.

Optimizing DDR4 performance using Quick Path Interconnect (QPI) Snoop Modes In NUMA architected systems—where memory channels are distributed across multiple processors—all memory requests require snoop operations in order to maintain data coherency. Snooping is the act of checking the content of the caches of processors to determine if a copy of the requested data resides in any of the caches.

Whenever memory is requested, the requesting processor must snoop both the local processor caches and the caches in the remote processor. The Local Snoop operation determines if the requested memory is currently stored in the caches of any of the processor’s cores. If is not found—and the memory address requested is local to that processor—then it performs a local memory access to retrieve the data. As part of the same operation, the processor’s Home Agent (part of the Integrated Memory Controller) also issues a Remote Snoop across the QPI bus. This operation determines if the requested memory is in one of the caches of the remote processor. If the requested memory address is one that is attached to the remote processor and is not found in cache, then the Remote Snoop process will retrieve the information from the remote processor’s attached memory. In all cases, the Remote Snoop will deliver its results to the local calling processor. At the end of the operation, the most current copy of the data is used.


Home Snoop Mode On HPE ProLiant Gen8 servers, the Snoop operations are performed through the processor’s memory controller and its Home Agent in the manner described above. On HPE ProLiant Gen9 servers, this is the default QPI Snoop Mode and is referred to as Home Snoop Mode (Figure 9).

Figure 9. Home Snoop Mode on E5-2600 v4 processors with 10 or more cores

E5-2600 v4 processors with 10 or more cores also feature two separate Home Agents and Integrated Memory Controllers (IMC) rather than one. Each Home Agent and controller works with two of the four memory channels. This parallelized architecture contributes to the improved memory performance of these processors. E5-2600 v4 processors with fewer than 10 cores feature a single Home Agent and IMC.

HPE ProLiant Gen9 servers with E5-2600 v4 processors also provide two new QPI Snoop Modes that were first introduced with the E5-2600 v3 generation of processors—Early Snoop Mode and Cluster-on-die Mode. Using these modes may deliver potential performance benefits for specific types of server application loads. You can configure the QPI Snoop Mode for the server using HPE’s configuration and management utilities.


Early Snoop Mode In Early Snoop Mode, the Cache Agent inside a given processor core initiates the Remote Snoop directly, bypassing the Home Agent within the memory controller and therefore decreasing the time to perform the Remote Snoop operation. Early Snoop Mode may provide better overall throughput and latency performance by decreasing the Remote Snoop times for the processor subsystem. However, Home Snoop mode may deliver better performance in application environments that have high throughput requirements.

Figure 10. Early Snoop Mode on E5-2600 v4 processors


Cluster-on-Die Mode In Cluster-on-Die (COD) Mode, the system configures each E5-2600 v4 processor as two separate clusters of processing cores. Each cluster has its own dedicated Ring Bus, Home Agent, and QPI channel (Figure 11). COD Mode requires servers with processors that have two Home Agents (i.e., those with more than 10 cores) and can be used in one and two processor configurations.

Figure 11. Cluster-on-Die Mode on E5-2600 v4 processor

By grouping cores and Home Agents in this manner, Cluster-on-die mode can provide the best throughput and latency of any of the three modes in situations when cores do not share data (i.e., they only access memory addresses associated with their Home Agent/IMC). Achieving this typically requires NUMA-aware applications that operate within the local memory spaces.

Choosing the correct QPI Snoop Mode Choosing the best QPI Snoop Mode for your server is not an apparent method. Many variables and interactions are involved in any specific server and application environment configuration. HPE recommends benchmarking and testing your specific application environment to determine which, if any, QPI Snoop mode may provide a distinct performance advantage.

Optimizing for lowest power consumption Several factors determine the power that a DIMM consumes in a system, including:

• DIMM technology

• DIMM capacity

• Number of DIMM ranks

• DIMM operating speed


DIMM type and capacity Overall, larger capacity DIMMs—which are powering multiple ranks of DRAMs—consume more power. However, on a per gigabyte basis they are more efficient. A 32 GB LRDIMM consumes 8.1 Watts under load, but this is a little over half the power per GB of an 8 GB RDIMM.

Figure 12. Power by DIMM capacity. Loaded and idle power for each DIMM type and capacity supported on HPE ProLiant Gen9 servers

Memory speed and power consumption DIMMs running at higher speeds consume more power than the same DIMMs running at a lower speed. Memory operating at 2400 MT/s consumes about 26% more power under loaded conditions than the same memory running at 1600 MT/s.

Figure 13. Power consumed by a single 16 GB RDIMM at different memory speed settings

0.00

2.00

4.00

6.00

8.00

10.00

12.00

8GB 1Rx8 RDIMM

16GB 1Rx4 RDIMM

16GB 2Rx4 RDIMM

32GB 2Rx4 RDIMM

32GB 2Rx4 LRDIMM

64GB 4Rx4 LRDIMM

128GB 8Rx4 LRDIMM

DIM

M P

ower

(W)

Power consumed by DIMM type and capacity HPE ProLiant Gen9 servers

Intel Xeon E5-2600 v4 processors

Idle Power Loaded Power (Read)

0

1

2

3

4

5

6

1600 MT/s 1866 MT/s 2133 MT/s 2400 MT/s

DIM

M P

ower

(Wat

ts)

DIMM Power consumption by memory speed Single 16 GB 2Rx4 RDIMM

Idle Power Loaded power


General guidelines when optimizing for power consumption When optimizing for lowest power consumption, you can use the following general rules.

• If you can meet your memory size requirements with them, use RDIMMs instead of LRDIMMs. You can configure 2-socket ProLiant 24 slot Gen9 servers with as much as 768 GB using RDIMMs.

• Use the smallest number of DIMMs possible, by using the highest capacity DIMM needed to configure your server with the required amount of total system memory.

• For additional power savings with any memory configuration, you can run memory at the slowest speed possible. With HPE ProLiant Gen9 Servers, this is 1600 MT/s.

Optimizing for Resiliency DDR4 DIMMs may be constructed using either 4-bit wide (x4) or 8-bit wide (x8) DRAM chips. Current ECC algorithms used in the memory controllers are capable of detecting and correcting memory errors up to 4 bits wide. For DIMMS constructed using x4 DRAMs, this means that an entire DRAM chip on the memory module can fail without causing a failure of the module itself. DIMMs constructed using x8 DRAMs cannot tolerate the failure of DRAM chip. The ECC algorithm can detect the failure, but it cannot correct it. As a result, systems configured with DIMMs using x4 DRAMs are safer from potential memory failures than those using memory consisting of x8 DRAMs. RDIMMs may be constructed with x4 or x8 DRAMs. With DDR4 memory, all LRDIMMs use x4 DRAMS.

Understanding unbalanced memory configurations Unbalanced memory configurations are those in which the installed memory is not distributed evenly across the memory channels and/or the processors. HPE discourages unbalanced configurations because they will always have lower performance than similar balanced configurations. There are two types of unbalanced configurations, each with its own performance implications.

• Unbalanced across channels. A memory configuration is unbalanced across channels if the memory capacities installed on each of the four channels of each installed processor are not identical.

• Unbalanced across processors. A memory configuration is unbalanced across processors if a different amount of memory is installed on each of the processors.

Memory configurations that are unbalanced across channels In unbalanced memory configurations across channels, the memory controller will split memory up into regions, as shown in Figure 14. Each region of memory will have different performance characteristics. The memory controller groups memory across channels as much as possible to create the regions. It will create as many regions as possible with DIMMs that span all four memory channels, since these have the highest performance. Next, it will move to create regions that span two memory channels and then to just one.

Figure 14. A memory configuration that is unbalanced across memory channels


The primary effect of memory configurations that are not balanced across channels is a decrease in memory throughput in those regions that span fewer memory channels. In the example above, measured memory throughput in Region 2 may be as little as 25% of the throughput in Region 1.

Memory configurations that are unbalanced across processors Figure 15 shows a memory configuration that is unbalanced across processors. The CPU1 threads operating on the larger memory capacity of CPU1 may have adequate local memory with relatively low latencies. The CPU2 threads operating on the smaller memory capacity of CPU2 may consume all available memory on CPU2 and request remote memory from CPU1. The longer latencies associated with the remote memory will result in reduced performance of those threads. In practice, this may result in non-uniform performance characteristics for program threads, depending on which processor executes them.

Figure 15. A memory configuration that is not balanced across processors

Settings for memory operation The HPE server BIOS provides control over several memory configuration settings for ProLiant Gen9 servers. You can access and change these settings using the ROM Based Setup Utility (RBSU), which is part of all HPE ProLiant servers. To launch RBSU, press the F9 key during the server boot sequence.


Controlling memory speed Using RBSU, you can set the speed at which the system memory runs to a specific value. This function is available from the Power Management Options menu inside RBSU. With ProLiant Gen9 servers, memory bus speed can be set to any of the following:

• Automatic (speed determined according to normal population rules)

• 2133 MT/s

• 1866 MT/s

• 1600 MT/s

You can set the memory speed to a lower value than maximum to lower power consumption. However, this will also reduce the performance of the memory system.

Setting memory interleave Disabling Memory Interleaving This option is available from the Advanced Power Management menu in RBSU. Disabling memory interleaving saves some power per DIMM, but also decreases overall memory system performance.

Setting Node Interleaving This feature controls how the server maps the system memory across the processors. Activate Node Interleaving from the RBSU Advanced Options menu. The system’s default is Node Interleaving disabled. When disabled, BIOS maps the system memory such that the memory addresses for the DIMMs attached to a given processor are together, or contiguous. For conventional applications this arrangement is more efficient, because the processors will access directly the memory addresses containing the code and data for the programs they are executing. When Node Interleaving is enabled, system memory addresses are alternated, or interleaved, across the DIMMs installed on both processors. In this case, each successive page in the system memory map is physically located on a DIMM attached to a different processor. For a subset of specific workloads—in particular those using shared data sets—the system may operate at a higher level of performance with Node Interleaving enabled.


Appendix A—Sample Configurations for 2P ProLiant Gen9 servers 24 DIMM slot servers using the Intel Xeon processor E5-2600 v3 product family DIMM Type

SKU Description DIMM Size

DIMM Ranks

DIMMs per Channel

Number of DIMMs

Total System Memory (GB)

Oper. Data Rate

Unloaded Latency (ns)

Through put (Reads) (GB/s)

Total Idle Power (W)

Total Loaded Power (W)

RDIMM 726717-B21 4 GB 1Rx8 4 GB 1Rx8 1 8 32 2133 52.6 118.6 0.8 21.6

2 16 64 2133 52.2 123.0 1.6 33.6

3 24 96 1600 54.3 92.6 2.4 40.8

RDIMM 726718-B21 8 GB 1Rx4 8 GB 1Rx4 1 8 64 2133 52.7 118.8 1.6 32.8

2 16 128 2133 52.6 122.6 3.2 48

3 24 192 1600 54.5 93.0 4.8 60

RDIMM 759934-B21 8 GB 2Rx8 8 GB 2Rx8 1 8 64 2133 53.2 122.7 1.6 28.8

2 16 128 2133 53.5 125.6 3.2 43.2

3 24 192 1600 60.8 92.7 4.8 52.8

RDIMM 726719-B21 16 GB 2Rx4 16 GB 2Rx4 1 8 128 2133 53.0 123.0 2.4 44

2 16 256 2133 53.5 124.7 4.8 62.4

3 24 384 1600 60.8 92.7 7.2 79.2

RDIMM 728629-B21 32 GB 2Rx4 32 GB 2Rx4 1 8 256 2133 54.7 121.8 3.0 53.2

2 16 512 2133 56.1 124 6.4 81.0

3 24 768 1600 60.9 92 9.1 103.4

LRDIMM 726720-B21 16 GB 2Rx4 16 GB 2Rx4 1 8 256 2133 55.6 120.9 4 62.4

2 16 512 2133 56.2 117.9 8 91.2

3 24 768 1866 61.0 106.9 12 127.2

LRDIMM 726722-B21 32 GB 4Rx4 32 GB 4Rx4 1 8 768 2133 56.1 122.0 6.4 79.2

2 16 768 2133 55.2 119.0 12.8 118.4

3 24 768 1866 60.8 101.6 19.2 163.2

LRDIMM 726724-B21 64 GB 4Rx4 64 GB 4Rx4 1 8 512 2133 60.4 120.7 5.6 72

2 16 1024 2133 57.5 115.3 11.2 102.4

3 24 1536 1866 60.9 75.4 16.8 136.8


Sign up for updates

© Copyright 2015, 2017 Hewlett Packard Enterprise Development LP. The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein.

Intel and Intel Xeon are trademarks of Intel Corporation in the U.S. and other countries.

4AA6-2997ENW, June 2017, Rev. 1

16 DIMM Slot Servers using the Intel Xeon processor E5-2600 v3 product family

DIMM Type

SKU Description DIMM Size

DIMM Ranks

DIMMs per Channel

Number of DIMMs

Total System Memory (GB)

Oper. Data Rate

Unloaded Latency (ns)

Through put (Reads) (GB/s)

Total Idle Power (W)

Total Loaded Power (W)

RDIMM 726717-B21 4 GB 1Rx8 4 GB 1Rx8 1 8 32 2133 52.6 118.6 0.8 21.6

2 16 64 2133 52.2 123.0 1.6 33.6

RDIMM 726718-B21 8 GB 1Rx4 8 GB 1Rx4 1 8 64 2133 52.7 118.8 1.6 32.8

2 16 128 2133 52.6 122.6 3.2 48

RDIMM 759934-B21 8 GB 2Rx8 8 GB 2Rx8 1 8 64 2133 53.2 122.7 1.6 28.8

2 16 128 2133 53.5 125.6 3.2 43.2

RDIMM 726719-B21 16 GB 2Rx4 16 GB 2Rx4 1 8 128 2133 53.0 123.0 2.4 44

2 16 256 2133 53.5 124.7 4.8 62.4

RDIMM 728629-B21 32 GB 2Rx4 32 GB 2Rx4 1 8 256 2133 54.7 121.8 3.0 53.2

2 16 512 2133 56.1 124 6.4 81.0

LRDIMM 726720-B21 16 GB 2Rx4 16 GB 2Rx4 1 8 256 2133 55.6 120.9 4 62.4

2 16 512 2133 56.2 117.9 8 91.2

LRDIMM 726722-B21 32 GB 4Rx4 32 GB 4Rx4 1 8 256 2133 56.1 122.0 6.4 79.2

2 16 512 2133 55.2 119.0 12.8 118.4

LRDIMM 726724-B21 64 GB 4Rx4 64 GB 4Rx4 1 8 512 2133 60.4 120.7 5.6 72

2 16 1024 2133 57.5 115.3 11.2 102.4

Resources, contacts, or additional links HPE servers Technical White Papers library hpe.com/docs/servertechnology

HPE ProLiant Server Memory main page hpe.com/us/en/servers/memory.html

HPE DDR4 Smart Memory Configurator h22195.www2.hpe.com/DDR4memoryconfig/

http://www.hpe.com/info/getupdated

http://www.facebook.com/sharer.php?u=https://www.hpe.com/h20195/V2/GetDocument.aspx?docname=4AA6-2997ENW

http://www.hpe.com/docs/servertechnology

http://www.hpe.com/us/en/servers/memory.html

http://h22195.www2.hpe.com/DDR4memoryconfig

https://www.hpe.com/global/hpechat/index.html?jumpid=Collaterals_4AA6-2997ENW

http://twitter.com/home/?status=Populating%20DDR4%20memory%20in%20ProLiant%20Gen9%20servers+%40+https://www.hpe.com/h20195/V2/GetDocument.aspx?docname=4AA6-2997ENW

http://www.linkedin.com/shareArticle?mini=true&ro=true&url=https://www.hpe.com/h20195/V2/GetDocument.aspx?docname=4AA6-2997ENW&title=Populating%20DDR4%20memory%20in%20ProLiant%20Gen9%20servers+&armin=armin

overview of ddr4 memory in hpe proliant gen9 servers … · improvement over the 1866 mt/s memory...

Documents