he Sun SPARC Enterprise T5120/T5220 Servers are the first of Suns servers to implement the Sun UltraSPARC T2 processor. This processor uses four, six, or eight 64-bit SPARC cores, each of which supports eight threads running concurrently. The servers support one processor, and therefore support a maximum of 64 threads. The Sun SPARC Enterprise T5120/T5220 Servers use the on-chip multithreaded processor to minimize the CPU idle time, thus ensuring that multiple threads have access to the processor and do not have to wait for the processor to be completely freed to gain CPU access time. The processor is designed for highly threaded transactional processing. This means that the time usually spent waiting for memory access has been minimized, which maximizes the core utilization processes.The Sun SPARC Enterprise T5120/T5220 Servers implement the PCI-Express, or PCIe, as the primary I/O bus on the server. PCIe offers an advantage of speed and data transmission rate; it operates at 2.5 GHz and transmits data at approximately 250 Mbytes/ sec in each direction. PCIe provides a true bidirectional link, so with one link available, the server offers a data transmission rate of approximately 500 Mbytes/sec. The Sun SPARC Enterprise T5120 Server has two PCIe x4 slots and one PCIe x8 slot while the Sun SPARC Enterprise T5220 Server has four PCIe x4 and two PCIe x8 slots.Note: The x4 and x8 nomenclature refers to the number of lanes. Each lanes supports 250 Mbytes/second

he Sun SPARC Enterprise T5120/T5220 Servers utilize the UltraSPARC sun4v architecture, a hyper-privileged architecture.Enhancements in this line include:Use of the Sun UltraSPARC T2 processor which supports multiple threads on multiple cores and its conformance to the new architecture by segregating hardware-specific drivers from the OS.Implementation of virtualization, which introduces a layer between the operating system and the platform that removes the need for the operating system to have direct register access to the processor, memory, and critical I/O devices. This benefits customers because hardware can be upgraded without changing the software infrastructure.The Sun SPARC Enterprise T5120/T5220 Servers provide support for a new Ethernet driver, referred to as e1000g. This new driver provides the following features:Support for data link provider interface (DLPI) version 2, which enables a data link service user to access and use a variety of conforming data link service providers without special knowledge of the provider's protocol;Physical layer static configuration using FORTH code, or FCode properties;Portability to Solaris X86 and SPARC platforms through device driver interface, or DDI frameworks;Fault management infrastructure support, which provides error handling and management capabilities;Message signaled interrupts, or MSI support, which allows the devices to communicate in a peer-to-peer manner without the involvement of the host CPU.These implementations bring the gigabit Ethernet interface in line with Suns other network interface drivers.

The Sun SPARC Enterprise T5120/T5220 Servers implement RAS features, which enable the system to maintain a higher uptime rate. RAS features are incorporated across both hardware and software components, including:System bussesPCI bridgesProcessorsMemory managementSystem powerAnd system coolingThe inclusion of RAS features in these entry-level servers continues Suns commitment to providing enterprise-level services to lower-cost solutions. The RAS features included in the Sun SPARC Enterprise T5120/T5220 Servers include:N+1 redundant hot-pluggable power supplies and hot-swappable fan modulesExtended ECC protection on L2 cache data path and memory interfaceDRAM extended ECC, which allows the detection of up to 4-bits in error, as long as they are on the same DRAMStandardized error message generation Environmental monitoring with the inter- integrated circuit (I2C) control serial busRemote access using ILOM-based hardware and software componentsRemote monitoring using Sun Net Connect

The Sun SPARC Enterprise T5120/T5220 Servers implement several key technologies that give it a great advantage in the entry-level server market for companies that require database servers, web servers, and application servers. In this module, we have discussed:An overview of the Sun SPARC Enterprise T5120/T5220 Servers, including their features, target markets, and target applicationsThe key technologies driving the Sun SPARC Enterprise T5120/T5220 Servers, including the new processor, architecture, and use of a faster PCI-based bus.The Sun SPARC Enterprise T5120 server is a 1U rack mountable server.The Sun SPARC Enterprise T5220 server is a 2U rack mountable server.Both servers implement the newly available chip multithreading (CMT), UltraSPARC T2 processor. These entry-level servers are enterprise servers, designed to meet the needs of Suns customers who use Java servers and databases, such as Oracle, on the back end.These Sun servers implement the sun4v architecture and on-chip multithreading technology that allows them to process more information per processor, in highly threaded workloads, than competitors. The Sun SPARC Enterprise T5120/T5220 Servers implement the PCI-Express (PCIe) I/O bus, which provides a greater data transfer rate than previous PCI standards.The Sun SPARC Enterprise T5120/T5220 Servers implement several new features and enhancements. Overall, the servers feature improvements in processor throughput and IO technology.The next few slides describe the system specifications for the Sun SPARC Enterprise T5120/T5220 Servers, starting with the processor.The Sun SPARC Enterprise T5120/T5220 Servers are equipped with a single UltraSPARC T2 processor on-board that operates at 1.2 and 1.4 gigahertz per second (GHz/sec). The processor contains a 4 megabyte Level 2, or L2, cache, and four, six, or eight cores, each of which has one floating point unit, or FPU, and supports eight threads. Each core has a 16 kilobyte instruction cache and an 8 kilobyte data cache. The CPU also supports up to 16 FBDIMM slots, each of which supports 1, 2, and 4 gigabyte DIMMs with a maximum of 64 gigabytes supported in the Sun SPARC Enterprise T5120/T5220 Servers. The DIMMs are controlled by four memory channels, each of which controls four DIMM slots.The system I/O is built on the PCIe bus, which provides support for:Three standard half length/half height PCIe expansion slots on three riser boards in the Sun SPARC Enterprise T5120 server orSix standard half length/half height PCIe expansion slots on three riser boards in the Sun SPARC Enterprise T5220 serverIn addition to PCI Express expansion cards, two of the expansion slots can alternately accept Sun proprietary XAUI-based cards to support 10Gbps Ethernet networking.Note: XAUI cards are only supported in slots 0 and 1.Internal storage in the Sun SPARC Enterprise T5120 server is handled by four small form factor (SFF), 2.5, internal SAS hard disk drives. Both the Sun SPARC Enterprise T5120 and T5220 Servers currently support 73 or 146 gigabyte disks at 10,000 revolutions per minute, or RPM. The disks are attached to a SAS and SATA disk backplane and managed through an LSI 1068E SAS/SATA disk controller.The Sun SPARC Enterprise T5220 server supports eight small form factor (SFF) SAS disk drives.System components that affect the Sun SPARC Enterprise T5120 servers environment include two 650 watt, hot-swappable power supply units (PSUs) and four hot-swappable system fan trays, 2 fans/ tray, providing N+1 redundancy.System components that affect the Sun SPARC Enterprise T5220 servers environment include two 750 watt, hot-swappable power supply units (PSUs) and three hot-swappable system fan trays, 2 fans/tray, providing N+1 redundancy.For monitoring environmental conditions, both servers have an Integrated Lights Out Management, or ILOM, controller with an RJ-45 serial management port and a 10/ 100BaseT Ethernet Management portSystem status LEDs provide system and component status information to administrators.The Sun SPARC Enterprise T5120/T5220 Servers incorporate several reliability, availability, and serviceability, or RAS features, which have become standard in Suns offering. These features include extended error correction code, or ECC, and CRC checking on busses, cache, memory channels, and address and data paths, power supply and fan module redundancy, the inclusion of a remote access tool, environmental monitoring solutions, and remote monitoring solutions.The Sun SPARC Enterprise T5120/T5220 Servers are designed to fulfill the needs of customers who require real-world application performance while optimizing the computing power that can be achieved within the expensive and shrinking data center space. As servers become increasingly more power hungry, maintaining an environment to balance power input and thermal output for each rack becomes increasingly important.The Sun SPARC Enterprise T5120/T5220 Servers utilize Cool Computing to meet the needs of these customers. With the Sun SPARC Enterprise T5120/T5220 Servers, changes in the architecture have improved the power input to thermal output ratio. CMT- based processors are designed for the thread-rich network computing environment. These processors maximize application throughput by processing multiple threads simultaneously on a single chip. Essentially, any application that is vertically and/or horizontally scalable today can benefit from CMT. These include real-world applications such as:DatabasesApplications serversTransaction processingAnd web-based servicesPrimary market segments that require these services include:Financial servicesTelecommunicationsGovernmentRetail organizationsAnd professional services

With the introduction of the new processor and architecture, the Sun SPARC Enterprise T5120/T5220 Servers are the right servers for customers whose applications are highly threaded, using parallel threads, or require large instruction or data working sets. The CMT chips can handle certain types of software and tasks better than other high-end 64-bit chips. Any application that creates network traffic, such as Web services software, Java, Web servers, and application servers, can benefit from spreading lots of software threads across low-powered processor cores.Several servers compete in the same market as the Sun SPARC Enterprise T5120/T5220 Servers. Competition arises primarily from IBM, HP, Fujitsu, and Dell.The trend toward multi-core processors is nothing new: IBM currently ships servers with the dual-core Power4+ and Power5 chips; HP uses the dual-core with PA-RISC processor; Fujitsu implements the SPARC64-V chip; Dell ships servers with the Intel Xeon MP and EM64T processors; while Advanced Micro Devices is supplying an Opteron processor to the industry. Sun is well positioned to handle the boost in performance with the Solaris Operating System. Unlike most desktop operating systems, the Solaris OS can leverage CMT to run 64 simultaneous threads. For threaded applications, future UltraSPARC processors will deliver up to 50 times the performance of today's fastest UltraSPARC processor, without a significantly higher cost-per-chip.And just as important as speed, CMT enables administrators to consolidate the infrastructure of dozens of servers onto a single server. Imagine the savings in administration, maintenance, power, cooling, and floor space. The management and maintenance savings are as phenomenal as the technology itself.Several key technologies are being used in the Sun SPARC Enterprise T5120/T5220 Servers. These technologies help to increase bandwidth, connectivity, system throughput, and computing throughput through enhancements to the architecture, processors, and system busses.The Sun SPARC Enterprise T5120/T5220 Servers offer the following major features:Dense packaging in a reduced footprintImplementation of a four, six, or eight core Sun UltraSPARC T2 processorOn-chip multithreading technology to maximize the use of the system processorHigh performance busses and memory interconnect systemsImproved input and output (I/O) throughput through the incorporation of PCIeArchitectural enhancements to system platforms to improve system upgrades, memory management, and the I/O infrastructureIncreased on-board network connectivity

========================This module, Sun SPARC Enterprise T5120/ T5220 Servers Architectural Overview, presents information on the hardware architecture for the Sun SPARC Enterprise T5120/T5220 servers.The references shown here provide additional information about the Sun SPARC Enterprise T5120/T5220 Servers.Most of the documentation shown is available on the www.sun.com/ documentation web site. A link is provided on your screen.Well pause for just a bit so that you can review the information on this slide.he Sun SPARC Enterprise T5120/T5220 servers have the following circuit boards installed in the chassis:MotherboardPower distribution board (2 in the T5220)Paddle boardUSB boardDisk backplaneFan boards (2)PCIe riser cards (3)

The motherboard is actually an assembly made up of the motherboard itself and a tray or carrier. The motherboard assembly comes in several different versions, with the only difference being processor speed and the number of cores.The motherboard includes a direct-attach CPU module, slots for 16 DIMMs, memory control subsystems, and all system controller (ILOM) logic.In addition, a removable NVRAM contains all Mac addresses, host ID, and OpenBoot PROM configuration data. When replacing the motherboard, the NVRAM can be transferred to a new board to retain system configuration data.The service processor (ILOM) subsystem contains a PowerPC Extended Core, and a communications processor that controls the host power and monitors host system events (power and environmental). The ILOM controller draws power from the hosts 3.3V standby supply rail, which is available whenever the system is receiving AC input power, even when the system is turned off.

The power distribution board distributes main 12v power from the power supplies to the rest of the system. It is directly connected to the paddle card, and to the motherboard via a bus bar and ribbon cable.The paddle board is an assembly made up of the board, a metal mounting bracket, and a top cover interlock or kill switch.The paddle board serves as the interconnect between the fan connector boards, and SAS backplane.The paddle board is an assembly made up of the board, a metal mounting bracket, and a top cover interlock or kill switch.The paddle board serves as the interconnect between the fan connector boards, and SAS backplane.The USB board connects directly to the SAS backplane. It is packaged with the DVD drive as a single customer-replaceable unit (CRU).The fan boards carry power to the system fan modules.In addition, they contain fan module status LEDs, and transfer I2C data for the fan modules.Note: The fan boards in the Sun SPARC Enterprise T5120 and T5220 are different boards, but function the same.There are three PCI riser cards per system, each attached in slots to the rear of the motherboard. In 1U systems, each riser card supports one card.In 2U systems, each riser supports two cards.PCI riser cards come in 2 different versions.One that can support both the x8 PCIe card or an XAUI card, andOne that supports x16 PCIe.Slots on the motherboard are keyed so that you can only plug in the correct type of riser card into the motherboard.Note: The slots that you see on the motherboard are not industry standard PCIe slots. They are Sun proprietary slots that only accommodate the Sun riser cards.Most of the electrical connectivity in the Sun SPARC Enterprise T5120/T5220 servers is accomplished through connectors on the systems infrastructure boards. The only system cables in the chassis are:PDB to MB ribbon cableHorizontal to vertical PDB ribbon cable (2U only)Disk backplane to MB cable (1 in the 1U, 2 in the 2U)Top cover, interlock switch cableThe diagram shown illustrates the overall system architecture for the Sun SPARC Enterprise T5120/T5220 servers.The top, center portion of the diagram depicts the UltraSPARC T2 CPU and the memory architecture. The processor uses four memory channels, each of which manages one of four banks, each with four FBDIMM slots.There are three DC-DC converters required to deliver power to the processor and FBDIMM connectors.To the right of the UltraSPARC T2 processor, you will find the service processor architecture. The service processor connects to the CPU through the serial system interface (SSI) communications bus.To the left of the UltraSPARC T2 processor, is the LSI 1068E SAS/SATA disk controller.Finally, the lower half of the architectural diagram shows the I/O architecture. The UltraSPARC T2 CPU interfaces with the I/O through three PCIe switches. We will provide an in depth explanation of all of these sections.

The illustration depicts all of the on-die components and data paths among the different components for the UltraSPARC T2 processor. Here, you have eight cores communicating through the Cache Crossbar (CCX) to the 4 MB total of 16-way associative L2 cache and then through the MCU controller channel, labeled MCU 0 through 3.Youll notice in the diagram that each CPU core has its own floating point unit (FPU). Each core also has its own crypto unit. Also there are two 10GB (XAUI) interfaces and two PCI-Express interfaces are integrated onto the chip.The UltraSPARC T2 processor is designed to operate with the first generation industry standard Fully Buffered Dual In-line Memory Modules (FBDIMMs).This feature dictates the memory architecture of the Sun SPARC Enterprise T5120/T5220 servers.The UltraSPARC T2 processor has four memory controllers, referred to as memory branches. Each branch has two channels. Each channel supports 10 Southbound (from MCU to memory) and 14 Northbound (from memory to MCU) high-speed serial lanes utilizing differential pair signaling.The UltraSPARC T2 processor allows up to a maximum of 32 FBDIMMs to be accessed, but is limited to 16 FBDIMMs in the Sun SPARC Enterprise T5120/T5220 servers.An FBDIMM module is comprised of standard DDR2 memory chips with an additional high speed serial link device, called an Advanced Memory Buffer (AMB) chip, which buffers all memory chip address, data and control signals from the outside world. The AMB serializes this information into high-speed, differentially driven data links that form point to point connections from the processor to the first FBDIMM, or from one FBDIMM to another FBDIMM in a daisy-chain fashion. Since the Sun SPARC Enterprise T5120/ T5220 Servers utilize FBDIMM memory modules that contain the AMB chip within the module, only the processor-to-DIMM and DIMM-to-DIMM link signals must be routed on the motherboard.The Sun SPARC Enterprise T5120/T5220 servers support three DIMM sizes at revenue release, including:1 GB, 2 GB and 4 GB.Because all four branches need to be populated, the following three memory configurations are supported:4 FBDIMMs, 8 FBDIMMs and 16 FBDIMMs.In total, the system supports a maximum total system memory of 128GB. The DIMMs supported are FBDIMMs which must all be the same density within a memory branch..Individual, faulty FBDIMMs can be replaced, meaning FBDIMMS do not have to be replaced in pairs.Well start our discussion of the I/O architecture with the two XAUI ports on the UltraSPARC T2 processor. These control the two XAUI slots on the motherboard.To the left of the XAUI ports is a PCIe port on the UltraSPARC T2 processor.The first component on the PCIe port is the ST Probe. This is a Soft Touch probe which is used for diagnostics.Below the ST Probe is the first PCIe switch.Coming off the bottom of the first switch is a PCIe x8 slot. Topologically, this is the closest to the processor. It will have the least latency. All the other slots, which includes two x4 slots in the Sun SPARC Enterprise T5120 and an additional x8 slot and two additional x4 slots in the case of the Sun SPARC Enterprise T5220, are connected to another PCIe switch, which is cascaded off the first PCIe switch.The third PCIe switch, shown on the left side of the diagram, controls the remaining on-board I/O which includes:Four Gigabit Ethernet interfacesFour USB portsTwo in the frontTwo in the rearDVD-ROM drive

The SAS/SATA controller provides support for embedded mirroring for the internal disks of the Sun SPARC Enterprise T5120/T5220 servers. It supports RAID levels 0 and 1, striping, and mirroring, respectively. This allows you to mirror the boot drive. Support for mirroring would not require software intervention, but instead relies on the controller.The SAS/SATA controller also provides external 32-bit support for Flash ROM and non-volatile static random access memory (NVSRAM).The controller implements the Fusion-MPT, or message passing technology architecture, which features a performance-based message passing protocol. By managing all I/O and coalescing interrupts to minimize system bus overhead, the controller requires small device drivers that are independent of the I/O bus. This represents a savings as a single device driver is used for SAS/SATA, SCSI, and fiber channel (FC).The service processor is the hardware portion of the lights-out-management system implemented on the Sun SPARC Enterprise T5120/T5220 servers. Unlike previous versions of Sun service processors, the service processor hardware is not on a separate card, but is integrated on the motherboard.The hardware components of the service processor include:A field-programmable gate array (FPGA) device that controls aspects of system power and acts as the primary ILOM to host server communications gatewayI2C devices responsible for monitoring the servers environment and FRUID dataA Motorola MPC885 microprocessor, which contains its own instruction and data caches, and a built-in memory controllerManagement portsClick the links provided to view additional information on the topics presented.The I2C bus is used on the service processor for the TOD functionality, several SEEPROMs, and to monitor the host servers environmental monitoring devices.

The OSP card supports several devices monitored on the I2C bus for itself, including:

IMAXILOMNVRAMFRU PROMTOD clockSocketed system configuration PROM containing host identification and MAC addressThe I2C bus is also responsible for monitoring the host servers monitoring and control devices, which includes temperature sensors, power supply and fan status, and LEDs.Motorola MPC885 Microprocessor

The Motorola MPC885 microprocessor offers a highly integrated, high computing power, lower power dissipation device that acts as the mini-computer for the service processor. The microprocessor contains an embedded PowerPC core that includes its own cache, system interface unit, and an interface to a communications processor. The microprocessor includes the following features:

Support for 66 MHz, 80 MHz, and 133 MHz core frequency2 KB of data cache1 KB of instruction cacheSupport for an 8-bit, 16-bit, and 32-bit external busBuilt-in memory controllerTwo serial communications controllers, one of which provides an external RS-232 UART channelTwo serial management channels used to communicate with the host serverTwo 10/100BaseT Ethernet controllers, where one provides an external network management interface and the second one communicates with the host serverAn I2C port for providing information on the configuration and statusLow-power mode357-pin plastic ball grid array (PBGA), which is a method of packaging high I/O devicesManagement Ports

The service processor supports two management ports: a serial port and a network management port. Both ports use RJ-45 connectors.

The serial port implements the full complement of RS-232-style modem controls. It can be connected to a terminal server or to a modem to gain access to the service processor and therefore console access to the Sun SPARC Enterprise T5120/T5220 servers.The service processor draws power from the 3.3V standby that is routed through the motherboard from the PSUs, regardless of whether the server is on. When power is applied to the system and the 3.3VDC receives power, the service processor boots.==============================As predicted by Gordon Moore in his article in Electronics magazine in 1965, the number of transistors on a chip has doubled approximately every two years. This has come to be known as Moore's Law.In addition, the law shows an equivalent doubling of the clock frequency within the same time frame. However, the memory speeds have not kept pace. DRAM doubled in speed every 6 years, leaving a growing speed gap between processors and memory.High speed processors can execute instructions at a fast rate. This execution instruction, called a thread, is made up of compute time and, for memory access instructions, memory latency time.The compute time can be fast, but overall processor performance is stalled whenever a memory access instruction is performed. Due to the speed gap that exists between processors and memory, the processor can spend up to 75 percent of its processing cycles waiting on memory. One way to take advantage of this memory latency is to start another thread that the processor can execute, while the first thread is stalled. This concept, known as multithreading, or MT, allows multiple instruction streams to be executed within the same period of time.An MT processor has multiple sets of registers and other thread states which allow threads to execute either simultaneously, if the processor can physically support this, or to be switched off when one thread is delayed waiting for data or instructions, such as memory access.Adding more transistors within the chip to support more threads in parallel, improves the overall processor performance, and significantly lowers the effect of the memory latency. With multiple threads, cycles that would otherwise be wasted are available as compute cycles for another thread.Another method of improving processor throughput is through chip multiprocessing, or CMP.CMP duplicates the processing unit of the processor, where multiple cores are included on a single chip. Functionally, this allows one thread to be active at a time on each core. This improves utilization of chip resources.Because physical resources are now available with a multiple cores, that core processes its thread independently from the other cores. The number of parallel threads that can be executed has increased.Chip multithreading combines the concept of multithreading and chip multiprocessing. This combination provides an increase in throughput and a gain in thread-level parallelism, or TLP, so you can now process multiple threads on a core in a processor with multiple cores. For example, a processor with 4 cores and 4 threads per core can execute 16 parallel threads. Sun's UltraSPARC T2 processor implements the CMT design. The UltraSPARC T2 processor is available with 4, 6, or 8 cores. Each of these cores supports up to eight threads. This provides support for up to 64 threads on the server.Each thread within a core contains a full set of system registers but shares the Level 1 instruction cache, which is 16 Kbytes, and the data cache, which is 8 Kbytes. By providing each core with its own L1 cache, we can dramatically decrease memory latency as compared to having L1 cache shared by all cores.L1 data and instruction cache communicates with the 4 megabyte L2 cache through a cache crossbar interface that enables L2 cache sharing among the available cores.To realize the large opportunity for throughput that the processor provides, the operating system must be able to effectively manage the individual cores and threads of the processor.Applications and the operating system should be able to fully utilize all cores so that the CPU utilization rate is high. Threads are switched in and out quickly, depending on their need. If a thread requires memory access as a result of a memory cache miss, another thread can continue to execute its instructions until it has been moved off to make room for another thread. Keep in mind that each thread has its own set of registers, so its state is maintained.The operating system, specifically the scheduler, is vital in scheduling threads for the CMT environment. While for all appearances, the operating system appears to treat these cores as individual processors, support is added to various OS subsystems to ensure that they are CMT-aware.The Solaris OS treats these logical processors like traditional SMP processors or symmetrical multiprocessing processors, scheduling runnable threads across them. However, there is an awareness of grouping, in which threads are organized by the core they are associated with. This helps the kernel with scheduling, as it alerts the system to which threads are sharing resources so that it can better manage scheduling policies to improve performance, primarily through cache utilization and maximizing aggregate data path bandwidth.Unless bound to a processor by the user, the OS scheduler must decide which threads should be run on the same processor or on separate processors.The scheduler uses several methods to balance loads and improve performance, including:Load balancing in which the scheduler uses a scheduling policy to distribute the workload across the logical processors to help maximize per-chip resource availability. It looks at the current load on a core to determine the best fit for the thread requiring execution time. If the core is underloaded, it can handle another thread without straining access to resources.Thread-to-cache affinity looks at a threads timestamp to see the last time it ran on its prior CPU. If it is less than a predetermined time, the thread is reassigned to the CPU. This method also takes into account the size of the cache to ensure the thread has the amount of resources required to complete its tasks efficiently.Shared run queues provide an advantage with shared cache. Once a thread runs on a strand within a specific core, there may be no disadvantage to it running on any other strands in the same core. Hence, the logical processors on a core share a dispatch queue.Updating CPU performance counters, or CPCs, to make them more flexible and available to the kernel, as well as to help make better scheduling decisions based on workload characterizationsField Programmable Gate Array (FPGA) Device

The FPGA device controls system power and acts as the primary gateway for ILOM-to-host server communications. It provides the interface to the UltraSPARC T2 CPU so that it can gain access to the boot Flash ROM, SRAM, which acts as a mailbox and POST scratch pad. The FPGA also provides the following functions for the server:

Reset controlClock controlPower controlInterrupts for the ILOM and UltraSPARC T2 processorA mailbox communications system is used by both the UltraSPARC T2 processor and the ILOM to gain indirect access to these functions. The mailbox pointers are held in the FPGA registers while data is maintained in the SRAM. Both the processor and the ILOM use the mailbox, and therefore the SRAM, to pass data between each other.

The Virtual Blade System Controller (vBSC) controls most FPGA functionality and provides interfaces for ILOM to call these functions when required, such as booting the system and configuring specific options.====================================================he following Sun SPARC Enterprise T5120/ T5220 components can be serviced by the customer:Fan modulesPower supply units of which there are 2 units accessible from the rear of the server.PCI cards and riser cardsDDR2 FBDIMMs represents the memory available in 1G, 2GB and 4G FBDIMMs.System batterySAS disk drivesRail mount kit which is used to rack mount the Sun SPARC Enterprise T5120/T5220 serverCable management arm which is used to streamline the cables in the rear of the server.The following Sun SPARC Enterprise T5120/ T5220 server components should be serviced by an authorized, trained engineer:DVD ROM which is replaceable from the front of the unit.Disk backplane boardFan power board which distributes power to the system fans.Paddle boardPower distribution board which takes the power from the power supplies and distributes it to the rest of the boards in the serverBus barsSystem configuration PROMCable Kit which contains two SAS cablesMotherboard (MB) assemblyIt should be noted that in the event of a CPU failure, the entire motherboard must be replaced.When servicing the internal components to the Sun SPARC Enterprise T5120/T5220 server, be sure to:Turn off all peripheral devices connected to the server.Turn off the server itself, except in the case of hot-swap components.Label and disconnect all of the cables coming into the server.And finally, ensure that you follow ESD precautions.Before working on the Sun SPARC Enterprise T5120/T5220 server be sure to have:A Flat blade No.1 screwdriver and a No. 2 Phillips head screwdriverAn Electrostatic discharge (ESD) matA grounding wrist or foot strapHot-swappable components are those that you can install or remove while the system is running, without affecting the systems performance. However, you might have to prepare the operating system before the hot-swap operation is performed. The following components are hot-swappable in a Sun SPARC Enterprise T5120/T5220 server:The two power supply unitsThe hard disk drivesNote: The system fans are hot-pluggable. No preparation of the operating system is required before removing and replacing system fans.he top cover for both the Sun SPARC Enterprise T5120 and T5220 server includes an integrated, latched door for access to the hot plug fans.Depending on the component that you are servicing, you might need to remove the top cover.Click the top link on your screen for a printable text based procedure on the removal and replacement of the top cover.Click the bottom link on your screen for an animated demonstration of the removal of the top cover.For each of the CRUs listed, click the component name for a printable text based procedure on the removal and replacement of that component.Click the link provided to view an animated demonstration on servicing CRUs.Procedure for Removing and Installing a Fan Module

Fans are replaced in pairs (2 fans per module).

To remove a fan module:

Slide the system out of the rack.

Push the 2 fan door latches toward the rear of the server and open the fan door.

The faulty fan module is identified with a corresponding Service Required LED.

On the Sun SPARC Enterprise T5120, the Fan Fault indicators are located on the fan board.

On the Sun SPARC Enterprise T5220, the Fan Fault indicators are located on the fan modules.

Pull up on the fan module handle until the fan module is removed from the chassis.

Note: The removal procedure for the Sun SPARC Enterprise T5120 and T5220 fan modules is the same. The only difference is that the Sun SPARC Enterprise T5120 server has 4 fan modules (8 fans) and the Sun SPARC Enterprise T5220 server has 3 fan modules (6 fans).

To install a fan module:1. With the top cover door open, install the replacement fan module into the server. The fan modules are keyed to ensure they are installed in the correct orientation.2. Apply firm pressure to fully seat the fan module.3. Verify that the Fan Fault indicator on the replaced fan module is not lit.4. Close the fan door.5. Verify that the Top Fan indicator, Service Required indicators, and the Locator indicator/Locator button are not lit.6. Procedure for Removing and Installing a FBDIMM7. Complete the following steps to remove an FBDIMM:8. 1. Power off the system and slide it out of the rack.

Note: FBDIMMs should only be removed with the power cord disconnected from the chassis.

9. 2. Remove the server top cover.10. 3. Lift the hinged air baffle.11. 4. Locate the memory module socket in which you will remove an FBDIMM. Press the FBDIMM fault button.12. The FBDIMM fault button is located on the motherboard near the FBDIMMs.13. Faulty FBDIMMs are identified with a corresponding amber LED on the motherboard.14. 5. Push down on the ejector tabs on each side of the FBDIMM until the FBDIMM is released15. 6. Carefully lift the FBDIMM straight up to remove it from the socket.16. Complete the following steps to install an FBDIMM:17. 1. Power off the system and slide it out of the rack.18. 2. Remove the top cover.19. 3. Lift the hinged air baffle.20. 4. Locate the memory module socket in which you will install an FBDIMM.21. 5. Ensure that the ejectors, at each end of the memory socket, are fully open (rotated downward) to accept the new FBDIMM.22. 6. Align the FBDIMM with the key in the socket.23. 7. Press the FBDIMM straight down until it snaps into place and the ejectors engage the cutouts in the FBDIMM's left and right edges.Procedure for Removing and Installing a Power Supply UnitComplete the following steps to remove a power supply:1. If the server is in a rack with a cable management arm attached, swivel open the cable management arm to view the power supplies.2. Identify which power supply you will replace. Each power supply has an amber LED that you can view from the rear of the server. If the amber LED is on, the power supply is faulty and should be replaced.3. Disconnect the AC power cord from the power supply that you are replacing. The power supplies are hot-swappable, so you do not have to shut down the server or disconnect the other power supply.

Note: The Service Action Required LEDs on the front panel and back panel blink when a power supply is unplugged.

4. Remove the power supply:a. Grasp the power supply handle and push the thumb latch toward the center of the power supply.b. While continuing to push on the latch, use the handle to pull the power supply from the chassis.Remove the power supply:1. Align the power supply with the empty bay in the chassis.2. Press the power supply into the bay until it firmly engages the connector on the power distribution board. It is fully seated when the thumb-latch clicks into place.3. Connect the AC power cord to the new power supply.4. Swivel any cable management arm back into the closed position.Procedure for Removing and Installing the System BatteryComplete the following steps to remove the system battery:1. Power off the system and slide the system out of the rack.2. Remove the top cover.3. Remove PCIe/XAUI riser 0.4. Using a small (No. 1 flat-blade) screwdriver, press the latch and remove the battery from the motherboard.To install the battery, reverse this procedure.

Note: Install the new battery with the plus sign (+) facing up.

Procedure for Removing and Installing a PCI Card and Riser CardProcedure for Installing and Removing a PCI Card and Riser Card PCIe/XAUI cards are installed on vertical risers. You must remove the relevant riser to access a PCIe/XAUI card.Complete the following steps to install a PCI card:1. Unpackage the replacement PCIe or XAUI card and place it on an antistatic mat.2. Locate the proper PCIe/XAUI slot for the card you are replacing or3. If necessary, review the PCIe and XAUI Card Guidelines to plan your installation.4. Disconnect any data cables connected to the cards on the PCIe/XAUI riser being removed. Label the cables to ensure proper connection later.5. Remove the riser board.a. Remove the #2 Phillips screw securing the riser to the motherboard.b. Slide the riser forward and out of the system.6. Insert the PCIe/XAUI card into the correct slot on the riser board.7. Replace the riser board.a. Slide the riser back until it seats in its slot in the back panel.b. Replace the #2 Phillips screw securing the riser to the motherboard.8. Re-install any data cables connected to the cards on the PCIe/XAUI riser being installed.9. Install the top cover.To remove a PCI card, reverse this procedure.

Note: The procedures are the same for both the Sun SPARC Enterprise T5120 and the Sun SPARC Enterprise T5220.

Procedure for Removing and Installing a Hard Disk DriveComplete the following steps to remove and replace a hard disk drive:1. On the drive you plan to remove, push the hard drive release button.The latch opens.

Caution: The latch is not an ejector. Do not bend it too far to the left. Doing so can damage the latch.

2. Grasp the latch and pull the drive out of the drive slot.To install a hard disk drive:1. If necessary, remove the blank panel from the chassis. Press the FBDIMM's fault button.

Note: Sun SPARC Enterprise T5120 servers might have three blank panels covering unoccupied drive slots. Sun SPARC Enterprise T5220 servers might have as many as seven blank panels covering unoccupied hard drive slots.

2. Align the replacement drive to the drive slot.Hard drives are physically addressed according to the slot in which they are installed. If you removed an existing hard drive from a slot in the server, you must install the replacement drive in the same slot as the drive that was removed.3. Slide the drive into the drive slot until it is fully seated.4. Close the latch to lock the drive in place.Procedure for Removing and Installing a DVD AssemblyTo remove a DVD assembly:1. Remove the following hard drive: Sun SPARC Enterprise T5120: HDD3 Sun SPARC Enterprise T5220: HDD72. Release the DVD/USB module from the disk drive backplane.Use the finger detent in the disk drive bay below the DVD/USB module to detach the module from the backplane.3. Slide the DVD/USB module out of the disk drive cage.To install the DVD assembly:1. Slide the DVD/USB module into the front of the chassis until it seats.2. Install the hard drive you removed during the DVD/USB module removal procedure.Procedure for Removing and Installing the Power Distribution Board to Motherboard Ribbon CableTo remove the PDB to MB ribbon cable:1. Power off the system and slide the system out of the rack.2. Remove the server top cover.3. Lift the hinged air baffle.4. Disconnect one end of the ribbon cable from the power distribution board and the other end from the motherboard.To install the PDB to MB ribbon cable, reverse this procedure.Procedure for Removing and Installing a Fan BoardTo remove a fan board:1. Power off the system and slide the system out of the rack.2. Remove the top cover.3. Remove the fan modules.

Note: If you are replacing a defective fan module connector board, remove only the fan modules that are necessary to remove the defective fan module connector board.

4. Remove the Phillips screw that secures the fan module connector board to the chassis.5. Slide the fan board toward the left side of the chassis approximately 0.5 inch (12 mm) to disengage the fan board from the paddle board and the bottom of the chassis.6. Lift the fan board up and out of the chassis.To install a fan board, reverse this procedure.Procedure for Removing and Installing the Paddle BoardTo remove the paddle board:1. Power off the system and slide the system out of the rack.2. Remove the top cover.3. Remove the motherboard assembly.4. Remove both power supplies from the chassis.5. Remove all the disk drives from the server.6. Remove the DVD, DVD carrier and USB board from the server.7. Remove the 4 screws securing the disk cage assembly to the chassis. There are 2 screws on the side of the chassis near the right front and 2 screws on the side of the chassis near the left front.8. Disconnect the disk cable from the motherboard, so that it does not obstruct access to the power distribution board.9. Remove the PDB to MB ribbon cable.10. Disconnect the top cover intrusion switch cable connector from the power distribution board.11. Remove the 4 screws connecting the power distribution board to the bus bar.12. Remove the single screw securing the power distribution board to the bottom of the chassis.13. Slide the power distribution board toward the left approximately 0.5 inch (12 mm) to release the power distribution board from the paddle board and the captive standoffs on the bottom of the chassis.14. Lift the power distribution board out of the chassis.15. Remove the 2 screws that secure the paddle board bracket to the chassis.

Note: Do not remove the 2 screws that secure the paddle board to the paddle board bracket.

16. Slide the paddle board bracket toward the rear of the chassis approximately 0.5 inch (12 mm) to release the paddle board bracket from the chassis.17. Lift the paddle board and bracket out of the chassis.To install the paddle board, reverse this procedure.Procedure for Removing and Installing a Disk BackplaneTo remove a disk backplane:1. Power off the system and slide the system out of the rack.2. Remove the top cover.3. Remove all the disk drives from the server.4. Remove the DVD, DVD carrier and USB board from the server.5. Remove the 4 screws securing the disk cage assembly to the chassis. There are 2 screws on the side of the chassis near the right front and 2 screws on the side of the chassis near the left front.6. Slide the disk cage assembly toward the front of the chassis approximately 0.5 inch (12 mm). This releases the disk cage assembly from the chassis bottom.7. Disconnect the disk cable from the disk backplane. The Sun SPARC Enterprise T5120 server has 1 disk cable. The Sun SPARC Enterprise T5220 server has 2 disk cables.8. Remove the 2 screws that secure the disk backplane to the disk cage assembly. The Sun SPARC Enterprise T5220 server has 4 screws securing the disk backplane to the disk cage assembly.9. Slide the disk backplane down approximately 0.25 inch (6 mm) to release the disk backplane from the "fingers" on the disk cage assembly that protrude through keyhole slots in the disk backplane.10. Pull the disk backplane away from the disk cage assembly.To install a disk backplane, reverse this procedure.Procedure for Removing and Installing the System Configuration PROMTo remove the system configuration PROM:1. Power off the system and slide the system out of the rack.2. Remove the top cover.3. Remove the right most PCI and riser card.4. The system configuration PROM is the chip located at j7901 on the motherboard. Lift it straight up off the motherboard.To install the system configuration PROM, reverse the procedure.

Note: The PROM is keyed, so it can only be installed one way.

Procedure for Removing and Installing the Motherboard AssemblyThe Sun SPARC Enterprise T5120 and Sun SPARC Enterprise T5220 use the same motherboard.The motherboard assembly consists of the motherboard and the tray that the motherboard sits in. They should be removed and installed as a single unit.To remove the motherboard:1. Power off the system and slide the system out of the rack.2. Remove the top cover.3. Remove the air baffle.a. Open the air baffle.b. Disengage the rear of the air baffle from the motherboard and rotate the air baffle forward.c. Press in the edges of the air baffle to disengage its pins from the chassis.4. Disconnect the motherboard to power distribution board ribbon cable.5. Disconnect the disk cable from the motherboard. The Sun SPARC Enterprise T5220 server has 2 disk cables.6. Remove all PCI boards and riser cards.7. Remove all FBDIMMs.8. Remove the 4 screws connecting the motherboard to the bus bar.9. Loosen the captive screw securing the motherboard to the chassis.The captive screw is colored green, and is located to the left of the bus bar screws.10. Using the green handles, slide the motherboard back and tilt the motherboard assembly to lift it out of the chassis.Grab the handles and move the motherboard toward the back of the system and lift it out of the chassis.To install the motherboard assembly, reverse this procedure.Procedure for Removing and Installing the Front Panel Light Pipe AssemblyTo remove the front panel light pipe assembly:1. Remove the hard drive cage.2. Remove the hard drive backplane.3. Remove the #2 Phillips screw securing the front control panel light pipe assembly to the hard drive cage.4. Slide the light pipe assembly out of the hard drive cage.To install the front panel light pipe assembly:1. Align the light pipe assembly with the mounting holes on the hard drive cage.2. Secure the light pipe assembly with a #2 Phillips screw.3. Install the hard drive backplane.4. Install the hard drive cage.

Procedure for Removing and Installing a Power Distribution Board

Note: It is easier to service the power distribution board (PDB) with the bus bar assembly attached.

To remove a power distribution board:1. Power off the system and slide the system out of the rack.2. Remove the top cover.3. Remove the motherboard assembly.4. Remove both power supplies from the chassis.5. Remove all the disk drives from the server.6. Remove the DVD, DVD carrier and USB board from the server.7. Remove the 4 screws securing the disk cage assembly to the chassis. There are 2 screws on the side of the chassis near the right front and 2 screws on the side of the chassis near the left front.8. Disconnect the disk cable from the motherboard, so that it does not obstruct access to the power distribution board. Note: There are 2 disk cables connected to the motherboard in the Sun SPARC Enterprise T5220 server.9. Remove the PDB to MB ribbon cable.10. Disconnect the top cover intrusion switch cable connector from the power distribution board.11. Remove the 4 screws securing the power distribution board to the bus bar.

Note: The Sun SPARC Enterprise T5220 has 4 additional screws to remove. They are connected to 2 additional bus bars that connect to a vertical power distribution board.

12. The Sun SPARC Enterprise T5220 server also has a ribbon cable that connects the horizontal and vertical power distribution boards. This must be disconnected also.13. Remove the single screw securing the power distribution board to the bottom of the chassis.14. Slide the power distribution board toward the left approximately 0.5 inch (12 mm) to release the power distribution board from the paddle board and the captive standoffs on the bottom of the chassis.15. Lift the power distribution board out of the chassis.To install a power distribution board, reverse this procedure.To set up the service processor with initial network configuration information, you must establish a connection through ILOM to the service processor. Until the service processor has an IP address assigned to it, you must use a serial connection to communicate with the service processor. After establishing a serial connection to the service processor, you can choose to configure the service processor with a static or DHCP IP address.The default action for the service processor is to try and use DHCP for its network configuration information. When you apply power to the system for the first time, ILOM broadcasts a DHCPDISCOVER packet. If you have an established DHCP server on the network, the DHCP server returns a DHCPOFFER packet containing an IP address and other network configuration information to the service processor.If you prefer to configure the service processor with a static IP address, and you have a DHCP server established on your network, you can configure the static IP address prior to attaching a LAN cable to the NET MGT port of the server.Note: Sun recommends a static IP address for the service processor.Whether static or DHCP IP addresses are assigned, you must initially establish a serial console connection to communicate with ILOM.You can access the ILOM CLI at any time by connecting a terminal or a PC running terminal emulation software to the serial management port on the chassis.To connect to the ILOM using a serial connection:Step 1. Verify that your terminal, laptop, or terminal server is operational.You need a display device to interface to the serial port. This can be one of the following devices:Laptop or desktop computerTerminal devicePersonal Digital Assistant

Step 2. Before trying to connect to the port, make sure that your display device is properly configured. In the case of the laptop, desktop computer, and PDA, make sure that you have a terminal simulation program loaded and started.In the case of all the devices, make sure that they are configured with the following default parameters:8N1: eight data bits, no parity, one stop bit9600 baudDisable hardware flow control

Step 3. Connect a serial cable from the serial management port on the rear of the chassis to a terminal device. The serial management port is the left-most RJ-45 port, as viewed from the rear of the chassis.The pinout requirements for the serial cable connected to the serial port are displayed in the table on your screen.

Step 4. Press Enter on the terminal device.This establishes the connection between the terminal device and the ILOM.Note: If you connect a terminal or emulator to the serial port before it has been powered up or during its power-up sequence, you will see bootup messages.When the system has booted, the ILOM displays its login prompt: SUNSP 00:12:2F:4A:7A:3B login:The first string in the prompt is the default host name. It consists of the prefix SUNSP and the ILOMs MAC address. The MAC address for each ILOM is unique.

Step 5. Log in to the CLI:Type the default user name, root.Type the default password, changeme.Once you have successfully logged in, the ILOM displays the ILOM default command prompt:->You can now run CLI commands.Note: CLI is the only available user interface on the serial port.

The CLI architecture is based on a hierarchical namespace, which is a predefined tree that contains every managed object in the system. This namespace defines the targets for each command verb.The ILOM includes three namespaces: The /SP namespace, the /SYS namespace, and the /HOST namespace.The /SP namespace manages the ILOM. For example, you use this space to manage users, clock settings, and other ILOM issues.The /SYS namespace manages the host system. For example, you can change the host state, read sensor information, and access other information for managed system hardware.The /HOST namespace - Used for monitoring and managing the host operating system.A syntax diagram for the ILOM CLI is shown on your screen. An ILOM CLI command is made up of the following components:verb - The CLI supports a pre-defined list of verbs.options - All options are not supported for all commands. See a specific command section for the options that are valid with that command.target - Every object in your namespace is a target. All targets are not supported for all commands.properties - Properties are the configurable attributes specific to each object. An object can have one or more properties.Note: For information on using the CLI, see the Integrated Lights Out Manager (ILOM) Administration Guide.Use the set command to change properties and values for network settings.Network settings have two sets of properties:pending - which are the updated settings, not currently in useactive - read-only settings, currently in use by the ILOMTo change settings:First, enter the updated settings as the pending settings.Then, set the commitpending property to true.To display network settings, type the command:show /SP/networkIf you are already in the /SP/network directory, type:show

Follow these steps to assign a static IP address to the network management port:Step 1. At the ILOM prompt, type the following command to set the working directory.-> cd /SP/networkStep 2. Type the following commands to specify a static Ethernet configuration.-> set pendingipaddress=129.144.82.26-> set pendingipnetmask=255.255.255.0-> set pendingipgateway=129.144.82.254-> set pendingipdiscovery=static-> set commitpending=trueNote: The network values shown are samples only. You must specify the IP address, netmask, and gateway appropriate for your network configuration.

Ensure that the same IP address is always assigned to an ILOM by either assigning a static IP address to your ILOM after initial setup, or configuring your DHCP server to always assign the same IP address to an ILOM. This enables the ILOM to be easily located on the network.When using the network management port, the wire speed is set to 10/100 megabit, full duplex. Connection through this Ethernet port is allowed only after you have configured the service processor using the serial port to have a valid IP address on your network.The service processor accepts ssh only through its Ethernet port.The following interfaces can be used as ILOM management interfaces:CLI using ssh which meets the DMTF SMASH industry standardSNMP v1, v2c, v3Web browserIPMI 2.0In general, feature compatibility exists between all user interfaces.In addition, the CLI is backward compatible to ALOM which is the Advanced Lights Out Management firmware application used on previous versions of Suns service processors.The first time the service processor is initialized, the following default conditions exist:The network is enabledDHCP is enabledssh service is enabledThe shell for the root user is the DMTF CLI

As previously mentioned, there is an ALOM backward compatibility shell to accommodate users that are more comfortable with an ALOM interface.Some differences in the ALOM compatibility shell in Sun SPARC Enterprise T5120/T5220 service processor include:ssh access only - no telnettftp only - no ftpA tftp server must be configured to support commands like:- flashupdate- FlashupdateFPGA- frucapture As these commands use tftp as an underlying protocolUser names and passwordsPasswords have to be 8 to 16 characters (versus 8 characters)Maximum of 10 users allowed (compared to 16)Two user roles (compared to 4 in ALOM). They are:CUAR (Administrator)No bits set (Operator) Two CLI modes:default (ILOM), andalomA new parameter - netsc_commitUsing this, network settings take immediate effect, and there isNo need to reboot the SP And, finally,setdefaults -a The -a stands for all. This resets all configuration parameters to defaults, including all usernames and passwords, which means that the only user that is left is the root user.To create a new user, with the ALOM shell as their interface, follow the procedure on your screen. This procedure creates the user admin to emulate the ALOM administrator.Step 1. Log in as the root user.Step 2. Create the admin user.-> create /SP/users/adminStep 3. Set the role and cli mode.-> set /SP/users/admin role=Administrator-> set /SP/users/admin cli_mode=alomStep 4. Log out and log back in as the admin user. In subsequent logins as admin, youll get the ALOM shell.Over the course of a products life cycle, new versions of firmware are released. To verify which version of firmware your system is running, issue the version command. In the command output, you are looking for the ILOM version, the SC firmware version, and the OBP version.If you are considering updating the ILOM firmware, be aware of the following:It is likely that the firmware images available to download from the SunSolve database are more current than the image installed on your service processor at the factory.The BIOS and the SP firmware are simultaneously updated. A single firmware image contains both the BIOS and the SP firmware.A firmware upgrade will cause the server and ILOM to be reset. It is recommended that you perform a clean shutdown of the server prior to the upgrade procedure. An upgrade takes about five minutes to complete.ILOM will enter a special mode to load new firmware. No other tasks can be performed in ILOM until the firmware upgrade is complete and ILOM is reset.Note: Ensure that you have reliable power before upgrading your firmware. If power to the system fails (for example, if the wall socket power fails or the system is unplugged) during the firmware update procedure, the ILOM could be left in an unbootable state.To update the ILOM firmware using the CLI:Step 1. Log in to the service processor as root.Step 2. Update the ILOM firmware (and consequently the BIOS) as follows:-> load -source tftp:// servername:port/path/to/imageNote This functionality also exists in the ILOM Web GUI.From the service processor, you can power on the server by typing:start /SYSYou can use the stop command to perform an orderly shutdown of the server followed by a power off of the server, as shown on your screen.You can also skip the orderly shutdown and force an immediate power off with the -force option, as shown on your screen.Output to POST, OBP, and the Solaris OS are displayed to the system console, which is accessible through the service processor. To initiate a connection to the server console, execute the start /SP/console command from the service processor prompt. To terminate a connection to the server console, execute the stop /SP/console command from the service processor prompt.

Working in a data center with thousands of servers in racks can sometimes pose a problem when you are trying to locate one.The locate LED is a white LED that you can light to help you find your server in a crowded equipment room. The Locate LED has two states, fast blink and Off.To turn on the locate LED, type:-> /SYS/LOCATE value=Fast_BlinkTo turn off the locate LED, type:-> /SYS/LOCATE value=Off

The system event log accumulates various events, including administration changes to the ILOM, software events, warnings, alerts, and events from the IPMI log.You should note that the ILOM tags all events or actions with LocalTime=GMT (or UDT). Browser clients show these events in LocalTime. This can cause apparent discrepancies in the event log. When an event occurs on the ILOM, the event log shows it in UDT, but a client shows it in local time.

To view and clear the system event logs, perform the following steps:Step 1. Navigate to /SP/logs/event.-> cd /SP/logs/eventStep 2. From the CLI, enter the show list command: -> show listThe event log scrolls onto your screen.Step 3. To scroll down, press any key except q.Step 4. To stop displaying the log, press q.Step 5. To clear the system event log, use the command:set clear=trueStep 6. The CLI asks you to confirm.Type y.The CLI clears the system event log.Note: The system event log accumulates many types of events, including copies of entries that IPMI posts to the IPMI log. Clearing the system event log clears all entries, including the copies of the IPMI log entries. However, clearing the system event log does NOT clear the actual IPMI log.You must use IPMI commands to view and clear the IPMI log.The service processor also allows you to create additional users. The tasks that can be performed by a user are determined by the privileges that you assign to that users account. You can have up to a maximum of ten user accounts, including root.Each user account consists of a user name, a password, and a role.As the root user, you can add, delete, and list users on the service processor.To add a user, execute the create command, providing the following information:usernamepassword, androle - either administrator or operatorYou can remove users from the service processor using the delete command, as shown on your screen.To display information about all local user accounts, type show /SP/users.

As root user, you can use the set command to change passwords and roles for existing user accounts.For example, when changing the role for user1 from Administrator to Operator, type:-> set /SP/users/user1 role=operatorTo change user1's password, type:-> set /SP/users/user1 new_passwordTo log into the Sun ILOM Web GUI, follow these procedures:Step 1. Using secure http, type the IP address of the ILOM service processor into your web browser, as the example on your screen shows.The JavaTM Web Console login screen is displayed. Step 2. Type your user name and password.Step 3. Click OK.The ILOM Web GUI login page is displayed.Click the link provided on your screen to view a demonstration using the Web GUI.

The Sun SPARC Enterprise T5120/T5220 server has a virtual keyswitch with four different modes. The first mode is normal in which the service processor uses the diagnostic settings that you specified with the set command to determine how POST is executed.The second mode is diag in which the vBSC sets the diagnostic level to the interactive menus.The third mode is stby. In this mode, the service processor prevents you from powering up the server.And in the fourth mode, locked, the service processor does not allow you to send a reset to the server.To set the virtual keyswitch, execute the set command with /SYS/keyswitch_state as the target and provide a mode as a value for the target. To view the current keyswitch setting, execute the show /SYS/ keyswitch_state command.

The Sun SPARC Enterprise T5120/T5220 server has a virtual keyswitch with four different modes. The first mode is normal in which the service processor uses the diagnostic settings that you specified with the set command to determine how POST is executed.The second mode is diag in which the vBSC sets the diagnostic level to the interactive menus.The third mode is stby. In this mode, the service processor prevents you from powering up the server.And in the fourth mode, locked, the service processor does not allow you to send a reset to the server.To set the virtual keyswitch, execute the set command with /SYS/keyswitch_state as the target and provide a mode as a value for the target. To view the current keyswitch setting, execute the show /SYS/ keyswitch_state command.You can power on the system by executing the start /SYS command at the service processor prompt, or by pressing the power button located in the bottom left corner on the front of the machine.

Output from POST, OBP, and the Solaris OS are displayed to the system console, which is accessible through the service processor. To acquire the console, execute the start /SP/ console command from the service processor.When you issue the command the system will prompt you Are you sure you want to start /SP/console (y/n)?. You can suppress this prompt by using the option -script.Although multiple users can connect to the system console from ILOM, only one user at a time has write access to the console. This is referred to as a write-locked session. Any characters that other users type are ignored. These are referred to as read-only mode sessions, where users can only view the console. If no other users have access to the system console, then the user entering the console session first obtains the write lock automatically. To see if there are other users connected into the service processor and to see whether they are connected to the console, execute the show /SP/sessions command.Note: Terminate a console session by typing #. (pound, period).

If the Solaris OS is running, you can use the stop /SYS command from the ILOM shell to issue a graceful shutdown to Solaris. It is similar to one of the Solaris OS commands, such as shutdown, init, or uadmin.It can take up to 65 seconds for the poweroff command to completely shut down the system. This is because ILOM attempts to wait for a graceful shutdown to complete before the server is powered off.You can also force the server to power down by executing the -> stop -force /SYS command or by pressing and holding the power button on the front of the server. This does an immediate shutdown regardless of the state of the host.From the service processor console, you can issue the command: -> set /HOST/send_break_action true to bring down the server to the Open Boot PROM prompt, or ok prompt. It is the equivalent of executing an L1-A or Stop-A on a system with a keyboard attached.For the system to accept a break, the virtual keyswitch must not be in the locked position. If it is in the locked position, ILOM returns an error message.

To reset the service processor or the server, execute the reset command from the service processor.To reset the service processor, type reset /SPIf you reset /SYS, the server reboots using the Open Boot PROM settings that you have configured.The reset command does not perform a graceful shutdown of the operating system.You will be prompted to confirm a reset operation. The -script option instructs ILOM to proceed without prompting the confirmation question.

Whenever a device is physically added or removed from a Sun SPARC Enterprise T5120/T5220 server, structures that describe this device to the hardware and the operating system must be created or removed. The process of creating and removing these structures is the foundation of device configuration.In the Sun SPARC Enterprise T5120/T5220 servers, device configuration is initiated through the Hypervisor layer. The Hyrpervisor then passes this structure to the Open Boot PROM and then ultimately up to the Solaris OS. The Solaris OS maintains its knowledge of available devices using a set of hierarchically organized device files. These files are located in the /devices directory of the root file system. In addition, the Solaris OS uses the path_to_inst file in the /etc directory to manage device instance names. It uses links in the /dev directory to enable logical device addressing.

The Sun SPARC Enterprise T5120/T5220 servers run OBP 4.x. The capabilities of the Open Boot PROM in these servers have decreased as a lot of its functionality has been moved to the Hypervisor layer. Its key functions are to allow you to boot the operating system, modify system startup parameters, load and execute programs, and get help in troubleshooting.The Solaris OS comes pre-installed on the Sun SPARC Enterprise T5120/T5220 servers on the disk in slot 0. The operating system is not configured, that is, the sys-unconfig command was run after the OS was installed. When you boot the system for the first time from the disk, you are prompted to configure it for your environment. At the ok prompt, boot from the disk that contains the Solaris operating system. You might want to configure an alias for this as your boot disk.

There are some instances when you might want to reinstall or upgrade the Solaris OS in your server. The Sun SPARC Enterprise T5120/T5220 servers support a minimum OS release of the Solaris 10 Update 4 OS. It has an architecture type of sun4v, to coincide with the ILOM firmware running on the service processor. The v represents the virtualization of the hardware to the Open Boot PROM and the Solaris OS, which is performed by the Hypervisor layer. You can install or upgrade the operating system through traditional jumpstart procedures or using the DVD-ROM.Next, were going to take a look at some standard Solaris OS commands to see what is reflected differently in their output. The first is the date command. The system TOD (time of day) is managed by the service processor and is provided to the Solaris OS by the Hypervisor layer. You must set the date on the service processor using the set /SP/clock/datetime command. If you set the date using the date command within the Solaris OS, it holds until your next re-POST, when the service processor will then pass its date over to the Solaris OS once again.The second command is prtdiag. The prtdiag command displays system configuration and diagnostic information. Youll notice in the prtdiag output that the Solaris OS sees the UltraSPARC T2 processor as 64 CPUs. It is an 8-core processor with 8 threads running per core. Youll also see in the prtdiag output that you can obtain memory specifics from the service processor. You only see the total memory size in the prtdiag output.The next command is psrinfo. The psrinfo command reflects the same changes from a CPU perspective as the prtdiag command. You will see that the Solaris OS is presented as 64 processors.And finally, well take a look at the output to ifconfig -a. This output represents the four gigabit Ethernet interfaces as ipge interfaces, ipge0 through ipge3.Multipathing software lets you define and control redundant physical paths to I/O devices, such as storage arrays and network interfaces. If the active path to a device becomes unavailable, the software can automatically switch to an alternative path and maintain its availability. This is known as automatic failover. To take advantage of multipathing capabilities, you must first configure the server with redundant hardware for example, multiple network interfaces going to the same subnet or two controllers attached to the same storage array and then configure the software to make use of it.For the Sun SPARC Enterprise T5120/T5220 servers, there are three different types of multipathing software that are available. The first is the Solaris OS IPMP (IP Multipathing) which provides multipathing and load-balancing capabilities for IP network interfaces.Next, we have Sun StorEdge Traffic Manager Software (STMS), which enables I/O devices to be accessed through multiple host controller interfaces from a single instance of the I/O device. This software is fully integrated into the Solaris operating system.And finally, we have Veritas Volume Manager (VxVM), which includes Dynamic Multipathing (DMP). This software provides disk multipathing as well as disk load balancing to optimize I/O throughput.

Within the Sun SPARC Enterprise T5120/ T5220 servers, the SAS controller supports hardware mirroring and striping using the Solaris OS raidctl utility.A hardware RAID volume created using the raidctl utility behaves differently than one created using software RAID. When volumes are created using hardware RAID, only one device appears in the device tree. Member disk devices are invisible to the operating system and are accessed only by the SAS controller.Executing the raidctl command tells you whether there are any RAID volumes found. To create a RAID volume, execute the raidctl -c primary_drive secondary_drive command. The secondary drive will disappear from the device tree.To set up a striped volume, execute the raidctl -c -r 0 primary_drive secondary_drive tertiary_drive, and so on command. In this case, all but the primary drive disappears from the Solaris OS device tree.To delete the hardware RAID volume, execute the raidctl -d mirrored_volume command.

To perform a disk replacement operation through dynamic reconfiguration, verify which hard drive corresponds to the physical device you want to remove.Ensure that no applications or processes are accessing the hard drive. To view the state of the SCSI devices, execute the cfgadm -al command. To remove a hard drive from the device tree, execute the cfgadm -c unconfigure Ap_Id command, where the Ap_Id is the attachment point identifier represented in the left-most column of the cfgadm -al output. When this operation completes, the blue OK-to_remove LED lights on the drive. It is now safe to remove and replace the disk.After installing the new drive, execute the cfgadm -c configure Ap_Id command to configure the drive back into the Solaris OS.The green activity LED flashes as the new disk is added to the device tree.LEDs are placed throughout the Sun SPARC Enterprise T5120/T5220 servers to help pinpoint problem components in the server, as well as to give a visual indicator of the overall server status.LEDs are found on the following chassis locations and server components:The front and rear panelsThe system fan modulesThe power suppliesThe disk drivesThe DVD driveService indicators are categorized as either system status LEDs or component status LEDs.On the Sun SPARC Enterprise T5120/T5220 servers front panel, system status LEDs are on the left, and component status LEDs are on the right.Looking at the system status LEDs on the front panel of the server, starting on the top, and moving down, the system status LEDs are the:White-colored Locator indicator,The Amber-colored Service Required indicator, and theGreen-colored running or Power OK IndicatorClick the indicator name on your screen for additional information about the function of that indicator.The system status LEDs on the front panel of the server are replicated on the rear of the chassis, as shown on your screen. Starting on the left, and moving to the right, they are:a White-colored Locator indicatorAmber-colored Service Required indicator, andGreen-colored running or Power OK IndicatorEach power supply has three LEDs. From top to bottom, they are:a Green-colored PSU OK indicator,Amber-colored PSU fault indicator, andGreen-colored A/C PSU power indicator

The gigabit Ethernet ports each have two LEDs to show their current status. They are the:Green-colored Link/activity indicator, and theAmber-colored Speed indicatorClick the indicator name on your screen for additional information about the function of that indicator.The Sun SPARC Enterprise T5120/T5220 status indicators conform with the American National Standards Institute (ANSI) Status Indicator Standard (SIS).The table on your screen describes the SIS standard LED behaviors and their meanings.You can also obtain the LED status information through the service processor.To view LED status information in the ALOM CLI, use the showenvironment command. In the ILOM cli, use the show /SYS// command.Click as indicated on your screen to view a demonstration of these commands.Each DIMM slot on the motherboard has an associated fault LED that identifies a faulty FBDIMM diagnosed by POST or FMA.When power is removed, the FBDIMM fault LEDs are lit by pressing a fault reminder button located on the motherboard.To identify faulty FBDIMMs, follow this procedure:Step 1. Unplug all power cords.Step 2. Press the FBDIMM fault button.The FBDIMM fault button is located on the motherboard near the FBDIMMs as shown.Step 3. Note the location of faulty FBDIMMs.Faulty FBDIMMs are identified with a corresponding amber LED on the motherboard.Step 4. Ensure that all FBDIMMs are seated correctly in their slots. If re-seating the FBDIMM does not fix the problem, remove and replace the faulty FBDIMM.Note: The DIMM fault LEDs can be lit for only a minute or so with the fault reminder button.

Firmware diagnostic tests are executed on both the Sun SPARC Enterprise T5120/ T5220 server host and on its service processor. The purpose of these tests is to verify the core functionality of the service processor and the host.The output displayed during the resetting of the service processor is an excellent source of diagnostic information.Click as indicated on your screen to view a demonstration of the power reset sequence of the service processor.

When the service processor has finished booting and the ILOM firmware has loaded, you can log in to the service processor and power on the host server. You can do this using the poweron command in the ALOM cli, executing start /SYS in the ILOM cli, or by pressing the power button on the front panel of the chassis.When power is applied to the host server, the vBSC, which is responsible for calling POST and collecting POST status on completion, is initialized.Hosted on the service processor, vBSC can be thought of as an extension of Hypervisor. The functionality that OBP once provided has been moved to the service processor. This eliminates the need to tie OBP-specific settings to the entire server.When the server powers on, the following actions take place:vBSC is initialized from the service processor.POST is called from vBSC to perform a sanity check of the server as well as testing of the components based on the settings passed to the host server by vBSC.The ASR database is updated based on diagnostic tests performed.And Hypervisor is started for the host server.While POST is called automatically, the output is not displayed by default. Likewise, higher levels of POST testing are not called by default. The server must be configured for these actions to happen. The ILOM variables that affect POST are diag_trigger, diag_verbosity, diag_level and diag_mode. You can modify them with the set command in the ILOM or the setsc command in ALOM and view them with their current settings by executing the show commnad in ILOM and showsc command in ALOM.The service processors setkeyswitch command also affects the behavior of POST execution. If the keyswitch is set to the DIAG position, then vBSC sets the diagnostic levels to the interactive menus. If the keyswitch is set to the NORMAL position, then the service processor variables are used to determine if and how POST is executed.Click as indicated on your screen to view a table of these ILOM variables and their possible settings.OBP tests are executed once POST has completed and the Hypervisor has been loaded. OBP tests are affected by the diag-switch? OBP variable.The value of diag-switch? affects the verbosity of OBP. If diag_switch? is set to true, the output from the OBP initialization and probing is sent to the console. If diag-switch? is set to false, no initialization or probing output is displayed.In addition to normal POST and OBP output, there are several commands and interfaces that you can use to analyze and test the system hardware to assist in troubleshooting.In the remainder of this module, we will look at analyzing POST output, running POST menus, analyzing OBP diagnostic output, analyzing POST error messages, managing the ASR database, and viewing logs, console messages, and faults.There are two levels of POST tests that are executed on the host server. These are the integrity POST and, if configured, EPOST.Integrity POST executes the following tests on the server each time the system is powered on:Register tests on window registers and the Niagara CPU scratchpadFloating point unit access to check the path from all threadsL2 access to check the path from all threadsQuick FPGA check by the master thread, the first thread to jump to POST, to check the integrity of the SRAMAll threads returned from testingEPOST executes tests for the remainder of the server based on the diag_level setting.When executed with a diag_level set to min, EPOST:Initializes registers and global variables.Runs the basic memory tests in which all memory cells are touched with unique patterns and checked with hardware ECC.Tests I2C operation and clock frequency for the master thread only.Performs a basic test on the JBUS-to-PCI-E bridge for the master thread only.When executed with a diag_level set to max, EPOST:Performs all the steps of the minimum diagnostic mode.Runs memory tests on any arrays not covered by BIST, such as L1 tags and the internal register arrays.Tests the functional operation of the L1 and L2 caches and instruction and data memory management unit and interrupts for all of the hardware strands.Performs an extended memory test in addition to hardware ECC checks.Tests the functionality of the JBUS-to-PCI-E bridge for the master hardware thread, specifically direct memory access and interrupt testing.The POST interactive menu mode is called when two conditions are met. The first is that the systems virtual keyswitch is set to DIAG mode and the second is that the service processor diag_mode variable is set to menu. You can verify that these conditions are met with the output to the showkeyswitch and showsc commands.The POST menu mode provides access to all tests available within POST, including:built in self testTests performed when diag_level is set to minTests performed when diag_level is set to max

When POST has completed, OBP initializes and works with Hypervisor to build the device tree structure. Even though the Open Boot PROM does not have as much functionality as it previously did, you can still gather some information from it to help you troubleshoot.From the output to the banner command, you can see how much memory has passed POST. From the output to show-devs, show-disks and show-nets, you can see the device tree that has been built, along with the disks and network interfaces that the Open Boot PROM sees.POST displays a great deal of information on errors and warnings to assist you in troubleshooting the cause of a problem. The level of information given is affected by the value of the service processor diag_verbosity variable. Errors follow the standard format that identifies:The test that was executing.The hardware that was being tested at the time of the failureThe suggested repair instructions for which component to replaceAnd the error message generated by the fault.Automatic System Recovery, or ASR, lets you manually manage blacklisted items from the service processor as well as lets the service processor manage the blacklisted items through the vBSC.An ASR database is maintained with a list of any blacklisted items. When POST completes, it returns a status of the components tested to the vBSC. If any of the components are reported back as failed, the service processor attempts to unconfigure them and map them out of the server. An event is then logged to the service processor and to the console regarding this issue.ILOM provides commands to manage the ASR database. The command that you use is cli dependent.In the ILOM cli, you can use the show /SYS command to query the component_state of a specific component.You use the set command to disable and enable a specific component.You also use the set /SYS command to clear the ASR database for a specific component.

Both the ILOM and ALOM cli provide commands for viewing console messages, system messages, and system faults. The types of messages include errors, faults, notices and general system information.In the ILOM cli, execute the show /SP/ logs/event/list command to display messages, notices, and events that have been sent to the service processors event log.The equivalent ALOM CLI command is the showlogs command. The ALOM showfaults command displays current valid system faults.The ALOM cli also provides a command called consolehistory that shows the contents of the boot log and the run log. The boot log contains the messages from POST, OBP, and the booting of Solaris. The run log contains everything in the boot log plus the Solaris runtime messages.The ILOM cli does not provide the console history functionality.Click the links on your screen to view a demonstration of these commands.Several tools and features are available on the Sun SPARC Enterprise T5120/T5220 servers to help administrators monitor, troubleshoot, and diagnose issues. In this section we will discuss the following tools:Software-based commandsApplicationsLog filesYou can monitor and analyze the server status using a combination of these tools.Click as indicated on your screen to view a block diagram showing the diagnostic components on both the host server and the service processor.The Sun SPARC Enterprise T5120/T5220 servers implement the fault management architecture introduced in Solaris 10 OS. Incorporated into both the hardware and software of the Sun SPARC Enterprise T5120/T5220 servers, the FMA helps the server maintain a greater uptime rate by:Automatically and silently diagnosing underlying problemsUsing predictive self-healingDisabling faulty components if necessary and possibleIssuing alerts on problems and logging eventsProviding data to higher-level management services and, in the future, remote servicesBe sure to have your server fully patched for the latest FMA agents.The fault management architecture incorporates the following components to help achieve its goals: Error report creation, where the Hypervisor, vBSC, and hardened device drivers each create reports that in the end are handed off to the Solaris OS fault management daemon.The Solaris OS fault management daemon, fmd, is responsible for forwarding reports generated by the vBSC from the hypervisor and hardened system device drivers to a diagnosis engine.The diagnosis engine contains profiles with fault trees and rules for devices on the server. These rules determine how a fault on a device should be handled and who should be handling it.Fault response agents help the server and system administrator manage the hardware or software fault.Click as indicated on your screen to view the fault management architecture block diagram for the Sun SPARC Enterprise T5120/T5220 servers.

There are several Solaris OS commands associated with FMA. These include:The fmadm command, which lets you view, load, and unload modules. It also lets you view and update the resource cache, which is a list of faulty resources as seen by the fault management daemon, fmd.The fmdump command, which enables system administrators to view any log files associated with fmd and retrieve specific details of any diagnosis issued. By default, the fmdump command lists the fault log, displaying the time, the ID associated with that fault, and the message ID, SUNW-MSG-ID, that can be viewed on Suns message lookup website.The fmstat command, which reports the statistics of the fault management system. By default, the fmstat command lists the active modules and the statistics associated with the modules.

Intermittent problems can often times be difficult to diagnose. Diagnostic tools exercise the components to the point where they display an emerging failing condition. These tools are designed to stress the components to the point of failure. On the Sun SPARC Enterprise T5120/T5220 servers, the Sun Validation Test Suite (SunVTS) is used to exercise the server, as well as for hardware validation and repair verification.The minimum version of SunVTS that supports the Sun SPARC Enterprise T5120/ T5220 servers is SunVTS 6.4 and it ships with the Solaris 10 Update 4 OS.The following Sun SPARC Enterprise T5120/ T5220 components can be diagnosed through SunVTS:CPUFBDIMMsI/OGigabit Netwo