getting started with ssf for maxcore™ mc3000 · pdf filethis document is applicable only...

APPLICATION NOTE

Getting Started with SSF for MaxCore™ MC3000 Application Note P/N: 6806800T98E

March, 2018

Important notice

This document is applicable only for the MaxCore MC3000 systems installed with BBS and SSF for MaxCore releases prior to 2.0. This document is not required for the systems with MaxCore MC3000 Platform Software Release 2.0 or later installed.

2 Getting Started with SSF for MaxCore MC3000

Table of Contents 1 Reading Instructions ............................................................................................................................. 5

2 Introduction .......................................................................................................................................... 6

3 Scope of Document ............................................................................................................................... 7

4 MaxCore Infrastructure Resources ....................................................................................................... 9

4.1 MaxCore Modes ............................................................................................................................ 9

4.2 Power up Sequence .................................................................................................................... 11

4.3 System Resources Identification ................................................................................................. 11

4.4 Default Networks ........................................................................................................................ 12

4.5 Network Boot Service ................................................................................................................. 16

4.6 iSCSI Storage Service ................................................................................................................... 17

5 SSF ....................................................................................................................................................... 18

5.1 GUI .............................................................................................................................................. 18

5.1.1 SSF Configuration and Maintenance Modes ....................................................................... 18

5.2 CLI and XML ................................................................................................................................. 30

6 Scenarios ............................................................................................................................................. 31

6.1 Connect a PC to MaxCore ........................................................................................................... 31

6.2 Update MaxCore ......................................................................................................................... 32

6.2.1 BMC and shelfHost .............................................................................................................. 34

6.3 Reconnect to a lost BMC ............................................................................................................. 34

6.4 Recover a lost shelfHost .............................................................................................................. 35

6.5 Connect to a lost payload CPU .................................................................................................... 36

6.5.1 Open a serial console session over LAN .............................................................................. 36

6.6 Install shelfHost from scratch ..................................................................................................... 36

6.6.1 Install the Artesyn distribution ........................................................................................... 36

6.6.2 Install CentOS from the Internet ......................................................................................... 39

6.7 Install a new payloadCPU ............................................................................................................ 42

6.8 Dealing with the hidden CPU on PCIE-9205 ................................................................................ 43


7 Tutorial ................................................................................................................................................ 44

7.1 Update to BMC and update Firmware ........................................................................................ 44

7.2 Open two BMC consoles ............................................................................................................. 45

7.3 Select shelfHost ........................................................................................................................... 45

7.4 Open shelfHost consoles ............................................................................................................. 45

7.5 Boot shelfHost ............................................................................................................................. 46

7.6 Install CentOS with MaxCore Management Software to shelfHost ............................................ 46

7.7 Update shelfHost BIOS and programmable logic ........................................................................ 48

7.8 Connect to SSF ............................................................................................................................ 49

7.9 Update Baseboard CPLDs ............................................................................................................ 49

7.10 Configure network boot services on shelfHost ........................................................................... 49

7.11 Assign VF to payloadCPU ............................................................................................................ 50

7.12 Boot payloadCPU from shelfHost ............................................................................................... 50

7.13 Install OS to microSD card of payloadCPU .................................................................................. 51

7.14 Update FW on payloadCPUs ....................................................................................................... 51

7.15 Set up iSCSI server (network storage server) on the shelfHost .................................................. 51

7.16 Install OS from iSCSI Initiator to iSCSI Target .............................................................................. 53

7.17 Boot payloadCPU from iSCSI Target ........................................................................................... 54

7.18 Boot payloadCPU from iSCSI Target ........................................................................................... 54

7.19 Move shelfHost to another location ........................................................................................... 54

7.20 Install payload OS on new iSCSI server ....................................................................................... 55

7.21 Add network over the MaxCore Ethernet switch ....................................................................... 56

7.22 Conclusion ................................................................................................................................... 56

8 Glossary ............................................................................................................................................... 57

9 References .......................................................................................................................................... 59

10 Attachments .................................................................................................................................... 60

10.1 iSCSI initialization scripts ............................................................................................................. 60


1 Reading Instructions The table below describes the conventions used throughout this manual.

Notation Description

italic Used for references, tables and figure descriptions and for glossary terms.

Courier Used for filenames, CLI commands and their output.

Courier + Bold Used to characterize user input and to separate it from system output.

CIDR

Classless Inter-Domain Routing

Used for IP address and netmask. For example, 172.27.1.1/16 for 172.27.1.1 with netmask 255.255.0.0).

Bold Used to emphasize a word.

File > Exit Notation for selecting a submenu.

<text> Notation for variables and keys.


2 Introduction MaxCore™ MC3000 is a platform for designing a highly scalable rack-mount appliance. The overall objective of this platform is to enable a wide range of solution providers to create their own competitive appliances and achieve the best time-to-market advantages. This implies the ability to integrate building blocks from an extensive portfolio of existing hardware and software components that are available in the market or from within their own design centers. This also implies an abstraction which allows a primarily software focused solution design team to operate within their comfort zone. On the surface, MaxCore is a rack-mount server with PCIe card slots. MaxCore differs from that of classical enterprise server in the sheer number of PCIe card slots available and its ability to implement many network connected servers within the same enclosure. Many servers in the sense means, there are many independent CPUs, each representing a root for an individual PCI device tree. The innovative feature is in the fact that many CPUs can now share a single SR-IOV capable PCIe endpoint card, for example, a network I/O card or an acceleration card. In addition, MaxCore provides PCIe based network interconnects between CPUs. The MaxCore contains a Baseboard Management Controller (BMC), which is powered and accessible even when the main power is switched off. The BMC runs an off-the shelf web server, MegaRAC® SP from AMI, for remote management. Similar to the BMCs on enterprise servers, the MaxCore BMC provides backdoor entry to the chassis and main CPU.


3 Scope of Document After a successful power up of MaxCore system with the help of MaxCore MC3000 Platform Quick Start Guide (6806800T89) delivered along with the product, the Getting Started with SSF for MaxCore MC3000 manual should be the subsequent document to read for a new MaxCore user. The purpose of this document is to explain some relevant aspects of the MaxCore architecture and to guide the user through some use cases. The comprehensive description of all features and user interfaces is covered in other manuals. This document will point to those manuals and provides an understanding of the whole document kit.

The following are the main topics in this document. You can pick the required ones and get started.

• Update firmware and software of Baseboard and shelfHost: Before you start digging deeper into your MaxCore, we recommend you to update the firmware and software of the Baseboard and shelfHost (the main CPU of MaxCore). Because, even a MaxCore fresh from the factory may not be running the latest versions. Section 6.2 explains this process. If you are not sure what configuration you have, refer to the detailed instructions in sections 6.2.1 and 6.2.1.1 to reconfigure your MaxCore. The minimum requirement to perform this process is a MaxCore with a PCIE-7410 card in slot 1.

• Modes of MaxCore: The first decision for you to make is to select one of the two basic modes of MaxCore. Before selecting the modes, you need ask yourself following questions:

o Do I need to connect many PCIe cards to just a single CPU like you would do with an ordinary server?

o Am I going to build many servers within the same box, each with its individual PCIe device tree and all CPUs connected to each other over an IP network?

For information on which mode to select for your requirement, refer to section 4.1.

• MaxCore CPU Hierarchy: MaxCore implements a strict CPU Hierarchy. BMC is at the top level. It provides the reliable backdoor entry which is available as soon and as long as there is power provided to one of the PSUs. At the next hierarchy level is the shelfHost. It covers the whole scope of the MaxCore infrastructure and controls the third hierarchy level which is populated by the payloadCPUs. Read section 4.2 to understand how these levels are instantiated step by step.

• PCIe Cards Identification: PCIe cards are the key part of your solution. To understand how MaxCore identifies and collects those cards and the pre-installed PCIe devices on the MaxCore Baseboard, see section 4.3.

• MaxCore Default Networks: An IP network is important when you have more than one CPU in MaxCore. Specifically, SSF makes use of the MaxCore default networks. Section 4.4 explains the factory setup.

• SSF GUI: After having understood the MaxCore resources and their dependencies, start playing with MaxCore and learn about the big picture being provided by the shelfHost. Your tool is the


SSF Web GUI. Section 5.1 explains the use of SSF and in that context the more important architectural aspects of MaxCore.

There are situations where you have to touch the BIOS and OS. An obvious one is the situation where you lost your connection to SSF.

Also for installing new payloadCPUs and for doing the more sophisticated network configurations will require that you go down to that level. The Scenarios chapter addresses various cases of that kind and explains some valuable methods to deal with MaxCore. Run the tutorial in chapter 7.2 at any time. Starting with the bare metal MaxCore, it takes you step-by-step through a comprehensive installation process.

This version of the document describes features and issues of the following releases:

• SCML42368 - MC3000 Firmware Customer Package v1.2.0 • SCML42545 - MC3K_PCIE7410_CENTOS_7_2_SW_1_1_0_23 • SCML42546 - MC3K_PCIE9205_CENTOS_7_2_SW_1_1_0_23


4 MaxCore Infrastructure Resources To deal with MaxCore, you should have a brief understanding of its infrastructure. This section provides an overview of MaxCore infrastructure resources.

4.1 MaxCore Modes MaxCore™ MC3000 platform can work in two different architecture modes, namely Single-Host Mode and Multi-Host Mode. A standard MaxCore will be running the more sophisticated Multi-Host Mode when it is shipped from the factory.

Single-Host Mode

This is a trivial case of server compliance, where there is a single PCIe device tree. In this mode, there is exactly one active CPU with a single PCIe root complex and up to 14 PCIe endpoint cards in MaxCore. This CPU is located in either slot 1 or 15 of the card cage. Functionally it is the same as the CPU on a typical server motherboard.

Figure 1: SSF View of MaxCore in Single-Host Mode

Multi-Host Mode

In this mode, MaxCore is equipped with many CPUs. One CPU has to be a shelfHost to perform the central shelf management role. Only the shelfHost has a private connection to the BMC. It has exactly the same relationship as in Single-Host Mode. The shelfHost is a mandatory element in a Multi-Host


MaxCore system. It handles the interconnections between the other CPUs and their PCIe devices and also provides central network services.

Per factory default, the first CPU in slot 1 is assigned as the shelfHost. You can re-assign this role to any of the four CPUs in slots 1 and 15.

To change the MaxCore mode, go to System > MaxCore and then click the Change Mode button on the Overview screen in the work area.

Figure 2: SSF View of MaxCore in Multi-Host Mode.

In the SSF, go to MaxCore > Hardware Manager > Shelf Host Peripheral Assignment > Update Configuration to select the shelfHost location. Select one of the Active radio buttons. You can configure a set of individual servers within MaxCore. Each individual server of this kind combines one CPU with multiple physical and virtual PCIe devices. A network connects all these CPUs to each other, to the outside world, and to a set of network services provided by MaxCore. Boot services build the core of this set.

The System Services Framework (SSF) provides a centralized management interface to configure and monitor software and hardware components across multiple shelves of MaxCore. It provides a simple and uniform interface for configuring and managing the platform.


The scope of this document is to provide a step-by-step explanation of the MaxCore configuration via the interactive SSF GUI. It will familiarize you with the key elements of MaxCore and how they are mapped to the GUI. This document is intended to get you to a level where you can intuitively click and try those elements, which are not documented herein. Use the SSF for MaxCore GUI Online help embedded with SSF application (Click <F1> to view the comprehensive help) and other reference documents for getting the details.

A good understanding of the interactive SSF GUI interface will also help you to find the corresponding commands in the CLI and XML interfaces of MaxCore.

4.2 Power up Sequence MaxCore powers up in three stages. In the first stage, the BMC boots up as soon as the first power cord is plugged into MaxCore. Now, you will have network access to the BMC. In the second stage, BMC powers all the PCIe endpoint cards and the shelfHost CPU. The shelfHost will now enumerate all PCIe devices and it can be accessed over the network when the boot phase is finished. The third stage is applicable in Multi-Host Mode only. The shelfHost now assigns PFs and VFs to the still unpowered payloadCPUs.

There are two possible locations for the configuration file. It is either sourced from the /opt/bladeservices/etc/mccs/mccs_config.xml file on the shelfHost or it is sourced from the BMC, which holds a copy of the most recent write of the mccs_config.xml file to the MaxCore hardware. It is always the file with the more recent file revision or the file from the BMC when both revisions are the same. All configuration changes are stored in both locations. The rationale behind this is the shelfHost failover scenario, where the BMC is the source of the most recent system configuration and the new shelfHost will pull its copy from the BMC.

And lastly, the shelfHost powers the payloadCPUs. The rules for powering shelfHost and payloadCPUs are user controlled. The payloadCPU boots its OS and enumerates all the PFs and VFs which were previously assigned by the shelfHost.

The BMC is configured in Management Power Only mode per factory default. This means, when you plug in the power cord, only the BMC and the RJ45 port with the BMC label will be powered. All 15 card slots will be without power. In this mode, you can access the BMC web server (MegaRAC® SP from AMI) for remote management. The factory default IP address of the BMC is 192.168.201.9. Refer to the MaxCore MC3000 Platform Installation and Use manual (6806800T88) for more details about the BMC’s external interface.

4.3 System Resources Identification As part of the boot process, the shelfHost collects information about all the PCIeLeafs that are connected to the PCIe Baseboard of MaxCore. This is at least the on board dual channel Ethernet controller on the MaxCore baseboard and also each PCIe card in the slot domain. Application software


on the shelfHost can access all these devices through its respective drivers. The shelfHost behaves exactly like the motherboard CPU of an enterprise server (only its access bandwidth is limited by a single PCIe lane when MaxCore runs in Multi-Host Mode). The SSF GUI displays all the PCIeLeafs as a resource tree in the Navigation pane. SSF also identifies installed PCIe roots in the Multi-Host Mode (for example, SharpServer cards) and adds them to the resource tree.

Figure 3: System Resource View

You can browse through the resource tree to validate whether your installed hardware is visible.

4.4 Default Networks The MaxCore faceplate has five RJ45 connectors. They are labeled as BMC, ETH1, ETH2, ETH3, and ETH4.

The BMC connector allows direct access to the BMC. The default IP address is 192.168.201.9/24. This connection works when the main power is switched off. At least one power cord must be connected.

The ETH4 connector allows access to the BMC and to the shelfHost. The default IP address for the BMC is 172.26.1.1/16. The default shelfHost address is 172.26.1.2/16. The third decimal number represents the shelfID. It will automatically change when you change the shelfID.

Errata: The BMC software version 1.8.0 sets the ETH4 default IP address to 172.27.1.1 by mistake. If your network has a gateway, then enter that address. If not, use the eth0 IP address as gateway address. You can change it by going to Configuration > Network > eth0 in MegaRAC application.


The NIC channel behind the ETH3 connector has a Virtual Ethernet bridge (VEB), which is used as an internal network switch. The shelfHost and all payloadCPUs are nodes in this network. The default shelfHost address is 172.27.1.2/16. The payloadCPUs acquire their IP addresses from the shelfHost per DHCP protocol. Geographical IP addresses are assigned according to the formula in the header of the /opt/bladeservices/etc/dhcpd.conf.tpl file on the shelfHost.

The VEB will not work without a network termination at the ETH3 port. There are two ways to terminate it. You can either connect an active cable to the connector or you can configure the port in loopback mode. The MaxCore software should perform the configuration automatically dependent on the port’s link status. You can check and change the loopback mode using the following commands.

mccs_tool.py –-method=set–loopback --func=16, 2

mccs_tool.py –-method=set–loopback --func=16, 2 --mode=off

mccs_tool.py –-method=set–loopback --func=16, 2 --mode=on

MaxCore Ethernet switch has ETH1 and ETH2 ports. The shelfHost, all payloadCPUs, and all networkCPUs have 1G channels to this switch. ifconfig -a should show a pair of interfaces with names like enp4s0f0 and enp4s0f1. The first number in these interface names may not be a four in your situation. However, the first of these two interfaces is your connection to the MaxCore Ethernet Switch. If you want to use the switch, assign each interface a static IP address from a new subnet of your choice.

Figure 4 shows the default network for a MaxCore with a PCIE-7410 in slot 1 and a PCIE-9205 in slot 3. mCPU is used for shelfHost, aCPU is used for payloadCPU, and nCPU is used for networkCPU and RRC is used for the switch on the PCIE-9205. NIC16 is the dual channel Ethernet controller on the MaxCore baseboard. BCM is the Ethernet switch on the baseboard. Other red circles are IP packet switches. The blue arrows from CPUs to PCIe endpoints are the default assignments. You can add and modify the existing assignments through SSF.


Figure 4: Default Networks

The /opt/bladeservices/etc/ directory contains files with the .tpl extension. These template files are used at boot time to create the following files on the shelfHost:

• /opt/bladeservices/etc/dhcpd.conf (using dhcpd.conf.tpl file) • /etc/sysconfig/network-scripts/ifcfg-ETH4 (using ifcfg-mcpu-

extern.tpl file) • /etc/sysconfig/network-scripts/ifcfg-ETH3 (using ifcfg-mcpu-

intern.tpl file) • /etc/sysconfig/network-scripts/ifcfg-e_s16d2f… (using ifcfg-

acpu.tpl file). This file is available on each payloadCPU.

If there is at least one PCIE-9205 installed there will be bridge br0 created at boot time. This bridge gets the 172.27.1.2 IP address. The ETH3 interface and one interface to each PCIE-9205 use this bridge. In that case /etc/sysconfig/network-scripts/ifcfg-br0 (ifcfg-mcpu-br-intern.tpl) is the bridge and /etc/sysconfig/network-scripts/ETH3 (ifcfg-mcpu-br-part.tpl) /etc/sysconfig/network-scripts/enp210f0 and others (ifcfg-mcpu-br-part.tpl) are


the bridge members.

Modify the .tpl files when you want to use this automation. Delete a .tpl file or change its name to disable the automatic file generation. Now, you can edit the configuration file directly and it will not be overwritten at next boot.

Figure 5: shelfHost Interfaces

The interface name on payloadCPUs is derived automatically from the actual PCIe device. For example, the interface name for a PCIe device in slot 04, device 01, and virtual Function 06 will be eth_s04d01f06.


Figure 6: Interface List and Mapping to Target Function

4.5 Network Boot Service PayloadCPUs can boot from the shelfHost. You need to install this boot service. Follow these steps on the shelfHost when the /var/lib/tftpboot directory is empty:

Load the rootfs.tar.bz2 file from SCML42545 - MC3K_PCIE7410_CENTOS_7_2_SW_1_1_0_23 to /opt/bladeservices/tools using the following commands. cd /opt/bladeservices/tools ./disk_install.sh -t -e -s rootfs.tar.bz2

The script will now setup the TFTP server at /var/lib/tftpboot and it will generate a ramdisk image from the rootfs.tar.bz2 file. This takes several minutes. The /var/lib/tftpboot/pcie7410/release directory contains two files, which can be booted by a payloadCPU. They are the Linux kernel and the ramdisk image. The /var/lib/tftpboot/grub.cfg file initializes the boot loader.

To boot this image, reboot a payloadCPU and then enter the BIOS console with <F4> key. Boot from the network device with a 02-xx-xx-10-02-xx MAC address. This corresponds to the ETH3 VEB of the baseboard Niantic. If the boot loader offers options, select one and then wait for the login prompt. With the rootfs from Artesyn's Software download server (SWORDS) you will see the same Artesyn


distribution, which boots from the factory configured SD cards. But now it runs out of RAM exclusively. So any changes you make to the rootfs will not be persistent.

Section 6.6.1 explains how to install to a direct attached storage device.

4.6 iSCSI Storage Service The shelfHost or a payloadCPU can act as iSCSI server for any amount of iSCSI clients in the same box. The iSCSi Server provides many iSCSI Targets. A single target will be assigned to exactly one client. The iSCSI terminology calls the iSCSI client an iSCSI Initiator. The following will use the terms Target and Initiator.

A single Target offers one or many storage devices, the Logical Units (LUN). A storage device can have different behaviors. We will have a closer look at one device type, the block device. From the Initiator’s perspective it behaves like a HDD and can be partitioned, formatted and mounted. The user creates Targets on the selected PCIE-7410 iSCSI server CPU and adds at least one LUN. Such a LUN is associated with a partition on a direct attached storage device like a HDD or SSD. These LUNs are the block storage devices which can be mounted over the network by the assigned Initiator CPU in that same network. The user can then install his OS from USB, CD or over the network to this mounted LUN and finally configure the BIOS to use this LUN as the standard boot device.

An installation and runtime scenario for a payloadCPU in MaxCore would look like this:

1. Boot OS from the TFTP boot server on the shelfHost. 2. Connect to iSCSI Target on a payloadCPU (or on shelfHost). 3. Install an OS to a LUN on that target. The disk_install.sh script does that automatically

for the Artesyn Linux distribution. 4. Reboot and enter the BIOS to select Target and LUN as the new boot device 5. Finish the boot and run the freshly installed OS with its rootfs now located on the iSCSI LUN

The README.iscsi.txt file at /opt/bladeservices/tools location describes the installation and use of the iSCSI storage. You can also follow the instructions in sections 7.15 to 7.17 to run an example.


5 SSF SSF is your default vehicle for configuring and managing MaxCore including most plug-in cards. You can do that interactively with the graphical user interface (GUI) or you can use one of several programmatic APIs. This chapter introduces some relevant methods of SSF based on the GUI and it gives some guidance about the use of the programmatic XML interface.

5.1 GUI

5.1.1 SSF Configuration and Maintenance Modes

When you log in to SSF it will connect to your MaxCore (or a stack of MaxCores after an upcoming SSF update). This is now referred to as Maintenance Mode. You can watch and control your system, but you do not make persistent changes to the semi-static configuration of MaxCore. shelfHost, drive bay and USB assignments as well as the assignment of PFs and VFs from PCIe cards to CPUs are considered semi-static. All these will typically not change at runtime anymore. However, there may be exceptions from that rule.

Note: The SCML42545 - MC3K_PCIE7410_CENTOS_7_2_SW_1_1_0_23 software release still supports semi-static configuration changes to be performed from Maintenance Mode. In the interest of a more consistent GUI, these capabilities will be removed with a future update. We recommend not using this feature.

For changing the semi-static configuration, you need to connect your SSF GUI to a MaxCore emulator. This emulator will run a copy of your real MaxCore configuration. You now apply any kind of changes to your emulated MaxCore. You can store the modified configuration for later use or you can apply it to your real system. This will then initiate the required reboots and will re-connect SSF to your real system. As long as SSF is connected to the Emulator it is in its Configuration Mode. As said before, this mode will not display your real system. Only alarms are still reported. Whenever SSF is in Configuration Mode it will display the name of the connected emulator in its header. It is the active Configuration: … message. No message is displayed in the Maintenance Mode.

Figure 7 - Configuration Mode


The following are the steps to enter into and leave the Configuration Mode:

• Click Configuration in the Navigation pane. The status message should tell you that you are currently connected to the hardware (your MaxCore).

• Go to Copy to selected candidate and select one of the available emulators from the list. They all behave the same way. Press the Copy button which will load the emulator with the configuration of your MaxCore. Wait until the loading is finished.

• Go to Switch emulator candidate and select the same emulator once again. Press the Switch button and wait until the GUI refreshes. The GUI header now displays the connection to the emulator.

• Change shelfHost, drive bay, USB, PF and VF assignments. Nothing will impact your MaxCore. The following sub-sections in this section will elaborate on the specific configuration views.

• If you have to connect back to your MaxCore without applying the changes, click Configuration in the Navigation pane and then click Connect SSF to real hardware button. This will not delete the configuration of your emulator and you can switch back to your emulator at any time and can continue editing the configuration.

• If your edits are finished and you want to apply the new configuration to your MaxCore, click Configuration and then click Apply active candidate to hardware. Confirm the reboot message and wait for one or two reboots. Update your SSF GUI and it will connect to your freshly configured MaxCore. Note: Remember that the reboot may also affect all your payloadCPUs in the same box.

5.1.1.1 Configuration

5.1.1.1.1 Switch between Single_host and Multi-Host Mode

Note: The terminology of GUI and documentation is not consistent. The Single Host Mode is also referred to as Base Mode. The Multi-Host Mode is alternatively referred to as Fabric Mode or Express Fabric Mode. Please consider these terms as synonyms.

Your system should run in Multi-Host Mode when you received it from the Artesyn factory. But check it if you are not sure. To check the current mode, click MaxCore in the SYSTEM branch of the left Navigation pane of your screen.


This loads a new view in the display area to the right in the navigation area. Check if the PexMode is either BASE_MODE (Single-Host) or EXPRESS FABRIC_MODE (Multi-Host Mode). Proceed to the next section when it is Multi-Host Mode. If you have to change the mode, then switch to a MaxCore emulator as explained in section 5.1.1 and apply your changes to the emulator first.

1. Click the Change Mode button on the SYSTEM > MaxCore > Overview screen. 2. Select Multi-Host from the pull-down menu and confirm. 3. Click the Refresh icon and the changes are displayed on the screen.

Follow section 5.1.1 to apply this change to your MaxCore or else proceed to the next section.


Figure 8: Change to Multi-Host Mode

5.1.1.1.2 shelfHost Selection

The shelfHost is a mandatory component in a Multi-Host MaxCore. Its fundamental role is to pull the virtual wires between PCIe devices and many other CPUs. Go through the Navigation pane at the left side to find the CPU which has the shelfHost role assigned. Per factory default, the first CPU in slot 1 is assigned as the shelfHost. You can skip this chapter, if you are fine with the default selection and if this is your first learning experience with MaxCore.

Slot 1 and Slot 15 are special slots of MaxCore and can host the shelfHost. If you populate these slots with a PCIE-7410 card, each of these slots will offer two CPUs as candidates for the shelfHosts. As a result, there are up to four CPUs which can be used as shelfHost. A PCIE-9205 card offers a single CPU which can also take the shelfHost role.

A motivation for moving the shelfHost to the other slot is the available PCIe bandwidth off the respective slot. Slot 1 has 16 PCIe lanes and slot 15 has 8 PCIe lanes. Therefore, move the shelfHost to slot 15 when you need the 16 lanes of slot 1 for payload functions provided by either CPUs or a PCIe endpoint card.


Note: Other than payloadCPUs the shelfHost is connected over a single PCIe lane with Gen 2 speed. That means a net bandwidth of about 3Gbps for the active shelfHost. This is less than the 25Gbps per payloadCPU, but still sufficient for control activities and central network services. Keep this bandwidth restriction in mind.

Now, switch to a MaxCore emulator as explained in section 5.1.1 and apply your changes to the emulator first. The Shelf Host Peripheral Assignment pane view in SSF shows the available CPUs in slots 1 and 15. Select the Active radio button to select your shelfHost and then click the Update button to save the selection.

Figure 9: shelfHost Selection and Peripherals Assignment using SSF

5.1.1.1.3 Shelf Resources or Peripheral Assignments

The four drive bays and the two USB connectors (all accessible from the MaxCore front panel) can be assigned to the CPUs in slot 1 and slot 15 (not to any other slot). This is fully independent on the shelfHost assignment. So drives can be assigned to the shelfHost and/or to a payload CPU to implement a storage server with better access bandwidth. Two USBs can be assigned to two CPUs in slots 1 and 15 to make the next boot attempt from a USB stick.

Figure 10 illustrates the configuration options. The six yellow switches at left and right are directly configurable through SSF (See Figure 10).

The central switch is controlled by the shelfHost selection. Only the shelfHost has the connection to the BMC.


Figure 10: Drive Bay and USB Assignment

To change a drive bay or USB assignment, click the disk and USB icons (See Figure 10) and then click the Update button.

Note: Ensure that you do not disconnect an application relevant boot and storage devices unintentionally.

To apply the changes to your MaxCore, follow the instructions in section 5.1.1 or proceed to the next chapter.

5.1.1.1.4 Building Device Trees

The distinction of MaxCore is its capability to host many PCIe device trees. The following shows how to build such device trees. This can be considered as a virtual soldering exercise, where wires are pulled from PCIeRoots to multiple PCIeLeafs. After you have created your own virtual motherboard, all CPUs will reboot and find the virtually soldered PCIe devices. The following are the two possible connection schemes.

Note: If you are not still connected to an emulator, follow the instructions in section 5.1.1.

• The first and obvious scheme assigns a physical PCIe device (for example, a graphical display adapter) to a payloadCPU. This will remove the PCIe device from the shelfHost device tree and assigns the payloadCPU as the exclusive owner.

• The second and sophisticated scheme assigns just virtual functions to the payloadCPUs. This is possible only for the PCIe cards with SR-IOV support. A prominent example is the Ethernet network adapter card. In this scheme the shelfHost owns the physical function of that device. The payloadCPU will treat the assigned virtualFunction as a physical device. The term virtualFunction comes from the SR-IOV terminology. However, it is misleading in the MaxCore

CPU1CPU2

PCIe-7410

SATA3

SATA4

USB2

Slot 15:

Slot 1:

SATA1

SATA2

USB1

SATA1SATA2

SATA3SATA4

USB2

USB1

ShMC

CPU1CPU2


architecture context, as the OS treats such a virtualFunction like a physical device. Read the glossary for understanding the terminology. It is important to understand that the shelfHost plays an active role in the communication between the PCIeRoot and its virtualFunction. Special device events (for example, a link down) are reported over the physicalFunction only. It is the responsibility of the shelfHost to forward the special device events to the affected payloadCPUs. As these events happen rarely, this shelfHost task is not considered as compute intensive.

You can view and modify the PF and VF assignments in this view. The left frame displays the current assignments. The right frame shows all available PFs and VFs in MaxCore. To view the current assignment, select a port of a CPU.

To change the configuration move objects between the left and the right frame. If the Available Functions frame does not show enough VFs, you can create a new VF pool for a specific PF when you click the Edit VFs button. By doing this it will delete all former VF assignments from this pool.

Figure 11: Endpoint Function Association in SSF


For the purpose of this exercise assign a VF from the Onboard-NIC PF1 to the Port 1 of each CPU. Do not assign anything to CPU Port 2.

Finally apply your changes to your MaxCore. Follow the instructions in section5.1.1.

5.1.1.2 Maintenance

5.1.1.2.1 Server View

The Server View section under the Hardware Manager tab shows the complete system. It shows the real image and slots populated, which are mapped to the real hardware placed at the deployment location.

Figure 12: Server View

5.1.1.2.2 Configuring BMC

SSF covers most of the BMC functionality and is recommended as the primary interface for MaxCore management, except the functions which can only be provided by the always powered on BMC, such as remote power button and serial console over LAN.


Figure 13: Configuring BMC

5.1.1.2.3 Monitoring System Parameters

One of the major functionalities of SSF in the deployment environment is to monitor the system parameters and provide updates about the events and alarms.


The Dashboard tab at Shelf level and at the PCIe card level allows the monitoring of system parameters, such as fan speed, CPU usage, and memory usage. The Dashboard tab shows live data and gets refreshed every five seconds.

Figure 14: Monitoring System Parameters

You can also view the alarms and existing events at the system level using the alarms and events buttons on the top left of the screen. Refer to the SSF for MaxCore GUI help, for more information about the events and alarms. Alternatively, you can also receive these alarms using the XML interface.


Figure 15: System Event Logs

Figure 16: System Alarm List

5.1.1.2.4 Software Update

The MaxCore Software Update feature allows you to update the software installed on various hardware components available in a MaxCore System. It also verifies the compatibility of the latest software image with the current software version to confirm whether you have picked the correct software image for the upgrade. The Software Update button is available at the landing page of the SSF login. Refer the SSF for MaxCore GUI help, for more information.


Figure 17: Software Update using SSF

5.1.1.2.5 Log collection

The Collect Log feature enables you to collect the required system log files from the specified Managed Object Identifier (MOID) level and store the files in .tar format at the pre-defined location in the SSF host. After successful collection, you can download the log using Download log button.

Figure 18: Log Collection

5.1.1.2.6 Configurations

MaxCore configuration is stored in database and configuration files. Complete system hierarchy and platform configuration is stored in the PostgreSQL database and Linux application configurations are stored in the configuration files. SSF supports multiple configurations. Each SSF session has an active configuration. When the active configuration is running, then all the configuration actions (set/get) are updated on the system and the database. You can select any of the existing configurations to be set as the active configuration. If the active configuration is not running, then all the configuration actions (get/set) are stored to the database only.


5.1.1.2.6.1 Creating and Reloading a Configuration SnapShot

You can create a snapshot of the running configuration of MaxCore or of an emulated configuration. The snapshot captured will be of node/object that is being selected in the navigation pane. So, select a specific MaxCore, if you want to store the configuration of the whole MaxCore. The platform related snapshot will be in XML format and the snapshot of Linux applications will be in text file format. All these are packaged in a single file.

You can use this snapshot to replicate the same configuration on another MaxCore or you can update the configuration as per your requirement and reload it. But you need to be careful in editing the configuration, as any wrong configuration can put the system in an unusable state.

Note: Ensure that you reload the snapshot to the same object category, where it was taken from. For example, a MaxCore snapshot to a MaxCore or a PCIE-7410 snapshot to a PCIE-7410.

Figure 19: Configurations in SSF

5.2 CLI and XML Important SSF features for deployment are its programmatic interfaces. You can choose between CLI and XML. Both are documented in their respective reference manuals.

The SSF GUI uses the XML interface internally. All XML commands and responses to those commands are logged in /opt/MaxCore/tmp/applicationinfo.log file on the shelfHost. It is a convenient way to learn about the XML interface and to find the correct syntax when you take an action via the GUI and then read the XML exchange in the log file.


6 Scenarios SSF for MaxCore is designed to support the most common configuration and maintenance tasks. However, there are scenarios which require access at BIOS and OS level. This chapter showcases some of these tasks and explains some interesting methods to deal with MaxCore.

6.1 Connect a PC to MaxCore The BMC connector provides access to the BMC even when the payload power is down. This is the only network connection that supports a remote power up.

The ETH4 connector supports simultaneous access to BMC and shelfHost and BMC as long as the payload power is up. The factory configured IP addresses for BMC and shelfHost over ETH4 are 172.27.1.1/16 and 172.26.1.2/16. A PC IP address with a zero in the 3rd decimal will not collide with MaxCore addresses.

Errata: The BMC software version 1.8.0 sets the ETH4 default IP address to 172.27.1.1 by mistake. You should change it to 172.26.1.1 with MegaRAC placing BMC and shelfHost in the same network. Go to Configuration > Network > eth0

Use Configuration > Network in MegaRAC to configure the two network interfaces of the BMC. eth0 is associated with the ETH4 connector and eth1 with the BMC connector.

You can configure the shelfHost interface by modifying the /opt/bladeservices/etc/ifcfg-mcpu-extern.tpl file. The changes will be applied after the next shelfHost reboot. Rename this file and you can directly modify the /etc/sysconfig/network-scripts/ifcfg-ETH4 file without being overwritten at next reboot.

PayloadCPUs on PCIE-7410 and PCIE-9205 can always be accessed from the shelfHost over the internal default network (Factory default: 172.27.1.<100 + 10*slotID + cpuID>). If you want to access a payloadCPU directly over ETH4, use SSF to assign a VF from device 1 in slot 16 (this belongs to the ETH4) to your payloadCPU.

Use ifconfig -a command on payloadCPU to show an interface with a MAC address like this xx:xx:xx:xx:10:01:VF.

Create a /etc/sysconfig/network-scripts/ifcfg-<interfacename> file for this interface to have a direct access over ETH4. This is helpful when installing software from the internet.

In the case of connection problems in the 172.27.0.0 network, check the loopback mode configuration of the ETH3 channel described in section 4.4.

A convenient setup for debug and integration purpose connects PC, ETH3, and BMC to an unmanaged Gbit Ethernet switch. The eth1 IP address in the BMC is changed to 172.27.1.1 and the ETH3 loopback


mode is OFF. This provides simple access to BMC and all MaxCore CPUs and MaxCore can be powered down and up remotely. ETH4 remains the only official interface for your deployment scenario.

6.2 Update MaxCore You need to first update the core of MaxCore that consists of the BMC, shelfHost and related firmware. After that, you need to update the payload of MaxCore, basically all CPUs except the shelfHost.

If you have to install the OS from scratch, follow the instructions in section 6.4 of this document.

In any case you need to download the software from the Artesyn SWORDS server:

1. Log in to Artesyn’s Customer Resource Center (CRC) web portal: http://crcportal.artesyn.com a. If you need to be registered, please reference the How to Register for an Account

document under the Artesyn Links on the left-hand side of the login webpage. b. If you still require help, select the Contact Us link on the webpage to email our Contact

Center for assistance with CRC/SWORDS access. 2. Under Support Links, select SWORDS (Software Download) to access the SWORDS server.

3. Log in with the same credentials you used for the CRC. To continue, select Agree. Note: You can also select to be notified of new versions of software for your products when they are available.

http://crcportal.artesyn.com/


4. You will now see the Search screen in SWORDS, using the heeds under Note shown in the following screen.

a. After entering field information, click Search, you will see all the software available for the products you purchased. You can sort on a number of the fields or refine your search by using the four fields as shown above. For example, enter %7410% in the Product Part Number field to see the software associated only with the PCIE-7410 card, as shown below.

You can now select and download the software of your choosing.


6.2.1 BMC and shelfHost

The following is the sequence for updating the core of MaxCore:

1. BMC software. The BMC (Baseboard management controller) is located on the Baseboard. It runs on its own, which means that it is independent on the shelfHost and other CPUs in MaxCore.

2. MaxCore baseboard Firmware. This is for programmable logic devices (PLDs). 3. shelfHost OS with Basic Blade Services (BBS). 4. BIOS and other firmware on shelfHost. 5. shelfHost applications like SSF and ViewCheck.

To update the baseboard firmware, open SSF application and click the Software Update button. Four individual devices (BMC and three CPLDs) that belong to the baseboard are listed.

Open an SSH session on the shelfHost and use the same IP address that is being used by SSF and login with root/root.

Use SCP or WinSCP to copy the .iso file of the 7410 installation update package from your PC to the shelfHost.

You can follow the update instructions in the MaxCore MC3000 Platform Installation and Use (6806800T88) manual to update the installed OS.

You can follow the update instructions in the MaxCore MC3000 Platform Installation and Use (6806800T88) manual to update the BIOS and programmable logic of the shelfHost. When you perform the OS update you would have already loaded the required images to the shelfHost.

Open SSF and check if the Navigation pane lists your installed hardware and then check the recently installed firmware and software revisions.

6.2.1.1 Payload CPUs

Updating payloadCPUs requires the same process as described for the shelfHost. You can either copy the .iso image from the shelfHost to each payloadCPU with the SCP command or you can mount the shelfHost directory with the .iso file.

6.3 Reconnect to a lost BMC Note: This should never happen in the real scenario, in case it does, follow the information provided in this section.

First attempt is to reboot the BMC. Before doing that, shutdown all CPUs for which you have the access to in a graceful manner. Then unplug all the power cords and plug them in again after waiting for at least 10 seconds. Wait a minute or two to allow BMC to reboot and then try to connect over the RJ45 connector, which is labeled as BMC. Try the IP address that you have configured earlier. If this does not


work, press the reset button for at least 10 seconds. This will set the BMC IP addresses and all other interface parameters to their default values. Now, try 192.168.201.9.

Note: Do not forget to configure the BMC eth0 for access over the ETH4 connector, if you have used that before. It changed to its default as well.

If it still does not work, most likely you have defective hardware or corrupted firmware and you may need support from Artesyn.

6.4 Recover a lost shelfHost The following is going to install OS, BBS, and SSF to the shelfHost. This will overwrite your storage media.

Use the remote console provided by MegaRAC. In the MegaRAC application, go to Remote Control > Console Redirection and click Java Console button.

When your PC is set up correctly, the JViewer window will pop up as shown in the following figure.

If the JViewer window does not pop up, run the Java engine from http://www.java.com. The Java security settings must enable the IP address of the BMC.

In such case, use the Java Control Panel to change the security settings and add the BMC’s IP address to the Java Site List. In your PC, go to Start > Configure Java > Security > Edit Site List > Add to add the IP address.

https://www.java.com/en/


Press <Enter> and the OS login will indicate if the OS is up and running. Change the Keyboard Layout as needed. Sometimes, you have to select the English keyboard even if you have something different. Try it.

In MegaRAC application, go to Remote Control > Server Power Control to reboot the shelfHost and get access to the shelfHost BIOS in the remote console.

6.5 Connect to a lost payload CPU

6.5.1 Open a serial console session over LAN

Open an SSH session on the BMC using the same IP address and login credentials as MegaRAC. /opt/fru/bin/sol <slotID> <cpuID>

The slotID ranges from 1 to 15 and the cpuID is either 1 or 2. Press <ESC + t> key, if you want to terminate the session and open a new one for another CPU. Be aware that the session terminates automatically after 1800 seconds. Re-open it when you see the BMC prompt.

6.6 Install shelfHost from scratch

6.6.1 Install the Artesyn distribution

The following process requires several reboots of the shelfHost. Be aware that each reboot of the shelfHost will result in a non-graceful reset of all payloadCPUs. If you want to make sure that nothing wrong happens to your payloadCPUs, shut them down before you reboot the shelfHost.

MegaRAC is your vehicle to access the console of the shelfHost when nothing else works anymore. Follow section 6.4 to connect via MegaRAC remote console.

1. Go to the JViewer console and open the Virtual Media Wizard from the Media menu.


2. Click the Browse button in the Hard Disk frame to select the install_disk.img file. Section 6.1 explains you where to find the file.

3. Connect the Hard Disk by clicking the Connect button. You have created a remote hard disk which is now physically connected to the shelfHost CPU in your MaxCore.

4. Booting the shelfHost In the MegaRAC GUI, go to Remote Control > Server Power Control and then reset or power cycle the CPU. Wait until the BIOS shows up in the JViewer window and press <F4> to enter the boot console. Select Virtual HDi from the list and allow the CPU to boot from that location.


It may take 30-60 minutes to boot. Go to the next step when the OS login shows up in the JViewer window. Note: No progress will be shown on the screen during boot time.

5. Install OS When the login screen appears, follow the instructions in the MaxCore MC3000 Platform Installation and Use manual to install the OS. Execute the disk_install.sh command with the -s parameter to enable SoL access to the new OS. Add the -t parameter to create the TFTP server with the PCIE-7410 boot image files. Before you reboot your newly installed OS, go back to the Virtual Media Wizard and disconnect the Hard Disk.

You can always use the MegaRAC remote console for BIOS access and also access to the OS when the network does not work. However, the JViewer console is not as convenient as using a remote terminal like PuTTY. MaxCore provides a feature called Serial over LAN (SoL), which allows one to connect the serial console of any CPU in MaxCore to a remote terminal, always one at a time. Now, you have the OS and an access over SoL. Refer to section 6.5 to open the SoL session.


6. Gain network access to the shelfHost. Connect your PC to the ETH4 connector. The default IP address is 172.27.1.2. You can change the default by modifying the /opt/bladeservices/etc/ifcfg-mcpu-extern.tpl file. This is a template which is used to create /etc/sysconfig/network-scripts/ifcfg-ETH4 file each time you boot. You can also delete or rename this file and it will not be used anymore for automatic file generation. You can also directly modify the /etc/sysconfig/network-scripts/ifcfg-ETH4 file. Note: When you allow MaxCore to acquire an IP address from your office network be aware that the hostname of the shelfHost is the same for all MaxCores in your system by default. You can customize the hostname with a leading string. Modify the hostname parameter in the /opt/bladeservices/etc/init.d/hostname file and reboot.

7. SSF Installation Before SSF installation, install other packages of your choice. It is recommended because the SSF installation procedure does not update the database of the yum package manager and the subsequent installations may corrupt things. For SSF installation, follow the instructions in the MaxCore MC3000 Platform Installation and Use manual.

With each reboot the shelfHost will recreate some configuration files to assign geographically associated IP addresses to the payloadCPUs. If this is what you want, edit the .tpl files in the /opt/bladeservices/etc/ directory and do not touch the files being generated. If you do not want this automation, rename the .tpl files and the next reboot will not overwrite the dhcpd.conf and the ifcfg-files anymore.

Note: The dhdpd.conf file is no longer available in its usual location. Modify this file in the /opt/bladeservices/etc/ directory. The /opt/bladeservices/etc/dhcpd.conf.tpl file contains the parameters for calculating the IP addresses.

6.6.2 Install CentOS from the Internet

This chapter provides a short overview/outline on how to setup a network installation environment for OS installation from a standard CentOS ISO-file on the PCIE-7410 management CPU (shelfHost) of the MaxCore system so that it can be used for installation of any payloadCPU in the chassis.

First check if the TFTP service is installed on your shelfHost. The procedure is described in section 4.4.

Note: Lines starting with the > character as OS prompt placeholder contain commands, which may be executed on the target, possibly followed by lines starting with the < character containing example responses to the executed commands.

1. Create a dedicated sub-directory and copy the respective CentOS ISO-file to this location. An example with CentOS 7.2 minimal ISO is shown below. > mkdir /var/lib/tftpboot/centos72


> cd /var/lib/tftpboot/centos72 > scp <external-server-ip>:/path/to/CentOS-7-x86_64-Minimal-1511.iso

2. Unpack the ISO-file content parts needed for network installation. > mkdir -p /mnt/centos_iso > mount -o loop CentOS-7-x86_64-Minimal-1511.iso /mnt/centos_iso > cp -r /mnt/centos_iso/LiveOS . > cp -r /mnt/centos_iso/images . > umount /mnt/centos_iso

3. Configure and start NFS server on shelfHost. > echo "/var/lib/tftpboot/ 172.27.1.0/24(ro)" > /etc/exports > systemctl restart nfs-server

4. Add the following menu entry to the grub.cfg file. menuentry 'Install CentOS 7 from ISO' { macappend "ifname=bootnet:" linuxefi /centos72/images/pxeboot/vmlinuz

inst.repo=nfs:nfsvers=3:172.27.1.2:/var/lib/tftpboot/centos72/inst.text quiet console=ttyS1,38400 ip=:::::bootnet:dhcp modprobe.blacklist=fm10k

initrdefi /centos72/images/pxeboot/initrd.img }

a. Preparation in step 4 will lead to the interactive installation menu.

If an automated install via anaconda kickstart file is desired, copy the kickstart file contents shown at the end of this document to the /var/lib/tftpboot/centos72/anaconda-ks.cfg file on the shelfHost and change the menu entry as follows:

menuentry 'Install CentOS 7 from ISO (auto-install with anaconda)' --class fedora --class gnu-linux --class gnu --class os {

macappend "ifname=bootnet:" linuxefi /centos72/images/pxeboot/vmlinuz

inst.ks=nfs:nfsvers=3:172.27.1.2:/var/lib/tftpboot/centos72/anaconda-ks.cfg inst.text console=ttyS1,38400 ip=:::::bootnet:dhcp modprobe.blacklist=fm10k

initrdefi /centos72/images/pxeboot/initrd.img }

After booting, as described in step 5, the installation will be done automatically.

5. Connect to the serial over LAN (SoL) console of the payloadCPU.

Reboot or reset the target payloadCPU and enter the Boot Menu with <F4> key, enter the Boot Manager and select the internal network device by MAC address scheme 02-01-??-10-02-?? and boot from it.


Inside the grub menu, select the above-defined entry 'Install CentOS 7 from ISO' and start booting.

After interactive (4.) or automated (4a.) installation, reboot the payloadCPU and select the device, where the installation was done, as the boot device.

The following are the contents of the sample kickstart file. /var/lib/tftpboot/centos72/anaconda-ks.cfg ### content begin ### #version=DEVEL # System authorization information auth --enableshadow --passalgo=sha512 # Use text mode install text # Use NFS installation media nfs --server=172.27.11.2 --dir=/var/lib/tftpboot/centos72/ --opts="nfsvers=3" # Run the Setup Agent on first boot firstboot --enable ignoredisk --only-use=sda # Keyboard layouts keyboard --vckeymap=us --xlayouts='us' # System language lang en_US.UTF-8 # Network information #network --bootproto=dhcp --device=enp3s0f0 --onboot=off --ipv6=auto network --bootproto=dhcp --device=bootnet --ipv6=auto --activate #network --bootproto=dhcp --device=enp3s0f1 --onboot=off --ipv6=auto #network --bootproto=dhcp --device=enp11s24f4 --onboot=off --ipv6=auto network --hostname=localhost.localdomain # Root password (here: "root") rootpw --iscrypted $6$LTcRHgKYDCjaR8ri$FdoGQd..D287WKkkknHQxVGl..Ghq6ODJpFE6Dr64sen9uWbPt5ILIud0d5KToI9g6RXlqBaRwTo8Wdjj4L/O0 # Do not configure the X Window System skipx


# System timezone timezone Europe/Berlin --isUtc # System bootloader configuration bootloader --append=" crashkernel=auto" --location=mbr --boot-drive=sda autopart --type=plain # Partition clearing information clearpart --all --initlabel --drives=sda %packages @core kexec-tools %end %addon com_redhat_kdump --enable --reserve-mb='auto' %end ### content end ### (non-management) CPU (payloadCPU) in the chassis.

6.7 Install a new payloadCPU

You can do the following now with the second CPU on the already installed PCIE-7410 card or you can install additional PCIE-7410 cards in any other slot of MaxCore. Follow the instructions in the MaxCore MC3000 Platform Installation and Use (6806800T88) manual for installing new cards into the box.

Install the TFTP server on the shelfHost, if not done already. The installation is described in section 4.5.

The Remote Console of MegaRAC connects to the shelfHost only. You have to establish the initial connection to your payloadCPU over the serial console.

Follow the instructions in section 6.5.

Now, reset your payload CPU and execute the following command on the shelfHost. mccs_tool.py --method=set-cpu-power --cpu=<slotID,cpuID> --power=reset

The BIOS of the payloadCPU should now show up. Press <F4> to enter the Boot menu and select the EFI boot device with the xx:xx:xx:10:02:xx MAC address and then boot from location. Identify the target device for the OS installation using the parted -l command.

Start the installation using the following commands.

cd /opt/bladeservices/tools


./diskinstall -d /dev/<your storage device> -e -s -i 172.27.1.2 /pcie7410/release/rootfs.tar.bz2

The -s parameter creates an OS with console output over SoL. Without that parameter, the OS will the use the microUSB port at the panel. The installation image is sourced from the TFTP server on the shelfHost.

Reboot the CPU when the installation is finished and enter the BIOS menu by pressing <F2>. Select your newly installed device for subsequent boots in the BIOS BOOT submenu.

6.8 Dealing with the hidden CPU on PCIE-9205 This CPU has no direct connection to the baseboard NIC, which is used by the PCIE-7410 CPUs for switching. But, the default network setup connects this CPU to the default network over the RedRockCanyon (RRC) switch as illustrated in Figure 4.

A different configuration of the RRC may confuse this network setup and disconnect the CPU from the default network. SoL is the connection which is always available. Use it to check your network settings and to adjust those manually.

A network over the Ethernet switch on the baseboard is quite helpful, because this switch is readily configured before the CPUs power up. Figure 4 shows the relevant interface names on shelfHost, payload CPUs and the PCIE-9205 CPU. You can assign IP addresses to make use of this network. Section 7.21 shows how to setup this network with DHCP service.

This network is the best option for the remote installation from a TFTP. Copy the ramdisk.image.gz file and the kernel files from SCML42546 - MC3K_PCIE9205_CENTOS_7_2_SW_1_1_0_23 to /var/lib/tftpboot/pcie9205/release and add a new menu entry to /var/lib/tftpboot/grub.cfg


7 Tutorial The following is an exemplary setup of a MaxCore. This exercise starts with a MaxCore system (without any OS being installed) and demonstrates most of the methods described in this document. The sub- sections in this section are interrelated. Therefore, follow each step accurately to create a system with network, network boot, and network provided storage devices at the end of this section.

This tutorial uses a simplified nomenclature which looks like the following way.

MegaRAC: Remote Control > Server Power Control > Power Cycle Server > Perform action

The text before the colon points to a tool or console which shall be used. The text next to the colon is either a CLI command or an > separated sequence of mouse clicks. Login credentials are shortly described with a slash (/) between username and password (for example, admin/admin).

Equipment needed - A Windows PC with two network interfaces. One to the Internet (may be WiFi) and the second

one an RJ45 to the MaxCore. The RJ45 must support 1Gbps link speed or you have to add a Gigabit switch between PC and MaxCore.

- A MaxCore with a PCIE-7410 in slot 1. This is the factory configuration of the standard product. - Two microSD cards with 16GB per card at least. These cards are already mounted on the PCIE-

7410. - mc3000-BMC-<version number>.bin, mc3000-cpld-companion-<version

number>.hpm, mc3000-cpld-glue-<version number>.hpm, and mc3000-cpld-iomod-<version number>.hpm files (You can download the latest SCML (SCML42368 - MC3000 Firmware Customer Package v1.2.0) from SWORDS or contact your local Artesyn executive)

- install_disk.img file. You can download the latest SCML (SCML42545 - MC3K_PCIE7410_CENTOS_7_2_SW_1_1_0_23) from SWORDS.

- A 2.5” SATA storage device (HDD or SSD) and a second PCIE-7410 to perform the final step of this exercise.

Note: Make sure that all four drive bays are empty when you start the process. This shall avoid confusion with the device naming.

7.1 Update to BMC and update Firmware To connect to BMC and update firmware:

1. Configure PC cable interface: 192.168.201.1/24 (CIDR notation for a 24-bit netmask). 2. Connect PC to ETH3 connector. 3. Connect power to MaxCore and push the power button for more than 10 seconds. This resets

the BMC IP addresses to their factory defaults. Allow the BMC a minute to boot. 4. Open browser session to 192.168.201.9. MegaRAC will pop up.


5. Login to MegaRAC with admin/admin. 6. Check the Firmware Revision. The firmware version has to be 1.8.0 or later.

If you see a revision older than 1.8.0, follow these steps: a. In the MegaRAC, go to Firmware Update > Firmware Update and click Enter Update

Mode button. A message box asking confirmation is displayed. b. Click OK. Wait until you are being asked to browse for the image file and then select the

image file that you have loaded to your PC. Note: Contact your Artesyn executive for the latest version. The file name of the firmware version image is mc3000-BMC-<version number>.bin.

c. Click Upload and wait for the next dialog. d. Select the Full Flash check box, click Proceed and then click OK to confirm.

Wait for some time. The new image will be flashed now and the BMC will reboot automatically.

e. Open the browser session to 192.168.201.9 and login with admin/admin. 7. Align the ETH4 IP address with the shelfHost IP address and change it to 172.26.1.1/16. Gateway

172.26.1.1 MegaRAC: Configuration > Network > LAN Interface > eth0

8. Change the IP address of your PC's cable interface to 172.26.0.1/16. Then, connect your PC to ETH4 and use your web browser to connect to MegaRAC via http://172.26.1.1

7.2 Open two BMC consoles

• Open browser session to 172.26.1.1. MegaRAC will pop up. • Open SSH session on BMC (sshBMC).

o Start PuTTY terminal emulator on the PC and connect with SSH to 172.26.1.1 o Login with admin/admin.

The following steps will make use of these two sessions (MegaRAC and sshBMC).

7.3 Select shelfHost Select cpu 1 in slot 1 to become the shelfHost after the next power cycle (This is the case already for a product fresh from the factory).

sshBMC: /usr/local/bin/ipmitool -I lan -H 127.0.0.1 -U admin -P admin raw

0x2e 0x30 0xae 0 4 01 03 0x60 4 01 03 0x61 0 15 03 0x60 0 15 03 0x61 0

7.4 Open shelfHost consoles To open shelfHost consoles, perform the following steps:


- Start VGA console (vgaCPU) session to shelfHost (JViewer). This gives you guaranteed console access and you can also run software with graphical output.

o Install Java from www.java.com to your PC (if it is not already installed). o Go to Java Control Panel> Security > Exception Site List and add 172.26.0.1 to that list. o Go to MegaRAC > Remote Control > Console Redirection and click Launch to view the

JViewer window. Note: Accept security warnings message to view JViewer window.

- Start serial console TTY session (ttyCPU) to shelfHost: This gives you the more convenient CLI console, if the OS is configured to redirect its console to ttyS1

o sshBMC: /opt/fru/bin/sol 1 1

(This connects to the serial port (ttyS1) of CPU 2 in slot 15).

7.5 Boot shelfHost To boot shelfHost, perform the following steps:

1. In MegaRAC application, go to Remote Control > Server Power Control > Reset Server and click Perform Action. Warning: Avoid Power off Server because it will disconnect your network connection over ETH4. You have to power up MaxCore by pushing the power button when, if power off occurs.

2. Press <F2> to enter the BIOS menu.

7.6 Install CentOS with MaxCore Management Software to shelfHost To install CentOS with MaxCore Management software to shelfHost:

1. Create remote HDD with installation image. In JViewer window, go to Media > Virtual Media Wizard.

a. Browse the install_disk.img file (downloaded from SCML42545 - MC3K_PCIE7410_CENTOS_7_2_SW_1_1_0_23) in the hard disk frame.

b. Click Connect Harddisk. c. Click Close.

2. Reboot shelfHost. In MegaRAC application, go to Remote Console > Server Power Control, select Reset Server and click Perform Action.

3. Observe the JViewer session and press <F4> to enter the boot menu. It takes a while to enter the boot menu.

http://www.java.com/


4. Select Virtual HDisk as next boot device and press Enter to boot. This may take about 30 minutes to boot 1GB over the network between PC and MaxCore. The screen will remain black during that time.

5. vgaCPU: Log in to shelfHost with root/root. a. Identify the storage device name for the microSD card.

Note: We are currently shipping a Model: Generic Ultra HS-COMBO microSD card, but that may change. The assumption for this tutorial is /dev/sda parted -l

b. Install OS to microSD card. cd /opt/bladeservices/tools ./disk_install.sh -d /dev/sda -e -s rootfs.tar.bz2

Wait until installation is finished. c. Copy a file to the installed disk for later use.

mount /dev/sda3 /mnt mkdir /mnt/root/archive/pcie7410 cp rootfs.tar.bz2 /mnt/root/archive/pcie7410 umount /mnt

6. Disconnect remote storage. In JViewer menu, go to Media > Virtual Media Wizard, click Disconnect Hard disk and close the window.\

7. Set next boot device vgaCPU:

reboot

o Enter BIOS with <F2> BIOS: BOOT > EFI Device First [Disabled] BIOS: BOOT > Legacy > Boot Type Order

Move the USB (BIOS sees the microSD card as a USB device) to the top of the list, save with F10 and let the shelfHost boot from its microSD card.

8. ttyCPU: o We will no longer use the JViewer in this example because the PuTTY terminal

emulation is more convenient to use. But, JViewer remains being our final fallback to gain connection to the shelfHost.

o Look at the TTY session. It timed out after 30 minutes when you see the BMC $ prompt. In that case re-open the tty session with cpu 1 in slot 1

/opt/fru/bin/sol 1 1

o Watch the boot messages and login with root/root. The login prompt shows pcie7410-s1-c1.

o You can edit the file below to change the external IP address of the shelfHost. Changes will be applied with a reboot. Leave it unchanged now.


vi /opt/bladeservices/etc/ifcfg-mcpu-extern.tpl

o ifconfig ETH4 shows the IP address for network access from the PC. It is 172.26.1.2 when you did not change it.

9. sshCPU: o Start another PuTTY session and connect with ssh to 172.26.1.2 o Login with root/root o This is now our preferred access window to the OS. The TTY window will only be used

when we need access to BIOS. 10. Check software releases.

cat /etc/blade-release

(Shall be pcie-rootfs-1.4.2) cat /etc/ssf-release

(Shall be SSF_VERSION=1.1.0.21)

7.7 Update shelfHost BIOS and programmable logic

Go to the cd /opt/bladeservices/rom/ directory and perform the following steps:

1. Check latest available revisions of CPLD and BIOS (CPU). cd /opt/bladeservices/rom/ ls

2. Check installed revisions of CPLD and BIOS (CPU). fcu -q

3. Load CPLD to hardware. fcu -uf pcie7410-cpld-<version number>.fri

4. Load BIOS image to hardware. fcu -uf pcie7410-cpu-<version number>.fri

5. Perform MaxCore power cycle to apply the firmware images. o TTY console: halt o Wait for System halted message. o In the MegaRAC application, go to Remote Control > Server Power Control > Power

Cycle Server and click Perform Action. o Check updated firmware revisions using the following command.

fcu –q


7.8 Connect to SSF To connect to SSF:

1. Open the browser session on the PC and connect to 172.26.1.2 2. Log in with Admin/Admin.

Note: If you receive an error message in response to your login attempt then allow SSF some more minutes to initialize and to collect the MaxCore resources. The SSF core services are not fully setup when you receive error message. You may have to wait several minutes after a shelfHost boot.

3. Check which VFs are connected to which payload CPU ports. These VFs will show up as network interfaces on the payloadCPUs. Do not change anything now. In SSF GUI, go to SYSTEM > MaxCore > Endpoint Function Association > select CPU Port

4. Use the Navigation pane at the left to select an object and browse through the main pane to the right to see object attributes, sensors, and controls.

o SSF: SYSTEM > MaxCore: Hardware Manager o SSF: SYSTEM > MaxCore: Overview o SSF: SYSTEM > MaxCore: Dashboard o SSF: SYSTEM > slot.1 > PCIE-7410 > CPU.1 > Host OS: OS/Services o SSF: SYSTEM > slot.1 > PCIE-7410 > CPU.2 > Overview

7.9 Update Baseboard CPLDs To the update Baseboard, perform the following steps in SSF application:

1. In the Navigation pane, select MaxCore. 2. Click the Software Update button on the top-right corner to open MaxCore Software Update

dialog box. 3. Select the required images from Select Image drop-down list for update, if required.

You can get the latest image files of BMC CPLD Companion, BMC CPLD Glue and BMC CPLD I/O Module as mc3000-cpld-companion-<version number>.hpm, mc3000-cpld-glue-<version number>.hpm, mc3000-cpld-iomod-<version number>.hpm respectively, which can be downloaded from SCML42368 - MC3000 Firmware Customer Package v1.2.0.

7.10 Configure network boot services on shelfHost To configure network boot services on shelfHost:

1. On shelfHost, run the following command. cd /opt/bladeservices/tools ./disk_install.sh -t -e -s /root/downloads/pcie7410/rootfs.tar.bz2


If you did not copy the rootfs.tar.bz2 file during the shelfHost installation you can load it from SCML42545 - MC3K_PCIE7410_CENTOS_7_2_SW_1_1_0_23 o /var/lib/tftpboot is now the root of the TFTP server.

o /opt/bladeservices/etc/dhcpd.conf.tpl generates /opt/bladeservices/etc/dhcpd.conf at each reboot. dhcpd.conf hands over the TFTP server parameters to the payload CPUs. You can edit the content of dhcpd.conf.tpl.

o Check the dhcpd service status after a reboot whenever you applied changes to dhcpd.conf.tpl file systemctl status dhcpd.service

7.11 Assign VF to payloadCPU To assign VF to payloadCPU:

1. In the SSF GUI, go to MaxCore > Endpoint Function Association > Port 1 of CPU 2. This should have a VF assigned from PF 2 in Slot 16. If there is no VF assigned, click Enable Configuration to assign VF(s).

2. After assigning VFs, click Apply and wait until MaxCore does a complete system reboot. 3. Check connection to payload CPU.

sshCPU: ping 172.27.1.112

7.12 Boot payloadCPU from shelfHost To boot payloadCPU from shelfHost:

1. ttyCPU: Press <ESC + t> to terminate the tty session to the shelfHost. You will get the $ prompt from the BMC.

2. Open TTY session to cpu 2 in slot 1. /opt/fru/bin/sol 1 2

3. Reset your payload CPU. Execute the following command on the shelfHost. mccs_tool.py --method=set-cpu-power --cpu=1,2 --power=reset

4. ttyCPU: Enter the BIOS boot console with <F4> key. 5. Boot from network interface with MAC address 02:01:00:10:02:xx (xx is the number of the

assigned VF). 6. Login with root/root.


7.13 Install OS to microSD card of payloadCPU To install OS to microSD card of payloadCPU:

1. Identify the storage device name for the microSD card. It is /dev/sda for this example. parted -l

2. Install OS to microSD card. cd /opt/bladeservices/tools ./disk_install.sh -d /dev/sda -e -s -i 172.27.1.2

/pcie7410/release/rootfs.tar.bz2

(The path to the rootfs.tar.bz2 file starts at the root of the TFTP server. That is /var/lib/tftpboot on the shelfHost) reboot

3. Enter BIOS with <F2> key and modify the following parameters. BIOS: BOOT > EFI Device First [Disabled] BIOS: BOOT > Legacy > Boot Type Order

Note: Move the USB (BIOS sees the microSD card as a USB device) to the top of the list, save with <F10> and let the shelfHost boot from its microSD card.

4. Open new PuTTY session and login to the shelfHost. 5. Open an SSH session to the payloadCPU (cpu 2 in slot 1).

ssh 172.27.1.112

6. ttyCPU: Press <ESC + t> to terminate the tty session to the first payloadCPU. You will get the $ prompt from the BMC.

7.14 Update FW on payloadCPUs To updated firmware on payloadCPUs:

1. Run the fcu -q (already described) command to check the installed firmware versions on each payloadCPU and compare them with the version numbers of the image files in the /opt/bladeservices/rom directory.

2. Update the firmware, if needed and do not forget the MaxCore power cycle to apply the images. It is good to run just one power cycle after images have been loaded to all payloadCPUs.

7.15 Set up iSCSI server (network storage server) on the shelfHost To set up iSCSI server on the shelfHost:

1. Plug a 2.5” storage device into the leftmost drive bay of MaxCore and install a second PCIE-7410 in slot 15. Before you do that, shut down your payloadCPU first and then the shelfHost. Then power down MaxCore and finally unplug the power cables. Power up MaxCore after installing the hardware.


2. In the SSF application, go to SYSTEM > MaxCore> Hardware Manager > Shelf Host Peripheral Assignment > Update Configuration

a. Select the disk icon number 1, next to cpu 1 in slot 1. b. Click Update and then click Apply. This will reboot shelfHost and payloadCPUs. c. cpu1 in slot 1:

o Find the disk drive in first bay with parted -l The following assumes /dev/sda, change it, if needed.

d. Use fdisk /dev/sda to create several GPT partitions. Typically one per payloadCPU e. Format partition with mkfs.ext4 /dev/sda1 f. Format all other partitions the same way.

3. Start the iSCSI server. systemctl enable target.service systemctl start target.service

The above task requires a reboot.

4. Configure an iSCSI Target on the server. targetcli

cd / clearconfig confirm=TRUE

a. Connect block devices. cd /backstores/block create name=bay1disk1 dev=/dev/sda1

b. Create target with single LUN. /iscsi create iqn.2015-07.com.artesyn:target-mc1s1c2 cd /iscsi/iqn.2015-07.com.artesyn:target-mc1s1c2/tpg1 set attribute authentication=0 demo_mode_write_protect=0

generate_node_acls=1 cache_dynamic_acls=1 cd luns create /backstores/block/bay1disk1

c. Save cd / saveconfig ls

d. Exit this shell. exit

5. Add more iSCSI targets as needed (one per payloadCPU). targetcli

a. Connect block devices. cd /backstores/block create name=bay1disk2 dev=/dev/sda2


b. Create target with single LUN. /iscsi create iqn.2015-07.com.artesyn:target-mc1s15c1 cd /iscsi/iqn.2015-07.com.artesyn:target-mc1s15c1/tpg1 set attribute authentication=0 demo_mode_write_protect=0

generate_node_acls=1 cache_dynamic_acls=1 cd luns create /backstores/block/bay1disk2

c. Save cd / saveconfig ls

d. Exit this shell. e. Exit.

7.16 Install OS from iSCSI Initiator to iSCSI Target To install OS from iSCSI initiator to iSCSI target:

1. TTY: Open session to cpu 2 in slot 1. /opt/fru/bin/sol 1 2

2. Login with root/root You do not have to install the microSD card. Without microSD, you need to boot from TFTP server and then perform the following steps. systemctl enable iscsid.service systemctl start iscsid.service

3. Connect to iSCSI Target. iscsiadm -m discovery -t st -p 172.27.1.2:3260 iscsiadm -m node --targetname iqn.2015-07.com.artesyn:target-mc1s1c2 --

portal "172.27.1.2:3260,1" --login

4. Install OS. cd /opt/bladeservices/tools ./disk_install.sh -d iscsi:172.27.1.2:::0:iqn.2015-

07.com.artesyn:target-mc1s1c2 -n iqn.2015-07.com.artesyn:initiator-mc1s1c2 -s -i 172.27.1.2 /pcie7410/release/rootfs.tar.bz2

5. Identify the Linux device name for the newly created storage device. The device name is at the end of the output. iscsiadm -m session -P 3

6. Do not reboot the payladCPU now. Proceed to the next section.


7.17 Boot payloadCPU from iSCSI Target

Follow the instructions in chapter 10 to add iSCSI root-path options to dhcpd-conf at boot time.

7.18 Boot payloadCPU from iSCSI Target To boot payloadCPU from iSCSi target:

1. Configure BIOS of payloadCPU. Here, cpu 2 is in slot 1. a. Reboot cpu 2 in slot 1 and press <F4> to enter BIOS console. b. BIOS: Device Management > iSCSI Configuration > iSCSI Initiator name

i. Enter iqn.2015-07.com.artesyn.initiator-mc1s1c2 c. BIOS: Device Management > iSCSI Configuration > Add an Attempt

i. Select MAC 02:01:00:10:02:xx and press <Enter> iSCSI Mode <Enabled> Enable DHCP [X] Get target info via DHCP [X] Authentication Type <None>

ii. Press <F10> to save changes then press <ESC> several times. iii. BIOS: Setup Utility

Boot > Boot Type <Dual Boot Type> Boot > EFI Device First <Enabled> Boot > PXE Boot capability <UEFI:IPv4>

iv. Save with <F10> key and enter the next BIOS console with <F4> key. v. Check if the first device on the list is now a SCSI device and boot from that

device.

7.19 Move shelfHost to another location To move shelfHost to another location:

1. Make cpu 2 in slot 15 the new shelfHost. SSF: MaxCore > Hardware Manager > Shelf Host Peripheral Assignment > Update Configuration > Active (bottom left) > Update > Apply Wait until the Apply action finished successfully before you click the Update button. This will reboot the whole system and bring up cpu 2 in slot 15 as the new shelfHost

2. Watch vgaCPU and allow the CPU to reboot two times. It will take a minute after the login prompt before the second reboot starts. Instead of JViewer you can also open a SoL session with sol 15 2. That will print the boot progress.

3. Login and check if the hostname shows the new location. It should be pcie7410-s15-c2. 4. Connect with PC browser to 172.26.1.2 and then login to SSF.


5. Repeat the steps of section 7.10 to install a new TFTP boot server on this CPU. 6. Follow sections 7.11 and 7.12 to validate the TFTP boot server.

7.20 Install payload OS on new iSCSI server The iSCSI server is still in the same location but its IP address changed after moving the shelfHost. This requires a new installation of the payload OSes.

1. Update DHCP server on cpu2 in slot 15. a. Disable the automatic generation of dhcpd.conf

mv /opt/bladeservices/etc/dhcpd.conf.tpl /opt/bladeservices/etc/dhcpd.conf.tpl.bak

b. Edit dhcpd.conf file. vi /opt/bladeservices/etc/dhcpd.conf

c. Add the following line to the end of the host mc1s1c2 definition (just before the closing bracket) option root-path "iscsi:172.27.1.111:::0:iqn.2015- 07.com.artesyn:target-s1c2";

d. Add the following line to the end of the host mc1s15c1 definition. For now just as a comment. option root-path "iscsi:172.27.1.111:::0:iqn.2015-07.com.artesyn: target-s15c1";

e. Save file and restart the dhcp service. There is something wrong with your edits when you get an error message. systemctl restart dhcpd.service

2. Open a SoL session to the payloadCPU, cpu 2 in slot 1, reboot and enter the BIOS with F4. 3. Boot over network from 01:02:00:10:02:xx 4. Login with root/root. 5. To connect to the network use ifconfig -a command to identify the interface name of the

interface with MAC 02:01:00:10:02:xx. Here, it is enp8s16f3. ifconfig enp8s16f3 172.27.1.112

6. Connect to iSCSI Target. iscsiadm -m discovery -t st -p 172.27.1.111:3260 iscsiadm -m node --targetname iqn.2015-07.com.artesyn:target-s1c2 --

portal "172.27.1.111:3260,1" --login

7. Install OS cd /opt/bladeservices/tools ./disk_install.sh -d iscsi:172.27.1.111:::0:iqn.2015-

07.com.artesyn:target-s1c2 -n iqn.2015-07.com.artesyn:initiator-s1c2 -s rootfs.tar.bz2


8. Reboot and enter BIOS with <F4> key. Check if the iSCSI shows up as boot device and boot. As an exercise you could now switch to cpu 1 in slot 15 and you install another OS on iqn.2015-07.com.artesyn:target-s15c1

7.21 Add network over the MaxCore Ethernet switch To add network over the MaxCore Ethernet Switch:

1. Open sshCPU session to shelfHost. 2. Edit file.

vi /opt/bladeservices/etc/dhcpd.conf.tpl Duplicate the whole subnet block at the end of the file and modify the duplicated block as follows: Replace 172.27.0.0 with 192.168.201.0 Replace 255.255.0.0 with 255.255.255.0 Replace #INTERFACE# with enp3s0f0; Replace #NEXT-SERVER# with 192.168.201.100; Replace #BASENET_HOSTS# with range 192.168.201.50 192.168.201.99;

3. Create file. vi /etc/sysconfig/network-scripts/ifcfg-enp3s0f0 DEVICE=enp3s0f0 ONBOOT=yes BOOTPROTO=static IPADDR=192.168.201.100 NETMASK=255.255.255.0

4. Reboot shelfHost. reboot

payloadCPUs including the CPU on a PCIE-9205 can now also use dhcp and tftp services over their network interface towards the MaxCore Ethernet switch. This is specifically helpful to perform a network installation of the PCIE-9205 CPU which has no other network connection at boot time. The name of the interface is either enp3s0f0 or enp4s0f0.

7.22 Conclusion You now have the shelfHost running on cpu 2 in slot 15 while your iSCSI server is still on cpu 1 in slot 1. Your payload CPUs are cpu 2 in slot 1 and cpu 1 in slot 15. The iSCSI server now benefits from the much better x4 PCIe Gen3 bandwidth since it is no longer located on the shelfHost. Thinking ahead, you can improve the iSCSI server performance even further when you plug an NVMe SSD card into a card slot and then connect it to PCIe port 2 of cpu 1 in slot 1.


8 Glossary BMC BMC stands for Baseboard Management Controller. It is similar to the BMC in an enterprise server. It is exclusively assigned to the shelfHost and provides a backdoor entry to MaxCore over standardized remote interfaces.

Baseboard

This is a component of the MaxCore chassis. It cannot be replaced in the field. It hosts the BMC and user programmable logic for configuring some aspects of the MaxCore platform.

managedObject This is an entity which can be monitored and controlled by the shelfHost.

networkCPU This is the CPU on a PCIE-9205 card. It runs the switch management software with a user interface provided by the SSF core software on the shelfHost. The networkCPU may also run user applications of any kind.

payloadCPU These CPUs run the application payload. The shelfHost controls BIOS configuration, remote power button and firmware updates, except the BIOS update.

PCIeLeaf Each PCIe device tree can have multiple PCIeLeafs. These are also referred as PCIe devices. Typical examples for such PCIe devices are I/O controllers and graphic adapters. PCIe devices with SR-IOV capabilities contain many PCIeLeafs. These are also referred to as virtual devices of virtualFunctions (VFs). VFs are important elements of MaxCore solution architecture.

PCIeRoot Each PCIe device tree has a single root. Typically it is a CPU. But be careful. There are CPUs within MaxCore which do not represent a PCIe Root for the PCIe backbone in MaxCore.

physicalFunction (PF) The PF is part of the enterprise server virtualization architecture. Each PCIeLeaf with SR-IOV has exactly one PF which is owned by the hypervisor in the case of an enterprise server. With few exceptions the MaxCore shelf Host is the owner of the PF. So, PF and VF can be assigned to different PCIeRoots.

shelfHost The shelfHost is the central management entity of each MaxCore with many CPUs. This term is not used for a single-CPU MaxCore.


SharpServer MaxCore PCIeRoot card with two CPUs with part number PCIE-7410. Four PCIe3x4 ports from the two CPUs.

SharpSwitch MaxCore PCIeLeaf card with Ethernet I/O, Ethernet switch, and single CPU with part number PCIE-9205. Four PCIe3x4 ports to the Ethernet switch.

systemHost The systemHost is a central entity which manages a stack of MaxCores, building one bigger system context. SSF running on the systemHost combines the resources of all connected MaxCore shelves within a single resource tree.

virtualFunction (VF) This term comes from classical virtualization in enterprise servers. In such scenarios it offers a physical PCIeLeaf many VFs which can be directly connected to many virtual machines within the central CPU. The term virtual is misleading in the MaxCore context because MaxCore offers VFs like physical devices to the payloadCPUs. So, be careful not to mix things up.


9 References The following are the documents are available at https://www.artesyn.com/computing/products/product/max-core

- MaxCore MC3000 Platform Quick Start Guide (6806800T89) - SharpServer PCIE-7410 Quick Start Guide (6806800T96) - SSF For MaxCore MC3000 Install and Use (6806800T81) - MaxCore MC3000 Platform Installation and Use (6806800T88) - iSCSI_MaxCore_MC3000

The following are the SCML files currently available at Artesyn portal: - SCML42368 - MC3000 Firmware Customer Package v1.2.0 - SCML42545 - MC3K_PCIE7410_CENTOS_7_2_SW_1_1_0_23 - SCML42546 - MC3K_PCIE9205_CENTOS_7_2_SW_1_1_0_23

https://www.artesyn.com/computing/products/product/max-core


10 Attachments

10.1 iSCSI initialization scripts

On the shelfHost, create the three files mentioned below. Replace the IP address in iscsi.tpl with the address of your iSCSI server. Leave it unchanged when you follow the instructions in the Tutorial section. After that, enable the service with the three commands provided at the end of this chapter. Finally reboot.

When the shelfHost comes up again, the /opt/bladeservice/etc/dhcpd.conf file contains an “option root-path …” line for each host. This option specifies the name of the iSCSI Target, which will be provided by the DHCP service to the dhclients.

/opt/bladeservices/etc/iscsi.tpl

#iscsi-svr=172.27.1.2 /opt/bladeservices/etc/init.d/iscsi.py

#!/usr/bin/python import sys import os def

get_iscsisvr_addr(dhcp_cfg_template='/opt/bladeservices/etc/iscsi.tpl'): basenet = "" file = open(dhcp_cfg_template,'r') for line in file: if line.startswith('#iscsi-svr'): vals=line.split('=') basenet = vals[1].strip() file.close() return basenet def add_option_rootpath(dhcp_cfg='/opt/bladeservices/etc/dhcpd.conf'): host = "" output = "" file = open(dhcp_cfg,'r') for line in file: if line.strip().startswith('option root-path'): return if line.strip().startswith('host'): host = line.strip().split(' ')[1] line=line.replace(line,line+' option root-path

"iscsi:'+get_iscsisvr_addr()+':::0:iqn.2015-07.com.artesyn:target-'+host+'";\n')

output += line


file.close() file = open(dhcp_cfg,'w') file.write(output) file.close() add_option_rootpath()

/etc/systemd/system/mc3k-iscsi.service

[Unit] Description=dhcpd.conf update for iSCSI After=pciemgmt.service Before=dhcpd.service [Service] Type=oneshot ExecStart=/opt/bladeservices/etc/init.d/iscsi.py [Install] WantedBy=multi-user.target

chmod +x /opt/bladeservices/etc/init.d/iscsi.py chmod +x /etc/systemd/system/mc3k-iscsi.service systemctl enable mc3k-iscsi.service systemctl start mc3k-iscsi.service systemctl restart dhcpd.service

getting started with ssf for maxcore™ mc3000 · pdf filethis document is applicable only...

Documents