Setup of NetApp gFiler Gateway Series

Installation Steps

•List of what the customer must provide•Physical hardware & cabling setup•Issues for clustering or stand alone setup•Configuration of storage subsystem – 9970V•Setup Console for initial Gateway access•Boot Gateway from flash compactdisk•Initial boot up of Gateway•Set environmental variables •Re-boot Gateway to initialize Luns or Lun x/Vol0•Special boot menu (called Floppy boot menu)•Configure Network interfaces on Gateway•Add Licenses for NFS•Install On-Tap OS 6.4.1 from Unix Admin (NFS)•(or) Install On-Tap OS 6.4.1 from Windows Admin (CIFS)•Configure NTP•Configure Time Zone to match network service•Confirm installation (some handy commands)

<

Racking and Cabling

• Unpack– Save license papers!

• Rack (3 man job!)• Cable up fibre to Array (cable all paths)• Power

– Both supplies & cluster ps

• Ethernet– Management port– Gig-e Ports

• Console port (for initial config & possible recovery)

What the customer must provide

• 2 IP addresses (Access & Management)– Gateway address– Netmasks– DNS servers

• 3 power feeds (110/220)• CIFS (Windows file server)

– Domain Name– WINS/DC ip address– Membership in Domain

• NFS– NIS server?– UNIX admin host IP (trusted host for administration)

Cluster Configuration

• Make sure both gFilers are cabled together and that they are both powered up.

• Set the environmental variables FIRST on both gFilers first before proceedings to install the Data ONTAP – setenv LUN-ownership low/high– setenv oprah-style true– setenv java? true– setenv jit-enable? true

Configure storage

• If direct connect – FCAL– Fabric Off– No LUN security

• If through a switch – Fabric On– Use LUN Security & Zoning(Each NetApp HBA should see only one copy of each LUN.See Appendix A for gFiler/Storage configurations)

• Present consecutive LUNs in order starting at 0 (Will not detect high numbered LUNs without lower ones for no-clustered configuration)

Boot Flash Upgrade

You may need to upgrade the Boot Flash if you are unable to discover LUNs from an HDS storage subsystem. See Appendix B for detailed instructions on how to upgrade the Boot Flash .

Configure NetApp ROM

• Attach to the NetApp console using the provided serial cable with an application like HyperTerm or SecureCRT– Settings

• Bits per Second: 9600• Data Bits: 8• Parity: None• Stop Bits: 1• Flow Control: No flow control

• Expect that the first boot will fail and yield an “OK>” prompt

Boot messagesIntel Open Firmware by FirmWorksCopyright 1995-2003 FirmWorks, Network Appliance. All Rights Reserved.Firmware release 4.2_i2Press Del to abort boot, Esc to skip POST

Memory size is 4096 MBTesting SIOTesting LCDProbing devicesMemory size: 4096MBTesting 512MBCompleteFinding image...Loading /pc-card:\x86\kernel\primary.krn

Starting Press CTRL-C for special boot menu................................................................................................................................................<1> MTL_MODULE(1): Initializing IB interconnect.NetApp Release 6.4X11: Fri Jan 31 08:02:30 PST 2003Copyright (c) 1992-2003 Network Appliance, Inc.Starting boot on Sun Feb 16 00:27:37 GMT 2003

Boot Post Messages(Make sure firmware is 4.2.i2 or later)

Loading kernel Image from boot flash

Special Boot menu(not used for first config)

Successful load of flash image

Boot messagesFound 3 LUNs on device 4a.0L0Found 3 LUNs on device 4b.0L0Found 3 LUNs on device 8a.0L0Found 3 LUNs on device 8b.0L0DBG: OFW LUN_ownership not setDBG: OFW LUN_ownership not setDBG: OFW LUN_ownership not setDBG: OFW LUN_ownership not setDBG: OFW LUN_ownership not setDBG: OFW LUN_ownership not setSun Feb 16 00:27:45 GMT [config.noPartnerLUNs:CRITICAL]: No LUNs were detectedor the partner; this node will be unable to takeover correctly

PANIC: OFW LUN_ownership not set in process dmgr_thread on release NetApp Releas

e 6.4X11 on Sun Feb 16 00:27:45 GMT 2003

version: NetApp Release 6.4X11: Fri Jan 31 08:02:30 PST 2003cc flags: 3O

DUMPCORE: START

DUMPCORE: END -- coredump *NOT* written.halt after panic during system initializationProgram terminatedok

We found 3 LUNSOn each path

However the NetAppWants to know who owns

These LUNs?

Since there are no LUNs with a bootable OS & no ownership is set we cant boot

So since we can’t boot acore dump occurs and yields an OK> prompt

Configure NetApp Prom

• Did it see any storage?– Link lights– Messages showing storage discovery “Found X LUNs on device

AA.bbb”• Set enviornmental variables

– OK> setenv oprah-style true– If it is the first cluster (or unclustered) ok> setenv LUN-

ownership low– If it is the second member of the cluster: ok> setenv LUN-

ownership high– OK> setenv java? true

OK> setenv jit-enable? true• Reboot

– Ok> boot

Initializing/zeroing out disks

• If there is data already on the disks the NetApp will see the drives as unavailable

• Need to enter special boot menu and zero out disks

• If the NetApp sees the disks as already zeroed out then the boot menu will skip to the first time configuration questions.

Example Using special boot menu to zero out disks

Intel Open Firmware by FirmWorksCopyright 1995-2003 FirmWorks, Network Appliance. All

Rights Reserved.Firmware release 4.2_i2Press Del to abort boot, Esc to skip POST

Memory size is 4096 MBTesting SIOTesting LCDProbing devicesMemory size: 4096MBTesting 512MBCompleteFinding image...Loading /pc-card:\x86\kernel\primary.krnStarting Press CTRL-C for special boot menu................................................................................................................................................Special boot options menu will be available.<1> MTL_MODULE(1): Initializing IB interconnect.

Press CTRL-C to go tospecial boot menu

CTRL-C recognizedWill boot into menu

Example Using special boot menu to zero out disks

NetApp Release 6.4X11: Fri Jan 31 08:02:30 PST 2003Copyright (c) 1992-2003 Network Appliance, Inc.Starting boot on Sun Feb 16 15:51:28 GMT 2003Found 3 LUNs on device 4a.0L0LUN Ownership using low rangeSun Feb 16 15:52:00 GMT [config.noPartnerLUNs:CRITICAL]:No LUNs were detected for the partner; this node will be unableto takeover correctlyThis boot is of OS version: NetApp Release 6.4X11.The last time this filer booted, it used OS version: <unknown>.The WAFL/RAID versions of the previously booted OS are unknown.If you choose a boot option other than Maintenance mode orInitialize disks, the file system of your filer might be upgradedto a new version of the OS.If you do not want to risk having your file system upgraded, chooseMaintenance mode or reboot using the correct OS version.

(1) Normal boot.(2) Boot without /etc/rc.(3) Change password.(4) Initialize all disks.(5) Maintenance mode boot.

We want the NetApp toInitialize the disks that itcan see. So we select 4

Example Using special boot menu to zero out disks

Selection (1-5)? 4Install a new file system? yThis will erase all the data on the disks, are you sure? yes

Sun Feb 16 15:52:27 GMT [raid.vol.disk.add.done:notice]: Addition of disk 4a.0L2 (S/N 045150060002) to volume vol0 has completed successfully

Sun Feb 16 15:52:28 GMT [config_async_0:notice]: Volume vol0 has been added to the system.

Booting the OS

• The gFiler will boot up and prompt you to answer questions to configure it.

• If you make a mistake it can be corrected afterwards (options setting <value>)

• On the first boot the NetApp will attempt to DHCP e0 (the onboard Ethernet). We will change the IP to the customer specified value.

First Boot (Configuring System Parameters)

ifconfig e0 mediatype autoConfiguring onboard ethernet e0.Contacting DHCP server.Ctrl-C to skip DHCP search ...Sun Feb 16 23:15:22 GMT [rc:info]: Contacting DHCP serverSun Feb 16 23:15:23 GMT [10/100-IV/e0:info]: Ethernet e0: Link up.Configuring e0 using IP 10.3.21.238 netmask 255.255.254.0.ifconfig e0 up mediatype auto netmask 255.255.254.0 10.3.21.238Sun Feb 16 23:15:34 GMT [rc:info]: DHCP config successful NetApp Release 6.4X11: Fri Jan 31 08:02:30 PST 2003 System ID: 0050390937 (); partner ID: <unknown> () System Serial Number: 1034194 () System Rev: C2 Backplane Part Number: 104-00013 Backplane Rev: C2 Backplane Serial Number: 1034194 slot 0: System Board Processors: 1 Memory Size: 3072 MB slot 0: 10/100 Ethernet Controller IV e0 MAC Address: 00:a0:98:01:12:15 (auto-100tx-fd-up) slot 0: NetApp ATA/IDE Adapter ata0a (1f0) ata0a.0 30MB slot 1: Remote Management Controller slot 4: FC Host Adapter 4a 3 Disks: 90.3GB slot 4: FC Host Adapter 4b 3 Disks: 90.3GB slot 5: Dual 10/100/1000 Ethernet Controller V e5a MAC Address: 00:07:e9:15:08:d0 (auto-unknown-cfg_down) e5b MAC Address: 00:07:e9:15:08:d1 (auto-unknown-cfg_down) slot 6: Interconnect HBA slot 7: NVRAM Memory Size: 256 MB slot 8: FC Host Adapter 8a 3 Disks: 90.3GB slot 8: FC Host Adapter 8b 3 Disks: 90.3GB slot 9: Gigabit Ethernet Controller IV e9 MAC Address: 00:02:b3:c1:e2:63 (auto-unknown-cfg_down)

First Boot (Configuring System Parameters)

(auto-unknown-cfg_down)Please enter the new hostname []: dexterDo you want to configure virtual network interfaces? [n]: nPlease enter the IP address for Network Interface e0 [10.3.21.238]: 10.3.20.134Please enter the netmask for Network Interface e0 [255.255.254.0]:Should interface e0 take over a partner IP address during failover? [n]:Please enter media type for e0 {100tx-fd, tp-fd, 100tx, tp, auto} [auto]: 100tx-fdPlease enter the IP address for Network Interface e5a []:Should interface e5a take over a partner IP address during failover? [n]:Please enter the IP address for Network Interface e5b []:Should interface e5b take over a partner IP address during failover? [n]:Please enter the IP address for Network Interface e9 []:Should interface e9 take over a partner IP address during failover? [n]:Please enter the IP address for Network Interface e10a []:Should interface e10a take over a partner IP address during failover? [n]:Please enter the IP address for Network Interface e10b []:Should interface e10b take over a partner IP address during failover? [n]:Please enter the IP address for Network Interface e11 []:Should interface e11 take over a partner IP address during failover? [n]:Would you like to continue setup through the web interface? [n]:Please enter the name or IP address of the default gateway: 10.3.20.1 The administration host is given root access to the filer's /etc files for system administration. To allow /etc root access to all NFS clients enter RETURN below.Please enter the name or IP address of the administration host: 10.3.20.91Where is the filer located? []: HDS TRC San Diego Lab (Row 2, Rack0)Do you want to run DNS resolver? [n]: yPlease enter DNS domain name [hds.com]: trc.hds.comYou may enter up to 3 nameserversPlease enter the IP address for first nameserver [10.1.7.34]: 10.3.20.215Do you want another nameserver? [y]: yPlease enter the IP address for alternate nameserver [10.1.7.33]: 140.243.186.22

Do you want another nameserver? [n]:Do you want to run NIS client? [n]:

First Boot (Configuring System Parameters)

This system will send event messages and weekly reports to Network Appliance Technical Support. To disable this feature, enter "options autosupport.support.enable off" within 24 hours.

Enabling Autosupport can significantly speed problem determination and resolution should a problem occur on your system. For further information on Autosupport, please see:. http://now.netapp.com/autosupport/

Press the return key to continue.

The remote management controller (RMC) extends autosupport by sending additional system event alerts. Your autosupport settings are used for sending these alerts via email over the RMC LAN interface.Would you like to configure the RMC LAN interface [y]: yThe MAC address for the dedicated RMC LAN interface is 00:30:d3:06:29:7e.Would you like to enable DHCP on the RMC LAN interface [y]: nPlease enter the IP address for the RMC []: 10.3.20.135Please enter the netmask for the RMC []: 255.255.254.0Please enter the IP address for the RMC gateway []: 10.3.20.1Please enter media type for the RMC {100tx-fd, tp-fd, 100tx, tp, auto} [auto]: 100tx-fd The mail host is required by your system to send RMC alerts and local autosupport email.Please enter the name or IP address of the mail host: 10.3.20.91You may use the autosupport options to configure alert destinations. Now unpack the tar file provided on the tar floppies into the root directory of the filer. When you are done, type "download" to install the boot image on the hard disk and "reboot" to begin using the new image.

Proceed to the administration host you specified earlier in the setup

First Boot (Configuring System Parameters)

Sun Feb 16 23:20:24 GMT [init_java:warning]: Java disabled: Missing /etc/java/rt131.jar.Sun Feb 16 23:20:24 GMT [init_zapi:warning]: Zapi RPC is disabled: java not enabledSun Feb 16 23:20:24 GMT [httpd_servlet:warning]: Java Virtual Machine not accessibleSun Feb 16 23:20:24 GMT [ems.engine.startReplay:info]: Replay begins for EMS boot-time cacheSun Feb 16 23:20:24 GMT [ems.engine.endReplay:info]: Replay ends for EMS boot-time cacheSun Feb 16 23:20:24 GMT [dyn_dev_qual_admin:info]: Firmware is up-to-date on all disk drives

Sun Feb 16 23:20:26 GMT [10/100-IV/e0:info]: Ethernet e0: Link being reconfigured.add net default: gateway 10.3.20.1There are 5 spare disks; you may want to use the vol command to create new volumes or add disks to the existing volume.Sun Feb 16 23:20:27 GMT [savecore_admin:info]: saving 102M to core.0.nz ("raid:there are no data or parity disks in process rc on release NetApp Release 6.4X11")Sun Feb 16 23:20:27 GMT [mgr.boot.floppy_done:info]: NetApp Release 6.4X11 floppy boot complete.Sun Feb 16 23:20:27 GMT [cf.fm.unexpectedAdapter:warning]: Warning: clustering is not licensed yet an interconnect adapter was found. NVRAM will be divided into two parts

until adapter is removedSun Feb 16 23:20:27 GMT [mgr.boot.reason_ok:notice]: System rebooted.Sun Feb 16 23:20:27 GMT [asup_main:warning]: minimal Autosupports unavailable.Could not read /etc/asup_content.confSun Feb 16 23:20:27 GMT [asup_main:warning]: minimal Autosupports unavailable.Could not read /etc/asup_content.confSun Feb 16 23:20:28 GMT [sysconfig.sysconfigtab.openFailed:notice]: sysconfig: table of valid configurations (/etc/sysconfigtab) is missing.

dexter> Sun Feb 16 23:20:28 GMT [console_login_mgr:info]: root logged in from consoleSun Feb 16 23:20:29 GMT [sysconfig.sysconfigtab.openFailed:notice]: sysconfig: table of valid configurations (/etc/sysconfigtab) is missing.

Sun Feb 16 23:20:31 GMT [savecore_admin:info]: core.0.nz processing completed.Sun Feb 16 23:20:31 GMT [savecore_admin:info]: Generating /etc/crash/mini-core.0Sun Feb 16 23:20:31 GMT [10/100-IV/e0:info]: Ethernet e0: Link up.Sun Feb 16 23:20:31 GMT [savecore_admin:info]: (/etc/crash/mini-core.0) generation completed.Sun Feb 16 23:20:41 GMT [rmc.connect:warning]: The remote management controllerin slot 1 is missing its LAN cable. Please connect it as soon as possible. dexter>

Add LicensesIn order to install the Data ONTAP software you will need to activateeither CIFS, NFS or HTTP (depending on the administration host) which requires a license.

dexter> license add QAPXIA nfs license has been installed. nfs enabled.dexter> Sun Feb 16 23:21:54 GMT [rc:notice]: nfs licensed

dexter> license add HHJNLA http license has been installed. Use "options httpd.enable" to enable http.dexter> Sun Feb 16 23:22:13 GMT [rc:notice]: http licensed

Update Data ONTAP software

• Even if it is already at 6.4 update the filer (no java)• Download code• Download release notes• Download install docs

Installing/Updating Data ONTAP software from UNIX

• Mount filer• Copy code to filer root• Extract tar file• Run install script• Unmount filer• From filer console run

“download” command• Reboot filer

ADMINHOST /># mount filer:/ /mnt

ADMINHOST /># cp 64_sysfiles_I.tar /mnt

ADMINHOST /># cd /mnt

ADMINHOST /mnt># tar –xvf 64_sysfiles_I.tar

ADMINHOST /mnt># ./install_netapp /mnt

(completes sucessfully)

ADMINHOST /mnt># cd /

ADMINHOST /># umount /mnt

Installing/Updating Data ONTAP software from Windows 2K

After successfully installing the CIFS license, map the C$ drive from the gFiler to a drive letter on the administration host.

Load the NetApp Data ONTAP CD that came with the gFiler on the administration host and change directories on the CD to “\ONTAP\6_4_1\X86”. Execute the “setup.exe” command making sure you are extracting the files into the “/etc” directory under \\<filer_name>\C$.

From the gFiler console run the command “download”. Wait until the “download” command finished then reboot the gFiler.

Note: If you are unable or are having problems adding the gFiler into the Windows domain choose “Windows Workgroup authentication using the filer’s user accounts”, option 2 in “cifs setup”. You can rerun “cifs setup” again to add the gFiler to the domain at a later date.

Configure NTP

Time synchronization is Important!

• 2 options NTP or rdate. NTP is preferred, but it is important that Windows Doman Controllers, NFS clients, and Filer all be synchronized to a common time source.

From filer console:

>options set timed.proto ntp

>options set timed.servers 10.3.20.1

>options set timed.enable on

Set Time Zone• Easily set by console (or web)>timezone America/Los_AngelesOr for the rest of the US (Not the whole list)America/PhoenixAmerica/IndianaAmerica/IndianapolisAmerica/ChicagoAmerica/Los_AngelesAmerica/DenverAmerica/DetroitAmerica/Mexico_CityAmerica/MontrealAmerica/VancouverAmerica/New_York

Configure CIFSdeedee*> cifs setup Enable CIFS access to the filer by a Windows(tm) PC Your filer is currently only visible to PCs on the same net.

Do you wish to make the system visible via WINS? [no]: This filer is currently configured as a Multiprotocol filer.

Do you want to configure this filer as a NTFS-only filer? [no]:

The default name of this filer will be 'DEEDEE'. Do you want to modify this name? [no]: CIFS supports three types of user authentication: 1. Windows Domain authentication. 2. Windows Workgroup authentication using the filer's user accounts.

3. /etc/passwd and/or NIS/LDAP based authentication.

What type of authentication will this filer use? [1]: 1 The filer will use Windows Domain authentication.

Configure CIFSEnter the Windows Domain for the filer []:trc.com TRC.COM is a Windows 2000(tm) domain. In order to create this filer's domain account, you must supply the name and password of an administrator account with sufficient privilege to add the filer to the TRC.COM domain.Please enter the Windows 2000 user [Administrator@TRC.COM]: administrator@trc.comPassword for administrator@trc.com:CIFS - Logged in as administrator@trc.com.CIFS - Updating filer account 'cn=deedee,cn=computers,dc=trc,dc=com'CIFS - Connecting to domain controller.Sun Feb 23 09:53:53 PST [rc:info]: Starting DC address discovery for TRC.Sun Feb 23 09:53:58 PST [rc:info]: Found 2 addresses using DNS site query (Default-First-Site-Name).Sun Feb 23 09:53:58 PST [rc:info]: Found 3 addresses using generic DNS query.Sun Feb 23 09:53:58 PST [rc:info]: DC address discovery for TRC complete. 2 unique addresses found.Sun Feb 23 09:53:58 PST [rc:info]: Connection with \\W2KDC01 establishedWelcome to the TRC (TRC.COM) Windows 2000(tm) domain.

Configure CIFS

CIFS local server is running.

deedee*> Sun Feb 23 09:5 4:06 PST [nbns_timer:info]: All CIFS name registrations

complete for local server

Confirm Available storage

dexter> dfFilesystem kbytes used avail capacity Mounted on/vol/vol0/ 22185864 105868 22079996 0% /vol/vol0//vol/vol0/.snapshot 5546464 0 5546464 0% /vol/vol0/.snapshot

dexter> storage show

Slot: 8aDescription: Fibre Channel Host Adapter 8a (Dual-channel, QLogic 2312 rev. 2)Firmware Rev: 3.1.15PCI Bus Width: 64-bitPCI Clock Speed: 33 MHzFC Node Name: 2:000:00e08b:09911dCacheline Size: 16FC Packet Size: 2048Link Data Rate: 1 GbitSRAM Parity: YesExternal GBIC: NoState: EnabledIn Use: YesRedundant: Yes

Slot: 8bDescription: Fibre Channel Host Adapter 8b (Dual-channel, QLogic 2312 rev. 2)Firmware Rev: 3.1.15PCI Bus Width: 64-bitPCI Clock Speed: 33 MHzFC Node Name: 2:001:00e08b:29911dCacheline Size: 16FC Packet Size: 2048Link Data Rate: 1 GbitSRAM Parity: YesExternal GBIC: NoState: EnabledIn Use: YesRedundant: Yes

Confirm Available storageSlot: 4aDescription: Fibre Channel Host Adapter 4a (Dual-channel, QLogic 2312 rev. 2)Firmware Rev: 3.1.15PCI Bus Width: 64-bitPCI Clock Speed: 33 MHzFC Node Name: 2:000:00e08b:097d03Cacheline Size: 16FC Packet Size: 2048Link Data Rate: 1 GbitSRAM Parity: YesExternal GBIC: NoState: EnabledIn Use: YesRedundant: Yes

Slot: 4bDescription: Fibre Channel Host Adapter 4b (Dual-channel, QLogic 2312 rev. 2)Firmware Rev: 3.1.15PCI Bus Width: 64-bitPCI Clock Speed: 33 MHzFC Node Name: 2:001:00e08b:297d03Cacheline Size: 16FC Packet Size: 2048Link Data Rate: 1 GbitSRAM Parity: YesExternal GBIC: NoState: EnabledIn Use: YesRedundant: Yes

DISK SHELF BAY SERIAL VENDOR MODEL REV--------------------- --------- ---------------- -------- ---------- ----4a.0L0 - - 045150060000 HITACHI OPEN-L 21038a.0L1 - - 045150060001 HITACHI OPEN-L 21038a.0L2 - - 045150060002 HITACHI OPEN-L 21038b.0L0 - - 045150060003 HITACHI OPEN-L 21034b.0L1 - - 045150060004 HITACHI OPEN-L 21034b.0L2 - - 045150060005 HITACHI OPEN-L 2103

Appendix B – gFiler/Storage Connectivity Cluster / Direct-Attach 9570V configuration

0A 0B

Filer11A 2B

Filer21A 2B

1A 1B

Up to 84/168 LUNs on each port (84 per node)

LUN_ownership = high LUN_ownership = low

Host Group 00Port 0A

Host Group 00Port 0B

Host Group 00Port 1A

Host Group 00Port 1B

Controller 0 Controller 19570V

High

High

High

High

Low

Low

Low

Low

Appendix B – Upgrading the Boot FlashUpgrading the Boot Flash image for NetApp gFilers Gateways

1) Hitachi Data Systems gets kernel code via FTP from Network Appliance:a) ftp ftp.netapp.comb) Login: anonymousc) Passwd: [your email address]d) bin (change to binary mode)e) cd frm-ntap/HDSf) get netapp_6.4.1-x86g) Save it to some easily-remembered directory on your PC

2) Remove Compact Flash card from gFiler

3) Insert the Compact Flash card into a Compact Flash reader/writer. This piece of equipment can be purchased for about $20.00 from a computer supply store like Fry’s, Circuit City or Best Buy.

Appendix B (continued)

4) CD to "X86" folder on the Compact Flash card

5) CD to "KERNEL" folder

6) Remove file called "PRIMARY.KRN"

7) Copy "netapp_6.4.1-x86" from wherever you stored it on your PC to the KERNEL folder and rename it "PRIMARY.KRN"

8) Insert the Compact Flash card into gFiler, button it up, and reboot.