Cisco SFS 3001 Multifabric Server Switch

Quick Start Guide (Topspin 90 Quick Start Guide)

Release 2.1.0

Part Number: OL-7712-01

Product ID: 140252

Copyright ?? 2003-2004 Topspin Communications, Inc. All rights reserved.

The Topspin Switched Computing System, Topspin Host Channel Adapter, Topspin Element Manager, and collateral software programs and documentation are subject to and made available only pursuant to the license agreement signed by you and Topspin, Communications, Inc. (or if no signed license agreement exists, the license agreement included with your original media) and may only be used in accordance with the terms of that agreement. Making copies, modifications, or compilation works of the software except as specifically allowed in that agreement is prohibited by law and constitutes a punishable violation of the law. This documentation may not be copied, modified or reduced to any electronic or machine-readable form without Topspin Communication, Inc.'s prior written consent.

As defined in FAR section 2.101, DFAR section 252.227-7014(a)(1) and DFAR section 252.227-7014(a)(5) or otherwise, the collateral software programs provided to you are ???commercial items,??? ???commercial computer software??? and/or ???commercial computer software documentation.??? Consistent with DFAR section 227.7202, FAR section 12.212 and other sections, any use, modification, reproduction, release, performance, display, disclosure or distribution thereof by or for the U.S. Government shall be governed solely by the terms of the license agreement and shall be prohibited except to the extent expressly permitted by the terms of that agreement. Any technical data provided that is not covered by the above provisions shall be deemed ???technical data-commercial items??? pursuant to DFAR section 227.7015(a) or equivalent. Any use, modification, reproduction, release, performance, display or disclosure of such technical data shall be governed by the terms of DFAR section 227.7015(b) or equivalent.

This documentation may include technical inaccuracies or typographical errors and is subject to correction and other revision without notice. TOPSPIN PROVIDES THIS DOCUMENTATION "AS IS" AND WITHOUT ANY

WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states or jurisdictions do not allow disclaimer of express or implied warranties in certain transactions; therefore, this statement may not apply to you.

??Copyright 2003-2004, Topspin Communications, Inc. All rights reserved. Topspin is a registered trademark and the Topspin logo, TopspinOS, and Topspin Switched Computing System are trademarks of Topspin Communications, Inc. Other company, product, or service names are the property of their respective owners.

Document Version: 2.1.0

Part Number: OL-7712-01

Product ID: 140252

Printed in the United States of America.

iii

Contents

v

Regulatory Notices

FCC Statement

This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to Part 15 of the FCC Rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment. This equipment generates, uses, and can radiate radio frequency energy and, if not installed and used in accordance with the instruction manual, may cause harmful interference to radio communications. Operation of this equipment in a residential area is likely to cause harmful interference in which case the user will be required to correct the interference at his own expense.

Safety Information

??????Electrical Cautions??? on page v

??????General Cautions??? on page vii

??????Laser Cautions??? on page vii

Electrical Cautions

Use only the power cable provided with your Topspin system. Inspect the power cord and determine if it provides the proper plug and is appropriately certified for use with your electrical system. Discard the cord if it is inappropriate for your country???s electrical system and obtain the proper cord, as required by your national electrical codes or ordinances.

CAUTION: Grounding is supplied by the ground-prong on the 3-prong power cable. Do not attach a separate ground cable. Do not use adapter plugs. Do not remove the ground prong from the cable.

vi

Ensure the ground connection on the power supply is correct and functioning before applying power to the Topspin system.

CAUTION: Always ground yourself before touching any internal Topspin system components to avoid damage from electrostatic discharge (ESD).

CAUTION: The Topspin system has two power cables. The operator must disconnect all power cables before attempting to remove the system from the rack.

CAUTION: Remove power cables by grasping the cable connector and pulling straight out.If the chassis comes equipped with a power-plug retainer, move the bail wire used to retain the plug to the side before attempting to remove the power plug. Replace power-supply cables be grasping the cable connector, align the connector with the power jack, and insert the connector straight into the jack.If the chassis comes equipped with a power-plug retainer, move the bail wire back into place to keep the power plug firmly in place.

CAUTION: The Topspin system contains lithium batteries. Do not attempt to replace or discard these batteries. The batteries may only be serviced by Topspin service personnel.

CAUTION: Observe and follow service markings. Do not service the Topspin system, except as explained in the Topspin documentation supplied with your system. Opening the chassis or removing the enclosure cover exposes you to electrical shock and may damage system components.

If one or more of the following situations occurs, remove power to the Topspin system chassis, and contact your Topspin support representative:

???The power cable or power plug is damaged.

???The system has been exposed to water.

???The system has been dropped or damaged in any way.

???The system does not operate correctly after following the installation instructions.

CAUTION: You must ensure the operating environment has adequate air circulation. Keep the Topspin in a cool, well-ventilated room. Do not block cooling vents. Install filler panels into all unused slots.

CAUTION: Use only Topspin-approved gateway expansion modules, controller cards, fans, and power supply modules.

vii

CAUTION: Never place your hand inside an empty card or module bay. You should never have cause to place a hand anywhere inside the Topspin chassis. Unused card and module bays should always have a Topspin cover over the bay to ensure proper safety, ventilation, and cooling.

General Cautions

CAUTION: No user is authorized to remove the Topspin system enclosure cover. The internal chassis contains no user-serviceable components. Removing the enclosure cover voids your warranty. See the warranty card for further details regarding the servicing of your chassis.

CAUTION: Opening and/or modifying expansion and system modules may void your warranty. Refer to your warranty to determine access restrictions.

In general:

???Do not spill food or liquids on your system components.

???Protect your Topspin system from sudden power-surges and interruptions by using a surge suppressor, line conditioner, or uninterruptable power supply (UPS).

???Place cables appropriately so that they do not obstruct any egress within the data center and they do not block any ventilation inlets or outlets.

???Rack mount the Topspin chassis according to the rack manufacturer recommendations.

Laser Cautions

CAUTION: Only qualified service technicians should repair laser devices.

CAUTION: Only open, modify, or service laser devices as specified in the manufacturer documentation.

CAUTION: Use of controls or adjustments or performance of procedures other than those specified herein may result in hazardous laser radiation exposure.

CAUTION: In order to ensure eye safety, only the fiber-optic transceivers indicated in the following table may be used.

viii

Laser Specs

Table III-1:

* Standards used: CDRH - 21CFR Chapter I ??1040.10; IEC - IEC60825-1:1994

Contact Information

Table III-2: Customer Contact Information

1

1

Topspin 90 Quick Start

This document describes the basic steps required to install and configure the Topspin 90 system.

For more detailed information about the Topspin 90 server switch, we recommend that you have the

Topspin 90 Hardware Guide available.

??????Prepare the Site??? on page 1

??????Install the Topspin 90 in an Equipment Rack??? on page 2

??????Install Expansion Modules (Optional)??? on page 5

??????Cable the Switch??? on page 5

??????Configure Basic Connectivity??? on page 6

??????Connect and Configure InfiniBand Hosts??? on page 8

??????Install the Element Manager (GUI)??? on page 10

??????Launch Chassis Manager??? on page 11

??????Configure Additional Settings??? on page 12

??????(Optional) Configure the Ethernet Gateway??? on page 13

??????Configure the Fibre Channel Gateway (Optional)??? on page 15

Prepare the Site

1.Read the cautionary statements in ???Safety Information??? on page v.

2.Unpack the Topspin 90 package, using the packing list provided.

3.Register the Topspin system online. Go to http://topspin.com/support/.

4.Prepare a management workstation, such as a PC or laptop running a terminal program, and a straight-through M/F DB-9 serial cable (included).

2

5.Prepare the physical environment for the Topspin 90.

a.Make sure that you have an approved grounding wrist-strap.

b.Make sure the system has the right cables and sufficient ventilation.

c.Unpack the Topspin 90 package, using the packing list provided.

d.Verify that you have all of the items on the Requirements list. See ???Requirements??? on page 2.

Install the Topspin 90 in an Equipment Rack

There are two different methods that can be used to rack the Topspin 90. This guide provides instructions for one method. To view instructions on the alternative method, refer to the Topspin 90 Hardware Guide.

Requirements

In addition to the accessories provided with the switch, you should have:

???Phillips screw driver

???Eight screws and any associated mounting clips to secure the brackets to your rack (2 for each rail of the rack)

???Two people are recommended to perform the install

Prepare for the Installation

1.Open the shipping package, if you have not already done so. Remove the chassis, rack brackets, CD-ROM, parts bag, and documentation.

2.Place the chassis on a secure, clean surface.

3.Open the plastic bag containing mounting parts.

4.Check the slot in the rack for sufficient clearance.

5.Determine the direction that the switch will be installed and justified in the rack. The direction that you mount the switch will determine the direction that the brackets will need to be attached.

Rack Mount Installation

NOTE: There are two different ways to perform the rack installation of the Topspin 90. In this procedure, you will attach the rails to the rack before inserting the switch. This procedure is slightly easier to perform if you only have one person. Refer to the Topspin 90 Hardware Guide to view the second installation method.

3

1.Take one connected pair of rack brackets and separate them. Each side arrives assembled to its counterpart, but should be separated before attaching to the switch when using this method of installation.

Separate Brackets

Figure 1-1: Assembled Rack Bracket Pair

2.Attach the rack bracket that has screw holes to the side of the chassis with the flange facing away from the switch, as shown in Figure 1-2.

The standard method is to face the flange toward the front of the chassis. However, you can also mount the flange toward the back if you want to mount the switch backward in the chassis (service-side forward).

Flange

Figure 1-2: Attaching One Rail to Server Switch Chassis

3.Repeat step 1 and step 2 for the other side of the switch chassis.

NOTE: The two counterparts to these sliding rails (without screw holes) should still be unattached.

4.Check your rack for clearance for the switch. The switch can be installed either directly on top of another device (using the rubber feet, which are provided), or be suspended from the rack brackets.

4

5.Attach the remaining two rail brackets to your rack.

a.Orient a bracket toward the back of the rack with the flange facing away from the rack. The flange should go around the outside of the rack post, as shown in, as shown in Figure 1-3.

NOTE: If you are mounting the switch backward, the rack bracket should be installed toward the front of the rack.

b.Secure the brackets with your screws through the back of the rack posts, as shown in Figure 1-4.

c.Repeat on both sides of the rack.

Figure 1-3: Holding Rack Bracket Against Inside of Rack

Figure 1-4: Secure Rack Brackets to Rack with Screws

6.Lift the switch unit and align the bracket on the switch with the bracket on the rack before sliding the two sets of brackets together.

7.Carefully push the switch unit into the rack.

5

If the brackets do not slide easily, the alignment may be off. Pull the switch back toward you and realign the brackets.

Figure 1-5: Sliding the Switch Into the Rack

8.Secure the switch with your screws through the front of the rack, as show in Figure 1-6.

Figure 1-6: Secure Switch with Screws through the Front of the Rack

Install Expansion Modules (Optional)

1.Install any optional Fibre Channel or Ethernet Gateway modules.

2.Install all hardware components before the Topspin 90 is powered on.

3.A gateway module or blanking panel must always be installed.

Cable the Switch

Connect Management Ports

1.Connect one end of the RJ-45 console cable to the InfiniBand chassis serial console port.

2.Connect the other end of the serial cable to your terminal server or management workstation.

3.Connect an Ethernet cable to the Ethernet port (bottom port).

4.Check the LEDs on the Management ports. The link status should be lit.

6

Connect InfiniBand Cables

1.Plug InfiniBand Cables from the InfiniBand-enabled host to the InfiniBand switch ports. This can also be done in a later step. Refer to step 5.

2.Plug InfiniBand cables from the Topspin 90 to any additional InfiniBand switches that you want to connect.

(Optional) Connect to Ethernet Devices

1.Connect the optional Ethernet gateway to workstations, routers or switches. The Ethernet gateway auto-detects what kind of cable is connected and can use either straight-through or cross-over UTP cabling.

(Optional) Connect to Fibre Channel Devices

1.Connect the optional Fibre Channel gateway to Fibre Channel storage arrays or switches with fibre cabling. Standard LC pluggable connectors are required.

Connect the Power Cable

1.Insert the power cords to the power jacks on the rear of the chassis.

2.Plug the other end of each AC power cable into an outlet operating at 90 - 132 VAC or 180-264V AC auto-ranging, 47-63Hz.

3.Check the System LED on the chassis to see if it is green. It will be blinking yellow/green until the power-on test is complete.

Configure Basic Connectivity

1.Configure an IP address for the Ethernet Management port.

It is necessary to configure connectivity to the out-of-box Ethernet management port in order to use Telnet, the Web GUI, or the Java GUI. The system is not pre-configured when it ships from the factory.

IP addresses can be assigned statically (which is the default), or dynamically assigned using DHCP.

??????Assigning an Address Statically??? on page 6

??????Assigning an Address Dynamically (DHCP)??? on page 7

Assigning an Address Statically

Refer to the static example below:

a.When the system has completed booting, press <Enter> several times to display the CLI prompt.

Login:

b.Enter the user name and password. The default user name is super, and the default password is super.

Login: super

Password: super

Topspin-90>

7

c.Enter the enable command.

Topspin-90> enable

Topspin-90#

d.Enter the configure command.

Topspin-90# configure

Topspin-90(config)#

e.Enter the interface mgmt-ethernet command.

Topspin-90(config)# interface mgmt-ethernet

f.Enter the IP address of the management port followed by the netmask.

Topspin-90(config-if-mgmt-ethernet)# ip address 10.10.0.22 255.255.255.0

g.Set the default gateway for the management port. For example:

Topspin-90(config-if-mgmt-ethernet)# gateway 10.10.0.1

h.Enable the management port by entering the no shutdown command.

Topspin-90(config-if-mgmt-ethernet)# no shutdown

i.Save the configuration to preserve it between reboots.

Topspin-90(config-if-mgmt-ethernet)# exit

Topspin-90(config)# exit

Topspin-90# copy running-config startup-config

Assigning an Address Dynamically (DHCP)

The Topspin 90 supports dhcp to assign the IP address.

a.When the system has completed booting, press <Enter> several times to display the CLI prompt.

Login:

b.Enter the user name and password. The default user name is super, and the default password is super.

Login: super

Password: super

Topspin-90>

c.Enter the enable command.

Topspin-90> enable

Topspin-90#

d.Enter the configure command.

Topspin-90# configure

Topspin-90(config)#

e.Enter the interface mgmt-ethernet command.

Topspin-90(config)# interface mgmt-ethernet

8

f.Enter the option dhcp command. The chosen method will be persistent across module reboots.

Topspin-90(config-if-mgmt-ethernet)# option dhcp

g.Enable the management port by entering the no shutdown command.

Topspin-90(config-if-mgmt-ethernet)# no shutdown

h.Save the configuration to preserve it between reboots.

Topspin-90(config-if-mgmt-ethernet)# exit

Topspin-90(config)# exit

Topspin-90# copy running-config startup-config

i.Use the show interface mgmt-ethernet command to view your IP address:.

Topspin-90# show interface mgmt-ethernet

===============================================================

Mgmt-Ethernet Information

===============================================================

mac-address : 00:05:ad:00:15:19 auto-negotiate : enabled

admin-status : up

ip-addr : 10.3.102.66 mask : 255.255.0.0

gateway-addr : 10.3.0.1 addr-option : static

Topspin-90#

Connect and Configure InfiniBand Hosts

This section describes a brief overview for configuring your InfiniBand hosts. For more detailed instructions, please see the HCA/Host Side Drivers Users Guide for Linux, which is included with your HCA(s).

If you are installing windows drivers, refer to the HCA Users Guide for Windows.

1.Install the HCA(s).

a.Power down the host system following proper grounding procedures.

b.Unpack the HCA card and insert it into an open 64-bit PCI or PCI-X slot. For more information, see the Host Channel Adapter Installation Guide.

2.Power on the host system.

3.Install the HCA driver(s).

An integrated suite of drivers is provided, including IPoIB, SRP, SDP, uDAPL and MPI.

a.Make sure your Linux kernel is supported. Refer to the Topspin Web site for a list of supported versions.

b.Insert the supplied CD-ROM into the CD-ROM drive. The CD-ROM is provided in the package with your HCA.

c.(Optional) Mount the file-system for the CD-ROM drive if your platform does not auto-mount devices.

# mount /mnt/cdrom

9

d.Open a terminal or graphic window and access the top-level directory on the CD-ROM

e.Execute the command ./tsinstall in a terminal window. This program automatically detects the available kernel and installs the appropriate drivers as RPM packages. By default, topspin files are saved in the /usr/local/topspin directory.

#cdmount ./tsinstall

4.Configure the HCA drivers.

a.Configure the IPoIB driver by assigning an IP address to interface ib0 and/or ib1, which corresponds to the two HCA ports. IPoIB drivers load when the interfaces are used for the first time.

To assign ib interfaces, enter ifconfig ib# ip-address netmask netmask:

Example of One HCA

#ifconfig ib0 192.168.0.0 netmask 255.255.255.0

#

# ifconfig ib1 192.168.0.1 netmask 255.255.255.0

#

Example of Two HCAs

#ifconfig ib0 192.168.0.0 netmask 255.255.255.0

#ifconfig ib1 192.168.0.1 netmask 255.255.255.0

#ifconfig ib2 192.168.0.2 netmask 255.255.255.0

#ifconfig ib3 192.168.0.3 netmask 255.255.255.0

a.(Optional) Configure MPI.

???Make sure that your connections to the HCA uses port 1 of the first HCA. MPI requires the use of HCA 1, port 1.

???Setup rsh or ssh on two nodes so that you can run commands between a remote and local node without entering a login or password. This is required to use the MPI protocol.

???Add paths to your environment PATH: /usr/local/topspin/mpi/mpich/bin /usr/local/topspin/bin

or

???Add the paths for all users by adding the following line to your /etc/profile.d script: export PATH=$PATH:/usr/local/topspin/mpi/mpich/bin:/usr/local/topspin/bin

b.Configure all other drivers, including SDP.

Refer to the Host Channel Adapter Installation Guide for instructions.

10

5.Plug InfiniBand Cables from the host to the InfiniBand switch.

a. To plug in an InfiniBand cable, push the connector into the interface until you hear/feel a click.

Figure 1-7: Fully Installed IB Cable with Pinch Connector

NOTE: If your host does not provide an ample amount of free space around a given IB port, double-check that your IB cable connector engages fully. Wiggle your connector back and forth to be sure that both sides of the connector have locked firmly into place.

b.To remove a cable with a pinch connector, pinch both sides of the back of the connector and pull the connector away from the port.

Figure 1-8: Removing a Pinch Connector

Install the Element Manager (GUI)

1.Check that you have sufficient system resources. You will need the following:

???32 MB free RAM

???50 MB disk space + 50MB additional temporary space during installation

11

???300 Mhz processor

???800x600 screen resolution, 16bit color depth

2.Insert the Topspin CD-ROM into the CD-ROM drive of a server that has network access to a Topspin management port.

3.Access the top folder of the CD-ROM. The GUI can be found in the following location on the CD-ROM.

???Linux/install_linux_ia64.bin

???Linux/install_linux_x86.bin

???Windows/install_x86.exe

???Solaris/install_solaris_sparc.bin

4.Run the installation wizard:

a.On Windows, double-click install.exe.

b.On Solaris and Linux systems, enter sh ./install_yourOS.bin.

5.Follow the on-screen instructions.

??? Enter location and preference information as requested by the installation program.

6.Start the Element Manager.

a.On Linux platforms, change to the installation directory and run the Element Manager with

./TopspinEM.

b.On Windows platforms, click

Programs -> Topspin Element Manager -> TopspinEM.

c.When prompted to enter a device name, enter the DNS host name or IP address of the Topspin 90, and community string for the unrestricted user. The default community string for the unrestricted user is secret.

Launch Chassis Manager

To launch Chassis Manager, perform the following steps:

1.Launch your Web browser.

Chassis Manager supports the following browsers

???Microsoft Internet Explorer version 6

???Netscape Navigator version 6

???Mozilla version 1.4

2.Type the Management IP address of your Server Switch in the address field of your browser and press Enter. (You configured the IP address in Step f.)

Figure 1-9: Web Browser Window

A log-in window opens.

3.Enter your Server Switch user name and password in the log-in window and click the OK button.

12

4.Chassis Manager loads in your browser window.

Figure 1-10: Chassis Manager Opening Display for the Topspin 90 Server Switch

Configure Additional Settings

You can now fully manage the device with CLI over Telnet/SSH or the GUI.

1.Verify that you are running the latest version of software.

a.Refer to the Topspin support Web site: http://topspin.com/support/

b.Use the show version command in the CLI to show the current firmware version of your switch:

Login: super

Password: xxxxx

Topspin-90> enable Topspin-90# show version

================================================================================

System Version Information

================================================================================

system-version : Topspin-90 TopspinOS 2.1.0 releng #117 06/18/2004 1

7:48:38

contact : support@topspin.com name : Topspin-90

location : 515 Ellis Street, Mountain View, CA 94043 up-time : 0(d):19(h):25(m):38(s)

last-change : Wed Jun 23 06:02:11 2004 last-config-save : Wed Jun 23 00:55:34 2004

action : none

result : none

Topspin-90#

2.(Optional) Configure time settings.

The Topspin 90 has internal clocks that are set at the factory.

You can also configure this manually or via NTP servers. If you do not want to configure the clock at this time, skip this step.

??? In the Element Manager GUI, go to Maintenance-->Time. Enter your settings.

13

???In the CLI, enter the configuration mode and enter the ntp command with the IP address of the NTP server.

Example

Topspin-90(config)# ntp server-one 10.2.0.50

3.Configure administrators and privilege levels. See Creating User Accounts in the User Guide.

(Optional) Configure the Ethernet Gateway

Configure the Ethernet Gateway if you have installed an Ethernet card into the chassis. Refer to the

Ethernet Gateway User Guide for more information.

There are three common topologies that can be deployed with Layer 2 Bridging:

???Bridging One Subnet.

???Action: Bridge one InfiniBand subnet to one IP subnet.

???Bridging Multiple Subnets

???Action: Bridge multiple IB and IP subnets across one gateway.

???Bridging Multiple Subnets across a single port

???Action: Bridge using VLAN tags and multiple subnets.

The following procedure describes the first topology: (bridging one subnet); this topology includes the following main steps:

???creating a bridge-group for a non-tagged Ethernet port.

???creating a bridge-group that uses the default pkey.

For information regarding configuring bridge-ports for tagged Ethernet ports, or for configuring more complex topologies, refer to the Ethernet Gateway User Guide.

Layer 2 Bridging Procedure

Refer to the Ethernet Gateway User Guide for more options and more detailed steps.

1.Enable the Ethernet gateway

Enter the configure command in global-configuration mode.

Example

Topspin-90# configure

Topspin-90(config)#

2.Create a bridge-group.

Enter bridge-group bridge-group# subnet-prefix

Leave the actual subnet-prefix address and length blank to auto-discover all available subnets.

Example

Topspin-90(config)#bridge-group 21 subnet-prefix

Topspin-90(config)#

3.If you are using DHCP, enable broadcast forwarding.

14

Use the broadcast-forwarding command.

Example

Topspin-90(config)# bridge-group 21 broadcast-forwarding

Topspin-90(config)#

or

If you are not using DHCP, configure the next InfiniBand hop. Use the ib-next-hop command.

Example

Topspin-90(config)# bridge-group 1 ib-next-hop

Topspin-90(config)#

4.Specify the Ethernet Gateway to which you want to assign the new bridge-group.

???Enter interface ethernet slot/port

???Specify the new bridge-group

Example

Topspin-90(config)# interface ethernet 2/1

Topspin-90(config-if-ether-2/1)# bridge-group 21 pkey 00:01

5.Specify the internal interface gateway to which you want to assign the new bridge-group.

The interface gateway number is the same as the physical slot number of the Ethernet gateway that you are configuring (in this example, the number is 2).

???Enter interface gateway physical slot#.

???Specify the new bridge group.

Example

Topspin-90(config-if-ether-2/1)# interface gateway 2

Topspin-90(config-if-gw-2)# bridge-group 21

Topspin-90(config-if-gw-2)#

6.Return to User-Exec mode and view the bridge-group.

???Enter exit.

???Enter exit.

???Enter show bridge-group:

Example

Topspin-90(config-if-gw-2)# exit

Topspin-90(config)# exit

Topspin-90# show bridge-group

All configured bridging-ports are displayed.

7.Configure the Ethernet hosts by entering ifconfig eth# ip-address netmask netmask.

???Log in to the host as a root user.

???Enter ifconfig eth# subnet# netmask #

Example

# ifconfig eth1 61.0.0.0 netmask 255.255.0.0

Repeat for all Ethernet hosts.

15

Configure the Fibre Channel Gateway (Optional)

To configure ITLs, perform the following steps:

1.Log in to your host.

2.Enter the /etc/init.d/ts_srp stop command at the host CLI to force the host to log out of any storage. You must disable connections now and enable them later in this process.

3.Enter the /usr/local/topspin/bin/vstat --verbose|grep -i guid command at the host CLI to display the host GUID.

NOTE: Record the GUID value. You will enter it repeatedly.

4.Enter the no shutdown command on your Server Switch CLI to bring up any Fibre Channel gateway cards that connect your device to FC storage. The gateway automatically discovers all storage.

Example

Topspin-90# configure terminal

Topspin-90(config)# card 6

Topspin-90(config-card-6)# no shutdown

5.(Optional) Enter the show fc srp target command to view the storage that your gateway(s) discovered.

Example

Topspin-90# show fc srp target

================================================================================

SRP Targets

================================================================================

wwpn: 21:00:00:04:cf:f6:c2:ab

wwnn: 20:00:00:04:cf:f6:c2:ab description: SRP.T10:21000004CFF6C2AB

ioc-guid: 00:05:ad:00:00:00:15:1a service-name: SRP.T10:21000004CFF6C2AB protocol-ids: 04:00:00:00:00:00:00:00:00:00

fc-address: 00:00:ef mtu: 0

connection-type: nl-port

physical-access: 6/1 Total: 1 targets.

Topspin-90#

6.Enter the fc srp initiator command with

???the GUID of the host (from step 3)

???the GUID extension (always 00:00:00:00:00:00:00:00)

???the auto-bind keyword

to configure your host.

NOTE: For an initiator to successfully connect to storage, each physical FC gateway port requires a virtual port with a unique WWPN that points to the initiator. Also, the initiator requires a unique WWNN. You can configure these values manually, but we strongly recommend that you take

16

advantage of CLI options to assign these values dynamically. The remainder of this process dynamically allocates WWNNs and WWPNs.

The auto-bind keyword:

???Assigns a unique WWNN to the initiator.

???Creates a virtual port (that maps to the initiator) for all possible (actual & potential) physical FC gateway port and assigns a WWPN to each virtual port.

Example

Topspin-90# configure terminal

Topspin-90(config)# fc srp initiator 00:05:ad:00:00:01:29:5c 00:00:00:00:00:00:00:00 auto-bind

7.(Optional) Enter the fc srp initiator command with

???the GUID of the host

???the GUID extension

???the description keyword

to assign an easily-recognizable identifier to the initiator.

Example

Topspin-90(config)# fc srp initiator 00:05:ad:00:00:01:29:c5 00:00:00:00:00:00:00:00 description Bender

17

8.(Optional) Enter the show fc srp initiator command to verify the new initiator.

Example

Topspin-90# show fc srp initiator

================================================================================

SRP Initiators

================================================================================

guid: 00:05:ad:00:00:01:29:c5 extension: 00:00:00:00:00:00:00:00

description: Bender

wwnn: 20:01:00:05:ad:00:40:00 credit: 0

active-ports: none pkeys:

action: auto-bind

result: success

wwpns: port wwpnfc-addr 2/1 20:01:00:05:ad:20:40:00 00:00:00 2/2 20:01:00:05:ad:24:40:00 00:00:00 3/1 20:01:00:05:ad:30:40:00 00:00:00 3/2 20:01:00:05:ad:34:40:00 00:00:00 4/1 20:01:00:05:ad:40:40:00 00:00:00 4/2 20:01:00:05:ad:44:40:00 00:00:00 5/1 20:01:00:05:ad:50:40:00 00:00:00 5/2 20:01:00:05:ad:54:40:00 00:00:00 6/1 20:01:00:05:ad:60:40:00 00:00:02 6/2 20:01:00:05:ad:64:40:00 00:00:00 7/1 20:01:00:05:ad:70:40:00 00:00:00 7/2 20:01:00:05:ad:74:40:00 00:00:00 8/1 20:01:00:05:ad:80:40:00 00:00:00 8/2 20:01:00:05:ad:84:40:00 00:00:00 9/1 20:01:00:05:ad:90:40:00 00:00:00 9/2 20:01:00:05:ad:94:40:00 00:00:00

10/1 20:01:00:05:ad:a0:40:00 00:00:00 10/2 20:01:00:05:ad:a4:40:00 00:00:00 11/1 20:01:00:05:ad:b0:40:00 00:00:00 11/2 20:01:00:05:ad:b4:40:00 00:00:00 12/1 20:01:00:05:ad:c0:40:00 00:00:00 12/2 20:01:00:05:ad:c4:40:00 00:00:00 13/1 20:01:00:05:ad:d0:40:00 00:00:00 13/2 20:01:00:05:ad:d4:40:00 00:00:00 14/1 20:01:00:05:ad:e0:40:00 00:00:00 14/2 20:01:00:05:ad:e4:40:00 00:00:00

Total: 1 initiators.

Topspin-90#

9.Enter the fc srp initiator command with

???the GUID of the host

???the GUID extension of the host

???the discover-itl keyword

to prompt your device to discover the potential ITLs that your initiator can form.

18

NOTE: This command may take some time to execute.

Example

Topspin-90(config)# fc srp initiator 00:05:ad:00:00:01:29:c5 00:00:00:00:00:00:00:00 discover-itl

10.Enter the show fc srp itl command to display the potential ITLs and record the target-wwpn and fc-lunid values.

Example

Topspin-90(config)# exit

Topspin-90# show fc srp itl

================================================================================

SRP ITL

================================================================================

guid: 00:05:ad:00:00:01:29:c5 extension: 00:00:00:00:00:00:00:00 target-wwpn: 21:00:00:04:cf:f6:c2:ab fc-lunid: 00:00:00:00:00:00:00:00 srp-lunid: 00:00:00:00:00:00:00:00

logical-id (raw 64 bytes): 01:03:00:08:20:00:00:04:cf:f6:c2:ab:00:00:00:00

:

00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00

:

00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00

:

00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00

logical-id (formatted display): 2000000000000000 description: itl

device-category: random lun-policy: restricted

non-restricted-ports: none active-ports: none

physical-access: 6/1 hi-mark: 16

max-retry: 5 min-io-timeout: 10

dynamic-path-affinity: false dynamic-gateway-port-loadbalancing: true dynamic-storage-port-loadbalancing:

dynamic-gateway-port-failover: false dynamic-storage-port-failover:

active-slots: none

Total: 1 itls.

Topspin-90#

19

NOTE: The fc-lunid field displays the LUN ID of the ITL that you will configure. For non-sequential LUNs, this value will always appear as 00:00:00:00:00:00:00:00. For sequential LUNs, the value will iterate by 1 for each LUN in the target device.

11.Enter the no fc srp itl command with

???the GUID of the host

???the GUID extension of the host

???the WWPN of the target (from step 10)

???the LUN ID of the storage disk (from step 10)

???the lun-policy keyword

???the restricted keyword

to configure the ITL through LUN masking.

Example

Topspin-90(config)# no fc srp itl 00:05:ad:00:00:01:29:c5 00:00:00:00:00:00:00:00 21:00:00:04:cf:f6:c2:ab 00:00:00:00:00:00:00:00 lun-policy restricted

12.Enter the no fc srp it command with

???the GUID of the host

???the GUID extension of the host

???the WWPN of the target

???the gateway-portmask-policy keyword

???the restricted keyword

???the all keyword or the ports you want to unmask

to grant the initiator portmask-policy access to the target.

Example

Topspin-90(config)# no fc srp it 00:05:ad:00:00:01:29:c5 00:00:00:00:00:00:00:00 21:00:00:04:cf:f6:c2:ab gateway-portmask-policy restricted all

13.Enter the no fc srp itl command with

???the GUID of the host

???the GUID extension of the host

???the WWPN of the target

???the LUN ID of the storage disk

???the gateway-portmask-policy keyword

???the restricted keyword

???the all keyword or the ports you want to unmask

to grant the initiator portmask-policy access to the LUN.

Example

Topspin-90(config)# no fc srp itl 00:05:ad:00:00:01:29:c5 00:00:00:00:00:00:00:00: 21:00:00:04:cf:f6:c2:ab 00:00:00:00:00:00:00:00 gateway-portmask-policy restricted all

14. Enter the /etc/init.d/ts_srp start command at the host.

20

15.Return to User Exec mode or Privileged Exec mode and enter the show fc srp itl command to verify that the ITL is active. Port numbers appear in the physical-access and active-ports fields to indicate that the ITL is active.

Index