wiki

Contents

  1 Definitions
  2 Link Generation
  3 Creating FCIP Connections
  4 Creating LSAN zones for FC Routing Communications
  5 Useful Commands to monitoring and status

Definitions

  Backbone Fabric – Silkworm 7500’s connected together via VE-Ports or VEX-Ports
  Edge Fabric – Normal switched fabric connected to the backbone via E-Ports or EX-Ports
  VE-Port – FCIP interconnected E-Port/ISL, i.e. fabrics will merge
  VEX-Port – FCIP interconnected EX-Port, routing needed via lsan zoning to connect initiator to a target.
  E-Port – Normal expansion connection between one switch to another.
  EX-Port – A connection that connects 2 fabrics together without merging services
  LSAN – A special zone that allows 2 fabrics interconnected via EX-Ports to communicate. 

Link Generation

  On the Backbone/Edge ensure that interopmode is disabled
      Check to ensure you aren’t running in Mcdata Compatibility Mode 

      >>interopmode 

  On the Backbone/Edge ensure that secmode is disabled
      Check to ensure Secure-Fabrics isn’t enabled 

      >>secmodeshow 

  On the Backbone ensure that msplatform DB is disabled
      Check to ensure there is no external management to Brocade Fabrics 

      >>msplatshow 

  Enable the FC Routing, by default its disabled on the backbone
      Ensure its disabled first 

      >>fosConfig –disable fcr 

      Set the Backbone Fabric ID 

      >>fcrconfigure 

      Ensure you use the same backbone id for all the routers in that backbone.
      Enable the FC routing 

      >>fosconfig –enable fcr 

  Perform these steps on all Routers in the Backbone Fabric 

Creating FCIP Connections

  Step to enabling FCIP is to turn on your EX-Port from the backbone fabric (Routers) into the edge fabric (e.g. Tape SAN). On both sides of your WAN link.
      Pick the same port for each backbone fabric i.e. port-4 on Source Router and Destination Router, to keep things easy to manage, on both sides
      If plugging into the same switch types plug the router FC port into the same port on the other side of the EX-Port/Edge Fabric
      By default all ports on the routers are persistently disabled, ensuring no-one can just plug in and go.
      Here are the steps to configure the EX-Ports

          >>portcfgexport 4 -a 1 -f 30 (fcrap14c001) 
          >>portcfgexport 4 –a 1 –f 31 (fcrap14c101) 

          This will configure port-4 to be an ex-port once its been enabled, with (-a 1, enabling), (-f 30, defines the fabric id) 
      On the other side of the backbone, you simply flip the EX-port fabric id’s, other wise the backbone fabric router will not merge properly

          >>portcfgexport 4 –a 1 –f 31 (fcrrp21001) 
          >>portcfgexport 4 –a 1 –f 30 (fcrrp21101) 

  Next you need to enable the port
      >>portcfgpersistentenable 4 on all routers, you have configured EX-ports on.
      This will enable the port, and assuming its plugged into the edge fabric it will synch up, and you have a routable environment from the FC-Side
      On the backbone you will see:

          EX-Port 10:00:00:05:1e:35:2c:9d "fcsap14c501" (fabric id = 30 )(Trunk master) 

      On the edge fabrics you will see:

          E-Port 50:00:51:e3:81:2e:4e:1e "fcr_fd_160" (downstream)(Trunk master) 

      One thing to remember the main reason for configuration of an EX-Port on the backbone, is so the fabrics will not merge themselves, thus ensuring WAN link outages won’t cause multi-fabric reconfigurations. 
  IPSEC is supported on these routers, however this configuration uses a PVC, not a internet based VPN.
      You can configure only 1 tunnel per Ge port if IPSec is enabled, this configuration takes advantage of multiple tunnels over the same ports.
      As well Tape Pipelining and Fast-Write aren’t supported over a secure-tunnel, and we plan to use both of these items. 
  Next you need to ensure you enable the virtual ports on both routers
      Ge0 - 16 – 23 correspond to Physical 0-8
      Ge1 - 24 – 31 correspond to Physical 9-15 
  Go ahead and enable the virtual ports you need based on the number of tunnels you are running across the ge-ports.

          <code> >>portcfgpersistentenable 16 enables a tunnel for ge0 
          >>portcfgpersistentenable 24 enables a tunnel for ge1 

          Ensure this is done on both sides of the WAN, and both set of routers 
  Create the IP Interfaces for the ge-ports on each router
      LC-Router >>portcfg ipif ge0 create 10.255.2.3 255.255.255.0 2250
          2250 is the MTU size (2148, is FC Frame size) 
      FC-Router >>portcfg ipif ge0 create 10.255.2.4 255.255.255.0 2250
          In a flat subnet there is no need to configure a ip route, once the tunnel is up the route will be generated for you
          If you wish to enable tunnels on ge1 then you need to perform the same procedure on those ports on each router. 
  Next are is to create the tunnels on both sides
      LC-Router >>portcfg fciptunnel ge0 create 0 10.255.2.4 10.255.2.3 512000 -c
          Tunnel ID = 0
          Dest IP = 10.255.2.4
          Src IP = 10.255.2.3
          comm_rate = 512000 (500 mbps) (What Networks has given us)
          -c (compression enable)
          -t (enables tapepiplining)
          -f (enables fastwrite) 

• *Note** HP Currently does not support the Fast Write option as it pertains to Continuous Access Journaling, if this is enabled you will not be able to create the needed pairs

      FC-Router >>portcfg fciptunnel ge0 create 0 10.255.2.3 10.255.2.4 512000 -c
          You will need to ensure that you create tunnels on the other pais of routers as well. 
  Once this is complete the tunnels should come online, provided you have done everything correctly

      Online VE-Port 10:00:00:05:1e:37:f2:fc "fcrrp21001" (upstream) 

Creating LSAN zones for FC Routing Communications

At this point you are ready to create some LSAN zones and start moving some data

  On the Edge Fabrics
      Create an alias in the LC Fabric for the Target-Port on the FC Storage Platform

          aliCreate "Storage_1B_Alias","50:06:00:00:01:02:03:04" 

      Create a zone with the remote alias and the local one

          zoneCreate "lsan_CAXP_Jrnl_Zone"," Storage_1B_Alias; XPAP14C001_1B_Alias" 

      Add the new zone into the Defined zone schema

          cfgAdd "CFG_3"," lsan_CAXP_Jrnl_Zone" 

      Add the new zone into the Effective zone schema

          cfgEnable "CFG_3" 

  FC- Edge Fabric

          aliCreate "XPAP14C001_1B_Alias"," 50:06:0e:80:04:7e:69:01" 
          zoneCreate "lsan_CAXP_Jrnl_Zone"," XPAP14C001_1B_Alias; XPRP21001_1B_Alias" 

      Add the new zone into the Defined zone schema

          cfgAdd "FC_CFG4"," lsan_CAXP_Jrnl_Zone" 

      Add the new zone into the Effective zone schema

          cfgEnable "Fc_CFG4" 

      You will need to do the same process above for the other sides of the XP
          XPAP14C001_2B_Alias – XPRP21001_2B_Alias (Initiator – RCU Target)
          Since we are going both ways you will need to configure another lsan zone coming back the other way for the XPAP14C001_1D_Alias – XPRP21001_1D_Alias (RCU Target – Initiator) 
      Must use LSAN or lsan in the beginning of any zone that will be routed. 

Useful Commands to monitoring and status

 > portcmd --ping ge0 -s 10.255.2.3 -d 10.255.2.4
 Pinging 10.255.2.4 from ip interface 10.255.2.3 on 0/ge0 with 64 bytes of data
 Reply from 10.255.2.4: bytes=64 rtt=12ms ttl=64
 Reply from 10.255.2.4: bytes=64 rtt=12ms ttl=64
 Reply from 10.255.2.4: bytes=64 rtt=12ms ttl=64
 Reply from 10.255.2.4: bytes=64 rtt=12ms ttl=64
 Ping Statistics for 10.255.2.4:
 Packets: Sent = 4, Received = 4, Loss = 0 ( 0 percent loss)
 Min RTT = 12ms, Max RTT = 12ms Average = 12ms

 > portshow ipif ge0
 Port: ge0
 Interface IP Address NetMask MTU
 0 10.255.2.3 255.255.255.0 2250

 > portshow ipif ge1
 Port: ge1
 Interface IP Address NetMask MTU
 0 10.255.2.5 255.255.255.0 2250

 > portshow iproute ge0
 Port: ge0
 IP Address Mask Gateway Metric Flags
 10.255.2.0 255.255.255.0 10.255.2.3 0 Interface

 > portshow iproute ge1
 Port: ge1
 IP Address Mask Gateway Metric Flags
 10.255.2.0 255.255.255.0 10.255.2.5 0 Interface

 > portshow fciptunnel ge0 all
 Port: ge0
 Tunnel ID 0
 Remote IP Addr 10.255.2.4
 Local IP Addr 10.255.2.3
 Remote WWN Not Configured
 Local WWN 10:00:00:05:1e:38:12:e4
 Compression on
 Fastwrite on
 Tape Pipelining on
 Committed Rate 636928 Kbps (0.636928 Gbps)
 SACK on
 Min Retransmit Time 100
 Keepalive Timeout 10
 Max Retransmissions 8
 Status : Active
 Uptime 19 hours, 11 minutes, 2 seconds

 > portshow fciptunnel ge1 all
 Port: ge1
 Tunnel ID 0
 Remote IP Addr 10.255.2.6
 Local IP Addr 10.255.2.5
 Remote WWN Not Configured
 Local WWN 10:00:00:05:1e:38:12:e4
     Compression on
 Fastwrite on
 Tape Pipelining on
 Committed Rate 636928 Kbps (0.636928 Gbps)
 SACK on
 Min Retransmit Time 100
 Keepalive Timeout 80
 Max Retransmissions 9
 Status : Active
 Uptime 19 hours, 11 minutes, 13 seconds

 > portshow arp ge0
 Port: ge0
 IP Address Mac Address Flags
 10.255.2.4 00:05:1e:37:f2:e0 Resolved

 > portshow arp ge1
 Port: ge1
 IP Address Mac Address Flags
 10.255.2.6 00:05:1e:37:f2:e1 Resolved

 > portshow ge0
 Eth Mac Address: 00.05.1e.38.12.c8
 Port State: 1 Online
 Port Phys: 6 In_Sync
 Port Flags: 0x3 PRESENT ACTIVE
 Port Speed: 1G

 > portshow ge1
 Eth Mac Address: 00.05.1e.38.12.c9
 Port State: 1 Online
 Port Phys: 6 In_Sync
 Port Flags: 0x3 PRESENT ACTIVE
 Port Speed: 1G

 > portshow mode ge0
 Port: ge0
 Mode: FCIP

 > portshow mode ge1
 Port: ge1
 Mode: FCIP

 > fcrfabricshow
 FC Router WWN: 10:00:00:05:1e:38:12:e4, Dom ID: 50, Info: 10.236.190.51, "fcrap14c001"
   EX_Port FID Neighbor Switch Info (WWN, enet IP, name)
   4 30 10:00:00:05:1e:35:2c:9d 10.236.189.121 "fcsap14c301"
 FC Router WWN: 10:00:00:05:1e:37:f2:fc, Dom ID: 51, Info: 10.12.23.194, "fcrrp21001"
   EX_Port FID Neighbor Switch Info (WWN, enet IP, name)
   4 31 10:00:00:05:1e:03:11:e1 10.12.23.188 "fcsrp21001"

 > fcrresourceshow
 Daemon Limits:
 Max Allowed Currently Used
 LSAN Zones: 3000 0
 LSAN Devices: 10000 0
 Proxy Device Slots: 10000 0
 WWN Pool Size Allocated
 Phantom Node WWN: 8192 0
 Phantom Port WWN: 32768 0
 Port Limits:
 Max proxy devices: 2000
 Max NR_Ports: 1000

 > fcrrouteshow
 Destination NR_Port FCRP Cost WWN of Principal
 30 320400 1000 10:00:00:05:1e:35:2b:c7
 31 330400 1000 10:00:00:05:1e:03:11:e1

Enter the portCfgShowcommand to view ports that are persistently disabled. Enter the portCfgPersistentDisablecommand to disable any VE_Ports that you will use in the FCIP tunnel configuration

switch:admin> portcfggemediatype ge1 optical The command options are as follows: ge0|ge1 geOfor port 0 or ge1for port 1. copper|optical The media type.

The following command creates an IP interface for port ge0 on the Brocade 7800 switch. switch:admin> portcfg ipif ge0 create 192.168.1.78 255.255.255.0 1500

Validating IP connectivity

The -s option specifies the source address, and the -d option specifies the destination address. switch:admin> portcmd –ping ge0 -s 192.168.11.78 -d 192.168.1.24

switch:admin> portcfg fciptunnel 16 create 192.168.1.24 192.168.11.78 -b 5500 -B 6200

portshow fciptunnel

portCfgPersistentEnable

switch63:root> portshow fciptunnel all -c

Tunnel Circuit OpStatus Flags Uptime TxMBps RxMBps ConnCnt CommRt Met
-----------------------------------------------------------------------------
--
16 - Up c------ 4m22s 0.00 0.00 1 - -
16 0 ge0 Up ---4--s 4m22s 0.00 0.00 1 1000/1000 0