wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
===== How to setup FCIP on Brocade Switch =====

Contents

    1 Definitions
    2 Link Generation
    3 Creating FCIP Connections
    4 Creating LSAN zones for FC Routing Communications
    5 Useful Commands to monitoring and status

Definitions

    Backbone Fabric – Silkworm 7500’s connected together via VE-Ports or VEX-Ports
    Edge Fabric – Normal switched fabric connected to the backbone via E-Ports or EX-Ports
    VE-Port – FCIP interconnected E-Port/ISL, i.e. fabrics will merge
    VEX-Port – FCIP interconnected EX-Port, routing needed via lsan zoning to connect initiator to a target.
    E-Port – Normal expansion connection between one switch to another.
    EX-Port – A connection that connects 2 fabrics together without merging services
    LSAN – A special zone that allows 2 fabrics interconnected via EX-Ports to communicate. 

Link Generation

    On the Backbone/Edge ensure that interopmode is disabled
        Check to ensure you aren’t running in Mcdata Compatibility Mode 

        >>interopmode 

    On the Backbone/Edge ensure that secmode is disabled
        Check to ensure Secure-Fabrics isn’t enabled 

        >>secmodeshow 

    On the Backbone ensure that msplatform DB is disabled
        Check to ensure there is no external management to Brocade Fabrics 

        >>msplatshow 

    Enable the FC Routing, by default its disabled on the backbone
        Ensure its disabled first 

        >>fosConfig –disable fcr 

        Set the Backbone Fabric ID 

        >>fcrconfigure 

        Ensure you use the same backbone id for all the routers in that backbone.
        Enable the FC routing 

        >>fosconfig –enable fcr 

    Perform these steps on all Routers in the Backbone Fabric 

Creating FCIP Connections

    Step to enabling FCIP is to turn on your EX-Port from the backbone fabric (Routers) into the edge fabric (e.g. Tape SAN). On both sides of your WAN link.
        Pick the same port for each backbone fabric i.e. port-4 on Source Router and Destination Router, to keep things easy to manage, on both sides
        If plugging into the same switch types plug the router FC port into the same port on the other side of the EX-Port/Edge Fabric
        By default all ports on the routers are persistently disabled, ensuring no-one can just plug in and go.
        Here are the steps to configure the EX-Ports

            >>portcfgexport 4 -a 1 -f 30 (fcrap14c001) 
            >>portcfgexport 4 –a 1 –f 31 (fcrap14c101) 

            This will configure port-4 to be an ex-port once its been enabled, with (-a 1, enabling), (-f 30, defines the fabric id) 
        On the other side of the backbone, you simply flip the EX-port fabric id’s, other wise the backbone fabric router will not merge properly

            >>portcfgexport 4 –a 1 –f 31 (fcrrp21001) 
            >>portcfgexport 4 –a 1 –f 30 (fcrrp21101) 

    Next you need to enable the port
        >>portcfgpersistentenable 4 on all routers, you have configured EX-ports on.
        This will enable the port, and assuming its plugged into the edge fabric it will synch up, and you have a routable environment from the FC-Side
        On the backbone you will see:

            EX-Port 10:00:00:05:1e:35:2c:9d "fcsap14c501" (fabric id = 30 )(Trunk master) 

        On the edge fabrics you will see:

            E-Port 50:00:51:e3:81:2e:4e:1e "fcr_fd_160" (downstream)(Trunk master) 

        One thing to remember the main reason for configuration of an EX-Port on the backbone, is so the fabrics will not merge themselves, thus ensuring WAN link outages won’t cause multi-fabric reconfigurations. 
    IPSEC is supported on these routers, however this configuration uses a PVC, not a internet based VPN.
        You can configure only 1 tunnel per Ge port if IPSec is enabled, this configuration takes advantage of multiple tunnels over the same ports.
        As well Tape Pipelining and Fast-Write aren’t supported over a secure-tunnel, and we plan to use both of these items. 
    Next you need to ensure you enable the virtual ports on both routers
        Ge0 - 16 – 23 correspond to Physical 0-8
        Ge1 - 24 – 31 correspond to Physical 9-15 
    Go ahead and enable the virtual ports you need based on the number of tunnels you are running across the ge-ports.

            <code> >>portcfgpersistentenable 16 enables a tunnel for ge0 
            >>portcfgpersistentenable 24 enables a tunnel for ge1 

            Ensure this is done on both sides of the WAN, and both set of routers 
    Create the IP Interfaces for the ge-ports on each router
        LC-Router >>portcfg ipif ge0 create 10.255.2.3 255.255.255.0 2250
            2250 is the MTU size (2148, is FC Frame size) 
        FC-Router >>portcfg ipif ge0 create 10.255.2.4 255.255.255.0 2250
            In a flat subnet there is no need to configure a ip route, once the tunnel is up the route will be generated for you
            If you wish to enable tunnels on ge1 then you need to perform the same procedure on those ports on each router. 
    Next are is to create the tunnels on both sides
        LC-Router >>portcfg fciptunnel ge0 create 0 10.255.2.4 10.255.2.3 512000 -c
            Tunnel ID = 0
            Dest IP = 10.255.2.4
            Src IP = 10.255.2.3
            comm_rate = 512000 (500 mbps) (What Networks has given us)
            -c (compression enable)
            -t (enables tapepiplining)
            -f (enables fastwrite) 

            **Note** HP Currently does not support the Fast Write option as it pertains to Continuous Access Journaling, if this is enabled you will not be able to create the needed pairs 

        FC-Router >>portcfg fciptunnel ge0 create 0 10.255.2.3 10.255.2.4 512000 -c
            You will need to ensure that you create tunnels on the other pais of routers as well. 
    Once this is complete the tunnels should come online, provided you have done everything correctly

        Online VE-Port 10:00:00:05:1e:37:f2:fc "fcrrp21001" (upstream) 

Creating LSAN zones for FC Routing Communications

At this point you are ready to create some LSAN zones and start moving some data

    On the Edge Fabrics
        Create an alias in the LC Fabric for the Target-Port on the FC Storage Platform

            aliCreate "Storage_1B_Alias","50:06:00:00:01:02:03:04" 

        Create a zone with the remote alias and the local one

            zoneCreate "lsan_CAXP_Jrnl_Zone"," Storage_1B_Alias; XPAP14C001_1B_Alias" 

        Add the new zone into the Defined zone schema

            cfgAdd "CFG_3"," lsan_CAXP_Jrnl_Zone" 

        Add the new zone into the Effective zone schema

            cfgEnable "CFG_3" 

    FC- Edge Fabric

            aliCreate "XPAP14C001_1B_Alias"," 50:06:0e:80:04:7e:69:01" 
            zoneCreate "lsan_CAXP_Jrnl_Zone"," XPAP14C001_1B_Alias; XPRP21001_1B_Alias" 

        Add the new zone into the Defined zone schema

            cfgAdd "FC_CFG4"," lsan_CAXP_Jrnl_Zone" 

        Add the new zone into the Effective zone schema

            cfgEnable "Fc_CFG4" 

        You will need to do the same process above for the other sides of the XP
            XPAP14C001_2B_Alias – XPRP21001_2B_Alias (Initiator – RCU Target)
            Since we are going both ways you will need to configure another lsan zone coming back the other way for the XPAP14C001_1D_Alias – XPRP21001_1D_Alias (RCU Target – Initiator) 
        Must use LSAN or lsan in the beginning of any zone that will be routed. 


Useful Commands to monitoring and status

   > portcmd --ping ge0 -s 10.255.2.3 -d 10.255.2.4
   Pinging 10.255.2.4 from ip interface 10.255.2.3 on 0/ge0 with 64 bytes of data
   Reply from 10.255.2.4: bytes=64 rtt=12ms ttl=64
   Reply from 10.255.2.4: bytes=64 rtt=12ms ttl=64
   Reply from 10.255.2.4: bytes=64 rtt=12ms ttl=64
   Reply from 10.255.2.4: bytes=64 rtt=12ms ttl=64
   Ping Statistics for 10.255.2.4:
   Packets: Sent = 4, Received = 4, Loss = 0 ( 0 percent loss)
   Min RTT = 12ms, Max RTT = 12ms Average = 12ms

   > portshow ipif ge0
   Port: ge0
   Interface IP Address NetMask MTU
   0 10.255.2.3 255.255.255.0 2250

   > portshow ipif ge1
   Port: ge1
   Interface IP Address NetMask MTU
   0 10.255.2.5 255.255.255.0 2250

   > portshow iproute ge0
   Port: ge0
   IP Address Mask Gateway Metric Flags
   10.255.2.0 255.255.255.0 10.255.2.3 0 Interface

   > portshow iproute ge1
   Port: ge1
   IP Address Mask Gateway Metric Flags
   10.255.2.0 255.255.255.0 10.255.2.5 0 Interface

   > portshow fciptunnel ge0 all
   Port: ge0
   Tunnel ID 0
   Remote IP Addr 10.255.2.4
   Local IP Addr 10.255.2.3
   Remote WWN Not Configured
   Local WWN 10:00:00:05:1e:38:12:e4
   Compression on
   Fastwrite on
   Tape Pipelining on
   Committed Rate 636928 Kbps (0.636928 Gbps)
   SACK on
   Min Retransmit Time 100
   Keepalive Timeout 10
   Max Retransmissions 8
   Status : Active
   Uptime 19 hours, 11 minutes, 2 seconds

   > portshow fciptunnel ge1 all
   Port: ge1
   Tunnel ID 0
   Remote IP Addr 10.255.2.6
   Local IP Addr 10.255.2.5
   Remote WWN Not Configured
   Local WWN 10:00:00:05:1e:38:12:e4
       Compression on
   Fastwrite on
   Tape Pipelining on
   Committed Rate 636928 Kbps (0.636928 Gbps)
   SACK on
   Min Retransmit Time 100
   Keepalive Timeout 80
   Max Retransmissions 9
   Status : Active
   Uptime 19 hours, 11 minutes, 13 seconds

   > portshow arp ge0
   Port: ge0
   IP Address Mac Address Flags
   10.255.2.4 00:05:1e:37:f2:e0 Resolved

   > portshow arp ge1
   Port: ge1
   IP Address Mac Address Flags
   10.255.2.6 00:05:1e:37:f2:e1 Resolved

   > portshow ge0
   Eth Mac Address: 00.05.1e.38.12.c8
   Port State: 1 Online
   Port Phys: 6 In_Sync
   Port Flags: 0x3 PRESENT ACTIVE
   Port Speed: 1G

   > portshow ge1
   Eth Mac Address: 00.05.1e.38.12.c9
   Port State: 1 Online
   Port Phys: 6 In_Sync
   Port Flags: 0x3 PRESENT ACTIVE
   Port Speed: 1G

   > portshow mode ge0
   Port: ge0
   Mode: FCIP

   > portshow mode ge1
   Port: ge1
   Mode: FCIP

   > fcrfabricshow
   FC Router WWN: 10:00:00:05:1e:38:12:e4, Dom ID: 50, Info: 10.236.190.51, "fcrap14c001"
     EX_Port FID Neighbor Switch Info (WWN, enet IP, name)
     4 30 10:00:00:05:1e:35:2c:9d 10.236.189.121 "fcsap14c301"
   FC Router WWN: 10:00:00:05:1e:37:f2:fc, Dom ID: 51, Info: 10.12.23.194, "fcrrp21001"
     EX_Port FID Neighbor Switch Info (WWN, enet IP, name)
     4 31 10:00:00:05:1e:03:11:e1 10.12.23.188 "fcsrp21001"

   > fcrresourceshow
   Daemon Limits:
   Max Allowed Currently Used
   LSAN Zones: 3000 0
   LSAN Devices: 10000 0
   Proxy Device Slots: 10000 0
   WWN Pool Size Allocated
   Phantom Node WWN: 8192 0
   Phantom Port WWN: 32768 0
   Port Limits:
   Max proxy devices: 2000
   Max NR_Ports: 1000

   > fcrrouteshow
   Destination NR_Port FCRP Cost WWN of Principal
   30 320400 1000 10:00:00:05:1e:35:2b:c7
   31 330400 1000 10:00:00:05:1e:03:11:e1

===== Additional ===== 

Enter the portCfgShowcommand to view ports that are persistently disabled.
Enter the portCfgPersistentDisablecommand to disable any VE_Ports that you will use in the FCIP tunnel configuration

switch:admin> portcfggemediatype ge1 optical
The command options are as follows:
ge0|ge1 geOfor port 0 or ge1for port 1.
copper|optical The media type.

The following command creates an IP interface for port ge0 on the Brocade 7800 switch.
switch:admin> portcfg ipif ge0 create 192.168.1.78 255.255.255.0 1500

Validating IP connectivity

The -s option specifies the source address, and 
the -d option specifies the destination address. 
switch:admin> portcmd --ping ge0 -s 192.168.11.78 -d 192.168.1.24

switch:admin> portcfg fciptunnel 16 create 192.168.1.24 192.168.11.78 -b 5500 -B 6200

portshow fciptunnel

portCfgPersistentEnable

<code>
switch63:root> portshow fciptunnel all -c

Tunnel Circuit OpStatus Flags Uptime TxMBps RxMBps ConnCnt CommRt Met
-----------------------------------------------------------------------------
--
16 - Up c------ 4m22s 0.00 0.00 1 - -
16 0 ge0 Up ---4--s 4m22s 0.00 0.00 1 1000/1000 0
</code>
wiki

User Tools

Site Tools

Page Tools
