Table of Contents
Brocade Advanced settings
Enable root access
To enable the root account
switch:admin> userconfig --change root -e yes
To set root access level to 'all':
switch:admin> rootaccess --set all
To set root access level to 'all' without prompting for confirmation:
switch:admin> rootaccess --set all -force
To display root access settings:
switch:admin> rootaccess --show RootAccess: all
Clear / remove SAN configuration
If you want to clear the zoning only on one switch, then first disable E-Ports to prevent the deletion on the whole Fabric.
Then do
FLEX-A1-BLUE:admin> cfgdisable -f FLEX-A1-BLUE:admin> switchshow switchName: FLEX-A1-BLUE zoning: OFF FLEX-A1-BLUE:admin> cfgclear The Clear All action will clear all Aliases, Zones, FA Zones and configurations in the Defined configuration. Run cfgSave to commit the transaction or cfgTransAbort to cancel the transaction. Do you really want to clear all configurations? (yes, y, no, n): [no] y FLEX-A1-BLUE:admin> cfgshow Defined configuration: no configuration defined Effective configuration: no configuration in effect
Configuring XISL
I’ll be creating something similar to the image below. The images comes straight out of the admin guide. XISL will act as virtual ISL for all virtual fabric with same FID. The XISL shares traffic among different logical fabrics.
Base switches do not support direct device connectivity. A base switch can have only E_Ports, VE_Ports, or EX_Ports but no F_Ports.
The XISL is using LWL SFPs for the 10KM distance. The XISL will take care of the “trunking” of the fabrics on either sides.
Logical switch 1 and 5 will form the default fabric. Logical switch 2 and 6 will form fabric 1. Logical switch 3 and 7 will form fabric 15 Lastly, the base switches will form fabric 8, but this is a special XISL only fabric. Only ISL ports can be configured in the base switches.
Use the switchshow command and check the value of the Allow XISL Use parameter.
Else activate it with configure
Allow XISL Use (yes, y, no, n): y
Select your base Virtual switch (default is 128, you can keep it, else)
FID128:admin> lscfg --create 8 -base Creation of a base switch requires that the proposed new base switch on this system be disabled. Would you like to continue [y/n]?: y About to create switch with fid=8. Please wait... Logical Switch with FID (8) has been successfully created. switch_8:FID8:admin> switchenable
Enable XISL if not yet done. [CTRL+D] to stop and validate
switch_8:FID8:admin> switchdisable switch_8:FID8:admin> configure Configure... Fabric parameters (yes, y, no, n): [no] y Domain: (1..239) [1] 4 ... Allow XISL Use (yes, y, no, n): y ... switch_8:FID8:admin>switchenable
Logical ISL are only software based, not based on a physical port. You can enable or disable it by using
admin> lfcfg --lisldisable admin> lfcfg --lislenable
To check the status of LISL and XISL, use the following command
admin> lfcfg --showall -xisl
Configuring QOS
Quality Of Service can easily activated per zone
You can enable high or low priority traffic, based on a priority ID.
The id range is from 1 through 5 for high-priority traffic, which corresponds to VCs 10 through 14. For low-priority traffic, the id range is from 1 through 2, which corresponds to VCs 8 and 9. The id is optional; if it is not specified, the virtual channels are allocated through a round-robin scheme.
Example of High priority zone with ID 2
QOSH2_myzone1
For low priority 1
QOSL1_myzone2
After enabling zone, your ISL have to be configured for QOS:
sw0:admin> portcfgqos --enable 3
Configuring Virtual Fabric
Step 1, enabling VF
I read in the manual, VF would be enabled by default. This was not the case.
admin> fosconfig --show FC Routing service: disabled iSCSI service: Service not supported on this Platform iSNS client service: Service not supported on this Platform Virtual Fabric: disabled Ethernet Switch Service: Service not supported on this Platform
so first enable the feature. Warning though!! The switch will reboot immediately after hitting “Y”
admin> fosconfig --enable vf WARNING: This is a disruptive operation that requires a reboot to take effect. All EX ports will be disabled upon reboot. Would you like to continue [Y/N]: Y VF has been enabled. Your system is being rebooted.
Step 2, create a base switch
Notice we’ve got a default fabric 128 after enabling VF. You can see in which FID context you currently are in the identifier in front of your username.
FID128:admin> lscfg --create 8 -base Creation of a base switch requires that the proposed new base switch on this system be disabled. Would you like to continue [y/n]?: y About to create switch with fid=8. Please wait... Logical Switch with FID (8) has been successfully created.
Logical Switch has been created with default configurations. Please configure the Logical Switch with appropriate switch and protocol settings before activating the Logical Switch.
FID 8 is created, so let’s assign some ports. Lets also switch context to FID8 when configuring the ports for the base switch by
FID128:admin>setcontext 8
Configure the port for the base switch. Port 6 for example, as shown in the figure above.
FID8:admin> lscfg --config 8 -port 6 This operation requires that the affected ports be disabled. Would you like to continue [y/n]?: y Making this configuration change. Please wait... Configuration change successful.
Use port compression to increase bandwith
FID8:admin> portcfgcompress --enable 6
Please enable your ports/switch when you are ready to continue.
It’s also possible to change the name of the logical switch, a base switch in this case:
FID8:admin>switchname baseswitch
Step 3, create the logical switches
The number of logical switches you can create depends on the platform you use. It’s mentioned in the guides. I’m working with a 5300, so i’m limited to 4 logical switches at time of writing this.
Besides the base switche and the default switch, i can create two more logical switches on the 5300. It does not really matter in which context you create the switches, as long you have permissions to do so.
FID8:admin> lscfg --create 1 FID8:admin> setcontext 1 FID1:admin> switchname fab1-lsw1 FID1:admin> lscfg --config 1 -port 2 FID1:admin> lscfg --create 15 FID1:admin> setcontext 15 FID15:admin> switchname fab15-lsw1 FID15:admin> lscfg --config 15 -port 5
after setting up my first 5300 switch, the second one can be setup in the same way. The logical switches dont have to have the same port assignment configuration. Above config follows the picture earlier in this post. I don’t think i have to write up the commands for the second switch, you’ll get it 🙂 Step 3, Connect the (X)ISL
When you physically connect the 5300 switches (port 6 on the first switch to port 8 on the second) fabric merges for fabrics with the same FIDs start to happen, just as they normally would when not using VF. When done right, you’ll have 4 Fabrics on your switches: One default, one base, and two normal logical switches. Each one can be configured individually.
Step 4, check fabric status
The fabricshow command will output two switches in each fabric, depending on the cli context you’re in.
When you login using Web Tools GUI, you’ll be able to zone each fabric individually. Misc. Notes
Ok these are some basic configuration steps. I’ll add notes where/when necessary .
Next post is a separate post about the configupload and configdownload command in a Virtual Fabric environment. It’s a bit different….
How do I create Virtual Fabric Zones with Commands?
Enable / Disable FOS components:
IBM_SAN384B_27:admin> fosconfig --show FC Routing service: enabled iSCSI service: Service not supported on this Platform iSNS client service: Service not supported on this Platform Virtual Fabric: disabled Ethernet Switch Service: disabled IBM_SAN384B_27:admin> fosconfig --disable fcr FC Routing service is disabled
Enable VF:
fosconfig --enable vf ... enter 'y' and then a reboot
Create logical switch:
lscfg --create FabricID
Set context to newly created switch:
setcontext FabricID
Disable to configure (set domain id etc.) and then enable:
switchdisable configure switchenable
After that, add ports from the physical switch:
e.g. lscfg --config FabricID -port 8-16
brcsan1:FID128:admin> lscfg --show Created switches: 128(ds) 120 Port 0 1 2 3 4 5 6 7 8 9 ------------------------------------------------------------------- FID 128 | 128 | 128 | 128 | 128 | 128 | 128 | 128 | 128 | 128 |
To display the logical switch details with the FID name:
switch:admin> lscfg --show -n ------------------------------------------------------------------ Switch Information ------------------------------------------------------------------ FID: 30 SwitchType: DS DomainID: 3 SwitchName: switch1 FabricName: thor2fab ------------------------------------------------------------------ FID: 2 SwitchType: BS DomainID: 1 SwitchName: switch_2 FabricName: base_switch
How to put a Blade internal switch as pass through
On the Blade switch, on all external ports disable all E_ports function, and force it as N_ports. Do the same on the external switch to prevent ISL use.
switch:admin> portcfgshow Ports of Slot 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 -------------------+--+--+--+--+----+--+--+--+----+--+--+--+----+--+ Speed AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN Fill Word 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 AL_PA Offset 13 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Trunk Port ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON Long Distance .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. VC Link Init .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Locked L_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Locked G_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Disabled E_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Locked E_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ISL R_RDY Mode .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. RSCN Suppressed .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Persistent Disable.. .. ON ON .. .. .. .. .. .. .. .. .. .. .. .. LOS TOV enable .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. NPIV capability ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON NPIV PP Limit 80 80 80 80 80 80 80 80 95 95 95 95 95 95 95 95 QOS E_Port .. .. AE AE AE AE AE AE AE AE AE AE AE AE AE AE EX Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Mirror Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
http://emmanuel.iffly.free.fr/doku.php?id=storage:brocade_gateway
Increase buffer credit to increase performance
To optimize bandwidth, you can increase memory buffer for each port (mostly E-ports) that will need buffer, but consider also to use compression, that can be very efficient and useful.
switch:admin> portcfgcompress --enable 2 switch:admin> portcfgshow Ports of Slot 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 -------------------+--+--+--+--+----+--+--+--+----+--+--+--+----+--+ Speed AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN Fill Word 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 AL_PA Offset 13 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Trunk Port ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON Long Distance .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ... Disabled E_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Locked E_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ISL R_RDY Mode .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. RSCN Suppressed .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Compression.. .. ON ON .. .. .. .. .. .. .. .. .. .. .. .. LOS TOV enable .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. NPIV capability ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ... QOS E_Port .. .. AE AE AE AE AE AE AE AE AE AE AE AE AE AE ...
Buffer credit are spread on a switch per ASIC (look at portbuffershow), if more buffer credit are needed, then you 'll find it using the following command:
SWSAN1:admin> portbuffershow User Port Lx Max/Resv Buffer Needed Link Remaining Port Type Mode Buffers Usage Buffers Distance Buffers -------------------------------------------------------------- 0 E - - 16 24 10km 1 - - 0 - - 2 - - 0 - - 3 F - - 16 - - 76 --------------------------------------------------------------
Change the buffercredit value for the port, and don't forget to clear the port statistics
besw32:admin> portcfgfportbuffers --enable 0 24 besw32:admin> statsclear
portbuffershow islshow trunkshow portstats64show
Check the port buffer usage, on each port you can check if you have enough buffer credit if the parameter stat64_inputBuffersFull is equal to zero, else you have to increase the buffercredit on this port, and if it's an ISL (E-port), add also buffercredit on the paired switch.
besw32:admin> portstats64show 0 ..... stat64_inputBuffersFull 20 .....
Trunk ports
However, to ensure in-order delivery, trunks can only be built between ports in the same ASIC.
Check with portcfgshow if the parameter Trunk Port is ON, else : portcfgtrunkport <port_number> 1
If you don't know the topology of your switch, the portbuffershow command list all ports on the same ASIC. If you want to create a trunk for example use port 0,1,2,3 on the following screen:
SWSAN1:admin> portbuffershow User Port Lx Max/Resv Buffer Needed Link Remaining Port Type Mode Buffers Usage Buffers Distance Buffers ---- ---- ---- ------- ------ ------- --------- ---------- 0 F - - 16 - - 1 - - 0 - - 2 - - 0 - - 3 F - - 16 - - 76 -------------------------------------------------------------- 4 F - - 16 - - 5 - - 0 - - 6 - - 0 - - 7 F - - 16 - - 76 -------------------------------------------------------------- 8 - - 0 - - 9 - - 0 - - 10 F - - 16 - - 11 F - - 16 - - 76 -------------------------------------------------------------- 12 F - - 16 - - 13 - - 0 - - 14 - - 0 - - 15 L - - 16 - - 76
SWSAN1:admin> portcfgshow Ports of Slot 0 0 1 2 3 16 17 18 19 20 21 22 23 24 25 26 27 ----------------+---+---+---+---+-----+---+---+---+----+---+---+---+-----+---+---+--- Speed AN AN AN AN AN AN 8G AN AN AN 16G 16G AN AN AN AN Fill Word 0 0 0 0 0 0 3 0 0 0 3 3 3 0 0 0 AL_PA Offset 13 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Trunk Port ON ON .. .. .. .. .. .. ON ON .. .. .. .. .. .. Long Distance .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. VC Link Init .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
There are 2 thrunk modes
- out-of-order delivery (OOD) : portcfgtrunkport 20 0
- in-order-delivery (IOD) : portcfgtrunkport 20 1
If multiple ISLs form a trunk, configure the fabric for IOD. If multiple ISLs cannot form a trunk, configure the fabric for OOD.
Show trunk usage
switch:admin> trunkshow -perf 1:105-> 27 10:00:d8:1f:cc:03:a5:43 10 deskew 0 E-Port MASTER Tx: Bandwidth 32.00Gbps, Capacity 27.20Gbps, Utilization 17.33Gbps (63.72%) Rx: Bandwidth 32.00Gbps, Capacity 27.20Gbps, Utilization 14.09Gbps (51.80%) Tx+Rx: Bandwidth 64.00Gbps, Capacity 54.40Gbps, Utilization 31.42Gbps (57.76%)
Long distance modes
Brocade has basically 3 long distance modes:
LE mode - merges all user-data virtual channels and assigns the amount of buffers necessary to cover a 10 km distance based on the full frame size for the given speed. It requires no license. LS mode - like LE mode, but is used for distances > 10 km and requires the "Extended Fabric License". You configure it with a fixed distance. LD mode - similar to LS mode, but the distance is measured automatically and the buffers are assigned according to the measured distance. You configure it with a "desired distance".
If you have two data centers with a distance of 30 km between them and you configure 60 km, the switch will only assign the buffers for the measured 30 km. Increasing the desired distance doesn't change anything.
Wait! Why should I increase it anyway? Use LS instead of LD!
LS mode gives you the full control. And use it with enough buffers by configuring a multiple of the physical distance. 3x is a good practice but you can increase it even more if there are buffers left. You can always check the available buffers with the command “portbuffershow”.
Example for port 5 assigned 250km
portcfglongdistance 5 LS 1 -distance 250
Steps to configure long distance port
On both site configure the E-Port of the SAN switch to LD (auto-detect) with a approximate distance
SWSAN1:admin> portcfglongdistance 5 LD 1 -distance 30
Use portbuffershow to print real distance mesearued
SWSAN1:admin> portbuffershow User Port Lx Max/Resv Buffer Needed Link Remaining Port Type Mode Buffers Usage Buffers Distance Buffers -------------------------------------------------------------- 5 E - - 30 40 10km 6 - - 0 - -
You can calculate required buffercredit (Example)
switch:admin> portbuffercalc 9/4 -distance 100 -speed 8 -framesize 512 1606 buffers required for 100km at 8G and framesize of 512 bytes
ISL settings for DWDM
Connect to a switch without password
Generate on your server a pair of ssh keys without password, and import them into brocade switch:
[root@nim] > ssh admin@brocade1 admin@brocade1's password: brocade1:admin> sshutil importpubkey Enter user name for whom key is imported:admin Enter IP address:10.10.2.144 Enter remote directory:/root/.ssh Enter public key name(must have .pub suffix):id_dsa.pub Enter login name:root root@10.10.2.144's password: public key is imported successfully. brocade1:admin> exit [root@nim] # ssh admin@brocade1 brocade1:admin> exit
Maximum number of simultaneous sessions
As root user change /etc/profile
# cd /etc # sed -e 's/max_telnet_sessions=2/max_telnet_sessions=1/' profile >profile.new # cp /etc/profile.new /etc/profile # cp /etc/profile.new /mnt/etc/profile
Check also the file limits.conf
# cat /etc/security/limits.conf | grep maxlo # for maxlogin limit # - maxlogins - max number of logins for this user #@student - maxlogins 4
Configuring SNMPv3 user/trap in Brocade Switch
http://himanshu-khare.blogspot.com/2015/04/configuring-snmpv3-usertrap-in-brocade.html?m=1
switch:admin> snmpconfig --set snmpv3 -index 4 -user user1 -auth_proto 2 -auth_passwd "password1" -priv_proto 3 -priv_passwd "password2" -engine_id "80:00:08:08:08:98:AB:5C:01"
Enter snmpconfig –set snmpv3 to create the SNMPv3 user.
switch:admin> snmpconfig --set snmpv3 SNMP Informs Enabled (true, t, false, f): [true] t SNMPv3 user configuration(snmp user not configured in FOS user database will have physical AD and admin role as the default): User (rw): [snmpadmin1] fabric_admin Auth Protocol [MD5(1)/SHA(2)/noAuth(3)]: (1..3) [3] 1 New Auth Passwd: Verify Auth Passwd: Priv Protocol [DES(1)/noPriv(2)/AES128(3)/AES256(4)]): (1..4) [2] 4 New Priv Passwd: Verify Priv Passwd: Engine ID: [00:00:00:00:00:00:00:00:00] 80:00:05:23:01:0A:xx:xx:xx User (rw): [snmpadmin2] Auth Protocol [MD5(1)/SHA(2)/noAuth(3)]: (1..3) [3] 1 New Auth Passwd: Verify Auth Passwd: Priv Protocol [DES(1)/noPriv(2)/AES128(3)/AES256(4)]): (1..4) [2] Engine ID: [00:00:00:00:00:00:00:00:00] User (rw): [snmpadmin3] Auth Protocol [MD5(1)/SHA(2)/noAuth(3)]: (1..3) [3] Priv Protocol [DES(1)/noPriv(2)/AES128(3)/AES256(4)]): (2..2) [2] Engine ID: [00:00:00:00:00:00:00:00:00] User (ro): [snmpuser1] Auth Protocol [MD5(1)/SHA(2)/noAuth(3)]: (1..3) [3] Priv Protocol [DES(1)/noPriv(2)/AES128(3)/AES256(4)]): (2..2) [2] Engine ID: [00:00:00:00:00:00:00:00:00] User (ro): [snmpuser2] Auth Protocol [MD5(1)/SHA(2)/noAuth(3)]: (1..3) [3] Priv Protocol [DES(1)/noPriv(2)/AES128(3)/AES256(4)]): (2..2) [2] Engine ID: [00:00:00:00:00:00:00:00:00] User (ro): [snmpuser2] Auth Protocol [MD5(1)/SHA(2)/noAuth(3)]: (1..3) [3] Priv Protocol [DES(1)/noPriv(2)/AES128(3)/AES256(4)]): (2..2) [2] Engine ID: [00:00:00:00:00:00:00:00:00] SNMPv3 trap recipient configuration: SNMPv3 trap recipient configuration: Trap Recipient's IP address : [0.0.0.0] xx.yy.zz.00 UserIndex: (1..6) [1] Trap recipient Severity level : (0..5) [0] 5 Trap recipient Port : (0..65535) [162] Trap Recipient's IP address : [0.0.0.0] xx.yy.zz.11 UserIndex: (1..6) [2] Trap recipient Severity level : (0..5) [0] 5 Trap recipient Port : (0..65535) [162] Trap Recipient's IP address : [0.0.0.0] Trap Recipient's IP address : [0.0.0.0] Trap Recipient's IP address : [0.0.0.0] Trap Recipient's IP address : [0.0.0.0] Committing configuration.....done. switch:admin> snmpconfig --show snmpv3 SNMP Informs = 1 (ON) SNMPv3 USM configuration: User 1 (rw): fabric_admin Auth Protocol: MD5 Priv Protocol: AES256 Engine ID: 80:00:05:23:01:0A:xx:xx:xx User 2 (rw): snmpadmin2 Auth Protocol: MD5 Priv Protocol: noPriv Engine ID: 00:00:00:00:00:00:00:00:00 User 3 (rw): snmpadmin3 Auth Protocol: noAuth Priv Protocol: noPriv Engine ID: 00:00:00:00:00:00:00:00:00 User 4 (ro): snmpuser1 Auth Protocol: noAuth Priv Protocol: noPriv Engine ID: 00:00:00:00:00:00:00:00:00 User 5 (ro): snmpuser2 Auth Protocol: noAuth Priv Protocol: noPriv Engine ID: 00:00:00:00:00:00:00:00:00 User 6 (ro): admin Auth Protocol: noAuth Priv Protocol: noPriv Engine ID: 00:00:00:00:00:00:00:00:00 SNMPv3 Trap configuration: Trap Entry 1: xx.yy.zz.00 Trap Port: 162 Trap User: fabric_admin Trap recipient Severity level: 5 Trap Entry 2: xx.yy.zz.11 Trap Port: 162 Trap User: snmpadmin2 Trap recipient Severity level: 5 Trap Entry 3: No trap recipient configured yet Trap Entry 4: No trap recipient configured yet Trap Entry 5: No trap recipient configured yet Trap Entry 6: No trap recipient configured yet
To display the test traps associated with the real MIBs supported in Fabric OS:
switch:admin> snmpTraps --show # |Mib Name |Supported Traps ---|----------------|-------------------------------- 001|SW-MIB |sw-track-changes-trap | |sw-fabric-watch-trap | |sw-fc-port-scn | |ip-v6-change-trap | |sw-pmgr-event-trap | |sw-event-trap | |sw-fabric-reconfig-trap | |sw-fabric-segment-trap | |sw-state-change-trap | |sw-zone-config-change-trap | |sw-port-move-trap | |sw-brcd-generic-trap | |sw-device-status-trap 002|FICON-MIB |link-rnid-device-registration | |link-rnid-device-deregistration | |link-lirr-listener-added | |link-lirr-listener-removed | |link-rlir-failure-incident 003|FA-MIB |conn-unit-status-change | |conn-unit-sensor-status-change | |conn-unit-port-status-change | |conn-unit-event-trap 004|MIB-2 |cold-restart-trap | |warm-restart-trap 005|IF-MIB |if-link-up-trap | |if-link-down-trap 006|RFC1157 |snmp-authetication-trap 007|HA-MIB |fru-status-change-trap | |fru-history-trap | |cp-status-change-trap 008|BD-MIB |bd-trap | |bd-clear-trap 009|T11-FC-ZONE-SERVER-MIB|t11ZsRequestRejectNotify | |t11ZsMergeSuccessNotify | |t11ZsMergeFailureNotify | |t11ZsDefZoneChangeNotify | |t11ZsActivateNotify
To send all traps to the configured recipients:
switch:admin> snmpTraps --send Number of traps sent : 30
To send all traps to the recipient xx:yy:zz:00:
switch:admin> snmpTraps --send -ip_address xx:yy:zz:00 Number of traps sent : 30
To send the sw-fc-port-scn trap to the configured recipients:
switch:admin> snmpTraps --send -trap_name sw-fc-port-scn Number of traps sent : 1
To send the sw-fc-port-scn trap to the recipient xx:yy:zz:00:
switch:admin> snmpTraps --send -trap_name sw-fc-port-scn -ip_address xx:yy:zz:00 Number of traps sent : 1
To unblock port traps on all the ports or on a specific port:
switch:admin> snmptraps --unblock -ports ALL switch:admin> snmptraps --unblock -port 1/10
To block port traps on slot 1 and port 10:
Switch:admin> snmptraps --block -port 1/10
Configuring principal switch
Principal switch (master) is responsible of
- time synchronization in the fabric
- maintain domain ID in the fabric
Who is the master ?
- use fabricshow, master is shown with '>'
- principal is the switch with higher priority
- if same priority, then the lowest wwn is used to define the principal
You can change the priority using
fabricprincipal --show fabricprincipal --enable -priority 0x01
You can disable the ports on principal en enable to activate, or use -force option (diruptive)
Priority acn be set to 0x01, 0x03-0xFE (0x02 is a reserved value, do not use)
Database size
san01:FID128:admin> cfgsize Chassis-Wide Max Zone DB size - 16777216 bytes Chassis-Wide Committed Zone DB size - 167888 bytes Current Logical Switch Max Zone DB size - 4194304 bytes Fabric-Wide Max Zone DB size - (Domain 121) 1045274 bytes Available Zone DB size - 878937 bytes Current Logical switch zone config sizes: committed - 166337 transaction - 0 Compressed sizes: committed - 21306 transaction - 0
RestAPI configuration
https://www.ibm.com/support/pages/planning-adding-brocade-switches-using-rest-api
RestAPI use the port 443 (https)
RestAPI user configuration
Check your user rights depending on usage, maybe it needs an access to all resources
san01:FID128:admin> userconfig --show monitor Account name: monitor Description: monitoring admin Enabled: Yes Password Last Change Date: Thu Sep 15 2016 (UTC) Password Expiration Date: Not Applicable (UTC) Locked: No Home LF Role: admin Role-LF List: admin: 1-128 Chassis Role: admin Home LF: 128 Day Time Access: N/A
Certificate configuration for https
Add a certificat for https, mandatory!
san01:FID128:admin> seccertmgmt generate -cert https -type rsa -keysize 2048 -hash sha1 -years 5 Generating a new certificate will automatically do the following 1. Delete existing switch certificate(s). 2. Disable secure protocol HTTPS Warning: Certificate generation is CPU intensive and can cause high CPU usage Continue (yes, y, no, n): [no] y Generating ... ...Generated self-signed https certificate successfully. san01:FID128:admin> seccertmgmt show -cert https Issued To countryName = US stateOrProvinceName = California localityName = San Jose organizationName = Brocade organizationalUnitName = Eng commonName = 10.1X.XX.XX Issued By countryName = US stateOrProvinceName = California localityName = San Jose organizationName = Brocade organizationalUnitName = Eng commonName = 10.1X.XX.XX Period Of Validity Begins On May 5 13:34:40 2023 GMT Expires On May 3 13:34:40 2028 GMT Fingerprints SHA1 Fingerprint 4B:AC:49:9E:8A:F8:57:DB:45:xx:xx:xx:xx:F:2B:A3 SHA256 Fingerprint FD:4F::xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:76:CC:8E:F7:E3:78:44:88 san01:FID128:admin> seccertmgmt show -all ssh private key: Does not Exist ssh public keys available for users: report Certificate Files: -------------------------------------------------------------------------------------------------------------------- Protocol Client CA Server CA SW CSR PVT Key Passphrase -------------------------------------------------------------------------------------------------------------------- FCAP Empty NA Empty Empty Empty Empty RADIUS Empty Empty Empty Empty Empty NA LDAP Empty Empty Empty Empty Empty NA SYSLOG Empty Empty Empty Empty Empty NA HTTPS NA Empty Exist Empty Exist NA KAFKA NA Empty NA NA NA NA ASC NA Empty NA NA NA NA
You can also import a custom certificate.
Ex: seccertmgmt import -cert https -protocol scp -ipaddr 10.xx.xx.xx -remotedir /home/localpath/CERT -certname 92.pem -login admin
RestAPI Activation
Check if RestAPI is enabled, else change it:
san01:FID128:admin> mgmtapp --show REST Interface State: Enabled REST Session Count: 3 REST Throttling Configurations: Sample Requests : 30 Sample Time (in sec) : 30 Idle Time (in sec) : 3 KeepAlive : Enabled KeepAliveTimeout : 15sec
Usage: ------ mgmtapp --config <configuration parameters> mgmtapp --terminate <session id> mgmtapp --enable <rest | keepalive> mgmtapp --disable <rest | keepalive> mgmtapp --unbind mgmtapp --show Configuration parameters: ------------------------- -maxrestsession <rest session count> -sampletime <sampling time in sec> -samplerequest <sampling request count> -idletime <throttling time in sec>
Hardware info on SAN switch
Serial number, switch type (here an example with Brocade gen7 G720 –> type 181, on firmware package you'll find a folder SWBD181) ….
DCX01:FID128:admin> switchshow switchName: ssw-green02 switchType: 181.0 switchState: Online switchMode: Native ...
DCX01:FID128:admin> chassisshow ... CHASSIS/WWN Unit: 1 System AirFlow: Non-portside Intake Header Version: 2 Factory Part Num: 40-1001115-44 Factory Serial Num: FME193xxxxJ Manufacture: Day: 13 Month: 5 Year: 2023 Update: Day: 12 Month: 9 Year: 2024 Time Alive: 1821 hours Time Awake: 11717 hours ID: IBM0000CA Part Num: 0089600000P64 Serial Num: 786xxxx
Hardware info on FC ports
DCX01:FID128:admin> nscamshow -t nscam show for remote switches: Switch entry for 21 state rev owner cap_available known v823 0xfffc0b 1 Device list: count 119 Type Pid COS PortName NodeName N 150400; 2,3;30:19:50:eb:1a:bb:6c:fd;10:00:50:eb:1a:bb:6c:fd; FC4s: FCP Fabric Port Name: 20:08:50:eb:1a:88:bc:40 Permanent Port Name: 30:19:50:eb:1a:bb:6c:fd Device type: Physical Initiator+Target Port Index: 4 Share Area: No Redirect: No Partial: No Slow Drain Device: No Device Link speed: 16G Connected through AG: No Real device behind AG: No FCoE: No FC4 Features [FCP]: Initiator Target N 156108; 3;10:00:f6:66:78:00:00:94;10:00:f6:66:78:00:00:95; FC4s: FCP FC-NVMe PortSymb: [34] "Emulex PPN-10:00:f6:66:78:00:00:94" NodeSymb: [72] "Emulex 5330C FV14.2.589.21 DV14.2.567.0 HN:sptv242l OS:VMware ESXi 7.0.3" Fabric Port Name: 20:61:00:33:f8:1c:8d:04 Permanent Port Name: 20:19:00:33:38:c8:c3:1c Device type: NPIV Initiator Port Index: 97 Share Area: No Redirect: No Partial: No Slow Drain Device: No Device Link speed: 16G Connected through AG: Yes Real device behind AG: Yes FCoE: No FC4 Features [FCP]: Initiator FC4 Features [FC-NVMe]: Initiator
DCX01:FID128:admin> fdmishow 51:40:2e:c0:33:2a:db:0e Ports: 1 51:40:2e:c0:33:2a:db:0e Port attributes: FC4 Types: FCP Supported Speed: 4 8 16 Gb/s Port Speed: 16 Gb/s Max Frame Size: 2048 bytes Device Name: VMHBA Host Name: esx01 Node Name: 51:40:2e:c0:33:2a:db:0f Port Name: 51:40:2e:c0:33:2a:db:0e Port Type: Nx_Port (0x7f) Port Symb Name: SN1100Q FW:v9.14.00 DVR:v5.3.81.2 port Class of Service: 3 Fabric Name: 20:a5:00:27:33:1c:8b:04 FC4 Active Type: FCP Port State: 0x2 Discovered Ports: 0x1 Port Identifier: 0x0ba540 HBA attributes: Node Name: 51:40:2e:c0:33:2a:db:0f Manufacturer: Marvell Semiconductor Inc Serial Number: 3Y213337YJ Model: SN1100Q Model Description: HPE SN1100Q 16Gb 2p FC HBA Hardware Version: BK3210407-20 J Driver Version: 5.3.81.2 Option ROM Version: 7.25 Firmware Version: 9.14.00 (d0d5) OS Name and Version: VMware ESXi-7.0.3 (Releasebuild-23307199) Max CT Payload Length: 512 words Symbolic Name: SN1100Q FW:v9.14.00 DVR:v5.3.81.2 Vendor Info: 0x00001077 Number of Ports: 1 Fabric Name: 10:00:00:27:33:1c:8b:04 Bios Version: BIOS 3.68 Vendor Identifier: QLOGIC
configuring security policies
- Fabric configuration server policy (FCS): Restricts which switches can change the configuration of the fabric.
- Device connection control (DCC) policy: Restricts which Fibre Channel device ports can connect to which Fibre Channel switch ports.
- Switch connection control (SCC) policy: Restricts which switches can join with a switch.
Displaying the Database Distribution Settings
switch:admin> fddcfg --showall Local Switch Configuration for all Databases:- DATABASE - Accept/Reject --------------------------------- SCC - accept DCC - accept PWD - accept FCS - accept AUTH - accept IPFILTER - accept Fabric Wide Consistency Policy:- ""
Enable Database Distribution Settings
switch:admin> fddcfg --fabwideset "SCC" switch:admin> fddcfg --showall Local Switch Configuration for all Databases:- DATABASE - Accept/Reject --------------------------------- SCC - accept DCC - accept PWD - accept FCS - accept AUTH - accept IPFILTER - accept Fabric Wide Consistency Policy:- "SCC"
Best way to configure SCC is to first enable all switches to join the fabric:
On each SAN switch you can list the WWN:
switch:admin> wwn
Do either of the following:
- Manually add the front domain switch WWN to the SCC policy, or the Domain ID, or name.
switch:admin> secpolicycreate SCC_POLICY "WWA;WWB"
- Use the command to automatically add all switches in the fabric
switch:admin> secpolicycreate SCC_POLICY "*"
List the policy settings
switch:admin> secpolicyshow ___________________________________________________ ACTIVE POLICY FCS_POLICY Pos Primary WWN DId swName __________________________________________________ 1 Yes 10:00:00:60:69:30:15:5c 1 primaryfcs 2 No 10:00:00:60:69:30:1e:62 4 switch ____________________________________________________
Activate and distribute the SCC policy.
switch:admin> secpolicysave switch:admin> secpolicyactivate
distribute -p <policy_list> -d <switch_list>
switch:admin> distribute -p "SCC;DCC" -d "3;5"
Or
switch:admin> distribute -p "FCS;PWD" -d "*"
configuring crypto policies
switch:admin> setcryptocfg --show SSH Crypto: SSH Cipher : aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc SSH Kex : ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1 SSH MAC : hmac-sha1,hmac-sha2-256,hmac-sha2-512 TLS Ciphers: HTTPS : ECDSA:ECDH:RSA:AES:3DES:!RSAPSK:!DHEPSK:!PSK:!DSS:!AESCCM8:!AESCCM:!ARIAGCM:!CAMELLIA:!CHACHA20:!SEED:!RC4 HTTPS_TLS_v1.3 : TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256 RADIUS : ECDSA:ECDH:RSA:AES:3DES:!RSAPSK:!DHEPSK:!PSK:!DSS:!AESCCM8:!AESCCM:!ARIAGCM:!CAMELLIA:!CHACHA20:!SEED:!RC4 LDAP : ECDSA:ECDH:RSA:AES:3DES:!RSAPSK:!DHEPSK:!PSK:!DSS:!AESCCM8:!AESCCM:!ARIAGCM:!CAMELLIA:!CHACHA20:!SEED:!RC4 SYSLOG : ECDSA:ECDH:RSA:AES:3DES:!RSAPSK:!DHEPSK:!PSK:!DSS:!AESCCM8:!AESCCM:!ARIAGCM:!CAMELLIA:!CHACHA20:!SEED:!RC4 RSA : ECDSA:ECDH:RSA:AES:!3DES:!RSAPSK:!DHEPSK:!PSK:!DSS:!ARIAGCM:!CAMELLIA:!CHACHA20:!SSLv3:!TLSv1:!AESCCM FA : ECDSA:ECDH:RSA:AES:!3DES:!RSAPSK:!DHEPSK:!PSK:!DSS:!ARIAGCM:!CAMELLIA:!CHACHA20:!SSLv3:!TLSv1:!AESCCM TLS Protocol: HTTPS : Any RADIUS : Any LDAP : Any SYSLOG : Any RSA : TLSv1.2 FA : TLSv1.2 X509v3: Validation : Basic Compliance: CryptoVersion : 9.2.1 FIPS Inside : Disabled BootUp Selftests : Disabled switch:admin> seccryptocfg --lstemplates List of templates: default_cc default_generic default_strong default_fips switch:admin> seccryptocfg --show default_strong [Ver] 0.2 [SSH] Enc:aes128-ctr,aes192-ctr,aes256-ctr Kex:ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,curve25519-sha256 Mac:hmac-sha2-256,hmac-sha2-512 [AAA] RAD_Ciphers:ECDSA:ECDH:RSA:AES:!3DES:!RSAPSK:!DHEPSK:!PSK:!DSS:!ARIAGCM:!CAMELLIA:!CHACHA20:!SSLv3:!TLSv1:!AESCCM LDAP_Ciphers:ECDSA:ECDH:RSA:AES:!3DES:!RSAPSK:!DHEPSK:!PSK:!DSS:!ARIAGCM:!CAMELLIA:!CHACHA20:!SSLv3:!TLSv1:!AESCCM RAD_Protocol:TLSv1.2 LDAP_Protocol:TLSv1.2 [LOG] Syslog_Ciphers:ECDSA:ECDH:RSA:AES:!3DES:!RSAPSK:!DHEPSK:!PSK:!DSS:!ARIAGCM:!CAMELLIA:!CHACHA20:!SSLv3:!TLSv1:!AESCCM Syslog_Protocol:TLSv1.2 [HTTPS] Ciphers:ECDSA:ECDH:RSA:AES:!3DES:!RSAPSK:!DHEPSK:!PSK:!DSS:!ARIAGCM:!CAMELLIA:!CHACHA20:!SSLv3:!TLSv1:!AESCCM Protocol:TLSv1.3 Ciphers_tlsv1.3:TLS_AES_256_GCM_SHA384:TLS_AES_128_GCM_SHA256:TLS_AES_128_CCM_8_SHA256:TLS_AES_128_CCM_SHA256 [X509v3] Validation:Basic
