====== PureFlex with FCoE switches ======
===== Firmware Upgrade on FCoE switch =====
Be careful, if you have a Lenovo (CMM2) or an IBM (CMM) then the firmware is different, don't go higher than 7.x with IBM.
1) Download the CN4093 software package (zip file) to the machine running the browser application.
2) Extract the Boot and OS image files into a directory.
3) Establish a connection between the Ethernet port of the Chassis Management Module (CMM) and the machine running the browser application. https://fcoe_sw_ip
4) Point your browser to the IP address of the ScSE and login to the Browser-based user interface (BBI):
(default) username = admin (or USERID)
(default) password = admin (or PASSW0RD)
a) select the CONFIGURE tab
b) select the IBM Flex System CN4093 10Gb ScSE folder
c) select the System folder
d) select the Config/Image Control page
e) scroll down to the "Image Setting" box
- on the "Image for Transfer" dropdown menu, select the OS image bank
(1 or 2) that is NOT the Active Image Selection.
NOTE: It is STRONGLY recommended that you retain the previous OS version by loading the upgrade into the non-active image block.
- click on the Browse button, navigate your local file system to select the GbScSE-10G-FC-7.8.11.0_OS.img file.
- click the "Download via Browser" button
NOTE: at this point the file transfer begins, followed by flashing non-volatile memory on the ScSE.
f) when the operation completes (about 2-3 minutes), the browser window returns and you will see the following message at the bottom of the page:
Status of Previous Transfer ...
... Image downloaded via Browser GbScSE-10G-FC-7.8.11.0_OS.img - Successful
g) scroll down to the "Image Setting" box
- on the "Image for Transfer" dropdown menu, select the Boot image bank
- click on the Browse button, navigate your local file system to select the GbScSE-10G-FC-7.8.11.0_Boot.img file.
- click the "Download via Browser" button
NOTE: at this point the file transfer begins, followed by flashing non-volatile memory on the ScSE.
h) when the operation completes, the browser window returns and you will see the following message at the bottom of the page:
Status of Previous Transfer ...
... Image downloaded via Browser GbScSE-10G-FC-7.8.11.0_Boot.img - Successful
i) set the Next Boot Image Selection to the image bank (1 or 2) that contains the 7.8.11.0 firmware
NOTE: this action may trigger an internal FC Firmware update (if required).
j) click on the Submit button at the bottom of the page
k) click on the REBOOT! button at the bottom of the page
l) wait for switch to reboot
===== Configure FCoE switch CN4094 for FC part =====
**Be careful:** some VLAN Id doesn't work for FC (firmware 7.8.11)
tested and working : 1000, 1002, 1024, 1025, 2001, 4001, 4002...
not working : 4000...
to see NPIV ports connected, you have to connect fiber from ports EXT17, and EXT18 to a FC SAN switch:
swfcoe-dc1-1#sh interface status
-----------------------------------------------------------------------
Alias Port Speed Duplex Flow Ctrl Link Description
------- ---- ----- -------- --TX-----RX-- ------ -------------
INTA1 1 1G/10G full no no down INTA1
INTA2 2 10000 full no no up INTA2
INTA3 3 10000 full no no up INTA3
INTA4 4 10000 full no no up INTA4
INTA5 5 10000 full no no up INTA5
INTA6 6 10000 full no no up INTA6
INTA7 7 10000 full no no up INTA7
INTA8 8 10000 full no no up INTA8
INTA9 9 10000 full no no up INTA9
INTA10 10 10000 full no no up INTA10
INTA11 11 10000 full no no up INTA11
INTA12 12 1G/10G full no no down INTA12
INTA13 13 1G/10G full no no down INTA13
INTA14 14 1G/10G full no no down INTA14
INTB1 15 1G/10G full no no down INTB1
INTB2 16 1G/10G full no no down INTB2
INTB3 17 10000 full no no up INTB3
INTB4 18 10000 full no no up INTB4
INTB5 19 10000 full no no up INTB5
INTB6 20 10000 full no no up INTB6
INTB7 21 10000 full no no up INTB7
INTB8 22 10000 full no no up INTB8
INTB9 23 10000 full no no up INTB9
INTB10 24 10000 full no no up INTB10
INTB11 25 1G/10G full no no down INTB11
INTB12 26 1G/10G full no no down INTB12
INTB13 27 1G/10G full no no down INTB13
INTB14 28 1G/10G full no no down INTB14
EXT1 43 1000 full no no up EXT1
EXT2 44 1000 full no no down EXT2
EXT11 53 10000 full no no down EXT11
EXT12 54 10000 full no no down EXT12
EXT13 55 10000 full no no down EXT13
EXT14 56 10000 full no no down EXT14
EXT15 57 10000 full no no down EXT15
EXT16 58 10000 full no no down EXT16
EXT17 59 Auto full no no up EXT17
EXT18 60 Auto full no no down EXT18
EXT19 61 10000 full no no down EXT19
EXT20 62 10000 full no no down EXT20
EXT21 63 10000 full no no down EXT21
EXT22 64 10000 full no no down EXT22
EXTM 65 any auto yes yes down EXTM
MGT1 66 1000 full no no up MGT1
swfcoe-dc1-1#sh npv flogi-table
--------------------------------------------------------------------------
VLAN Port WWN MAC Login
--------------------------------------------------------------------------
4010 INTA2 10:00:00:90:fa:a5:6f:a1 0e:fc:00:0a:00:01 FLOGI
4010 INTB4 10:00:00:90:fa:a8:2a:3d 0e:fc:00:0a:00:02 FLOGI
4010 INTB7 10:00:00:90:fa:a6:22:79 0e:fc:00:0a:00:03 FLOGI
4010 INTB3 10:00:00:90:fa:a6:72:45 0e:fc:00:0a:00:04 FLOGI
4010 INTB5 10:00:00:90:fa:a8:29:0d 0e:fc:00:0a:00:05 FLOGI
swfcoe-dc1-1#sh npv status
Unit:0 VLAN: 4010 NPV enabled
swfcoe-dc1-1#sh npv traffic-map
-------------------------------------------------------------------
VLAN Source Ports NP-Uplink Dest Ports
-------------------------------------------------------------------
4010 empty EXT17,EXT18
swfcoe-dc1-1#sh fcoe database
-----------------------------------------------------------------------
VLAN FCID WWN MAC Port
-----------------------------------------------------------------------
4010 0a0001 10:00:00:90:fa:a6:72:45 0e:fc:00:0a:00:01 INTB3
4010 0a0002 10:00:00:90:fa:a8:29:0d 0e:fc:00:0a:00:02 INTB5
4010 0a0003 10:00:00:90:fa:a5:6f:a1 0e:fc:00:0a:00:03 INTA2
4010 0a0004 10:00:00:90:fa:a6:22:79 0e:fc:00:0a:00:04 INTB7
4010 0a0005 10:00:00:90:fa:a8:2a:3d 0e:fc:00:0a:00:05 INTB4
Total number of entries = 5.
-----------------------------------------------------------------------
swfcoe-dc1-1#sh fcoe fips
FIP snooping current configuration:
FIP snooping is currently enabled.
swfcoe-dc1-1#sh fcoe fips fcoe
Total number of FCoE connections: 4
VN_PORT MAC FCF MAC Port Vlan
------------------------------------------------------
0e:fc:00:0a:00:01 a8:97:dc:69:cc:c9 INTB3 4010
0e:fc:00:0a:00:05 a8:97:dc:69:cc:c9 INTB4 4010
0e:fc:00:0a:00:02 a8:97:dc:69:cc:c9 INTB5 4010
0e:fc:00:0a:00:04 a8:97:dc:69:cc:c9 INTB7 4010
swfcoe-dc1-1#sh run
Current configuration:
!
version "7.8.11"
switch-type "IBM Flex System Fabric CN4093 10Gb Converged Scalable Switch(Upgrade2)"
iscli-new
!
system timezone 308
! Europe/Luxembourg
system daylight
!
snmp-server name "swfcoe-dc1-1"
!
!
system port EXT17,EXT18 type fc
interface fc EXT17
switchport trunk allowed vlan 1,4010
interface fc EXT18
switchport trunk allowed vlan 1,4010
!
hostname "swfcoe-dc1-1"
system idle 60
!
!
access user 1 password "xxxxxxxxxxxxxxxxxxxxxx"
!
interface port INTA2
switchport mode trunk
switchport trunk allowed vlan 1,4010
exit
!
interface port INTB1
switchport mode trunk
switchport trunk allowed vlan 1,4010
exit
!
interface port INTB2
switchport mode trunk
switchport trunk allowed vlan 1,4010
exit
!
interface port INTB3
switchport mode trunk
switchport trunk allowed vlan 1,4010
exit
!
interface port INTB4
switchport mode trunk
switchport trunk allowed vlan 1,4010
exit
!
interface port INTB5
switchport mode trunk
switchport trunk allowed vlan 1,4010
exit
!
interface port INTB6
switchport mode trunk
switchport trunk allowed vlan 1,4010
exit
!
interface port INTB7
switchport mode trunk
switchport trunk allowed vlan 1,4010
exit
!
interface port INTB8
switchport mode trunk
switchport trunk allowed vlan 1,4010
exit
!
interface port INTB9
switchport mode trunk
switchport trunk allowed vlan 1,4010
exit
!
interface port INTB10
switchport mode trunk
switchport trunk allowed vlan 1,4010
exit
!
interface port INTB11
switchport mode trunk
switchport trunk allowed vlan 1,4010
exit
!
interface port INTB12
switchport mode trunk
switchport trunk allowed vlan 1,4010
exit
!
interface port INTB13
switchport mode trunk
switchport trunk allowed vlan 1,4010
exit
!
interface port INTB14
switchport mode trunk
switchport trunk allowed vlan 1,4010
exit
!
vlan 4010
name "SAN FCOE"
npv enable
npv traffic-map external-interface EXT17,EXT18
!
spanning-tree stp 113 vlan 4010
!
no logging console
!
fcoe fips enable
!
cee enable
!
ntp enable
ntp ipv6 primary-server fe80::211:25ff:fec3:1ef2 MGT
ntp interval 15
ntp authenticate
ntp primary-key 44461
!
ntp message-digest-key 44461 md5-ekey "1273dcde000xxxxxxxxxxxxxxx1e27be"
!
ntp trusted-key 44461
!
end
=== Full commands to use NPV (NPIV) ===
Convert switch to Cisco compatibility:\\
ssh USERID@swfcoe
boot/mode iscli-new
save
reset
Then set the Time Zone (Europe/Luxembourg) and hostname
enable
conf t
system timezone 308
system daylight
hostname swfcoe-dc1-1
end
write
Enable the cee and fcoe feature required for FCoE and NPV:
conf t
cee enable
fcoe fips enable
end
write
Set the external omiports EXT17 and EXT18 as FC ports, and enable npv on vlan ID 4010:
conf t
vlan 4010
name "SAN FCOE"
system port EXT17-EXT18 type fc
interface fc EXT17-EXT18
switchport trunk allowed vlan 1,4010
vlan 4010
npv enable
npv traffic-map external-interface EXT17,EXT18
end
write
Put internal ports B into the VLAN 4010, to see NPIV ports connected, you have to connect fiber from ports EXT17, and EXT18 to a FC SAN switch:
conf t
interface port INTB1-INTB14
switchport mode trunk
switchport trunk allowed vlan 1,4010
end
write
sh npv flogi
sh npv status