Cisco Nexus can be very temperamental or capricious (pick the one you prefer 🙂 ) and the vPC technology is not an isolated case.
There is a certain way to configure vPC and we will see that in that blogpost.
The following topology will be used:
Enabling the feature
Obviously we need to activate the vPC and LACP feature in order to build a proper vPC configuration.
|
1 2 3 4 5 |
N5K-1(config)# feature vpc N5K-1(config)# feature fex N5K-2(config)# feature vpc N5K-2(config)# feature fex |
Peer Keepalive connectivity
The Peer Keepalive link is used to detect failure between the peers. It is definitely not used in the data plane. On N5K, the management interface is usually used for the Peer Keepalive Link and management purpose. On N7K, Cisco does not recommend to use the supervisor interface as a vPC peer-keepalive link since it can introduce failure if that supervisor crashes…..
In our example we will use the management interface in order to have IPÂ connectivity between our vPC peers.
|
1 2 3 4 5 6 7 |
N5K-1(config)# int mgmt 0 N5K-1(config-if)# ip address 10.2.8.83/24 N5K-1(config-if)# no shut N5K-2(config)# int mgmt 0 N5K-2(config-if)# ip address 10.2.8.84/24 N5K-2(config-if)# no shut |
Now let’s check if N5K-1 can reach N5K-2
|
1 2 3 4 5 6 7 8 9 10 |
N5K-1(config)# ping 10.2.8.84 PING 10.2.8.84 (10.2.8.84): 56 data bytes ping: sendto 10.2.8.84 64 chars, No route to host ping: sendto 10.2.8.84 64 chars, No route to host ping: sendto 10.2.8.84 64 chars, No route to host ping: sendto 10.2.8.84 64 chars, No route to host ping: sendto 10.2.8.84 64 chars, No route to host ^C --- 10.2.8.84 ping statistics --- 5 packets transmitted, 0 packets received, 100.00% packet loss |
The behaviour above is normal since we are trying to ping 10.2.8.84 using the global RIB and the mgmt 0 interface does not belong to that RIB. Instead it does belong to the management vrf . So if we add the “vrf management”  keywords, everything should be fine !
|
1 2 3 4 5 6 7 8 9 10 11 |
N5K-1(config)# ping 10.2.8.84 vrf management PING 10.2.8.84 (10.2.8.84): 56 data bytes 64 bytes from 10.2.8.84: icmp_seq=0 ttl=254 time=1.14 ms 64 bytes from 10.2.8.84: icmp_seq=1 ttl=254 time=1.14 ms 64 bytes from 10.2.8.84: icmp_seq=2 ttl=254 time=0.538 ms 64 bytes from 10.2.8.84: icmp_seq=3 ttl=254 time=0.532 ms 64 bytes from 10.2.8.84: icmp_seq=4 ttl=254 time=0.522 ms --- 10.2.8.84 ping statistics --- 5 packets transmitted,5 packets received, 0.00% packet loss round-trip min/avg/max = 0.522/0.682/1.14 ms |
and it is 🙂
vPC domain and vPC peer link configuration
Now let’s create the vPC domain and the vPC peer link.
Please be aware that it is a best practice to configure an unique vpc domain ID for every pair of Nexus. The vPC domain ID will be used to build a system mac-address. So if 4 nexus are connected together created 2 vPC domains and if both vPC domains have the same system mac-address, you will experience something funny 🙂
I decided that N5K-1 will be elected as the Primary vPC peer and N5K-2 will remain Secondary vPC peer.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
N5K-1(config)# vpc domain 100 N5K-1(config-vpc-domain)# role priority 1 Warning: !!:: vPCs will be flapped on current primary vPC switch while attempting role change ::!! Note: --------:: Change will take effect after user has re-initd the vPC peer-link ::-------- N5K-1(config-vpc-domain)# peer-keepalive destination 10.2.8.84 Note: --------:: Management VRF will be used as the default VRF ::-------- N5K-2(config)# vpc domain 100 N5K-2(config-vpc-domain)# role priority 2 Warning: !!:: vPCs will be flapped on current primary vPC switch while attempting role change ::!! Note: --------:: Change will take effect after user has re-initd the vPC peer-link ::-------- N5K-2(config-vpc-domain)# peer-keepalive destination 10.2.8.83 Note: --------:: Management VRF will be used as the default VRF ::-------- N5K-2(config-vpc-domain)# |
Please note that if you do not specify a keyword for the peer-keepalive destination command, the switch automatically use the management vrf.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
N5K-1# sh vpc Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 100 Peer status : peer link not configured vPC keep-alive status : peer is alive Configuration consistency status : failed Per-vlan consistency status : failed Configuration inconsistency reason: vPC peer-link does not exist Type-2 consistency status : failed Type-2 inconsistency reason : vPC peer-link does not exist vPC role : none established Number of vPCs configured : 0 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Disabled (due to peer configuration) Auto-recovery status : Disabled |
As we can see above, the vPC keep-alive status is OK since we have reachability between our mgmt 0 interface.
We will now create our vPC peer-link that will be mainly used for CSF and many other things (will be detailed in a future blogpost)
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
N5K-1(config)# int eth1/17-18 N5K-1(config-if-range)# shut N5K-1(config-if-range)# channel-group 100 mode active N5K-1(config-if-range)# exit N5K-1(config)# N5K-1(config)# int po100 N5K-1(config-if)# switchport mode trunk N5K-1(config-if)# switch trunk allow vlan all N5K-1(config-if)# vpc peer-link Please note that spanning tree port type is changed to "network" port type on vPC peer-link. This will enable spanning tree Bridge Assurance on vPC peer-link provided the STP Bridge Assurance (which is enabled by default) is not disabled. N5K-2(config-vpc-domain)# int eth1/17-18 N5K-2(config-if-range)# shut N5K-2(config-if-range)# channel-group 100 mode active N5K-2(config-if-range)# exit N5K-2(config-if)# int po100 N5K-2(config-if)# switchport mode trunk N5K-2(config-if)# switchport trunk allow vlan all N5K-2(config-if)# vpc peer-link Please note that spanning tree port type is changed to "network" port type on vPC peer-link. This will enable spanning tree Bridge Assurance on vPC peer-link provided the STP Bridge Assurance (which is enabled by default) is not disabled. |
Enabling the vPC peer-link on the port-channel automatically set the type to network for that interface. This means that we just enabled bridge assurance on that link. In a vPC topology, Cisco recommends to enable Bridge Assurance only on the vPC peer-link.
We can now enable the interface to check the status of our port-channel.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
N5K-1(config-if)# int eth1/17-18 N5K-1(config-if-range)# no shut N5K-2(config-if)# int eth1/17-18 N5K-2(config-if-range)# no shut N5K-1# show port-channel summary Flags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed S - Switched R - Routed U - Up (port-channel) M - Not in use. Min-links not met -------------------------------------------------------------------------------- Group Port- Type Protocol Member Ports -------------------------------------------------------------------------------- 100 Po100(SU) Eth LACP Eth1/17(P) Eth1/18(P) |
Our Port-channel is functionning properly so we can check the status of our vPC peer link now
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 |
vPC domain id : 100 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : primary Number of vPCs configured : 0 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Disabled vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ -------------------------------------------------- 1 Po100 up 1 N5K-1# N5K-1# N5K-1# N5K-1# N5K-1# show vpc role vPC Role status ---------------------------------------------------- vPC role : primary Dual Active Detection Status : 0 vPC system-mac : 00:23:04:ee:be:64 vPC system-priority : 32667 vPC local system-mac : 00:05:73:ca:f7:01 vPC local role-priority : 1 N5K-2# show vpc Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 100 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : secondary Number of vPCs configured : 0 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Disabled vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ -------------------------------------------------- 1 Po100 up 1 N5K-2# show vpc role vPC Role status ---------------------------------------------------- vPC role : secondary Dual Active Detection Status : 0 vPC system-mac : 00:23:04:ee:be:64 vPC system-priority : 32667 vPC local system-mac : 00:05:73:b2:0e:bc vPC local role-priority : 2 |
Everything looks so far so good !
Do you remember when I explain about the dependencies between vPC domain and vPC system mac-address ? 🙂 here is the result
|
1 2 3 4 5 6 7 |
N5K-1# show vpc role | inc system-mac vPC system-mac : 00:23:04:ee:be:64 vPC local system-mac : 00:05:73:ca:f7:01 N5K-2# show vpc role | inc system-mac vPC system-mac : 00:23:04:ee:be:64 vPC local system-mac : 00:05:73:b2:0e:bc |
vPC Configuration
We will now create a vPC towards an LACP neighbor which in this case will be a server.
Here is the configuration used on both Nexus
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
N5K-1(config-if)# int eth101/1/1-2 N5K-1(config-if-range)# shut N5K-1(config-if-range)# channel-group 111 mode active N5K-1(config-if-range)# int po111 N5K-1(config-if)# switchport mode access N5K-1(config-if)# switchport access vlan 930 N5K-1(config-if)# vpc 111 N5K-1(config-if)# int eth101/1/1-2 N5K-1(config-if-range)# no shut N5K-2(config-if)# int eth101/1/1-2 N5K-2(config-if-range)# shut N5K-2(config-if-range)# channel-group 111 mode active N5K-2(config-if-range)# int po111 N5K-2(config-if)# switchport mode access N5K-2(config-if)# switchport access vlan 930 N5K-2(config-if)# vpc 111 N5K-2(config-if)# int eth101/1/1-2 N5K-2(config-if-range)# no shut |
Let’s assume that our server’s NIC team is already configured for LACP. Let’s now check the vPC status
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 |
N5K-1# show vpc Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 100 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : primary Number of vPCs configured : 1 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Disabled vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ -------------------------------------------------- 1 Po100 up 1,930 vPC status ---------------------------------------------------------------------------- id Port Status Consistency Reason Active vlans ------ ----------- ------ ----------- -------------------------- ----------- 111 Po111 up success success 930 N5K-1# show vpc consistency-parameters vpc 111 Legend: Type 1 : vPC will be suspended in case of mismatch Name Type Local Value Peer Value ------------- ---- ---------------------- ----------------------- Shut Lan 1 No No STP Port Type 1 Default Default STP Port Guard 1 None None STP MST Simulate PVST 1 Default Default lag-id 1 [(7f9b, [(7f9b, 0-23-4-ee-be-64, 806f, 0-23-4-ee-be-64, 806f, 0, 0), (ffff, 0, 0), (ffff, 0-4-23-8-fb-80, 11, 0, 0-4-23-8-fb-80, 11, 0, 0)] 0)] mode 1 active active Speed 1 1000 Mb/s 1000 Mb/s Duplex 1 full full Port Mode 1 access access MTU 1 1500 1500 Admin port mode 1 Allowed VLANs - 930 930 Local suspended VLANs - - - N5K-2# show vpc Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 100 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : secondary Number of vPCs configured : 1 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Disabled vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ -------------------------------------------------- 1 Po100 up 1,930 vPC status ---------------------------------------------------------------------------- id Port Status Consistency Reason Active vlans ------ ----------- ------ ----------- -------------------------- ----------- 111 Po111 up success success 930 N5K-2# show vpc consistency-parameters vpc 111 Legend: Type 1 : vPC will be suspended in case of mismatch Name Type Local Value Peer Value ------------- ---- ---------------------- ----------------------- Shut Lan 1 No No STP Port Type 1 Default Default STP Port Guard 1 None None STP MST Simulate PVST 1 Default Default lag-id 1 [(7f9b, [(7f9b, 0-23-4-ee-be-64, 806f, 0-23-4-ee-be-64, 806f, 0, 0), (ffff, 0, 0), (ffff, 0-4-23-8-fb-80, 11, 0, 0-4-23-8-fb-80, 11, 0, 0)] 0)] mode 1 active active Speed 1 1000 Mb/s 1000 Mb/s Duplex 1 full full Port Mode 1 access access MTU 1 1500 1500 Admin port mode 1 Allowed VLANs - 930 930 Local suspended VLANs - - - |
Consistency parameters are matching on both vPC peers and our vPC towards that server is functional 🙂
LACP verification
The following output will confirm that we are indeed having an LACP port-channel between the server and the vPC peers
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
N5K-1# show lacp neighbor interface po111 Flags: S - Device is sending Slow LACPDUs F - Device is sending Fast LACPDUs A - Device is in Active mode P - Device is in Passive mode port-channel111 neighbors Partner's information Partner Partner Partner Port System ID Port Number Age Flags Eth101/1/1 65535,0-4-23-8-fb-80 0x1 2944 SA LACP Partner Partner Partner Port Priority Oper Key Port State 255 0x11 0x3d Partner's information Partner Partner Partner Port System ID Port Number Age Flags Eth101/1/2 65535,0-4-23-8-fb-80 0x2 2944 SA LACP Partner Partner Partner Port Priority Oper Key Port State 255 0x11 0x3d N5K-2# show lacp neighbor interface po111 Flags: S - Device is sending Slow LACPDUs F - Device is sending Fast LACPDUs A - Device is in Active mode P - Device is in Passive mode port-channel111 neighbors Partner's information Partner Partner Partner Port System ID Port Number Age Flags Eth102/1/1 65535,0-4-23-8-fb-80 0x6 2109 SA LACP Partner Partner Partner Port Priority Oper Key Port State 255 0x11 0x3d Partner's information Partner Partner Partner Port System ID Port Number Age Flags Eth102/1/2 65535,0-4-23-8-fb-80 0x5 2109 SA LACP Partner Partner Partner Port Priority Oper Key Port State 255 0x11 0x3d |
This post was a bit basic but it can get tricky to troubleshoot a vPC so be sure to follow the steps above in order to save time in your deployment/labs.
We will dig deeper on vPC tricks soon 🙂
Nicolas
