SDN in Action: Deploy VXLAN wi

SDN in Action: Deploy VXLAN with MP-BGP EV_P_N

薛国锋     xueguofeng2011@gmail.com

The initial VXLAN standard relies on the data-plane flood-and-learn behavior for remote VTEP peer discovery and end-host learning, which presents a challenge for its scalability. In order to overcome the limitations of the flood-and-learn, MP-BGP EV_P_N is designed and used as the control plane for VXLAN, and provides protocol-based VTEP peer discovery and end-host reachability information distribution. Futuremore it provides control-plane and data-plane separation, and serves as the unfied control plane for both Layer 2 and Layer 3 forwarding – IRB(Integrated Routing and Bridging) in a VXLAN overlay network. VXLAN and MP-BGP EV_P_N are the important additons to SDN allowing it to scale better and interact with standards-track protocols.

Today we are going to build a mini-lab environment with GNS3 and NX-OSv 9000, and get some hands-on experience with VXLAN and MP-BGP EV_P_N:

-          Setup the mini-lab environment

-          VXLAN with MP-BGP EV_P_N – Bridging

-          VXLAN with MP-BGP EV_P_N – Integrated Routing and Bridging

Setup the mini-lab environment

https://docs.gns3.com/appliances/cisco-nxosv9k.html

https://www.virtualizationhowto.com/2017/03/how-to-add-cisco-nx-osv-to-gns3/

Download the appliance file of NX-Osv 9000 for GNS3 and its images:

-          ‘cisco-nxosv9k.gns3a’;

-          ‘hda.qcow2’;

-          ‘OVMF-20160813.fd’.

Import ‘cisco-nxosv9k.gns3a’ to GNS3:

Run NX-OSv 9000 the first time:

Abort Auto Provisioning and continue with normal setup ?(yes/no)[n]: yes

Do you want to enforce secure password standard (yes/no) [y]: yes

Enter the password for "admin": 2018ciscoUS

Confirm the password for "admin": 2018ciscoUS

Would you like to enter the basic configuration dialog (yes/no): no

User Access Verification

 login: admin

Password: 2018ciscoUS

switch# conf

switch(config)# hostname spine

spine# copy running-config startup-config

spine# copy running-config myconfig.bak

spine# copy myconfig.bak startup-config

spine# reload

This command will reboot the system. (y/n)?  [n] yes

loader > boot nxos.7.0.3.I7.2.bin   

VXLAN with MP-BGP EV_P_N – Bridging

In this scenario, MP-BGP EV_P_N is deployed on the leaf nodes to distribute MAC addresss-only routes, and VTEP devices in a given VNI can know about other MAC address of end hosts in the same VNI. Distribution of MAC addresses through MP-BGP EV_P_N allows unknown unicast flooding in the VXLAN to be reduced significantly.

The EV_P_N NLRI is carried in BGP using the BGP multiprotocol extension with a new address family called L2V_P_N EV_P_N. Similar to the V_P_Nv4 address family in the BGP MPLS-based IP V_P_N, the L2V_P_N EV_P_N address family for EV_P_N uses route distinguishers (RDs) to maintain uniqueness among identical routes in different VRF instances, and route targets (RTs) to define the policies that determine how routes are advertised and shared by different VRF instances.

pc11> ip 192.168.1.1/16

pc11> save

pc12> ip 192.168.1.2/16

pc12> save

pc21> ip 192.168.2.1/16

pc11> save

pc22> ip 192.168.2.2/16

pc22> save

spine1# show ip route

spine2# show ip mroute

leaf1# show nve peers

leaf1# show nve vni

leaf1# show vxlan interface

leaf1# show bgp l2V_P_N eV_P_N summary

leaf1# show system internal l2fwder mac

leaf1# show bgp l2V_P_N eV_P_N

leaf1# show l2route eV_P_N mac all

leaf1# show l2route eV_P_N mac-ip all

VXLAN Bridging: 192.168.1.1 -> 192.168.2.2, VNI 100192

VXLAN Bridging: 192.168.2.2 -> 192.168.1.1, VNI 100192

VXLAN with MP-BGP EV_P_N – Integrated Routing and Bridging

MP-BGP EV_P_N is designed to distribute NLRI – network layer reachability information for the network. It can advertise both the MAC and IP addresses of EV_P_N VXLAN end hosts, and accordingly support the integrated routing and bridging. Each VTEP performs local-data-plane learning to obtain MAC and IP address information from its locally attached hosts and then distribute the informtion through the MP-BGP EV_P_N control plane to other VTEPs. The underlay network provides IP reachability for all the VTEP addresses that are used to route the encapsulated VXLAN packets toward the egress VTEP, and doesn’t need to learn the EV_P_N routes, which simplifies the underlay network operation and increases its stability and scalability.

Two integrated routing and bridging models are defined: asymmetric IRB and symmetric IRB. Symmetric IRB is widely adopted for its scalability advantages and simplified Layer 2 and Layer 3 multitenancy support. With symmetric IRB, both the ingress and egress VTEPs perform Layer 2 and Layer 3 lookups.

For a tenant, its VRF instance in each VTEP is mapped to a unique Layer 3 VNI in the network. All inter-VXLAN routed traffic is encapsulated with the Layer 3 VNI in the VXLAN header. The receiving VTEP uses this VNI to determine the VRF context in which the inner IP packet needs to be forwarded. This VNI also provides the basis for enforcing Layer 3 segmentation in the data plane. Each VTEP has a unique router MAC address that other VTEPs can use as the inner destination MAC address for the routed VXLAN packet.

An EV_P_N VXLAN tenant can have multiple Layer 2 networks, each with a corresponding VNI. These Layer 2 networks are bridge domains in the overlay network. The VNIs that are associated with them are often referred to as Layer 2 VNIs. If VXLAN routing is required, then each tenant also needs a Layer 3 VNI in symmetric IRB. Although a VTEP can have all or a subset of the Layer 2 VNIs in a VXLAN EV_P_N overlay, it must have the Layer 3 VNI for inter-VXLAN routing. All VTEPs in an EV_P_N must have the same Layer 3 VNI.

When the destination MAC address in the original packet header does not belong to the local VTEP, then VXLAN bridging needs to occur. In other words, the originating VTEP will perform a Layer 2 lookup and bridge the packet to the destination VTEP. If the destination MAC address matches the anycast gateway MAC address, then VXLAN routing needs to occur. In this case, the originating VTEP will perform a Layer 3 lookup and then encapsulate the packet with the Layer 3 VNI. The destination VTEP will then receive the traffic and will perform another routing lookup based on the inner IP header.

In MP-BGP EV_P_N, any VTEP participating in a VNI can use the distributed anycast gateway feature for end hosts in its IP subnet by supporting the same virtual gateway IP address and the virtual gateway MAC address. With the anycast gateway function in EV_P_N, an end host in a VNI can always use its local VTEP for this VNI as its default gateway to send traffic outside its IP subnet. This capability enables optimal forwarding for northbound traffic from end hosts in the VXLAN overlay network. A distributed anycast gateway also offers the benefit of transparent host mobility in the VXLAN overlay network. Because the gateway IP address and virtual MAC address are identically provisioned on all VTEPs within a VNI, when an end host moves from one VTEP to another, it doesn’t need to send another ARP request to relearn the gateway MAC address.

pc1_1> ip 4.1.1.10/24 4.1.1.1

pc1_1> save

pc1_2> ip 4.1.1.11/24 4.1.1.1

pc1_2> save

pc2_1> ip 4.2.2.10/24 4.2.2.1

pc2_1> save

pc2_2> ip 4.2.2.11/24 4.2.2.1

pc2_2> save

leaf1# show nve peers

leaf1# show nve vni

leaf1# show vxlan interface

leaf1# show l2route eV_P_N mac all

leaf1# show l2route eV_P_N mac-ip all

leaf1# show ip route vrf vxlan-900001

VXLAN Bridging: 4.1.1.10 -> 4.1.1.11, VNI 2001001 

VXLAN Bridging: 4.2.2.10 -> 4.2.2.11, VNI 2001002

VXLAN Routing: 4.1.1.10 -> 4.2.2.11, VNI 900001

leaf1# ping 4.1.1.1 vrf vxlan-900001

leaf1# ping 4.1.1.10 vrf vxlan-900001

leaf1# ping 4.2.2.1 vrf vxlan-900001

leaf1# ping 4.2.2.10 vrf vxlan-900001

leaf1# ping 4.1.1.11 vrf vxlan-900001   // would not work because of the anycast GW IP

leaf1# ping 4.2.2.11 vrf vxlan-900001   // would not work because of the anycast GW IP

VXLAN BGP EV_P_N ON NEXUS 9000V:

https://nfvguy.mas-net.at/2017/05/23/vxlan-bgp-e***-on-nexus-9000v-part-1/

Deploy a VXLAN Network with an MP-BGP EV_P_N Control Plane:

https://www.cisco.com/c/en/us/products/collateral/switches/nexus-7000-series-switches/white-paper-c11-735015.pdf

VXLAN Design and Deployment:

https://www.cisco.com/c/dam/m/sl_si/events/2016/cisco_dan_inovativnih_resitev/pdf/cisco_day_slovenia_2016_vxlan_marian_klas_final.pdf

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide:

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/7-x/vxlan/configuration/guide/b_Cisco_Nexus_9000_Series_NX-OS_VXLAN_Configuration_Guide_7x.html