VXLAN Header Format and Encapsulation – Part 2
Encapsulation. That thing where you start with a small piece of information, and you glue more onto it. And then some more, and then more. You will see plenty of that with VxLAN. This will be part 2 of our series about VxLAN. If you did not read the first part yet, I strongly recommend you to click here:
https://forum.huawei.com/enterprise/en/how-vxlan-works-what-are-vxlan-benefits-part-1/thread/784511-861
In the last article, we saw how VxLAN works. We saw that VxLAN creates virtual layer 2 segments, called VNI’s. VNI’s run over the top of a layer 3 network. VxLAN switches use a special interface called VTEP. This bridges VNIs to the layer 3 networks. When traffic comes in, the VTEP encapsulates the traffic and sends it to a destination VTEP where it is decapsulated.
In this article, we are going to look deeper into the process, and look at headers that VxLAN uses.
Encapsulation
We start with an ordinary Ethernet frame that a host would send. Consider VM 2 in the image below. We call this the “inner MAC frame”. This includes data, MAC address information, and other Ethernet fields. It also may include a VLAN tag. In this example, traffic will stay within the VNI, so there is no routing required. The host sends the frame to the switch. The switch adds a VxLAN header, which contains the VNI. The VTEP now adds several additional headers, preserving the inner frame.
Figure 1 – How an Ethernet frame is processed.
VxLANuses UDP for transport. The destination UPD port is 4789 and the source port is random. ECMP (Equal-Cost Multiple Path), if available, uses a hashing algorithm to decide which link to put the traffic on. The random source port helps this algorithm to utilize the links evenly. An IP header is now added, with the address of the destination VTEP. An Ethernet header with a MAC address is added for delivery to the next physical device. As normal, the source and destination MAC change with each device they pass through.
Figure 2 – VXLAN packet encapsulation process. NVE is the switch with the VTEP interface, for example.
When the traffic arrives at the destination VTEP (the destination Network Virtualization Edge in the previous figure), the headers are removed, leaving the original frame (the inner frame), which can now be delivered to the host.
VXLAN Header
Of all these headers, the VXLAN header is the only one that is a little bit different. All the others are well known. Fortunately, it is not difficult to understand it.
Figure 3 – VXLAN Header
There are four parts to the VxLAN header. At the start of the frame is eight flag bits. Right now, only bit 3 is used. Bit three is the “I” field, which is set to 1 for a valid VNI. The rest is reserved and ignored. Then there are 8 bits reserved for future use. This is set to zero and ignored by the receiving VTEP. The VNI field is 24 bits long and contains the VxLAN ID. This large address space is what makes it possible to have so many VNI’s. Another 24 bits are reserved. As before, this field is ignored by the receiver.
As can be seen, there are a lot of unused bits in the VXLAN Header, and it will be very interesting to see how they will be used in the future.
The extra VxLAN, UDP, and IP headers add up to around 50-70 bytes of overhead. VxLAN adds 8 bytes, then UDP 8 more. IP adds more than 20 bytes (IPv4) or 40 bytes (IPv6). Ethernet more 18 bytes. To account for this, you will need to enable jumbo frames everywhere. Otherwise, you will get fragmentation, which, as you may know, decreases performance. In the next part of the series, we are going to look at the spine and leaf topology that is so commonly used with VxLAN.
