Exploring Openstack Networking with OVN

Let’s dig how two virtual machines deployed on OpenStack communicate with each other. I have deployed two ubuntu focal instances on top of a Charmed OpenStack deployment in the existing network int_net – overcloud-focal-1 and overcloud-focal-2. These will sometimes be called focal-1 and focal-2 respectively for ease.

Let’s look at how focal-1 and focal-2 talk to each other.

The following are the internal IP addresses and MAC addresses of these virtual machines:

Machine	IP address	MAC address
overcloud-focal-1	192.168.21.52	fa:16:3e:50:1a:a3
overcloud-focal-2	192.168.21.66	fa:16:3e:e4:e8:04

The two machines are in the same network int_net which has an IP range 192.168.21.0/24. This network can also be seen in the horizon dashboard as a graph, which intuively shows the two vms in the same network.

First, let’s send icmp packets from focal-1 to focal-2

Listening for these packets on focal-2 with tcpdump, we see that the connection is successful.

The packets go through a logical switch as they are in the same (logical) network. To see which switch they go through, let’s look at the OVN Northbound database. To learn more about OVN Northbound db, and OVN in general, refer to the OVN architecture. In the machine where ovn is deployed, we will use the ovn-nbctl command to see the information about the virtual switches.

root@juju-868395-overcloud0-20:/home/ubuntu# ovn-nbctl show
switch c268ca84-f20a-44a3-aadd-047408839634
(neutron-1b80dac9-eed2-4fc2-9540-55f624aaec81) (aka int_net)
	port de4f2069-5265-4e23-acc0-6a7d84ee1ef2
	addresses: ["fa:16:3e:50:1a:a3 192.168.21.52", "unknown"] # focal-1
	port f10e5288-5279-4ce0-9cfc-3908ca44c416
	type: localport
	addresses: ["fa:16:3e:ab:ea:9a 192.168.21.10"]
	port 2324719c-16c4-47a6-ab8e-74885c80c2c2
	type: router
	router-port: lrp-2324719c-16c4-47a6-ab8e-74885c80c2c2
	port 585e6e2e-a699-4aa8-b6b0-4d5e2fd66e06
	addresses: ["fa:16:3e:e4:e8:04 192.168.21.66", "unknown"] # focal-2

- - - snip - - - -

Note that the lines with comment # focal-1 and # focal-2 show the MAC addresses and IP addresses of these VMs respectively. Hence, it is seen that the packets between these VMs pass through this logical switch c268ca84-f20a-44a3-aadd-047408839634.

Now, let’s use ovn-trace to trace the logical flow between these machines. We will filter only the flows which have input port as de4f2069-5265-4e23-acc0-6a7d84ee1ef2, where machine focal-1 is logically connected. We will also filter only the flows which have destination as focal-2’s MAC address ie. fa:16:3e:e4:e8:04

ovn-trace int_net 'inport == "de4f2069-5265-4e23-acc0-6a7d84ee1ef2" &&
eth.dst == fa:16:3e:e4:e8:04'
#
reg14=0x3,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=fa:16:3e:e4:e8:04
,dl_type=0x0000
#
reg14=0x3,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=fa:16:3e:e4:e8:04
,dl_type=0x0000
ingress(dp="int_net", inport="de4f20")
--------------------------------------
0. ls_in_port_sec_l2 (northd.c:5205): inport == "de4f20", priority 50,
uuid 29e6778f
next;
3. ls_in_lookup_fdb (northd.c:5241): inport == "de4f20", priority 100,
uuid 2f738c79reg0[11] = lookup_fdb(inport, eth.src);
/* lookup mac failed in mac learning table. */
next;
4. ls_in_put_fdb (northd.c:5249): inport == "de4f20" && reg0[11] == 0,
priority 100, uuid 433c911c
put_fdb(inport, eth.src);
next;
8. ls_in_acl_hint (northd.c:5735): !ct.trk, priority 5, uuid 344dc4b0
reg0[8] = 1;
reg0[9] = 1;
next;
22. ls_in_l2_lkup (northd.c:7963): eth.dst == fa:16:3e:e4:e8:04, priority
50, uuid 6bb1b3ae
outport = "585e6e";
output;
egress(dp="int_net", inport="de4f20", outport="585e6e")
-------------------------------------------------------
3. ls_out_acl_hint (northd.c:5735): !ct.trk, priority 5, uuid d6f991ab
reg0[8] = 1;
reg0[9] = 1;
next;
9. ls_out_port_sec_l2 (northd.c:5302): outport == "585e6e", priority 50,
uuid c4cff143
output;
/* output to "585e6e", type "" */

Notice the outport which matches with the port where machine focal-2 is logically connected.

Taking the analogy of physical computer networks, we can verify that from the VM’s perspective, the machines are in the same ‘real’ network. This can be done by tracing the route from inside the vm. As seen below, the packets go to the destination directly without any routers in between:

Thus, as far the machine overcloud-focal-1 and it’s processes are concerned, overcloud-focal-2 is in the same network connected directly on the same switch.