Problem with bridging two networks with a VM

asked 2014-02-21 10:04:59 -0600

capzulu gravatar image

updated 2014-02-21 10:06:16 -0600


I'm struggling with this problem the last two weeks.

I'm using Openstack Havana and I want to implement this network topology. (image description)

The network configuration is the follow:

  • All the VM's are using Ubuntu Server 12.04;
  • The orange network have a DHCP server and the gateway is the router connected to this network;
  • The green network don't have DHCP Server as I configure a static IP in the VM connected to this network.
  • The VM with a blue circle have eth0 connected to orange network and eth1 connected to green network.

I want the VM from the green network to be able to talk to other VM's from orange network and to the router. All the comunications will go through the VM (blue circle) that are connecting the two networks, like a layer 2 bridge.

Info: Don't tell me that I can use a router to connect the two networks, as I need it to be connected using a VM.

All the VM's from orange network are talking with each other and with the Internet. All is good.

So, I need to configure the bridge VM (blue circle VM) to act like a layer 2 bridge. This is done inside the VM:

sudo ifconfig eth0 down
sudo ifconfig eth1 down
sudo brctl addbr br0
sudo brctl addif br0 eth0
sudo brctl addif br0 eth1
sudo ifconfig eth0 promisc up
sudo ifconfig eth1 promisc up
sudo ifconfig br0 promisc up

Ok, now I have the two interfaces bridged and theoretically all the packets can be passed from one network to the other network, because of the promisc mode....theoretically.

Next I go inside the VM connected in the green network and do this:

sudo ifconfig netmask
sudo route add default gw

I give it an IP and I add a default gateway, that is the router in the orange network.

Now it's time to test the connection. I place tshark in one of the tap network in the bridge VM

sudo tshark -n -i tap383f8a7c-86

And I do a ping from the VM that are connected to the green network

ping (This is the IP of the router)

This is when the strange things happen. The network traffic at the bridge goes crazy. So as you can see I created a pastebin output. (ARP craziness)

With the tshark information, I can see that the bridge catches the ARP Request (broadcast) and the ARP Reply (unicast)

But the VM in the green network outputs me this:

PING ( 56(84) bytes of data.
From icmp_seq=1 Destination Host Unreachable
From icmp_seq=2 Destination Host Unreachable
From ...
edit retag flag offensive close merge delete



Why are you trying to do this?

darragh-oreilly gravatar imagedarragh-oreilly ( 2014-02-21 11:29:14 -0600 )edit

I am doing virtualization of network functions (NFV). In this scenario I'm trying to implement a WOC (WAN Optimization Controllers). A WOC needs a layer 2 VM that can connect two network segments.

capzulu gravatar imagecapzulu ( 2014-02-24 07:09:41 -0600 )edit

The nova vif driver enables haripin mode on the tap interface in the linux bridge to workaround some floating ip issue with nova-network. It's definitely not needed in neutron, so it might be worth trying to disable that. There is a patch to disable it upstream as it causes issues with neutron ipv6.

darragh-oreilly gravatar imagedarragh-oreilly ( 2014-02-24 09:44:08 -0600 )edit

Thank you for your help. I did turned off hairpin and the "ARP craziness" stopped. My VM in the green network receives the ARP Reply from the router. It populates it's arp table. But for the ICMP packets (ping), I can they are going out but the VM connecting the two networks don't see that packets.

capzulu gravatar imagecapzulu ( 2014-02-25 07:19:21 -0600 )edit

Be aware that the security groups implementation places anti-spoofing rules to prevent a VM sending packets that don't have the source mac or IP address that Neutron assigned to it. Do iptables-save on the compute nodes to see them.

darragh-oreilly gravatar imagedarragh-oreilly ( 2014-02-25 08:02:22 -0600 )edit