Ask Your Question
1

Nova-Network on Hetzner. instance cant pass bridge.

asked 2015-10-22 04:08:49 -0500

mickey gravatar image

updated 2015-10-23 11:42:45 -0500

Hi. I have installed a 3 Node Cluster on Hetzner with Compute Legacy Networking. There is a well known problem with Hetzner’s IP Policy. The given subnet for my Instances are not in the same Subnet as the Physical Interface (AAA.BBB.76.148). So i told the Host that the Subnet XXX.VVV.216.144/28 belongs to him, so that Openstack can assign it as Floating IP to the VM. The Default Gateway for the Server is AAA.BBB.76.129.

The Bridge took the IP and Gateway from the eth0 interface and uses the Default gateway (XXX.VVV.216.145) for the VMs.

After the first VM starts i sign in and try to ping 8.8.8.8

m009@ubuntu-14:~$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=56 time=5.55 ms
From 8.8.8.8: icmp_seq=2 Redirect Host(New nexthop: 8.8.8.8)
64 bytes from 8.8.8.8: icmp_seq=2 ttl=56 time=5.55 ms
From 8.8.8.8 icmp_seq=3 Destination Host Unreachable
From 8.8.8.8 icmp_seq=4 Destination Host Unreachable
From 8.8.8.8 icmp_seq=5 Destination Host Unreachable

It always automatically drops every other Packet after 2 received.

So i made a tcpdump on the Compute Host to check the traffic and what i found out:

223 27.766181   fa:16:3e:2e:db:34   Broadcast   ARP 42  Who has AAA.BBB.76.129?  Tell XXX.VVV.216.148
...
Target MAC address: 00:00:00:00:00:00 (00:00:00:00:00:00)
Target IP: AAA.BBB.76.129

my route config:

default via AAA.BBB.76.129 dev br100 
AAA.BBB.76.128/26 dev br100  proto kernel  scope link  src AAA.BBB.76.148 
XXX.VVV.216.144/28 dev br100  proto kernel  scope link  src XXX.VVV.216.145 
192.168.122.0/24 dev virbr0  proto kernel  scope link  src 192.168.122.1 
213.133.98.64/26 dev eth1  proto kernel  scope link  src 213.133.98.99

What could be the problem?

edit retag flag offensive close merge delete

2 answers

Sort by » oldest newest most voted
1

answered 2015-10-23 11:35:04 -0500

mickey gravatar image

updated 2015-10-23 11:36:36 -0500

FIX! Problem solved.

I was totally confused, because basically Hetzner means that your VM´s must have a specific MAC Address beginning with 54:52:00. http://wiki.hetzner.de/index.php/KVM_... All I could find was the solution to solve the bridge and MAC-Address NAT problem in combination with Neutron. Like mangelajo described in his blog. I am very thankful for his Post! :) But i am going in another direction where i want the smallest setup with enough storage using the nova network to provide public connection for the instances.

The first time I started my Ubuntu Cloud Instance, i saw that ICMP receives 2 Packages and stops receiving more. So i checked the compute hosts bridge with tcpdumb and had a neverending Broadcast. The Bridge had 2 Subnets to provide and couldn't decide to who the the target mac belongs.
ARP "Who has the gateway IP? Tell VM". That means to me i have to seperate the public interface IP from the bridge by another virtual interface providing the Gateway.

... long story short result.

modprobe dummy
ifconfig dummy0 x.x.x.x
add route x.x.x.x dummy0

/etc/nova/nova.conf
...
...
flat_network_bridge = br100
flat_interface = dummy0
public_interface = dummy0

I created a dummy Interface using the gateway IP of my floating-pool subnet. route add x.x.x.x dummy0. Accordingly i´ve changed the public_interface and flat_interface to "dummy0".

thanks!

edit flag offensive delete link more
0

answered 2015-10-23 01:15:34 -0500

mangelajo gravatar image

I cannot give you a fully working solution, but some orientation. there are two issues:

1) Neutron does not support on-link routes to be configured in the routers (that means, you cannot create set a router outside of your subnet for your router).

2) I'm not fully aware of how hetzner works, but if it's similar to OVH, what I did there was, setting all my IP RIPE ranges to a single VMAC, then I used this:

https://github.com/mangelajo/ovh-ebtables-agent (https://github.com/mangelajo/ovh-ebta...)

to setup a linux bridge in front of my network adapter, and insert ebtables rules to translate router port mac addresses to match that VMAC back and forth. Right' now I'd do the same thing by using open flow, but that's a different history.

The daemon and neutron patches only work for juno, I wish I had time to contribute on-link routes at least to neutron, which will happen eventually, but it's taking it's time as I always have higher priority work to do.

If you try to amend the on-link routes patch, I think the mtu patch is not necessary because you can now tune MTU on networks for neutron (since kilo).

But you'd still need to run (manually or an equivalent of ..) the bin/ script you have there, to build the switch in the middle. and plug br-ex and your system to such bridge,

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2015-10-22 04:08:49 -0500

Seen: 928 times

Last updated: Oct 23 '15