Ask Your Question
0

Nested virtualization with KVM/OpenVSwitch

asked 2017-05-04 01:50:26 -0500

Micha gravatar image

updated 2017-05-10 04:07:31 -0500

Hi all together,

I have a OpenStack Installation running on an bunch of old hardware we put together from old PCs. Now I want to set up OpenStack on a single host, but inside multiple VMs virtualized with KVM. The KVMs are connected via OpenVSwitch. The network setup is the following: https://imgur.com/a/HZzYw (Openstack Network Setup)

I can successfully ping each instance over the Management network, running over intbr0. I can also ping the compute1, compute2 and network via their IP Addresses on the second NIC, which is connected to the tunbr0. A third NIC on the network node is set to 'manual' and attached to the br-ext. I then set up keystone, glance, neutron and nova. Inside the VMs I use OpenVSwitch again, which is this time managed by OpenStack, for the inter-vm traffic I use vxlan tunnels. The connection to the outer world is done via VLAN, where I implemented trunking on both OpenVSwitches (br-ext and databr0)

I was able to create multiple instances which appear correctly on the compute nodes and are shown without an error. I also created two networks - one internal Test-Customer network and one external network. I also created the needed subnets and the router and attached it to both networks correctly.

After doing so I was able to: Ping the Router IPs from Routers Namespace (Internal IP and gateway IP)

I was NOT able to: Ping the attached VMs Ping another machine in the same Subnet attached to databr0 Ping the Router Gateway from the machine attached to databr0

However, my instances got IPs from the correct subnet after booting up, so I assume some DHCP magic was working (but why is my ping failing then?).

I had a setup like that running on ESXI. I remember I had to set the vSwitch to promiscous mode. However, I did nothing like that on databr0, tunbr0 or intbr0 which is - as I assume - the reason packets may get dropped there. I found https://darabi.wordpress.com/2014/10/06/howto-set-promiscuous-mode-and-enable-it-in-openvswitch/ (THIS) article, however I'm not sure how to set that up.

If anyone could help me to get this working, I would highly appreciate it. OpenVSwitch and OpenFlow looks like some dark art to me right now and I'm stuck at this point for over a week. If you need more information, please feel free to ask :)

Greetings from Germany Micha


EDIT: More infos: Oh, just remembered I can't be sure the instance got the IP - I was just checking the metadata (which may be incorrect).

The network is not configured in virsh (yet - should I do that?), I attached the instances by editing their config file and adding another interface like this:

<interface type='bridge'>   
  <mac address='52:54:00:43:68:01'/>  
  <source bridge='ostun0'/>  
  <virtualport type='openvswitch'>
    <parameters interfaceid='85dc9976-0f7b-4995-891d-c1ce2f5902a6'/>
  </virtualport>
  <model type='virtio'/>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> 
</interface>

However, I also found out that virsh apparently does ... (more)

edit retag flag offensive close merge delete

Comments

Could you check the security group rules? Allow icmp must be set. By default this is disabled.

sanjana gravatar imagesanjana ( 2017-05-04 23:30:02 -0500 )edit

Since the router resides (afaik) in the same security group as the node, it should be pingable by the default mitaka rules.

This would also not explain why I can't ping the external public IPs.

Can somebody tell me if the IP after boot is assigned via DHCP or if there is some voodoo happening?

Micha gravatar imageMicha ( 2017-05-05 04:12:26 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
0

answered 2017-05-07 12:36:25 -0500

mlschuler gravatar image

An instance usually gets its IP address assigned by DHCP. In your installation, what is the configuration of the network within KVM? Would you show us the output of virsh net-dumpxml?

edit flag offensive delete link more

Comments

Added info to the start post since there is no 'reply' function here.

Micha gravatar imageMicha ( 2017-05-10 04:07:06 -0500 )edit

I have done a similar installation for a 'virtualized' SUSE OpenStack Cloud. The difference to what I see in your setting, is that I used portgroups in openvswitch for each of the node types in OpenStack (compute, controller, storage etc). I do not see any VLANs in your xml.

mlschuler gravatar imagemlschuler ( 2017-05-10 04:21:07 -0500 )edit

I added the VLANs via ovs-vsctl set port vnet8 trunks=2003,2004

Micha gravatar imageMicha ( 2017-05-10 05:16:03 -0500 )edit
1

I do not understand why dev dev1 on databr0 is not a VLAN trunk with IDs 2003 and 2004? The traffic from vnet8 does not go to anywhere.

mlschuler gravatar imagemlschuler ( 2017-05-10 09:50:58 -0500 )edit
1

Let me repeat my previous question in another way. It does not make sense to define a single switch port (vnet8) as a VLAN trunk and all other ports as native VLANs. 2003 and 2004 need to be defined for other ports too. Did I miss something?

mlschuler gravatar imagemlschuler ( 2017-05-11 10:42:26 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2017-05-04 01:50:26 -0500

Seen: 579 times

Last updated: May 10 '17