VMs on Controller Node can't properly send/recieve broadcasts
On our private OpenStack essex cloud (single host, flatDHCP networking), VMs spawned on the controller can't properly send or receive broadcasts (looks like a NAT), while VMs spawned on the compute nodes don't experience any issues at all.
We have two independent networks:
Network 1: 10.10.32.0/21
Network 2: 10.10.20.0/23
and the VMs are all connected to 'network 1' and are given addresses in the subnet 10.10.38.0/24. When a broadcast is sent from a VM on the controller node, machines on 'network 1' see it as being sent from the controller's 'network 2' interface address. Simlararly, when a broadcast is sent from a machine on 'network 1' a VM on the controller node will see it as being sent from the controller's 'network 2' interface address.
This does not happen at all on the compute node guests.
Our config setup:
/etc/network/interfaces (eth0 is connected to 'network 1', eth1 to 'network 2')
# This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). auto lo iface lo inet loopback # eth0 is not defined here, because it is used in the bridge auto eth1 iface eth1 inet dhcp auto eth0 iface eth0 inet dhcp auto br100 iface br100 inet dhcp bridge_ports eth0 bridge_stp off bridge_maxwait 0 bridge_fd 0
/etc/nova/nova.conf
--sql_connection=mysql://novadbadmin:novasecret@10.10.33.233/nova --dhcpbridge_flagfile=/etc/nova/nova.conf --dhcpbridge=/usr/bin/nova-dhcpbridge --logdir=/var/log/nova --state_path=/var/lib/nova --lock_path=/run/lock/nova --allow_admin_api=true --use_deprecated_auth=false --auth_strategy=keystone --scheduler_driver=nova.scheduler.filter_scheduler.FilterScheduler --scheduler_available_filters=nova.scheduler.filters.standard_filters --scheduler_default_filters=AvailabilityZoneFilter,RamFilter,ComputeFilter,CoreFilter --cpu_allocation_ratio=1.0 --s3_host=10.10.33.233 --ec2_host=10.10.33.233 --cc_host=10.10.33.233 --nova_url=http://10.10.33.233:8774/v1.1/ --glance_api_servers=10.10.33.233:9292 --image_service=nova.image.glance.GlanceImageService --ec2_url=http://10.10.33.233:8773/services/Cloud --keystone_ec2_url=http://10.10.33.233:5000/v2.0/ec2tokens --api_paste_config=/etc/nova/api-paste.ini --multi_host=true --iscsi_helper=tgtadm --root_helper=sudo nova-rootwrap --verbose --connection_type=libvirt --libvirt_type=kvm --start_guests_on_host_boot=true --resume_guests_state_on_host_boot=true --rpc_response_timeout=3600 # vnc specific configuration --novnc_enabled=true --novncproxy_base_url=http://10.10.33.233:6080/vnc_auto.html --vncserver_proxyclient_address=10.10.33.233 --vncserver_listen=10.10.33.233 # network specific settings --network_manager=nova.network.manager.FlatDHCPManager --libvirt_use_virtio_for_bridges=true --fixed_range=10.10.32.0/21 --flat_interface=eth0 --flat_network_bridge=br100 --flat_injected=False # RabbitMQ settings --rabbit_host=10.10.33.233 # Metadata settings --metadatahost=10.10.33.233
iptables for instance on controller
Chain nova-compute-inst-229 (1 references) target prot opt source destination DROP all -- anywhere anywhere state INVALID ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED nova-compute-provider all -- anywhere anywhere ACCEPT udp -- 10.10.38.1 anywhere udp spt:bootps dpt:bootpc ACCEPT all -- 10.10.38.0/24 anywhere ACCEPT tcp -- anywhere anywhere tcp dpt:3389 ACCEPT icmp -- anywhere anywhere ACCEPT tcp -- anywhere anywhere multiport dports 50080:50110 ACCEPT udp ...