I've set up Neutron public and tenant networking according to the Ubuntu Liberty install guide. I'm having trouble getting tenant networks workingI have created a public and private network and assigned a subnet to each of them. I can create an instance on the public network and everything works. When I create an instance on the private network and assign an IP to it, I can't communicate with it. Also, I can't ping the 'public' side of the router attached to the private subnet and when I VNC to the instance running on the private network, it can only ping as far as its gateway on the private network, nothing past that works.
I've created two networks in Neutron:
$ neutron subnet-list
+--------------------------------------+---------+----------------+--------------------------------------------------+
| id | name | cidr | allocation_pools |
+--------------------------------------+---------+----------------+--------------------------------------------------+
| 5ae8659a-525f-4173-b413-e3934428e840 | public | 172.18.0.0/22 | {"start": "172.18.1.195", "end": "172.18.1.230"} |
| 41805d59-3b7d-4a5f-8cc7-9f795368ea58 | private | 192.168.1.0/24 | {"start": "192.168.1.2", "end": "192.168.1.254"} |
+--------------------------------------+---------+----------------+--------------------------------------------------+
Here are the details of each:
$ neutron subnet-show private
+-------------------+--------------------------------------------------+
| Field | Value |
+-------------------+--------------------------------------------------+
| allocation_pools | {"start": "192.168.1.2", "end": "192.168.1.254"} |
| cidr | 192.168.1.0/24 |
| dns_nameservers | 8.8.8.8 |
| enable_dhcp | True |
| gateway_ip | 192.168.1.1 |
| host_routes | |
| id | 41805d59-3b7d-4a5f-8cc7-9f795368ea58 |
| ip_version | 4 |
| ipv6_address_mode | |
| ipv6_ra_mode | |
| name | private |
| network_id | 42067c22-3499-4d92-adc2-017f4085ce9a |
| subnetpool_id | |
| tenant_id | 5648ab9f768f4fc0afc541371dacf2f9 |
+-------------------+--------------------------------------------------+
$ neutron subnet-show public
+-------------------+--------------------------------------------------+
| Field | Value |
+-------------------+--------------------------------------------------+
| allocation_pools | {"start": "172.18.1.195", "end": "172.18.1.230"} |
| cidr | 172.18.0.0/22 |
| dns_nameservers | 172.18.1.5 |
| enable_dhcp | True |
| gateway_ip | 172.18.0.1 |
| host_routes | |
| id | 5ae8659a-525f-4173-b413-e3934428e840 |
| ip_version | 4 |
| ipv6_address_mode | |
| ipv6_ra_mode | |
| name | public |
| network_id | 11a30862-96f0-40d3-a826-f42924d10086 |
| subnetpool_id | |
| tenant_id | cf7bd0047cec46c9a51675aef6add576 |
+-------------------+--------------------------------------------------+
My neutron system and my compute node have two network interfaces, one for public networks (eth1) and the other for management/private networks (eth0):
On the NEUTRON/CONTROL system:
$ ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:50:56:a3:41:ec
inet addr:172.18.1.80 Bcast:172.18.3.255 Mask:255.255.252.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1319835 errors:0 dropped:12079 overruns:0 frame:0
TX packets:622529 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1804763786 (1.8 GB) TX bytes:764407696 (764.4 MB)
$ ifconfig eth1
eth1 Link encap:Ethernet HWaddr 00:50:56:a3:2e:df
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1462139 errors:9204 dropped:18341 overruns:0 frame:0
TX packets:78 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:121271062 (121.2 MB) TX bytes:6141 (6.1 KB)
/etc/nova/nova.conf:
[neutron]
url = http://openstack-control1:9696
auth_url = http://openstack-control1:35357
auth_plugin = password
project_domain_id = default
user_domain_id = default
region_name = RegionOne
project_name = service
username = neutron
password = XXXXXX
/etc/neutron/plugins/ml2/linuxbridge_agent.ini:
[linux_bridge]
physical_interface_mappings = public:eth1
[vxlan]
enable_vxlan = True
local_ip = 172.18.1.80
l2_population = True
[agent]
prevent_arp_spoofing = True
[securitygroup]
enable_security_group = True
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver
/etc/neutron/l3_agent.ini:
[DEFAULT]
interface_driver = neutron.agent.linux.interface.BridgeInterfaceDriver
external_network_bridge =
/etc/neutron/dhcp_agent.ini:
[DEFAULT]
interface_driver = neutron.agent.linux.interface.BridgeInterfaceDriver
dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq
enable_isolated_metadata = True
dnsmasq_config_file = /etc/neutron/dnsmasq-neutron.conf
/etc/neutron/dnsmasq-neutron.conf:
dhcp-option-force=26,1450
On the COMPUTE node:
$ ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:30:48:f5:a4:34
inet addr:172.18.1.100 Bcast:172.18.3.255 Mask:255.255.252.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:334155 errors:0 dropped:4037 overruns:0 frame:0
TX packets:183071 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:67517074 (67.5 MB) TX bytes:56809460 (56.8 MB)
Memory:c0000000-c001ffff
$ ifconfig eth1
eth1 Link encap:Ethernet HWaddr 00:30:48:f5:a4:35
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:867906 errors:0 dropped:1943 overruns:0 frame:0
TX packets:154 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:75534410 (75.5 MB) TX bytes:11766 (11.7 KB)
Memory:c0060000-c007ffff
/etc/nova/nova.conf:
[neutron]
url = http://openstack-control1:9696
auth_url = http://openstack-control1:35357
auth_plugin = password
project_domain_id = default
user_domain_id = default
region_name = RegionOne
project_name = service
username = neutron
password = XXXXXXX
/etc/neutron/plugins/ml2/linuxbridge_agent.ini:
[linux_bridge]
physical_interface_mappings = public:eth1
[vxlan]
enable_vxlan = True
local_ip = 172.18.1.100
l2_population = True
[agent]
prevent_arp_spoofing = True
[securitygroup]
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver
enable_security_group = True
I put an instance on the public network and it works great:
$ nova list | grep ubuntu
| e8b816a5-82cf-45c4-98b6-6a488a001633 | ubuntu_test | ACTIVE | - | Running | public=172.18.1.204 |
I can ping, ssh, etc to that box
I made a instance on the private network and I can't communicate with it. I tried assigning a 'public' IP to it:
$ nova floating-ip-create public
+--------------------------------------+--------------+-----------+----------+--------+
| Id | IP | Server Id | Fixed IP | Pool |
+--------------------------------------+--------------+-----------+----------+--------+
| 420783e9-cfc8-482d-ac0f-0c4718aaf495 | 172.18.1.201 | - | - | public |
+--------------------------------------+--------------+-----------+----------+--------+
$ nova floating-ip-list
+--------------------------------------+--------------+-----------+----------+--------+
| Id | IP | Server Id | Fixed IP | Pool |
+--------------------------------------+--------------+-----------+----------+--------+
| 420783e9-cfc8-482d-ac0f-0c4718aaf495 | 172.18.1.201 | - | - | public |
+--------------------------------------+--------------+-----------+----------+--------+
$ nova list
+--------------------------------------+-----------------+--------+------------+-------------+---------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-----------------+--------+------------+-------------+---------------------+
| 7b523d5d-9c8e-42ff-84d0-c82b18a46797 | cirros_private | ACTIVE | - | Running | private=192.168.1.7 |
$ nova floating-ip-associate cirros_private 172.18.1.201
$ nova list
+--------------------------------------+-----------------+--------+------------+-------------+-----------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-----------------+--------+------------+-------------+-----------------------------------+
| 7b523d5d-9c8e-42ff-84d0-c82b18a46797 | cirros_private | ACTIVE | - | Running | private=192.168.1.7, 172.18.1.201 |
I can't ping 172.18.1.201 from the machine running Neutron, or from any of my Nova nodes, or any other machine on the 172.18.0.0/22 network
I open a VNC connection to the 'cirros_private' instance and I can't ping anything on 172.18.0.0/22 or on the internet. I can ping 192.168.1.1 though, which is the gateway of that machine
That instance is using the 'default' security group (as is my instance on the public nework, which works)
$ nova show cirros_private | grep security_group
| security_groups | default |
And that security group has the following rules, allowing ping/ssh from anywhere:
$ nova secgroup-list-rules default
+-------------+-----------+---------+-----------+--------------+
| IP Protocol | From Port | To Port | IP Range | Source Group |
+-------------+-----------+---------+-----------+--------------+
| | | | | default |
| tcp | 22 | 22 | 0.0.0.0/0 | |
| icmp | -1 | -1 | 0.0.0.0/0 | |
| | | | | default |
+-------------+-----------+---------+-----------+--------------+
Let's take a look at the router that instance is connected to:
$ neutron router-list
| id | name | external_gateway_info | distributed | ha |
| 2fde1cba-6a31-45cb-b674-9e3db1a20f47 | router | {"network_id": "11a30862-96f0-40d3-a826-f42924d10086", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "5ae8659a-525f-4173-b413-e3934428e840", "ip_address": "172.18.1.196"}]} | False | False |
And the ports on that router:
$ neutron router-port-list 2fde1cba-6a31-45cb-b674-9e3db1a20f47
+--------------------------------------+------+-------------------+-------------------------------------------------------------------------------------+
| id | name | mac_address | fixed_ips |
+--------------------------------------+------+-------------------+-------------------------------------------------------------------------------------+
| 82a9704f-dca3-4d06-a15b-a7365a090bf3 | | fa:16:3e:38:ee:b5 | {"subnet_id": "5ae8659a-525f-4173-b413-e3934428e840", "ip_address": "172.18.1.196"} |
| c4be4fe5-fc7a-4c27-aef1-8494200bab48 | | fa:16:3e:4c:1d:9b | {"subnet_id": "41805d59-3b7d-4a5f-8cc7-9f795368ea58", "ip_address": "192.168.1.1"} |
+--------------------------------------+------+-------------------+-------------------------------------------------------------------------------------+
I can't ping the external interface of that router from the Neutron box or my compute node:
root@openstack-control1:/etc/neutron# ping 172.18.1.196
PING 172.18.1.196 (172.18.1.196) 56(84) bytes of data.
^C
--- 172.18.1.196 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms
Where did I go wrong?