cannot ping tenant network in cirros using neutron networking - Juno
I have setup an Juno OpenStack cloud in my lab. I have 3 machines running on a VMware vCenter cluster:
Box #1 - Controller & Glance Box #2 - Compute Box #3 - Neutron
Box #1 has 1 network connection (management network) Box #2 has 2 network connections (management network & GRE) Box #3 has 3 network connections (management, GRE, & External net)
I am able to start instances without any issue. I can get to them via VNC proxy, but they have no connectivity to the tenant network or the router.
There are no errors or warnings in any of the nova or neutron logs.
The ovs-vsctl show output shows the connection between the 2 boxes on the GRE network is up:
[root@neutron neutron]# ovs-vsctl show
Bridge br-tun
fail_mode: secure
Port "gre-0f000002"
Interface "gre-0f000002"
type: gre
options: {df_default="true", in_key=flow, local_ip="15.0.0.3", out_key=flow, remote_ip="15.0.0.2"}
[root@compute neutron]# ovs-vsctl show
Port "gre-0f000003"
Interface "gre-0f000003"
type: gre
options: {df_default="true", in_key=flow, local_ip="15.0.0.2", out_key=flow, remote_ip="15.0.0.3"}
ovs_version: "2.3.1"
I tried setting the ip address manually in the Cirros instance and it still won't ping the tenant router.
I used the following article for troubleshooting: http://dischord.org/2015/03/09/troubleshooting-openstack-neutron-networking-part-one/ (http://dischord.org/2015/03/09/troubl...)
There is traffic flowing between compute and neutron as I am seeing the DHCP requests on the neutron server via tcpdump.
I did a tcpdump on the tap interface in the br-tun bridge. It's not showing any data flowing at all.
[root@neutron ~]# ip netns list | grep c7c59f68-e276-4899-87ac-1ad17c3e28ad
qdhcp-c7c59f68-e276-4899-87ac-1ad17c3e28ad
[root@neutron ~]# ip netns exec qdhcp-c7c59f68-e276-4899-87ac-1ad17c3e28ad ip li
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
12: tapa4f79f35-6f: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT
link/ether fa:16:3e:04:63:15 brd ff:ff:ff:ff:ff:ff
[root@neutron ~]# ovs-vsctl show | grep -A1 tapa4f79f35-6f
Port "tapa4f79f35-6f"
tag: 1
Interface "tapa4f79f35-6f"
type: internal
[root@neutron ~]# ip netns exec qdhcp-c7c59f68-e276-4899-87ac-1ad17c3e28ad tcpdump -i tapa4f79f35-6f
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tapa4f79f35-6f, link-type EN10MB (Ethernet), capture size 65535 bytes
Looking at the flows on the OVS br-tun, it appears something is wrong there (the segmenation_id is 1)
[root@neutron ~]# ovs-ofctl dump-flows br-tun | grep 0x1
cookie=0x0, duration=3553.900s, table=3, n_packets=0, n_bytes=0, idle_age=3553, priority=1,tun_id=0x1 actions=mod_vlan_vid:2,resubmit(,10)
cookie=0x0, duration=3554.071s, table=22, n_packets=2, n_bytes=140, idle_age=3316, dl_vlan=2 actions=strip_vlan,set_tunnel:0x1,output:2
Please help.
Thanks, Doug
Troubleshooting tips here : https://www.hastexo.com/system/files/...