Ask Your Question
1

Instance cannot ping the external gateway

asked 2013-09-05 11:55:38 -0500

nko321 gravatar image

updated 2013-09-06 15:06:59 -0500

darragh-oreilly gravatar image

Hello,

I had previously asked a question in this thread. After much progress, darragh-oreilly suggested that I post an update as a new question, so this is the continuation of that thread.

I'm currently at the point where my OpenStack environment works almost perfectly. The one remaining things is that my OpenStack instances can't connect to the external network- by which I mean my actual network outside of OpenStack, not my "External Network" configured within OpenStack. Within OpenStack, I can connect from any point to any point that makes sense- as long as the routing is configured, as long as IPs are assigned, etc. It all makes sense.

This part is the meat of this post. As for my outside network configuration, I think that is all in order as well. If I turn off OpenVSwitch and assign an IP to my external interface (eth2) of 172.16.86.160 (this being the IP I have assigned to the router connected to my External Network), I can ping my gateway (172.16.86.5). If I remove that IP configuration, turn OVS back on, and attempt to ping from the netns context of my Router that has an IP on the External Network, I get no ping response. This happens when the router has the same IP I'd tried with the physical interface (172.16.86.160) or a different IP in that subnet (172.16.86.90). /post_meat

I suspect I have misconfigured either:

  • My Bridge configuration
  • My External Network or Router
  • iptables is turned off. I've read a few things about configuring iptables, but I've also read iptables can be turned off. Not sure on that one.

darragh-oreilly requested that I provide the output of a few commands that may help shed some light on my configuration. I've provided that output on OpenStack's paste site. Commands run there include:

  • quantum net-show {ext-net-id}
  • quantum subnet-show {ext-net-subid}
  • ip -4 address
  • ip netns exec qrouter-{uuid} ip -4 a
  • ip netns exec qrouter-{uuid} route -n
  • ovs-vsctl show
  • quantum router-show {router-id}
  • ovs_quantum_plugin.ini

Any help, insights, questions, or commentary is appreciated. I'm this, this close to a fully functional environment and it feels good!

edit retag flag offensive close merge delete

4 answers

Sort by » oldest newest most voted
1

answered 2013-09-05 13:34:43 -0500

darragh-oreilly gravatar image

updated 2013-09-05 15:28:33 -0500

Iptables should not be turned off - it is what implements the NAT. You should enable them and restart the l3-agent.

I assume eth2 is up, and yes it should not have an IP address. The external subnet shouldn't really have dhcp enabled, but I don't think that causes problems.

If the network node is a VirtualBox or VMware VM, then you will need to set the NIC for eth2 to be in promiscuous mode in VirtualBox. What exactly is your gateway - the device with address 172.16.86.5?

Can you provide a tcpdump:

tcpdump -nei eth2

and from the instance do:

ping -c1 172.16.86.5

Tcpdump analysis: the 3 icmp request packets left eth2 ok. I don't see 172.16.86.5 doing an ARP broadcast request for 172.16.86.160, but maybe it done this earlier and has fa:16:3e:6f:29:c5 for it in its cache. Then the gateway device would be sending the reply with this MAC, but the microsoft virtual nic ignores it because that mac does not match its own, and it is not in promiscuous mode. Maybe you could try changing the virtual nic's mac to fa:16:3e:6f:29:c5 in hyper-v?

edit flag offensive delete link more

Comments

> If the network node is a VirtualBox or VMware VM, then you will need to set the NIC for eth2 to be in promiscuous mode in VirtualBox. I think this is the smoking gun. I'm running this in a VM hosted on Hyper-V. Google says Hyper-V doesn't do promisc. mode. I'll try setting up on a physical box!

nko321 gravatar imagenko321 ( 2013-09-05 14:01:09 -0500 )edit

wow - but it does not necessary mean it is the problem - it depends on how microsoft have implemented their virtual switch. You should do the tcpdump first.

darragh-oreilly gravatar imagedarragh-oreilly ( 2013-09-05 14:20:22 -0500 )edit

OK, thanks! tcpdump: http://paste.openstack.org/show/45829/ iptables -L: http://paste.openstack.org/show/45830/ 172.16.86.5 is a Cisco ASA. I attach to a VLAN tagged port on a Catalyst switch. The ASA has an interface on that VLAN as well.

nko321 gravatar imagenko321 ( 2013-09-05 14:46:34 -0500 )edit

Also, the qrouter's iptables: http://paste.openstack.org/show/45832/

nko321 gravatar imagenko321 ( 2013-09-05 14:49:58 -0500 )edit

I have V2V migrated my network node to VMware. My issue now is that I can't see my ICMP traffic on eth2 when coming from the qrouter, so it appears my flow is more broken than before. I suspect my qrouter's iptables, which looks different from before: http://paste.openstack.org/show/45941/ .

nko321 gravatar imagenko321 ( 2013-09-06 12:38:08 -0500 )edit
0

answered 2013-09-10 07:39:53 -0500

bishoy gravatar image

make sure the br-ex takes the conf static ip and give it a dns server and give this dns to the external and internal network from the dashboard this will make it work. i met this before.

edit flag offensive delete link more

Comments

2

no - br-ex does not need an ip address. Some people do this in non-production test installations as it makes it convenient to access the internal networks from the network node's IP stack without the bother of opening ip namespaces.

darragh-oreilly gravatar imagedarragh-oreilly ( 2013-09-13 06:33:35 -0500 )edit
0

answered 2014-07-25 17:18:13 -0500

duran gravatar image

I have this same problem using devstack icehouse , ml2 plugin, running in virtual box on win2k8 machine. The have bridged virtual box to guest ubuntu 12.04. If i ping from my launched instance to out side ip, i see arp request come to the eth0 of my guest os. The am able to reach outside and outside from/to but my launched instances are not. wondering if anybody has been able to figure out this issue

edit flag offensive delete link more
-1

answered 2013-09-09 17:26:03 -0500

dathomir gravatar image

updated 2013-09-09 18:25:11 -0500

Adding another data point here. It seems like i have a very similar problem, so I'll post here first. If it turns out to be significantly different problem, i'll start another thread; but perhaps my troubleshooting will give you some ideas.

It appears that the quantum gateway is not operating correctly.

Admin (controller)            Compute                      10.0.8.0/22 vpn network  | google
eth0 10.0.11.82                eth0  10.0.11.84          10.0.10.1   gateway        | 8.8.8.8
eth1 vnic assigned but no ip      eth1  vnic assigned no ip                         | 
                                                                                    |
       |   |                               |  |                                     |
| -------------  Vmware ESX vSphere Server ---       |                              | external sites

Setup is done with an ansible script https://github.com/ansible/ansible-redhat-openstack which sets up a provider router with private networks, aka vlan architecture. Admin and compute nodes are centos6.4 I differ slightly from the very well described vlan here http://www.mirantis.com/blog/openstack-networking-vlanmanager/ in that it uses a gre tunnel which alleviates the requirement of setting the switch to use vlan tags. 10.0.8.0/22 is the pre-existing corporate vpn subnet used for connecting to external ip addresses.

I've tried to get things going with two different external subnet configurations. In either case, a vm added to the compute node can ping 10.0.10.1, and anything in the vpn network.

A. Admin and Compute ip's (10.0.11.82, 10.0.11.84) in vpn subnet; external subnet is same as vpn subnet. Hosts Linux-OpenStack-Admin, Linux-OpenStack-Compute. external_subnet_cidr: 10.0.11.80/22 ip range [10.0.8.0 - 10.0.11.255] if this is supposed to match the large vpn network

The ansible-playbook creates the following routes.

[[root@Linux-OpenStack-Admin ~]# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
3.3.3.0         *               255.255.255.0   U     0      0        0 qr-03f621ea-8d
10.0.8.0        *               255.255.252.0   U     0      0        0 eth0
10.0.8.0        *               255.255.252.0   U     0      0        0 qg-33582141-61
link-local      *               255.255.0.0     U     1002   0        0 eth0
link-local      *               255.255.0.0     U     1003   0        0 eth1
default         10.0.8.1        0.0.0.0         UG    0      0        0 eth0
default         ciscoasa.21tech 0.0.0.0         UG    0      0        0 eth0

I can ssh into my vm at 3.3.3.2 on the compute node and ping 8.8.8.8 If i sit on admin and listen to qr, i see traffic, but not on qg.

[root@Linux-OpenStack-Admin ~]# tcpdump -i qr-03f621ea-8d | grep ICMP
17:16:17.902604 IP n003-000-000-000.static.ge.com > n003-000-000-000.static.ge.com: ICMP host google-public-dns-a.google.com unreachable, length 92
17:16:17.902633 IP n003-000-000-000.static.ge.com > n003-000-000-000.static.ge.com: ICMP host google-public-dns-a.google.com unreachable, length 92

listnening to eth0 i get something similar.

17:22:21.280780 IP Linux-OpenStack-Compute > Linux-OpenStack-Admin: GREv0, key=0x3, length 110: IP n003-000-000-000.static.ge.com > google-public-dns-a.google.com: ICMP echo request, id 24577, seq 20, length 64 ...
(more)
edit flag offensive delete link more

Comments

This is not an answer. If it's a different question, please ask a new one. https://ask.openstack.org/en/faq/

smaffulli gravatar imagesmaffulli ( 2014-01-10 15:38:18 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

3 followers

Stats

Asked: 2013-09-05 11:55:38 -0500

Seen: 8,259 times

Last updated: Sep 10 '13