Ask Your Question
0

Quantum Multi-node setup instances cannot ping each other from different nodes

asked 2012-09-19 22:12:12 -0600

eoghank gravatar image

I’ve been trying to get Quantum multi-node with openvswitch working but cannot get the instances to be able to ping each other across nodes. I have tried with the milestone-proposed as well as the master branch of Quantum and in neither case can instances on different nodes ping each other.

I tried the devstack and used the flags indicated in the Quantum wiki for services on the second node so stackrc on the second node has this: ENABLED_SERVICES=n-cpu,rabbit,g-api,quantum,q-agt,q-dhcp

Localrc on the controller node (running all services) has this ENABLE_TENANT_VLAN=True TENANT_VLAN_RANGE=1:1000 PHYSICAL_NETWORK=eth0 OVS_PHYSICAL_BRIDGE=br-eth0

The physical host networking is eth0.160 as the primary and eth1.820 for the floating IP’s.

Ovs-vsctl on the controller node 8d3731cf-1ac2-4df8-86e5-47ed0a5106ac Bridge br-ex Port "qg-fe7b62d0-51" Interface "qg-fe7b62d0-51" type: internal Port br-ex Interface br-ex type: internal Bridge "br-eth0" Port "br-eth0" Interface "br-eth0" type: internal Port "phy-br-eth0" Interface "phy-br-eth0" Bridge br-int Port "tap55d1e5e8-ab" tag: 1 Interface "tap55d1e5e8-ab" type: internal Port "qr-4b50a17d-3c" tag: 1 Interface "qr-4b50a17d-3c" type: internal Port br-int Interface br-int type: internal Port "int-br-eth0" Interface "int-br-eth0" ovs_version: "1.4.0+build0"

Ovs-vsctl on the second node 38dbbac7-f643-4bdb-8aa1-83f06e612ec4 Bridge br-int Port br-int Interface br-int type: internal Port "tap26583155-34" tag: 1 Interface "tap26583155-34" type: internal ovs_version: "1.4.0+build0"

Instances come up and can be pinged in the namespace of the server where they are spun up but cannot ping from instances on the controller node to the second node.

I also tried the steps in this question https://answers.launchpad.net/quantum/+question/208822 (https://answers.launchpad.net/quantum...) but that did not resolve the problem either.

I checked the router namespace and I see the SNAT and DNAT rules are set correctly. What else needs to be added/corrected here to make multi-node work?

Thanks Eoghan

edit retag flag offensive close merge delete

5 answers

Sort by » oldest newest most voted
0

answered 2012-09-20 10:42:16 -0600

gongysh gravatar image

u need add physical eth device into br-eth0: for example ovs-vsctl add-port br-eth0 eth0

edit flag offensive delete link more
0

answered 2012-09-19 23:25:40 -0600

salvatore-orlando gravatar image

Hi Eoghan,

the first thing that struck me is that br-int on the second node does not have a port towards the physical bridge. As you're using the OVS plugin in VLAN mode, this port is required for sending packets over the physical network.

On the second node, is the ovs agent starting with the same settings as the controller node? In particular, I would check that the following flag is set:

bridge_mappings=eth0:br-eth0

edit flag offensive delete link more
0

answered 2012-09-20 00:55:50 -0600

eoghank gravatar image

Hi Salvatore

I edited the ovs_quantum_plugin.ini file on the second node to include this

[OVS] bridge_mappings = eth0:br-eth0

Then restarted the agent but I had to manually create br-eth0 before it restarted and when it did start this is what ovs-vsctl looks like on the second node but instances on different nodes still could not ping each other.

38dbbac7-f643-4bdb-8aa1-83f06e612ec4 Bridge "br-eth0" Port "br-eth0" Interface "br-eth0" type: internal Port "phy-br-eth0" Interface "phy-br-eth0" Bridge br-int Port "int-br-eth0" Interface "int-br-eth0" Port "tap6d40cc67-9b" tag: 1 Interface "tap6d40cc67-9b" Port "tap06b6c317-bf" tag: 1 Interface "tap06b6c317-bf" Port "tapd59af817-c0" tag: 1 Interface "tapd59af817-c0" Port br-int Interface br-int type: internal Port "tap26583155-34" tag: 1 Interface "tap26583155-34" type: internal ovs_version: "1.4.0+build0"

edit flag offensive delete link more
0

answered 2012-09-20 21:08:27 -0600

eoghank gravatar image

I added eth0 to br-eth0 on both nodes but instances on nodeA still cannot ping instances on nodeB and vice versa. ovs-vsctl show for br-eth0 is the same on both nodes:

Bridge "br-eth0" Port "br-eth0" Interface "br-eth0" type: internal Port "eth0" Interface "eth0" Port "phy-br-eth0" Interface "phy-br-eth0"

When I looked at tcpdump I see the requests with the instance IP going out of int-br-eth0 and out of phy-br-eth0 but they do not appear to get up to eth0. I do see traffic form phy-br-eth0 to etho but nothing that indicates any traffic from the instance IP. Are there any specific paramaters required for setting up the ovs bridges other that what is being used in stack.sh?

edit flag offensive delete link more
0

answered 2012-09-21 14:21:59 -0600

eoghank gravatar image

Thanks for the help Salvator and Yong.

I got the answer to the rest of this problem in this thread https://answers.launchpad.net/quantum/+question/209168 (https://answers.launchpad.net/quantum...)

Eoghan

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2012-09-19 22:12:12 -0600

Seen: 73 times

Last updated: Sep 21 '12