Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

How can I fix a compute node network problem, in a Fedora 19 Havana 1 controller/1 compute node setup, 3 NIC's per server, using Neutron OVS vlans, built with packstack?

Greetings,

I have a 2 node (1 controller/compute, 1 compute) Centos6 Folsom Openstack cluster up and working great for a long time, and now I'm building a sister Fedora 19 Havana 2 node (1 controller/compute, 1 compute) setup using existing VLANs on my router. I successfully got this working in Grizzly using provider networks, but having trouble getting Havana to work.

My problem is: The instances launched on the compute node don't have external network access, can't ping any gateways, floating ip's do not work, and meta data is unavailable. They successfully get a dhcp address, and I can ssh to one via ip netns exec. To get metadata working based upon info in the metadata_agent.ini file, I used this iptables command on the compute node:

iptables -I PREROUTING -t nat -s 0.0.0.0/0 -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination ${CONTROLLER}:${METADATAPORT}.

The route to 169.254.0.0 already exists in the routing table on all interfaces. The meta data agent is not listening for requests on the compute node, dnsmasq is running in the process list. Tcpdump showed the metadata request going out, but not getting any replies until I added this rule. I thought the metadata agent should forward this request to the target in the metadata_agent.ini, is this correct?

I used packstack to configure both the controller and compute nodes. This one is with CONFIG_NEUTRON_L3_EXT_BRIDGE=provider. Here is my answer file

Goal: I would like to have a management vlan2 NIC em1, data trunk NIC em2, and external network vlan3 NIC p1p1, use the router to handle traffic flow and neutron router to handle floating IP's, using vlans, on a one controller/ multiple compute node setup.

If I have to ditch using my own router in the setup, that is fine.

Background:

For the Havana setup, each node has three NIC's: - em1 (in management vlan2 on the switch, ip 10.55.2.156), - em2 (trunked on the switch, no ip, used for data network bridge br-em2), - p1p1 (in external vlan3, no ip. I want to use this for external network access).

Router has the first ip in each subnet 10.55.2.1 (management) , 10.55.3.1 ( DMZ ), 10.55.4.1 (VM Private) , 10.55.5.1 (Ops). All IP's are pingable, and is in a known good state.

Since I have a router on my network, with gateway ip's for each VLAN, I think I should use provider networking for the CONFIG_NEUTRON_L3_EXT_BRIDGE. When I tried using 'br-em2' in the answer file for the L3 bridge I had a different problem: the vm's were unable to get anything network related, but I see the requests going out the interfaces using tcpdump.

*Although I reference p1p1 above, I can't a compute node to work with just the management/em1 and data/em2. When I tried to use p1p1, I created two physical networks and vlan ranges: "inter-vlan:vlan:2:100,ext-net:vlan:2:100" and mapped them to: "ext-net:br-p1p1,inter-vlan:br-em2". Then I created the bridges, added the interfaces to the bridges. When I create the neutron routers, I indicate which network, ext-net or inter-vlan, the router should attach to. This should work, right? I would like to do that, but the following setup doesn't use p1p1/ext-net at all.


After running packstack, the compute node has l3_agent, dhcp agent, and metadata agent running, where most tutorials say these should be off (packstack run with provider network turns them on, and when I ran it with br-em2 instead they were off and instance networking didn't work. I had to turn them on to get a dhcp ip). It has ip namespaces, but no qrouter namespace. (*another question: when I ran packstack with br-em2 for the external bridge instead of provider, no namespaces are available. is this correct?) I can only ping the vm on the compute node within the namespace, not on the controller node within the namespace, or anywhere.

Since provider network setup says it doesn't use the external bridge, but I set the external bridge in the answer file (br-em2), and it created the bridge and so I put the em2 interface in. I created the ifcfg-br-em2 file, and edited the ifcfg-em2 so it would persist after reboot.

I created a router within Neutron to be the gateway to the external network (is this the problem?), and attaching the VM Private subnet's interface, so I can assign floating IP's. Here are my neutron commands

Controller configuration output of iptables, ip netns tests, various config files, ovs

Compute node configuration output of iptables, ip netns tests, various config files, ovs

It looks like an iptables problem on the compute node, as if a rule is not getting added to flow traffic from one interface to another, or perhaps I don't need to have a neutron router, but everything works on the controller node. Should the qrouter namespace exist on the compute node? It does not, just the qdhcp namespaces. Could I have my ports trunked wrong on the switch? Any help is appreciated.

Thanks!

How can I fix a compute node network problem, in a Fedora 19 Havana 1 controller/1 compute node setup, 3 NIC's per server, using Neutron OVS vlans, built with packstack?

Greetings,

I have a 2 node (1 controller/compute, 1 compute) Centos6 Folsom Openstack cluster up and working great for a long time, and now I'm building a sister Fedora 19 Havana 2 node (1 controller/compute, 1 compute) setup using existing VLANs on my router. I successfully got this working in Grizzly using provider networks, but having trouble getting Havana to work.

My problem is: The instances launched on the compute node don't have external network access, can't ping any gateways, floating ip's do not work, and meta data is unavailable. They successfully get a dhcp address, and I can ssh to one via ip netns exec. To get metadata working based upon info in the metadata_agent.ini file, I used this iptables command on the compute node:

iptables -I PREROUTING -t nat -s 0.0.0.0/0 -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination ${CONTROLLER}:${METADATAPORT}.  

${CONTROLLER}:${METADATAPORT}.

The route to 169.254.0.0 already exists in the routing table on all interfaces. The meta data agent is not listening for requests on the compute node, dnsmasq is running in the process list. Tcpdump showed the metadata request going out, but not getting any replies until I added this rule. I thought the metadata agent should forward this request to the target in the metadata_agent.ini, is this correct?

I used packstack to configure both the controller and compute nodes. This one is with CONFIG_NEUTRON_L3_EXT_BRIDGE=provider. CONFIG_NEUTRON_L3_EXT_BRIDGE=provider. Here is my answer file

Goal: I would like to have a management vlan2 NIC em1, data trunk NIC em2, and external network vlan3 NIC p1p1, use the router to handle traffic flow and neutron router to handle floating IP's, using vlans, on a one controller/ multiple compute node setup.

If I have to ditch using my own router in the setup, that is fine.

Background:

For the Havana setup, each node has three NIC's: - NIC's:

  • em1 (in management vlan2 on the switch, ip 10.55.2.156), - 10.55.2.156),
  • em2 (trunked on the switch, no ip, used for data network bridge br-em2), - br-em2),
  • p1p1 (in external vlan3, no ip. I want to use this for external network access).

Router has the first ip in each subnet 10.55.2.1 (management) , 10.55.3.1 ( DMZ ), 10.55.4.1 (VM Private) , 10.55.5.1 (Ops). All IP's are pingable, and is in a known good state.

Since I have a router on my network, with gateway ip's for each VLAN, I think I should use provider networking for the CONFIG_NEUTRON_L3_EXT_BRIDGE. CONFIG_NEUTRON_L3_EXT_BRIDGE. When I tried using 'br-em2' br-em2 in the answer file for the L3 bridge I had a different problem: the vm's were unable to get anything network related, but I see the requests going out the interfaces using tcpdump.

*Although I reference p1p1 above, I can't a compute node to work with just the management/em1 and data/em2. When I tried to use p1p1, I created two physical networks and vlan ranges: "inter-vlan:vlan:2:100,ext-net:vlan:2:100" and mapped them to: "ext-net:br-p1p1,inter-vlan:br-em2". Then I created the bridges, added the interfaces to the bridges. When I create the neutron routers, I indicate which network, ext-net or inter-vlan, the router should attach to. This should work, right? I would like to do that, but the following setup doesn't use p1p1/ext-net at all.


After running packstack, the compute node has l3_agent, dhcp agent, and metadata agent running, where most tutorials say these should be off (packstack run with provider network turns them on, and when I ran it with br-em2 instead they were off and instance networking didn't work. I had to turn them on to get a dhcp ip). It has ip namespaces, but no qrouter namespace. (*another question: when I ran packstack with br-em2 for the external bridge instead of provider, no namespaces are available. is this correct?) I can only ping the vm on the compute node within the namespace, not on the controller node within the namespace, or anywhere.

Since provider network setup says it doesn't use the external bridge, but I set the external bridge in the answer file (br-em2), and it created the bridge and so I put the em2 interface in. I created the ifcfg-br-em2 file, and edited the ifcfg-em2 so it would persist after reboot.

I created a router within Neutron to be the gateway to the external network (is this the problem?), and attaching the VM Private subnet's interface, so I can assign floating IP's. Here are my neutron commands

Controller configuration output of iptables, ip netns tests, various config files, ovs

Compute node configuration output of iptables, ip netns tests, various config files, ovs

It looks like an iptables problem on the compute node, as if a rule is not getting added to flow traffic from one interface to another, or perhaps I don't need to have a neutron router, but everything works on the controller node. Should the qrouter namespace exist on the compute node? It does not, just the qdhcp namespaces. Could I have my ports trunked wrong on the switch? Any help is appreciated.

Thanks!