Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Icehouse Instance cannot ping Compute node

I'm running Icehouse on 3 nodes (Controller, Compute and Network (Neutron)) each on 3 different Ubuntu 14.04 servers and 1 instance running cirros.

From the compute and network nodes I can ping the external router address and the instance floating IP. From the instance I can ping both the compute and the network nodes mgmt IPs as well as the router. However from the compute node I cannot ping the router nor the floating IP and from the instance I cannot ping the compute mgmt IP.

I can SSH to the instance from another computer on my network however the instance itself cannout resolve any DNS names or access the internet:

$ ssh cirros@192.168.1.201
cirros@192.168.1.201's password: 
$ nslookup www.google.com
Server:    172.21.21.2
Address 1: 172.21.21.2 host-172-21-21-2.openstacklocal

nslookup: can't resolve 'www.google.com'
$ ping -c 4 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
^C
--- 8.8.8.8 ping statistics ---
4 packets transmitted, 0 packets received, 100% packet loss

I also get PORT/LINK DOWN on br-int on the compute node:

root@stack-compute:~# ovs-ofctl show br-int
OFPT_FEATURES_REPLY (xid=0x2): dpid:00006677f5503e4a
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE
 1(patch-tun): addr:5a:79:46:d9:cd:5f
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 2(qvofaa9d401-aa): addr:5a:aa:ef:d4:84:6b
     config:     0
     state:      0
     current:    10GB-FD COPPER
     speed: 10000 Mbps now, 0 Mbps max
 LOCAL(br-int): addr:56:24:3f:87:6d:27
     config:     PORT_DOWN
     state:      LINK_DOWN
     speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0

This is var/log/neutron/openvswitch-agent.log on the compute node:

2014-06-02 11:49:43.113 1938 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-fbc17718-4980-4cd6-8a3f-a7570aa769ba None] Agent initialized successfully, now running...
2014-06-02 11:49:43.118 1938 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-fbc17718-4980-4cd6-8a3f-a7570aa769ba None] Agent out of sync with plugin!
2014-06-02 11:49:43.120 1938 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-fbc17718-4980-4cd6-8a3f-a7570aa769ba None] Agent tunnel out of sync with plugin!

Everything looks normal on /var/log/openvswitch/ovs-vswitchd.log:

2014-06-02T01:49:39.320Z|00014|bridge|INFO|ovs-vswitchd (Open vSwitch) 2.0.1
2014-06-02T01:49:42.410Z|00015|bridge|INFO|bridge br-tun: added interface br-tun on port 65534
2014-06-02T01:49:42.410Z|00016|bridge|INFO|bridge br-tun: using datapath ID 0000fee6495d2340
2014-06-02T01:49:42.410Z|00017|connmgr|INFO|br-tun: added service controller "punix:/var/run/openvswitch/br-tun.mgmt"
2014-06-02T01:49:42.463Z|00018|bridge|INFO|bridge br-int: added interface patch-tun on port 1
2014-06-02T01:49:42.555Z|00019|bridge|INFO|bridge br-tun: added interface patch-int on port 1
2014-06-02T01:49:43.281Z|00020|bridge|INFO|bridge br-tun: added interface gre-0a010116 on port 2
2014-06-02T01:49:43.450Z|00021|bridge|INFO|bridge br-int: added interface qvofaa9d401-aa on port 2
2014-06-02T01:49:49.304Z|00022|memory|INFO|32000 kB peak resident set size after 10.1 seconds
2014-06-02T01:49:49.304Z|00023|memory|INFO|facets:4 ports:6 rules:19 subfacets:4
2014-06-02T01:49:52.307Z|00024|ofproto|INFO|br-int: 3 flow_mods in the 4 s starting 10 s ago (1 adds, 2 deletes)
2014-06-02T01:49:53.308Z|00025|ofproto|INFO|br-tun: 13 flow_mods in the 3 s starting 10 s ago (11 adds, 1 deletes, 1 modifications)

Any ideas on how to get the instance to access other subnets and the internet? It seems to be a problem between the instance and the compute node.

Icehouse Instance cannot ping Compute node

I'm running Icehouse on 3 nodes (Controller, Compute and Network (Neutron)) each on 3 different Ubuntu 14.04 servers and 1 instance running cirros.

From the compute controller and network nodes I can ping the external router address and the instance floating IP. From the instance I can ping both the compute controller and the network nodes mgmt IPs as well as the router. However from the compute node I cannot ping the router nor the floating IP and from the instance I cannot ping the compute mgmt IP.

I can SSH to the instance from another computer on my network however the instance itself cannout resolve any DNS names or access the internet:

$ ssh cirros@192.168.1.201
cirros@192.168.1.201's password: 
$ nslookup www.google.com
Server:    172.21.21.2
Address 1: 172.21.21.2 host-172-21-21-2.openstacklocal

nslookup: can't resolve 'www.google.com'
$ ping -c 4 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
^C
--- 8.8.8.8 ping statistics ---
4 packets transmitted, 0 packets received, 100% packet loss

I also get PORT/LINK DOWN on br-int on the compute node:

root@stack-compute:~# ovs-ofctl show br-int
OFPT_FEATURES_REPLY (xid=0x2): dpid:00006677f5503e4a
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE
 1(patch-tun): addr:5a:79:46:d9:cd:5f
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 2(qvofaa9d401-aa): addr:5a:aa:ef:d4:84:6b
     config:     0
     state:      0
     current:    10GB-FD COPPER
     speed: 10000 Mbps now, 0 Mbps max
 LOCAL(br-int): addr:56:24:3f:87:6d:27
     config:     PORT_DOWN
     state:      LINK_DOWN
     speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0

This is var/log/neutron/openvswitch-agent.log on the compute node:

2014-06-02 11:49:43.113 1938 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-fbc17718-4980-4cd6-8a3f-a7570aa769ba None] Agent initialized successfully, now running...
2014-06-02 11:49:43.118 1938 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-fbc17718-4980-4cd6-8a3f-a7570aa769ba None] Agent out of sync with plugin!
2014-06-02 11:49:43.120 1938 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-fbc17718-4980-4cd6-8a3f-a7570aa769ba None] Agent tunnel out of sync with plugin!

Everything looks normal on /var/log/openvswitch/ovs-vswitchd.log:

2014-06-02T01:49:39.320Z|00014|bridge|INFO|ovs-vswitchd (Open vSwitch) 2.0.1
2014-06-02T01:49:42.410Z|00015|bridge|INFO|bridge br-tun: added interface br-tun on port 65534
2014-06-02T01:49:42.410Z|00016|bridge|INFO|bridge br-tun: using datapath ID 0000fee6495d2340
2014-06-02T01:49:42.410Z|00017|connmgr|INFO|br-tun: added service controller "punix:/var/run/openvswitch/br-tun.mgmt"
2014-06-02T01:49:42.463Z|00018|bridge|INFO|bridge br-int: added interface patch-tun on port 1
2014-06-02T01:49:42.555Z|00019|bridge|INFO|bridge br-tun: added interface patch-int on port 1
2014-06-02T01:49:43.281Z|00020|bridge|INFO|bridge br-tun: added interface gre-0a010116 on port 2
2014-06-02T01:49:43.450Z|00021|bridge|INFO|bridge br-int: added interface qvofaa9d401-aa on port 2
2014-06-02T01:49:49.304Z|00022|memory|INFO|32000 kB peak resident set size after 10.1 seconds
2014-06-02T01:49:49.304Z|00023|memory|INFO|facets:4 ports:6 rules:19 subfacets:4
2014-06-02T01:49:52.307Z|00024|ofproto|INFO|br-int: 3 flow_mods in the 4 s starting 10 s ago (1 adds, 2 deletes)
2014-06-02T01:49:53.308Z|00025|ofproto|INFO|br-tun: 13 flow_mods in the 3 s starting 10 s ago (11 adds, 1 deletes, 1 modifications)

Any ideas on how to get the instance to access other subnets and the internet? It seems to be a problem between the instance and the compute node.

Icehouse Instance cannot ping Compute node

I'm running Icehouse on 3 nodes (Controller, Compute and Network (Neutron)) each on 3 different Ubuntu 14.04 servers and 1 instance running cirros.

From the controller and network nodes I can ping the external router address and the instance floating IP. From the instance I can ping both the controller and the network nodes mgmt IPs as well as the router. However from the compute node I cannot ping the router nor the floating IP and from the instance I cannot ping the compute mgmt IP.

I can SSH to the instance from another computer on my network however the instance itself cannout cannot resolve any DNS names or access the internet:

$ ssh cirros@192.168.1.201
cirros@192.168.1.201's password: 
$ nslookup www.google.com
Server:    172.21.21.2
Address 1: 172.21.21.2 host-172-21-21-2.openstacklocal

nslookup: can't resolve 'www.google.com'
$ ping -c 4 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
^C
--- 8.8.8.8 ping statistics ---
4 packets transmitted, 0 packets received, 100% packet loss

I also get PORT/LINK DOWN on br-int on the compute node:

root@stack-compute:~# ovs-ofctl show br-int
OFPT_FEATURES_REPLY (xid=0x2): dpid:00006677f5503e4a
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE
 1(patch-tun): addr:5a:79:46:d9:cd:5f
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 2(qvofaa9d401-aa): addr:5a:aa:ef:d4:84:6b
     config:     0
     state:      0
     current:    10GB-FD COPPER
     speed: 10000 Mbps now, 0 Mbps max
 LOCAL(br-int): addr:56:24:3f:87:6d:27
     config:     PORT_DOWN
     state:      LINK_DOWN
     speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0

When I execute on the compute node:

service openvswitch-switch restart

The link changes to UP however the instance loses all connectivity to anything on the network, including the servers it could talk to before.

This is var/log/neutron/openvswitch-agent.log on the compute node:

2014-06-02 11:49:43.113 1938 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-fbc17718-4980-4cd6-8a3f-a7570aa769ba None] Agent initialized successfully, now running...
2014-06-02 11:49:43.118 1938 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-fbc17718-4980-4cd6-8a3f-a7570aa769ba None] Agent out of sync with plugin!
2014-06-02 11:49:43.120 1938 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-fbc17718-4980-4cd6-8a3f-a7570aa769ba None] Agent tunnel out of sync with plugin!

Everything looks normal on /var/log/openvswitch/ovs-vswitchd.log:

2014-06-02T01:49:39.320Z|00014|bridge|INFO|ovs-vswitchd (Open vSwitch) 2.0.1
2014-06-02T01:49:42.410Z|00015|bridge|INFO|bridge br-tun: added interface br-tun on port 65534
2014-06-02T01:49:42.410Z|00016|bridge|INFO|bridge br-tun: using datapath ID 0000fee6495d2340
2014-06-02T01:49:42.410Z|00017|connmgr|INFO|br-tun: added service controller "punix:/var/run/openvswitch/br-tun.mgmt"
2014-06-02T01:49:42.463Z|00018|bridge|INFO|bridge br-int: added interface patch-tun on port 1
2014-06-02T01:49:42.555Z|00019|bridge|INFO|bridge br-tun: added interface patch-int on port 1
2014-06-02T01:49:43.281Z|00020|bridge|INFO|bridge br-tun: added interface gre-0a010116 on port 2
2014-06-02T01:49:43.450Z|00021|bridge|INFO|bridge br-int: added interface qvofaa9d401-aa on port 2
2014-06-02T01:49:49.304Z|00022|memory|INFO|32000 kB peak resident set size after 10.1 seconds
2014-06-02T01:49:49.304Z|00023|memory|INFO|facets:4 ports:6 rules:19 subfacets:4
2014-06-02T01:49:52.307Z|00024|ofproto|INFO|br-int: 3 flow_mods in the 4 s starting 10 s ago (1 adds, 2 deletes)
2014-06-02T01:49:53.308Z|00025|ofproto|INFO|br-tun: 13 flow_mods in the 3 s starting 10 s ago (11 adds, 1 deletes, 1 modifications)

Any ideas on how to get the instance to access other subnets and the internet? It seems to be a problem between the instance and the compute node.