Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Cannot connect to Openstack instances

Setup is as follows, workstation <-> Maas server <-> 3 VM's - Instances

Maas Server and the VM's are deployed in vSphere.

The Maas server's eth1 has 10.4.x.x which is acting as DHCP manager for the VM's. My workstation's ip is 10.x.x.x and I am able to connect to the Maas server and the VM's via ssh. I have one dedicated compute node and the network node has two interfaces. eth0 (br0) has an ip-address from the maas-server and eth1 (br-ex) gets a publicly accessible ip address from 192.168.x.x range( external network )

The problem is I cannot connect to the instances directly from my workstation or any other terminal in our network. I have identified the problem to be within the router namespace. I cannot ping 8.8.8.8 from the neutron router namespace. It is able to ping br-ex, but shows 'destination unreachable' when I ping 192.168.x.1. Please have a look at the output of the following commands,

ip netns

qrouter-1ff7cd3f-7d73-4c7a-a9aa-0f8efb14a718

ip netns exec qrouter-1ff7cd3f-7d73-4c7a-a9aa-0f8efb14a718 ifconfig

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:16 errors:0 dropped:0 overruns:0 frame:0 TX packets:16 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1776 (1.7 KB) TX bytes:1776 (1.7 KB)

qg-80014d1d-df Link encap:Ethernet HWaddr fa:16:3e:b8:62:2e
inet addr:192.168.x.x Bcast:192.168.x.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:feb8:622e/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:21036 errors:0 dropped:0 overruns:0 frame:0 TX packets:3095 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:4857288 (4.8 MB) TX bytes:1047570 (1.0 MB)

ip netns exec qrouter-1ff7cd3f-7d73-4c7a-a9aa-0f8efb14a718 ip route list

default via 192.168.x.1 dev qg-80014d1d-df

192.168.x.0/24 dev qg-80014d1d-df proto kernel scope link src 192.168.x.x

I have seen neutron setups where the router interface( qg-80014d1d-df )is included in the bridge br-ex. Also the diagram at openstack.org for neutron troubleshooting shows that br-ex should contain a 'qg' interface.

enter image description here

But in my node it shows different,

ovs-vsctl show

f50f0af5-a5ab-4ab8-935a-9eaf71b40eea

Bridge br-int

    fail_mode: secure
    Port br-int
        Interface br-int
            type: internal
    Port patch-tun
        Interface patch-tun
            type: patch
            options: {peer=patch-int}

Bridge br-ex
    Port "tap80014d1d-df"
        Interface "tap80014d1d-df"
    Port br-ex
        Interface br-ex
            type: internal
    Port "eth1"
        Interface "eth1"

Bridge br-tun
    Port br-tun
        Interface br-tun
            type: internal
    Port patch-int
        Interface patch-int
            type: patch
            options: {peer=patch-tun}
    Port "gre-0a04546a"
        Interface "gre-0a04546a"
            type: gre
            options: {in_key=flow, local_ip="10.4.x.x", out_key=flow, remote_ip="10.4.x.x"}
ovs_version: "2.0.2"

Here br-ex does get an additional interface but that is 'tap'. The numeric value for this 'tap' and 'qg' interface are same though. I have checked the log files on both the compute node and controller, but did not find anything significant. I can spawn instances and assign them floating ip's. I am also able to ping br-ex, but not the router interface within it. Not sure if the problem is caused by the 'tap' interface but there is no flow between the router namespace and outside world. Another thought came to my mind if the problem is caused by VMware vSphere !!!

One thing should be mentioned, while posting the outputs there is only external network and a router to check connectivity. There is no instances or internal network.

I am stuck on this thing like forever. Any suggestion would be highly appreciated.

Cannot connect to Openstack instances

Setup is as follows, workstation <-> Maas server <-> 3 VM's - Instances

Maas Server and the VM's are deployed in vSphere.

The Maas server's eth1 has 10.4.x.x which is acting as DHCP manager for the VM's. My workstation's ip is 10.x.x.x and I am able to connect to the Maas server and the VM's via ssh. I have one dedicated compute node and the network node has two interfaces. eth0 (br0) has an ip-address from the maas-server and eth1 (br-ex) gets a publicly accessible ip address from 192.168.x.x range( external network )

The problem is I cannot connect to the instances directly from my workstation or any other terminal in our network. I have identified the problem to be within the router namespace. I cannot ping 8.8.8.8 from the neutron router namespace. It is able to ping br-ex, but shows 'destination unreachable' when I ping 192.168.x.1. Please have a look at the output of the following commands,

ip netns

qrouter-1ff7cd3f-7d73-4c7a-a9aa-0f8efb14a718

ip netns exec qrouter-1ff7cd3f-7d73-4c7a-a9aa-0f8efb14a718 ifconfig

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:16 errors:0 dropped:0 overruns:0 frame:0 TX packets:16 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1776 (1.7 KB) TX bytes:1776 (1.7 KB)

qg-80014d1d-df Link encap:Ethernet HWaddr fa:16:3e:b8:62:2e
inet addr:192.168.x.x Bcast:192.168.x.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:feb8:622e/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:21036 errors:0 dropped:0 overruns:0 frame:0 TX packets:3095 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:4857288 (4.8 MB) TX bytes:1047570 (1.0 MB)

ip netns exec qrouter-1ff7cd3f-7d73-4c7a-a9aa-0f8efb14a718 ip route list

default via 192.168.x.1 dev qg-80014d1d-df

192.168.x.0/24 dev qg-80014d1d-df proto kernel scope link src 192.168.x.x

I have seen neutron setups where the router interface( qg-80014d1d-df )is included in the bridge br-ex. Also the diagram at openstack.org for neutron troubleshooting shows that br-ex should contain a 'qg' interface.

enter image description here

But in my node it shows different,

ovs-vsctl show

f50f0af5-a5ab-4ab8-935a-9eaf71b40eea

Bridge br-int

    fail_mode: secure
    Port br-int
        Interface br-int
            type: internal
    Port patch-tun
        Interface patch-tun
            type: patch
            options: {peer=patch-int}

Bridge br-ex
    Port "tap80014d1d-df"
        Interface "tap80014d1d-df"
    Port br-ex
        Interface br-ex
            type: internal
    Port "eth1"
        Interface "eth1"

Bridge br-tun
    Port br-tun
        Interface br-tun
            type: internal
    Port patch-int
        Interface patch-int
            type: patch
            options: {peer=patch-tun}
    Port "gre-0a04546a"
        Interface "gre-0a04546a"
            type: gre
            options: {in_key=flow, local_ip="10.4.x.x", out_key=flow, remote_ip="10.4.x.x"}
ovs_version: "2.0.2"

Here br-ex does get an additional interface but that is 'tap'. The numeric value for this 'tap' and 'qg' interface are same though. I have checked the log files on both the compute node and controller, but did not find anything significant. I can spawn instances and assign them floating ip's. I am also able to ping br-ex, but not the router interface within it. Not sure if the problem is caused by the 'tap' interface but there is no flow between the router namespace and outside world. Another thought came to my mind if the problem is caused by VMware vSphere !!!

One thing should be mentioned, while posting the outputs there is only external network and a router to check connectivity. There is no instances or internal network.

I am stuck on this thing like forever. Any suggestion would be highly appreciated.

Cannot Router namespace issue;cannot connect to Openstack instances

Setup is as follows, workstation <-> Maas server <-> 3 VM's - Instances

Maas Server and the VM's are deployed in vSphere.

The Maas server's eth1 has 10.4.x.x which is acting as DHCP manager for the VM's. My workstation's ip is 10.x.x.x and I am able to connect to the Maas server and the VM's via ssh. I have one dedicated compute node and the network node has two interfaces. eth0 (br0) has an ip-address from the maas-server and eth1 (br-ex) gets a publicly accessible ip address from 192.168.x.x range( external network )

The problem is I cannot connect to the instances directly from my workstation or any other terminal in our network. I have identified the problem to be within the router namespace. I cannot ping 8.8.8.8 from the neutron router namespace. It is able to ping br-ex, but shows 'destination unreachable' when I ping 192.168.x.1. Please have a look at the output of the following commands,

ip netns

qrouter-1ff7cd3f-7d73-4c7a-a9aa-0f8efb14a718

ip netns exec qrouter-1ff7cd3f-7d73-4c7a-a9aa-0f8efb14a718 ifconfig

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:16 errors:0 dropped:0 overruns:0 frame:0 TX packets:16 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1776 (1.7 KB) TX bytes:1776 (1.7 KB)

qg-80014d1d-df Link encap:Ethernet HWaddr fa:16:3e:b8:62:2e
inet addr:192.168.x.x Bcast:192.168.x.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:feb8:622e/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:21036 errors:0 dropped:0 overruns:0 frame:0 TX packets:3095 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:4857288 (4.8 MB) TX bytes:1047570 (1.0 MB)

ip netns exec qrouter-1ff7cd3f-7d73-4c7a-a9aa-0f8efb14a718 ip route list

default via 192.168.x.1 dev qg-80014d1d-df

192.168.x.0/24 dev qg-80014d1d-df proto kernel scope link src 192.168.x.x

I have seen neutron setups where the router interface( qg-80014d1d-df )is included in the bridge br-ex. Also the diagram at openstack.org for neutron troubleshooting shows that br-ex should contain a 'qg' interface.

But in my node it shows different,

ovs-vsctl show

f50f0af5-a5ab-4ab8-935a-9eaf71b40eea

Bridge br-int

    fail_mode: secure
    Port br-int
        Interface br-int
            type: internal
    Port patch-tun
        Interface patch-tun
            type: patch
            options: {peer=patch-int}

Bridge br-ex
    Port "tap80014d1d-df"
        Interface "tap80014d1d-df"
    Port br-ex
        Interface br-ex
            type: internal
    Port "eth1"
        Interface "eth1"

Bridge br-tun
    Port br-tun
        Interface br-tun
            type: internal
    Port patch-int
        Interface patch-int
            type: patch
            options: {peer=patch-tun}
    Port "gre-0a04546a"
        Interface "gre-0a04546a"
            type: gre
            options: {in_key=flow, local_ip="10.4.x.x", out_key=flow, remote_ip="10.4.x.x"}
ovs_version: "2.0.2"

Here br-ex does get an additional interface but that is 'tap'. The numeric value for this 'tap' and 'qg' interface are same though. I have checked the log files on both the compute node and controller, but did not find anything significant. I can spawn instances and assign them floating ip's. I am also able to ping br-ex, but not the router interface within it. Not sure if the problem is caused by the 'tap' interface but there is no flow between the router namespace and outside world. Another thought came to my mind if the problem is caused by VMware vSphere !!!

One thing should be mentioned, while posting the outputs there is only external network and a router to check connectivity. There is no instances or internal network.

I am stuck on this thing like forever. Any suggestion would be highly appreciated.

Router namespace issue;cannot connect to Openstack instances

Setup is as follows, workstation <-> Maas server <-> 3 VM's - Instances

Maas Server and the VM's are deployed in vSphere.

The Maas server's eth1 has 10.4.x.x which is acting as DHCP manager for the VM's. My workstation's ip is 10.x.x.x and I am able to connect to the Maas server and the VM's via ssh. I have one dedicated compute node and the network node has two interfaces. eth0 (br0) has an ip-address from the maas-server and eth1 (br-ex) gets a publicly accessible ip address from 192.168.x.x range( external network )

The problem is I cannot connect to the instances directly from my workstation or any other terminal in our network. I have identified the problem to be within the router namespace. I cannot ping 8.8.8.8 from the neutron router namespace. It is able to ping br-ex, but shows 'destination unreachable' when I ping 192.168.x.1. Please have a look at the output of the following commands,

ip netns

qrouter-1ff7cd3f-7d73-4c7a-a9aa-0f8efb14a718

ip netns exec qrouter-1ff7cd3f-7d73-4c7a-a9aa-0f8efb14a718 ifconfig

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:16 errors:0 dropped:0 overruns:0 frame:0 TX packets:16 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1776 (1.7 KB) TX bytes:1776 (1.7 KB)

qg-80014d1d-df Link encap:Ethernet HWaddr fa:16:3e:b8:62:2e
inet addr:192.168.x.x Bcast:192.168.x.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:feb8:622e/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:21036 errors:0 dropped:0 overruns:0 frame:0 TX packets:3095 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:4857288 (4.8 MB) TX bytes:1047570 (1.0 MB)

ip netns exec qrouter-1ff7cd3f-7d73-4c7a-a9aa-0f8efb14a718 ip route list

default via 192.168.x.1 dev qg-80014d1d-df

192.168.x.0/24 dev qg-80014d1d-df proto kernel scope link src 192.168.x.x

I have seen neutron setups where the router interface( qg-80014d1d-df )is included in the bridge br-ex. Also the diagram at openstack.org for neutron troubleshooting shows that br-ex should contain a 'qg' interface.

But in my node it shows different,

ovs-vsctl show

f50f0af5-a5ab-4ab8-935a-9eaf71b40eea

Bridge br-int

    fail_mode: secure
    Port br-int
        Interface br-int
            type: internal
    Port patch-tun
        Interface patch-tun
            type: patch
            options: {peer=patch-int}

Bridge br-ex
    Port "tap80014d1d-df"
        Interface "tap80014d1d-df"
    Port br-ex
        Interface br-ex
            type: internal
    Port "eth1"
        Interface "eth1"

Bridge br-tun
    Port br-tun
        Interface br-tun
            type: internal
    Port patch-int
        Interface patch-int
            type: patch
            options: {peer=patch-tun}
    Port "gre-0a04546a"
        Interface "gre-0a04546a"
            type: gre
            options: {in_key=flow, local_ip="10.4.x.x", out_key=flow, remote_ip="10.4.x.x"}
ovs_version: "2.0.2"

Here br-ex does get an additional interface but that is 'tap'. The numeric value for this 'tap' and 'qg' interface are same though. I have checked the log files on both the compute node and controller, but did not find anything significant. I can spawn instances and assign them floating ip's. I am also able to ping br-ex, but not the router interface within it. Not sure if the problem is caused by the 'tap' interface but there is no flow between the router namespace and outside world. Another thought came to my mind if the problem is caused by VMware vSphere !!!

One thing should be mentioned, while posting the outputs there is only external network and a router to check connectivity. There is no instances or internal network.

I am stuck on this thing like forever. Any suggestion would be highly appreciated.