Ask Your Question
2

No internet connectivity on Instances with each other and external Network [closed]

asked 2015-08-07 02:25:58 -0600

smcas gravatar image

updated 2015-09-09 06:02:19 -0600

We have configured 3-node setup of openstack icehouse on CentOS : Controller , Compute and Network Node.

We have following networks :
Public Network - 192.168.10.0/16
Management Network - 10.0.0.0/24
GRE TUNNEL Network - 10.0.1.0/24
VM TRAFFIC-FloatingIp Network - 192.168.12.0/16

image description image description

we create another network i.e, 172.16.24.0/24 which is available for instances running on compute node.

We are facing following problems:

1. Instances are getting IP from Floating IP through DHCP. Though, IP is displayed on the dashboard but when we execute 'ifconfig' from VM terminal, it does not show any IP.
2. No internet access  in the virtual machine.
3. Router interface and gateway always down. 
4. Ports are showing down on br-int .
5. VMs are not able to ping the compute node and external network.
6. ip a  on any node shows UNKNOWN or DOWN status of interfaces.
7. ip netns does not show router namespace.
8. ovs-vsctl shows dead tag 4095.

OUR CONFIGURATION IS :

Bridge config:

DEVICE=br-ex
DEVICETYPE=ovs
TYPE=OVSIntPort
OVS_BRIDGE=br-ex
NM_CONTROLLED=nO
MACADDR=f8:0f:41:fc:48:65
BOOTPROTO=static
ONBOOT=yes
IPADDR=192.168.12.2
PREFIX=16
NETMASK=255.255.0.0
GATEWAY=192.168.0.3
DNS1=202.138.96.2
DNS2=8.8.8.8
DEFROUTE=yes
IPV4_FAILURE_FATAL=yes
IPV6INIT=no

Ethernet port config:

DEVICE=eth3
HWADDR=F8:0F:41:FC:48:65
TYPE=OVSPort
DEVICETYPE=ovs
OVS_BRIDGE=br-ex
UUID=e84b351d-bfc7-427f-8dda-84a1b1df5f26
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=none
IPV4_FAILURE_FATAL=yes
IPV6INIT=no

ovs-vsctl show on network node:

[root@newnetwork ~]# ovs-vsctl show
821c052b-827b-41db-9412-cfa33407d615
    Bridge br-ex
        Port phy-br-ex
            Interface phy-br-ex
        Port "eth3"
            Interface "eth3"
        Port br-ex
            Interface br-ex
                type: internal
    Bridge br-tun
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
    Bridge br-int
        fail_mode: secure
        Port "tap9300d1c6-f4"
            tag: 4095
            Interface "tap9300d1c6-f4"
                type: internal
        Port br-int
            Interface br-int
                type: internal
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port int-br-ex
            Interface int-br-ex
    ovs_version: "2.1.3"

ovs-vsctl on compute node:

[root@newcompute1 ~]# ovs-vsctl show
8290fd1a-7636-4887-b108-82768338455d
    Bridge br-tun
        Port br-tun
            Interface br-tun
                type: internal
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port "gre-0a000104"
            Interface "gre-0a000104"
                type: gre
                options: {in_key=flow, local_ip="10.0.1.5", out_key=flow, remote_ip="10.0.1.4"}
    Bridge br-int
        fail_mode: secure
        Port "qvo625284a7-2c"
            tag: 4095
            Interface "qvo625284a7-2c"
        Port br-int
            Interface br-int
                type: internal
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
    ovs_version: "2.1.3"

As per the suggestions we have made changes and the internet connectivity on network node is fine after adding bridge but still have issues.. please suggest what we are still missing. We are waiting for the response.

edit retag flag offensive reopen merge delete

Closed for the following reason duplicate question by smcas
close date 2015-10-28 06:20:18.702022

6 answers

Sort by » oldest newest most voted
2

answered 2015-08-07 04:01:11 -0600

dbaxps gravatar image

updated 2015-08-12 06:24:30 -0600

Working configuration should be as follows :-
Remove public network and create new external network "public" ( or whatever ) 172.16.3.0/24 with gateway having IP matching IP of real router to Internet. Floating IPs of your VMs should be also on 172.16.3.0/24
Br-ex on Network Node should have IP on 172.16.3.0/24, one of Ethernet interfaces on Network Node
should be OVS port of OVS bridge br-ex

image description

[root@ip-192-169-142-147 network-scripts]# cat ifcfg-br-ex
DEVICE="br-ex"
BOOTPROTO="static"
IPADDR="172.24.4.232"
NETMASK="255.255.255.240"
DNS1="83.221.202.254"
BROADCAST="172.24.4.239"
GATEWAY="172.24.4.225"
NM_CONTROLLED="no"
TYPE="OVSIntPort"
OVS_BRIDGE=br-ex
DEVICETYPE="ovs"
DEFROUTE="yes"
IPV4_FAILURE_FATAL="yes"
IPV6INIT=no


[root@ip-192-169-142-147 network-scripts]# cat ifcfg-eth2
DEVICE="eth2"
HWADDR=00:22:15:63:E4:E2
ONBOOT="yes"
TYPE="OVSPort"
DEVICETYPE="ovs"
OVS_BRIDGE=br-ex
NM_CONTROLLED=no
IPV6INIT=no
edit flag offensive delete link more

Comments

Hi Boris, we followed your ovs configuration but something went wrong and our network node is unable to access the internet. Can you please share your email ID or linkedIn ID so that we remain in touch with you.

smcas gravatar imagesmcas ( 2015-08-13 07:00:31 -0600 )edit
dbaxps gravatar imagedbaxps ( 2015-08-13 07:05:28 -0600 )edit
0

answered 2015-08-12 05:20:09 -0600

smcas gravatar image

updated 2015-09-03 07:58:53 -0600

ROUTER GATEWAY AND INTERNAL INTERFACE STATUS ARE DOWN ?

As per your suggestion for the above question we have created new external network "public" 192.168.12.0/24 with gateway having IP matching IP of real router to Internet(which is 192.168.0.3). After adding bridge to the network node, internet connectivity is working fine but, following issues comes

  • when checked in horizon we found the routers internal interface and external interface/gateway status are down and admin status is up(refer the image attached).
  • ifconfig command doesn't show any ip address in VMs but both ip's(internal address as well as floating ip) are displayed on dashboard.
  • virtual machine are not able to ping each other as well as internet

    image description

For OVS configuration on one of the ethernet interface on network node and OVS bridge, we have made following configuration in ethernet and bridge device:

Bridge device configuration:

DEVICE=br-ex
DEVICETYPE=ovs
TYPE=OVSBridge
MACADDR=f8:0f:41:fc:48:65
BOOTPROTO=static
ONBOOT=yes
IPADDR=192.168.12.2
PREFIX=16
NETMASK=255.255.0.0
GATEWAY=192.168.0.3
DNS1=202.138.96.2
DNS2=8.8.8.8

Ethernet port configuration :

DEVICE=eth3
HWADDR=F8:0F:41:FC:48:65
#TYPE=Ethernet
TYPE=OVSPort
DEVICETYPE=ovs
OVS_BRIDEGE=br-ex
UUID=e84b351d-bfc7-427f-8dda-84a1b1df5f26
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=none
IPADDR=192.168.12.2
PREFIX=16
NETMASK=255.255.0.0
GATEWAY=192.168.0.3
DNS=202.138.96.2
DNS2=8.8.8.8
DEFROUTE=yes
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
NAME="System eth3"

In file /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini , we updated

bridge_mapping=external:br-ex

In file /etc/neutron/plugin.ini , we updated

type_drivers=gre,flat,vxlan,vlan

and ovs-vsctl show

[root@newnetwork network-scripts]# ovs-vsctl show
821c052b-827b-41db-9412-cfa33407d615
    Bridge br-tun
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
    Bridge br-ex
        Port br-ex
            Interface br-ex
                type: internal
        Port "eth3"
            Interface "eth3"
    Bridge br-int
        fail_mode: secure
        Port int-br-ex
            Interface int-br-ex
        Port "qr-b1dc84d3-a8"
            tag: 4095
            Interface "qr-b1dc84d3-a8"
                type: internal
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port "tap4137ae1e-fb"
            tag: 2
            Interface "tap4137ae1e-fb"
                type: internal
        Port "tap41d2aa33-8f"
            tag: 1
            Interface "tap41d2aa33-8f"
                type: internal
        Port br-int
            Interface br-int
                type: internal
    ovs_version: "2.1.3"

According to suggestions which are given do changes some issues gone but some the issue remains the same. Please suggest the modifications to be done for successful deployment.

edit flag offensive delete link more

Comments

Router interface should be UP. Try create new private sub-net , new router, and attach interface to private sub-net

dbaxps gravatar imagedbaxps ( 2015-08-12 05:39:08 -0600 )edit

"After adding bridge to the network node, internet connectivity is working fine but, following issues comes". Can you describe in more details what you did ( just as update to question) ?

dbaxps gravatar imagedbaxps ( 2015-08-12 05:40:22 -0600 )edit

We did same as you and mohit suggested. We removed previous network on network node and created external network 192.168.12.0/16 with gateway having IP 192.168.0.3. Bridge and floating IP are also connected on Network Node with IP on 192.168.12.0/16 . Refer the current answer for more info.

smcas gravatar imagesmcas ( 2015-08-12 06:03:33 -0600 )edit

OVS bridge br-ex && OVS port are configured wrong.
See samples here :- http://bderzhavets.blogspot.com/2015/...

dbaxps gravatar imagedbaxps ( 2015-08-12 06:23:05 -0600 )edit

I added samples to my answer.

dbaxps gravatar imagedbaxps ( 2015-08-12 06:26:02 -0600 )edit
0

answered 2015-08-09 21:14:19 -0600

Mohit gravatar image

updated 2015-08-09 21:20:49 -0600

Hello,

I am writing this assuming the network diagram prepared in MS paint with black fonts

On network node Set the bridging correctly as

Assuming eth1 interface is having internet access and br-ex brige is set up ,

ovs-vsctl add-port br-ex eth1

This command would disconnect the access of network node, Login to console and set

ifcfg eth1 0

Also ensure that br-ex is having the public ip address or the inter access network , If not , se it via CLI

ifconfig br-ex 172.24.4.224 netmask 255.x.x.x up

This would bring back the connectivity,

In your case ifconfig should return the

ifconfig br-ex = ip address of internet network (172.24.4.224) eth1 = This is up but no ip address is displayed

Check the ip routing table in network node ,assuming internet access is on 172.24.4.224 network

route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 172.24.4.1 0.0.0.0 UG 100 0 0 br-ex 10.40.96.0 0.0.0.0 255.255.255.0 U 0 0 0 br-ex 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0

ip route show default via 172.24.4.1 dev br-ex metric 100 172.24.4.0/24 dev br-ex proto kernel scope link src 172.24.4.224 . .

In above results i assume 172.24.4.1 is you default gateway of your internet network Ensure that the default path is set correctly , if not set it and check if the network node is able to resolve DNS ping http://google.com

After this you are done on CLI , Now login to dashboard and set the networking topology correctly

Create a public network.

Starting with the Openstack Dashboard, logged in as admin, create a public network is 172.24.4.0/28. The IP address 172.24.4.1 is the address of your hardware router, gateway to the internet. Network host is 172.24.4.224 . And Openstack needs a subnet, referred to as public, that sits in this address range.

Openstack Dashboard Menu: Admin->Networks->Create Network Name - Public Project - admin Admin state = yes External network = yes

Create network

Openstack Dashboard Menu: Admin->Networks->public->Create Subnet Subnet Name - public_subnet Network address - 172.24.4.0/28 IP Version - IPV4 Gateway IP - 172.24.4.224

Openstack Dashboard Menu: Admin->Networks->public->Create Subnet 2 Allocation Pools - Set the range of ip's free on 172.24.4.0/28 DNS servers - Set the DNS ip

openstack Dashboard Menu: Project->Network->Network Topology->Create Network — “private”

Network Name - Private Admin state = yes

Openstack Dashboard Menu: Project->Network->Network Topology->Create Network->Subnet Create subnet = yes Subnet name = Private subnet Network address - 192.168.0.0/24 (Please set this as per your enviroment IP Version - v4 Gateway IP = 192.168.0.1

Openstack Dashboard Menu: Project->Network->Network Topology- ... (more)

edit flag offensive delete link more

Comments

Thanks Mohit.Now we are able to ping the 8.8.8.8 from network node but as you suggested to check that is it able to resolve DNS, we unable to ping google.com from CLI. we are not heading forward and got stuck here, plz suggest how to resolve it..?

smcas gravatar imagesmcas ( 2015-08-12 02:11:46 -0600 )edit

we were trying to resolve the issue and now it has been resolved by following action: add DNS server name in the /etc/resolv.conf file

smcas gravatar imagesmcas ( 2015-08-12 02:37:59 -0600 )edit

Hi Mohit, thanks for your help. I did the same but the router interfaces shows down.Do we need to make the bridge ovs and the port to which bridge is attached ovs port? We tried making it ovs due to which internet stops working on network node as well as instances are also not launching.Please help

smcas gravatar imagesmcas ( 2015-08-17 00:45:09 -0600 )edit

set eth3 to 0

ifconfig eth3 0

Ensure that the br-ex is having the access to internet. This would bring back the internet , If not set it via CLI

ifconfig br-ex x.x.x.x netmask 255.x.x.x up

This would bring back the connectivity. Also Check the ip routing table in network node.

Mohit gravatar imageMohit ( 2015-08-17 01:36:20 -0600 )edit

What abt eth1

Mohit gravatar imageMohit ( 2015-08-17 01:36:47 -0600 )edit
0

answered 2015-08-07 18:36:33 -0600

Loss of network generally means br-ex does not have an IP assigned on public. You may have to let Neutron know about your bridge. Have a look at the external network setup at the RDO project (info is not really specific to RDO) https://www.rdoproject.org/Neutron_wi...

edit flag offensive delete link more
0

answered 2015-08-08 05:52:02 -0600

Ams70 gravatar image

Hi salman sheikh, I have exactly your problems, it is very important for me to know the answers of this problems. If you find the reason of this problems, please share it. thanks

edit flag offensive delete link more
0

answered 2015-08-10 00:39:36 -0600

"Instances are getting IP from Floating IP through DHCP. Though, IP is displayed on the dashboard but when we execute 'ifconfig' from VM terminal, it does not show any IP."

Whenever we assign a network(mostly internal n/w) to the VM it will display from the ifconfig as well as from the Dashboard also. But when we assign a Floating IP it will display only in Dashboard and not using ifconfig.

salman, can you please post a snapshot of the network diagram taken from the Dashboard.

edit flag offensive delete link more

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2015-08-07 02:25:58 -0600

Seen: 9,942 times

Last updated: Sep 09 '15