Revision history [back]

Cannot reach floating IP from outside network

Host OS

  • Distributor ID: Debian
  • Description: Debian GNU/Linux 8.6
  • (jessie) Release: 8.6
  • Codename: jessie

Openstack Environement

  • RDO All-In-One installed on VMware workstation VM + Centos7

-During installation, Openstack is configured to use the external interface (192.168.0.[1-155]/24) (connecting VM to host) as "public" network..
-Configured a subnet range for floating IPs (not assigned by external network:192.168.0.[170-199]/24).
-Created an instance, assigned a floating IP (192.168.0.176).

selection_888

Results:
==> Can ping/ssh to instance from within Openstack, but not from outside.

Observations:
I can observe the floatng IP (192.168.0.176) assigned to router external interface

[root@RDO-AIO ~(keystone_admin)]# ip netns exec qrouter-a2dd3739-fe62-4e79-8795-e3023419dc30 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
9: qg-913f6089-a8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
    link/ether fa:16:3e:31:a4:f9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.173/24 brd 192.168.0.255 scope global qg-913f6089-a8
       valid_lft forever preferred_lft forever
    inet 192.168.0.176/32 brd 192.168.0.176 scope global qg-913f6089-a8    <<<<<<<<<<<<<<<<<<<<<<<<  
       valid_lft forever preferred_lft forever
    inet6 f816:3eff:fe31:a4f9/64 scope global mngtmpaddr dynamic 
       valid_lft 86318sec preferred_lft 86318sec
    inet6 fe80::f816:3eff:fe31:a4f9/64 scope link 
       valid_lft forever preferred_lft forever

routing and arp on openstack looks OK

[root@RDO-AIO ~(keystone_admin)]# cat /proc/sys/net/ipv4/ip_forward
1

[root@RDO-AIO ~(keystone_admin)]# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         gateway         0.0.0.0         UG    0      0        0 br-ex
link-local      0.0.0.0         255.255.0.0     U     1002   0        0 eno16777736
link-local      0.0.0.0         255.255.0.0     U     1006   0        0 br-ex
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 br-ex   <<<<<<<<<<<<<<
[root@RDO-AIO ~(keystone_admin)]# 
[root@RDO-AIO ~(keystone_admin)]# arp
Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.0.146            ether   b4:b5:2f:b1:fa:ec   C                     br-ex
192.168.0.2              ether   a2:c6:c7:14:c5:49   C                     br-ex
192.168.0.173            ether   fa:16:3e:31:a4:f9   C                     br-ex
192.168.0.176            ether   fa:16:3e:31:a4:f9   C                     br-ex   <<<<<<<<<<<<<  
gateway                  ether   f4:ca:e5:4c:ed:44   C                     br-ex

From Openstack host, I can see the ping coming but Openstack not forwarding it to the router namespace:

[root@RDO-AIO ~(keystone_admin)]# tcpdump icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br-ex, link-type EN10MB (Ethernet), capture size 65535 bytes
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 1, length 64
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 2, length 64
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 3, length 64
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 4, length 64
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 5, length 64
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 6, length 64

From host1, arp resolution for the VM instance inside Openstack is correct:

/ # arp
Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.0.108            ether   00:0c:29:41:b7:6c   C                     eth0
192.168.0.176            ether   fa:16:3e:31:a4:f9   C                     eth0  <<<<<<<<<<<<<<<

From Openstack host

[root@RDO-AIO ~(keystone_admin)]# arp
Address                  HWtype  HWaddress           Flags Mask            Iface
gateway                  ether   f4:ca:e5:4c:ed:44   C                     br-ex
192.168.0.2              ether   16:da:a1:da:c3:f7   C                     br-ex
192.168.0.173            ether   fa:16:3e:31:a4:f9   C                     br-ex
192.168.0.146            ether   b4:b5:2f:b1:fa:ec   C                     br-ex
192.168.0.176            ether   fa:16:3e:31:a4:f9   C                     br-ex  <<<<<<<<<<<<<<<

Cannot reach floating IP from outside network

Host OS

  • Distributor ID: Debian
  • Description: Debian GNU/Linux 8.6
  • (jessie) Release: 8.6
  • Codename: jessie

Openstack Environement

  • RDO All-In-One installed on VMware workstation VM + Centos7

-During installation, Openstack is configured to use the external interface (192.168.0.[1-155]/24) (connecting VM to host) as "public" network..
-Configured a subnet range for floating IPs (not assigned by external network:192.168.0.[170-199]/24).
-Created an instance, assigned a floating IP (192.168.0.176).

selection_888

Results:
==> Can ping/ssh to instance from within Openstack, but not from outside.

Observations:
I can observe the floatng IP (192.168.0.176) assigned to router external interface

[root@RDO-AIO ~(keystone_admin)]# ip netns exec qrouter-a2dd3739-fe62-4e79-8795-e3023419dc30 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
9: qg-913f6089-a8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
    link/ether fa:16:3e:31:a4:f9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.173/24 brd 192.168.0.255 scope global qg-913f6089-a8
       valid_lft forever preferred_lft forever
    inet 192.168.0.176/32 brd 192.168.0.176 scope global qg-913f6089-a8    <<<<<<<<<<<<<<<<<<<<<<<<  
       valid_lft forever preferred_lft forever
    inet6 f816:3eff:fe31:a4f9/64 scope global mngtmpaddr dynamic 
       valid_lft 86318sec preferred_lft 86318sec
    inet6 fe80::f816:3eff:fe31:a4f9/64 scope link 
       valid_lft forever preferred_lft forever

routing and arp on openstack looks OK

[root@RDO-AIO ~(keystone_admin)]# cat /proc/sys/net/ipv4/ip_forward
1

[root@RDO-AIO ~(keystone_admin)]# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         gateway         0.0.0.0         UG    0      0        0 br-ex
link-local      0.0.0.0         255.255.0.0     U     1002   0        0 eno16777736
link-local      0.0.0.0         255.255.0.0     U     1006   0        0 br-ex
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 br-ex   <<<<<<<<<<<<<<
[root@RDO-AIO ~(keystone_admin)]# 
[root@RDO-AIO ~(keystone_admin)]# arp
Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.0.146            ether   b4:b5:2f:b1:fa:ec   C                     br-ex
192.168.0.2              ether   a2:c6:c7:14:c5:49   C                     br-ex
192.168.0.173            ether   fa:16:3e:31:a4:f9   C                     br-ex
192.168.0.176            ether   fa:16:3e:31:a4:f9   C                     br-ex   <<<<<<<<<<<<<  
gateway                  ether   f4:ca:e5:4c:ed:44   C                     br-ex

From Openstack host, I can see the ping coming but Openstack not forwarding it to the router namespace:

[root@RDO-AIO ~(keystone_admin)]# tcpdump icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br-ex, link-type EN10MB (Ethernet), capture size 65535 bytes
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 1, length 64
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 2, length 64
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 3, length 64
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 4, length 64
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 5, length 64
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 6, length 64

From host1, arp resolution for the VM instance inside Openstack is correct:

/ # arp
Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.0.108            ether   00:0c:29:41:b7:6c   C                     eth0
192.168.0.176            ether   fa:16:3e:31:a4:f9   C                     eth0  <<<<<<<<<<<<<<<

From Openstack host

[root@RDO-AIO ~(keystone_admin)]# arp
Address                  HWtype  HWaddress           Flags Mask            Iface
gateway                  ether   f4:ca:e5:4c:ed:44   C                     br-ex
192.168.0.2              ether   16:da:a1:da:c3:f7   C                     br-ex
192.168.0.173            ether   fa:16:3e:31:a4:f9   C                     br-ex
192.168.0.146            ether   b4:b5:2f:b1:fa:ec   C                     br-ex
192.168.0.176            ether   fa:16:3e:31:a4:f9   C                     br-ex  <<<<<<<<<<<<<<<

In my understanding, Openstack should be able to forward traffic coming from outside to the instance knowing that:

  • routing is enabled
  • floating IP belongs to outside network (different pool)
  • outside hosts successfully resolve arp
  • instance floating IP is ping'able/ssh'able from inside

Did I miss something?

Cannot reach floating IP from outside network

Host OS

  • Distributor ID: Debian
  • Description: Debian GNU/Linux 8.6
  • (jessie) Release: 8.6
  • Codename: jessie

Openstack Environement

  • RDO All-In-One installed on VMware workstation VM + Centos7

-During installation, Openstack is configured to use the external interface (192.168.0.[1-155]/24) (connecting VM to host) as "public" network..
-Configured a subnet range for floating IPs (not assigned by external network:192.168.0.[170-199]/24).
-Created an instance, assigned a floating IP (192.168.0.176).

selection_888

Results:
==> Can ping/ssh to instance from within Openstack, but not from outside.

Observations:
I can observe the floatng IP (192.168.0.176) assigned to router external interface

[root@RDO-AIO ~(keystone_admin)]# ip netns exec qrouter-a2dd3739-fe62-4e79-8795-e3023419dc30 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
9: qg-913f6089-a8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
    link/ether fa:16:3e:31:a4:f9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.173/24 brd 192.168.0.255 scope global qg-913f6089-a8
       valid_lft forever preferred_lft forever
    inet 192.168.0.176/32 brd 192.168.0.176 scope global qg-913f6089-a8    <<<<<<<<<<<<<<<<<<<<<<<<  
       valid_lft forever preferred_lft forever
    inet6 f816:3eff:fe31:a4f9/64 scope global mngtmpaddr dynamic 
       valid_lft 86318sec preferred_lft 86318sec
    inet6 fe80::f816:3eff:fe31:a4f9/64 scope link 
       valid_lft forever preferred_lft forever

routing and arp on openstack looks OK

[root@RDO-AIO ~(keystone_admin)]# cat /proc/sys/net/ipv4/ip_forward
1

[root@RDO-AIO ~(keystone_admin)]# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         gateway         0.0.0.0         UG    0      0        0 br-ex
link-local      0.0.0.0         255.255.0.0     U     1002   0        0 eno16777736
link-local      0.0.0.0         255.255.0.0     U     1006   0        0 br-ex
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 br-ex   <<<<<<<<<<<<<<
[root@RDO-AIO ~(keystone_admin)]# 
[root@RDO-AIO ~(keystone_admin)]# arp
Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.0.146            ether   b4:b5:2f:b1:fa:ec   C                     br-ex
192.168.0.2              ether   a2:c6:c7:14:c5:49   C                     br-ex
192.168.0.173            ether   fa:16:3e:31:a4:f9   C                     br-ex
192.168.0.176            ether   fa:16:3e:31:a4:f9   C                     br-ex   <<<<<<<<<<<<<  
gateway                  ether   f4:ca:e5:4c:ed:44   C                     br-ex

From Openstack host, I can see the ping coming but Openstack not forwarding it to the router namespace:

[root@RDO-AIO ~(keystone_admin)]# tcpdump icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br-ex, link-type EN10MB (Ethernet), capture size 65535 bytes
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 1, length 64
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 2, length 64
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 3, length 64
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 4, length 64
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 5, length 64
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 6, length 64

From host1, arp resolution for the VM instance inside Openstack is correct:

/ # arp
Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.0.108            ether   00:0c:29:41:b7:6c   C                     eth0
192.168.0.176            ether   fa:16:3e:31:a4:f9   C                     eth0  <<<<<<<<<<<<<<<

From Openstack host

[root@RDO-AIO ~(keystone_admin)]# arp
Address                  HWtype  HWaddress           Flags Mask            Iface
gateway                  ether   f4:ca:e5:4c:ed:44   C                     br-ex
192.168.0.2              ether   16:da:a1:da:c3:f7   C                     br-ex
192.168.0.173            ether   fa:16:3e:31:a4:f9   C                     br-ex
192.168.0.146            ether   b4:b5:2f:b1:fa:ec   C                     br-ex
192.168.0.176            ether   fa:16:3e:31:a4:f9   C                     br-ex  <<<<<<<<<<<<<<<

In my understanding, Openstack should be able to forward traffic coming from outside to the instance knowing that:

  • Openstack public subnet is mapped to outside network(during installation)
  • routing is enabled
  • floating IP belongs to outside network (different pool)
  • outside hosts successfully resolve arp
  • instance floating IP is ping'able/ssh'able from inside

Did I miss something?

Cannot reach floating IP from outside network

Host OS

  • Distributor ID: Debian
  • Description: Debian GNU/Linux 8.6
  • (jessie) Release: 8.6
  • Codename: jessie

Openstack Environement

  • RDO All-In-One installed on VMware workstation VM + Centos7

-During installation, Openstack is configured to use the external interface (192.168.0.[1-155]/24) (connecting VM to host) as "public" network..
-Configured a subnet range for floating IPs (not assigned by external network:192.168.0.[170-199]/24).
-Created an instance, assigned a floating IP (192.168.0.176).

selection_888

Results:
==> Can ping/ssh to instance from within Openstack, but not from outside.

Observations:
I can observe the floatng IP (192.168.0.176) assigned to router external interface

[root@RDO-AIO ~(keystone_admin)]# ip netns exec qrouter-a2dd3739-fe62-4e79-8795-e3023419dc30 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
9: qg-913f6089-a8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
    link/ether fa:16:3e:31:a4:f9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.173/24 brd 192.168.0.255 scope global qg-913f6089-a8
       valid_lft forever preferred_lft forever
    inet 192.168.0.176/32 brd 192.168.0.176 scope global qg-913f6089-a8    <<<<<<<<<<<<<<<<<<<<<<<<  
       valid_lft forever preferred_lft forever
    inet6 f816:3eff:fe31:a4f9/64 scope global mngtmpaddr dynamic 
       valid_lft 86318sec preferred_lft 86318sec
    inet6 fe80::f816:3eff:fe31:a4f9/64 scope link 
       valid_lft forever preferred_lft forever

routing and arp on openstack looks OK

[root@RDO-AIO ~(keystone_admin)]# cat /proc/sys/net/ipv4/ip_forward
1

[root@RDO-AIO ~(keystone_admin)]# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         gateway         0.0.0.0         UG    0      0        0 br-ex
link-local      0.0.0.0         255.255.0.0     U     1002   0        0 eno16777736
link-local      0.0.0.0         255.255.0.0     U     1006   0        0 br-ex
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 br-ex   <<<<<<<<<<<<<<
[root@RDO-AIO ~(keystone_admin)]# 
[root@RDO-AIO ~(keystone_admin)]# arp
Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.0.146            ether   b4:b5:2f:b1:fa:ec   C                     br-ex
192.168.0.2              ether   a2:c6:c7:14:c5:49   C                     br-ex
192.168.0.173            ether   fa:16:3e:31:a4:f9   C                     br-ex
192.168.0.176            ether   fa:16:3e:31:a4:f9   C                     br-ex   <<<<<<<<<<<<<  
gateway                  ether   f4:ca:e5:4c:ed:44   C                     br-ex

From outside network is pointing (route) to floatng iP's, used by instances, via Openstack host:

/ # ip a 192.168.0.176 via 192.168.0.108 dev ovsbr0

From Openstack host, I can see the ping coming but Openstack not forwarding it to the router namespace:

[root@RDO-AIO ~(keystone_admin)]# tcpdump icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br-ex, link-type EN10MB (Ethernet), capture size 65535 bytes
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 1, length 64
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 2, length 64
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 3, length 64
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 4, length 64
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 5, length 64
IP 192.168.0.146 > 192.168.0.176: ICMP echo request, id 29797, seq 6, length 64

From host1, arp resolution for the VM instance inside Openstack is correct:

/ # arp
Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.0.108            ether   00:0c:29:41:b7:6c   C                     eth0
192.168.0.176            ether   fa:16:3e:31:a4:f9   C                     eth0  <<<<<<<<<<<<<<<

From Openstack host

[root@RDO-AIO ~(keystone_admin)]# arp
Address                  HWtype  HWaddress           Flags Mask            Iface
gateway                  ether   f4:ca:e5:4c:ed:44   C                     br-ex
192.168.0.2              ether   16:da:a1:da:c3:f7   C                     br-ex
192.168.0.173            ether   fa:16:3e:31:a4:f9   C                     br-ex
192.168.0.146            ether   b4:b5:2f:b1:fa:ec   C                     br-ex
192.168.0.176            ether   fa:16:3e:31:a4:f9   C                     br-ex  <<<<<<<<<<<<<<<

In my understanding, Openstack should be able to forward traffic coming from outside to the instance knowing that:

  • Openstack public subnet is mapped to outside network(during installation)
  • routing is enabled
  • floating IP belongs to outside network (different pool)
  • outside hosts successfully resolve arp
  • instance floating IP is ping'able/ssh'able from inside

Did I miss something?