Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Can't reuse floating IP in DVR - Kilo (neutron)

Hi @ all.

I have a problem with DVR if I reuse my floating IP after I disassociate it from the instance. After reusing the same floatin ip on the same insance, I can't ping my external gateway anymore for 15 minutes. After this, it works from it's own.

And the really strange thing is, that my instances and my routers always use the next ip, and not the ip again, that is free again. Example:

create router:
virtual router ip : 192.168.102.10
delete router
create new router:
virtual router ip : 192.168.102.11

The same with the instances.
create instance:
ip : 192.168.0.20
delete instance
create new instance:
ip : 192.168.0.21

First. DVR on the Compute node, without a associated floatin ip. Ping to external Network works fine.

192.168.0.0/24 --> internal IP 192.168.102.0/23 --> external IP

[root@ostacktbl14 ~]# ip netns exec qrouter-7aa5ca7f-1c1e-43ca-9ec6-6478d739efc4 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
48: qr-41e6384c-1d: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
    link/ether fa:16:3e:01:ce:06 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.1/24 brd 192.168.0.255 scope global qr-41e6384c-1d
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe01:ce06/64 scope link
       valid_lft forever preferred_lft forever

[root@ostacktbl14 ~]# ip netns exec qrouter-7aa5ca7f-1c1e-43ca-9ec6-6478d739efc4 route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 qr-41e6384c-1d

[root@ostacktbl14 ~]# ip netns exec qrouter-7aa5ca7f-1c1e-43ca-9ec6-6478d739efc4 ssh cirros@192.168.0.6
Warning: Permanently added '192.168.0.6' (RSA) to the list of known hosts.
cirros@192.168.0.6's password:

$ ping 192.168.102.10
PING 192.168.102.10 (192.168.102.10): 56 data bytes
64 bytes from 192.168.102.10: seq=0 ttl=63 time=3.135 ms
64 bytes from 192.168.102.10: seq=1 ttl=63 time=1.284 ms

With a new associated floatin ip: Floating IP = 192.168.102.253

[root@ostacktbl14 ~]# ip netns
fip-615aa46d-b9eb-45d0-b323-5e8cf3afbc39
qrouter-7aa5ca7f-1c1e-43ca-9ec6-6478d739efc4

[root@ostacktbl14 ~]# ip netns exec fip-615aa46d-b9eb-45d0-b323-5e8cf3afbc39 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: fpr-7aa5ca7f-1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 3a:cc:90:83:f8:83 brd ff:ff:ff:ff:ff:ff
    inet 169.254.31.29/31 scope global fpr-7aa5ca7f-1
       valid_lft forever preferred_lft forever
    inet6 fe80::38cc:90ff:fe83:f883/64 scope link
       valid_lft forever preferred_lft forever
49: fg-a4f6c7c3-96: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
    link/ether fa:16:3e:d3:ae:a9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.102.254/23 brd 192.168.103.255 scope global fg-a4f6c7c3-96
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fed3:aea9/64 scope link
       valid_lft forever preferred_lft forever

[root@ostacktbl14 ~]# ip netns exec fip-615aa46d-b9eb-45d0-b323-5e8cf3afbc39 route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         192.168.102.10    0.0.0.0         UG    0      0        0 fg-a4f6c7c3-96
192.168.102.0     0.0.0.0         255.255.254.0   U     0      0        0 fg-a4f6c7c3-96
192.168.102.253   169.254.31.28   255.255.255.255 UGH   0      0        0 fpr-7aa5ca7f-1
169.254.31.28   0.0.0.0         255.255.255.254 U     0      0        0 fpr-7aa5ca7f-1

[root@ostacktbl14 ~]# ssh cirros@192.168.102.253
cirros@192.168.102.253's password:

$ ping 192.168.102.10
PING 192.168.102.10 (192.168.102.10): 56 data bytes
64 bytes from 192.168.102.10: seq=0 ttl=62 time=684.050 ms
64 bytes from 192.168.102.10: seq=1 ttl=62 time=1.105 ms
64 bytes from 192.168.102.10: seq=2 ttl=62 time=0.993 ms

Now I Disassociate the IP from my instance: The fip router is not there any more.

[root@ostacktbl14 ~]# ip netns
qrouter-7aa5ca7f-1c1e-43ca-9ec6-6478d739efc4

[root@ostacktbl14 ~]# ip netns exec qrouter-7aa5ca7f-1c1e-43ca-9ec6-6478d739efc4 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
48: qr-41e6384c-1d: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
    link/ether fa:16:3e:01:ce:06 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.1/24 brd 192.168.0.255 scope global qr-41e6384c-1d
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe01:ce06/64 scope link
       valid_lft forever preferred_lft forever

[root@ostacktbl14 ~]# ip netns exec qrouter-7aa5ca7f-1c1e-43ca-9ec6-6478d739efc4 ssh cirros@192.168.0.6
cirros@192.168.0.6's password:

$ ping 192.168.102.10
PING 192.168.102.10 (192.168.102.10): 56 data bytes
64 bytes from 192.168.102.10: seq=0 ttl=63 time=1.945 ms
64 bytes from 192.168.102.10: seq=1 ttl=63 time=1.410 ms

So far so good. I can ping my Gateway and everythink works fine.

And now. After I Assocate my floating IP again: -->>

[root@ostacktbl14 ~]# ip netns
fip-615aa46d-b9eb-45d0-b323-5e8cf3afbc39
qrouter-7aa5ca7f-1c1e-43ca-9ec6-6478d739efc4

[root@ostacktbl14 ~]# ip netns exec fip-615aa46d-b9eb-45d0-b323-5e8cf3afbc39 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: fpr-7aa5ca7f-1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 92:0a:a8:67:85:7a brd ff:ff:ff:ff:ff:ff
    inet 169.254.31.29/31 scope global fpr-7aa5ca7f-1
       valid_lft forever preferred_lft forever
    inet6 fe80::900a:a8ff:fe67:857a/64 scope link
       valid_lft forever preferred_lft forever
50: fg-7005be18-4f: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
    link/ether fa:16:3e:c0:db:67 brd ff:ff:ff:ff:ff:ff
    inet 192.168.102.255/23 brd 192.168.103.255 scope global fg-7005be18-4f
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fec0:db67/64 scope link
       valid_lft forever preferred_lft forever

[root@ostacktbl14 ~]# ip netns exec fip-615aa46d-b9eb-45d0-b323-5e8cf3afbc39 route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         192.168.102.10    0.0.0.0         UG    0      0        0 fg-7005be18-4f
192.168.102.0     0.0.0.0         255.255.254.0   U     0      0        0 fg-7005be18-4f
192.168.102.253   169.254.31.28   255.255.255.255 UGH   0      0        0 fpr-7aa5ca7f-1
169.254.31.28   0.0.0.0         255.255.255.254 U     0      0        0 fpr-7aa5ca7f-1

[root@ostacktbl14 ~]# ssh cirros@192.168.102.253
cirros@192.168.102.253's password:
$ ping 192.168.102.10
PING 192.168.102.10 (192.168.102.10): 56 data bytes
^C

I can see two strange thinks. The first one is the external IP of my fip namespace. Why I always get the next IP of the external network here and not the first available ip?

The second one is, that i can ping the external ip of my instance from my host, but not the gateway from my instance.

Some TCP Dumps:

[root@ostacktbl14 ~]#
[root@ostacktbl14 ~]# tcpdump -nni br-ex | grep 192.168.102.10
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br-ex, link-type EN10MB (Ethernet), capture size 65535 bytes
08:59:49.274858 IP 192.168.102.10 > 192.168.102.253: ICMP echo reply, id 26113, seq 17, length 64
08:59:50.274993 IP 192.168.102.10 > 192.168.102.253: ICMP echo reply, id 26113, seq 18, length 64
^C185 packets captured
186 packets received by filter
0 packets dropped by kernel

[root@ostacktbl14 ~]# ip netns exec fip-615aa46d-b9eb-45d0-b323-5e8cf3afbc39 tcpdump -nni fg-7005be18-4f | grep 192.168.102.10
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on fg-7005be18-4f, link-type EN10MB (Ethernet), capture size 65535 bytes
^C17 packets captured
17 packets received by filter
0 packets dropped by kernel

[root@ostacktbl14 ~]# tcpdump -nni br-ex | grep 192.168.102.10
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br-ex, link-type EN10MB (Ethernet), capture size 65535 bytes
09:00:07.277752 IP 192.168.102.10 > 192.168.102.253: ICMP echo reply, id 26113, seq 35, length 64
09:00:07.360536 ARP, Request who-has 192.168.103.5 tell 192.168.102.10, length 46
09:00:08.278037 IP 192.168.102.10 > 192.168.102.253: ICMP echo reply, id 26113, seq 36, length 64
09:00:09.278028 IP 192.168.102.10 > 192.168.102.253: ICMP echo reply, id 26113, seq 37, length 64
^C141 packets captured
142 packets received by filter
0 packets dropped by kernel

[root@ostacktbl14 ~]#
[root@ostacktbl14 ~]# tcpdump -nni br-ex | grep 192.168.102.10
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br-ex, link-type EN10MB (Ethernet), capture size 65535 bytes
09:00:13.278788 IP 192.168.102.10 > 192.168.102.253: ICMP echo reply, id 26113, seq 41, length 64
09:00:14.278991 IP 192.168.102.10 > 192.168.102.253: ICMP echo reply, id 26113, seq 42, length 64
^C98 packets captured
99 packets received by filter
0 packets dropped by kernel

[root@ostacktbl14 ~]# ip netns exec fip-615aa46d-b9eb-45d0-b323-5e8cf3afbc39 tcpdump -nni fg-7005be18-4f
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on fg-7005be18-4f, link-type EN10MB (Ethernet), capture size 65535 bytes
09:00:21.279337 IP 192.168.102.253 > 192.168.102.10: ICMP echo request, id 26113, seq 49, length 64
09:00:21.280000 IP 192.168.102.10 > 192.168.102.253: ICMP echo reply, id 26113, seq 49, length 64
09:00:22.279559 IP 192.168.102.253 > 192.168.102.10: ICMP echo request, id 26113, seq 50, length 64
09:00:22.280196 IP 192.168.102.10 > 192.168.102.253: ICMP echo reply, id 26113, seq 50, length 64
09:00:22.350164 IP 192.168.102.228.47556 > 192.168.102.253.22: Flags [.], ack 3297952770, win 159, options [nop,nop,TS val 144800816 ecr 309364], length 0
09:00:22.350665 IP 192.168.102.253.22 > 192.168.102.228.47556: Flags [.], ack 1, win 3707, options [nop,nop,TS val 310616 ecr 144750737], length 0
09:00:22.669930 IP 192.168.102.228.47547 > 192.168.102.253.22: Flags [.], ack 2339158537, win 141, options [nop,nop,TS val 144801136 ecr 309443], length 0
09:00:22.670268 IP 192.168.102.253.22 > 192.168.102.228.47547: Flags [.], ack 1, win 4044, options [nop,nop,TS val 310695 ecr 144225257], length 0