Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Cannot ssh with second floating ip to guest vm.

Installation: OpenStack Kilo Single Node installed with openstack-install on ubuntu-trusty.

My Setup has 4 networks, one external and three internal. All of them connected with single router. I Created a guest VM (test4) with two NICs each with different ip from different subnet/network (int-net-2 and int-net-3) (10.101.3.5 and 10.101.2.5). Assigned two floating ips ( from external network ext-net) to these NICs. (10.0.5.15 and 10.0.5.12)

The issue is when i ssh the vm with floating-ip1 (10.0.5.12) it works, but from floating-ip2 (10.0.5.15) it does not. Here is the configuration

nova list --all-tenants
+--------------------------------------+-------+----------------------------------+--------+------------+-------------+------------------------------------------------------------------+
| ID                                   | Name  | Tenant ID                        | Status | Task State | Power State | Networks                                                         |
+--------------------------------------+-------+----------------------------------+--------+------------+-------------+------------------------------------------------------------------+
| 539548c2-1237-4f5e-b5e9-fb27907fea39 | test1 | b6d48b43e10d4753b433de6c3b875b98 | ACTIVE | -          | Running     | ubuntu-net=10.101.0.3, 10.0.5.9                                  |
| a8852d1e-9cda-412a-ba24-c67ad073bee0 | test2 | b6d48b43e10d4753b433de6c3b875b98 | ACTIVE | -          | Running     | int-net-2=10.101.2.4, 10.0.5.10                                  |
| fdf2d441-df8f-4276-a356-73b45539fad5 | test3 | b6d48b43e10d4753b433de6c3b875b98 | ACTIVE | -          | Running     | int-net-3=10.101.3.4, 10.0.5.11                                  |
| 6576b1d2-5049-453e-9728-401f4310c742 | test4 | b6d48b43e10d4753b433de6c3b875b98 | ACTIVE | -          | Running     | int-net-3=10.101.3.5, 10.0.5.15; int-net-2=10.101.2.5, 10.0.5.12 |
+--------------------------------------+-------+----------------------------------+--------+------------+-------------+------------------------------------------------------------------+

neutron net-list
+--------------------------------------+------------+----------------------------------------------------+
| id                                   | name       | subnets                                            |
+--------------------------------------+------------+----------------------------------------------------+
| 19eb2c8c-c27d-4911-95ea-41d578ae2b9d | ubuntu-net | b699671a-f1ed-44ac-bf72-d879b0d9dc98 10.101.0.0/24 |
| 28564098-7f1a-42be-ab9c-2ca86e11abe6 | int-net-3  | c316f687-2bc5-4c1f-8225-0f8950f6e67c 10.101.3.0/24 |
| 521e57cd-9492-4097-b75b-f825d79dcb46 | int-net-2  | 836a10a7-a563-4c78-9d27-9b6f6f4c57a4 10.101.2.0/24 |
| d5fea3a2-5c8c-42ab-a614-56463ed08b5c | ext-net    | 54d2695a-8b13-4816-8c22-a27511fbeff3 10.0.5.0/24   |
+--------------------------------------+------------+----------------------------------------------------+

neutron subnet-list 
+--------------------------------------+---------------+---------------+------------------------------------------------+
| id                                   | name          | cidr          | allocation_pools                               |
+--------------------------------------+---------------+---------------+------------------------------------------------+
| 54d2695a-8b13-4816-8c22-a27511fbeff3 |               | 10.0.5.0/24   | {"start": "10.0.5.2", "end": "10.0.5.254"}     |
| 836a10a7-a563-4c78-9d27-9b6f6f4c57a4 | int-subnet-2  | 10.101.2.0/24 | {"start": "10.101.2.2", "end": "10.101.2.254"} |
| b699671a-f1ed-44ac-bf72-d879b0d9dc98 | ubuntu-subnet | 10.101.0.0/24 | {"start": "10.101.0.2", "end": "10.101.0.254"} |
| c316f687-2bc5-4c1f-8225-0f8950f6e67c | int-subnet-3  | 10.101.3.0/24 | {"start": "10.101.3.2", "end": "10.101.3.254"} |
+

--------------------------------------+---------------+---------------+------------------------------------------------+

 neutron router-list
+--------------------------------------+---------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+-------+
| id                                   | name          | external_gateway_info                                                                                                                                                                | distributed | ha    |
+--------------------------------------+---------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+-------+
| 6367b9e2-11b4-427d-9d01-e9d9e82cf5ab | ubuntu-router | {"network_id": "d5fea3a2-5c8c-42ab-a614-56463ed08b5c", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "54d2695a-8b13-4816-8c22-a27511fbeff3", "ip_address": "10.0.5.3"}]} | False       | False |
+--------------------------------------+---------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+-------+

On Neutron

sudo ip netns list
qdhcp-28564098-7f1a-42be-ab9c-2ca86e11abe6
qdhcp-521e57cd-9492-4097-b75b-f825d79dcb46
qrouter-6367b9e2-11b4-427d-9d01-e9d9e82cf5ab
qdhcp-19eb2c8c-c27d-4911-95ea-41d578ae2b9d
qdhcp-d5fea3a2-5c8c-42ab-a614-56463ed08b5c

Working Case

$ sudo ip netns exec qrouter-6367b9e2-11b4-427d-9d01-e9d9e82cf5ab ping -c 5 10.0.5.12
PING 10.0.5.12 (10.0.5.12) 56(84) bytes of data.
64 bytes from 10.0.5.12: icmp_seq=1 ttl=64 time=2.19 ms
--- 10.0.5.12 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms

Not Working

$ sudo ip netns exec qrouter-6367b9e2-11b4-427d-9d01-e9d9e82cf5ab ping -c 5 10.0.5.15
PING 10.0.5.15 (10.0.5.15) 56(84) bytes of data.
--- 10.0.5.15 ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 3999ms

However i am able to ping both private addresses

$ sudo ip netns exec qrouter-6367b9e2-11b4-427d-9d01-e9d9e82cf5ab ping -c 5 10.101.3.5
PING 10.101.3.5 (10.101.3.5) 56(84) bytes of data.
64 bytes from 10.101.3.5: icmp_seq=1 ttl=64 time=1.84 ms
64 bytes from 10.101.3.5: icmp_seq=2 ttl=64 time=1.10 ms
^C
--- 10.101.3.5 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 1.104/1.474/1.844/0.370 ms

$ sudo ip netns exec qrouter-6367b9e2-11b4-427d-9d01-e9d9e82cf5ab ping -c 5 10.101.2.5
PING 10.101.2.5 (10.101.2.5) 56(84) bytes of data.
64 bytes from 10.101.2.5: icmp_seq=1 ttl=64 time=1.50 ms
64 bytes from 10.101.2.5: icmp_seq=2 ttl=64 time=1.10 ms
^C
--- 10.101.2.5 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 1.101/1.301/1.502/0.203 ms


$ sudo ip netns exec qrouter-6367b9e2-11b4-427d-9d01-e9d9e82cf5ab iptables -S -t nat
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-N neutron-postrouting-bottom
-N neutron-vpn-agen-OUTPUT
-N neutron-vpn-agen-POSTROUTING
-N neutron-vpn-agen-PREROUTING
-N neutron-vpn-agen-float-snat
-N neutron-vpn-agen-snat
-A PREROUTING -j neutron-vpn-agen-PREROUTING
-A OUTPUT -j neutron-vpn-agen-OUTPUT
-A POSTROUTING -j neutron-vpn-agen-POSTROUTING
-A POSTROUTING -j neutron-postrouting-bottom
-A neutron-postrouting-bottom -m comment --comment "Perform source NAT on outgoing traffic." -j neutron-vpn-agen-snat
-A neutron-vpn-agen-OUTPUT -d 10.0.5.15/32 -j DNAT --to-destination 10.101.3.5
-A neutron-vpn-agen-OUTPUT -d 10.0.5.10/32 -j DNAT --to-destination 10.101.2.4
-A neutron-vpn-agen-OUTPUT -d 10.0.5.12/32 -j DNAT --to-destination 10.101.2.5
-A neutron-vpn-agen-OUTPUT -d 10.0.5.11/32 -j DNAT --to-destination 10.101.3.4
-A neutron-vpn-agen-OUTPUT -d 10.0.5.9/32 -j DNAT --to-destination 10.101.0.3
-A neutron-vpn-agen-POSTROUTING ! -i qg-6dbc8102-a9 ! -o qg-6dbc8102-a9 -m conntrack ! --ctstate DNAT -j ACCEPT
-A neutron-vpn-agen-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 9697
-A neutron-vpn-agen-PREROUTING -d 10.0.5.15/32 -j DNAT --to-destination 10.101.3.5
-A neutron-vpn-agen-PREROUTING -d 10.0.5.10/32 -j DNAT --to-destination 10.101.2.4
-A neutron-vpn-agen-PREROUTING -d 10.0.5.12/32 -j DNAT --to-destination 10.101.2.5
-A neutron-vpn-agen-PREROUTING -d 10.0.5.11/32 -j DNAT --to-destination 10.101.3.4
-A neutron-vpn-agen-PREROUTING -d 10.0.5.9/32 -j DNAT --to-destination 10.101.0.3
-A neutron-vpn-agen-float-snat -s 10.101.3.5/32 -j SNAT --to-source 10.0.5.15
-A neutron-vpn-agen-float-snat -s 10.101.2.4/32 -j SNAT --to-source 10.0.5.10
-A neutron-vpn-agen-float-snat -s 10.101.2.5/32 -j SNAT --to-source 10.0.5.12
-A neutron-vpn-agen-float-snat -s 10.101.3.4/32 -j SNAT --to-source 10.0.5.11
-A neutron-vpn-agen-float-snat -s 10.101.0.3/32 -j SNAT --to-source 10.0.5.9
-A neutron-vpn-agen-snat -j neutron-vpn-agen-float-snat
-A neutron-vpn-agen-snat -o qg-6dbc8102-a9 -j SNAT --to-source 10.0.5.3
-A neutron-vpn-agen-snat -m mark ! --mark 0x2 -m conntrack --ctstate DNAT -j SNAT --to-source 10.0.5.3