Nova Docker Networking problem

asked 2014-12-29 12:11:55 -0500

Hello,

I've installed OpenStack with Docker as hypervisor on a cubietruck, everything seems to work ok but the container ip does not respond to pings nor respond to the service i'm running inside the container (nginx por 80).

I checked how nova created the container and it looks like everything is in place:

nova list

+--------------------------------------+---------------+--------+------------+-------------+----------------------+
| ID                                          | Name        | Status | Task State | Power State | Networks             |
+--------------------------------------+---------------+--------+------------+-------------+----------------------+
| 249df778-b2b6-490c-9dce-1126f8f337f3 | test_nginx_13 | ACTIVE | -          | Running     | public=192.168.1.135 |
+--------------------------------------+---------------+--------+------------+-------------+----------------------+

# docker ps
CONTAINER ID        IMAGE COMMAND             CREATED             STATUS PORTS                  NAMES
89b59bf9f442        sotolitolabs/nginx_arm:latest "/usr/sbin/nginx"   6 hours ago         Up 6 hours nova-249df778-b2b6-490c-9dce-1126f8f337f3

A funny thing that i noticed but i'm not really sure it's relevant, the docker container does not show network info when created by nova:

 # docker inspect 89b59bf9f442

.... unnecesary output....

"NetworkSettings": {
        "Bridge": "",
        "Gateway": "",
        "IPAddress": "",
        "IPPrefixLen": 0,
        "PortMapping": null,
        "Ports": null
    },

# neutron router-list

+--------------------------------------+---------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+-------+
| id                                   | name    | external_gateway_info | distributed | ha    |
+--------------------------------------+---------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+-------+
| f8dc7e15-1087-4681-b495-217ecfa95189 | router1 | {"network_id": "160add9a-2d2e-45ab-8045-68b334d29418", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "1ae33c0b-a04e-47b6-bdba-bbdf9a3ef14d", "ip_address": "192.168.1.120"}]} | False       | False |
+--------------------------------------+---------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+-------+

# neutron subnet-list

+--------------------------------------+----------------+----------------+----------------------------------------------------+
| id                                   | name           | cidr           | allocation_pools |
+--------------------------------------+----------------+----------------+----------------------------------------------------+
| 34995548-bc2b-4d33-bdb2-27443c01e483 | private_subnet | 10.0.0.0/24    | {"start": "10.0.0.2", "end": "10.0.0.254"} |
| 1ae33c0b-a04e-47b6-bdba-bbdf9a3ef14d | public_subnet  | 192.168.1.0/24 | {"start": "192.168.1.120", "end": "192.168.1.200"} |
+--------------------------------------+----------------+----------------+----------------------------------------------------+

# neutron port-list

+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
| id                                   | name | mac_address       | fixed_ips |
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
| 863eb9a3-461c-4016-9bd1-7c4c7210db98 |      | fa:16:3e:24:7b:2c | {"subnet_id": "34995548-bc2b-4d33-bdb2-27443c01e483", "ip_address": "10.0.0.2"}      |
| bbe59188-ab4e-4b92-a578-bbc2d6759295 |      | fa:16:3e:1c:04:6a | {"subnet_id": "1ae33c0b-a04e-47b6-bdba-bbdf9a3ef14d", "ip_address": "192.168.1.135"} |
| c8b94a90-c7d1-44fc-a582-3370f5486d26 |      | fa:16:3e:6f:69:71 | {"subnet_id": "34995548-bc2b-4d33-bdb2-27443c01e483", "ip_address": "10.0.0.1"}      |
| f108b583-0d54-4388-bcc0-f8d1cbe6efd4 |      | fa:16:3e:bb:3a:1b | {"subnet_id": "1ae33c0b-a04e-47b6-bdba-bbdf9a3ef14d", "ip_address": "192.168.1.120"} |
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+

the network namespace is being created:

 # ip netns exec 89b59bf9f442a0d468d9d4d8c9370c53f8e4a3ba4d8affcd6be8b2dde84fff64 ifconfig

lo: flags=73<up,loopback,running>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

nsbbe59188-ab: flags=4163<up,broadcast,running,multicast>  mtu 1500
        inet 192.168.1.135  netmask 255.255.255.0  broadcast 192.168.1.255
        inet6 fe80::f816:3eff:fe1c:46a  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:1c:04:6a  txqueuelen 1000  (Ethernet)
        RX packets 8  bytes 648 (648.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8  bytes 648 (648.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

when i try a ping it does not return:

# ping -c3 192.168.1.135

PING 192.168.1.135 (192.168.1.135) 56(84) bytes of data.
From 192.168.1.65 icmp_seq=1 Destination Host Unreachable
From 192.168.1.65 icmp_seq=2 Destination Host Unreachable
From 192.168.1.65 icmp_seq=3 Destination Host Unreachable

--- 192.168.1.135 ping statistics ---
3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2008ms ...

(more)

edit retag flag offensive close merge delete

add a comment

2 answers

Sort by » oldest newest most voted

answered 2014-12-30 11:58:22 -0500

ichavero

26 ●1 ●1 ●3 http://ivan.chavero.co...


# docker ps
CONTAINER ID        IMAGE                                COMMAND             CREATED             STATUS              PORTS                  NAMES
9081b29f54bc        sotolitolabs/nginx-test-net:latest   "/usr/sbin/nginx"   10 hours ago        Up 10 hours                                nova-5f3309cf-06df-4969-8810-45bd917b3d4a

# ip netns
9081b29f54bc276a87770f058013b33a86019bda747052f2e2ba350bea1b8573
6240bfee62fcfe2b8546b06bb051ee34a1e2476c1da8a231b0f54895ac2f7884
51a4a54efbb9185b9112b73bd354ee19c02e376df91d674e5cda59cef1da7ca5
89b59bf9f442a0d468d9d4d8c9370c53f8e4a3ba4d8affcd6be8b2dde84fff64
3e0514527a598ab9262a2d5353ee5fe3f0687dc8fbf8a16eb8f05388bcd6a23e
c39d11c069a81b1473c7961852daf691c0588d2e5ad467ff6b65b3eb5483ac3d
96759c5e11b3041c30750b95a8a1d68da2c0e7ea716b926361b948ed58e5d4e6
557c4e73d6cec26a89a569ba22414b54b016d855e88e85338e0640247cf028fa
d04079cd207db77604f8544d03f784da8235a55bd339baef03323e0fcb841231
69f35240de4fc1bf26a981f3db26ccc19903deac152144f2d34088e5c63b68de
990a92c1f5fc9f591285a3bb90fdeac9991aa96fd8fe1bb3f7facbaa165e018f
1e11bffe192e96568553a1e7ad80aacb9f8cc205c48eb358d3cded6be4d71b92
3a833584bb0459d84e25154d10443c87eaf0cc5cf58312cc3f3fd6f2e36236a4
qdhcp-f7f6b1c3-3431-4831-b32c-6a00811d7056


# ovs-vsctl show
2de4ca9a-78cc-426c-84d0-a43537c66d62
    Bridge br-tun
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
    Bridge br-int
        fail_mode: secure
        Port "tapd2d0eea1-6a"
            tag: 1
            Interface "tapd2d0eea1-6a"
        Port br-int
            Interface br-int
                type: internal
        Port "tap6b2917cb-4c"
            tag: 1
            Interface "tap6b2917cb-4c"
        Port "tap7893b669-3e"
            tag: 1
            Interface "tap7893b669-3e"
        Port "tap38e65528-a9"
            tag: 1
            Interface "tap38e65528-a9"
        Port "tapdf6d26d4-c8"
            tag: 1
            Interface "tapdf6d26d4-c8"
        Port "tap451d2e6f-93"
            tag: 1
            Interface "tap451d2e6f-93"
        Port "tapc9b6577a-ac"
            tag: 1
            Interface "tapc9b6577a-ac"
        Port "tape4d2a4f7-36"
            tag: 1
            Interface "tape4d2a4f7-36"
        Port "tapd77a11f0-33"
            tag: 1
            Interface "tapd77a11f0-33"
        Port "tap2a930a1f-c5"
            tag: 1
            Interface "tap2a930a1f-c5"
        Port "tap5116e690-9a"
            tag: 1
            Interface "tap5116e690-9a"
        Port "tapbbe59188-ab"
            tag: 1
            Interface "tapbbe59188-ab"
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port "tapa24789ad-23"
            tag: 1
            Interface "tapa24789ad-23"
        Port "tap863eb9a3-46"
            tag: 4095
            Interface "tap863eb9a3-46"
                type: internal
    Bridge br-ex
        Port "eth0"
            Interface "eth0"
        Port "nsbbe59188-ab"
            Interface "nsbbe59188-ab"
        Port br-ex
            Interface br-ex
                type: internal
    ovs_version: "2.3.0"

Couldn't find the router in the ip netns output

I don't know what i did wrong here. But i guess this is the source of my problems How can i create this router??

This part i do have:


# ip netns exec qdhcp-f7f6b1c3-3431-4831-b32c-6a00811d7056 route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.0.0.1        0.0.0.0         UG    0      0        0 tap863eb9a3-46
10.0.0.0        0.0.0.0         255.255.255.0   U     0      0        0 tap863eb9a3-46

# ip netns exec qdhcp-f7f6b1c3-3431-4831-b32c-6a00811d7056 ifconfig
lo: flags=73<up,loopback,running>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

tap863eb9a3-46: flags=4163<up,broadcast,running,multicast>  mtu 1500
        inet 10.0.0.2  netmask 255.255.255.0  broadcast 10.0.0.255
        inet6 fe80::f816:3eff:fe24:7b2c  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:24:7b:2c  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8  bytes 648 (648.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Thanks!
Ivan

edit flag offensive delete link

Comments

View ovs-vsctl show on my Nova-Docker RDO Juno AIO Node :-
http://textuploader.com/6553

dbaxps ( 2014-12-30 13:20:55 -0500 )edit

View also " Fork Juno branch from https://github.com/stackforge/nova-do... " :-
http://bderzhavets.blogspot.com/2014/...

dbaxps ( 2014-12-30 13:25:41 -0500 )edit

add a comment

answered 2014-12-29 23:24:07 -0500

dbaxps

10778 ●57 ●112 ●224 http://bderzhavets.blo...

updated 2014-12-30 14:17:43 -0500

UPDATE 12/30/2014 21:56 MSK

You have to create neutron router for particular tenant. Add gateway to external network and interface to private network, your Nova-Docker Container is supposed to run on this private network and obtain private IP from dnsmasq serving this subnet, floating IP has to be assigned from neutron external pool. Qrouter-namespace is created when you activate gateway on router to external network.

View :- http://www.linux.com/community/blogs/...
View also :- http://textuploader.com/1hey

Section "Create Neutron networks on Controller node "

Use as comment field due to formatting
First

[root@junodocker ~(keystone_admin)]# neutron net-list
+--------------------------------------+--------------+-----------------------------------------------------+
| id                                   | name         | subnets                                             |
+--------------------------------------+--------------+-----------------------------------------------------+
| 38147773-5746-4ca7-bf4a-6be58c7be942 | demo_network | 7c020d0e-2621-4f45-a7f8-1f9984db596a 70.0.0.0/24    |
| 2b642289-8144-4032-8452-30ea88b37f29 | public       | 569da7f2-57a0-4017-a12c-d0b1a3752782 192.168.1.0/24 |
+--------------------------------------+--------------+------------------------------------------

Second

[root@junodocker ~(keystone_admin)]# ip netns
d7ea3f7b9c5c57046fa3e569f9267d3d334a5089900f5cca544fab77769da387
qrouter-ffaea514-6639-44f2-980f-20ccc99a9f33
qdhcp-38147773-5746-4ca7-bf4a-6be58c7be942

Third

[root@junodocker ~(keystone_admin)]# ip netns exec d7ea3f7b9c5c57046fa3e569f9267d3d334a5089900f5cca544fab77769da387 ifconfig
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

nsd4626b6e-15: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 70.0.0.35  netmask 255.255.255.0  broadcast 70.0.0.255
        inet6 fe80::f816:3eff:fef0:f317  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:f0:f3:17  txqueuelen 1000  (Ethernet)
        RX packets 68  bytes 9896 (9.6 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 68  bytes 5546 (5.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Fourth

[root@junodocker ~(keystone_admin)]# ovs-vsctl list-ports br-ex
enp2s0
qg-744ca793-72

Your ovs-vsctl report doesn't look good && ip netns docker-id shows public IP ( not private)
Please, post

$ ip netns
$ ovs-vsctl show
$ ip netns exec qrouter-your-router-id iptables -S -t net
$ ip netns exec qrouter-your-router-id route -n
$ ip netns exec qrouter-your-router-id  ifconfig
$ ip netns exec qdhcp-private-net-id route -n
$ ip netns exec qdhcp-private-net-id ifconfig

edit flag offensive delete link

Comments

i created a new router assigned to the admin tenant but it didn't create the qroute netns when i added the external gateway

ichavero ( 2014-12-31 14:28:27 -0500 )edit

Then
4. Creating interface to private net
5.Creating gateway to external net

dbaxps ( 2014-12-31 15:56:30 -0500 )edit

Precisely a year since this post went out, I'm running into the exact same behavior but with the Liberty release. The Install guide no longer directs to use openvswitch, but instead sets up a vxlan across the private and public networks, has anyone on this thread made "breakthrough" findings?

vincent ( 2015-12-30 13:05:36 -0500 )edit

add a comment

Nova Docker Networking problem

nova list

2 answers

Comments

Comments

Your Answer

Get to know Ask OpenStack

Question Tools

Stats

Related questions

Feedback About This Page

OpenStack

Community

Documentation

Branding & Legal

Nova Docker Networking problem edit

nova list

2 answers

Comments

Comments

Your Answer

Get to know Ask OpenStack

Question Tools

Stats

Related questions

Feedback About This Page

OpenStack

Community

Documentation

Branding & Legal

Nova Docker Networking problem