Ask Your Question
0

network is unreachable in instance [closed]

asked 2015-08-20 08:44:48 -0500

twskay gravatar image

updated 2015-08-23 09:03:17 -0500

I create a instance. it looks fine.

| ID                                   |  Name  | Status   | Task State | Power State | Networks          +--------------------------------------+--------+----------+------------+-------------+----------+  
| 6639e727-d888-4999-93f0-8e70202674ab | demo2 | ACTIVE    | -          | Running     | demoNet=10.0.98.4   |

I logined instance,and I ping myself ip in instance.(10.0.98.4) it show error message,""network is unreachable"".

I look neutron's server.log but not error message.

after... i found ""network fail"" in intance's console log

Starting network...                 
udhcpc (v1.20.1) started            
Sending discover...              
Sending discover...        
Sending discover...       
Usage: /sbin/cirros-dhcpc <up|down>         
No lease, failing          
WARN: /etc/rc3.d/S40-network failed        
cirros-ds 'net' up at 181.24          
checking http://169.254.169.254/2009-04-04/instance-id         
failed 1/20: up 181.25. request failed        
failed 2/20: up 183.34. request failed           
failed 3/20: up 185.34. request failed        
failed 4/20: up 187.35. request failed
......
failed to read iid from metadata. tried 20
no results found for mode=net. up 221.49. searched: nocloud configdrive ec2
failed to get instance-id of datasource

I don't what happened. Could anybody help me,please?

dnsmasq.log

Aug 22 10:30:23 dnsmasq[19579]: LOUD WARNING: use --bind-dynamic rather than --bind-interfaces to avoid DNS    amplification attacks via these interface(s)
Aug 22 10:30:23 dnsmasq[19579]: warning: no upstream servers configured
Aug 22 10:30:23 dnsmasq-dhcp[19579]: DHCP, static leases only on x.x.x.x, lease time 2m
Aug 22 10:30:23 dnsmasq-dhcp[19579]: DHCP, sockets bound exclusively to interface tapc9c752ba-f0
Aug 22 10:30:23 dnsmasq[19579]: read /var/lib/neutron/dhcp/8eff37ed-df00-4efa-86ba-7a56d69bdbad/addn_hosts - 3 addresses
Aug 22 10:30:23 dnsmasq-dhcp[19579]: read /var/lib/neutron/dhcp/8eff37ed-df00-4efa-86ba-7a56d69bdbad/host
Aug 22 10:30:23 dnsmasq-dhcp[19579]: read /var/lib/neutron/dhcp/8eff37ed-df00-4efa-86ba-7a56d69bdbad/opts
Aug 22 10:31:38 dnsmasq-dhcp[19576]: 0 available DHCP subnet: 10.0.98.0/255.255.255.0
Aug 22 10:31:38 dnsmasq-dhcp[19576]: 0 DHCPRELEASE(tap221451c4-ea) 10.0.98.11 fa:16:3e:be:80:c5 unknown lease
Aug 22 10:31:38 dnsmasq[19576]: read /var/lib/neutron/dhcp/f9ab40f0-d20d-4b5a-a513-2449d1e1c321/addn_hosts - 2 addresses
Aug 22 10:31:38 dnsmasq-dhcp[19576]: read /var/lib/neutron/dhcp/f9ab40f0-d20d-4b5a-a513-2449d1e1c321/host
Aug 22 10:31:38 dnsmasq-dhcp[19576]: read /var/lib/neutron/dhcp/f9ab40f0-d20d-4b5a-a513-2449d1e1c321/opts
Aug 22 10:31:58 dnsmasq[19576]: read /var/lib/neutron/dhcp/f9ab40f0-d20d-4b5a-a513-2449d1e1c321/addn_hosts - 3 addresses
Aug 22 10:31:58 dnsmasq-dhcp[19576]: read /var/lib/neutron/dhcp/f9ab40f0-d20d-4b5a-a513-2449d1e1c321/host
Aug 22 10:31:58 dnsmasq-dhcp[19576]: read /var/lib/neutron/dhcp/f9ab40f0-d20d-4b5a-a513-2449d1e1c321/opts

create private network successfully

neutron net-create demoNet
neutron subnet-create demoNet --name demoSubnet --gateway 10.0.98.1 10.0.98.0/24
neutron router-create demoRouter
neutron router-interface-add demoRouter demoSubnet
neutron router-gateway-set demoRouter public

ovs-vsctl show (network node)

Bridge br-int
    fail_mode: secure
    Port patch-tun
        Interface patch-tun
            type: patch
            options: {peer=patch-int}
    Port "qr-d8413e34-99"
        tag: 5
        Interface "qr-d8413e34-99"
            type: internal
    Port br-int
        Interface br-int
            type: internal
Bridge br-ex
    Port br-ex
        Interface br-ex
            type: internal
    Port "qg-94b419b5-9e"
        Interface "qg-94b419b5-9e"
            type: internal
Bridge br-tun
    fail_mode: secure ...
(more)
edit retag flag offensive reopen merge delete

Closed for the following reason the question is answered, right answer was accepted by twskay
close date 2015-08-31 03:45:04.679072

Comments

Run ip netns on Network Node.

dbaxps gravatar imagedbaxps ( 2015-08-20 09:22:34 -0500 )edit

request:

qrouter-5e5fc9f3-d61c-483e-bdf7-e16b9c936591

qdhcp-70f2f011-1b69-403f-b957-231a4f31bb10

twskay gravatar imagetwskay ( 2015-08-20 10:42:04 -0500 )edit

i found "network fail" in instance's console.log

Usage: /sbin/cirros-dhcpc <up|down>

No lease, failing

WARN: /etc/rc3.d/S40-network failed

cirros-ds 'net' up at 181.24

checking http://169.254.169.254/2009-04-04/instance-id (http://169.254.169.254/2009-04-04/ins...)

twskay gravatar imagetwskay ( 2015-08-20 10:44:18 -0500 )edit

Activate dnsmasq.log and check what happens with dhcp request ( instance ) dhcp responce ( dnsmasq )

dbaxps gravatar imagedbaxps ( 2015-08-21 04:33:24 -0500 )edit

hello, I don't find dnsmasq.log in /var/log/neutron/.

but in dhcp_agent.ini have this setting dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq

should i install dnsmasq? or what could i do ?

thank you.

twskay gravatar imagetwskay ( 2015-08-21 10:57:01 -0500 )edit

2 answers

Sort by ยป oldest newest most voted
0

answered 2015-08-21 11:50:01 -0500

dbaxps gravatar image

updated 2015-08-24 12:34:48 -0500

Using field as comment :-
UPDATE 4
SOLVED via troubleshooting VXLAN tunnel && Resetting OVS
UPDATE 3
I guess, that message in neutron-server.log

   2015-08-24 09:35:27.692 16994 WARNING keystonemiddleware.auth_token [-] Authorization failed for token
   2015-08-24 09:35:27.692 16994 INFO keystonemiddleware.auth_token [-] Invalid user token - rejecting request

Causes failure to create a port when nova boot instance
See : - http://sanenthusiast.com/tag/keystone...

Now its very evident that the keystone service is not accepting the credentials. If its an authentication error verify the password that is set for the neutron user account residing in keystone database. The next option is to verify neutron.conf in network node. In my case while re-configuring neutron password for neutron user in /etc/neutron/neutron.conf is incorrect. After changing the password keystone accepts authentication and neutron service is running successfully.

UPDATE 2

    Check neutron-server.log regarding successful connection to rabbitmq
    When you start instance , issue :-
    $ neutron port-list --device-id   instance-id

UPDATE 1
Pease, reproduce on your system

[root@ip-192-169-142-147 ~(keystone_admin)]# ip netns exec qdhcp-238e83f5-864f-44e1-83e6-c9dc33ba752f netstat -antp
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 50.0.0.10:53            0.0.0.0:*               LISTEN      4052/dnsmasq        
tcp6       0      0 fe80::f816:3eff:fe19:53 :::*                    LISTEN      4052/dnsmasq

[root@ip-192-169-142-147 ~(keystone_admin)]# ps -ef | grep 4052

nobody    4052     1  0 15:30 ?        00:00:00 dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=tapd31feefd-b6 --except-interface=lo --pid-file=/var/lib/neutron/dhcp/238e83f5-864f-44e1-83e6-c9dc33ba752f/pid --dhcp-hostsfile=/var/lib/neutron/dhcp/238e83f5-864f-44e1-83e6-c9dc33ba752f/host --addn-hosts=/var/lib/neutron/dhcp/238e83f5-864f-44e1-83e6-c9dc33ba752f/addn_hosts --dhcp-optsfile=/var/lib/neutron/dhcp/238e83f5-864f-44e1-83e6-c9dc33ba752f/opts --leasefile-ro --dhcp-authoritative --dhcp-range=set:tag0,50.0.0.0,static,86400s --dhcp-lease-max=256 --conf-file=/etc/neutron/dnsmasq-neutron.conf --domain=openstacklocal

END UPDATE

[root@fedora22wks neutron(keystone_admin)]# cat dhcp_agent.ini | grep -v ^# | grep -v ^$
[DEFAULT]
debug = False
resync_interval = 30
interface_driver =neutron.agent.linux.interface.OVSInterfaceDriver
dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq
use_namespaces = True
enable_isolated_metadata = False
enable_metadata_network = False
dhcp_domain = openstacklocal
dnsmasq_config_file =/etc/neutron/dnsmasq-neutron.conf <==
dhcp_delete_namespaces = False
state_path=/var/lib/neutron
root_helper=sudo neutron-rootwrap /etc/neutron/rootwrap.conf

and

# cat  dnsmasq-neutron.conf
log-facility = /var/log/neutron/dnsmasq.log
log-dhcp
# Line added
dhcp-option=26,1454

Set dhcp_lease_time=120 in nova.conf
Restart nova services
Restart neutron-dhcp service
$ps -ef | grep dnsmasq
Check content /var/log/neutron/dnsmasq.log

Can you reproduce analog :-

[root@fedora22wks ~(keystone_admin)]# ip netns exec qdhcp-cdbcae12-3bfa-4690-a653-e7541fc16b77 ifconfig
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

tapee48740e-37: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 50.0.0.10  netmask 255.255.255.0  broadcast 50.0.0.255
        inet6 fe80::f816:3eff:fed8:3a5e  prefixlen 64  scopeid 0x20<link>
        ether fa ...
(more)
edit flag offensive delete link more

Comments

hello, dnsmasq-neutron.conf is not existed in /etc/neutron/ ,then i create the file and add config.

dnsmasq.log is updated in the above post.

but I am confused what happened on log.

the problem is also existing.

twskay gravatar imagetwskay ( 2015-08-21 20:59:50 -0500 )edit

I have a question. I using vxlan mode. Is it different setting config?

thank you!

twskay gravatar imagetwskay ( 2015-08-21 22:04:11 -0500 )edit

Try to recreate private network and post corresponding CLI command && ovs-vsctl show && ifconfig as UPDATE 1 to your question.

dbaxps gravatar imagedbaxps ( 2015-08-22 01:59:51 -0500 )edit

Post also:-

ps -ef | grep dnsmasq
dbaxps gravatar imagedbaxps ( 2015-08-22 02:00:30 -0500 )edit

hello, i have updated . I recreate private network,but problem is the same.(above I post instance console log)

twskay gravatar imagetwskay ( 2015-08-22 05:06:45 -0500 )edit
0

answered 2015-08-20 09:11:59 -0500

Mohit gravatar image

How was the image created from which you created an instance , Can you test out this issue via default cirros image

$ mkdir images
$ cd images/
$ wget http://cdn.download.cirros-cloud.net/0.3.1/cirros-0.3.1-x86_64-disk.img

Load the source file

glance image-create --name="CirrOS-0.3.1" --disk-format=qcow2    --container-format=bare --is-public=true < cirros-0.3.1-x86_64-disk.img

Create an instance using this image an observe what happens to ipconfiguration, Is the DHCP server enabled on the demonet.

edit flag offensive delete link more

Comments

hello, i created image as you say.(i use cirros-0.3.3-x86_64.img) but i don't know how to observe what happens to ipconfiguration, could you tell me,please?

twskay gravatar imagetwskay ( 2015-08-20 10:55:26 -0500 )edit

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2015-08-20 08:44:48 -0500

Seen: 6,749 times

Last updated: Aug 24 '15