Ask Your Question
0

Launch instance: Connection to neutron failed: Maximum attempts reached

asked 2015-03-31 01:15:14 -0500

enojado gravatar image

updated 2015-03-31 01:17:03 -0500

Hi,

When Im launching an instance I get an error.
I have following the docs: http://docs.openstack.org/juno/instal...
Every node running ubuntu 14.04 and using the same network.

- Reinstalled compute and nova with new fresh databases on the controller two times.

I think the main error her is: ""Returning exception Connection to neutron failed: Maximum attempts reached to caller""
I have pasted the list over services and logs under. If I miss something please tell me and I will provide

 neutron agent-list
    +--------------------------------------+--------------------+-----------------+-------+----------------+
    | id                                   | agent_type         | host            | alive | admin_state_up |
    +--------------------------------------+--------------------+-----------------+-------+----------------+
    | 412cc8b0-1204-4e21-8860-19dcc3d4ed72 | HyperV agent       | WIN-431A67622PG | :-)   | True           |
    | 6356cf8c-6057-43bf-b9f9-b80daebaaa97 | Open vSwitch agent | vitkenhost01    | :-)   | True           |
    | 7366b551-c547-4ea0-938e-6bedfe464edc | Metadata agent     | vitkencas03     | :-)   | True           |
    | 86b64640-6134-4221-9d1e-07761cb6eedf | Open vSwitch agent | vitkencas03     | :-)   | True           |
    | c5906abf-7be1-4d37-897d-4fd9f3e5b9b0 | L3 agent           | vitkencas03     | :-)   | True           |
    | e2051c72-ec71-4629-9f69-11b20924dd7c | DHCP agent         | vitkencas03     | :-)   | True           |
    +--------------------------------------+--------------------+-----------------+-------+----------------+


    nova service-list
    +------------------+--------------+----------+---------+-------+----------------------------+-----------------+
    | Binary           | Host         | Zone     | Status  | State | Updated_at                 | Disabled Reason |
    +------------------+--------------+----------+---------+-------+----------------------------+-----------------+
    | nova-conductor   | vitkenxfs03  | internal | enabled | up    | 2015-03-30T18:25:37.000000 | -               |
    | nova-consoleauth | vitkenxfs03  | internal | enabled | up    | 2015-03-30T18:25:36.000000 | -               |
    | nova-cert        | vitkenxfs03  | internal | enabled | up    | 2015-03-30T18:25:32.000000 | -               |
    | nova-scheduler   | vitkenxfs03  | internal | enabled | up    | 2015-03-30T18:25:39.000000 | -               |
    | nova-compute     | vitkenhost01 | nova     | enabled | up    | 2015-03-30T18:25:37.000000 | -               |
    +------------------+--------------+----------+---------+-------+----------------------------+-----------------+


    root@cas03:/etc/neutron# nova list
    +--------------------------------------+----------------+--------+------------+-------------+----------+
    | ID                                   | Name           | Status | Task State | Power State | Networks |
    +--------------------------------------+----------------+--------+------------+-------------+----------+
    | 73d29fd0-f559-4b87-815a-28cb880e5f30 | demo-instance1 | ERROR  | -          | NOSTATE     |          |
    | d5b09e3c-6bae-4c42-85a1-f623ab87f3bb | demo-instance1 | ERROR  | -          | NOSTATE     |          |
    ...





    From network node
    cat /var/log/neutron/openvswitch-agent.log

    2015-03-30 17:48:23.760 26766 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Port ee1ce6e2-9004-4100-b85d-bbab19eff2bc updated. Details: {u'admin_state_up': True, u'network_id': u'2d5bb6ad-1d47-4d65-8e4a-5db897cc5950', u'segmentation_id': 1, u'physical_network': None, u'device': u'ee1ce6e2-9004-4100-b85d-bbab19eff2bc', u'port_id': u'ee1ce6e2-9004-4100-b85d-bbab19eff2bc', u'network_type': u'gre'}
    2015-03-30 17:48:23.761 26766 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Assigning 1 as local vlan for net-id=2d5bb6ad-1d47-4d65-8e4a-5db897cc5950
    2015-03-30 17:48:23.941 26766 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Configuration for device ee1ce6e2-9004-4100-b85d-bbab19eff2bc completed.
    2015-03-30 17:48:24.038 26766 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Port b79d0f53-245c-4725-ac88-d45ed996dbe2 updated. Details: {u'admin_state_up': True, u'network_id': u'2d5bb6ad-1d47-4d65-8e4a-5db897cc5950', u'segmentation_id': 1, u'physical_network': None, u'device': u'b79d0f53-245c-4725-ac88-d45ed996dbe2', u'port_id': u'b79d0f53-245c-4725-ac88-d45ed996dbe2', u'network_type': u'gre'}
    2015-03-30 17:48:24.114 26766 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Configuration for device b79d0f53-245c-4725-ac88-d45ed996dbe2 completed.
    2015-03-30 18:38:58.979 26766 ERROR neutron.agent.linux.ovsdb_monitor [-] Error received from ovsdb monitor: ovsdb-client: unix:/var/run/openvswitch/db.sock: receive failed (End of file)
    2015-03-30 18:38:59.159 26766 ERROR neutron.agent.linux.ovs_lib [-] Unable to execute ['ovs-ofctl', 'dump-flows', 'br-int', 'table=22']. Exception:
    Command: ['sudo', '/usr/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-ofctl', 'dump-flows', 'br-int', 'table=22']
    Exit code: 1
    Stdout: ''
    Stderr: 'ovs-ofctl: br-int is not a bridge or a socket\n'
    2015-03-30 18:38:59.217 26766 ERROR neutron.agent.linux.ovs_lib [-] Unable to execute ['ovs-vsctl', '--timeout=10', '--', '--may-exist', 'add-br', 'br-int']. Exception:
    Command: ['sudo', '/usr/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--', '--may-exist', 'add-br', 'br-int']
    Exit code: 1
    Stdout: ''
    Stderr: '2015-03-30T16:38:59Z|00002|reconnect|WARN|unix:/var/run/openvswitch/db.sock: connection attempt failed (No such file or directory)\novs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory)\n'
    2015-03-30 ...
(more)
edit retag flag offensive close merge delete

Comments

Your openvswitch service is not running ? Start openvswitch service first and see what happens ?
Also run lsmod | grep openvswitch

dbaxps gravatar imagedbaxps ( 2015-03-31 01:34:29 -0500 )edit

On Ubuntu it's called :-

service openvswitch-switch start
dbaxps gravatar imagedbaxps ( 2015-03-31 01:37:10 -0500 )edit

Hi thanks! root@vitkencas03:/etc/neutron# service openvswitch-switch status openvswitch-switch start/running

root@vitkenhost01:/etc/nova# service openvswitch-switch status openvswitch-switch start/running

enojado gravatar imageenojado ( 2015-03-31 01:50:54 -0500 )edit

But still problem. That is the problem. Every service seem to be in good state, and neutron is running.

enojado gravatar imageenojado ( 2015-03-31 01:52:04 -0500 )edit

2 answers

Sort by ยป oldest newest most voted
0

answered 2015-03-31 01:47:08 -0500

enojado gravatar image

updated 2015-03-31 15:42:49 -0500

From both - network node and compute node.

root@vitkenhost01:/etc/nova# lsmod | grep openvswitch
openvswitch            66901  0
gre                    13796  1 openvswitch
vxlan                  37619  1 openvswitch
libcrc32c              12644  1 openvswitch

root@vitkencas03:/etc/neutron# lsmod | grep openvswitch
openvswitch            66901  0
gre                    13796  1 openvswitch
vxlan                  37619  1 openvswitch
libcrc32c              12644  1 openvswitch

EDITED NEW

After this on compute node
 ovs-vsctl del-br br-int
 ovs-vsctl add-br br-int 

This hang

    root@vitkenhost01:/etc/nova# nova boot --flavor m1.tiny --image cirros-0.3.3-x86_64 --nic net-id=2d5bb6ad-1d47-4d65-8e4a-5db897cc5950   --security-group default --key-name demo-key demo-instance9

        root@vitkenhost01:/etc/nova# nova service-list
        +------------------+--------------+----------+---------+-------+----------------------------+-----------------+
        | Binary           | Host         | Zone     | Status  | State | Updated_at                 | Disabled Reason |
        +------------------+--------------+----------+---------+-------+----------------------------+-----------------+
        | nova-conductor   | vitkenxfs03  | internal | enabled | up    | 2015-03-31T07:24:09.000000 | -               |
        | nova-consoleauth | vitkenxfs03  | internal | enabled | up    | 2015-03-31T07:24:04.000000 | -               |
        | nova-cert        | vitkenxfs03  | internal | enabled | up    | 2015-03-31T07:24:10.000000 | -               |
        | nova-scheduler   | vitkenxfs03  | internal | enabled | up    | 2015-03-31T07:24:05.000000 | -               |
        | nova-compute     | vitkenhost01 | nova     | enabled | down  | 2015-03-30T23:06:31.000000 | -               |
        +------------------+--------------+----------+---------+-------+----------------------------+-----------------+


    root@vitkenhost01:/etc/nova# neutron agent-list
    +--------------------------------------+--------------------+-----------------+-------+----------------+
    | id                                   | agent_type         | host            | alive | admin_state_up |
    +--------------------------------------+--------------------+-----------------+-------+----------------+
    | 412cc8b0-1204-4e21-8860-19dcc3d4ed72 | HyperV agent       | WIN-431A67622PG | xxx   | True           |
    | 6356cf8c-6057-43bf-b9f9-b80daebaaa97 | Open vSwitch agent | vitkenhost01    | xxx   | True           |
    | 7366b551-c547-4ea0-938e-6bedfe464edc | Metadata agent     | vitkencas03     | xxx   | True           |
    | 86b64640-6134-4221-9d1e-07761cb6eedf | Open vSwitch agent | vitkencas03     | xxx   | True           |
    | c5906abf-7be1-4d37-897d-4fd9f3e5b9b0 | L3 agent           | vitkencas03     | xxx   | True           |
    | e2051c72-ec71-4629-9f69-11b20924dd7c | DHCP agent         | vitkencas03     | xxx   | True           |
    +--------------------------------------+--------------------+-----------------+-------+----------------+



New log compute node:

root@vitkenhost01:/etc/nova# cat /var/log/neutron/openvswitch-agent.log
Stderr: 'ovs-ofctl: br-int is not a bridge or a socket\n'
2015-03-31 09:13:06.944 28870 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Agent tunnel out of sync with plugin!
2015-03-31 09:13:07.118 28870 INFO neutron.openstack.common.rpc.common [-] Connected to AMQP server on 10.55.0.28:5672
2015-03-31 09:17:33.063 28870 ERROR neutron.agent.linux.ovsdb_monitor [-] Error received from ovsdb monitor: ovsdb-client: unix:/var/run/openvswitch/db.sock: receive failed (End of file)
2015-03-31 09:17:48.239 11277 INFO neutron.common.config [-] Logging enabled!
2015-03-31 09:17:48.735 11277 INFO neutron.openstack.common.rpc.common [req-dda37389-e24e-4cd7-9058-9d8adadd8073 None] Connected to AMQP server on 10.55.0.28:5672
2015-03-31 09:17:48.766 11277 INFO neutron.openstack.common.rpc.common [req-dda37389-e24e-4cd7-9058-9d8adadd8073 None] Connected to AMQP server on 10.55.0.28:5672
2015-03-31 09:17:49.938 11277 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-dda37389-e24e-4cd7-9058-9d8adadd8073 None] Agent initialized successfully, now running...
2015-03-31 09:17:49.943 11277 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-dda37389-e24e-4cd7-9058-9d8adadd8073 None] Agent out of sync with plugin!
2015-03-31 09:17:50.007 11277 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-dda37389-e24e-4cd7-9058-9d8adadd8073 None] Agent tunnel out of sync with plugin!
2015-03-31 09:17:50.031 11277 INFO neutron.openstack.common.rpc.common [req-dda37389-e24e-4cd7-9058-9d8adadd8073 None] Connected to AMQP server on 10.55.0.28:5672
2015-03-31 09:17:50.050 11277 INFO neutron.openstack.common.rpc.common [req-dda37389-e24e-4cd7-9058-9d8adadd8073 None] Connected to AMQP server on 10.55.0.28:5672
2015-03-31 09:27:30.700 11277 ERROR neutron.agent.linux.ovsdb_monitor [req-dda37389-e24e-4cd7-9058-9d8adadd8073 None] Error received from ovsdb monitor: ovsdb-client: unix:/var/run/openvswitch/db.sock: receive failed (End of file)


network node
root@vitkencas03:/etc/neutron# cat /var/log/neutron/openvswitch-agent.log 
2015-03-31 ...
(more)
edit flag offensive delete link more

Comments

ml2_conf.ini is missing

integration_bridge = br-int
tunnel_bridge = br-tun
dbaxps gravatar imagedbaxps ( 2015-03-31 03:57:45 -0500 )edit

Thanks man. Should I add them to network node also?

enojado gravatar imageenojado ( 2015-03-31 04:40:36 -0500 )edit

I have restarted openvswitch -services and both logs started with:
2015-03-31 11:40:06.887 2156 ERROR neutron.agent.linux.ovsdb_monitor [req-b44ed548-479c-4517-87b0-911e9d6bbc73 None] Error received from ovsdb monitor: ovsdb-client: unix:/var/run/openvswitch/db.sock: receive failed (End of file)

enojado gravatar imageenojado ( 2015-03-31 04:41:58 -0500 )edit

Also added openvswitch-agent.log from both nodes below in this answer.

enojado gravatar imageenojado ( 2015-03-31 04:55:18 -0500 )edit

Added also ovs-vsctl show from both network and compute node.

enojado gravatar imageenojado ( 2015-03-31 05:20:05 -0500 )edit
0

answered 2015-03-31 01:54:25 -0500

dbaxps gravatar image

updated 2015-03-31 03:26:48 -0500

From your report :-

   2015-03-30 18:38:58.979 26766 ERROR neutron.agent.linux.ovsdb_monitor [-] Error received from ovsdb monitor: ovsdb-client: unix:/var/run/openvswitch/db.sock: receive failed (End of file)
2015-03-30 18:38:59.159 26766 ERROR neutron.agent.linux.ovs_lib [-] Unable to execute ['ovs-ofctl', 'dump-flows', 'br-int', 'table=22']. Exception:
Command: ['sudo', '/usr/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-ofctl', 'dump-flows', 'br-int', 'table=22']
Exit code: 1
Stdout: ''
Stderr: 'ovs-ofctl: br-int is not a bridge or a socket\n'

Add br-int on all Compute nodes :-

$ sudo ovs-vsctl add br-int

Verify Compute node set up per
http://docs.openstack.org/havana/inst...
Just sections 1,2

Configure the Neutron OVS plug-in for GRE tunneling on a dedicated compute node

Tell the OVS plug-in to use GRE tunneling with a br-int integration bridge, a br-tun tunneling bridge, and a local IP for the tunnel of DATA_INTERFACE's IP Edit the /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini file:
[ovs]
tenant_network_type = gre
tunnel_id_ranges = 1:1000
enable_tunneling = True
integration_bridge = br-int
tunnel_bridge = br-tun
local_ip = DATA_INTERFACE_IP
edit flag offensive delete link more

Comments

No, only did this on network node: ovs-ofctl dump-flows and ovs-vsctl add-br br-ex and ovs-vsctl add-port br-ex eth2

enojado gravatar imageenojado ( 2015-03-31 02:02:05 -0500 )edit

br-int is required on Compute Node as well.

dbaxps gravatar imagedbaxps ( 2015-03-31 02:03:50 -0500 )edit

Ah, so i need to do this on the network node: ovs-vsctl add-br br-int, ovs-vsctl add-br br-eth1 and ovs-vsctl add-port br-eth1 eth1. On the compute node, i need to do this: ovs-vsctl add-br br-int, ovs-vsctl add-br br-eth1 and ovs-vsctl add-port br-eth1 eth1 ??? etho is mng and eth1 tunnel.

enojado gravatar imageenojado ( 2015-03-31 02:10:03 -0500 )edit

Im looking into havana guide: http://docs.openstack.org/havana/inst...

enojado gravatar imageenojado ( 2015-03-31 02:10:36 -0500 )edit

" ovs-vsctl add-port br-eth1 eth1" seems to me VLAN set up step, but you have VXLAN or GRE tunneling.

dbaxps gravatar imagedbaxps ( 2015-03-31 02:15:16 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2015-03-31 01:15:14 -0500

Seen: 4,379 times

Last updated: Mar 31 '15