Controller backup and recovery rabbitmq error

asked 2015-03-08 23:03:03 -0500

rafael canelas gravatar image

updated 2015-03-15 20:25:32 -0500

Hello world,

hope you doing fine while reading this issue...me????...im losing my hair when trying to setup fail recovery on openstack(legacy) controller node...Please...i need your suggestions and recommendations to solve this issue.

I follow the link below:

http://docs.openstack.org/openstack-o...

i tried to setup two controller node on ubuntu12.04 using chef,everything is working well,except for fail recovery.

172.50.0.206 controller master

172.50.0.207 controller standby

Please see my steps below.

im using database replication technique.

1. Controller Master

mysql> SHOW MASTER STATUS; +------------------+----------+-------------------------------------------------------------------+------------------+ | File | Position | Binlog_Do_DB
| Binlog_Ignore_DB | +------------------+----------+-------------------------------------------------------------------+------------------+ | mysql-bin.000007 | 1380647 | ceilometer,cinder,glance,heat,horizon,keystone,mysql,neutron,nova | | +------------------+----------+-------------------------------------------------------------------+------------------+ 1 row in set (0.00 sec)

2. Controller-standby

> mysql> SHOW SLAVE STATUS\G
> *************************** 1. row ***************************
>                Slave_IO_State: Waiting for master to send event
>                   Master_Host: 172.50.0.206
>                   Master_User: controller-standby
>                   Master_Port: 3306
>                 Connect_Retry: 60
>               Master_Log_File: mysql-bin.000007
>           Read_Master_Log_Pos: 1394002
>                Relay_Log_File: mysql-relay-bin.000007
>                 Relay_Log_Pos: 1394148
>         Relay_Master_Log_File: mysql-bin.000007
>              Slave_IO_Running: Yes
>             Slave_SQL_Running: Yes

3.Rsync all files and database backup from controller-master to controller-standby

> nova1="/etc/nova/" 
> nova2="/var/lib/nova/" 
> glance1="/etc/glance/" 
> glance2="/var/lib/glance/" 
> glance3="/var/lib/glance/images/" 
> keystone1="/etc/keystone/" 
> keystone2="/var/lib/keystone/" 
> cinder1="/etc/cinder/" 
> cinder2="/var/lib/cinder/" 
> mysql="/etc/mysql" 
> dashboard="/etc/openstack-dashboard/" 
> rabbit="/etc/rabbitmq/"

And controller-standby recovery

@controller-standby node

gunzip < /root/dbbackup/all-databases-controller-master.20150219.sql.gz | mysql -u root -psecrete

4.shutdown controller-master

5.change controller-standby configuration to become controller-master

controller-standby ip address change from 172.50.0.207 to 172.50.0.206
controller-standby database:change from slave to master
controller-standby hostname from controller-standby to controller-master

After i restart all the services,error occured on the new controller-master.

#Error 1:rabbitmq

root@controller-master:/var/log/rabbitmq# tail -f rabbit@controller-master.log
                 {amqp_error,access_refused,
                             "access to vhost '/nova' refused for user 'guest'",
                             'connection.open'}}

=ERROR REPORT==== 9-Mar-2015::11:08:44 ===
closing AMQP connection <0.15268.77> (172.50.0.206:45275 -> 172.50.0.206:5672):
{handshake_error,opening,0,
                 {amqp_error,access_refused,
                             "access to vhost '/nova' refused for user 'guest'",
                             'connection.open'}}

#Error 2 all are triple X

root@controller-master:~# nova-manage service list
Binary           Host                                 Zone             Status     State Updated_At
nova-conductor   controller-master                     internal         enabled    XXX   2015-02-19 08:54:59
nova-scheduler   controller-master                     internal         enabled    XXX   2015-02-19 08:54:57
nova-cert        controller-master                     internal         enabled    XXX   2015-02-19 08:54:57
nova-consoleauth controller-master                     internal         enabled    XXX   2015-02-19 08:54:57
nova-network     controller-master                     internal         enabled    XXX   2015-02-19 08:54:54

Please i need your help..Thank you in advance.

edit retag flag offensive close merge delete

Comments

Could you make sure that the rabbitmq server in the second server is also having same password as in primary server.

If you haven't set the password try setting the password.

rahulrajvn gravatar imagerahulrajvn ( 2015-03-10 12:17:06 -0500 )edit

Oh thank you rahulrajvn for your reply.I guess you are correct because i thought the password for the rabbitmq server running on a chef is --- > guest.but when i double check it,the actual password is --> pass

root@workstation-node:~/# cat /rabbitmq-server.rb
user user
password pass
rafael canelas gravatar imagerafael canelas ( 2015-03-10 22:13:30 -0500 )edit