Ask Your Question
1

Service MySQL start fail on galera 2nd node

asked 2013-08-23 09:45:46 -0500

RohitGupta gravatar image

updated 2013-08-27 06:21:31 -0500

Hi Experts,

I am using 2 nodes for Galera clustering on ubuntu 12.04 and receiving the following output on node 1:

mysql> SHOW STATUS LIKE 'wsrep%'
 -> ;
+----------------------------+--------------------------------------+
| Variable_name | Value |
+----------------------------+--------------------------------------+
| wsrep_local_state_uuid | 7dbb763b-0bea-11e3-0800-36c52d70d461 |
| wsrep_protocol_version | 4 |
| wsrep_last_committed | 1 |
| wsrep_replicated | 1 |
| wsrep_replicated_bytes | 203 |
| wsrep_received | 2 |
| wsrep_received_bytes | 133 |
| wsrep_local_commits | 0 |
| wsrep_local_cert_failures | 0 |
| wsrep_local_bf_aborts | 0 |
| wsrep_local_replays | 0 |
| wsrep_local_send_queue | 0 |
| wsrep_local_send_queue_avg | 0.000000 |
| wsrep_local_recv_queue | 0 |
| wsrep_local_recv_queue_avg | 0.500000 |
| wsrep_flow_control_paused | 0.000000 |
| wsrep_flow_control_sent | 0 |
| wsrep_flow_control_recv | 0 |
| wsrep_cert_deps_distance | 1.000000 |
| wsrep_apply_oooe | 0.000000 |
| wsrep_apply_oool | 0.000000 |
| wsrep_apply_window | 1.000000 |
| wsrep_commit_oooe | 0.000000 |
| wsrep_commit_oool | 0.000000 |
| wsrep_commit_window | 1.000000 |
| wsrep_local_state | 4 |
| wsrep_local_state_comment | Synced (6) |
| wsrep_cert_index_size | 2 |
| wsrep_causal_reads | 0 |
| wsrep_cluster_conf_id | 1 |
| wsrep_cluster_size | 1 |
| wsrep_cluster_state_uuid | 7dbb763b-0bea-11e3-0800-36c52d70d461 |
| wsrep_cluster_status | Primary |
| wsrep_connected | ON |
| wsrep_local_index | 0 |
| wsrep_provider_name | Galera |
| wsrep_provider_vendor | Codership Oy <info@codership.com> |
| wsrep_provider_version | 23.2.1(r129) |
| wsrep_ready | ON |
+----------------------------+--------------------------------------+
39 rows in set (0.00 sec)

On 2nd node: The Service MySQL fails to start - in /var/log/syslog

Aug 23 17:26:23 control2 mysqld: 130823 17:26:23 [Note] WSREP: GMCast version 0
Aug 23 17:26:23 control2 mysqld: 130823 17:26:23 [Note] WSREP: (c008d869-0c10-11e3-0800-146570776113, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
Aug 23 17:26:23 control2 mysqld: 130823 17:26:23 [Note] WSREP: (c008d869-0c10-11e3-0800-146570776113, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
Aug 23 17:26:23 control2 mysqld: 130823 17:26:23 [Note] WSREP: EVS version 0
Aug 23 17:26:23 control2 mysqld: 130823 17:26:23 [Note] WSREP: PC version 0
Aug 23 17:26:23 control2 mysqld: 130823 17:26:23 [Note] WSREP: gcomm: connecting to group 'controller_cluster', peer '10.0.0.145:'
Aug 23 17:26:23 control2 mysqld: 130823 17:26:23 [Note] WSREP: declaring 130214d4-0c0e-11e3-0800-27da27176580 stable
Aug 23 17:26:23 control2 mysqld: 130823 17:26:23 [Note] WSREP: view(view_id(PRIM,130214d4-0c0e-11e3-0800-27da27176580,6) memb {
Aug 23 17:26:23 control2 mysqld: #011130214d4-0c0e-11e3-0800-27da27176580,
Aug 23 17:26:23 control2 mysqld: #011c008d869-0c10-11e3-0800-146570776113,
Aug 23 17:26:23 control2 mysqld: } joined {
Aug 23 17:26:23 control2 mysqld: } left {
Aug 23 17:26:23 control2 mysqld: } partitioned {
Aug 23 17:26:23 control2 mysqld: })
Aug 23 17:26:24 control2 mysqld: 130823 17:26:24 [Note] WSREP: gcomm: connected
Aug 23 17:26:24 control2 mysqld: 130823 17:26:24 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
Aug 23 17:26:24 control2 mysqld: 130823 17:26:24 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
Aug 23 17:26:24 control2 mysqld: 130823 17:26:24 [Note] WSREP: Opened channel 'controller_cluster'
Aug 23 17:26:24 control2 mysqld: 130823 17:26:24 [Note] WSREP: Waiting for SST to complete.
Aug 23 17:26:24 control2 mysqld: 130823 17:26:24 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 2
Aug 23 17:26:24 control2 mysqld: 130823 17:26:24 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
Aug 23 17:26:24 control2 mysqld: 130823 17:26:24 [Note] WSREP: STATE EXCHANGE: sent state msg: c059b4f5-0c10-11e3-0800-90932b673ef2 ...
(more)
edit retag flag offensive close merge delete
add a comment

2 answers

Sort by ยป oldest newest most voted
0

answered 2013-08-23 15:25:36 -0500

foexle gravatar image

So i don't know enough about wsrep but do you have open the communication ports ? So check your firewall rules

edit flag offensive delete link more

Comments

Hi, Thanks for the reply!! -- I have disabled the firewall "ufw disable" on both the nodes. The following are the iptables --list output on both the nodes: root@control2:~# iptables --list Chain INPUT (policy ACCEPT) target prot opt source destination ACCEPT tcp -- anywhere anywhere tcp dpt:4567 ACCEPT tcp -- anywhere anywhere tcp dpt:MySQL The netstat -ntp output on node2 is: root@control2:~# netstat -ntp Active Internet connections (w/o servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 10.0.0.146:4444 10.0.0.145:60256 ESTABLISHED 10077/rsync tcp 0 0 10.0.0.146:60747 10.0.0.145:4567 ESTABLISHED 10021/mysqld tcp 0 0 192.168.4.229:22 192.168.4.26:56064 ESTABLISHED 2060/sshd: control2 Please check the PS: section on the questions - I have added the error message.

RohitGupta gravatar imageRohitGupta ( 2013-08-27 06:19:42 -0500 )edit

i mean not the local fw rules on each vm. I mean your securitygroups in OpenStack. But there is an other problem: 130827 11:34:55 [ERROR] WSREP: gcs/src/gcs.c:gcs_open():1290: Failed to open channel 'controller_cluster' at 'gcomm://10.0.0.145': -98 (Address already in use) 130827 11:34:55 [ERROR] WSREP: gcs connect failed: Address already in use 130827 11:34:55 [ERROR] WSREP: wsrep::connect() failed: 6 as the answer before, you're using the same address an port

foexle gravatar imagefoexle ( 2013-09-19 09:06:13 -0500 )edit
add a comment
0

answered 2013-09-05 02:35:08 -0500

Ashraf Sharif gravatar image

Taken from the log: "[ERROR] WSREP: Address already in use"

You may need to kill all MySQL-wsrep related processes on Node 2 and restart again. Galera/Wsrep required following ports to be opened and available:

  • 3306 - mysql
  • 4567 - used by Galera for inter-node communication
  • 4444 - used for SST
  • 4568 - used for IST

Use netstat to check for PID of above said listening ports. Manually kill those PIDs and restart back mysqld with --wsrep-cluster-address option.

edit flag offensive delete link more
add a comment

Get to know Ask OpenStack

Resources for moderators

Question Tools

subscribe to rss feed

Stats

Asked: 2013-08-23 09:45:46 -0500

Seen: 11,320 times

Last updated: Sep 05 '13

Related questions

How to restore Openstack mysql database and to sync the database?

Failed to notify nova on events

IPsec or OpenVPN for Swift?

Move instances, flavors and images from one install to another

[Errno 111] Connection refused [closed]

neutron.service no such table: ml2_vlan_allocations [closed]

Openstack Cluster or parallel execution

Cinder SQL & Volume Creation Failure

Trove doesn't work on devstack: ModelNotFoundError: InstanceServiceStatus Not Found

main/binary-i386/Packages 404 Not Found