Controller Node out of sync, can't add it back to environment

asked 2016-09-14 06:56:25 -0500

katbeaulieu gravatar image

Hi Everyone, I am pretty new to the world of openstack and find myself with a bit of a problem. I noticed that one of our nodes was not connecting at the right speed so decided to try to just power it off and back on and see what happened. Once I did this, I am told this resulted in out of sync databases. Unable to fix the problem I dropped the controller node from the environment in fuel and tried to add it back but keep getting this error:

2016-09-13 19:31:45 ERR  (/Stage[main]/Galera/Exec[wait-for-synced-state]/returns) change from notrun to 0
failed: /usr/bin/mysql -uwsrep_sst -phTUuaexv -Nbe "show status like 'wsrep_local_state_comment'" | /bin/grep -q Synced && sleep 10 returned 1 instead of one of [0]
 2016-09-13 19:31:45    ERR  /usr/bin/puppet:4:in `<main>'
 2016-09-13 19:31:45    ERR  /usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:91:in `execute'
 2016-09-13 19:31:45    ERR  /usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:137:in `run'  
 2016-09-13 19:31:45    ERR  /usr/lib/ruby/vendor_ruby/puppet/application.rb:364:in `run'
 2016-09-13 19:31:45    ERR  /usr/lib/ruby/vendor_ruby/puppet/util.rb:478:in `exit_on_fail'
 2016-09-13 19:31:45    ERR  /usr/lib/ruby/vendor_ruby/puppet/application.rb:364:in `block in run'
 2016-09-13 19:31:45    ERR  /usr/lib/ruby/vendor_ruby/puppet/application.rb:470:in `plugin_hook'
 2016-09-13 19:31:45    ERR  /usr/lib/ruby/vendor_ruby/puppet/application.rb:364:in `block (2 levels) in run'
 2016-09-13 19:31:45    ERR  /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:146:in `run_command'
 2016-09-13 19:31:45    ERR  /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:218:in `main'
 2016-09-13 19:31:45    ERR  /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:268:in `apply_catalog'
 2016-09-13 19:31:45    ERR  /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:192:in `run'
 2016-09-13 19:31:45    ERR  /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:124:in `apply_catalog'
 2016-09-13 19:31:45    ERR  /usr/lib/ruby/vendor_ruby/puppet/util.rb:160:in `benchmark'
 2016-09-13 19:31:45    ERR  /usr/lib/ruby/1.9.1/benchmark.rb:295:in `realtime'
 2016-09-13 19:31:45    ERR  /usr/lib/ruby/vendor_ruby/puppet/util.rb:161:in `block in benchmark'
 2016-09-13 19:31:45    ERR  /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:125:in `block in apply_catalog'
 2016-09-13 19:31:45    ERR  /usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:163:in `apply'
 2016-09-13 19:31:45    ERR  /usr/lib/ruby/vendor_ruby/puppet/transaction/report.rb:108:in `as_logging_destination'
2016-09-13 19:31:45 ERR  /usr/lib/ruby/vendor_ruby/puppet/util/log.rb:149:in `with_destination'
2016-09-13 19:31:45 ERR  /usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:164:in `block in apply'
2016-09-13 19:31:45 ERR  /usr/lib/ruby/vendor_ruby/puppet/transaction.rb:108:in `evaluate'
2016-09-13 19:31:45 ERR  /usr/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:in `traverse'
2016-09-13 19:31:45 ERR  /usr ...
(more)
edit retag flag offensive close merge delete