nova serive-list ERROR (InternalServerError): An unexpected error prevented the server from fulfilling your request. (HTTP 500)

asked 2018-04-09 02:50:14 -0600

Ekta gravatar image

Hi,

When I am execution few commands on openstack controller getting below error and also I unable to login to dashboard as well once this error occurred

  1. [root@control-internal tempest]# nova service-list ERROR (InternalServerError): An unexpected error prevented the server from fulfilling your request. (HTTP 500) (Request-ID: req-3c890354-3c66-43b4-af05-8a05946ec710)

  2. [root@control-internal tempest]# glance image-list An unexpected error prevented the server from fulfilling your request. (HTTP 500) (Request-ID: req-1f43f14a-bbb1-4a60-a13b-971c52c62600)

  3. [root@control-internal tempest]# cinder service-list ERROR: An unexpected error prevented the server from fulfilling your request. (HTTP 500) (Request-ID: req-225ca293-68f5-4203-ba84-32ac10334ed3)

  4. [root@control-internal tempest]# openstack endpoint list An unexpected error prevented the server from fulfilling your request. (HTTP 500) (Request-ID: req-333c8014-62e0-4479-8859-22c5cc3a4d00)

When looked into keystone.log it was flooded with MySQL error

2018-04-09 07:39:27.482 22176 ERROR keystone.common.wsgi DBError: (pymysql.err.InternalError) (1129, u"Host 'control-internal' is blocked because of many connection errors; unblock with 'mysqladmin flush-hosts'") 2018-04-09 07:39:27.482 22176 ERROR keystone.common.wsgi

As recommended in the above log itself I did FLUSH HOSTS on open stack database and after this everything looks fine. But I need to flush host very frequently after few hours while my backup job is running.

I just wanted to know what could be the possible reason behind this issue and is "flush host" only remedy for this. I have already increased database max_connection parameter but no luck.

edit retag flag offensive close merge delete

Comments

Deepa gravatar imageDeepa ( 2018-04-09 03:16:26 -0600 )edit
1

Hello, Thank you Deepa for your prompt reply. So for above issue, I increased max_connect_errors parameter on openstack to 100 as its default value was set to 10. Monitored the system for 2 days. I am not observing this issue on my setup as of now.

Ekta gravatar imageEkta ( 2018-04-12 00:25:33 -0600 )edit

@Ekta cool good to know :)

Deepa gravatar imageDeepa ( 2018-04-12 03:13:33 -0600 )edit