Ask Your Question

Failed to upgrade rocky, all services get http 503

asked 2019-09-02 10:49:51 -0500

joey-opens gravatar image


I starting an upgrade from queens 17.1.10 to rocky 18.1.9, using OpenStack ansible as a deploy tool, follow this upgrade guide "" all those steps go well without error till the last one "setup-OpenStack" failed on the first setup configuring keystone, as I saw from this point all endpoints of all the services are unreachable and drop HTTP 503!, the step failed on :

TASK [os_keystone : Add service project] ********************************************************** FAILED - RETRYING: Add service project (5 retries left). FAILED - RETRYING: Add service project (4 retries left). FAILED - RETRYING: Add service project (3 retries left). FAILED - RETRYING: Add service project (2 retries left). FAILED - RETRYING: Add service project (1 retries left). fatal: [node01_keystone_container-097ad5b5 -> localhost]: FAILED! => {"attempts": 5, "changed": false, "extra_data": null, "msg": "Unable to establish connection to http://<ipaddress>:35357/tenants: ('Connection aborted.', error(104, 'Connection reset by peer'))"}

I think something at the start of this task change something in keystone that causes this...unfortunately I didn't success to find it can anyone help me or know about some change in keyston (rocky version) that can be the reason for that?


edit retag flag offensive close merge delete


http://<ipaddress>:35357/tenants is a broken request. It doesn't include the API version (v2 or v3), and tenants indicates v2, which is obsolete by Rocky (not sure if still supported at Queens). This is where I would start my investigation.

DISCLAIMER: I can't help much with OpenStack-Ansible.

Bernd Bausch gravatar imageBernd Bausch ( 2019-09-02 18:54:25 -0500 )edit

Hey thanks for that, In keystone service its looks like they drop the service admin that pointer to port 35357, I will try to check this it may be related

joey-opens gravatar imagejoey-opens ( 2019-09-04 02:49:20 -0500 )edit

It would be strange that the endpoint is automatically changed when upgrading. The endpoint should be under your control.

Bernd Bausch gravatar imageBernd Bausch ( 2019-09-04 03:03:56 -0500 )edit

I have a different env with queen version that I have two services for keystone (public{:5000}, admin{:35357}) after rocy they remove the service file from systemd and from what I read with the removal of the 2.0 API(in rocky) keystone does not distinguish between admin and public endpoints.

joey-opens gravatar imagejoey-opens ( 2019-09-04 06:45:22 -0500 )edit

but still,the playbook running and fail on"oskeystone:Add service project"with no connection to that this port is not used anymore following these changes but I trying to understand if its some that can be changed in my configuration or its a bug in openstackAnsible

joey-opens gravatar imagejoey-opens ( 2019-09-04 06:49:38 -0500 )edit

1 answer

Sort by ยป oldest newest most voted

answered 2019-09-05 08:51:32 -0500

joey-opens gravatar image

upgrade to rocky remove the assign port type(admin) in keystone that pointer to port 35357,

where its failed, the module os_projects have var endpoint_type: admin that from what I saw take it from endpoint list in OpenStack (CLI = openstack endpoint list ), so for me set the admin type to port 5000 was work.

edit flag offensive delete link more

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower


Asked: 2019-09-02 10:49:51 -0500

Seen: 117 times

Last updated: Sep 02 '19