Undercloud upgrade failed
Hi,
After updating openstack-tripleo-heat-templates from 10.5 to 10.6 (using centos) and other tripleo packages (minor versions) undercloud fails to upgrade. Ran:
openstack undercloud upgrade
Got as result:
Exception: Post Upgrade failed
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1262, in _standalone_deploy
raise exceptions.DeploymentError('Post Upgrade failed')
DeploymentError: Post Upgrade failed
None
** Found ansible errors for undercloud deployment! **
[
[
"Clean podman images",
{
"stderr_lines": [
"Error: failed to prune image: Image used by 9c7b8d36f95751d86f9c0287a9552c5c3dae4e511e0fcb3f89890620c0ac31f5: image is in use by a container"
],
"changed": true,
"end": "2019-08-29 11:42:30.877390",
"_ansible_no_log": false,
"stdout": "",
"cmd": "podman image prune -a",
"start": "2019-08-29 11:42:30.682528",
"delta": "0:00:00.194862",
"stderr": "Error: failed to prune image: Image used by 9c7b8d36f95751d86f9c0287a9552c5c3dae4e511e0fcb3f89890620c0ac31f5: image is in use by a container",
"rc": 125,
"invocation": {
"module_args": {
"creates": null,
"executable": null,
"_uses_shell": true,
"strip_empty_ends": true,
"_raw_params": "podman image prune -a",
"removes": null,
"argv": null,
"warn": true,
"chdir": null,
"stdin_add_newline": true,
"stdin": null
}
},
"stdout_lines": [],
"msg": "non-zero return code"
}
]
]
Install artifact is located at /home/stack/undercloud-install-20190829084230.tar.bzip2
The heat stack undercloud virtual state/action is reset to CREATE. Use "--force-stack-update" to set it forcefully to UPDATE
Removing the stack virtual update mark file /var/lib/tripleo-heat-installer/update_mark_undercloud
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Deployment Failed!
ERROR: Heat log files: /var/log/heat-launcher/undercloud_deploy-EofYwP
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
reset failed: reset: standard error: Inappropriate ioctl for device
Forgot to mention, process running for that image ID:
/usr/libexec/podman/conmon -s -c 9c7b8d36f95751d86f9c0287a9552c5c3dae4e511e0fcb3f89890620c0ac31f5 -u 9c7b8d36f95751d86f9c0287a9552c5c3dae4e511e0fcb3f89890620c0ac31f5 -n ironic_api -r /usr/bin/runc -b /var/lib/containers/storage/overlay-containers/9c7b8d36f95751d86f9c0287a9552c5c3dae4e511e0fcb3f89890620c0ac31f5/userdata -p /var/run/containers/storage/overlay-containers/9c7b8d36f95751d86f9c0287a9552c5c3dae4e511e0fcb3f89890620c0ac31f5/userdata/pidfile --exit-dir /var/run/libpod/exits --conmon-pidfile /var/run/ironic_api.pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /var/run/containers/storage --exit-command-arg --log-level --exit-command-arg error --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /var/run/libpod --exit-command-arg --runtime --exit-command-arg runc --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg container --exit-command-arg cleanup --exit-command-arg 9c7b8d36f95751d86f9c0287a9552c5c3dae4e511e0fcb3f89890620c0ac31f5 --socket-dir-path /var/run/libpod/socket -l k8s-file:/var/log/containers/stdouts/ironic_api.log --log-level error
Any ideas on what might be wrong?
I managed to get it reinstalled by stopping all containers reported by "podman image prune -a" and removing them. From what I can see my modes are ok. Am not sure about: Deployment successful! AND reset failed: reset: standard error: Inappropriate ioctl for device (say success but with an error)