Error during ceph nodes setup in copy ceph keys
I'm setting up a small test lab for HA Openstack environment based on Queens and CentOS 7 (to be the best similar with OSP 13, as my target). Director is CentOS 7.8 with ansible ansible-2.6.19-1.el7.ans.noarch and undercloud deployed.
Idea is to have 2 compute, 3 controllers, 3 ceph storage nodes
As the first step of overcloud deploy the correct 3 ceph nodes candidates for OSD role are powered on and begin to be configured, but at a certain point I have with this error
2020-05-03 22:31:07Z [AllNodesDeploySteps.CephStorageDeployment_Step1.0]: CREATE_COMPLETE state changed 2020-05-03 22:31:07Z [AllNodesDeploySteps.CephStorageDeployment_Step1]: CREATE_COMPLETE Stack CREATE completed successfully 2020-05-03 22:31:08Z [AllNodesDeploySteps.CephStorageDeployment_Step1]: CREATE_COMPLETE state changed 2020-05-03 22:31:08Z [AllNodesDeploySteps.WorkflowTasks_Step2]: CREATE_IN_PROGRESS state changed 2020-05-03 22:31:09Z [AllNodesDeploySteps.WorkflowTasks_Step2]: CREATE_COMPLETE state changed 2020-05-03 22:31:09Z [AllNodesDeploySteps.WorkflowTasks_Step2_Execution]: CREATE_IN_PROGRESS state changed 2020-05-03 22:33:49Z [AllNodesDeploySteps.WorkflowTasks_Step2_Execution]: CREATE_FAILED resources.WorkflowTasks_Step2_Execution: Failure caused by error in tasks: ceph_base_ansible_workflow ceph_base_ansible_workflow [task_ex_id=b9eee634-5f66-47f9-bef2-57e46cd6d80f] -> Failure caused by error in tasks: ceph_install ceph_install [task_e 2020-05-03 22:33:49Z [AllNodesDeploySteps]: CREATE_FAILED Resource CREATE failed: resources.WorkflowTasks_Step2_Execution: Failure caused by error in tasks: ceph_base_ansible_workflow ... overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution: resource_type: OS::TripleO::WorkflowSteps physical_resource_id: 71e920fe-b781-4d41-b082-cf27c7bbdb4c status: CREATE_FAILED status_reason: | resources.WorkflowTasks_Step2_Execution: Failure caused by error in tasks: ceph_base_ansible_workflow ceph_base_ansible_workflow [task_ex_id=b9eee634-5f66-47f9-bef2-57e46cd6d80f] -> Failure caused by error in tasks: ceph_install ceph_install [task_ex_id=567a8737-4abe-4b24-89e9-6116d4e8eff2] -> One or more actions had failed. ... Unexpected error while running command. Command: ansible-playbook /usr/share/ceph-ansible/site-docker.yml.sample --user tripleo-admin --become --become-user root --extra-vars {"ireallymeanit": "yes", "osd_pool_default_pgp_num": 16, "osd_pool_default_pg_num": 16} --inventory-file /tmp/ansible-mistral-actionV7vbpj/inventory.yaml --private-key /tmp/ansible-mistral-actionV7vbpj/ssh_private_key --skip*** package-install,with_pkg Exit code: 2
And this in /var/log/mistral/ceph-install-workflow.log:
2020-05-04 00:33:45,311 p=17037 u=mistral | TASK [ceph-osd : include_tasks common.yml] ************************************* 2020-05-04 00:33:45,311 p=17037 u=mistral | Monday 04 May 2020 00:33:45 +0200 (0:00:00.145) 0:01:48.161 ************ 2020-05-04 00:33:45,517 p=17037 u=mistral | included: /usr/share/ceph-ansible/roles/ceph-osd/tasks/common.yml for 172.23.0.239, 172.23.0.229, 172.23.0.234 2020-05-04 00:33:45,562 p=17037 u=mistral | TASK [ceph-osd : create bootstrap-osd and osd directories] ********************* 2020-05-04 00:33:45,562 p=17037 u=mistral | Monday 04 May 2020 00:33:45 +0200 (0:00:00.251) 0:01:48.413 ************ 2020-05-04 00:33:45,793 p=17037 u=mistral | ok: [172.23.0.239] => (item=/var/lib/ceph/bootstrap-osd/) 2020-05-04 00:33:45,845 p=17037 u=mistral | ok: [172.23.0.229] => (item=/var/lib/ceph/bootstrap-osd/) 2020-05-04 00:33:45,891 p=17037 u=mistral | ok: [172.23.0.234] => (item=/var/lib/ceph/bootstrap-osd/) 2020-05-04 00:33:45,995 p=17037 u=mistral | ok: [172.23.0.239] => (item=/var/lib/ceph/osd/) 2020-05-04 00:33:46,044 p=17037 u=mistral | ok: [172.23.0.229] => (item=/var/lib/ceph/osd/) 2020-05-04 00:33:46,086 p=17037 u=mistral | ok: [172.23 ...