How to add Ceph OSD node into existing kolla environment?

asked 2019-04-15 03:27:10 -0500

AKSN74 gravatar image

Hi, I'm try to add compute node which contains Ceph OSD role together.

The existing environment is single all-in-one node. Using kolla-ansible to deploy based on stable/rocky version from git. I changed to use multinode inventory and fill the nodes information.

Note that kolla-sv1 is existed single node, kolla-sv2 is additional node.

[control]
kolla-sv1       network_interface="eno2.3832" neutron_external_interface="eno2.1111" storage_interface="eno2.3832" api_interface="eno1" cluster_interface="eno2.3832" tunnel_interface="eno2.3832" dns_interface="eno2.3832"

[network]
kolla-sv1       network_interface="eno2.3832" neutron_external_interface="eno2.1111" storage_interface="eno2.3832" api_interface="eno1" cluster_interface="eno2.3832" tunnel_interface="eno2.3832" dns_interface="eno2.3832"

[inner-compute]

[external-compute]
kolla-sv1       network_interface="eno2.3832" neutron_external_interface="eno2.1111" storage_interface="eno2.3832" api_interface="eno1" cluster_interface="eno2.3832" tunnel_interface="eno2.3832" dns_interface="eno2.3832"
kolla-sv2       network_interface="enp8s0f2.3832" neutron_external_interface="enp8s0f2.1111" storage_interface="enp8s0f2.3832" api_interface="enp8s0f1" cluster_interface="enp8s0f2.3832" tunnel_interface="enp8s0f2.3832" dns_interface="enp8s0f2.3832"

[compute:children]
inner-compute
external-compute

[monitoring]
kolla-sv1       network_interface="eno2.3832" neutron_external_interface="eno2.1111" storage_interface="eno2.3832" api_interface="eno1" cluster_interface="eno2.3832" tunnel_interface="eno2.3832" dns_interface="eno2.3832"

[storage]
kolla-sv1       network_interface="eno2.3832" neutron_external_interface="eno2.1111" storage_interface="eno2.3832" api_interface="eno1" cluster_interface="eno2.3832" tunnel_interface="eno2.3832" dns_interface="eno2.3832"
kolla-sv2       network_interface="enp8s0f2.3832" neutron_external_interface="enp8s0f2.1111" storage_interface="enp8s0f2.3832" api_interface="enp8s0f1" cluster_interface="enp8s0f2.3832" tunnel_interface="enp8s0f2.3832" dns_interface="enp8s0f2.3832"

[deployment]
localhost       ansible_connection=local

Then use below commands to doing kolla-sv2's bootstrap and prechecks

kolla-ansible -i multinode --limit kolla-sv2 bootstrap-servers
kolla-ansible -i multinode --limit kolla-sv2 prechecks

Because I found that using whole deploy command will flush all existed settings on this environment. So I using "limit" options.

kolla-ansible -i multinode deploy --limit kolla-sv2

But I got error during deployment.

TASK [ceph : Copying over config.json files for services] ************************************************************************************************************************************************************************************************************************
skipping: [kolla-sv2] => (item={u'group': u'ceph-mon', u'name': u'ceph-mon'}) 
changed: [kolla-sv2] => (item={u'group': u'ceph-osd', u'name': u'ceph-osd'})
skipping: [kolla-sv2] => (item={u'group': u'ceph-rgw', u'name': u'ceph-rgw'}) 
skipping: [kolla-sv2] => (item={u'group': u'ceph-mgr', u'name': u'ceph-mgr'}) 
skipping: [kolla-sv2] => (item={u'group': u'ceph-mds', u'name': u'ceph-mds'}) 
skipping: [kolla-sv2] => (item={u'group': u'ceph-nfs', u'name': u'ceph-nfs'}) 

TASK [ceph : Copying over ceph.conf] *********************************************************************************************************************************************************************************************************************************************
changed: [kolla-sv2] => (item=ceph-mon)
changed: [kolla-sv2] => (item=ceph-osd)
changed: [kolla-sv2] => (item=ceph-rgw)
changed: [kolla-sv2] => (item=ceph-mgr)
changed: [kolla-sv2] => (item=ceph-mds)
changed: [kolla-sv2] => (item=ceph-nfs)

TASK [ceph : include_tasks] ******************************************************************************************************************************************************************************************************************************************************
skipping: [kolla-sv2]

TASK [ceph : include_tasks] ******************************************************************************************************************************************************************************************************************************************************
included: /usr/share/kolla-ansible/ansible/roles/ceph/tasks/distribute_keyrings.yml for kolla-sv2

TASK [ceph : Fetching Ceph keyrings] *********************************************************************************************************************************************************************************************************************************************
fatal: [kolla-sv2]: FAILED! => {"msg": "'delegate_host' is undefined"}

NO MORE HOSTS LEFT ***************************************************************************************************************************************************************************************************************************************************************
    to retry, use: --limit @/usr/share/kolla-ansible/ansible/site.retry

PLAY RECAP ***********************************************************************************************************************************************************************************************************************************************************************
kolla-sv2                  : ok=48   changed=29   unreachable=0    failed=1   

Command failed ansible-playbook -i multinode -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla  --limit kolla-sv2 -e kolla_action=deploy /usr/share/kolla-ansible/ansible/site.yml

So I also tried option "reconfigure".

kolla-ansible -i multinode reconfigure

After reconfiguration, compute node services launched, but Ceph OSD didn't exist on kolla-sv2.

How to add Ceph OSD into exist cluster in kolla? Anyone have any experience?

edit retag flag offensive close merge delete