Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

RDO TripleO Mitaka Non-HA Overcloud Failing

Hi All,

I am currently working on Tripleo Mitaka Openstack deployment on baremetal servers: Undercloud – 1 baremetal server with 2 NIC (1 for provisioning and 2nd for external network connectivity) Controller – 1 baremetal server ( 6 NICs with each openstack VLANs on separate NIC) Compute – 1 baremetal server

I followed Graeme's instructions here : https://www.redhat.com/archives/rdo-list/2016-June/msg00049.html to set up Undercloud . Undercloud deployment was successful and all the images required for overcloud deployment was properly built as per the instruction. I would like to mention that I used libvirt tools to modify the root password on overcloud-full.qcow2 and we also modified the grub file to include “net.ifnames=0 biosdevname=0” to restore old interface naming.

I was able to successfully introspect 2 serves to be used for controller and compute nodes. Also , we added the serial device discovered during introspection as root device: ironic node-update 604f7dfc-38af-4fe0-8986-4c8ac5f956e2 add properties/root_device='{"serial": "618e728372833010c79bead9066f0f9e"}' ironic node-update afcfbee3-3108-48da-a6da-aba8f422642c add properties/root_device='{"serial": "618e7283728347101f2107b511603adc"}'

Next, we added compute and control tag to respective introspected node with local boot option:

ironic node-update 604f7dfc-38af-4fe0-8986-4c8ac5f956e2 add properties/capabilities='profile:control,boot_option:local' ironic node-update afcfbee3-3108-48da-a6da-aba8f422642c add properties/capabilities='profile:compute,boot_option:local'

We used multiple NIC templates for control and compute node which has been attached along with network-environment.yaml file. Default network isolation template file has been used.

Deployment script looks like this :

!/bin/bash

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" template_base_dir="$DIR" ntpserver=<sprint.ntp.server.ip> #Sprint LAB openstack overcloud deploy --templates \ -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \ -e ${template_base_dir}/environments/network-environment.yaml \ --control-flavor control --compute-flavor compute \ --control-scale 1 --compute-scale 1 \ --ntp-server $ntpserver \ --neutron-network-type vxlan --neutron-tunnel-types vxlan --debug

Heat stack deployment goes on more really long time (more than 4 hours) and gets stuck at postdeployment configurations. Please find below the capture during install :

Every 2.0s: ironic node-list && nova list && heat stack-list && heat resource-list -n5 overcloud | grep -vi complete Wed Aug 3 17:33:37 2016

+--------------------------------------+------+--------------------------------------+-------------+--------------------+-------------+ | UUID | Name | Instance UUID | Power State | Provisioning State | Maintenance | +--------------------------------------+------+--------------------------------------+-------------+--------------------+-------------+ | 604f7dfc-38af-4fe0-8986-4c8ac5f956e2 | None | 9e7aae15-cabc-4489-a1b2-778915a78df2 | power on | active | False | | afcfbee3-3108-48da-a6da-aba8f422642c | None | c1ab52a9-461a-4a11-a13e-e57ff0a3ae2a | power on | active | False | +--------------------------------------+------+--------------------------------------+-------------+--------------------+-------------+ +--------------------------------------+-------------------------+--------+------------+-------------+------------------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+-------------------------+--------+------------+-------------+------------------------+ | 9e7aae15-cabc-4489-a1b2-778915a78df2 | overcloud-controller-0 | ACTIVE | - | Running | ctlplane=192.168.149.9 | | c1ab52a9-461a-4a11-a13e-e57ff0a3ae2a | overcloud-novacompute-0 | ACTIVE | - | Running | ctlplane=192.168.149.8 | +--------------------------------------+-------------------------+--------+------------+-------------+------------------------+ +--------------------------------------+------------+---------------+---------------------+--------------+ | id | stack_name | stack_status | creation_time | updated_time | +--------------------------------------+------------+---------------+---------------------+--------------+ | 26ee0150-4cfa-4268-9107-8bfbf6712913 | overcloud | CREATE_FAILED | 2016-08-03T08:11:34 | None | +--------------------------------------+------------+---------------+---------------------+--------------+ +---------------------------------------------+-----------------------------------------------+------------------------------------------------------------------------ ---------+--------------------+---------------------+---------------------------------------------------------------------------------------------------------------+ | resource_name | physical_resource_id | resource_type | resource_status | updated_time | stack_name | +---------------------------------------------+-----------------------------------------------+------------------------------------------------------------------------ ---------+--------------------+---------------------+---------------------------------------------------------------------------------------------------------------+ | ComputeNodesPostDeployment | 3797aec6-e543-4dda-9cd1-c7261e827a64 | OS::TripleO::ComputePostDeployment | CREATE_FAILED | 2016-08-03T08:11:35 | overcloud | | ControllerNodesPostDeployment | 6ad9f88c-5c55-4125-97f1-eb0e33329d16 | OS::TripleO::ControllerPostDeployment | CREATE_FAILED | 2016-08-03T08:11:35 | overcloud | | ComputePuppetDeployment | 8b199f85-e4f9-48ad-9aee-b1cdf4900b9f | OS::Heat::StructuredDeployments | CREATE_FAILED | 2016-08-03T08:29:19 | overcloud-ComputeNodesPostDeployment-6vxfu2g2qucy | | ControllerOvercloudServicesDeployment_Step4 | 15509f59-ff28-43af-95dd-6247a6a32c2d | OS::Heat::StructuredDeployments | CREATE_FAILED | 2016-08-03T08:29:20 | overcloud-ControllerNodesPostDeployment-35y7uafngfwj | | 0 | 7cd0aa3d-742f-4e78-99ca-b2a575913f8e | OS::Heat::StructuredDeployment | CREATE_IN_PROGRESS | 2016-08-03T08:30:04 | overcloud-ComputeNodesPostDeployment-6vxfu2g2qucy-ComputePuppetDeployment-cpahcct3tfw3 | | 0 | 5e9308f7-c3a9-4a94-a017-e1acb694c036 | OS::Heat::StructuredDeployment

[stack@mitaka-uc ~]$ openstack software deployment show 5e9308f7-c3a9-4a94-a017-e1acb694c036 +---------------+--------------------------------------+ | Field | Value | +---------------+--------------------------------------+ | id | 5e9308f7-c3a9-4a94-a017-e1acb694c036 | | server_id | 9e7aae15-cabc-4489-a1b2-778915a78df2 | | config_id | 86d49e66-2f25-4cb1-b623-5ae87b01bb64 | | creation_time | 2016-08-03T08:32:10 | | updated_time | | | status | IN_PROGRESS | | status_reason | Deploy data available | | input_values | {} | | action | CREATE | +---------------+--------------------------------------+

[stack@mitaka-uc ~]$ openstack software deployment show --long 5e9308f7-c3a9-4a94-a017-e1acb694c036 +---------------+--------------------------------------+ | Field | Value | +---------------+--------------------------------------+ | id | 5e9308f7-c3a9-4a94-a017-e1acb694c036 | | server_id | 9e7aae15-cabc-4489-a1b2-778915a78df2 | | config_id | 86d49e66-2f25-4cb1-b623-5ae87b01bb64 | | creation_time | 2016-08-03T08:32:10 | | updated_time | | | status | IN_PROGRESS | | status_reason | Deploy data available | | input_values | {} | | action | CREATE | | output_values | None | +---------------+--------------------------------------+

[stack@mitaka-uc ~]$ openstack stack resource list 3797aec6-e543-4dda-9cd1-c7261e827a64 +-------------------------+--------------------------------------+-------------------------------------------------+-----------------+---------------------+ | resource_name | physical_resource_id | resource_type | resource_status | updated_time | +-------------------------+--------------------------------------+-------------------------------------------------+-----------------+---------------------+ | ComputeArtifactsConfig | a33cd04d-61ab-4429-8565-182409c2b97f | file:///usr/share/openstack-tripleo-heat- | CREATE_COMPLETE | 2016-08-03T08:29:19 | | | | templates/puppet/deploy-artifacts.yaml | | | | ComputePuppetConfig | 5bb712b0-5358-46c7-a444-f9adedfedd50 | OS::Heat::SoftwareConfig | CREATE_COMPLETE | 2016-08-03T08:29:19 | | ComputePuppetDeployment | 8b199f85-e4f9-48ad-9aee-b1cdf4900b9f | OS::Heat::StructuredDeployments | CREATE_FAILED | 2016-08-03T08:29:19 | | ComputeArtifactsDeploy | 1d13bf34-fc66-4bf1-a3b7-1dd815f58f5a | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2016-08-03T08:29:19 | | ExtraConfig | | OS::TripleO::NodeExtraConfigPost | INIT_COMPLETE | 2016-08-03T08:29:19 | +-------------------------+--------------------------------------+-------------------------------------------------+-----------------+---------------------+

[stack@mitaka-uc ~]$ openstack stack resource list 8b199f85-e4f9-48ad-9aee-b1cdf4900b9f +---------------+--------------------------------------+--------------------------------+--------------------+---------------------+ | resource_name | physical_resource_id | resource_type | resource_status | updated_time | +---------------+--------------------------------------+--------------------------------+--------------------+---------------------+ | 0 | 7cd0aa3d-742f-4e78-99ca-b2a575913f8e | OS::Heat::StructuredDeployment | CREATE_IN_PROGRESS | 2016-08-03T08:30:04 | +---------------+--------------------------------------+--------------------------------+--------------------+---------------------+ [stack@mitaka-uc ~]$ openstack software deployment show 7cd0aa3d-742f-4e78-99ca-b2a575913f8e +---------------+--------------------------------------+ | Field | Value | +---------------+--------------------------------------+ | id | 7cd0aa3d-742f-4e78-99ca-b2a575913f8e | | server_id | c1ab52a9-461a-4a11-a13e-e57ff0a3ae2a | | config_id | 24e5c0db-f84f-4a94-8f8e-8e38e73ccc86 | | creation_time | 2016-08-03T08:30:05 | | updated_time | | | status | IN_PROGRESS | | status_reason | Deploy data available | | input_values | {} | | action | CREATE | +---------------+--------------------------------------+

Keystonerc file was not generated. Please find below openstack status command result on controller and compute.

[heat-admin@overcloud-controller-0 ~]$ openstack-status == Nova services == openstack-nova-api: active openstack-nova-compute: inactive (disabled on boot) openstack-nova-network: inactive (disabled on boot) openstack-nova-scheduler: activating(disabled on boot) openstack-nova-cert: active openstack-nova-conductor: active openstack-nova-console: inactive (disabled on boot) openstack-nova-consoleauth: active openstack-nova-xvpvncproxy: inactive (disabled on boot) == Glance services == openstack-glance-api: active openstack-glance-registry: active == Keystone service == openstack-keystone: inactive (disabled on boot) == Horizon service == openstack-dashboard: uncontactable == neutron services == neutron-server: failed (disabled on boot) neutron-dhcp-agent: inactive (disabled on boot) neutron-l3-agent: inactive (disabled on boot) neutron-metadata-agent: inactive (disabled on boot) neutron-lbaas-agent: inactive (disabled on boot) neutron-openvswitch-agent: inactive (disabled on boot) neutron-metering-agent: inactive (disabled on boot) == Swift services == openstack-swift-proxy: active openstack-swift-account: active openstack-swift-container: active openstack-swift-object: active == Cinder services == openstack-cinder-api: active openstack-cinder-scheduler: active openstack-cinder-volume: active openstack-cinder-backup: inactive (disabled on boot) == Ceilometer services == openstack-ceilometer-api: active openstack-ceilometer-central: active openstack-ceilometer-compute: inactive (disabled on boot) openstack-ceilometer-collector: active openstack-ceilometer-notification: active == Heat services == openstack-heat-api: inactive (disabled on boot) openstack-heat-api-cfn: active openstack-heat-api-cloudwatch: inactive (disabled on boot) openstack-heat-engine: inactive (disabled on boot) == Sahara services == openstack-sahara-api: active openstack-sahara-engine: active == Support services == libvirtd: active openvswitch: active dbus: active target: active rabbitmq-server: active memcached: active

[heat-admin@overcloud-novacompute-0 ~]$ openstack-status == Nova services == openstack-nova-api: inactive (disabled on boot) openstack-nova-compute: activating(disabled on boot) openstack-nova-network: inactive (disabled on boot) openstack-nova-scheduler: inactive (disabled on boot) openstack-nova-cert: inactive (disabled on boot) openstack-nova-conductor: inactive (disabled on boot) openstack-nova-console: inactive (disabled on boot) openstack-nova-consoleauth: inactive (disabled on boot) openstack-nova-xvpvncproxy: inactive (disabled on boot) == Glance services == openstack-glance-api: inactive (disabled on boot) openstack-glance-registry: inactive (disabled on boot) == Keystone service == openstack-keystone: inactive (disabled on boot) == Horizon service == openstack-dashboard: uncontactable == neutron services == neutron-server: inactive (disabled on boot) neutron-dhcp-agent: inactive (disabled on boot) neutron-l3-agent: inactive (disabled on boot) neutron-metadata-agent: inactive (disabled on boot) neutron-lbaas-agent: inactive (disabled on boot) neutron-openvswitch-agent: active neutron-metering-agent: inactive (disabled on boot) == Swift services == openstack-swift-proxy: inactive (disabled on boot) openstack-swift-account: inactive (disabled on boot) openstack-swift-container: inactive (disabled on boot) openstack-swift-object: inactive (disabled on boot) == Cinder services == openstack-cinder-api: inactive (disabled on boot) openstack-cinder-scheduler: inactive (disabled on boot) openstack-cinder-volume: inactive (disabled on boot) openstack-cinder-backup: inactive (disabled on boot) == Ceilometer services == openstack-ceilometer-api: inactive (disabled on boot) openstack-ceilometer-central: inactive (disabled on boot) openstack-ceilometer-compute: inactive (disabled on boot) openstack-ceilometer-collector: inactive (disabled on boot) openstack-ceilometer-notification: inactive (disabled on boot) == Heat services == openstack-heat-api: inactive (disabled on boot) openstack-heat-api-cfn: inactive (disabled on boot) openstack-heat-api-cloudwatch: inactive (disabled on boot) openstack-heat-engine: inactive (disabled on boot) == Sahara services == openstack-sahara-all: inactive (disabled on boot) == Support services == libvirtd: active openvswitch: active dbus: active rabbitmq-server: inactive (disabled on boot) memcached: inactive (disabled on boot)

Please let me know if there is any other logs which I can provide that can help in troubleshooting.

Thanks a lot in Advance for your help and support.

Best Regards, Milind Gunjan

RDO TripleO Mitaka Non-HA Overcloud Failing

Hi All,

I am currently working on Tripleo Mitaka Openstack deployment on baremetal servers: Undercloud – 1 baremetal server with 2 NIC (1 for provisioning and 2nd for external network connectivity) Controller – 1 3 baremetal server ( 6 7 NICs with each openstack VLANs on separate NIC) Compute – 1 baremetal serverserver ( 7 NICs with each openstack VLANs on separate NIC)

I followed Graeme's instructions here : https://www.redhat.com/archives/rdo-list/2016-June/msg00049.html to set up Undercloud . Undercloud deployment was successful and all the images required for overcloud deployment was properly built as per the instruction. I would like to mention that I used libvirt tools to modify the root password on overcloud-full.qcow2 and we also modified the grub file to include “net.ifnames=0 biosdevname=0” biosdevname=0” to restore old interface naming.

I was able to successfully introspect 2 serves to be used for controller and compute nodes. Also , we added the serial device discovered during introspection as root device: ironic node-update 604f7dfc-38af-4fe0-8986-4c8ac5f956e2 add properties/root_device='{"serial": "618e728372833010c79bead9066f0f9e"}' ironic node-update afcfbee3-3108-48da-a6da-aba8f422642c add properties/root_device='{"serial": "618e7283728347101f2107b511603adc"}'"618e7283728347101f2107b511603adc"}'

Next, we added compute and control tag to respective introspected node with local boot option:

ironic node-update 604f7dfc-38af-4fe0-8986-4c8ac5f956e2 add properties/capabilities='profile:control,boot_option:local' ironic node-update afcfbee3-3108-48da-a6da-aba8f422642c add properties/capabilities='profile:compute,boot_option:local'

We used multiple NIC templates for control and compute node which has been attached along with network-environment.yaml file. Default network isolation template file has been used.

Deployment script looks like this :

:

!/bin/bash

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" template_base_dir="$DIR" ntpserver=<sprint.ntp.server.ip> #Sprint LAB openstack overcloud deploy --templates \ -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \ -e ${template_base_dir}/environments/network-environment.yaml \ --control-flavor control --compute-flavor compute \ --control-scale 1 --compute-scale 1 \ --ntp-server $ntpserver \ --neutron-network-type vxlan --neutron-tunnel-types vxlan --debug--debug

Heat stack deployment goes on more really long time (more than 4 hours) and gets stuck at postdeployment configurations. Please find below the capture during install :

Every 2.0s: ironic node-list && nova list && heat stack-list && heat resource-list -n5 overcloud | grep -vi complete                           Wed Aug  3 17:33:37 2016

2016

+--------------------------------------+------+--------------------------------------+-------------+--------------------+-------------+ | UUID | Name | Instance UUID | Power State | Provisioning State | Maintenance | +--------------------------------------+------+--------------------------------------+-------------+--------------------+-------------+ | 604f7dfc-38af-4fe0-8986-4c8ac5f956e2 | None | 9e7aae15-cabc-4489-a1b2-778915a78df2 | power on | active | False | | afcfbee3-3108-48da-a6da-aba8f422642c | None | c1ab52a9-461a-4a11-a13e-e57ff0a3ae2a | power on | active | False | +--------------------------------------+------+--------------------------------------+-------------+--------------------+-------------+ +--------------------------------------+-------------------------+--------+------------+-------------+------------------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+-------------------------+--------+------------+-------------+------------------------+ | 9e7aae15-cabc-4489-a1b2-778915a78df2 | overcloud-controller-0 | ACTIVE | - | Running | ctlplane=192.168.149.9 | | c1ab52a9-461a-4a11-a13e-e57ff0a3ae2a | overcloud-novacompute-0 | ACTIVE | - | Running | ctlplane=192.168.149.8 | +--------------------------------------+-------------------------+--------+------------+-------------+------------------------+ +--------------------------------------+------------+---------------+---------------------+--------------+ | id | stack_name | stack_status | creation_time | updated_time | +--------------------------------------+------------+---------------+---------------------+--------------+ | 26ee0150-4cfa-4268-9107-8bfbf6712913 | overcloud | CREATE_FAILED | 2016-08-03T08:11:34 | None | +--------------------------------------+------------+---------------+---------------------+--------------+ +---------------------------------------------+-----------------------------------------------+------------------------------------------------------------------------ ---------+--------------------+---------------------+---------------------------------------------------------------------------------------------------------------+ | resource_name | physical_resource_id | resource_type | resource_status | updated_time | stack_name | +---------------------------------------------+-----------------------------------------------+------------------------------------------------------------------------ ---------+--------------------+---------------------+---------------------------------------------------------------------------------------------------------------+ | ComputeNodesPostDeployment | 3797aec6-e543-4dda-9cd1-c7261e827a64 | OS::TripleO::ComputePostDeployment | CREATE_FAILED | 2016-08-03T08:11:35 | overcloud | | ControllerNodesPostDeployment | 6ad9f88c-5c55-4125-97f1-eb0e33329d16 | OS::TripleO::ControllerPostDeployment | CREATE_FAILED | 2016-08-03T08:11:35 | overcloud | | ComputePuppetDeployment | 8b199f85-e4f9-48ad-9aee-b1cdf4900b9f | OS::Heat::StructuredDeployments | CREATE_FAILED | 2016-08-03T08:29:19 | overcloud-ComputeNodesPostDeployment-6vxfu2g2qucy | | ControllerOvercloudServicesDeployment_Step4 | 15509f59-ff28-43af-95dd-6247a6a32c2d | OS::Heat::StructuredDeployments | CREATE_FAILED | 2016-08-03T08:29:20 | overcloud-ControllerNodesPostDeployment-35y7uafngfwj | | 0 | 7cd0aa3d-742f-4e78-99ca-b2a575913f8e | OS::Heat::StructuredDeployment | CREATE_IN_PROGRESS | 2016-08-03T08:30:04 | overcloud-ComputeNodesPostDeployment-6vxfu2g2qucy-ComputePuppetDeployment-cpahcct3tfw3 | | 0 | 5e9308f7-c3a9-4a94-a017-e1acb694c036 | OS::Heat::StructuredDeployment

OS::Heat::StructuredDeployment

[stack@mitaka-uc ~]$ openstack software deployment show 5e9308f7-c3a9-4a94-a017-e1acb694c036 +---------------+--------------------------------------+ | Field | Value | +---------------+--------------------------------------+ | id | 5e9308f7-c3a9-4a94-a017-e1acb694c036 | | server_id | 9e7aae15-cabc-4489-a1b2-778915a78df2 | | config_id | 86d49e66-2f25-4cb1-b623-5ae87b01bb64 | | creation_time | 2016-08-03T08:32:10 | | updated_time | | | status | IN_PROGRESS | | status_reason | Deploy data available | | input_values | {} | | action | CREATE | +---------------+--------------------------------------+

+---------------+--------------------------------------+

[stack@mitaka-uc ~]$ openstack software deployment show --long 5e9308f7-c3a9-4a94-a017-e1acb694c036 +---------------+--------------------------------------+ | Field | Value | +---------------+--------------------------------------+ | id | 5e9308f7-c3a9-4a94-a017-e1acb694c036 | | server_id | 9e7aae15-cabc-4489-a1b2-778915a78df2 | | config_id | 86d49e66-2f25-4cb1-b623-5ae87b01bb64 | | creation_time | 2016-08-03T08:32:10 | | updated_time | | | status | IN_PROGRESS | | status_reason | Deploy data available | | input_values | {} | | action | CREATE | | output_values | None | +---------------+--------------------------------------+

+---------------+--------------------------------------+

[stack@mitaka-uc ~]$ openstack stack resource list 3797aec6-e543-4dda-9cd1-c7261e827a64 +-------------------------+--------------------------------------+-------------------------------------------------+-----------------+---------------------+ | resource_name | physical_resource_id | resource_type | resource_status | updated_time | +-------------------------+--------------------------------------+-------------------------------------------------+-----------------+---------------------+ | ComputeArtifactsConfig | a33cd04d-61ab-4429-8565-182409c2b97f | file:///usr/share/openstack-tripleo-heat- | CREATE_COMPLETE | 2016-08-03T08:29:19 | | | | templates/puppet/deploy-artifacts.yaml | | | | ComputePuppetConfig | 5bb712b0-5358-46c7-a444-f9adedfedd50 | OS::Heat::SoftwareConfig | CREATE_COMPLETE | 2016-08-03T08:29:19 | | ComputePuppetDeployment | 8b199f85-e4f9-48ad-9aee-b1cdf4900b9f | OS::Heat::StructuredDeployments | CREATE_FAILED | 2016-08-03T08:29:19 | | ComputeArtifactsDeploy | 1d13bf34-fc66-4bf1-a3b7-1dd815f58f5a | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2016-08-03T08:29:19 | | ExtraConfig | | OS::TripleO::NodeExtraConfigPost | INIT_COMPLETE | 2016-08-03T08:29:19 | +-------------------------+--------------------------------------+-------------------------------------------------+-----------------+---------------------+

+-------------------------+--------------------------------------+-------------------------------------------------+-----------------+---------------------+

[stack@mitaka-uc ~]$ openstack stack resource list 8b199f85-e4f9-48ad-9aee-b1cdf4900b9f +---------------+--------------------------------------+--------------------------------+--------------------+---------------------+ | resource_name | physical_resource_id | resource_type | resource_status | updated_time | +---------------+--------------------------------------+--------------------------------+--------------------+---------------------+ | 0 | 7cd0aa3d-742f-4e78-99ca-b2a575913f8e | OS::Heat::StructuredDeployment | CREATE_IN_PROGRESS | 2016-08-03T08:30:04 | +---------------+--------------------------------------+--------------------------------+--------------------+---------------------+ [stack@mitaka-uc ~]$ openstack software deployment show 7cd0aa3d-742f-4e78-99ca-b2a575913f8e +---------------+--------------------------------------+ | Field | Value | +---------------+--------------------------------------+ | id | 7cd0aa3d-742f-4e78-99ca-b2a575913f8e | | server_id | c1ab52a9-461a-4a11-a13e-e57ff0a3ae2a | | config_id | 24e5c0db-f84f-4a94-8f8e-8e38e73ccc86 | | creation_time | 2016-08-03T08:30:05 | | updated_time | | | status | IN_PROGRESS | | status_reason | Deploy data available | | input_values | {} | | action | CREATE | +---------------+--------------------------------------+

+---------------+--------------------------------------+

Keystonerc file was not generated. Please find below openstack status command result on controller and compute.

[heat-admin@overcloud-controller-0 ~]$ openstack-status
== Nova services ==
openstack-nova-api:                     active
openstack-nova-compute:                 inactive  (disabled on boot)
openstack-nova-network:                 inactive  (disabled on boot)
openstack-nova-scheduler:               activating(disabled on boot)
openstack-nova-cert:                    active
openstack-nova-conductor:               active
openstack-nova-console:                 inactive  (disabled on boot)
openstack-nova-consoleauth:             active
openstack-nova-xvpvncproxy:             inactive  (disabled on boot)
== Glance services ==
openstack-glance-api:                   active
openstack-glance-registry:              active
== Keystone service ==
openstack-keystone:                     inactive  (disabled on boot)
== Horizon service ==
openstack-dashboard:                    uncontactable
== neutron services ==
neutron-server:                         failed    (disabled on boot)
neutron-dhcp-agent:                     inactive  (disabled on boot)
neutron-l3-agent:                       inactive  (disabled on boot)
neutron-metadata-agent:                 inactive  (disabled on boot)
neutron-lbaas-agent:                    inactive  (disabled on boot)
neutron-openvswitch-agent:              inactive  (disabled on boot)
neutron-metering-agent:                 inactive  (disabled on boot)
== Swift services ==
openstack-swift-proxy:                  active
openstack-swift-account:                active
openstack-swift-container:              active
openstack-swift-object:                 active
== Cinder services ==
openstack-cinder-api:                   active
openstack-cinder-scheduler:             active
openstack-cinder-volume:                active
openstack-cinder-backup:                inactive  (disabled on boot)
== Ceilometer services ==
openstack-ceilometer-api:               active
openstack-ceilometer-central:           active
openstack-ceilometer-compute:           inactive  (disabled on boot)
openstack-ceilometer-collector:         active
openstack-ceilometer-notification:      active
== Heat services ==
openstack-heat-api:                     inactive  (disabled on boot)
openstack-heat-api-cfn:                 active
openstack-heat-api-cloudwatch:          inactive  (disabled on boot)
openstack-heat-engine:                  inactive  (disabled on boot)
== Sahara services ==
openstack-sahara-api:                   active
openstack-sahara-engine:                active
== Support services ==
libvirtd:                               active
openvswitch:                            active
dbus:                                   active
target:                                 active
rabbitmq-server:                        active
memcached:                              active

active

[heat-admin@overcloud-novacompute-0 ~]$ openstack-status == Nova services == openstack-nova-api: inactive (disabled on boot) openstack-nova-compute: activating(disabled on boot) openstack-nova-network: inactive (disabled on boot) openstack-nova-scheduler: inactive (disabled on boot) openstack-nova-cert: inactive (disabled on boot) openstack-nova-conductor: inactive (disabled on boot) openstack-nova-console: inactive (disabled on boot) openstack-nova-consoleauth: inactive (disabled on boot) openstack-nova-xvpvncproxy: inactive (disabled on boot) == Glance services == openstack-glance-api: inactive (disabled on boot) openstack-glance-registry: inactive (disabled on boot) == Keystone service == openstack-keystone: inactive (disabled on boot) == Horizon service == openstack-dashboard: uncontactable == neutron services == neutron-server: inactive (disabled on boot) neutron-dhcp-agent: inactive (disabled on boot) neutron-l3-agent: inactive (disabled on boot) neutron-metadata-agent: inactive (disabled on boot) neutron-lbaas-agent: inactive (disabled on boot) neutron-openvswitch-agent: active neutron-metering-agent: inactive (disabled on boot) == Swift services == openstack-swift-proxy: inactive (disabled on boot) openstack-swift-account: inactive (disabled on boot) openstack-swift-container: inactive (disabled on boot) openstack-swift-object: inactive (disabled on boot) == Cinder services == openstack-cinder-api: inactive (disabled on boot) openstack-cinder-scheduler: inactive (disabled on boot) openstack-cinder-volume: inactive (disabled on boot) openstack-cinder-backup: inactive (disabled on boot) == Ceilometer services == openstack-ceilometer-api: inactive (disabled on boot) openstack-ceilometer-central: inactive (disabled on boot) openstack-ceilometer-compute: inactive (disabled on boot) openstack-ceilometer-collector: inactive (disabled on boot) openstack-ceilometer-notification: inactive (disabled on boot) == Heat services == openstack-heat-api: inactive (disabled on boot) openstack-heat-api-cfn: inactive (disabled on boot) openstack-heat-api-cloudwatch: inactive (disabled on boot) openstack-heat-engine: inactive (disabled on boot) == Sahara services == openstack-sahara-all: inactive (disabled on boot) == Support services == libvirtd: active openvswitch: active dbus: active rabbitmq-server: inactive (disabled on boot) memcached: inactive (disabled on boot)

boot)

Please let me know if there is any other logs which I can provide that can help in troubleshooting.

Thanks a lot in Advance for your help and support.

Best Regards, Milind Gunjan

RDO TripleO Mitaka Non-HA Overcloud Failing

Hi All,

I am currently working on Tripleo Mitaka Openstack deployment on baremetal servers: Undercloud – 1 baremetal server with 2 NIC (1 for provisioning and 2nd for external network connectivity) Controller – 3 baremetal server ( 7 NICs with each openstack VLANs on separate NIC) Compute – 1 baremetal server ( 7 NICs with each openstack VLANs on separate NIC)

I followed Graeme's instructions here : https://www.redhat.com/archives/rdo-list/2016-June/msg00049.html to set up Undercloud . Undercloud deployment was successful and all the images required for overcloud deployment was properly built as per the instruction. I would like to mention that I used libvirt tools to modify the root password on overcloud-full.qcow2 and we also modified the grub file to include “net.ifnames=0 biosdevname=0” to restore old interface naming.

I was able to successfully introspect 2 serves to be used for controller and compute nodes. Also , we added the serial device discovered during introspection as root device: ironic node-update 604f7dfc-38af-4fe0-8986-4c8ac5f956e2 add properties/root_device='{"serial": "618e728372833010c79bead9066f0f9e"}' ironic node-update afcfbee3-3108-48da-a6da-aba8f422642c add properties/root_device='{"serial": "618e7283728347101f2107b511603adc"}'

Next, we added compute and control tag to respective introspected node with local boot option:

ironic node-update 604f7dfc-38af-4fe0-8986-4c8ac5f956e2 add properties/capabilities='profile:control,boot_option:local' ironic node-update afcfbee3-3108-48da-a6da-aba8f422642c add properties/capabilities='profile:compute,boot_option:local'

We used multiple NIC templates for control and compute node which has been attached along with network-environment.yaml file. Default network isolation template file has been used.

Deployment script looks like this : DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" template_base_dir="$DIR" ntpserver=<sprint.ntp.server.ip> #Sprint LAB openstack overcloud deploy --templates \ -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \ -e ${template_base_dir}/environments/network-environment.yaml \ --control-flavor control --compute-flavor compute \ --control-scale 1 --compute-scale 1 \ --ntp-server $ntpserver \ --neutron-network-type vxlan --neutron-tunnel-types vxlan --debug

Heat stack deployment goes on more really long time (more than 4 hours) and gets stuck at postdeployment configurations. Please find below the capture during install :

Every 2.0s: ironic node-list && nova list && heat stack-list && heat resource-list -n5 overcloud | grep -vi complete                           Wed Aug  3 17:33:37 2016

+--------------------------------------+------+--------------------------------------+-------------+--------------------+-------------+
| UUID                                 | Name | Instance UUID                        | Power State | Provisioning State | Maintenance |
+--------------------------------------+------+--------------------------------------+-------------+--------------------+-------------+
| 604f7dfc-38af-4fe0-8986-4c8ac5f956e2 | None | 9e7aae15-cabc-4489-a1b2-778915a78df2 | power on    | active             | False       |
| afcfbee3-3108-48da-a6da-aba8f422642c | None | c1ab52a9-461a-4a11-a13e-e57ff0a3ae2a | power on    | active             | False       |
+--------------------------------------+------+--------------------------------------+-------------+--------------------+-------------+
+--------------------------------------+-------------------------+--------+------------+-------------+------------------------+
| ID                                   | Name                    | Status | Task State | Power State | Networks               |
+--------------------------------------+-------------------------+--------+------------+-------------+------------------------+
| 9e7aae15-cabc-4489-a1b2-778915a78df2 | overcloud-controller-0  | ACTIVE | -          | Running     | ctlplane=192.168.149.9 |
| c1ab52a9-461a-4a11-a13e-e57ff0a3ae2a | overcloud-novacompute-0 | ACTIVE | -          | Running     | ctlplane=192.168.149.8 |
+--------------------------------------+-------------------------+--------+------------+-------------+------------------------+
+--------------------------------------+------------+---------------+---------------------+--------------+
| id                                   | stack_name | stack_status  | creation_time       | updated_time |
+--------------------------------------+------------+---------------+---------------------+--------------+
| 26ee0150-4cfa-4268-9107-8bfbf6712913 | overcloud  | CREATE_FAILED | 2016-08-03T08:11:34 | None         |
+--------------------------------------+------------+---------------+---------------------+--------------+
+---------------------------------------------+-----------------------------------------------+------------------------------------------------------------------------
---------+--------------------+---------------------+---------------------------------------------------------------------------------------------------------------+
| resource_name                               | physical_resource_id                          | resource_type
         | resource_status    | updated_time        | stack_name                                                                                                    |
+---------------------------------------------+-----------------------------------------------+------------------------------------------------------------------------
---------+--------------------+---------------------+---------------------------------------------------------------------------------------------------------------+
| ComputeNodesPostDeployment                  | 3797aec6-e543-4dda-9cd1-c7261e827a64          | OS::TripleO::ComputePostDeployment
         | CREATE_FAILED      | 2016-08-03T08:11:35 | overcloud                                                                                                     |
| ControllerNodesPostDeployment               | 6ad9f88c-5c55-4125-97f1-eb0e33329d16          | OS::TripleO::ControllerPostDeployment
         | CREATE_FAILED      | 2016-08-03T08:11:35 | overcloud                                                                                                     |
| ComputePuppetDeployment                     | 8b199f85-e4f9-48ad-9aee-b1cdf4900b9f          | OS::Heat::StructuredDeployments
         | CREATE_FAILED      | 2016-08-03T08:29:19 | overcloud-ComputeNodesPostDeployment-6vxfu2g2qucy                                                             |
| ControllerOvercloudServicesDeployment_Step4 | 15509f59-ff28-43af-95dd-6247a6a32c2d          | OS::Heat::StructuredDeployments
         | CREATE_FAILED      | 2016-08-03T08:29:20 | overcloud-ControllerNodesPostDeployment-35y7uafngfwj                                                          |
| 0                                           | 7cd0aa3d-742f-4e78-99ca-b2a575913f8e          | OS::Heat::StructuredDeployment
         | CREATE_IN_PROGRESS | 2016-08-03T08:30:04 | overcloud-ComputeNodesPostDeployment-6vxfu2g2qucy-ComputePuppetDeployment-cpahcct3tfw3                        |
| 0                                           | 5e9308f7-c3a9-4a94-a017-e1acb694c036          | OS::Heat::StructuredDeployment



[stack@mitaka-uc ~]$ openstack software deployment show 5e9308f7-c3a9-4a94-a017-e1acb694c036
+---------------+--------------------------------------+
| Field         | Value                                |
+---------------+--------------------------------------+
| id            | 5e9308f7-c3a9-4a94-a017-e1acb694c036 |
| server_id     | 9e7aae15-cabc-4489-a1b2-778915a78df2 |
| config_id     | 86d49e66-2f25-4cb1-b623-5ae87b01bb64 |
| creation_time | 2016-08-03T08:32:10                  |
| updated_time  |                                      |
| status        | IN_PROGRESS                          |
| status_reason | Deploy data available                |
| input_values  | {}                                   |
| action        | CREATE                               |
+---------------+--------------------------------------+

[stack@mitaka-uc ~]$ openstack software deployment show --long 5e9308f7-c3a9-4a94-a017-e1acb694c036
+---------------+--------------------------------------+
| Field         | Value                                |
+---------------+--------------------------------------+
| id            | 5e9308f7-c3a9-4a94-a017-e1acb694c036 |
| server_id     | 9e7aae15-cabc-4489-a1b2-778915a78df2 |
| config_id     | 86d49e66-2f25-4cb1-b623-5ae87b01bb64 |
| creation_time | 2016-08-03T08:32:10                  |
| updated_time  |                                      |
| status        | IN_PROGRESS                          |
| status_reason | Deploy data available                |
| input_values  | {}                                   |
| action        | CREATE                               |
| output_values | None                                 |
+---------------+--------------------------------------+

[stack@mitaka-uc ~]$ openstack stack resource list 3797aec6-e543-4dda-9cd1-c7261e827a64
+-------------------------+--------------------------------------+-------------------------------------------------+-----------------+---------------------+
| resource_name           | physical_resource_id                 | resource_type                                   | resource_status | updated_time        |
+-------------------------+--------------------------------------+-------------------------------------------------+-----------------+---------------------+
| ComputeArtifactsConfig  | a33cd04d-61ab-4429-8565-182409c2b97f | file:///usr/share/openstack-tripleo-heat-       | CREATE_COMPLETE | 2016-08-03T08:29:19 |
|                         |                                      | templates/puppet/deploy-artifacts.yaml          |                 |                     |
| ComputePuppetConfig     | 5bb712b0-5358-46c7-a444-f9adedfedd50 | OS::Heat::SoftwareConfig                        | CREATE_COMPLETE | 2016-08-03T08:29:19 |
| ComputePuppetDeployment | 8b199f85-e4f9-48ad-9aee-b1cdf4900b9f | OS::Heat::StructuredDeployments                 | CREATE_FAILED   | 2016-08-03T08:29:19 |
| ComputeArtifactsDeploy  | 1d13bf34-fc66-4bf1-a3b7-1dd815f58f5a | OS::Heat::StructuredDeployments                 | CREATE_COMPLETE | 2016-08-03T08:29:19 |
| ExtraConfig             |                                      | OS::TripleO::NodeExtraConfigPost                | INIT_COMPLETE   | 2016-08-03T08:29:19 |
+-------------------------+--------------------------------------+-------------------------------------------------+-----------------+---------------------+

[stack@mitaka-uc ~]$ openstack stack resource list 8b199f85-e4f9-48ad-9aee-b1cdf4900b9f
+---------------+--------------------------------------+--------------------------------+--------------------+---------------------+
| resource_name | physical_resource_id                 | resource_type                  | resource_status    | updated_time        |
+---------------+--------------------------------------+--------------------------------+--------------------+---------------------+
| 0             | 7cd0aa3d-742f-4e78-99ca-b2a575913f8e | OS::Heat::StructuredDeployment | CREATE_IN_PROGRESS | 2016-08-03T08:30:04 |
+---------------+--------------------------------------+--------------------------------+--------------------+---------------------+
[stack@mitaka-uc ~]$ openstack software deployment show 7cd0aa3d-742f-4e78-99ca-b2a575913f8e
+---------------+--------------------------------------+
| Field         | Value                                |
+---------------+--------------------------------------+
| id            | 7cd0aa3d-742f-4e78-99ca-b2a575913f8e |
| server_id     | c1ab52a9-461a-4a11-a13e-e57ff0a3ae2a |
| config_id     | 24e5c0db-f84f-4a94-8f8e-8e38e73ccc86 |
| creation_time | 2016-08-03T08:30:05                  |
| updated_time  |                                      |
| status        | IN_PROGRESS                          |
| status_reason | Deploy data available                |
| input_values  | {}                                   |
| action        | CREATE                               |
+---------------+--------------------------------------+

Keystonerc file was not generated. Please find below openstack status command result on controller and compute.

[heat-admin@overcloud-controller-0 ~]$ openstack-status
== Nova services ==
openstack-nova-api:                     active
openstack-nova-compute:                 inactive  (disabled on boot)
openstack-nova-network:                 inactive  (disabled on boot)
openstack-nova-scheduler:               activating(disabled on boot)
openstack-nova-cert:                    active
openstack-nova-conductor:               active
openstack-nova-console:                 inactive  (disabled on boot)
openstack-nova-consoleauth:             active
openstack-nova-xvpvncproxy:             inactive  (disabled on boot)
== Glance services ==
openstack-glance-api:                   active
openstack-glance-registry:              active
== Keystone service ==
openstack-keystone:                     inactive  (disabled on boot)
== Horizon service ==
openstack-dashboard:                    uncontactable
== neutron services ==
neutron-server:                         failed    (disabled on boot)
neutron-dhcp-agent:                     inactive  (disabled on boot)
neutron-l3-agent:                       inactive  (disabled on boot)
neutron-metadata-agent:                 inactive  (disabled on boot)
neutron-lbaas-agent:                    inactive  (disabled on boot)
neutron-openvswitch-agent:              inactive  (disabled on boot)
neutron-metering-agent:                 inactive  (disabled on boot)
== Swift services ==
openstack-swift-proxy:                  active
openstack-swift-account:                active
openstack-swift-container:              active
openstack-swift-object:                 active
== Cinder services ==
openstack-cinder-api:                   active
openstack-cinder-scheduler:             active
openstack-cinder-volume:                active
openstack-cinder-backup:                inactive  (disabled on boot)
== Ceilometer services ==
openstack-ceilometer-api:               active
openstack-ceilometer-central:           active
openstack-ceilometer-compute:           inactive  (disabled on boot)
openstack-ceilometer-collector:         active
openstack-ceilometer-notification:      active
== Heat services ==
openstack-heat-api:                     inactive  (disabled on boot)
openstack-heat-api-cfn:                 active
openstack-heat-api-cloudwatch:          inactive  (disabled on boot)
openstack-heat-engine:                  inactive  (disabled on boot)
== Sahara services ==
openstack-sahara-api:                   active
openstack-sahara-engine:                active
== Support services ==
libvirtd:                               active
openvswitch:                            active
dbus:                                   active
target:                                 active
rabbitmq-server:                        active
memcached:                              active

[heat-admin@overcloud-novacompute-0 ~]$ openstack-status
== Nova services ==
openstack-nova-api:                     inactive  (disabled on boot)
openstack-nova-compute:                 activating(disabled on boot)
openstack-nova-network:                 inactive  (disabled on boot)
openstack-nova-scheduler:               inactive  (disabled on boot)
openstack-nova-cert:                    inactive  (disabled on boot)
openstack-nova-conductor:               inactive  (disabled on boot)
openstack-nova-console:                 inactive  (disabled on boot)
openstack-nova-consoleauth:             inactive  (disabled on boot)
openstack-nova-xvpvncproxy:             inactive  (disabled on boot)
== Glance services ==
openstack-glance-api:                   inactive  (disabled on boot)
openstack-glance-registry:              inactive  (disabled on boot)
== Keystone service ==
openstack-keystone:                     inactive  (disabled on boot)
== Horizon service ==
openstack-dashboard:                    uncontactable
== neutron services ==
neutron-server:                         inactive  (disabled on boot)
neutron-dhcp-agent:                     inactive  (disabled on boot)
neutron-l3-agent:                       inactive  (disabled on boot)
neutron-metadata-agent:                 inactive  (disabled on boot)
neutron-lbaas-agent:                    inactive  (disabled on boot)
neutron-openvswitch-agent:              active
neutron-metering-agent:                 inactive  (disabled on boot)
== Swift services ==
openstack-swift-proxy:                  inactive  (disabled on boot)
openstack-swift-account:                inactive  (disabled on boot)
openstack-swift-container:              inactive  (disabled on boot)
openstack-swift-object:                 inactive  (disabled on boot)
== Cinder services ==
openstack-cinder-api:                   inactive  (disabled on boot)
openstack-cinder-scheduler:             inactive  (disabled on boot)
openstack-cinder-volume:                inactive  (disabled on boot)
openstack-cinder-backup:                inactive  (disabled on boot)
== Ceilometer services ==
openstack-ceilometer-api:               inactive  (disabled on boot)
openstack-ceilometer-central:           inactive  (disabled on boot)
openstack-ceilometer-compute:           inactive  (disabled on boot)
openstack-ceilometer-collector:         inactive  (disabled on boot)
openstack-ceilometer-notification:      inactive  (disabled on boot)
== Heat services ==
openstack-heat-api:                     inactive  (disabled on boot)
openstack-heat-api-cfn:                 inactive  (disabled on boot)
openstack-heat-api-cloudwatch:          inactive  (disabled on boot)
openstack-heat-engine:                  inactive  (disabled on boot)
== Sahara services ==
openstack-sahara-all:                   inactive  (disabled on boot)
== Support services ==
libvirtd:                               active
openvswitch:                            active
dbus:                                   active
rabbitmq-server:                        inactive  (disabled on boot)
memcached:                              inactive  (disabled on boot)

Please let me know if there is any other logs which I can provide that can help in troubleshooting.

Thanks a lot in Advance for your help and support.