nova-compute reporting XXX

asked 2012-11-27 20:10:49 -0600

jacobgodin gravatar image

I have a few nodes setup in a development env. to test an OpenStack deployment. Currently, I'm trying to get the cloud controller to communicate with the compute node. The setup is as follows:

controller1 - OpenStack services: openstack-glance-api openstack-glance-registry openstack-keystone openstack-nova-api openstack-nova-cert openstack-nova-console openstack-nova-consoleauth openstack-nova-metadata-api openstack-nova-network openstack-nova-objectstore openstack-nova-scheduler openstack-nova-volume openstack-nova-xvpvncproxy

compute1 - OpenStack services: openstack-nova-compute

compute1 is running qemu via libvirt and appears to be able to connect properly: 2012-11-27 14:01:03 DEBUG nova.utils [req-e73569ba-9a7e-48e0-b955-bee3defcbb7f None None] backend <module 'nova.db.sqlalchemy.api'="" from="" '="" usr="" lib="" python2.6="" site-packages="" nova="" db="" sqlalchemy="" api.pyc'=""> from (pid=4344) __get_backend /usr/lib/python2.6/site-packages/nova/utils.py:494 2012-11-27 14:01:04 DEBUG nova.utils [-] Got semaphore "compute_resources" for method "update_available_resource"... from (pid=4344) inner /usr/lib/python2.6/site-packages/nova/utils.py:713 2012-11-27 14:01:06 DEBUG nova.compute.resource_tracker [-] Hypervisor: free ram (MB): 3652 from (pid=4344) _report_hypervisor_resource_view /usr/lib/python2.6/site-packages/nova/compute/resource_tracker.py:447 2012-11-27 14:01:06 DEBUG nova.compute.resource_tracker [-] Hypervisor: free disk (GB): 5 from (pid=4344) _report_hypervisor_resource_view /usr/lib/python2.6/site-packages/nova/compute/resource_tracker.py:448 2012-11-27 14:01:06 DEBUG nova.compute.resource_tracker [-] Hypervisor: free VCPUs: 2 from (pid=4344) _report_hypervisor_resource_view /usr/lib/python2.6/site-packages/nova/compute/resource_tracker.py:453 2012-11-27 14:01:06 AUDIT nova.compute.resource_tracker [-] Free ram (MB): 3321 2012-11-27 14:01:06 AUDIT nova.compute.resource_tracker [-] Free disk (GB): 7 2012-11-27 14:01:06 AUDIT nova.compute.resource_tracker [-] Free VCPUS: 2 2012-11-27 14:01:06 INFO nova.compute.resource_tracker [-] Compute_service record updated for compute1

compute1 is also able to connect to the DB on controller1: [root@compute1 nova]# cat nova.conf | grep sql sql_connection = mysql://nova:nova@10.0.0.10/nova [root@compute1 nova]# mysql -u nova -pnova -h 10.0.0.10 -e "SHOW tables" nova +-------------------------------------+ | Tables_in_nova | +-------------------------------------+ | agent_builds | | aggregate_hosts | | aggregate_metadata | | aggregates | | block_device_mapping | | bw_usage_cache | | cells | | certificates | | compute_node_stats | | compute_nodes | | console_pools | | consoles | | dns_domains | ....

controller1 is not receiving status updates from compute1: [root@controller1 nova]# nova-manage service list 2012-11-27 16:08:42 DEBUG nova.utils [req-16c46196-f79b-4d5a-a9f1-0b3c2f1f253c None None] backend <module 'nova.db.sqlalchemy.api'="" from="" '="" usr="" lib="" python2.6="" site-packages="" nova="" db="" sqlalchemy="" api.pyc'=""> from (pid=30052) __get_backend /usr/lib/python2.6/site-packages/nova/utils.py:494 Binary Host Zone Status State Updated_At nova-compute compute1 nova enabled XXX 2012-11-27 17:41:33 nova-network controller1 nova enabled :-) 2012-11-27 20:08:41 nova-scheduler controller1 nova enabled :-) 2012-11-27 20:08:38 nova-volume controller1 nova enabled :-) 2012-11-27 20:08:37 nova-console controller1 nova enabled :-) 2012-11-27 20:08:38 [root@controller1 nova]# mysql -u nova -pnova -e "select * from services" nova +---------------------+---------------------+------------+---------+----+----------------------------------------+----------------+-----------+--------------+----------+-------------------+ | created_at | updated_at | deleted_at | deleted | id | host | binary | topic | report_count | disabled | availability_zone | +---------------------+---------------------+------------+---------+----+----------------------------------------+----------------+-----------+--------------+----------+-------------------+ | 2012-11-22 12:35:57 | 2012-11-27 17:41:33 | NULL | 0 | 1 | compute1 | nova-compute | compute | 0 | 0 | nova | | 2012-11-22 12:38:30 | 2012-11-27 20:09:32 | NULL | 0 | 2 | controller1 | nova-network | network | 35723 | 0 | nova | | 2012-11-22 12:38:44 | 2012-11-27 20:09:28 | NULL | 0 | 3 | controller1 | nova-scheduler | scheduler | 35722 | 0 | nova | | 2012-11-23 16:44:26 ... (more)

edit retag flag offensive close merge delete

5 answers

Sort by ยป oldest newest most voted
0

answered 2012-11-27 20:11:42 -0600

jacobgodin gravatar image

Just a quick note that I'm running FOLSOM RPMs from EPEL

edit flag offensive delete link more
0

answered 2012-11-28 18:44:03 -0600

if you tail the compute log file, you should see periodic tasks firing every few minutes (for a variety of purposes).

Are you seeing this? Or, once launched, does the node go silent?

edit flag offensive delete link more
0

answered 2012-11-28 18:49:06 -0600

jacobgodin gravatar image

[root@compute1 nova]# ls -las total 120 4 drwxr-xr-x. 2 nova root 4096 Nov 28 03:24 . 4 drwxr-xr-x. 12 root root 4096 Nov 27 12:48 .. 0 -rw-r--r-- 1 nova nova 0 Nov 28 03:24 compute.log 64 -rw-r--r-- 1 nova nova 64539 Nov 28 03:24 compute.log-20121128.gz 4 -rw-r--r--. 1 root root 3217 Nov 27 13:37 nova-manage.log [root@compute1 nova]# date Wed Nov 28 14:48:40 AST 2012 [root@compute1 nova]# cat compute.log [root@compute1 nova]#

edit flag offensive delete link more
0

answered 2012-11-29 13:25:58 -0600

jacobgodin gravatar image

Also seeing the following new line if I run the compute service directly with nova-compute:

2012-11-29 09:22:20 AUDIT nova.compute.resource_tracker [-] Free disk (GB): 7 2012-11-29 09:22:20 AUDIT nova.compute.resource_tracker [-] Free VCPUS: 2 2012-11-29 09:22:20 INFO nova.compute.resource_tracker [-] Compute_service record updated for compute1.osc.hfx.dynamichosting.biz No handlers could be found for logger "qpid.messaging" <--------

Looks like maybe the compute node either doesn't see the scheduler on the controller or can't communicate with it?

edit flag offensive delete link more
0

answered 2012-11-29 16:17:58 -0600

jacobgodin gravatar image

This appears to have been an issue with using Apache QPID. I removed QPID from my scheduler and replaced it with RabbitMQ. Everything seems happy now :-)

[root@controller1 rabbitmq]# nova-manage service list 2012-11-29 12:16:24 DEBUG nova.utils [req-8423af36-02ab-480f-a658-a1363a6e45ae None None] backend <module 'nova.db.sqlalchemy.api'="" from="" '="" usr="" lib="" python2.6="" site-packages="" nova="" db="" sqlalchemy="" api.pyc'=""> from (pid=5789) __get_backend /usr/lib/python2.6/site-packages/nova/utils.py:494 Binary Host Zone Status State Updated_At nova-compute compute1 nova enabled :-) 2012-11-29 16:16:19 nova-network controller1 nova enabled :-) 2012-11-29 16:16:21 nova-scheduler controller1 nova enabled :-) 2012-11-29 16:16:25 nova-volume controller1 nova enabled :-) 2012-11-29 16:16:20 nova-console controller1 nova enabled :-) 2012-11-29 16:16:24

edit flag offensive delete link more

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2012-11-27 20:10:49 -0600

Seen: 168 times

Last updated: Nov 29 '12