Ask Your Question
0

Internal Server error during "nova image-list" request

asked 2014-05-30 10:52:53 -0500

Diego Lima gravatar image

updated 2014-05-30 12:31:11 -0500

Hi! I've got an openstack environment (Havana on Ubuntu 12.04) that has been working for some time and now I can't list images using "nova image-list". Using "glance image-list" or "glance index" works just fine.

Nova, however, returns an internal server error message:

# nova image-list
ERROR: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-b3bca21a-0f98-4796-bde2-c7ed555aed68)

I've managed to track the error to Glance Registry, which seems to be rolling back a transaction after receiving an error from the database (further details regarding the query on the pastebin file at the end of the post):

2014-05-30 12:37:32.880 18921 INFO sqlalchemy.engine.base.Engine [-] ROLLBACK

On the database logs (I'm using Postgresql 9.3) I get the following error message:

2014-05-30 12:41:47 BRT ERROR:  invalid input syntax for type timestamp: "" at character 4184

I can also get details about the image that seems to be causing the error using "glance image-show 45db5061-9f61-460a-b78d-386fd70e9b5c" (Details on pastebin) and on the database:

glance=# select * from images where id = '45db5061-9f61-460a-b78d-386fd70e9b5c';
45db5061-9f61-460a-b78d-386fd70e9b5c | 117_AmbariSlave | 3270443008 | active | f         | 2014-05-13 13:37:54.803997 | 2014-05-13 13:43:24.105433 |            | f       | qcow2       | bare             | 0b00c947551e073af5510c9f3dc8ab3f | cb581997b681459f88d9f8b01dc0ebcc |       40 |    4096 | f

Update:

Comparing this site with another one where everything is working fine I've noticed the request sent to the Nova API on the working one does not include the parameter marker:

/v1/images/detail?is_public=none&limit=20

On the site where "nova image-list" doesn't work it includes a marker parameter on the request:

/v1/images/detail?marker=e869aa20-4aac-492e-b2f7-6a021d941d8e&is_public=none&limit=20

While this is obviously causing the problem (manually repeating the request without the marker works), I have no idea as to where this marker parameter is coming from.

Log Files:

Postgresql Error Log: http://pastebin.com/bcZUhmBN

Glance Registry Log: http://pastebin.com/93Xyhjn0

Nova API Log: http://pastebin.com/JczzDubw

Image details: http://pastebin.com/zxN4Q7bz

edit retag flag offensive close merge delete

Comments

Update: I've compared this environment with another site that I'm running, where everything is working fine. I've noticed that the request sent to nova-api on the functioning environment doesn't include the "marker" parameter.

Diego Lima gravatar imageDiego Lima ( 2014-05-30 12:26:56 -0500 )edit

2 answers

Sort by ยป oldest newest most voted
1

answered 2014-05-30 12:46:21 -0500

Diego Lima gravatar image

Answering myself so if anyone run into the same problem they won't be at a loss:

I've managed to fix the problem by upgrading the nova packages to the latest version (1:2013.2.3-0ubuntu1~cloud0) from the repository.

edit flag offensive delete link more
0

answered 2014-06-04 14:52:35 -0500

Hello,

I'm currently getting the same exact problem as yours and found in some other mailing list that it was a bug that was already fixed in the package you mentioned above. I manage to upgrade all nova packages in my controller node (where glance resides) but it did not resolve the issue. I'm wondering if you needed to upgrade the nova packages across all the compute node as well.

Thank you.

edit flag offensive delete link more

Comments

Hi! I believe you need to upgrade your glance packages to the latest version as well. I took the oportunity to upgrade all services on my management node (including glance-api and glance-registry).

I did not upgrade any packages on any other servers (compute nor dedicated network nodes)

Diego Lima gravatar imageDiego Lima ( 2014-06-05 20:38:51 -0500 )edit

Thanks Diego, I'll try that and post back results.

enarciso gravatar imageenarciso ( 2014-06-06 11:18:55 -0500 )edit

upgrading Glace's packages solve my issue.

Thank you

enarciso gravatar imageenarciso ( 2014-06-06 19:36:38 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

2 followers

Stats

Asked: 2014-05-30 10:52:53 -0500

Seen: 671 times

Last updated: Jun 04 '14