Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Internal Server error during "nova image-list" request

Hi! I've got an openstack environment (Havana on Ubuntu 12.04) that has been working for some time and now I can't list images using "nova image-list". Using "glance image-list" or "glance index" works just fine.

Nova, however, returns an internal server error message:

# nova image-list
ERROR: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-b3bca21a-0f98-4796-bde2-c7ed555aed68)

I've managed to track the error to Glance Registry, which seems to be rolling back a transaction after receiving an error from the database (further details regarding the query on the pastebin file at the end of the post):

2014-05-30 12:37:32.880 18921 INFO sqlalchemy.engine.base.Engine [-] ROLLBACK

On the database logs (I'm using Postgresql 9.3) I get the following error message:

2014-05-30 12:41:47 BRT ERROR:  invalid input syntax for type timestamp: "" at character 4184

Log Files:

Postgresql Error Log: http://pastebin.com/bcZUhmBN

Glance Registry Log: http://pastebin.com/93Xyhjn0

Nova API Log: http://pastebin.com/JczzDubw

Internal Server error during "nova image-list" request

Hi! I've got an openstack environment (Havana on Ubuntu 12.04) that has been working for some time and now I can't list images using "nova image-list". Using "glance image-list" or "glance index" works just fine.

Nova, however, returns an internal server error message:

# nova image-list
ERROR: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-b3bca21a-0f98-4796-bde2-c7ed555aed68)

I've managed to track the error to Glance Registry, which seems to be rolling back a transaction after receiving an error from the database (further details regarding the query on the pastebin file at the end of the post):

2014-05-30 12:37:32.880 18921 INFO sqlalchemy.engine.base.Engine [-] ROLLBACK

On the database logs (I'm using Postgresql 9.3) I get the following error message:

2014-05-30 12:41:47 BRT ERROR:  invalid input syntax for type timestamp: "" at character 4184

I can also get details about the image that seems to be causing the error using "glance image-show 45db5061-9f61-460a-b78d-386fd70e9b5c" (Details on pastebin) and on the database:

glance=# select * from images where id = '45db5061-9f61-460a-b78d-386fd70e9b5c';
45db5061-9f61-460a-b78d-386fd70e9b5c | 117_AmbariSlave | 3270443008 | active | f         | 2014-05-13 13:37:54.803997 | 2014-05-13 13:43:24.105433 |            | f       | qcow2       | bare             | 0b00c947551e073af5510c9f3dc8ab3f | cb581997b681459f88d9f8b01dc0ebcc |       40 |    4096 | f

Log Files:

Postgresql Error Log: http://pastebin.com/bcZUhmBN

Glance Registry Log: http://pastebin.com/93Xyhjn0

Nova API Log: http://pastebin.com/JczzDubw

Image details: http://pastebin.com/zxN4Q7bz

Internal Server error during "nova image-list" request

Hi! I've got an openstack environment (Havana on Ubuntu 12.04) that has been working for some time and now I can't list images using "nova image-list". Using "glance image-list" or "glance index" works just fine.

Nova, however, returns an internal server error message:

# nova image-list
ERROR: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-b3bca21a-0f98-4796-bde2-c7ed555aed68)

I've managed to track the error to Glance Registry, which seems to be rolling back a transaction after receiving an error from the database (further details regarding the query on the pastebin file at the end of the post):

2014-05-30 12:37:32.880 18921 INFO sqlalchemy.engine.base.Engine [-] ROLLBACK

On the database logs (I'm using Postgresql 9.3) I get the following error message:

2014-05-30 12:41:47 BRT ERROR:  invalid input syntax for type timestamp: "" at character 4184

I can also get details about the image that seems to be causing the error using "glance image-show 45db5061-9f61-460a-b78d-386fd70e9b5c" (Details on pastebin) and on the database:

glance=# select * from images where id = '45db5061-9f61-460a-b78d-386fd70e9b5c';
45db5061-9f61-460a-b78d-386fd70e9b5c | 117_AmbariSlave | 3270443008 | active | f         | 2014-05-13 13:37:54.803997 | 2014-05-13 13:43:24.105433 |            | f       | qcow2       | bare             | 0b00c947551e073af5510c9f3dc8ab3f | cb581997b681459f88d9f8b01dc0ebcc |       40 |    4096 | f

Update:

Comparing this site with another one where everything is working fine I've noticed the request sent to the Nova API on the working one does not include the parameter marker:

/v1/images/detail?is_public=none&limit=20

On the site where "nova image-list" doesn't work it includes a marker parameter on the request:

/v1/images/detail?marker=e869aa20-4aac-492e-b2f7-6a021d941d8e&is_public=none&limit=20

While this is obviously causing the problem (manually repeating the request without the marker works), I have no idea as to where this marker parameter is coming from.

Log Files:

Postgresql Error Log: http://pastebin.com/bcZUhmBN

Glance Registry Log: http://pastebin.com/93Xyhjn0

Nova API Log: http://pastebin.com/JczzDubw

Image details: http://pastebin.com/zxN4Q7bz