Ask Your Question
0

instances and images on the same disk: cp instead of curl [closed]

asked 2013-12-25 08:45:50 -0500

scsnow gravatar image

Hi, I'm running Havana multinode. Instances and images are located on SAN attached shared disk (GPFS). Glance images need to be copied by "cp" instead of "curl" to nova's "_base" directory. Here is my configs:

/etc/glance/glance-api.conf

filesystem_store_datadir = /gpfs/images/
show_multiple_locations = True
filesystem_store_metadata_file = /etc/glance/gpfs.json

/etc/glance/gpfs.json

{
    "id": "b2b3229e-f22f-4af1-a809-fcf72afe8577",
    "mountpoint": "/gpfs"
}

/etc/nova/nova.conf

allowed_direct_url_schemes=file
filesystems=gpfs
[image_file_url:gpfs]
id=b2b3229e-f22f-4af1-a809-fcf72afe8577
mountpoint=/gpfs

Nova log on compute node

2013-12-25 17:29:15.512 10058 INFO nova.virt.libvirt.driver [req-af8fc341-bc26-4217-ba47-51a63d39a934 3cb68bbdc8bf499d82dee70392fe1c62 d1944f8305224f00b7f1faf72937f448] [instance: 4c18cdd0-c852-406
7-bef7-de0b3e6db82b] Creating image
2013-12-25 17:29:16.109 10058 ERROR nova.image.glance [req-af8fc341-bc26-4217-ba47-51a63d39a934 3cb68bbdc8bf499d82dee70392fe1c62 d1944f8305224f00b7f1faf72937f448] Unexpected error while running com
mand.
Command: cp /gpfs/images/e70e8713-b96b-4e6e-85a6-eda501889315 /var/lib/nova/instances/_base/bbc9f62419d817181cf3f8f72530133bc0a1172e.part
Exit code: 1
Stdout: ''
Stderr: "cp: cannot open `/gpfs/images/e70e8713-b96b-4e6e-85a6-eda501889315' for reading: Permission denied\n"
2013-12-25 17:29:16.109 10058 TRACE nova.image.glance Traceback (most recent call last):
2013-12-25 17:29:16.109 10058 TRACE nova.image.glance   File "/usr/lib/python2.6/site-packages/nova/image/glance.py", line 338, in download
2013-12-25 17:29:16.109 10058 TRACE nova.image.glance     xfer_mod.download(context, o, dst_path, loc_meta)
2013-12-25 17:29:16.109 10058 TRACE nova.image.glance   File "/usr/lib/python2.6/site-packages/nova/image/download/file.py", line 164, in download
2013-12-25 17:29:16.109 10058 TRACE nova.image.glance     lv_utils.copy_image(source_file, dst_file)
2013-12-25 17:29:16.109 10058 TRACE nova.image.glance   File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/utils.py", line 462, in copy_image
2013-12-25 17:29:16.109 10058 TRACE nova.image.glance     execute('cp', src, dest)
2013-12-25 17:29:16.109 10058 TRACE nova.image.glance   File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/utils.py", line 50, in execute
2013-12-25 17:29:16.109 10058 TRACE nova.image.glance     return utils.execute(*args, **kwargs)
2013-12-25 17:29:16.109 10058 TRACE nova.image.glance   File "/usr/lib/python2.6/site-packages/nova/utils.py", line 177, in execute
2013-12-25 17:29:16.109 10058 TRACE nova.image.glance     return processutils.execute(*cmd, **kwargs)
2013-12-25 17:29:16.109 10058 TRACE nova.image.glance   File "/usr/lib/python2.6/site-packages/nova/openstack/common/processutils.py", line 178, in execute
2013-12-25 17:29:16.109 10058 TRACE nova.image.glance     cmd=' '.join(cmd))
2013-12-25 17:29:16.109 10058 TRACE nova.image.glance ProcessExecutionError: Unexpected error while running command.
2013-12-25 17:29:16.109 10058 TRACE nova.image.glance Command: cp /gpfs/images/e70e8713-b96b-4e6e-85a6-eda501889315 /var/lib/nova/instances/_base/bbc9f62419d817181cf3f8f72530133bc0a1172e.part
2013-12-25 17:29:16.109 10058 TRACE nova.image.glance Exit code: 1
2013-12-25 17:29:16.109 10058 TRACE nova.image.glance Stdout: ''
2013-12-25 17:29:16.109 10058 TRACE nova.image.glance Stderr: "cp: cannot open `/gpfs/images/e70e8713-b96b-4e6e-85a6-eda501889315' for reading: Permission denied\n"

File permissions on image

-rw-r-----. 1 glance glance 10718478336 Dec 23 19:21 /gpfs/images/e70e8713-b96b-4e6e-85a6-eda501889315

I assume that compute service was trying ... (more)

edit retag flag offensive reopen merge delete

Closed for the following reason the question is answered, right answer was accepted by dheeru
close date 2014-01-02 22:08:31.267408

2 answers

Sort by ยป oldest newest most voted
3

answered 2013-12-25 10:01:21 -0500

dheeru gravatar image

updated 2013-12-25 10:06:51 -0500

It is clearly permission issue. This directory has only read/write perms for owner and read only permission for group. I suggest you try to change the permission as chmod 777 and then try you stuff. It should work.

also you can try manual copy i.e you can try executing the command from cli

edit flag offensive delete link more

Comments

Did you try anything ? Did this work ?

dheeru gravatar imagedheeru ( 2013-12-26 01:53:52 -0500 )edit

Trick with chmod 777 would definitely work. But every time I upload new image I have to go to that directory and change permissions. That's not good.

scsnow gravatar imagescsnow ( 2013-12-26 02:00:54 -0500 )edit

Agree with you. Issue is about the process is started by different user/group and directory is owned by different user and group. No way this would work unless both users get into same group. We have to make either of these to make it work. You can put the nova user in glance group as well. This will work irrespective anything.

dheeru gravatar imagedheeru ( 2013-12-26 02:08:21 -0500 )edit
0

answered 2014-01-02 15:49:34 -0500

scsnow gravatar image

Opened bug report for this issue: https://bugs.launchpad.net/bugs/1264302

edit flag offensive delete link more

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2013-12-25 08:45:50 -0500

Seen: 186 times

Last updated: Jan 02 '14