Ask Your Question
0

Permission denied for cinder backed by NFS

asked 2015-12-03 00:18:58 -0500

t.goto gravatar image

updated 2015-12-03 02:39:48 -0500

Hello,

I setup Kilo OpenStack with RDO package and my cinder-volume is backed with NFS.

When I create a volume, I got a "permission denied" error in /var/log/cinder/volume.log.

  (snip)
2015-12-03 14:59:36.680 4193 TRACE oslo_messaging.rpc.dispatcher ProcessExecutionError: Unexpected error while running command.
2015-12-03 14:59:36.680 4193 TRACE oslo_messaging.rpc.dispatcher Command: truncate -s 1G /var/lib/cinder/mnt/07e2e40cd3318d2c3b7272887a58cd60/volume-0e1a4b72-eda6-4b58-
9ce5-49c913088f48
2015-12-03 14:59:36.680 4193 TRACE oslo_messaging.rpc.dispatcher Exit code: 1
2015-12-03 14:59:36.680 4193 TRACE oslo_messaging.rpc.dispatcher Stdout: u''
2015-12-03 14:59:36.680 4193 TRACE oslo_messaging.rpc.dispatcher Stderr: 'truncate: cannot open '/var/lib/cinder/mnt/07e2e40cd3318d2c3b7272887a58cd60/volume-0e1a4b72-eda6-4b58-9ce5-49c913088f48' for writing: Permission denied\n'

Then I dug in deeper and found I could successfully create a volume MANUALLY using cinder-rootwrap. So I think NFS setting is correct.

sudo cinder-rootwrap /etc/cinder/rootwrap.conf \
 truncate -s 1G /var/lib/cinder/mnt/07e2e40cd3318d2c3b7272887a58cd60/volume-0e1a4b72-eda6-4b58-9ce5-49c913088f48

Did anybody encountered such error?


(added)

I tried chmod the NFS export directory so that others can write to it.I can successfully create a volume now.

# before
rwxr-xr-x    2 root root 4096 12月  3 12:15 test1_cinder
# after
rwxr-xrwx    2 root root 4096 12月  3 12:15 test1_cinder

But I don't think it is an intended behavior. Because I could create a volume on 755 NFS directory in Juno.


I have found some disparity between Juno and Kilo.

when you create a volume in Juno, cinder-volume log looks like this.

2015-12-03 17:22:52.981 14443 DEBUG cinder.openstack.common.processutils [req-1f2d05 - -] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf truncate -s 1G /var/lib/cinder/mnt/dc4db500da4856a90baf5fc91774dcf0/volume-
10628be4-1e45-4fd9-87e7-4ede3134ba80 execute /usr/lib/python2.7/site-packages/cinder/openstack/common/processutils.py:158

However in Kilo, rootwrap is not used!

2015-12-03 17:26:04.098 11897 DEBUG oslo_concurrency.processutils [req] CMD "truncate -s 1G /var/lib/cinder/mnt/07e2e40cd3318d2c3b7272887a58cd60/volume-393a112a-7ffb-402f-94cc-6a505899f395" returned: 0 in 0.009s execute /usr/li
b/python2.7/site-packages/oslo_concurrency/processutils.py:225

maybe that's why chmod 777 is needed in Kilo, and not in Juno..

edit retag flag offensive close merge delete

2 answers

Sort by » oldest newest most voted
1

answered 2015-12-03 00:36:52 -0500

dbaxps gravatar image

updated 2015-12-03 02:06:09 -0500

UPDATE
Per https://kimizhang.wordpress.com/2015/...
RHEL7 + Juno

mkdir /nfsshare; chmod 777 /nfsshare
mkdir /nfsshare_glance; chmod 777 /nfsshare_glance
mkdir /nfsshare_cinder; chmod 777 /nfsshare_cinder

However, I remember very well it's not needed on RDO Liberty
END UPDATE

Make sure commands like bellow have been run :-

# Choose whatever NFS share is used
Check directory /volume/openstack/cinder  is exported properly on NFS_SERVER
cat > /etc/cinder/nfs_exports << EOF
NFS_SERVER_IP:/volume/openstack/cinder 
EOF

chown root:cinder /etc/cinder/nfs_exports
chmod 0640 /etc/cinder/nfs_exports
openstack-config --set /etc/cinder/cinder.conf DEFAULT nfs_shares_config /etc/cinder/nfs_exports
openstack-config --set /etc/cinder/cinder.conf DEFAULT nfs_sparsed_volumes true
openstack-config --set /etc/cinder/cinder.conf DEFAULT nfs_mount_options v3
openstack-config --set /etc/cinder/cinder.conf DEFAULT volume_driver cinder.volume.drivers.nfs.NfsDriver
edit flag offensive delete link more

Comments

Thank you for your quick reply, dpaxps.

I checked above configs you mentioned, but still got identical error.

on my NFS_SERVER

# cat /etc/exports
/test1_cinder 192.168.0.0/16(rw,no_root_squash)

from cinder-volume

# showmount -e <NFS_SERVER>
/test1_cinder   192.168.0.0/16
t.goto gravatar imaget.goto ( 2015-12-03 01:24:48 -0500 )edit

owner/group of nfs_exports and options in cinder.conf are properly set, but still got same error.

t.goto gravatar imaget.goto ( 2015-12-03 01:25:59 -0500 )edit

Can you just mount remote directory from Cinder Host ?

dbaxps gravatar imagedbaxps ( 2015-12-03 01:44:42 -0500 )edit

Yes. I can mount remote dir from cinder host.

t.goto gravatar imaget.goto ( 2015-12-03 01:48:30 -0500 )edit

Check also one more time with
https://kimizhang.wordpress.com/2015/...

dbaxps gravatar imagedbaxps ( 2015-12-03 01:57:09 -0500 )edit
0

answered 2015-12-03 20:59:32 -0500

t.goto gravatar image

updated 2015-12-03 21:38:46 -0500

(edited message to add another option)

Another solution.

Make NAS operations run as root so that cinder volume files are created as root.

If you want to use NFS export directory in mode 755, following settings in cinder.conf for cinder-volume work.

nas_secure_file_operations=False
nas_option=False

https://blueprints.launchpad.net/cind...

https://bugs.launchpad.net/openstack-...

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2015-12-03 00:18:58 -0500

Seen: 2,503 times

Last updated: Dec 03 '15