Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

gluster volume for nova instance

Description of problem: I configured glusterfs volume for nova instances in openstack havana version. Anyone who configured properly glusterfs volumes for nova instance?

case 1. local volume ( /var/lib/nova/instances ) If I launch instance, works well. case 2. 2 replica glusterfs volume ( /mnt/glusterfs/nova ) If I launch instance, I/O error occurs.

block I/O error in device 'drive-virtio-disk0': Bad file descriptor (9) block I/O error in device 'drive-virtio-disk0': Bad file descriptor (9) block I/O error in device 'drive-virtio-disk0': Bad file descriptor (9) block I/O error in device 'drive-virtio-disk0': Bad file descriptor (9) block I/O error in device 'drive-virtio-disk0': Bad file descriptor (9) block I/O error in device 'drive-virtio-disk0': Bad file descriptor (9)

gluster> volume info NOVA

Volume Name: NOVA Type: Replicate Volume ID: 4933cbf2-b3eb-4e64-b9fc-76195d50a3b5 Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: 192.168.100.231:/data1/nova Brick2: 192.168.100.233:/data1/nova Options Reconfigured: auth.allow: 192.168.100.*

[root@controller glusterfs]# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda2 41284928 28684504 10503272 74% / tmpfs 16281956 0 16281956 0% /dev/shm /dev/sda1 198337 63002 125095 34% /boot /dev/sda5 30962748 21539420 7850416 74% /var/www/html /dev/mapper/vg_gluster-lv_gluster1 1952559608 32137812 1920421796 2% /data1 192.168.100.231:/GLANCE 1952559488 32137728 1920421760 2% /mnt/gluster/glance 192.168.100.231:/NOVA 1952559488 32137728 1920421760 2% /mnt/gluster/nova 192.168.100.231:/CINDER 1952559488 32137728 1920421760 2% /var/lib/cinder/mnt/08b7742d46137bb49d1a7d9e6b993488

Actual results:

I can't access instance because it produced I/O error.

Additional info: In my logs, displayed messages

/var/log/glusterfs/mnt-gluster-nova.log [2014-03-17 03:17:45.247599] W [fuse-bridge.c:648:fuse_truncate_cbk] 0-glusterfs-fuse: 2526594: FTRUNCATE() ERR => -1 (Bad file descriptor) [2014-03-17 03:17:46.081142] W [fuse-bridge.c:2167:fuse_writev_cbk] 0-glusterfs-fuse: 2526914: WRITE => -1 (Bad file descriptor) [2014-03-17 03:37:49.608837] W [socket.c:514:__socket_rwv] 0-NOVA-client-1: readv failed (No data available)

In my instances log. Remounting root filesystem in read-write mode: Buffer I/O error on device dm-0, logical block 0 [ OK ] Buffer I/O error on device dm-0, logical block 3702785 Buffer I/O error on device dm-0, logical block 3703882 Buffer I/O error on device dm-0, logical block 3703887 Buffer I/O error on device dm-0, logical block 3709714 Mounting local filesystems: Buffer I/O error on device dm-0, logical block 3711039 end_request: I/O error, dev vda, sector 2050 [ OK ]end_request: I/O error, dev vda, sector 30831792

click to hide/show revision 2
No.2 Revision

gluster volume for nova instance

Description of problem: I configured glusterfs volume for nova instances in openstack havana version. Anyone who configured properly glusterfs volumes for nova instance?

case 1. local volume ( /var/lib/nova/instances ) If I launch instance, works well. case 2. 2 replica glusterfs volume ( /mnt/glusterfs/nova ) If I launch instance, I/O error occurs.

block I/O error in device 'drive-virtio-disk0': Bad file descriptor (9)
block I/O error in device 'drive-virtio-disk0': Bad file descriptor (9)
block I/O error in device 'drive-virtio-disk0': Bad file descriptor (9)
block I/O error in device 'drive-virtio-disk0': Bad file descriptor (9)
block I/O error in device 'drive-virtio-disk0': Bad file descriptor (9)
block I/O error in device 'drive-virtio-disk0': Bad file descriptor (9)

(9) gluster> volume info NOVA

NOVA Volume Name: NOVA Type: Replicate Volume ID: 4933cbf2-b3eb-4e64-b9fc-76195d50a3b5 Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: 192.168.100.231:/data1/nova Brick2: 192.168.100.233:/data1/nova Options Reconfigured: auth.allow: 192.168.100.*

192.168.100.* [root@controller glusterfs]# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda2 41284928 28684504 10503272 74% / tmpfs 16281956 0 16281956 0% /dev/shm /dev/sda1 198337 63002 125095 34% /boot /dev/sda5 30962748 21539420 7850416 74% /var/www/html /dev/mapper/vg_gluster-lv_gluster1 1952559608 32137812 1920421796 2% /data1 192.168.100.231:/GLANCE 1952559488 32137728 1920421760 2% /mnt/gluster/glance 192.168.100.231:/NOVA 1952559488 32137728 1920421760 2% /mnt/gluster/nova 192.168.100.231:/CINDER 1952559488 32137728 1920421760 2% /var/lib/cinder/mnt/08b7742d46137bb49d1a7d9e6b993488

/var/lib/cinder/mnt/08b7742d46137bb49d1a7d9e6b993488

Actual results:

I can't access instance because it produced I/O error.

Additional info: In my logs, displayed messages

/var/log/glusterfs/mnt-gluster-nova.log
[2014-03-17 03:17:45.247599] W [fuse-bridge.c:648:fuse_truncate_cbk] 0-glusterfs-fuse: 2526594: FTRUNCATE() ERR => -1 (Bad file descriptor)
[2014-03-17 03:17:46.081142] W [fuse-bridge.c:2167:fuse_writev_cbk] 0-glusterfs-fuse: 2526914: WRITE => -1 (Bad file descriptor)
[2014-03-17 03:37:49.608837] W [socket.c:514:__socket_rwv] 0-NOVA-client-1: readv failed (No data available)

available)

In my instances log. log:

  Remounting root filesystem in read-write mode:  Buffer I/O error on device dm-0, logical block 0
[  OK  ]
Buffer I/O error on device dm-0, logical block 3702785
Buffer I/O error on device dm-0, logical block 3703882
Buffer I/O error on device dm-0, logical block 3703887
Buffer I/O error on device dm-0, logical block 3709714
Mounting local filesystems:  Buffer I/O error on device dm-0, logical block 3711039
end_request: I/O error, dev vda, sector 2050
[  OK  ]end_request: I/O error, dev vda, sector 30831792

30831792