Ask Your Question
0

What happens if you delete /var/lib/nova/instances/_base/* ?

asked 2018-03-22 16:25:57 -0500

Wajahat gravatar image

I am trying to see the migration time and rebuild time differences when the images are not cached on the compute node, and I was wondering if it is safe to delete the contents of /var/lib/nova/instances/_base/ .

I tried doing it and it did not mess up the instance that was running on my cluster, and I rebooted the VM and it came back up just fine. And the image was not re-downloaded from glance to _base folder on the compute node.

Also, when exactly are the images in _base used ? (only for rebuild/migration ?)

edit retag flag offensive close merge delete
add a comment

1 answer

Sort by ยป oldest newest most voted
1

answered 2018-03-22 20:49:44 -0500

Bernd Bausch gravatar image

updated 2018-03-22 21:32:31 -0500

Instances' ephemeral storage is normally under /var/lib/nova/instances/INSTANCE_ID. I say "normally" because this directory, and practically all other details, can be configured.

If use_cow_images is configured, and images_type is qcow2, the files under the INSTANCE_ID directory are COW images based on files under the _base directory. You can verify that with the command qemu-img info --force-share FILENAME.qcow2.

Why does deleting a _base file have no effect? I removed the _base file and ran openstack server reboot. Let's have a look at the Nova Compute log.

Mar 23 11:17:39 devstack-compute nova-compute[1108]: : libvirtError: Cannot access backing file '/opt/stack/data/nova/instances/_base/fb9365317d7a6f6afbc6eb9f5ad40d5d0f4d8307' of storage file '/opt/stack/data/nova/instances/ef3367f5-6908-4099-b5c0-d6fffb7784c9/disk' (as uid:64055, gid:120): No such file or directory
Mar 23 11:17:39 devstack-compute nova-compute[1108]: DEBUG nova.virt.libvirt.driver [None req-0d26fbe2-d936-4f54-b4e1-18b29f9833a6 demo demo] [instance: ef3367f5-6908-4099-b5c0-d6fffb7784c9] Instance soft reboot failed: Cannot access backing file '/opt/stack/data/nova/instances/_base/fb9365317d7a6f6afbc6eb9f5ad40d5d0f4d8307' of storage file '/opt/stack/data/nova/instances/ef3367f5-6908-4099-b5c0-d6fffb7784c9/disk' (as uid:64055, gid:120): No such file or directory {{(pid=1108) reboot /opt/stack/nova/nova/virt/libvirt/driver.py:2632}}
Mar 23 11:17:39 devstack-compute nova-compute[1108]: WARNING nova.virt.libvirt.driver [None req-0d26fbe2-d936-4f54-b4e1-18b29f9833a6 demo demo] [instance: ef3367f5-6908-4099-b5c0-d6fffb7784c9] Failed to soft reboot instance. Trying hard reboot.: libvirtError: Cannot access backing file '/opt/stack/data/nova/instances/_base/fb9365317d7a6f6afbc6eb9f5ad40d5d0f4d8307' of storage file '/opt/stack/data/nova/instances/ef3367f5-6908-4099-b5c0-d6fffb7784c9/disk' (as uid:64055, gid:120): No such file or directory
Mar 23 11:17:39 devstack-compute nova-compute[1108]: INFO nova.virt.libvirt.driver [None req-0d26fbe2-d936-4f54-b4e1-18b29f9833a6 demo demo] [instance: ef3367f5-6908-4099-b5c0-d6fffb7784c9] Instance destroyed successfully.

At this point, Nova basically recreates the instance but keeps all its attributes like UUID, ephemeral storage and IP address.

Not sure what happens when you run reboot inside the instance. My guess is that the same qemu process keeps running, and that the "deleted" _base file still exists, although it has no directory entry anymore.

Or your instances use volume storage instead of ephemeral storage. This is the default when launching instances from Horizon.

In any case, I see no reason for deleting files under _base, except if you know that they are left-overs that won't be used anymore.

edit flag offensive delete link more

Comments

So in openstack if I wanted to disable caching of images for rebuild and migrate and force it to copy the image from glance everytime, what would I have to do instead of deleting contents of _base directory?

Wajahat gravatar imageWajahat ( 2018-03-22 21:29:01 -0500 )edit

I am not 100% sure, but try disabling the use_cow_images setting. This has the possibly undesired side effect that instances don't share disk space on a compute node. Or use a different images_type format, such as flat.

Bernd Bausch gravatar imageBernd Bausch ( 2018-03-22 21:36:02 -0500 )edit

Check the libvirt config options, and use the KVM config guide (assuming your hypervisor is KVM).

Bernd Bausch gravatar imageBernd Bausch ( 2018-03-22 21:37:32 -0500 )edit

Note that I updated my answer considerably after your comment.

Bernd Bausch gravatar imageBernd Bausch ( 2018-03-22 21:38:56 -0500 )edit

Interesting, after the example you gave I did the same thing to try out how it works on my system. I am using Mitaka release, and when I deleted the _base image and did openstack server reboot it did not give me an error, instead it just created/copied a new _base image from glance I assume.

Wajahat gravatar imageWajahat ( 2018-03-24 02:15:30 -0500 )edit

I tried it on Queens. But did you check the Nova Compute log?

Bernd Bausch gravatar imageBernd Bausch ( 2018-03-24 07:48:45 -0500 )edit

The compute log contains the entries that your log contained. But what does this mean? When I tried this after dumping some data onto the instance, and deleting _base, and then doing a cold-migrate to another host; the data was preserved which I thought would not have.

Wajahat gravatar imageWajahat ( 2018-03-26 16:00:04 -0500 )edit
see more comments

Get to know Ask OpenStack

Resources for moderators

Question Tools

2 followers

subscribe to rss feed

Stats

Asked: 2018-03-22 16:25:57 -0500

Seen: 2,412 times

Last updated: Mar 22 '18

Related questions

Return VM to the initial state using HEAT

Caching images from glance

how to use a disk, kernel and ramdisk from /var/lib/nova/instances

how to convert ami/aki/ari image to qcow2?

Why does Keystone not reuse tokens?

Is it feasibility to add SSD Cache to a volume?

multiple glance server for caching

How memcached helps keystone

"qcow2" image with Openstack in Multi Node System

snapshot of ubuntu-virtual machine