Understanding caching of imaged in openstack glance

asked 2020-05-29 09:36:02 -0500

Taslim gravatar image
 Hi everyone,

I am understanding caching of glance imgs in a multinode environment while I am setting up the env

Openstack set up:

Openstack version:5.2.0

Deployed as openstack-helm(Openstack deployed on K8S nodes)-All the services are running as PODS on controller 
node.There is 1 Controller node and 2 compute nodes.

Controller node glance API is configured for image caching i.e

middleware is configured which was done by default under 
paste.filter_factory = glance.api.middleware.cache_manage:CacheManageFilter.factory
flavor is set as keystone+cachemanagement 
and image_cache_max_size and image_cache_dir are also defined

And I created an Instance

And when I executed glance-cache-manage list-cached list on the controller node
it does list cached Images in the below format

Found 1 cached images...
|                  ID                  |    Last Accessed (UTC)     |    Last Modified (UTC)     |      Size | Hits |

My Question:

When does the column Hits(int) gets incremented?--Because Hits value changed from 0 to 1 and 
doesnt change further even though I have created multiple instances on the 2 compute nodes using this same Image 
which is cached. 

 1. Is it because the images are cached
    on both the compute nodes and
    nova-compute doesn't hit the glance
    from the 3rd  instance creation?
        - if its 'yes' what is the location where the image is cached
          on the compute node.
       - And if 'no' and the flow is Nova-compute -->Glance api-->Glance cache
        why is the Hits not getting increased? Did I miss any config?

Please correct me,if my understanding of caching is bad.

I sincerely thank for the time and patience.
edit retag flag offensive close merge delete



I think that the images on the compute nodes are used. By default, you find a copy of the image under /var/lib/nova, and each instance has a qcow2 copy of that.

What exactly happens depends on the image type and how ephemeral storage is configured. Try raw images instead of qcow2, or LVM storage.

Bernd Bausch gravatar imageBernd Bausch ( 2020-05-30 03:47:39 -0500 )edit

Or launch instances from images, but with volumes instead of ephemeral storage (Horizon's default). That should also lead to a hit count increase.

Bernd Bausch gravatar imageBernd Bausch ( 2020-05-30 03:48:22 -0500 )edit

Yes,the imgs are cached under /var/lib/nova/ in compute node,& for further instance creation nova-compute doesn't hit Glance but uses its cached img in /var/lib/nova,That is the reason hit count is not getting increased.Is caching of imgs in compute node disabled by cache_image=none in nova.conf?

Taslim gravatar imageTaslim ( 2020-05-31 06:36:18 -0500 )edit

I don't know.

Bernd Bausch gravatar imageBernd Bausch ( 2020-05-31 12:39:27 -0500 )edit