VMware: Clean up storage / image cache

asked 2014-01-16 06:06:51 -0500

Mathias Ewald gravatar image

Hi, I run my environment on ESXi 5.1 using VMwareESXDriver and just ran out of storage capacity. Looking unfortunately nova does not tell, instead a new deployment operations fails with no error message and no log entries... Whatever, I could track it down to storage capacity, I think:

~ # esxcli storage filesystem list
Mount Point                                        Volume Name         UUID                                 Mounted  Type           Size         Free
-------------------------------------------------  ------------------  -----------------------------------  -------  ------  -----------  -----------
/vmfs/volumes/516d467b-6c3dd967-877c-00505601001d  esxi01:datastore01  516d467b-6c3dd967-877c-00505601001d     true  VMFS-5  80530636800  22807576576
/vmfs/volumes/516d467c-9302ef8c-8833-00505601001d                      516d467c-9302ef8c-8833-00505601001d     true  vfat     4293591040   4219338752
/vmfs/volumes/db25c371-9d9574d1-f0d8-7d34bd7e488b                      db25c371-9d9574d1-f0d8-7d34bd7e488b     true  vfat      261853184    125358080
/vmfs/volumes/6180a573-1f8dee38-9722-90ed8136e5ef                      6180a573-1f8dee38-9722-90ed8136e5ef     true  vfat      261853184    110006272
/vmfs/volumes/516d4675-af07b34c-d35c-00505601001d                      516d4675-af07b34c-d35c-00505601001d     true  vfat      299712512     87957504
~ #

~ # df -h
Filesystem   Size   Used Available Use% Mounted on
VMFS-5      75.0G  53.8G     21.2G  72% /vmfs/volumes/esxi01:datastore01
vfat         4.0G  70.8M      3.9G   2% /vmfs/volumes/516d467c-9302ef8c-8833-00505601001d
vfat       249.7M 130.2M    119.6M  52% /vmfs/volumes/db25c371-9d9574d1-f0d8-7d34bd7e488b
vfat       249.7M 144.8M    104.9M  58% /vmfs/volumes/6180a573-1f8dee38-9722-90ed8136e5ef
vfat       285.8M 201.9M     83.9M  71% /vmfs/volumes/516d4675-af07b34c-d35c-00505601001d
~ #

So not a lot of space available. Looking into the datastore, I could see that the vmware_base directory is by far the largest one. I am only running 1 Cirros and 1 Ubuntu instance right now ;)

/vmfs/volumes/516d467b-6c3dd967-877c-00505601001d # du -h 
44.7G   ./vmware_base
4.1G    ./compute1
8.0K    ./vmware-tmp
2.7G    ./8e30520d-d826-43ec-ada2-be8409db1ae6
623.0M  ./a2364ccc-e277-4e0d-93c8-c6b65c5361a2
53.6G   .
/vmfs/volumes/516d467b-6c3dd967-877c-00505601001d #

This is what is inside the vmware_base directory:

/vmfs/volumes/516d467b-6c3dd967-877c-00505601001d/vmware_base # ls -hl
-rw-------    1 root     root       39.2M Jan  5 14:50 0949b205-f27b-4bde-8204-4eb72f730e68-flat.vmdk
-rw-------    1 root     root        1.0G Jan  5 14:50 0949b205-f27b-4bde-8204-4eb72f730e68.1-flat.vmdk
-rw-------    1 root     root         548 Jan 15 21:35 0949b205-f27b-4bde-8204-4eb72f730e68.1.vmdk
-rw-------    1 root     root       10.0G Jan 15 21:43 0949b205-f27b-4bde-8204-4eb72f730e68.10-flat.vmdk
-rw-------    1 root     root         551 Jan 15 21:44 0949b205-f27b-4bde-8204-4eb72f730e68.10.vmdk
-rw-------    1 root     root       20.0G Jan  5 20:13 0949b205-f27b-4bde-8204-4eb72f730e68.20-flat.vmdk
-rw-------    1 root     root         551 Jan 13 17:54 0949b205-f27b-4bde-8204-4eb72f730e68.20.vmdk
-rw-------    1 root     root       40.0G Jan 11 16:52 0949b205-f27b-4bde-8204-4eb72f730e68.40-flat.vmdk
-rw-------    1 root     root         551 Jan 12 08:31 0949b205-f27b-4bde-8204-4eb72f730e68.40.vmdk
-rw-------    1 root     root         541 Jan  5 14:50 0949b205-f27b-4bde-8204-4eb72f730e68.vmdk
-rw-------    1 root     root       39.2M Jan  5 14:00 0e48a956-a008-424c-99ee-41692eb03fe8-flat.vmdk
-rw-------    1 root     root        1.0G Jan  5 14:00 0e48a956-a008-424c-99ee-41692eb03fe8.1-flat.vmdk
-rw-------    1 root     root         548 Jan  5 14:42 0e48a956-a008-424c-99ee-41692eb03fe8.1.vmdk
-rw-------    1 root     root         541 Jan  5 14:00 0e48a956-a008-424c-99ee-41692eb03fe8.vmdk
-rw-------    1 root     root        2.0G Jan 15 20:58 2efd517e-8445-451a-a5e6-6fc1915a87cd-flat.vmdk
-rw-------    1 root     root       20.0G Jan 15 20:58 2efd517e-8445-451a-a5e6-6fc1915a87cd.20-flat.vmdk
-rw-------    1 root     root         551 Jan 15 21:36 2efd517e-8445-451a-a5e6-6fc1915a87cd.20.vmdk
-rw-------    1 root     root         545 Jan 15 20:58 2efd517e-8445-451a-a5e6-6fc1915a87cd.vmdk
-rw-------    1 root     root        2.0G Jan  5 13:12 7a81cac3-91c9-427d-a08c-9a34b301bae5-flat.vmdk
-rw-------    1 root     root       20.0G Jan  5 13:28 7a81cac3-91c9-427d-a08c-9a34b301bae5.20-flat.vmdk
-rw-------    1 root     root         551 Jan  5 13:39 7a81cac3-91c9-427d-a08c-9a34b301bae5.20.vmdk ...
(more)
edit retag flag offensive close merge delete