Hi Everyone,
Is there a way to upload large images in small chunks? My issue is that i have only 1GB in cashe memory and when i upload large images (around 3.5 GB) the Linux starts to use swap which slowes the system dramatically. Especially since it is swapping OpenStack services and not only the image.
Thank you for any input, Michael
There is only one API for uploading image data (description), and it doesn’t seem to allow uploading in pieces:
curl -i -X PUT -H "X-Auth-Token: $token" -H "Content-Type: application/octet-stream" \
-d @yoursourcefile.qcow2 $image_url/v2/images/{image_id}/file
You don’t describe your setup. How is your cloud configured; is it an all-in-one server? What storage backend do you use for Glance? From where do you upload the image? If you currently do everything on the same server that just has 1GB of cache, perhaps you gain something by keeping the source file and running the openstack CLI command on a separate PC.
Or you could stream the source file from somewhere else:
<program generating image data slowly> | curl -X PUT -H ... -d @- $image-url...
(that’s dash d at dash; I haven’t tried this)
Or you could ease the stress on the cache by throttling the network interface you use for uploading.
Of course, 1GB is woefully small, but it seems to me that there is a lot of experimentation you could do.
It's not a CLI solution, it's the raw API.
You can throttle the interface no matter how you access OpenStack. Look for the tc command. Out of the box, Horizon uploads a file located on the PC where the browser runs. I am, however, not aware of a CLI or GUI solution for uploading from stdin.
Correction: At least in Pike, Horizon could upload from a URL. This doesn't help you, though, since you can't use this method to control the upload speed.
Asked: 2018-06-05 10:43:31 -0500
Seen: 406 times
Last updated: Jun 05 '18
