Ask Your Question
0

Using Glance/Nova on multinode compute

asked 2011-04-20 19:36:28 -0500

dennis-demarco gravatar image

I have a question on using Glance/Nova

I have a glance setup that uses swift for the back-end storage. I have about 6 1U servers for swift nodes (1 proxy, 5 storage) and 9 1U servers for compute nodes (1 as a cloud-controller/glance,8 compute nodes)

As I understand it nova-objectstore does not work in multi-compute setups, and the suggested method is to use glance.

I can add a raw image using glance-add and I can verify it uploaded into the swift object store. However, I can not seem to use euca-upload-image or uec-publish-image to add images. Are these tools supported? It am getting S3 timeout errors.

Here are the packages I have installed on the cloud controller/glance node

ii python-swift 1.2.0-0ubuntu1~lucid0
ii swift 1.2.0-0ubuntu1~lucid0
ii glance 2011.2~bzr114-0ubuntu0ppa1~lucid1
ii python-glance 2011.2~bzr114-0ubuntu0ppa1~lucid1

ii nova-api 2011.2-0ubuntu0ppa1~lucid1
ii nova-common 2011.2-0ubuntu0ppa1~lucid1
ii nova-compute 2011.2-0ubuntu0ppa1~lucid1
ii nova-doc 2011.2-0ubuntu0ppa1~lucid1
ii nova-network 2011.2-0ubuntu0ppa1~lucid1
ii nova-scheduler 2011.2-0ubuntu0ppa1~lucid1
ii python-nova 2011.2-0ubuntu0ppa1~lucid1
ii python-novaclient 2.4-0ubuntu1~lucid1

This is my nova.config.

--dhcpbridge_flagfile=/etc/nova/nova.conf --dhcpbridge=/usr/bin/nova-dhcpbridge --logdir=/var/log/nova --state_path=/var/lib/nova --lock_path=/var/lock/nova --verbose --sql_connection=mysql://root:nova@10.239.220.10/nova --rabbit_host=10.239.220.10 --ec2_host=10.239.220.10 --ec2_url=http://10.239.220.10:8773/services/Cloud --network_manager=nova.network.manager.VlanManager --fixed_range=192.168.0.0/24 --network_size=254 --glance_host=10.239.220.10 --image_service=nova.image.glance.GlanceImageService --s3_host=10.239.220.10

edit retag flag offensive close merge delete

9 answers

Sort by ยป oldest newest most voted
0

answered 2011-04-26 18:50:48 -0500

dennis-demarco gravatar image

I understand a bit better now. Thank you. So in order for euca-bundle/etc to work both glance and nova-object store need to be running because those tools need a S3 service.

Does the image upload get placed into glance's backend? When I tried running both glance and nova-object, the location of the image never referenced a swift URL (glance details).

So images what were uploaded using glance-add/upload were in swift while images using euca toolset were referenced local. Is there a configuration setting for nova-object that I missed?

edit flag offensive delete link more
0

answered 2011-04-26 18:36:10 -0500

vishvananda gravatar image

It is my understanding nova-object should not run if you are running with glance. It should all go through glance. I just do not think the API is yet compatible with euca-bundle/uec yet. I'm running swift object store to store all my images and nova-object is local machine only storage.

I want to clarify something here. Nova-objectstore exists only to support euca-upload-bundle and to make euca-register work properly. It works fine with glance image service and local image service.

Nova-objectstore is just a a very simple bucket service (replicates s3 api).

There seems to be quite a bit of confusion on this point, so let me describe how the AWS image registration works by describing what each of the euca commands does.

euca-bundle-vol/euca-bundle-image: * compresses the volume/image * encrypts the compressed file * breaks the encrypted file into 10 mb chunks * creates a manifest.xml file with metadata about the image and all of the chunks

euca-upload-bundle: * uploads manifest and xml to an s3 bucket

euca-register * grabs the manifest from an s3 bucket * downloads all of the part files * combines the part files into one file * decrypts the file * decompresses the file * extracts metadata from the manifest * makes the image available (this involves uploading the decompressed file into the image service along with image metadata)

As you can see, the AWS model for uploading images requires an S3 service to make uploading work. Therefore, to run these commands, you need to have some sort of S3 service. Nova-objectstore is just an ultra-simple s3 service to support this particular set of commands.

Vish


You received this question notification because you are a member of Glance Core, which is an answer contact for Glance.

edit flag offensive delete link more
0

answered 2011-04-26 16:07:40 -0500

dennis-demarco gravatar image

What authentication type are you using in swift? I added the auth-server services and using that instead swauth. If your using swauth I think the URL is different in the glance.conf

ie

https://<proxy_hostname>:11000/1.0

vs. (swauth)

https://<proxy_hostname>:8080/auth/v1.0

edit flag offensive delete link more
0

answered 2011-04-26 15:51:50 -0500

dennis-demarco gravatar image

Try using glace-upload to upload an image. There are a few image style formats out there (Raw/tar/AMI) and it can get confusing. You will need to identify the type of image you have in order to upload use it correctly.

This is what I did to get the ttylinux-uec-amd64-12.1_2.6.35-22_1.tar.gz to work.

Untar it first. Then upload it as an AMI image:

glance-upload --type=ramdisk ttylinux-uec-amd64-12.1_2.6.35-22_1-initrd ttyinitrd glance-upload --type=kernel ttylinux-uec-amd64-12.1_2.6.35-22_1-vmlinuz ttykernel glance-upload --type=machine --kernel=14 --ramdisk=13 ttylinux-uec-amd64-12.1_2.6.35-22_1.img ttyimage

This seemed to work well for me.

It is my understanding nova-object should not run if you are running with glance. It should all go through glance. I just do not think the API is yet compatible with euca-bundle/uec yet. I'm running swift object store to store all my images and nova-object is local machine only storage.

edit flag offensive delete link more
0

answered 2011-04-26 02:13:07 -0500

everett-toews gravatar image

Hi Dennis,

I'm trying to do the same thing right now. As I understand it, if you want to use uec-publish-tarball or the euca-bundle-* commands then you need to run nova-objectstore. Those commands require an S3 API and nova-objectstore is the service that provides it.

However, when I run nova-objectstore and do an upload the image doesn't get stored in Swift. Instead the parts get stored in the usual image directory /var/lib/nova/buckets/ dir.

My relevant nova.conf config is:

--s3_host=127.0.0.1 --glance_host=127.0.0.1 --image_service=nova.image.glance.GlanceImageService

(I'm keeping everything on one machine for the moment while I'm testing, hence the 127 address)

Of course I haven't even been able to store my images in Swift using glance-add (see https://answers.launchpad.net/glance/+question/154253 (https://answers.launchpad.net/glance/...) ) so I might have something else misconfigured.

Would you be willing to share your glance config so I can compare it with my config? If I can get glance-add working, I'll be able to move on to getting the uec-publish-tarball and euca-bundle-* commands working with Glance/Swift and can try to continue to help.

Everett

edit flag offensive delete link more
0

answered 2011-04-25 02:04:59 -0500

I have the same problem. I can add image through glance, but i can't use euca-upload-bundle, uec-publish-tarball to upload. Below is my error message.

"uec-publish-tarball" error message:

root@clouder:~#uec-publish-tarball /data/images/ubuntu1010-UEC-localuser-image.tar.gz ubuntu1010 amd64 Mon Apr 25 18:02:44 CST 2011: ====== extracting image ====== Warning: no ramdisk found, assuming '--ramdisk none' kernel : maverick-server-uec-amd64-vmlinuz-virtual ramdisk: none image : maverick-server-uec-amd64.img kernel : maverick-server-uec-amd64-vmlinuz-virtual ramdisk: none image : maverick-server-uec-amd64.img rock 1 read: 200: arg count Mon Apr 25 18:03:17 CST 2011: ====== bundle/upload kernel ====== failed to register maverick-server-uec-amd64-vmlinuz-virtual.manifest.xml failed: euca-register --name maverick-server-uec-amd64-vmlinuz-virtual ubuntu1010/maverick-server-uec-amd64-vmlinuz-virtual.manifest.xml UnknownError: An unknown error has occurred. Please try your request again.failed to upload kernel

"euca-upload-bundle" error message:

euca-upload-bundle -b images -m /tmp/maverick-server-uec-amd64.img.manifest.xml Checking bucket: images Uploading manifest file Warning: failed to parse error message from AWS: http://www.w3.org/TR/html4/strict.dtd:81:4 (http://www.w3.org/TR/html4/strict.dtd...) : error in processing external entity reference Permission denied while writing: maverick-server-uec-amd64.img.manifest.xm

edit flag offensive delete link more
0

answered 2011-04-20 20:22:05 -0500

dennis-demarco gravatar image

I also tried setting my S3_URL to http://10.239.220.10:9292

This is what I got for client side:

root@nova-cc01:~# uec-publish-tarball ttylinux-uec-amd64-12.1_2.6.35-22_1.tar.gz test x86_64 Wed Apr 20 16:20:19 EDT 2011: ====== extracting image ====== kernel : ttylinux-uec-amd64-12.1_2.6.35-22_1-vmlinuz ramdisk: ttylinux-uec-amd64-12.1_2.6.35-22_1-initrd image : ttylinux-uec-amd64-12.1_2.6.35-22_1.img Wed Apr 20 16:20:20 EDT 2011: ====== bundle/upload kernel ====== Traceback (most recent call last): File "/usr/bin/euca-upload-bundle", line 231, in <module> main() File "/usr/bin/euca-upload-bundle", line 214, in main bucket_instance = ensure_bucket(conn, bucket, canned_acl) File "/usr/bin/euca-upload-bundle", line 92, in ensure_bucket bucket_instance = create_bucket(connection, bucket, canned_acl) File "/usr/bin/euca-upload-bundle", line 81, in create_bucket return connection.create_bucket(bucket, policy=canned_acl) File "/usr/lib/pymodules/python2.6/boto/s3/connection.py", line 321, in create_bucket raise S3ResponseError(response.status, response.reason, body) boto.exception.S3ResponseError: S3ResponseError: 404 Not Found 404 Not Found

The resource could not be found.

failed to upload bundle to test/ttylinux-uec-amd64-12.1_2.6.35-22_1-vmlinuz.manifest.xml failed: euca-upload-bundle --bucket test --manifest /tmp/uec-publish-image.f1ovSs/ttylinux-uec-amd64-12.1_2.6.35-22_1-vmlinuz.manifest.xml Checking bucket: test Warning: failed to parse error message from AWS: <unknown>:1:0: syntax error Creating bucket: test Warning: failed to parse error message from AWS: <unknown>:1:0: syntax errorfailed to upload kernel

This the glance side:

2011-04-20 16:21:03 DEBUG [routes.middleware] Matched GET /images/detail 2011-04-20 16:21:03 DEBUG [routes.middleware] Route path: '/images/detail', defaults: {'action': u'detail', 'controller': <glance.server.controller object="" at="" 0x1d41ad0="">} 2011-04-20 16:21:03 DEBUG [routes.middleware] Match dict: {'action': u'detail', 'controller': <glance.server.controller object="" at="" 0x1d41ad0="">} 2011-04-20 16:21:03 DEBUG [eventlet.wsgi.server] 10.239.220.10 - - [20/Apr/2011 16:21:03] "GET /images/detail HTTP/1.1" 200 540 0.007298 2011-04-20 16:21:03 DEBUG [routes.middleware] Matched GET /images/detail 2011-04-20 16:21:03 DEBUG [routes.middleware] Route path: '/images/detail', defaults: {'action': u'detail', 'controller': <glance.server.controller object="" at="" 0x1d41ad0="">} 2011-04-20 16:21:03 DEBUG [routes.middleware] Match dict: {'action': u'detail', 'controller': <glance.server.controller object="" at="" 0x1d41ad0="">} 2011-04-20 16:21:03 DEBUG [eventlet.wsgi.server] 10.239.220.10 - - [20/Apr/2011 16:21:03] "GET /images/detail HTTP/1.1" 200 540 0.006362 2011-04-20 16:21:04 DEBUG [routes.middleware] Matched GET /images/detail 2011-04-20 16:21:04 DEBUG [routes.middleware] Route path: '/images/detail', defaults: {'action': u'detail', 'controller': <glance.server.controller object="" at="" 0x1d41ad0="">} 2011-04-20 16:21:04 DEBUG [routes.middleware] Match dict: {'action': u'detail', 'controller': <glance.server.controller object="" at="" 0x1d41ad0="">} 2011-04-20 16:21:04 DEBUG [eventlet.wsgi.server] 10.239.220.10 - - [20/Apr/2011 16:21:04] "GET /images/detail HTTP/1.1" 200 540 0.007064 2011-04-20 16:21:04 DEBUG [routes.middleware] No route matched for GET /test/ 2011-04-20 16:21:04 DEBUG [eventlet.wsgi.server] 10.239.220.10 - - [20/Apr/2011 16:21:04] "GET /test/?max-keys=0 ... (more)

edit flag offensive delete link more
0

answered 2011-05-04 18:30:07 -0500

vishvananda gravatar image

one typo in my description above. this line:

  • uploads manifest and xml to an s3 bucket

should say:

  • uploads manifest.xml and part files to an s3 bucket
edit flag offensive delete link more
0

answered 2011-04-26 18:55:51 -0500

vishvananda gravatar image

if you specify: --glance_host=<ip of="" glance=""> --image_service=nova.image.glance.GlanceImageService in your flagfile, the euca commands, will end up storing the final decrypted image and metadata in whatever backend you have glance configured to use.

Vish

On Apr 26, 2011, at 11:50 AM, Dennis DeMarco wrote:

Question #153638 on Glance changed: https://answers.launchpad.net/glance/...

Status: Answered => Open

Dennis DeMarco is still having a problem: I understand a bit better now. Thank you. So in order for euca- bundle/etc to work both glance and nova-object store need to be running because those tools need a S3 service.

Does the image upload get placed into glance's backend? When I tried running both glance and nova-object, the location of the image never referenced a swift URL (glance details).

So images what were uploaded using glance-add/upload were in swift while images using euca toolset were referenced local. Is there a configuration setting for nova-object that I missed?


You received this question notification because you are a member of Glance Core, which is an answer contact for Glance.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2011-04-20 19:36:28 -0500

Seen: 85 times

Last updated: May 04 '11