How to work with S3 API using Swift3?

asked 2011-04-26 16:49:00 -0600

jinzishuai gravatar image

Hi there,

I am wondering if there is any more documentation on how to get swift3 to work with S3 API. Specifically, if I created a swift account by seki@OS-CC:/var/log$ swift-auth-add-user -K devauth -a system root testpass https://192.168.1.33:8080/v1/AUTH_365f77c9d523435dbcf12c9d2678d197 (https://192.168.1.33:8080/v1/AUTH_365...) And get the following seki@OS-CC:/var/log$ curl -k -v -H 'X-Storage-User: system:root' -H 'X-Storage-Pass: testpass' https://192.168.1.33:11000/v1.0 ... < X-Storage-Url: https://192.168.1.33:8080/v1/AUTH_365f77c9d523435dbcf12c9d2678d197 (https://192.168.1.33:8080/v1/AUTH_365...) < X-Storage-Token: AUTH_tka3599de5039545809d181637d0f010a9 < X-Auth-Token: AUTH_tka3599de5039545809d181637d0f010a9 ...

Should I set the following variables export EC2_ACCESS_KEY=AUTH_tka3599de5039545809d181637d0f010a9 export EC2_SECRET_KEY=testpass export S3_URL=https://192.168.1.33:8080/v1/AUTH_365f77c9d523435dbcf12c9d2678d197

I've added to the following to /etc/swift/proxy-server.conf [filter:swift3] use = egg:swift#swift3 log_facility = LOG_LOCAL1

But I am still getting 401 error: seki@OS-CC:~/s3-curl$ ./s3curl.pl --id $EC2_ACCESS_KEY --key $EC2_SECRET_KEY --get -- -s -v $S3_URL -k Unknown option: get WARNING: It isn't safe to put your AWS secret access key on the command line! The recommended key management system is to store your AWS secret access keys in a file owned by, and only readable by you.

For example:

%awsSecretAccessKeys = ( # personal account personal => { id => '1ME55KNV6SBTR7EXG0R2', key => 'zyMrlZUKeG9UcYpwzlPko/+Ciu0K2co0duRM3fhi', },

# corporate account
company => {
    id => '1ATXQ3HHA59CYF1CVS02',
    key => 'WQY4SrSS95pJUT95V6zWea01gBKBCL6PI0cdxeH8',
},

);

$ chmod 600 /home/seki/.s3curl

Will sleep and continue despite this problem. Please set up /home/seki/.s3curl for future requests. * About to connect() to 192.168.1.33 port 8080 (#0) * Trying 192.168.1.33... connected * Connected to 192.168.1.33 (192.168.1.33) port 8080 (#0) * successfully set certificate verify locations: * CAfile: none CApath: /etc/ssl/certs * SSLv3, TLS handshake, Client hello (1): * SSLv3, TLS handshake, Server hello (2): * SSLv3, TLS handshake, CERT (11): * SSLv3, TLS handshake, Server finished (14): * SSLv3, TLS handshake, Client key exchange (16): * SSLv3, TLS change cipher, Client hello (1): * SSLv3, TLS handshake, Finished (20): * SSLv3, TLS change cipher, Client hello (1): * SSLv3, TLS handshake, Finished (20): * SSL connection using AES256-SHA * Server certificate: * subject: C=CA; ST=AB; L=Edmonton; O=VRS; OU=RD; CN=OS-CC; emailAddress=Shi.Jin@vrstorm.com * start date: 2011-04-23 15:55:37 GMT * expire date: 2011-05-23 15:55:37 GMT * common name: OS-CC (does not match '192.168.1.33') * issuer: C=CA; ST=AB; L=Edmonton; O=VRS; OU=RD; CN=OS-CC; emailAddress=Shi.Jin@vrstorm.com * SSL certificate verify result: self signed certificate (18), continuing anyway.

GET /v1/AUTH_365f77c9d523435dbcf12c9d2678d197 HTTP/1.1 User-Agent: curl/7.21.0 (x86_64-pc-linux-gnu) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.18 Host: 192.168.1.33:8080 Accept: / Date: Tue, 26 Apr 2011 16:42:05 +0000 Authorization: AWS AUTH_tka3599de5039545809d181637d0f010a9:6L/VuKi4ZT5YQkI9JwnVIMT2TcI=

< HTTP/1.1 401 Unauthorized < Content-Type: text/html; charset=UTF-8 < Content-Length: 364 < Date: Tue, 26 Apr 2011 16:42:05 GMT < <html> <head> <title>401 Unauthorized</title> </head> <body>

401 Unauthorized

This server could ...
(more)
edit retag flag offensive close merge delete

5 answers

Sort by ยป oldest newest most voted
0

answered 2011-05-12 15:12:47 -0600

graham-hemingway gravatar image

I have setup my swift3 config per what you have here and want to test it out. You wouldn't have any idea perhaps about what a boto (python) connection script might look like would you?

Thanks for the great work on figuring this out. Cheers, Graham

edit flag offensive delete link more
0

answered 2011-05-10 18:13:45 -0600

following worked for me after a lot of trying(Swift 1.4):

-->proxy-server.conf [pipeline:main] pipeline = healthcheck cache swift3 swauth proxy-server

[filter:swift3] use = egg:swift#swift3

-->configure s3curl.pl Add your Host to the @endpoints array in line 33 my @endpoints = ( '1.2.3.4');

-->using s3curl.pl ./s3curl.pl --id 'myacc:myuser' --key mypw --get -- -s -v http://1.2.3.4:8080 Output: ...< HTTP/1.1 200 OK.... ?xml version="1.0" encoding="UTF-8"?> testdir2009-02-03T16:45:09.000Z

edit flag offensive delete link more
0

answered 2011-04-26 22:16:57 -0600

cthier gravatar image

When using the s3 compatibility layer, the access key needs to be in the form of account_name:user_name, and the secret key used to sign the request is the user's password. So in the above example, EC2_ACCESS_KEY=system:root

and in your config, the pipeline should look something like:

pipeline = healthcheck cache swift3 auth proxy-server

http://swift.openstack.org/misc.html#module-swift.common.middleware.swift3 (http://swift.openstack.org/misc.html#...)

Is the only documentation that we have currently.

edit flag offensive delete link more
0

answered 2011-04-26 18:10:28 -0600

jinzishuai gravatar image

Thanks. I didn't have swift3 there.

Now i added it

seki@OS-CC:/var/log$ cat /etc/swift/proxy-server.conf [DEFAULT] cert_file = /etc/swift/cert.crt key_file = /etc/swift/cert.key bind_port = 8080 workers = 8 user = swift

[pipeline:main] pipeline = healthcheck cache auth proxy-server swift3

[app:proxy-server] use = egg:swift#proxy allow_account_management = true

[filter:swift3] use = egg:swift#swift3 log_facility = LOG_LOCAL1

[filter:auth] use = egg:swift#auth ssl = true

[filter:healthcheck] use = egg:swift#healthcheck

[filter:cache] use = egg:swift#memcache memcache_servers = 127.0.0.1:11211

Then I cannot start the proxy service. syslog shows the following error:

Apr 26 12:00:54 OS-CC proxy-server UNCAUGHT EXCEPTION#012Traceback (most recent call last):#012 File "/usr/bin/swift-proxy-server", line 22, in <module>#012 run_wsgi(conf_file, 'proxy-server', default_port=8080, *options)#012 File "/usr/lib/pymodules/python2.6/swift/common/wsgi.py", line 126, in run_wsgi#012 app = loadapp('config:%s' % conf_file, global_conf={'log_name': log_name})#012 File "/usr/lib/pymodules/python2.6/paste/deploy/loadwsgi.py", line 204, in loadapp#012 return loadobj(APP, uri, name=name, *kw)#012 File "/usr/lib/pymodules/python2.6/paste/deploy/loadwsgi.py", line 224, in loadobj#012 global_conf=global_conf)#012 File "/usr/lib/pymodules/python2.6/paste/deploy/loadwsgi.py", line 248, in loadcontext#012 global_conf=global_conf)#012 File "/usr/lib/pymodules/python2.6/paste/deploy/loadwsgi.py", line 278, in _loadconfig#012 return loader.get_context(object_type, name, global_conf)#012 File "/usr/lib/pymodules/python2.6/paste/deploy/loadwsgi.py", line 405, in get_context#012 global_additions=global_additions)#012 File "/usr/lib/pymodules/python2.6/paste/deploy/loadwsgi.py", line 500, in _pipeline_app_context#012 APP, pipeline[-1], global_conf)#012 File "/usr/lib/pymodules/python2.6/paste/deploy/loadwsgi.py", line 363, in get_context#012 object_type, name=name)#012 File "/usr/lib/pymodules/python2.6/paste/deploy/loadwsgi.py", line 528, in find_config_section#012 self.filename))#012LookupError: No section 'swift3' (prefixed by 'app' or 'application' or 'composite' or 'composit' or 'pipeline' or 'filter-app') found in config /etc/swift/proxy-server.conf

Should I call it filter-app instead of filter? Where do I find the detailed reference on how to write this documentation? I am running ubuntu-10.10 and swift-1.2.0-0ubuntu1~maverick0. Thanks.

Shi

edit flag offensive delete link more
0

answered 2011-04-26 17:50:17 -0600

lgoikhburg gravatar image

Do you have swift3 enabled in the auth pipeline ? like:

[pipeline:main] pipeline = healthcheck cache swift3 swauth proxy-server

edit flag offensive delete link more

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2011-04-26 16:49:00 -0600

Seen: 4,057 times

Last updated: May 12 '11