Ask Your Question

clayg's profile - activity

2019-03-14 10:14:46 -0500 answered a question How to create symlinks in swift?
2014-04-16 18:51:06 -0500 answered a question How to delete old directory of container already deleted?

container's will be deleted by the container-replicator after their reclaim age has passed. The reclaim age is configurable:

go down the "container-replicator" section

It is not safe to delete to delete the containers any sooner that the consistency engine cycle time because it can lead to ghost entries in the accounts.

You can also turn down the concurrency and interval on the container updater.

But the container's won't actually updating if they're fully reported, so the only overhead should be the quick check on the container_stat table for the reported timestamps and the io of container-updaters file system walk - which you should be able to tune to the load and capabilities of your system.

2014-04-16 18:14:19 -0500 received badge  Commentator
2014-04-16 18:14:19 -0500 commented answer Error: Service s-proxy is not running - OpenStack installation
2014-04-16 18:04:07 -0500 commented answer Enable Swift in OpenStack Devstack

in your localrc?

2014-04-16 18:03:07 -0500 answered a question PHP-OpenCloud with OpenStack installed Locally

It certainly does seem to target the Rackspace cloud - which should be mostly compatible with openstack api's - but it may not be that throughly tested with a default devstack installation.

I think the auth integration may have some gaps:

If you're seeing some more specific issues you should open them up on their issue tracker:

2014-04-16 17:58:15 -0500 answered a question I need response please ( Swift Glance )

Sounds like a stand alone project that integrates with openstack apis. If it works with swift you can see about adding it to

2014-03-01 01:00:18 -0500 received badge  Good Answer (source)
2014-03-01 01:00:18 -0500 received badge  Enlightened (source)
2013-12-05 14:18:05 -0500 commented answer are results of functests and probetests right?

Nose should have been installed -> check that the /srv/1/node/sdb1 directories are being created, and that the symlinks connect them to /mnt/sdb1/1/node/sdb1. Anything tracebacks in /var/log/syslog?

2013-12-05 02:01:29 -0500 answered a question are results of functests and probetests right?

the functest results look good, the skip tests might be because you need to enable static large object support

the probetests output doesn't look right, but they call resetswift, and make a number of assumptions about your swift-all-in-one configuration. Even people who work on swift regularly have trouble with them, I think at one point everyone just assumed they only worked if you were gholt.

A few things you can try:

  1. run some common checks for misconfiguration based on the assumptions probetest need (you might need if you're using a loop back device)

    python swift/test/probe/

  2. run w/o redirected logging and stop after the first failure - then you can inspect /var/log/syslog for errors/exception

    nosetest -vsx swift/test/probe/

My guess is that your resetswift script isn't wiping your data disks and making the /srv/node links or setting up the permissions correctly.

If you are using a loopback device substitute /dev/sdb1 with /srv/swift-disk in the mkfs step.
2013-11-28 21:23:51 -0500 received badge  Good Answer (source)
2013-11-13 12:00:17 -0500 answered a question Why won't containers show when I run containers status command?

By default the swift command line tool will only show the containers for the tenant which is being authenticated under their own storage account.

You can use the --os-storage-url option to bypass the default storage-url and view the containers/account_info for another tenant/storage-account if you have an authorized token (i.e. .reseller_admin)

2013-11-13 11:55:55 -0500 answered a question Swift Object Expirer never deletes items only sends 404

i think the top level expirer account won't get created until you run the container updater.

so make sure those have run/are running.

2013-10-10 07:43:27 -0500 received badge  Nice Answer (source)
2013-10-07 22:21:24 -0500 received badge  Nice Answer (source)
2013-09-25 09:28:15 -0500 answered a question how to set up rsync for swift on CentOS?

Check that rsync 873/tcp is in /etc/services

Check that you're not blocked by /etc/hosts.deny

Try rsync rsync://<ip of node in the ring>

Note: no pub@

2013-09-18 06:17:01 -0500 received badge  Nice Answer (source)
2013-09-18 03:25:50 -0500 commented answer ImportError running Swift

well, that looks correct, seems like the code and entry_points are installed correctly (for this user/environment anyway)

2013-09-18 03:13:30 -0500 received badge  Enthusiast
2013-09-17 01:59:10 -0500 answered a question Dedicated replication network for swift

If you want to separate your replication traffic from your backend request traffic you'll need an isolated network (different ip) on each storage node.

I think does a really good job of laying out all of the steps needed to simulate a separate replication network for testing, and enough information for deployers with Swift experience to apply the concepts to their environment. I think it'd be nice to see updated; possibly with additional information on the dedicated replication network feature - but it's not always useful and even more rarely required - but that may change as more people experiment with geographically distributed clusters.


Generally moving replication traffic to a separate network requires moving two parts:

  • The data transfer channel (rsync)
  • The management channel (the HTTP REPLICATE method)

You'll also need to update the device info in your ring builder files.


The configuration of the named rsync endpoints (rsync calls them "modules") is not managed automatically by Swift, but it expects storage nodes to have named rsync endpoints in form rsync://<replication_ip>/<server-type>

max connections = 2
path = /srv/node/
read only = false
lock file = /var/lock/object.lock

or rsync://<replication_ip>/<server-type><replication_port>

max connections = 2
path = /srv/1/node/
read only = false
lock file = /var/lock/object6010.lock

... if you're running in vm_test_mode.

The replication "port" is my favorite part (rsync always runs on the same port), but it is used by the management channel.


The HTTP REPLICATE verb on the management channel does some network (especially on the account/container servers) - and you can separate that out to the dedicated replication network by having a separate process listening for REPLICATE requests on replication network. The swift process management tool swift-init can make this easier on you by launching one server for each config.

So if you currently have /etc/swift/object-server.conf you'll want to create /etc/swift/objet-server/request.conf and /etc/swift/object-server/replication.conf so that when you (or your rc scripts) call swift-init object-server start you get one instance started from request.conf and configured for requests, and another started from replication.conf for replication management commands.

You'll need to set the bind_ip and bind_port options in the [DEFAULT] section. You'll also want to set replication_server = true in the [app:object-server] section of replication.conf and after you're sure the replication network is working to false in request.conf.


It's easiest to add replication parameters when adding devices to the rings:

swift-ring-builder test.builder add --region 1 --zone 1 --ip --port 6010 --replication-ip --replication-port 8010 --device sda --weight 100

... but you can also update existing devices with set_info:

$ swift-ring-builder test.builder set_info R192.168.3.100
Device d0r1z1-"" is now d0r1z1-192.168 ...
2013-09-17 01:06:45 -0500 answered a question problem in preparing and labeling the drives in order to mount them

If you do not have a spare physical device (or are unsure) you can use a loopback device:

Four or more devices is useful if you want to develop or test swift's failure handling and consistency engine. If you only need to test the API - you can have only one.

2013-09-17 00:57:48 -0500 answered a question ImportError running Swift

This looks like a standard run of the mill install error. The following normally works:

cd ~/swift/  # /root/swift seems to be the current install location according to your traceback
pip install --upgrade distribute  # or maybe setuptools
pip install -U -r requirements.txt
python develop

If you're still having problems it may be something with how pkg_resources is exposing the entry_points from swift to paste - weird stuff been goin' on since distribute took over and setuptools started up versionin' again...


What does the following print your your system:

cd ~; python -c 'import pkg_resources, pprint; pprint.pprint(pkg_resources.get_entry_map(pkg_resources.get_distribution("swift")))'

Here's mine:

2013-09-17 00:41:11 -0500 answered a question Do we need to stop swift servers while adding new disk?
2013-09-01 12:10:53 -0500 answered a question 412 Precondition Failed on swift -a

Looks like tempauth or swauth? You probably want the AUTH_URL to be:


so try running swift -A http://localhost:8080/auth/v1.0 -U demo:admin -K password stat

2013-08-22 12:44:52 -0500 answered a question unable to start rsync, keystone, swift

sudo apt-get install rsync

2013-08-22 12:36:41 -0500 commented question Glance API shows authentication errno 1 during uploading image to swift's container

maybe somehow glance is using the wrong port to talk to swift? can you paste the swift section of your glance config?

2013-08-17 11:39:39 -0500 answered a question problem in creating builder files in swift

Looks like the ubuntu package for dnspython 1.9.4, and the Swift package you have is requiring dnspython 1.10.0

Try to install the newer dnspython with pip:

pip install "dnspython==1.10.0"
2013-08-17 02:42:25 -0500 commented answer swift admin password gives unexpected argument error.why??

It depends how you originally installed - can you provide more details?

2013-08-16 04:17:40 -0500 answered a question swift admin password gives unexpected argument error.why??

Looks like an old version of python-keystone client mixed up with a newerish version of pythonswiftclient? You should try to verify the versions of the installed software (system packages? pypi? source?) and update them.

Please update the question with the full tracebacks.

2013-08-09 10:52:16 -0500 answered a question The proxy node in swift
  1. There's no question in that statement - I think "ask" is primarily a QA forum - perhaps you'd get a more suitable response on the mailing list?

  2. I've already answered this question here:

2013-08-08 00:53:59 -0500 received badge  Critic (source)
2013-08-08 00:53:25 -0500 commented question Swift Error - unable to retrieve container list created yesterday but able to create new containers and upload them with objects

Is the dashboard getting a 404 or 503 from Swift? Is the dashboard doing any cacheing, what does the swift command line client say? Are all the services running and drives mounted? Proxy logs would be the best place to start, container logs next.

2013-08-08 00:49:16 -0500 answered a question The proxy node in swift

Most deployments have more throughput between the proxy and object tier than between their proxy/load-balancing core's and any one client; so the amplification on PUT is not a limiting factor during upload.

Regardless, the consequences of loosing an object between a "successful" upload and an asynchronous offload is too dangerous. The proxy will even go so far as to "hand off" to secondary nodes if one of the primary locations is not responsive in order to ensure full durability immediately upon response to the client.

I should clarify if it wasn't already obvious that the proxy writes all three copies simultaneously - not "one by one". If any single connection drops out mid-stream the remaining two can continue to service the request successfully.

Other storage systems with different goals have taken the approach you've described - DRDB for example has an asynchronous mirroring mode. Recently Swift added support for write-affinity specifically for handling the case of geographically distributed clusters - but you may be able to leverage it to achieve a similar effect if you have use case that requires it:

2013-08-08 00:28:43 -0500 answered a question Is there any scripts available to automate swift uploading?

the standard openstack swift command line client [1] upload command has a --changed option that might suit your needs:

  -c, --changed         Will only upload files that have changed since the
                        last upload

For comparison you might also consider trying swiftly [2] - who's put command has a --different option that might also fit your needs:

  -d, --different       For PUTs with an --input option, first performs a HEAD
                        on the object and compares the X-Object-Meta-Mtime
                        header with the modified time of the PATH obtained
                        from the --input option and then PUTs the object only
                        if the local time is different. It will also check the
                        local and remote sizes and PUT if they differ.
                        ETag/MD5sum checking are not done (an option may be
                        provided in the future) since this is usually much
                        more disk intensive. When the --input PATH is a
                        directory, this offers an easy way to upload only the
                        differing files since the last upload (at the expense
                        of HEAD requests). NOTE THAT THIS CAN UPLOAD OLDER
                        FILES OVER NEWER ONES! DIFFERENT does not mean NEWER.
2013-08-08 00:18:16 -0500 answered a question Is it possible to create a separate system to keep my object files is swift?


Many deployments separate their object nodes from the account/container nodes.

There is no requirement for the nodes or devices in one of the rings to be present in the others.

2013-08-07 17:18:28 -0500 commented answer saio error, swift instructions, don't work!!!
2013-08-04 21:24:38 -0500 answered a question problem with install swift
  1. I've always prefer to use partitions, but I don't think there's any technical reasons. Adding a drive in virtualbox just seems like less work that setting up a loopback. When I'm on a cloud server with only one device - it's the typically the opposite, I prefer to setup a loopback. If you're not sure if you have a extra device or how to add one - use the loopback method.

  2. If you've never used vi before, oh you're in for a treat -> (I'm a 15 year newb!) ... for a more "traditional" notepad like experience, some people get by with nano -> ... if you're not familiar with fstab start here ->

  3. <your-user-name> and <your-group-name> are place holders in the docs; they should never be used literally. They are representative of the logged in user's name and group. On my system it's clayg; on production systems it might be swift; if you're running as the superuser it'll be root.

Good Luck!

2013-08-03 15:53:35 -0500 received badge  Nice Answer (source)
2013-08-01 17:11:48 -0500 commented answer No swift endpoint after devstack installation

tenantName worked on my machine. Ask the Keystone guys why it's required all I know is: "cause roles"

2013-08-01 17:10:03 -0500 answered a question When I use swift to perform a HEAD, I am getting 401 Unauthorized.

The formatting there is a little weird.

Only think I can think of that would exhibit that behavior is memcached not running:

2013-08-01 03:49:45 -0500 answered a question No swift endpoint after devstack installation

Try adding "tenantName"

Here's my localrc:


And my curl foo:

curl -d '{"auth":{"passwordCredentials":{"username": "admin", "password": "admin"}, "tenantName": "admin"}}' -H "Content-type: application/json" <a href="http://localhost:5000/v2.0/tokens">http://localhost:5000/v2.0/tokens</a> | python -m json.tool

You should also just try using environment vars and the swift command line tool:

source ./devstack/openrc
swift stat -v`
2013-07-28 15:39:55 -0500 answered a question While adding nodes to swift, do uploads/downloads error?
2013-07-28 10:57:19 -0500 received badge  Editor (source)
2013-07-28 00:09:58 -0500 answered a question Adding nodes to swift

You don't need to restart the servers for them to pick up the new ring, they'll check for changes and reload them every so often automatically:

On a related note swift wsgi servers handle SIGHUP to signal a soft restart. The parent process will restart and spawn new workers to accept new connections while the rest of the workers will finish servicing existing connections before terminating.

2013-07-27 23:56:47 -0500 answered a question Is there anyway to enable inotify in swift?


I think most shops look at some form of home grown post-log processing.

2013-07-27 23:50:22 -0500 answered a question what's this (startmain): Exception: Could not bind to after trying for 30 seconds?

It means another process is already bound to that port - probably X.

If it's a swift process, swift-orphans should help you find it.

Otherwise maybe netstat -plnt