all requests to old files 404 after extending volume size in Swift SAIO [closed]

asked 2016-12-22 05:51:04 -0500

wenyuan gravatar image

We have a Swift SAIO installation on one of our servers. It is to store all the user uploaded files from a small internal app we developed. It works all fine until one of our ops tried to extend the swift loop volume this afternoon.

What he did is to stop swift, copy everything to a temp directory, truncate the loop file, remake the xfs, mount and copy files back.

After restarting swift, requests to every file that uploaded before became 404, but you can still list the files. The newly uploaded files are fine.

[root@host-10-20-4-44 swift]# curl -X GET -H "X-Auth-Token: AUTH_tkd6b685264d0846e2acb9cf76dd0515e8" http://10.20.4.44:9999/v1/AUTH_test/test
...
null/1481882937_1041165417646604798_12345/1.png
...
[root@host-10-20-4-44 swift]# curl -X GET -H "X-Auth-Token: AUTH_tkd6b685264d0846e2acb9cf76dd0515e8" http://10.20.4.44:9999/v1/AUTH_test/test/null/1481882937_1041165417646604798_12345/1.png
<html><h1>Not Found</h1><p>The resource could not be found.</p></html>[root@host-10-20-4-44 swift]#

The Swift version is "2.9.1.dev164", os is centos 7, and we followed http://docs.openstack.org/developer/swift/development_saio.html (http://docs.openstack.org/developer/s...) to set up a saio environment.

[root@host-10-20-4-44 swift]# curl http://10.20.4.44:9999/info
{"bulk_delete": {"max_failed_deletes": 1000, "max_deletes_per_request": 10000}, "versioned_writes": {}, "container_quotas": {}, "crossdomain": {}, "swift": {"max_file_size": 5368709122, "account_listing_limit": 10000, "account_autocreate": true, "max_meta_count": 90, "max_meta_value_length": 256, "container_listing_limit": 10000, "extra_header_count": 0, "max_meta_overall_size": 4096, "version": "2.9.1.dev164", "max_meta_name_length": 128, "max_header_size": 8192, "policies": [{"default": true, "name": "gold", "aliases": "gold"}, {"name": "silver", "aliases": "silver"}, {"name": "ec42", "aliases": "ec42"}], "max_object_name_length": 1024, "max_account_name_length": 256, "strict_cors_mode": true, "allow_account_management": true, "max_container_name_length": 256}, "tempurl": {"outgoing_remove_headers": ["x-object-meta-*"], "incoming_remove_headers": ["x-timestamp"], "incoming_allow_headers": [], "methods": ["GET", "HEAD", "PUT", "POST", "DELETE"], "outgoing_allow_headers": ["x-object-meta-public-*"]}, "bulk_upload": {"max_failed_extractions": 1000, "max_containers_per_extraction": 10000}, "container_sync": {"realms": {"SAIO": {"clusters": {"SAIO_ENDPOINT": {"current": true}}}}}, "ratelimit": {"account_ratelimit": 0.0, "max_sleep_time_seconds": 60.0, "container_ratelimits": [], "container_listing_ratelimits": []}, "slo": {"max_manifest_segments": 1000, "min_segment_size": 1, "max_manifest_size": 2097152}, "account_quotas": {}, "staticweb": {}, "tempauth": {"account_acls": true}}

Any idea? Can we recover the files? Thanks!

edit retag flag offensive reopen merge delete

Closed for the following reason duplicate question by wenyuan
close date 2016-12-22 08:07:23.559566