Ask Your Question
0

Account HEAD failed 503 Internal Server Error

asked 2013-12-04 14:53:25 -0600

vigneshb4u gravatar image

Hello all,

I am configuring Swift per the instructions in https://access.redhat.com/site/documentation/en-US/Red_Hat_OpenStack/3/html-single/Installation_and_Configuration_Guide/index.html#chap-Installing_the_OpenStack_Object_Storage_Service (https://access.redhat.com/site/docume...)

After completing configuration, I am getting a 503 internal server error while validating installation using "swift stat"

Message from syslogd@localhost at Dec 4 09:29:34 ... ?<131>proxy-server Account HEAD returning 503 for [507] (txn: tx62353887227d445fad636dbdda582119) (client_ip: 127.0.0.1)

Message from syslogd@localhost at Dec 4 09:29:35 ... ?<135>proxy-server Node error limited 127.0.0.1:6002 (sdb1) (txn: tx88716d69ada741d1aecdec79cd7dc28e)

Message from syslogd@localhost at Dec 4 09:29:35 ... ?<131>proxy-server Account HEAD returning 503 for [] (txn: tx88716d69ada741d1aecdec79cd7dc28e) (client_ip: 127.0.0.1)

Message from syslogd@localhost at Dec 4 09:29:37 ... ?<135>proxy-server Node error limited 127.0.0.1:6002 (sdb1) (txn: txbe9692dc2d6042b2b07de156c65db6ff)

Message from syslogd@localhost at Dec 4 09:29:37 ... ?<131>proxy-server Account HEAD returning 503 for [] (txn: txbe9692dc2d6042b2b07de156c65db6ff) (client_ip: 127.0.0.1)

Message from syslogd@localhost at Dec 4 09:29:41 ... ?<135>proxy-server Node error limited 127.0.0.1:6002 (sdb1) (txn: tx652a3f07adb54517a6070ff35434dedc)

Message from syslogd@localhost at Dec 4 09:29:41 ... ?<131>proxy-server Account HEAD returning 503 for [] (txn: tx652a3f07adb54517a6070ff35434dedc) (client_ip: 127.0.0.1)

Message from syslogd@localhost at Dec 4 09:29:49 ... ?<135>proxy-server Node error limited 127.0.0.1:6002 (sdb1) (txn: tx16b1fcbc6a18472bae28840b15bfacac)

Message from syslogd@localhost at Dec 4 09:29:49 ... ?<131>proxy-server Account HEAD returning 503 for [] (txn: tx16b1fcbc6a18472bae28840b15bfacac) (client_ip: 127.0.0.1)

Message from syslogd@localhost at Dec 4 09:30:05 ... ?<135>proxy-server Node error limited 127.0.0.1:6002 (sdb1) (txn: tx39d798d702334ebb80bd4a1d1683af07)

Message from syslogd@localhost at Dec 4 09:30:05 ... ?<131>proxy-server Account HEAD returning 503 for [] (txn: tx39d798d702334ebb80bd4a1d1683af07) (client_ip: 127.0.0.1) Account HEAD failed: http://localhost:8080/v1/AUTH_2dd248ff7c05446b801eba44c082c133 (http://localhost:8080/v1/AUTH_2dd248f...) 503 Internal Server Error

I checked all the similar questions in this forum, but nothing works.

I checked the port in both account-server.conf and also the rings. Everything looks good. I tried tempauth and got the same error.

Following are my ports in conf and ring: [DEFAULT] devices = /srv/node/1 bind_ip = 127.0.0.1 bind_port = 6002 workers = 2

swift-ring-builder /etc/swift/account.builder search /sdb1

Devices: id region zone ip address port name weight partitions balance meta 0 1 1 127.0.0.1 6002 sdb1 1.00 786432 0.00

I am not able to turn on DEBUG log for the account-server as well. This is the only log I got from the account-server log (is it a 507 error?):

Dec 4 09:29:34 localhost account-server 127.0.0.1 - - [04/Dec/2013:14:29:34 +0000] "HEAD /sdb1/23746/AUTH_2dd248ff7c05446b801eba44c082c133" 507 - "tx62353887227d445fad636dbdda582119" "-" "-" 0.0003 ""

Following is my log setting for account-server.conf

[app:account-server] use = egg:swift#account set log_level = DEBUG set log_name = account-server set log_facility = LOG_LOCAL0

Probably debug logs will help me troubleshoot ... (more)

edit retag flag offensive close merge delete

16 answers

Sort by ยป oldest newest most voted
0

answered 2013-12-04 21:43:29 -0600

vigneshb4u gravatar image

Sure, Here you go..

df

Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/vg_openstack-lv_root 51606140 9571576 39413124 20% / tmpfs 49568868 232 49568636 1% /dev/shm /dev/sda1 495844 38044 432200 9% /boot /dev/mapper/vg_openstack-lv_home 1097480696 234232 1041497652 1% /home /dev/sr0 3632776 3632776 0 100% /media/RHEL_6.4 x86_64 Disc 1

edit flag offensive delete link more
0

answered 2013-12-04 21:53:27 -0600

vigneshb4u gravatar image

Following is the content of /etc/fstab

/dev/mapper/vg_openstack-lv_root / ext4 defaults 1 1 UUID=d5049cfa-b43d-489f-95fa-ba5ebb5d7d8e /boot ext4 defaults 1 2 /dev/mapper/vg_openstack-lv_home /home ext4 acl,user_xattr 1 2 /dev/mapper/vg_openstack-lv_swap swap swap defaults 0 0 tmpfs /dev/shm tmpfs defaults 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0

Following is how I configured the device_mountpoint

mkdir /home/storage chown swift:swift /home/storage/ mkdir /home/storage/{1..4} chown swift:swift /home/storage/* mkdir /srv/node/ ln -s /home/storage/{1..4} /srv/node for S in {1..4}; do mkdir -p /srv/node/${S}/sdb${S}; done chown -R swift:swift /etc/swift/ /srv/node/ chown -R swift:swift /etc/swift/ /srv/node/{1..4} chown -R swift:swift /etc/swift/ /srv/node/{1..4}/* restorecon -R /srv

As I mentioned earlier I set "devices = /srv/node/1" in the account-server.conf, object-server.conf and container-server.conf.

Do you see any issues with the above procedure?

edit flag offensive delete link more
0

answered 2013-12-04 22:15:35 -0600

Yes, you need to now have device "names" that you use with your ring files that are directories under /srv/node/*/.

And since you don't have individual device mount points, you should probably set "mount_check = false" for account, container and object servers as well.

Are you following the SAIO procedure some how?

How are you building the rings?

edit flag offensive delete link more
0

answered 2013-12-04 22:22:21 -0600

vigneshb4u gravatar image

Sorry I didn't get what you meant in the first line. Could you please elaborate?

I suppose by device "names", you mean "sdb1" in my case, and I have already created under those directories. for S in {1..4}; do mkdir -p /srv/node/${S}/sdb${S}; done

Yes I am following SAIO procedure for a sandbox setup.

I built the rings as follows: swift-ring-builder /etc/swift/object.builder create 18 3 1 swift-ring-builder /etc/swift/container.builder create 18 3 1 swift-ring-builder /etc/swift/account.builder create 18 3 1 swift-ring-builder /etc/swift/account.builder add z1-127.0.0.1:6002/sdb1 1 swift-ring-builder /etc/swift/container.builder add z1-127.0.0.1:6001/sdb1 1 swift-ring-builder /etc/swift/object.builder add z1-127.0.0.1:6000/sdb1 1 swift-ring-builder /etc/swift/account.builder rebalance swift-ring-builder /etc/swift/container.builder rebalance swift-ring-builder /etc/swift/object.builder rebalance

edit flag offensive delete link more
0

answered 2013-12-04 22:32:25 -0600

vigneshb4u gravatar image

Gosh! This problem got resolved after I set "mount_check = false" for account, container and object servers.

swift stat

Account: AUTH_2dd248ff7c05446b801eba44c082c133 Containers: 0 Objects: 0 Bytes: 0 Accept-Ranges: bytes X-Timestamp: 1386196007.43202 Content-Type: text/plain; charset=utf-8

Nowhere in the configuration guide, "mount_check = false" settings was mentioned. Was stuck with this problem for 4 days.

Thanks a lot Peter, for all your time and expertise.

I didn't understand what you meant by "since you don't have individual device mount points". Do you mind elborating?

edit flag offensive delete link more
0

answered 2013-12-04 22:52:45 -0600

I am not sure how closely you have followed the SAIO procedures, but the .conf files from the SAIO document are supposed to have mount_check=false in them. So you might double check the SAIO procedure you are following to be sure you have copied the contents of the various .conf files into the actual ones on your system.

Outside of SAIO documentation, one actually sets up Swift by using real disks for the storage, and so you use mount points under the path specified by the "devices" configuration file parameter.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2013-12-04 14:53:25 -0600

Seen: 1,045 times

Last updated: Dec 04 '13