Revision history [back]

problem with new swift storage node

Hi,

im running a swift enviroment with about 3 Servers. I added a new one yersterday and get some trouble with it.

Maybe there is a way to fix it?

Thanks!

Log

Feb 11 20:58:06 cloud03 account-server: ERROR __call__ error with REPLICATE /sdb/9762/988822bcb7816b9a3e617a2a38e19940 : #012Traceback (most recent call last):#012 File "/usr/lib/python2.7/dist-packages/swift/account/server.py", line 269, in __call__#012 res = getattr(self, req.method)(req)#012 File "/usr/lib/python2.7/dist-packages/swift/common/utils.py", line 1672, in _timing_stats#012 resp = func(ctrl, args, *kwargs)#012 File "/usr/lib/python2.7/dist-packages/swift/account/server.py", line 234, in REPLICATE#012 ret = self.replicator_rpc.dispatch(post_args, args)#012 File "/usr/lib/python2.7/dist-packages/swift/common/db_replicator.py", line 708, in dispatch#012 return self.complete_rsync(drive, db_file, args)#012 File "/usr/lib/python2.7/dist-packages/swift/common/db_replicator.py", line 817, in complete_rsync#012 broker.newid(args[0])#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 427, in newid#012 ''' % self.db_type, (str(uuid4()),))#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 134, in execute#012 self.timeout, self.db_file, lambda: sqlite3.Cursor.execute(#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 72, in _db_timeout#012 return call()#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 135, in <lambda>#012 self, args, *kwargs))#012OperationalError: no such table: account_stat Feb 11 20:58:06 cloud03 account-server: ERROR __call__ error with REPLICATE /sdb/12107/bd2d86db130e0683c8eed777e7512146 : #012Traceback (most recent call last):#012 File "/usr/lib/python2.7/dist-packages/swift/account/server.py", line 269, in __call__#012 res = getattr(self, req.method)(req)#012 File "/usr/lib/python2.7/dist-packages/swift/common/utils.py", line 1672, in _timing_stats#012 resp = func(ctrl, args, *kwargs)#012 File "/usr/lib/python2.7/dist-packages/swift/account/server.py", line 234, in REPLICATE#012 ret = self.replicator_rpc.dispatch(post_args, args)#012 File "/usr/lib/python2.7/dist-packages/swift/common/db_replicator.py", line 708, in dispatch#012 return self.complete_rsync(drive, db_file, args)#012 File "/usr/lib/python2.7/dist-packages/swift/common/db_replicator.py", line 817, in complete_rsync#012 broker.newid(args[0])#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 427, in newid#012 ''' % self.db_type, (str(uuid4()),))#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 134, in execute#012 self.timeout, self.db_file, lambda: sqlite3.Cursor.execute(#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 72, in _db_timeout#012 return call()#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 135, in <lambda>#012 self, args, *kwargs))#012OperationalError: no such table: account_stat Feb 11 20:58:06 cloud03 account-server: ERROR __call__ error with REPLICATE /sdb/7120/6f402a36776fdbb206f4a0be432e5be5 : #012Traceback (most recent call last):#012 File "/usr/lib/python2.7/dist-packages/swift/account/server.py", line 269, in __call__#012 res = getattr(self, req.method)(req)#012 File "/usr/lib/python2.7/dist-packages/swift/common/utils.py", line 1672, in _timing_stats#012 resp = func(ctrl, args, *kwargs)#012 File "/usr/lib/python2.7/dist-packages/swift/account/server.py", line 234, in REPLICATE#012 ret = self.replicator_rpc.dispatch(post_args, args)#012 File "/usr/lib/python2.7/dist-packages/swift/common/db_replicator.py", line 708, in dispatch#012 return self.complete_rsync(drive, db_file, args)#012 File "/usr/lib/python2.7/dist-packages/swift/common/db_replicator.py", line 817, in complete_rsync#012 broker.newid(args[0])#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 427, in newid#012 ''' % self.db_type, (str(uuid4()),))#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 134, in execute#012 self.timeout, self.db_file, lambda: sqlite3.Cursor.execute(#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 72, in _db_timeout#012 return call()#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 135, in <lambda>#012 self, args, *kwargs))#012OperationalError: no such table: account_stat Feb 11 20:58:06 cloud03 account-server: ERROR __call__ error with REPLICATE /sdb/12238/bf39032008a75bc4e955cb526ae39807 : #012Traceback (most recent call last):#012 File "/usr/lib/python2.7/dist-packages/swift/account/server.py", line 269, in __call__#012 res = getattr(self, req.method)(req)#012 File "/usr/lib/python2.7/dist-packages/swift/common/utils.py", line 1672, in _timing_stats#012 resp = func(ctrl, args, *kwargs)#012 File "/usr/lib/python2.7/dist-packages/swift/account/server.py", line 234, in REPLICATE#012 ret = self.replicator_rpc.dispatch(post_args, args)#012 File "/usr/lib/python2.7/dist-packages/swift/common/db_replicator.py", line 708, in dispatch#012 return self.complete_rsync(drive, db_file, args)#012 File "/usr/lib/python2.7/dist-packages/swift/common/db_replicator.py", line 817, in complete_rsync#012 broker.newid(args[0])#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 427, in newid#012 ''' % self.db_type, (str(uuid4()),))#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 134, in execute#012 self.timeout, self.db_file, lambda: sqlite3.Cursor.execute(#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 72, in _db_timeout#012 return call()#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 135, in <lambda>#012 self, args, *kwargs))#012OperationalError: no such table: account_stat Feb 11 20:58:06 cloud03 account-server: ERROR __call__ error with REPLICATE /sdb/12170/be2b5de15f841caa12599c449d528173 : #012Traceback (most recent call last):#012 File "/usr/lib/python2.7/dist-packages/swift/account/server.py", line 269, in __call__#012 res = getattr(self, req.method)(req)#012 File "/usr/lib/python2.7/dist-packages/swift/common/utils.py", line 1672, in _timing_stats#012 resp = func(ctrl, args, *kwargs)#012 File "/usr/lib/python2.7/dist-packages/swift/account/server.py", line 234, in REPLICATE#012 ret = self.replicator_rpc.dispatch(post_args, args)#012 File "/usr/lib/python2.7/dist-packages/swift/common/db_replicator.py", line 708, in dispatch#012 return self.complete_rsync(drive, db_file, args)#012 File "/usr/lib/python2.7/dist-packages/swift/common/db_replicator.py", line 817, in complete_rsync#012 broker.newid(args[0])#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 427, in newid#012 ''' % self.db_type, (str(uuid4()),))#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 134, in execute#012 self.timeout, self.db_file, lambda: sqlite3.Cursor.execute(#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 72, in _db_timeout#012 return call()#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 135, in <lambda>#012 self, args, *kwargs))#012OperationalError: no such table: account_stat Feb 11 20:58:06 cloud03 account-server: ERROR __call__ error with REPLICATE /sdb/12207/bebd3295bc14576d4ec89e1398571632 : #012Traceback (most recent call last):#012 File "/usr/lib/python2.7/dist-packages/swift/account/server.py", line 269, in __call__#012 res = getattr(self, req.method)(req)#012 File "/usr/lib/python2.7/dist-packages/swift/common/utils.py", line 1672, in _timing_stats#012 resp = func(ctrl, args, *kwargs)#012 File "/usr/lib/python2.7/dist-packages/swift/account/server.py", line 234, in REPLICATE#012 ret = self.replicator_rpc.dispatch(post_args, args)#012 File "/usr/lib/python2.7/dist-packages/swift/common/db_replicator.py", line 708, in dispatch#012 return self.complete_rsync(drive, db_file, args)#012 File "/usr/lib/python2.7/dist-packages/swift/common/db_replicator.py", line 817, in complete_rsync#012 broker.newid(args[0])#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 427, in newid#012 ''' % self.db_type, (str(uuid4()),))#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 134, in execute#012 self.timeout, self.db_file, lambda: sqlite3.Cursor.execute(#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 72, in _db_timeout#012 return call()#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 135, in <lambda>#012 self, args, *kwargs))#012OperationalError: no such table: account_stat Feb 11 20:58:07 cloud03 account-server: ERROR __call__ error with REPLICATE /sdb/12004/bb909d08fa79965576872be9310f24ee : #012Traceback (most recent call last):#012 File "/usr/lib/python2.7/dist-packages/swift/account/server.py", line 269, in __call__#012 res = getattr(self, req.method)(req)#012 File "/usr/lib/python2.7/dist-packages/swift/common/utils.py", line 1672, in _timing_stats#012 resp = func(ctrl, args, *kwargs)#012 File "/usr/lib/python2.7/dist-packages/swift/account/server.py", line 234, in REPLICATE#012 ret = self.replicator_rpc.dispatch(post_args, args)#012 File "/usr/lib/python2.7/dist-packages/swift/common/db_replicator.py", line 708, in dispatch#012 return self.complete_rsync(drive, db_file, args)#012 File "/usr/lib/python2.7/dist-packages/swift/common/db_replicator.py", line 817, in complete_rsync#012 broker.newid(args[0])#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 427, in newid#012 ''' % self.db_type, (str(uuid4()),))#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 134, in execute#012 self.timeout, self.db_file, lambda: sqlite3.Cursor.execute(#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 72, in _db_timeout#012 return call()#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 135, in <lambda>#012 self, args, *kwargs))#012OperationalError: no such table: account_stat Feb 12 20:58:07 cloud03 container-replicator: Can't find itself 127.0.0.1, ::1, 192.168.0.209, fe80::250:ccff:fe6b:a723 with port 6001 in ring file, not replicating Feb 12 20:58:07 cloud03 container-replicator: Beginning replication run Feb 12 20:58:07 cloud03 container-replicator: Replication run OVER Feb 12 20:58:07 cloud03 container-replicator: Attempted to replicate 0 dbs in 0.00153 seconds (0.00000/s) Feb 12 20:58:07 cloud03 container-replicator: Removed 0 dbs Feb 12 20:58:07 cloud03 container-replicator: 0 successes, 0 failures Feb 12 20:58:07 cloud03 container-replicator: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0