Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

cinder cintainers are restarting again again

Using openstack kolla as my infrastructure. trying to bring cinder service up but cinder containers (cinder_scheduler, cinder_volume, cinder_backup and iscsid) are restarting again n again.

globals.yml

enable_cinder: "yes" enable_cinder_backend_lvm: "yes"

Could you please guide me on the same.

cinder cintainers are restarting again again

Using openstack kolla as my infrastructure. trying to bring cinder service up but cinder containers (cinder_scheduler, cinder_volume, cinder_backup and iscsid) are restarting again n again.

globals.yml

enable_cinder: "yes" enable_cinder_backend_lvm: "yes"

Could you please guide me on the same.

cinder cintainers containers are restarting again again

Using openstack kolla as my infrastructure. trying to bring cinder service up but cinder containers (cinder_scheduler, cinder_volume, cinder_backup and iscsid) are restarting again n again.

globals.yml

#globals.yml
enable_cinder: "yes"
enable_cinder_backend_lvm: "yes"

"yes"

Could you please guide me on the same.

logs for your reference

cinder-backup.log

2019-05-14 17:52:09.233 6 INFO cinder.cmd.backup [-] Backup running in single process mode.
2019-05-14 17:52:09.739 6 CRITICAL cinder [req-2eb0d613-3509-49f1-8927-fd6859437616 - - - - -] Unhandled error: ProgrammingError: (pymysql.err.ProgrammingError) (1146, u"Table 'cinder.services' doesn't exist") [SQL: u'SELECT services.created_at AS services_created_at, services.deleted_at AS services_deleted_at, services.deleted AS services_deleted, services.id AS services_id, services.uuid AS services_uuid, services.cluster_name AS services_cluster_name, services.host AS services_host, services.`binary` AS services_binary, services.updated_at AS services_updated_at, services.topic AS services_topic, services.report_count AS services_report_count, services.disabled AS services_disabled, services.availability_zone AS services_availability_zone, services.disabled_reason AS services_disabled_reason, services.modified_at AS services_modified_at, services.rpc_current_version AS services_rpc_current_version, services.object_current_version AS services_object_current_version, services.replication_status AS services_replication_status, services.active_backend_id AS services_active_backend_id, services.frozen AS services_frozen \nFROM services \nWHERE services.deleted = false AND services.`binary` = %(binary_1)s'] [parameters: {u'binary_1': 'cinder-backup'}] (Background on this error at: http://sqlalche.me/e/f405)
2019-05-14 17:52:09.739 6 ERROR cinder Traceback (most recent call last):
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/bin/cinder-backup", line 10, in <module>
2019-05-14 17:52:09.739 6 ERROR cinder     sys.exit(main())
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/cmd/backup.py", line 112, in main
2019-05-14 17:52:09.739 6 ERROR cinder     process_number=1)
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/service.py", line 392, in create
2019-05-14 17:52:09.739 6 ERROR cinder     cluster=cluster, **kwargs)
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/service.py", line 155, in __init__
2019-05-14 17:52:09.739 6 ERROR cinder     *args, **kwargs)
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/backup/manager.py", line 117, in __init__
2019-05-14 17:52:09.739 6 ERROR cinder     self.backup_rpcapi = backup_rpcapi.BackupAPI()
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/rpc.py", line 207, in __init__
2019-05-14 17:52:09.739 6 ERROR cinder     obj_version_cap = self.determine_obj_version_cap()
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/rpc.py", line 260, in determine_obj_version_cap
2019-05-14 17:52:09.739 6 ERROR cinder     cinder.context.get_admin_context(), cls.BINARY)
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/objects/service.py", line 222, in get_minimum_obj_version
2019-05-14 17:52:09.739 6 ERROR cinder     binary)
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/objects/service.py", line 192, in _get_minimum_version
2019-05-14 17:52:09.739 6 ERROR cinder     services = ServiceList.get_all_by_binary(context, binary)
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/objects/service.py", line 256, in get_all_by_binary
2019-05-14 17:52:09.739 6 ERROR cinder     disabled=disabled)
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/db/api.py", line 141, in service_get_all
2019-05-14 17:52:09.739 6 ERROR cinder     return IMPL.service_get_all(context, backend_match_level, **filters)
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 165, in wrapper
2019-05-14 17:52:09.739 6 ERROR cinder     return f(*args, **kwargs)
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 526, in service_get_all
2019-05-14 17:52:09.739 6 ERROR cinder     return [] if not query else query.all()
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2836, in all
2019-05-14 17:52:09.739 6 ERROR cinder     return list(self)
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2988, in __iter__
2019-05-14 17:52:09.739 6 ERROR cinder     return self._execute_and_instances(context)
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 3011, in _execute_and_instances
2019-05-14 17:52:09.739 6 ERROR cinder     result = conn.execute(querycontext.statement, self._params)
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 948, in execute
2019-05-14 17:52:09.739 6 ERROR cinder     return meth(self, multiparams, params)
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection
2019-05-14 17:52:09.739 6 ERROR cinder     return connection._execute_clauseelement(self, multiparams, params)
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1060, in _execute_clauseelement
2019-05-14 17:52:09.739 6 ERROR cinder     compiled_sql, distilled_params
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
2019-05-14 17:52:09.739 6 ERROR cinder     context)
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1409, in _handle_dbapi_exception
2019-05-14 17:52:09.739 6 ERROR cinder     util.raise_from_cause(newraise, exc_info)
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 265, in raise_from_cause
2019-05-14 17:52:09.739 6 ERROR cinder     reraise(type(exception), exception, tb=exc_tb, cause=cause)
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
2019-05-14 17:52:09.739 6 ERROR cinder     context)
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 509, in do_execute
2019-05-14 17:52:09.739 6 ERROR cinder     cursor.execute(statement, parameters)
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/pymysql/cursors.py", line 170, in execute
2019-05-14 17:52:09.739 6 ERROR cinder     result = self._query(query)
2019-05-14 17:52:09.739 6 ERROR cinder   File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/pymysql/cursors.py", line 328, in _query
2019-05-14 17:52:09.739 6 ERROR cinder     conn.query(q)

docker logs iscsid

+ CMD='iscsid -d 8 -f --pid=/run/iscsid.pid'
+ ARGS=
+ [[ ! -n '' ]]
+ . kolla_extend_start
++ [[ ! -f /etc/iscsi/initiatorname.iscsi ]]
+ echo 'Running command: '\''iscsid -d 8 -f --pid=/run/iscsid.pid'\'''
+ exec iscsid -d 8 -f --pid=/run/iscsid.pid
Running command: 'iscsid -d 8 -f --pid=/run/iscsid.pid'
iscsid: sysfs_init: sysfs_path='/sys'
iscsid: Can not bind IPC socket
+ sudo -E kolla_set_configs
INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json
INFO:__main__:Validating config file
INFO:__main__:Kolla config strategy set to: COPY_ALWAYS
INFO:__main__:Copying service configuration files
INFO:__main__:Writing out command to execute
++ cat /run_command
+ CMD='iscsid -d 8 -f --pid=/run/iscsid.pid'
+ ARGS=
+ [[ ! -n '' ]]
+ . kolla_extend_start
++ [[ ! -f /etc/iscsi/initiatorname.iscsi ]]
+ echo 'Running command: '\''iscsid -d 8 -f --pid=/run/iscsid.pid'\'''
Running command: 'iscsid -d 8 -f --pid=/run/iscsid.pid'
+ exec iscsid -d 8 -f --pid=/run/iscsid.pid
iscsid: sysfs_init: sysfs_path='/sys'
iscsid: Can not bind IPC socket
+ sudo -E kolla_set_configs
INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json
INFO:__main__:Validating config file
INFO:__main__:Kolla config strategy set to: COPY_ALWAYS
INFO:__main__:Copying service configuration files
INFO:__main__:Writing out command to execute
++ cat /run_command
+ CMD='iscsid -d 8 -f --pid=/run/iscsid.pid'
+ ARGS=
+ [[ ! -n '' ]]
+ . kolla_extend_start
++ [[ ! -f /etc/iscsi/initiatorname.iscsi ]]
+ echo 'Running command: '\''iscsid -d 8 -f --pid=/run/iscsid.pid'\'''
Running command: 'iscsid -d 8 -f --pid=/run/iscsid.pid'
+ exec iscsid -d 8 -f --pid=/run/iscsid.pid
iscsid: sysfs_init: sysfs_path='/sys'
iscsid: Can not bind IPC socket
+ sudo -E kolla_set_configs
INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json
INFO:__main__:Validating config file
INFO:__main__:Kolla config strategy set to: COPY_ALWAYS
INFO:__main__:Copying service configuration files
INFO:__main__:Writing out command to execute
++ cat /run_command
+ CMD='iscsid -d 8 -f --pid=/run/iscsid.pid'
+ ARGS=
+ [[ ! -n '' ]]
+ . kolla_extend_start
++ [[ ! -f /etc/iscsi/initiatorname.iscsi ]]
+ echo 'Running command: '\''iscsid -d 8 -f --pid=/run/iscsid.pid'\'''
Running command: 'iscsid -d 8 -f --pid=/run/iscsid.pid'
+ exec iscsid -d 8 -f --pid=/run/iscsid.pid
iscsid: sysfs_init: sysfs_path='/sys'
iscsid: Can not bind IPC socket