这篇文章主要介绍了登录openstack的dashboard发现无法启动云主机怎么办,具有一定借鉴价值,感兴趣的朋友可以参考下,希望大家阅读完这篇文章之后大有收获,下面让小编带着大家一起了解一下。
测试机器每天晚上都要关机,每天上午开机的时候总会出现一些莫名其妙的问题,登录openstack的dashboard,发现无法启动云主机,创建云硬盘失败。
查看底层存储ceph的状态,发现好几个osd并未启动,于是启动关闭的osd,ceph集群进入健康状态。
ceph集群一共是三台机器216、217和218,其中216机器有些问题,总是启动不来,但它又是主mon,导致217和218上的osd一直连接不上216的mon会被标记为down的状态,只有在216正常启动后再启动217和218的osd才会正常。
# view /var/log/cinder/volume.log
看到如下错误:
。。。
2016-08-02 15:01:44.865 9622 ERROR cinder.utils [req-0706d398-b2bf-4925-9613-04bdb1deb8f9 6d8248aff00546bb86f439342780e770 a3f5a48418e449578f9c0c56105287cc - - -] Volume driver RBDDriver not initialized
2016-08-02 15:01:44.866 9622 ERROR oslo_messaging.rpc.dispatcher [req-0706d398-b2bf-4925-9613-04bdb1deb8f9 6d8248aff00546bb86f439342780e770 a3f5a48418e449578f9c0c56105287cc - - -] Exception during message handling: Volume driver not ready.
2016-08-02 15:01:44.866 9622 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2016-08-02 15:01:44.866 9622 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2016-08-02 15:01:44.866 9622 TRACE oslo_messaging.rpc.dispatcher executor_callback))
2016-08-02 15:01:44.866 9622 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2016-08-02 15:01:44.866 9622 TRACE oslo_messaging.rpc.dispatcher executor_callback)
2016-08-02 15:01:44.866 9622 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
2016-08-02 15:01:44.866 9622 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2016-08-02 15:01:44.866 9622 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 105, in wrapper
2016-08-02 15:01:44.866 9622 TRACE oslo_messaging.rpc.dispatcher return f(*args, **kwargs)
2016-08-02 15:01:44.866 9622 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1076, in initialize_connection
2016-08-02 15:01:44.866 9622 TRACE oslo_messaging.rpc.dispatcher utils.require_driver_initialized(self.driver)
2016-08-02 15:01:44.866 9622 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 598, in require_driver_initialized
2016-08-02 15:01:44.866 9622 TRACE oslo_messaging.rpc.dispatcher raise exception.DriverNotInitialized()
2016-08-02 15:01:44.866 9622 TRACE oslo_messaging.rpc.dispatcher DriverNotInitialized: Volume driver not ready.
2016-08-02 15:01:44.866 9622 TRACE oslo_messaging.rpc.dispatcher
2016-08-02 15:01:44.870 9622 INFO oslo_messaging._drivers.impl_rabbit [req-0706d398-b2bf-4925-9613-04bdb1deb8f9 6d8248aff00546bb86f439342780e770 a3f5a48418e449578f9c0c56105287cc - - -] Connecting to AMQP server on 192.168.8.217:5672
2016-08-02 15:01:44.883 9622 INFO oslo_messaging._drivers.impl_rabbit [req-0706d398-b2bf-4925-9613-04bdb1deb8f9 6d8248aff00546bb86f439342780e770 a3f5a48418e449578f9c0c56105287cc - - -] Connected to AMQP server on 192.168.8.217:5672
2016-08-02 15:01:44.885 9622 ERROR oslo_messaging._drivers.common [req-0706d398-b2bf-4925-9613-04bdb1deb8f9 6d8248aff00546bb86f439342780e770 a3f5a48418e449578f9c0c56105287cc - - -] Returning exception Volume driver not ready. to caller
2016-08-02 15:01:44.885 9622 ERROR oslo_messaging._drivers.common [req-0706d398-b2bf-4925-9613-04bdb1deb8f9 6d8248aff00546bb86f439342780e770 a3f5a48418e449578f9c0c56105287cc - - -] ['Traceback (most recent call last):\n', ' File "/usr/
lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply\n executor_callback))\n', ' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch\n execut
or_callback)\n', ' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch\n result = func(ctxt, **new_args)\n', ' File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 105
, in wrapper\n return f(*args, **kwargs)\n', ' File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1076, in initialize_connection\n utils.require_driver_initialized(self.driver)\n', ' File "/usr/lib/python2.7/
site-packages/cinder/utils.py", line 598, in require_driver_initialized\n raise exception.DriverNotInitialized()\n', 'DriverNotInitialized: Volume driver not ready.\n']
2016-08-02 15:02:42.248 9622 INFO cinder.volume.manager [req-fe77cb0b-b8e0-4abc-a6a4-3127ec2dcbc3 - - - - -] Updating volume status
2016-08-02 15:02:42.250 9622 WARNING cinder.volume.manager [req-fe77cb0b-b8e0-4abc-a6a4-3127ec2dcbc3 - - - - -] Unable to update stats, RBDDriver -1.1.0 driver is uninitialized.
2016-08-02 15:03:42.249 9622 INFO cinder.volume.manager [req-eca34b20-dca4-4ce4-a9aa-0d782076aca4 - - - - -] Updating volume status
2016-08-02 15:03:42.251 9622 WARNING cinder.volume.manager [req-eca34b20-dca4-4ce4-a9aa-0d782076aca4 - - - - -] Unable to update stats, RBDDriver -1.1.0 driver is uninitialized.
。。。
其中出现次数最多的就是 RBDDriver -1.1.0 driver is uninitialized.
往上翻日志,可以追溯到机器刚启动的时候,发现是由于连接不上ceph集群,导致 RBDDriver -1.1.0 driver 未初始化。
2016-08-02 08:47:41.014 9622 INFO cinder.volume.manager [req-72b8c8a4-a831-4395-80fc-f18a6535ce90 - - - - -] Starting volume driver RBDDriver (1.1.0)
2016-08-02 08:52:41.074 9622 ERROR cinder.volume.drivers.rbd [req-72b8c8a4-a831-4395-80fc-f18a6535ce90 - - - - -] Error connecting to ceph cluster.
2016-08-02 08:52:41.074 9622 TRACE cinder.volume.drivers.rbd Traceback (most recent call last):
2016-08-02 08:52:41.074 9622 TRACE cinder.volume.drivers.rbd File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 314, in _connect_to_rados
2016-08-02 08:52:41.074 9622 TRACE cinder.volume.drivers.rbd client.connect()
2016-08-02 08:52:41.074 9622 TRACE cinder.volume.drivers.rbd File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
2016-08-02 08:52:41.074 9622 TRACE cinder.volume.drivers.rbd TimedOut: error connecting to the cluster
2016-08-02 08:52:41.074 9622 TRACE cinder.volume.drivers.rbd
2016-08-02 08:52:41.080 9622 ERROR cinder.volume.manager [req-72b8c8a4-a831-4395-80fc-f18a6535ce90 - - - - -] Error encountered during initialization of driver: RBDDriver
2016-08-02 08:52:41.081 9622 ERROR cinder.volume.manager [req-72b8c8a4-a831-4395-80fc-f18a6535ce90 - - - - -] Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
2016-08-02 08:52:41.081 9622 TRACE cinder.volume.manager Traceback (most recent call last):
2016-08-02 08:52:41.081 9622 TRACE cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 302, in init_host
2016-08-02 08:52:41.081 9622 TRACE cinder.volume.manager self.driver.check_for_setup_error()
2016-08-02 08:52:41.081 9622 TRACE cinder.volume.manager File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 105, in wrapper
2016-08-02 08:52:41.081 9622 TRACE cinder.volume.manager return f(*args, **kwargs)
2016-08-02 08:52:41.081 9622 TRACE cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 287, in check_for_setup_error
2016-08-02 08:52:41.081 9622 TRACE cinder.volume.manager with RADOSClient(self):
2016-08-02 08:52:41.081 9622 TRACE cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 242, in __init__
2016-08-02 08:52:41.081 9622 TRACE cinder.volume.manager self.cluster, self.ioctx = driver._connect_to_rados(pool)
2016-08-02 08:52:41.081 9622 TRACE cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 322, in _connect_to_rados
2016-08-02 08:52:41.081 9622 TRACE cinder.volume.manager raise exception.VolumeBackendAPIException(data=msg)
2016-08-02 08:52:41.081 9622 TRACE cinder.volume.manager VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
2016-08-02 08:52:41.081 9622 TRACE cinder.volume.manager
2016-08-02 08:52:41.127 9622 INFO oslo_messaging._drivers.impl_rabbit [req-d845763e-483f-4495-8dc4-70fb2757d522 - - - - -] Connecting to AMQP server on 192.168.8.217:5672
2016-08-02 08:52:41.146 9622 INFO oslo_messaging._drivers.impl_rabbit [req-d845763e-483f-4495-8dc4-70fb2757d522 - - - - -] Connected to AMQP server on 192.168.8.217:5672
2016-08-02 08:52:42.163 9622 INFO cinder.volume.manager [req-8e733ab2-45c7-41c6-b89f-46ad5319d291 - - - - -] Updating volume status
2016-08-02 08:52:42.164 9622 WARNING cinder.volume.manager [req-8e733ab2-45c7-41c6-b89f-46ad5319d291 - - - - -] Unable to update stats, RBDDriver -1.1.0 driver is uninitialized.
2016-08-02 08:53:42.163 9622 INFO cinder.volume.manager [req-89715cce-2cc6-44a7-a80f-c7da27d5074a - - - - -] Updating volume status
2016-08-02 08:53:42.165 9622 WARNING cinder.volume.manager [req-89715cce-2cc6-44a7-a80f-c7da27d5074a - - - - -] Unable to update stats, RBDDriver -1.1.0 driver is uninitialized.
2016-08-02 08:54:42.164 9622 INFO cinder.volume.manager [req-42e4bb17-c2b7-4800-bf86-9f80947d3ca9 - - - - -] Updating volume status
2016-08-02 08:54:42.165 9622 WARNING cinder.volume.manager [req-42e4bb17-c2b7-4800-bf86-9f80947d3ca9 - - - - -] Unable to update stats, RBDDriver -1.1.0 driver is uninitialized.
虽然现在集群已经恢复正常了,但是RBDDriver还是不能正常工作,这里就需要重启一下cinder的服务才行。
# systemctl restart openstack-cinder-api openstack-cinder-scheduler openstack-cinder-volume
重启完服务,等待一下再查看日志
2016-08-02 15:10:08.805 1953 INFO cinder.openstack.common.service [req-01aa405c-6b1b-4456-b38b-22fc86a74ba9 - - - - -] Caught SIGTERM, stopping children
2016-08-02 15:10:08.808 1953 INFO cinder.openstack.common.service [req-01aa405c-6b1b-4456-b38b-22fc86a74ba9 - - - - -] Waiting on 1 children to exit
2016-08-02 15:10:08.809 1953 INFO cinder.openstack.common.service [req-01aa405c-6b1b-4456-b38b-22fc86a74ba9 - - - - -] Child 9622 killed by signal 15
2016-08-02 15:10:10.046 59977 INFO cinder.volume.manager [req-2cae44c1-861c-43ab-9b5a-d4b71453daeb - - - - -] Determined volume DB was not empty at startup.
2016-08-02 15:10:10.066 59977 INFO cinder.openstack.common.service [req-2cae44c1-861c-43ab-9b5a-d4b71453daeb - - - - -] Starting 1 workers
2016-08-02 15:10:10.070 59977 INFO cinder.openstack.common.service [req-2cae44c1-861c-43ab-9b5a-d4b71453daeb - - - - -] Started child 60005
2016-08-02 15:10:10.071 60005 INFO cinder.service [-] Starting cinder-volume node (version 2015.1.2)
2016-08-02 15:10:10.073 60005 INFO cinder.volume.manager [req-dbf976a6-1253-4022-a52c-3a8ce3f1f5ae - - - - -] Starting volume driver RBDDriver (1.1.0)
2016-08-02 15:10:10.178 60005 INFO cinder.volume.manager [req-dbf976a6-1253-4022-a52c-3a8ce3f1f5ae - - - - -] Updating volume status
2016-08-02 15:10:10.226 60005 INFO oslo_messaging._drivers.impl_rabbit [req-dbf976a6-1253-4022-a52c-3a8ce3f1f5ae - - - - -] Connecting to AMQP server on 192.168.8.217:5672
2016-08-02 15:10:10.240 60005 INFO oslo_messaging._drivers.impl_rabbit [req-dbf976a6-1253-4022-a52c-3a8ce3f1f5ae - - - - -] Connected to AMQP server on 192.168.8.217:5672
2016-08-02 15:10:10.290 60005 INFO oslo_messaging._drivers.impl_rabbit [req-c1532ac0-ef22-424f-ba60-02b1de8967c9 - - - - -] Connecting to AMQP server on 192.168.8.217:5672
2016-08-02 15:10:10.303 60005 INFO oslo_messaging._drivers.impl_rabbit [req-c1532ac0-ef22-424f-ba60-02b1de8967c9 - - - - -] Connected to AMQP server on 192.168.8.217:5672
2016-08-02 15:11:43.288 60005 INFO cinder.volume.flows.manager.create_volume [req-e2cf8138-38dd-4812-98d7-a8ec27b53800 6d8248aff00546bb86f439342780e770 a3f5a48418e449578f9c0c56105287cc - - -] Volume 37b011a3-f5f2-4bb2-8cb6-e646bf955ce6:
being created as image with specification: {'status': u'creating', 'image_location': (None, None), 'volume_size': 1, 'volume_name': u'volume-37b011a3-f5f2-4bb2-8cb6-e646bf955ce6', 'image_id': u'5008d784-fa11-45d3-b910-653c7f2420fc', 'i
mage_service': <cinder.image.glance.GlanceImageService object at 0x4dd80d0>, 'image_meta': {'status': u'active', 'name': u'cirros_raw', 'deleted': False, 'container_format': u'bare', 'created_at': datetime.datetime(2016, 7, 28, 6, 15, 2
8, tzinfo=<iso8601.Utc>), 'disk_format': u'qcow2', 'updated_at': datetime.datetime(2016, 7, 28, 6, 15, 30, tzinfo=<iso8601.Utc>), 'id': u'5008d784-fa11-45d3-b910-653c7f2420fc', 'owner': u'aa61c20829e34bbeb67e9b8fc4b5a6b4', 'min_ram': 0,
'checksum': u'ee1eca47dc88f4879d8a229cc70a07c6', 'min_disk': 0, 'is_public': True, 'deleted_at': None, 'properties': {}, 'size': 13287936}}
2016-08-02 15:11:44.840 60005 INFO cinder.image.image_utils [req-e2cf8138-38dd-4812-98d7-a8ec27b53800 6d8248aff00546bb86f439342780e770 a3f5a48418e449578f9c0c56105287cc - - -] Image download 12.00 MB at 12.00 MB/s
2016-08-02 15:11:45.864 60005 INFO cinder.image.image_utils [req-e2cf8138-38dd-4812-98d7-a8ec27b53800 6d8248aff00546bb86f439342780e770 a3f5a48418e449578f9c0c56105287cc - - -] Converted 12.00 MB image at 12.00 MB/s
感谢你能够认真阅读完这篇文章,希望小编分享的“登录openstack的dashboard发现无法启动云主机怎么办”这篇文章对大家有帮助,同时也希望大家多多支持亿速云,关注亿速云行业资讯频道,更多相关知识等着你来学习!
免责声明:本站发布的内容(图片、视频和文字)以原创、转载和分享为主,文章观点不代表本网站立场,如果涉及侵权请联系站长邮箱:is@yisu.com进行举报,并提供相关证据,一经查实,将立刻删除涉嫌侵权内容。