Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

I had decrease the value of expiration from 86400 to 864 in the /etc/keystone/keystone.conf. Then database synchronization and start all the nova services and apache2 server.

This all did not helped me out. My cluster is still in waiting state from last 4 hour. My nova console-log for active instance (after cluster goes into error state) is as follows as:

Instance1: <-------------------------------------------------------------------------> .............................................................................. [ 0.407413] pnp: PnP ACPI: found 8 devices [ 0.408832] ACPI: ACPI bus type pnp unregistered [ 0.446954] NET: Registered protocol family 2 [ 0.448505] IP route cache hash table entries: 4096 (order: 3, 32768 bytes) [ 0.450843] TCP established hash table entries: 16384 (order: 6, 262144 bytes) [ 0.453757] TCP bind hash table entries: 16384 (order: 6, 262144 bytes) [ 0.456131] TCP: Hash tables configured (established 16384 bind 16384) [ 0.459077] TCP: reno registered [ 0.460269] UDP hash table entries: 256 (order: 1, 8192 bytes) [ 0.462055] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes) [ 0.464050] NET: Registered protocol family 1 [ 0.465487] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.467332] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.469206] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.471212] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11 [ 0.473788] audit: initializing netlink socket (disabled) [ 0.475512] type=2000 audit(1376301101.472:1): initialized [ 0.477389] Trying to unpack rootfs image as initramfs... [ 0.529422] HugeTLB registered 2 MB page size, pre-allocated 0 pages [ 0.533234] VFS: Disk quotas dquot_6.5.2 [ 0.534622] Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.544243] fuse init (API version 7.19) [ 0.545667] msgmni has been set to 939 [ 0.560223] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252) [ 0.562686] io scheduler noop registered [ 0.563987] io scheduler deadline registered (default) [ 0.572293] io scheduler cfq registered [ 0.573729] pci_hotplug: PCI Hot Plug PCI Core version: 0.5 [ 0.575454] pciehp: PCI Express Hot Plug Controller Driver version: 0.4 [ 0.577614] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 0.580044] ACPI: Power Button [PWRF] [ 0.583208] GHES: HEST is not enabled! [ 0.592651] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10 [ 0.600195] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled [ 0.624673] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A [ 0.674575] serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A [ 0.734591] 00:05: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A [ 0.779096] 00:06: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A [ 0.804373] Linux agpgart interface v0.103 [ 0.813653] brd: module loaded [ 0.815867] loop: module loaded [ 0.884614] vda: vda1 [ 0.912109] scsi0 : ata_piix [ 0.913322] scsi1 : ata_piix [ 0.914465] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc180 irq 14 [ 0.916462] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc188 irq 15 [ 0.919445] Fixed MDIO Bus: probed [ 0.920713] tun: Universal TUN/TAP device driver, 1.6 [ 0.922345] tun: (C) 1999-2004 Max Krasnyansky maxk@qualcomm.com [ 0.928295] PPP generic driver version 2.4.2 [ 0.936063] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver [ 0.938096] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver [ 0.939935] uhci_hcd: USB Universal Host Controller Interface driver [ 0.948243] uhci_hcd 0000:00:01.2: UHCI Host Controller [ 0.949875] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1 [ 0.952506] uhci_hcd 0000:00:01.2: irq 11, io base 0x0000c140 [ 0.954365] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001 [ 0.956464] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 0.958940] usb usb1: Product: UHCI Host Controller [ 0.960587] usb usb1: Manufacturer: Linux 3.5.0-31-generic uhci_hcd [ 0.962552] usb usb1: SerialNumber: 0000:00:01.2 [ 0.976323] hub 1-0:1.0: USB hub found [ 0.977725] hub 1-0:1.0: 2 ports detected [ 0.979290] usbcore: registered new interface driver libusual [ 0.988104] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 0.991840] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 1.000114] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 1.001988] mousedev: PS/2 mouse device common for all mice [ 1.004261] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 1.007236] rtc_cmos 00:01: RTC can wake from S4 [ 1.009431] rtc_cmos 00:01: rtc core: registered rtc_cmos as rtc0 [ 1.011520] rtc0: alarms up to one day, 114 bytes nvram, hpet irqs [ 1.020235] device-mapper: uevent: version 1.0.3 [ 1.021968] device-mapper: ioctl: 4.22.0-ioctl (2011-10-19) initialised: dm-devel@redhat.com [ 1.024862] cpuidle: using governor ladder [ 1.026275] cpuidle: using governor menu [ 1.027784] EFI Variables Facility v0.08 2004-May-17 [ 1.040144] ashmem: initialized [ 1.041575] TCP: cubic registered [ 1.043005] NET: Registered protocol family 10 [ 1.048564] NET: Registered protocol family 17 [ 1.050130] Key type dns_resolver registered [ 1.056166] registered taskstats version 1 [ 1.285058] Freeing initrd memory: 18524k freed [ 1.310450] usb 1-1: new full-speed USB device number 2 using uhci_hcd [ 1.320759] Key type trusted registered [ 1.330199] Key type encrypted registered [ 1.339076] Magic number: 9:206:879 [ 1.340740] rtc_cmos 00:01: setting system clock to 2013-08-12 09:51:41 UTC (1376301101) [ 1.343493] BIOS EDD facility v0.16 2004-Jun-25, 0 devices found [ 1.345448] EDD information not available. [ 1.348853] Freeing unused kernel memory: 932k freed [ 1.350895] Write protecting the kernel read-only data: 12288k [ 1.357383] Freeing unused kernel memory: 1472k freed [ 1.363047] Freeing unused kernel memory: 1132k freed Loading, please wait... [ 1.473090] Refined TSC clocksource calibration: 2394.002 MHz. [ 1.703454] usb 1-1: New USB device found, idVendor=0627, idProduct=0001 [ 1.705650] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=5 [ 1.708556] usb 1-1: Product: QEMU USB Tablet [ 1.710057] usb 1-1: Manufacturer: QEMU 1.0 [ 1.711524] usb 1-1: SerialNumber: 42 [ 2.136680] 8139cp: 8139cp: 10/100 PCI Ethernet driver v1.3 (Mar 22, 2004) [ 2.139182] ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 11 [ 2.176581] 8139cp 0000:00:03.0: eth0: RTL-8139C+ at 0xffffc9000015a000, fa:16:3e:67:9e:a7, IRQ 11 [ 2.200573] 8139too: 8139too Fast Ethernet driver 0.9.28 [ 2.444679] FDC 0 is a S82078B [ 2.464912] usbcore: registered new interface driver usbhid [ 2.466885] usbhid: USB HID core driver [ 2.482416] input: QEMU 1.0 QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/input/input2 [ 2.492436] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Pointer [QEMU 1.0 QEMU USB Tablet] on usb-0000:00:01.2-1/input0 FATAL: Error inserting vesafb (/lib/modules/3.5.0-31-generic/kernel/drivers/video/vesafb.ko): No such device error: unexpectedly disconnected from boot status daemon Begin: Loading essential drivers ... done. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done. Begin: Running /scripts/local-premount ... done. [ 3.446795] EXT4-fs (vda1): mounted filesystem with ordered data mode. Opts: (null) Begin: Running /scripts/local-bottom ... done. done. Begin: Running /scripts/init-bottom ... done. [ 5.602105] EXT4-fs (vda1): re-mounted. Opts: (null) mountall: Disconnected from Plymouth Cloud-init v. 0.7 running 'init-local' at Mon, 12 Aug 2013 09:51:57 +0000. Up 17.29 seconds. Cloud-init v. 0.7 running 'init' at Mon, 12 Aug 2013 09:51:59 +0000. Up 19.56 seconds. ci-info: ++++++++++++++++++++++++++Net device info++++++++++++++++++++++++++ ci-info: +--------+------+-----------+-----------------+-------------------+ ci-info: | Device | Up | Address | Mask | Hw-Address | ci-info: +--------+------+-----------+-----------------+-------------------+ ci-info: | lo | True | 127.0.0.1 | 255.0.0.0 | . | ci-info: | eth0 | True | 10.1.1.4 | 255.255.255.224 | fa:16:3e:67:9e:a7 | ci-info: +--------+------+-----------+-----------------+-------------------+ ci-info: +++++++++++++++++++++++++++++++Route info+++++++++++++++++++++++++++++++ ci-info: +-------+-------------+----------+-----------------+-----------+-------+ ci-info: | Route | Destination | Gateway | Genmask | Interface | Flags | ci-info: +-------+-------------+----------+-----------------+-----------+-------+ ci-info: | 0 | 0.0.0.0 | 10.1.1.3 | 0.0.0.0 | eth0 | UG | ci-info: | 1 | 10.1.1.0 | 0.0.0.0 | 255.255.255.224 | eth0 | U | ci-info: +-------+-------------+----------+-----------------+-----------+-------+ Generating public/private rsa key pair. Your identification has been saved in /etc/ssh/ssh_host_rsa_key. Your public key has been saved in /etc/ssh/ssh_host_rsa_key.pub. The key fingerprint is: d2:10:65:ff:b3:7b:c0:d7:38:cf:db:e6:c8:94:60:61 root@h-cluster-hadoop-master-001 The key's randomart image is: +--[ RSA 2048]----+ | ..o | | o . | | . . E | | o o . | | . S .= o | | . .o++..| | .oo+ | | +..=| | ..o++| +-----------------+ Generating public/private dsa key pair. Your identification has been saved in /etc/ssh/ssh_host_dsa_key. Your public key has been saved in /etc/ssh/ssh_host_dsa_key.pub. The key fingerprint is: 3f:4b:7a:6c:b2:6f:9f:d2:77:4e:0c:13:8b:3f:51:84 root@h-cluster-hadoop-master-001 The key's randomart image is: +--[ DSA 1024]----+ | ..| | E. | | . .| | . + | | S . = | | . . = | | .+. o o| | .o*o...o.| | oBooo. o.| +-----------------+ Generating public/private ecdsa key pair. Your identification has been saved in /etc/ssh/ssh_host_ecdsa_key. Your public key has been saved in /etc/ssh/ssh_host_ecdsa_key.pub. The key fingerprint is: 0b:a9:25:4f:16:d2:56:9e:8d:6f:92:1d:74:d0:5b:49 root@h-cluster-hadoop-master-001 The key's randomart image is: +--[ECDSA 256]---+ | . oo..E. | | . o = .. o | | . + + o o | | o o + .. | | . * S + | | B . + | | . . . | | | | | +-----------------+ * Starting AppArmor profiles [80G Cloud-init v. 0.7 running 'modules:config' at Mon, 12 Aug 2013 09:52:08 +0000. Up 28.09 seconds. Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd [74G[ OK ] [74G[ OK ]g iSCSI initiator service iscsid [80G * Setting up iSCSI targets [80G iscsiadm: No records found [74G[ OK ] [74G[ OK ]g network filesystems [80G landscape-client is not configured, please run landscape-config. + dib-first-boot + touch /var/log/first-boot.d.log + chmod 0600 /var/log/first-boot.d.log + run-parts /etc/first-boot.d Generating locales... en_US.UTF-8... up-to-date Generation complete. Cloud-init v. 0.7 running 'modules:final' at Mon, 12 Aug 2013 09:52:18 +0000. Up 38.04 seconds. /var/lib/cloud/instance/scripts/part-001: line 2: /home/test//.ssh/authorized_keys: No such file or directory /var/lib/cloud/instance/scripts/part-001: line 29: /home/test//.ssh/id_rsa: No such file or directory 2013-08-12 09:52:18,479 - util.py[WARNING]: Failed running /var/lib/cloud/instance/scripts/part-001 [1] 2013-08-12 09:52:18,526 - cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts) 2013-08-12 09:52:18,528 - util.py[WARNING]: Running scripts-user (<module 'cloudinit.config.cc_scripts_user'="" from="" '="" usr="" lib="" python2.7="" dist-packages="" cloudinit="" config="" cc_scripts_user.pyc'="">) failed 2013-08-12 09:52:18,615 - util.py[WARNING]: Running ssh-authkey-fingerprints (<module 'cloudinit.config.cc_ssh_authkey_fingerprints'="" from="" '="" usr="" lib="" python2.7="" dist-packages="" cloudinit="" config="" cc_ssh_authkey_fingerprints.pyc'="">) failed ec2: ec2: ############################################################# ec2: -----BEGIN SSH HOST KEY FINGERPRINTS----- ec2: 1024 3f:4b:7a:6c:b2:6f:9f:d2:77:4e:0c:13:8b:3f:51:84 root@h-cluster-hadoop-master-001 (DSA) ec2: 256 0b:a9:25:4f:16:d2:56:9e:8d:6f:92:1d:74:d0:5b:49 root@h-cluster-hadoop-master-001 (ECDSA) ec2: 2048 d2:10:65:ff:b3:7b:c0:d7:38:cf:db:e6:c8:94:60:61 root@h-cluster-hadoop-master-001 (RSA) ec2: -----END SSH HOST KEY FINGERPRINTS----- ec2: ############################################################# -----BEGIN SSH HOST KEY KEYS----- ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKsAux6imBWJSZrmbZkax72BHEdFWUxdghRHmChc2afgbhOvcwH/MtqdZb++Y95G8RUFuqSRnJ5PYBhbwQ5FYVM= root@h-cluster-hadoop-master-001 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCs4KsrodgccuMMJYUmW6fc/GKuMKn7yGf4lnq6Pn5ZVaMMzQ+3sLC8zRPqvnlsvBcz5s412ODSSqDQYhF8/+thTmmi6G+5aNLtY1jnAKjVfj/dyii4RY60cUQ4owxWhYGERFbBeT58G5NgA82f+oGbANosFzf2RF+13pq+fP/at4LdZYMxYEV88voV7c2m7PJy5DwiwnT+krreYe3yyCxauQaQ1JCsp9bOlBbB821priDRKtAgT+HwfU9eyaM6hJtTmpC4u8XkLh2PvAFIA2OOFF3obS/KgHvQPOJZQun1V4HSWuDMM5YG7IbPAjwYjymh4BukDxsNtKLq9YSaxpR7 root@h-cluster-hadoop-master-001 -----END SSH HOST KEY KEYS----- Cloud-init v. 0.7 finished at Mon, 12 Aug 2013 09:52:18 +0000. Datasource DataSourceEc2. Up 38.58 seconds