devstack - trove-guestagent downloaded from wrong address

asked 2015-06-22 22:58:30 -0600

updated 2015-06-22 23:17:31 -0600

Try as I might, I have never been able to create a Trove instance - the status remains BUILD for perhaps 10 minutes then turns into ERROR. The Trove taskmanager log file features a line "Build error: guestagent timeout".

While I can't ssh into the instance, and the console fails as well, thanks to guestmount I am able to look at the instance's filesystem. The core of the action is in /etc/init/trove-guest.conf, where an attempt is made to download the guestagent to the instance.

Strangely though, trove-guest.conf tries to download it from 10.0.0.1 - that's the router. This fails, and /var/log/upstart/trove-guest.log on the instance contains:

ssh: connect to host 10.0.0.1 port 22: Connection refused
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(605) [Receiver=3.0.9]

As a consequence the guestagent can't be copied to the instance. That explains the timeout.

I stop the instance and, with the help of guestmount, change 10.0.0.1 to the hypervisor's address (or controller's address; since this is devstack there is no difference). When restarting the instance, the initialization seems to complete without errors.

So my questions are:

  • is 10.0.0.1 in the original trove-guest.conf correct?
  • if yes, something's wrong with my networking I guess. So what can I do? (I have used this devstack config a lot without ever seeing any NW problems)
  • if not, where might 10.0.0.1 come from? I suspect trove-guest.conf is written by cloud-init; any way to influence this?

EDIT: It turns out the 10.0.0.1 is hard-coded in the original image, http://tarballs.openstack.org/trove/i..., which solves the third question.

Appendix: The full trove-guest.conf:

description "Trove Guest"
author "Auto-Gen"

start on (filesystem and net-device-up IFACE!=lo)
stop on runlevel [016]
chdir /var/run
pre-start script
    mkdir -p /var/run/trove
    chown ubuntu:root /var/run/trove/

    mkdir -p /var/lock/trove
    chown ubuntu:root /var/lock/trove/

    mkdir -p /tmp/
    chown ubuntu:root /tmp/
    chmod +r /etc/guest_info

    # If /etc/trove does not exist, copy the trove source and the
    # guest agent config from the user's development environment
    if [ ! -d /etc/trove ]; then
        sudo -u ubuntu rsync -e 'ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' -avz --exclude='.*' ubuntu@10.0.0.1:/opt/stack/trove/ /home/ubuntu/trove
        mkdir -p /etc/trove
        sudo -u ubuntu rsync -e 'ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' -avz --exclude='.*' ubuntu@10.0.0.1:/etc/trove/trove-guestagent.conf ~ubuntu/
        mv ~ubuntu/trove-guestagent.conf /etc/trove/trove-guestagent.conf
    fi

end script

exec su -c "/home/ubuntu/trove/contrib/trove-guestagent --config-file=/etc/guest_info --config-file=/etc/trove/trove-guestagent.conf" ubuntu
edit retag flag offensive close merge delete

Comments

Hi, I think i'm having the same issue. I can't ssh into my trove guest instance either. How do you guestmount a trove/nova instance to inspect its log files? Thanks

ChakLee gravatar imageChakLee ( 2015-08-03 16:08:21 -0600 )edit