Nodepool Raising SSHException

asked 2014-05-23 06:33:49 -0600

mstrisoline gravatar image

updated 2014-05-23 07:25:59 -0600

I have been looking into nodepool to expand our jenkins test infrastructure and keep running into this error.

It appears that paramiko is failing to establish the connection because of unknown host keys. I have tried to set the client ssh_config to use StrictHostKeyChecking no but it doesn't seem to acknowledge that within paramiko itself. I Have tested the keys being used and they do work successfully from the server running nodepool.

root@nodepool:/root# nodepool image-update nova00 fedora20-qcow2
INFO:nodepool.ImageUpdater:Creating image id: 36 with hostname fedora20-qcow2-1400787620.template.openstack.org for fedora20-qcow2 in nova00
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): api00.infradev.pitbpa0.priv.intermedix.com
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): api00.infradev.pitbpa0.priv.intermedix.com
INFO:paramiko.transport:Connected (version 2.0, client OpenSSH_6.3)
/usr/local/lib/python2.7/dist-packages/paramiko/client.py:517: UserWarning: Unknown ssh-rsa host key for 10.51.17.228: 58183977c83ae4c0d4fda09c181efaf7
  (key.get_name(), hostname, hexlify(key.get_fingerprint())))
ERROR:nodepool.ImageUpdater:Exception updating image fedora20-qcow2 in nova00:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/nodepool/nodepool.py", line 700, in _run
    self.updateImage(session)
  File "/usr/local/lib/python2.7/dist-packages/nodepool/nodepool.py", line 778, in updateImage
    self.bootstrapServer(server, key, use_password=use_password)
  File "/usr/local/lib/python2.7/dist-packages/nodepool/nodepool.py", line 821, in bootstrapServer
    timeout=CONNECT_TIMEOUT)
  File "/usr/local/lib/python2.7/dist-packages/nodepool/nodeutils.py", line 48, in ssh_connect
    client = SSHClient(ip, username, **connect_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/nodepool/sshclient.py", line 30, in __init__
    key_filename=key_filename)
  File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 273, in connect
    self._auth(username, password, pkey, key_filenames, allow_agent, look_for_keys)
  File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 457, in _auth
    raise SSHException('No authentication methods available')
SSHException: No authentication methods available
INFO:nodepool.NodePool:Deleted image id: 36

Included is the nodepool.yaml

script-dir: .
dburi: 'mysql://nodepool@localhost/nodepool'

cron:
  check: '*/15 * * * *'
  cleanup: '*/1 * * * *'
  update-image: '14 2 * * *'

zmq-publishers:
  - tcp://localhost:8888

labels:
  - name: fedora20-qcow2
    image: fedora20-qcow2
    min-ready: 1
    providers:
      - name: nova00

providers:
  - name: nova00
    keypair: 'jenkins-key'
    username: 'admin'
    password: 'admin'
    auth-url: 'http://keystone.example.com:5000/v2.0'
    project-id: 'admin'
    max-servers: 3
    pool: 'ext-net'
    boot-timeout: 120
    launch-timeout: 900
    networks:
      - net-id: 'b22df07b-0cb6-4226-91e3-ac206af727d4'
    images:
      - name: fedora20-qcow2
        base-image: 'fedora20-qcow2'
        min-ram: 2048
        username: fedora
        private-key: /var/lib/jenkins/.ssh/id_rsa

targets:
  - name: test-infra-jenkins
    jenkins:
      url: http://nodepool.example.com:8080/
      user: jenkins
      apikey: **************************

So I added some debugging to this:

lass SSHClient(object):
    def __init__(self, ip, username, password=None, pkey=None,
                 key_filename=None, log=None):
        print "ip: %s" % (ip)
        print "username: %s" % (username)
        print "password: %s" % (password)
        print "pkey: %s" % (pkey)
        print "key_filename: %s" % (key_filename)
        print log

        client = paramiko.SSHClient()
        client.set_missing_host_key_policy(paramiko.WarningPolicy())
        client.connect(ip, username=username, password=password, pkey=pkey,
                       key_filename=key_filename)
        self.client = client
        self.log = log

And this is the output:

root@nodepool:/# nodepool -c /etc/nodepool/nodepool.yaml image-update nova00 fedora20-qcow2
INFO:nodepool ...
(more)
edit retag flag offensive close merge delete