Remote debugging OpenStack Kilo "nova-scheduler" with Eclipse-Pydev ends in "ThreadSuspended" error messsage
Dear community,
my current development environment is:
- Eclipse Luna, version 4.4.2
- PyDev 4.2.0 (latest version)
- RDO OpenStack Kilo (latest version)
- Red Hat Enterprise Linux 7.1 (latest version)
when trying to remote debug a nova-scheduler related Python source file name "filter_scheduler.py" located in "/usr/lib/python2.7/site-packages/nova/scheduler".
I followed the following guidelines:
- http://pydev.org/manual_adv_remote_debugger.html (http://pydev.org/manual_adv_remote_de...)
- http://ilearnstack.com/2013/05/30/debug-openstack-code-local-remote-with-eclipse-and-pydev-plug-in/ (http://ilearnstack.com/2013/05/30/deb...)
- http://debugopenstack.blogspot.de/2012/12/debugopenstack-code-local-remote-with.html (http://debugopenstack.blogspot.de/201...)
- https://ask.openstack.org/en/question/815/how-do-i-debug-nova-service-with-eclipse-and-pydev/ (https://ask.openstack.org/en/question...)
So, I did the following steps:
- copied the Python source of "/usr/lib/python2.7/site-packages/nova/*" to my localhost and imported the Python sources in my PyDev Eclipse project
- copied the Eclipse "pydevd" Python debugger client implementation from "C:\Program Files (x86)\eclipse\dropins\plugins\org.python.pydev_4.0.0.201504132356" to the OpenStack remote host in "/tmp/pysrc"
- adapted the "/tmp/pysrc/pydevd_file_utils.py" file to match the source paths of the Pyhton code between the remote host and my local development environment (see PATHS_FROM_ECLIPSE_TO_PYTHON = [(r'c:\Workspace\PythonTest\src', r'/usr/lib/python2.7/site-packages')])
- modified "/usr/bin/nova-scheduler" to ensure enhancing the Python module search path for "pydevd" via "sys.path.append('/tmp/pysrc')" below "import sys"
- inserted the "import eventlet" and "eventlet.monkey_patch(all=False,socket=True,time=True,os=False,thread=False)" statements in "/usr/bin/nova-scheduler" to prevent getting "ThreadSuspended" error messages
- activated the breakpoint in "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py" by additionally importing "import pydevd" and inserting "pydevd.settrace('xxx.xxx.xxx.xxx', port=5678, stdoutToServer=True, stderrToServer=True,suspend=True)" where the debugging should start
- started the Eclipse PyDev debugger on standard port 5678
- restarted "nova-scheduler" via "systemctl restart nova-scheduler"
- launched a nova docker instance via "nova boot --image "docker.io/fedora:latest" --flavor m1.tiny --nic net-id=<nic_id> docker_test"
- checked the Eclipse PyDev debugger and always see an "Error reading ThreadSuspended" error message
Although, I inserted the famous "eventlet.monkey_patch" statement in "/usr/bin/nova-scheduler" I get this error message:
Unexpected XML error reading:<xml><thread id="pid22922_seq1" stop_reason="105" message="None"><frame id="31976272" name="allocate_lock" file="/usr/lib/python2.7/site-packages/eventlet/green/thread.py" line="78">"</frame><frame id="31416032" name="__init__" file="/usr/lib64/python2.7/threading.py" line="563">"</frame><frame id="31415280" name="Event" file="/usr/lib64/python2.7/threading.py" line="550">"</frame><frame id="31412496" name="__init__" file="/usr/lib64/python2.7/threading.py" line="690">"</frame><frame id="97266224" name="__init__" file="/tmp/pysrc42/pydevd_comm.py" line="250">"</frame><frame id="97358208" name="__init__" file="/tmp/pysrc42/pydevd.py" line="205">"</frame><frame id="97235984" name="_locked_settrace" file="/tmp/pysrc42/pydevd.py" line="1967">"</frame><frame id="97255072" name="settrace" file="/tmp/pysrc42/pydevd.py" line="1876">"</frame><frame id="97254496" name="select_destinations" file="/usr/lib/python2 ...