How do I debug OpenStack?
I stumbled upon an error that I cannot decipher: where do I find OpenStack's logs and how do I debug the system?
First time here? Check out the FAQ!
I stumbled upon an error that I cannot decipher: where do I find OpenStack's logs and how do I debug the system?
Hi,
The OpenStack Operations guide covers this:
answered 2013-04-05 19:27:03 -0500
This post is a wiki. Anyone with karma >75 is welcome to improve it.
Debugging a piece of code is an activity as important as writing it. The importance of debugging is much more noticeable on large projects like OpenStack, which require both developers and common users contribute and report when something is not going well.
Here are some pointers to collect useful logs and debugging processes.
Logging
Looking through the logs can be tricky... not if you know how to use regular expressions!
The first thing to do when we see a strange behavior that is repeated with the same input data, is to look at logs.
Logs are files, usually plain text and arranged chronologically, that contain information about the code that is executed to perform an action.
Analyzing the information found there, you may be able to detect what the problem is or, at least, where the error occurs.
The location of the logs in the system may vary depending on the implementation, but in GNU/Linux they usually are stored in /var/log.
In OpenStack the location of these logs varies according how have you deployed it: using DevStack or by hand, whether with the distribution repository packages you are using or from source.
If you're using DevStack keep in mind that by default, DevStack doesn’t log. To enable this option you will need to add the following lines to the localrc file.
LOGFILE=$DEST/stack.sh.log
– Name and location of the stack.sh script logs. Without this line the output of this script will appear in the terminal where you run it, without being stored in any location.
SCREEN_LOGDIR=$DEST/logs/screen
– Location of the screen logs. DevStack runs OpenStack services under GNU Screen, thus providing the facility to view its current execution status. The logs simply store the output of these screens..
LOGDAYS=1
– Number of days to be logged. The old logs are replaced with the latest runs.
$DEST is the location in which the logs will be located. You can set this value as you wish.
Screen logs are just a record of the screens initiated by DevStack but you can check them anytime during the execution. The information provided is very useful to debug any OpenStack service.
GNU Screen provides the same functions as the DEC VT100 terminal, plus other aggregates. If you have no experience with this terminal, make sure to write down the following shortcuts.
Attach to a screen screen -r stack
Switch screens ctrl+a <screen no.> (e.g. ctrl+a 1)
Scroll in a screen ctrl+a esc and up/down arrows to scroll
List screens ctrl+a " and select the screen with up/down arrows
Next screen ctrl+a n
Detach from the screen ctrl+a d
Manual deployment
Here you will find most services logs in /var/log/service, where service is the name of the desired service. For example, in /var/log/keystone you will find Keystone logs.
In my experience, the only service that differs from this default location is Horizon. You can find ... (more)
HI, are there any guide on debugging with eclipse and pydev with the latest branch. I have tried commented out eventlet.monkeypatch(os=False) and replaced it with eventlet.monkeypatch(all=False,socket=True,time=True,os=False) and added import sys;sys.path.append('path') but breakpoints are ignored
Here is the link to debug openstack using eclipse
http://ilearnstack.com/2013/05/30/debug-openstack-code-local-remote-with-eclipse-and-pydev-plug-in/
works fine with as horizon ocata integration with eclipse!!! https://chianingwang.blogspot.de/2015... but use from the link above Run Configurations -> Arguments -> ${workspace_loc:openstack-dashboard}
Please refer Grizzly docs http://docs.openstack.org/grizzly/openstack-ops/content/logging_monitoring.html
Compute specific docs here http://docs.openstack.org/grizzly/openstack-compute/admin/content/configuring-logging.html
Networking logs can be found at /var/log/quantum/
You should be seeing logs under: /var/log/* dir under respective service.
Just use pdb doesn't work fine with some projects like nova and or tempest, I'm not really sure, but I think it has something to do to how testr perform the tests run. Although there is valuable tip on testr page on the wiki https://wiki.openstack.org/wiki/Testr#Debugging_.28pdb.29_Tests
TL;DR: python -m testtools.run discover my.tests.regex
Check this guide. It seems good.
Asked: 2013-04-05 19:10:34 -0500
Seen: 9,737 times
Last updated: Jul 16 '15
How Many Physical Servers and with what Resources?
Recommended hardware to learn?
How can I debug network-related issues?
How do I fix errors with IP addresses for VMs?
Which network configuration for OpenStack in a VirtualBox?
How to start with Java and CentOS7
How to deploy mistral on Heroku ?
How do I add new features to Horizon?
How do you find what is using a security group of a deleted tenant?