Some text. Some more text.
Some bold text and
One of the factors that delay a bug being fixed is the way it is reported. By creating this guide, we hope to help improve the communication between developers and users in bug resolution. Getting bugs fixed is an important, if not crucial part of the quality assurance for any project and hopefully this guide will help make that a success.
You're emerge-ing a package or working with a program and suddenly the worst happens -- you find a bug. Bugs come in many forms like emerge failures or segmentation faults. Whatever the cause, the fact still remains that such a bug must be fixed. Here is a few examples of such bugs.
$ ./bad_code `perl -e 'print Ax100'` Segmentation fault
/usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.2/include/g++-v3/backward/backward_warning.h:32:2: warning: #warning This file includes at least one deprecated or antiquated header. Please consider using one of the 32 headers found in section 17.4.1.2 of the C++ standard. Examples include substituting the <X> header for the <X.h> header for C++ includes, or <sstream> instead of the deprecated header <strstream.h>. To disable this warning use -Wno-deprecated. In file included from main.cc:40: menudef.h:55: error: brace-enclosed initializer used to initialize ` OXPopupMenu*' menudef.h:62: error: brace-enclosed initializer used to initialize ` OXPopupMenu*' menudef.h:70: error: brace-enclosed initializer used to initialize ` OXPopupMenu*' menudef.h:78: error: brace-enclosed initializer used to initialize ` OXPopupMenu*' main.cc: In member function `void OXMain::DoOpen()': main.cc:323: warning: unused variable `FILE*fp' main.cc: In member function `void OXMain::DoSave(char*)': main.cc:337: warning: unused variable `FILE*fp' make[1]: *** [main.o] Error 1 make[1]: Leaving directory `/var/tmp/portage/xclass-0.7.4/work/xclass-0.7.4/example-app' make: *** [shared] Error 2 !!! ERROR: x11-libs/xclass-0.7.4 failed. !!! Function src_compile, Line 29, Exitcode 2 !!! 'emake shared' failed
These errors can be quite troublesome. However, once you find them, what do you
do? The following sections will look at two important tools for handling run
time errors. After that, we'll take a look at compile errors, and how to handle
them. Let's start out with the first tool for debugging run time errors --
GDB, or the (G)NU (D)e(B)ugger, is a program used to find run time errors that
normally involve memory corruption. First off, let's take a look at what
debugging entails. One of the main things you must do in order to debug a
program is to
(debug symbols stripped) -rwxr-xr-x 1 chris users 3140 6/28 13:11 bad_code(debug symbols intact) -rwxr-xr-x 1 chris users 6374 6/28 13:10 bad_code
Just for reference,
CFLAGS="-O1 -pipe -g -ggdb"
CXXFLAGS="${CFLAGS}"
Lastly, you can also add debug to the package's USE flags. This can be done
with the
# echo "category/package debug" >> /etc/portage/package.use
Then we re-emerge the package with the modifications we've done so far as shown below.
# FEATURES="nostrip" emerge package
Now that debug symbols are setup, we can continue with debugging the program.
Let's say we have a program here called "bad_code". Some person claims that the program crashes and provides an example. You go ahead and test it out:
$ ./bad_code `perl -e 'print Ax100'` Segmentation fault
It seems this person was right. Since the program is obviously broken, we have
a bug at hand. Now, it's time to use
$ gdb --args ./bad_code `perl -e 'print Ax100'` GNU gdb 6.3 Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i686-pc-linux-gnu"...Using host libthread_db library "/lib/libthread_db.so.1".
You should see a prompt that says "(gdb)" and waits for input. First, we have to
run the program. We type in
(gdb) run Starting program: /home/chris/bad_code Program received signal SIGSEGV, Segmentation fault. 0xb7ec6dc0 in strcpy () from /lib/libc.so.6
Here we see the program starting, as well as a notification of SIGSEGV, or
Segmentation Fault. This is GDB telling us that our program has crashed. It
also gives the last run function it could trace when the program crashes.
However, this isn't too useful, as there could be multiple strcpy's in the
program, making it hard for developers to find which one is causing the issue.
In order to help them out, we do what's called a backtrace. A backtrace runs
backwards through all the functions that occurred upon program execution, to the
function at fault. Functions that return (without causing a crash) will not show
up on the backtrace. To get a backtrace, at the (gdb) prompt, type in
(gdb) bt #0 0xb7ec6dc0 in strcpy () from /lib/libc.so.6 #1 0x0804838c in run_it () #2 0x080483ba in main ()
You can notice the trace pattern clearly. main() is called first, followed by
run_it(), and somewhere in run_it() lies the strcpy() at fault. Things such as
this help developers narrow down problems. There are a few exceptions to the
output. First off is forgetting to enable debug symbols with
(gdb) bt #0 0xb7e2cdc0 in strcpy () from /lib/libc.so.6 #1 0x0804838c in ?? () #2 0xbfd19510 in ?? () #3 0x00000000 in ?? () #4 0x00000000 in ?? () #5 0xb7eef148 in libgcc_s_personality () from /lib/libc.so.6 #6 0x080482ed in ?? () #7 0x080495b0 in ?? () #8 0xbfd19528 in ?? () #9 0xb7dd73b8 in __guard_setup () from /lib/libc.so.6 #10 0xb7dd742d in __guard_setup () from /lib/libc.so.6 #11 0x00000006 in ?? () #12 0xbfd19548 in ?? () #13 0x080483ba in ?? () #14 0x00000000 in ?? () #15 0x00000000 in ?? () #16 0xb7deebcc in __new_exitfn () from /lib/libc.so.6 #17 0x00000000 in ?? () #18 0xbfd19560 in ?? () #19 0xb7ef017c in nullserv () from /lib/libc.so.6 #20 0xb7dd6f37 in __libc_start_main () from /lib/libc.so.6 #21 0x00000001 in ?? () #22 0xbfd195d4 in ?? () #23 0xbfd195dc in ?? () #24 0x08048201 in ?? ()
This backtrace contains a large number of ?? marks. This is because without
debug symbols,
(gdb) bt #0 0xb7e4bdc0 in strcpy () from /lib/libc.so.6 #1 0x0804838c in run_it (input=0x0) at bad_code.c:7 #2 0x080483ba in main (argc=1, argv=0xbfd3a434) at bad_code.c:12
Here we see that a lot more information is available for developers. Not only is function information displayed, but even the exact line numbers of the source files. This method is the most preferred if you can spare the extra space. Here's how much the file size varies between debug, strip, and -ggdb enabled programs.
(debug symbols stripped) -rwxr-xr-x 1 chris users 3140 6/28 13:11 bad_code(debug symbols enabled) -rwxr-xr-x 1 chris users 6374 6/28 13:10 bad_code(-ggdb flag enabled) -rwxr-xr-x 1 chris users 19552 6/28 13:11 bad_code
As you can see, -ggdb adds about
(gdb) quit The program is running. Exit anyway? (y or n) y $
This ends the walk-through of
Programs often use files to fetch configuration information, access hardware or
write logs. Sometimes, a program attempts to reach such files incorrectly. A
tool called
$ ./foobar2 Configuration says: bar
Our previous configuration specifically had it set to foo, so let's use
We make
# strace -ostrace.log ./foobar2
This creates a file called
open(".foobar2/config", O_RDONLY) = 3
read(3, "bar", 3) = 3
Aha! So There's the problem. Someone moved the configuration directory to
Now we've taken care of finding run time bugs. These bugs prove to be
problematic when you try and run your programs. However, run time errors are
the least of your concerns if your program won't compile at all. Let's take a
look at how to address
Let's take a look at this very simple
gcc -D__TEST__ -D__GNU__ -D__LINUX__ -L/usr/lib -I/usr/include -L/usr/lib/nspr/ -I/usr/include/fmod -c -o foobar2-7.o foobar2-7.c gcc -D__TEST__ -D__GNU__ -D__LINUX__ -L/usr/lib -I/usr/include -L/usr/lib/nspr/ -I/usr/include/fmod -c -o foobar2-8.o foobar2-8.c gcc -D__TEST__ -D__GNU__ -D__LINUX__ -L/usr/lib -I/usr/include -L/usr/lib/nspr/ -I/usr/include/fmod -c -o foobar2-9.o foobar2-9.c gcc -D__TEST__ -D__GNU__ -D__LINUX__ -L/usr/lib -I/usr/include -L/usr/lib/nspr/ -I/usr/include/fmod -c -o foobar2.o foobar2.c foobar2.c:1:17: ogg.h: No such file or directory make: *** [foobar2.o] Error 1 !!! ERROR: sys-apps/foobar2-1.0 failed. !!! Function src_compile, Line 19, Exitcode 2 !!! Make failed! !!! If you need support, post the topmost build error, NOT this status message
The program is compiling smoothly when it suddenly stops and presents an error message. This particular error can be split into 3 different sections, The compile messages, the build error, and the emerge error message as shown below.
(Compilation Messages) gcc -D__TEST__ -D__GNU__ -D__LINUX__ -L/usr/lib -I/usr/include -L/usr/lib/nspr/ -I/usr/include/fmod -c -o foobar2-7.o foobar2-7.c gcc -D__TEST__ -D__GNU__ -D__LINUX__ -L/usr/lib -I/usr/include -L/usr/lib/nspr/ -I/usr/include/fmod -c -o foobar2-8.o foobar2-8.c gcc -D__TEST__ -D__GNU__ -D__LINUX__ -L/usr/lib -I/usr/include -L/usr/lib/nspr/ -I/usr/include/fmod -c -o foobar2-9.o foobar2-9.c gcc -D__TEST__ -D__GNU__ -D__LINUX__ -L/usr/lib -I/usr/include -L/usr/lib/nspr/ -I/usr/include/fmod -c -o foobar2.o foobar2.c(Build Error) foobar2.c:1:17: ogg.h: No such file or directory make: *** [foobar2.o] Error 1(emerge Error) !!! ERROR: sys-apps/foobar2-1.0 failed. !!! Function src_compile, Line 19, Exitcode 2 !!! Make failed! !!! If you need support, post the topmost build error, NOT this status message
The compilation messages are what lead up to the error. Most often, it's good to at least include 10 lines of compile information so that the developer knows where the compilation was at when the error occurred.
Make errors are the actual error and the information the developer needs. When you see "make: ***", this is often where the error has occurred. Normally, you can copy and paste 10 lines above it and the developer will be able to address the issue. However, this may not always work and we'll take a look at an alternative shortly.
The emerge error is what
PORT_LOGDIR is a portage variable that sets up a log directory for separate
emerge logs. Let's take a look and see what that entails. First, run your
emerge with PORT_LOGDIR set to your favorite log location. Let's say we have a
location
# PORT_LOGDIR=/var/log/portage emerge foobar2
Now the emerge fails again. However, this time we have a log we can work with, and attach to the bug later on. Let's take a quick look at our log directory.
# ls -la /var/log/portage total 16 drwxrws--- 2 root root 4096 Jun 30 10:08 . drwxr-xr-x 15 root root 4096 Jun 30 10:08 .. -rw-r--r-- 1 root root 7390 Jun 30 10:09 2115-foobar2-1.0.log
The log files have the format [counter]-[package name]-[version].log. Counter is a special variable that is meant to state this package as the n-th package you've emerged. This prevents duplicate logs from appearing. A quick look at the log file will show the entire emerge process. This can be attached later on as we'll see in the bug reporting section. Now that we've safely obtained our information needed to report the bug we can continue to do so. However, before we get started on that, we need to make sure no one else has reported the issue. Let's take a look at searching for bugs.
One of the most frustrating things for developers and bug-wranglers is finding duplicate bug reports. These cost them valuable time that they could otherwise use to work on more important bugs. Often, this can be prevented by a few simple search methods. So we're going to see how to search for bugs and find out if you have one that's similar. For this example, we're going to use the xclass emerge error that was used earlier.
/usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.2/include/g++-v3/backward/backward_warning.h:32:2: warning: #warning This file includes at least one deprecated or antiquated header. Please consider using one of the 32 headers found in section 17.4.1.2 of the C++ standard. Examples include substituting the <X> header for the <X.h> header for C++ includes, or <sstream> instead of the deprecated header <strstream.h>. To disable this warning use -Wno-deprecated. In file included from main.cc:40: menudef.h:55: error: brace-enclosed initializer used to initialize ` OXPopupMenu*' menudef.h:62: error: brace-enclosed initializer used to initialize ` OXPopupMenu*' menudef.h:70: error: brace-enclosed initializer used to initialize ` OXPopupMenu*' menudef.h:78: error: brace-enclosed initializer used to initialize ` OXPopupMenu*' main.cc: In member function `void OXMain::DoOpen()': main.cc:323: warning: unused variable `FILE*fp' main.cc: In member function `void OXMain::DoSave(char*)': main.cc:337: warning: unused variable `FILE*fp' make[1]: *** [main.o] Error 1 make[1]: Leaving directory `/var/tmp/portage/xclass-0.7.4/work/xclass-0.7.4/example-app' make: *** [shared] Error 2 !!! ERROR: x11-libs/xclass-0.7.4 failed. !!! Function src_compile, Line 29, Exitcode 2 !!! 'emake shared' failed
So to begin searching, we head over to the
We'll click on "Query Existing bug reports". The reason why we choose this over the basic bug search is because the basic bug search tends to give vague results and often hinders users from looking through the results and finding the duplicate bug. Once we click on the query screen, we reach the next page:
Proceed by clicking on the "Advanced Search" link to bring up the Advanced Search page.
This is how the Advanced Search Page looks like. While it may seem overwhelming at first, we're going to look at a few simple areas to narrow down the rather vague searches bugzilla returns.
The first field is the summary of the bug. Here we're simply going to put the name of the package that's crashing. If bugzie doesn't return results, try removing the package name, just in case someone didn't put that in the summary (highly unlikely, but we've seen a fair share of strange bug reports).
Product, Component, and Version should all be set to the default. This prevents us from being too specific and missing all the bugs.
Comment is the important part. Use the comment field to list what appears to be a specific instance of the error. Basically, don't use anything like the beginning of the build error, find a line that's before it stating a true error. Also, you'll want to filter out any punctuation to prevent bugzilla from interpreting the results the comment the wrong way. Example from the xclass emerge error:
menudef.h:78: error: brace-enclosed initializer used to initialize `OXPopupMenu'(Remove the quotes ' ') menudef.h 78 error brace-enclosed initializer used to initialize OXPopupMenu
The above is specific enough to where we'll find the bug without wading through other xclass compile failure candidates.
URI, Whiteboard, and Keywords can all be left alone. What we've entered so far should be enough to find our bug. Let's take a look at what we have filled out.
Now we click on the Search button and here come the results...
Only 2 bugs! That's a lot easier to deal with. We click on the first one to check, and sure enough it's the one we're looking for.
Not only is it the one we want, but it has also been resolved. By checking the last comment we see the solution and know what to do in order to resolve it. Now, let's see what would have happened if we had not used the advanced search.
4 more bugs to deal with! It gets even worse with larger packages. However, with these simple tools, we're able to significantly narrow down the search to try and locate a specific bug.
Let's say that you have searched and searched but still can't find a bug. You've found yourself a new bug. Let's take a look at the bug reporting process for submitting your new bug.
In this chapter, we'll figure out how to use Bugzilla to file a shiny, new bug.
Head over to
Click on "Report a Bug - Using the guided format".
As you can see, major emphasis has been placed on putting your bug in the right place. Gentoo Linux is where a large majority of bugs go.
Despite this, some people will file ebuild bugs in portage development (assumption that portage team handles the portage tree) or infra (assumption that infra has access to mirrors and rsync and can fix it directly). This is simply not how things work.
Another common misconception occurs with our Documentation bugs. For example, a
user finds a bug with the
Our bug goes in Gentoo Linux as it's an ebuild bug. We head over there and are presented with the multi-step bug reporting process. Let us now proceed with Step 1...
The first step here is really important (as the red text tells you). This is where you search to see that someone else hasn't hit the same bug you have, yet. If you do skip this step and a bug like yours already exists, it will be marked as a DUPLICATE thus wasting a large amount of QA effort. To give you an idea, the bug numbers that are struck out above are duplicate bugs. Now comes step 2, where we give the information.
Let us take a closer look at what's what.
So, for our example bug, we have :
Not including the category in the summary really isn't too bad, but it's recommended. If you don't include the package name, however, we won't know what you're filling a bug for, and will have to ask you about it later. The version number is important for people searching for bugs. If 20 people filed bugs and not one put a version number, how would people looking for similar bugs be able to tell if one was there's? They'd have to look through every single bug, which isn't too hard, but if there are say, 200 bugs.. it's not that easy. After all the package information, you'll want to include a small description of the incident. Here's an example:
These simple rules can make handling bugs a lot easier. Next are the details. Here we put in the information about the bug. We'll demonstrate with an example:
Now the developer knows why we're filing the bug. They can then try to reproduce it. Reproducibility tells us how often we were able to make the problem recur. In this example, we can reproduce it any time simply by running foobar2. Let's put that information in.
We have explained how we found the bug. The next step is to explain what were the results we got and what we think they should actually be.
We could then provide additional information. This could be things such as
stack traces, sections (since the whole log is usually big and of not
much use) of strace logs, but most importantly, your
Lastly we select the severity of the bug. Please look this over carefully. In most cases it's OK to leave it as is and someone will raise/lower it for you. However, if you raise the severity of the bug, please make sure you read it over carefully and make sure you're not making a mistake. A run down of the various levels is given below.
Here, we'll set it to Normal.
Now we can submit the bug report by clicking on the Submit Bug Report box. You
will now see your new bug come up. See
Looking at the bug, we see the information we provided earlier. You will notice that the bug has been assigned to bug-wranglers@gentoo.org. This is the default location for Application component bugs.
The details we entered about the bug are available as well.
However, bug-wranglers (usually) won't fix our bugs, so we'll reassign it to
someone that can (you can let bug-wranglers re-assign it for you as well). For
this we use the package's metadata.xml. You can normally find them in
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE pkgmetadata SYSTEM "http://www.gentoo.org/dtd/metadata.dtd"> <pkgmetadata> <herd>chriswhite</herd> <maintainer> <email>chriswhite@gentoo.org</email> <name>Chris White</name> </maintainer> <longdescription lang="en"> Foobar2 is a package that uses a configuration file to display a word. </longdescription> </pkgmetadata>
Notice the maintainer section. This lists the maintainer of the package, which in this case is myself, Chris White. The email listed is chriswhite@gentoo.org. We will use this to re-assign the bug to the proper person. To do this, click the bubble next to Reassign bug to, then fill in the email.
Then hit the Commit button for the changes to take place. The bug has been reassigned to me. Shortly afterward, you notice (by email usually) that I've responded to your bug. I've stated that I'd like to see an strace log to figure out how the program is trying to reach your configuration file. You follow the previous instructions on using strace and obtain an strace log. Now you need to attach it to the bug. In order to do this, click on "Create A New Attachment".
Now we have to attach the log. Let's go throught it step wise.
With respect to Content Type, here are a few more details. You can check the
"patch" checkbox if you're submitting a patch. Otherwise, you could ask
Bugzilla to "auto-detect" the file type (not advisable). The other options are
"select from list", which is most frequently used. Use plain text (text/plain)
for
We submit
We've mentioned before that sometimes ebuilds will tell you to attach a file in the emerge error. An example can be seen below.
configure: error: PNG support requires ZLIB. Use --with-zlib-dir=<DIR> !!! Please attach the config.log to your bug report: !!! /var/tmp/portage/php-5.0.3-r1/work/php-5.0.3/config.log !!! ERROR: dev-php/php-5.0.3-r1 failed. !!! Function econf, Line 485, Exitcode 0 !!! econf failed !!! If you need support, post the topmost build error, NOT this status message.
Please attach any file mentioned like this to your bug report.
While we're doing all this, suppose another person finds your bug by searching through bugzilla and is curious to keep track of the bug, they may do so by putting their email in the Add CC field of the bug as shown below. You could also keep track of other bugs by following the same method.
After all this work, the bug can undergo various status markings. This is usually done by the Gentoo Developers and sometimes by the reporter. The following are the various possible states a bug may go through during its lifetime.
Now shortly afterward, we find the error in the strace log and fix the bug and mark it as RESOLVED FIXED and mention that there was a change in the location of configuration files, and that I will update the ebuild with a warning about it. The bug now becomes resolved, and you are shown the following.
A little below, you'll see the following:
This gives you the option of Reopening the bug if you wish to (i.e. the developer thinks it's resolved but it's really not to your standards). Now our bug is fixed! However, different resolutions can occur. Here's a small list:
Sometimes, before the bug can be resolved, a developer may request that you test an updated ebulid. In the next chapter we'll take a look at testing ebuilds.
Let's say that you reported a bug for the foobar2 compile fix from earlier. Now developers might find out what the problem is and might need you to test the ebuild for them to be sure it works for you as well:
Some rather confusing vocabulary is used here. First off, let's see what an
overlay is. An overlay is a special directory like
PORTDIR_OVERLAY="/usr/local/portage"
Now we'll want to create the appropriate directories to put our test ebuild files in. In this case, we're supposed to put them in sys-apps/foobar2. You'll notice that the second comment asks for a files directory for the patch. The files directory holds the digests (md5sums of files for a particular version of a package) and any other required files that aren't included with the standard source archive (patches, init.d scripts, etc). This is a subdir in the package directory called files. Go ahead and create these directories:
# mkdir -p /usr/local/portage/sys-apps/foobar2/files
Ok now, we can go ahead and download the files. First, download the ebuild
into
The process to create an ebuild that can be used by emerge is fairly simple. You must create a Manifest and a digest file for the ebuild. This can be done with the ebuild command. Run it as shown.
# ebuild foobar2-1.0.ebuild digest >>> Generating digest file... <<< foobar2-1.0.tar.bz2 >>> Generating manifest file... <<< foobar2-1.0.ebuild <<< files/digest-foobar2-1.0 <<< files/foobar2-1.0-Makefile.patch >>> Computed message digests.
Now let's test to see if it works as it should.
# emerge -pv foobar2 These are the packages that I would merge, in order: Calculating dependencies ...done! [ebuild N ] sys-apps/foobar2-1.0 0 kB [1] Total size of downloads: 0 kB Portage overlays: [1] /usr/local/portage
It does seem to have worked! You'll notice the [1] next to the [ebuild] line.
That points to
# emerge foobar2 Calculating dependencies ...done!(compile info snipped) >>> Unpacking foobar2-1.0.tar.bz2 to /var/tmp/portage/foobar2-1.0/work * Applying foobar2-1.0-Makefile.patch ... [ ok ](compile info snipped) >>> Merging sys-apps/foobar2-1.0 to / >>> chris +sandbox(preinst) --- /usr/ --- /usr/bin/ >>> /usr/bin/foobar2
In the first section we see that the emerge started off as it should. The second section shows our patch being applied successfully by the "[ ok ]" status message to the right. The last section tells us the program compiled ok. The patch works! Now we can go and let the developer know that their patch works fine, and that they can commit the fix to portage.
This concludes the howto on working with Bugzilla. I hope you find this useful.
If you have any questions, comments, or ideas regarding this document, please
send them to me at