/[gentoo]/xml/htdocs/doc/en/bugzilla-howto.xml

Diff of /xml/htdocs/doc/en/bugzilla-howto.xml

Parent Directory | Revision Log | View Patch Patch

Revision 1.1		Revision 1.6
1	<?xml version="1.0" encoding="UTF-8"?>	1	<?xml version="1.0" encoding="UTF-8"?>
2	<!DOCTYPE guide SYSTEM "/dtd/guide.dtd">	2	<!DOCTYPE guide SYSTEM "/dtd/guide.dtd">
3	<!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/bugzilla-howto.xml,v 1.1 2005/07/07 12:02:21 fox2mike Exp $ -->	3	<!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/bugzilla-howto.xml,v 1.6 2005/08/30 02:51:12 vapier Exp $ -->
4		4
5	<guide link="/doc/en/bugzilla-howto.xml">	5	<guide link="/doc/en/bugzilla-howto.xml">
6	<title>Gentoo Bug Reporting Guide</title>	6	<title>Gentoo Bug Reporting Guide</title>
7		7
8	<author title="Author">	8	<author title="Author">
…		…
18		18
19	<!-- The content of this document is licensed under the CC-BY-SA license -->	19	<!-- The content of this document is licensed under the CC-BY-SA license -->
20	<!-- See http://creativecommons.org/licenses/by-sa/2.5 -->	20	<!-- See http://creativecommons.org/licenses/by-sa/2.5 -->
21	<license/>	21	<license/>
22		22
23	<version>1.0</version>	23	<version>1.5</version>
24	<date>2005-07-07</date>	24	<date>2005-08-29</date>
25		25
26	<chapter>	26	<chapter>
27	<title>Introduction</title>	27	<title>Introduction</title>
28	<section>	28	<section>
29	<title>Preface</title>	29	<title>Preface</title>
30	<body>	30	<body>
31		31
32	<p>	32	<p>
33	Often one of the factors that delay a bug being fixed is how it is reported. By	33	One of the factors that delay a bug being fixed is the way it is reported. By
34	creating this guide, I hope to help improve the communication between	34	creating this guide, we hope to help improve the communication between
35	developers and users in bug resolution. Getting bugs fixed is an important, if	35	developers and users in bug resolution. Getting bugs fixed is an important, if
36	not crucial part of the quality assurance of any project and hopefully this	36	not crucial part of the quality assurance for any project and hopefully this
37	guide will help make that a success.	37	guide will help make that a success.
38	</p>	38	</p>
39		39
40	</body>	40	</body>
41	</section>	41	</section>
42	<section>	42	<section>
43	<title>Initial Finding</title>	43	<title>Bugs!!!!</title>
44	<body>	44	<body>
45		45
46	<p>	46	<p>
47	You're emerge-ing a package or working with a program, then suddenly the worst	47	You're emerge-ing a package or working with a program and suddenly the worst
48	happens -- you find a bug. Bugs come in many forms, whether it be emerge	48	happens -- you find a bug. Bugs come in many forms like emerge failures or
49	failures or segmentation faults. Whatever the cause, the fact still remains that	49	segmentation faults. Whatever the cause, the fact still remains that such a bug
50	such a bug must be fixed. Here is a few examples of such bugs.	50	must be fixed. Here is a few examples of such bugs.
51	</p>	51	</p>
52		52
53	<pre caption="A run time error">	53	<pre caption="A run time error">
54	$ <i>./bad_code `perl -e 'print Ax100'`</i>	54	$ <i>./bad_code `perl -e 'print Ax100'`</i>
55	Segmentation fault	55	Segmentation fault
…		…
84	!!! Function src_compile, Line 29, Exitcode 2	84	!!! Function src_compile, Line 29, Exitcode 2
85	!!! 'emake shared' failed	85	!!! 'emake shared' failed
86	</pre>	86	</pre>
87		87
88	<p>	88	<p>
89	These errors can be quite troublesome. However, once you find them, what do	89	These errors can be quite troublesome. However, once you find them, what do you
90	you do? The following sections will look at 2 important tools for handling	90	do? The following sections will look at two important tools for handling run
91	run time errors. After that, we'll take a look at compile errors, and how to	91	time errors. After that, we'll take a look at compile errors, and how to handle
92	handle them. Let's start out with the first tool for debugging run time	92	them. Let's start out with the first tool for debugging run time errors --
93	errors -- <c>gdb</c>	93	<c>gdb</c>.
94	</p>	94	</p>
95		95
96	</body>	96	</body>
97	</section>	97	</section>
98	</chapter>	98	</chapter>
…		…
106		106
107	<p>	107	<p>
108	GDB, or the (G)NU (D)e(B)ugger, is a program used to find run time errors that	108	GDB, or the (G)NU (D)e(B)ugger, is a program used to find run time errors that
109	normally involve memory corruption. First off, let's take a look at what	109	normally involve memory corruption. First off, let's take a look at what
110	debugging entails. One of the main things you must do in order to debug a	110	debugging entails. One of the main things you must do in order to debug a
111	program is to <c>emerge</c> the program with FEATURES="nostrip". This prevents	111	program is to <c>emerge</c> the program with <c>FEATURES="nostrip"</c>. This
112	the stripping of debug symbols. Why are programs stripped by default? The reason	112	prevents the stripping of debug symbols. Why are programs stripped by default?
113	is the same as that for having gzipped man pages -- saving space. Here's how the	113	The reason is the same as that for having gzipped man pages -- saving space.
114	size of a program varies with and without debug symbol stripping.	114	Here's how the size of a program varies with and without debug symbol stripping.
115	</p>	115	</p>
116		116
117	<pre caption="Filesize Comparison">	117	<pre caption="Filesize Comparison">
118	<comment>(debug symbols stripped)</comment>	118	<comment>(debug symbols stripped)</comment>
119	-rwxr-xr-x 1 chris users 3140 6/28 13:11 bad_code	119	-rwxr-xr-x 1 chris users 3140 6/28 13:11 bad_code
…		…
123		123
124	<p>	124	<p>
125	Just for reference, <e>bad_code</e> is the program we'll be debugging with	125	Just for reference, <e>bad_code</e> is the program we'll be debugging with
126	<c>gdb</c> later on. As you can see, the program without debugging symbols is	126	<c>gdb</c> later on. As you can see, the program without debugging symbols is
127	3140 bytes, while the program with them is 6374 bytes. That's close to double	127	3140 bytes, while the program with them is 6374 bytes. That's close to double
128	the size! Two more things can be done for debugging. The first is adding -g to	128	the size! Two more things can be done for debugging. The first is adding ggdb3
129	your CFLAGS and CXXFLAGS. This flag adds more debugging information than is	129	to your CFLAGS and CXXFLAGS. This flag adds more debugging information than is
130	generally included. We'll see what that means later on. Lastly, you can also add	130	generally included. We'll see what that means later on. This is how
131	debug to the package's USE flags. This can be done with the	131	<path>/etc/make.conf</path> <e>might</e> look with the newly added flags.
132	<path>package.use</path> file.
133	</p>	132	</p>
		133
		134	<pre caption="make.conf settings">
		135	CFLAGS="-O1 -pipe -g -ggdb"
		136	CXXFLAGS="${CFLAGS}"
		137	</pre>
		138
		139	<p>
		140	Lastly, you can also add debug to the package's USE flags. This can be done
		141	with the <path>package.use</path> file.
		142	</p>
134		143
135	<pre caption="Using package.use to add debug USE flag">	144	<pre caption="Using package.use to add debug USE flag">
136	# <i>echo "category/package debug" >> /etc/portage/package.use</i>	145	# <i>echo "category/package debug" >> /etc/portage/package.use</i>
137	</pre>	146	</pre>
138		147
…		…
142	USE flags set in <path>package.use</path>, you will need to manually modify them	151	USE flags set in <path>package.use</path>, you will need to manually modify them
143	in your favorite editor.	152	in your favorite editor.
144	</note>	153	</note>
145		154
146	<p>	155	<p>
147	Now that that's done, you will need to re-emerge your package to set the	156	Then we re-emerge the package with the modifications we've done so far as shown
148	new debug settings into place. This can be done as follows:	157	below.
149	</p>	158	</p>
150		159
151	<pre caption="Re-emergeing a package with debugging">	160	<pre caption="Re-emergeing a package with debugging">
152	# <i>FEATURES="nostrip" emerge package</i>	161	# <i>FEATURES="nostrip" emerge package</i>
153	</pre>	162	</pre>
…		…
161	<section>	170	<section>
162	<title>Running the program with GDB</title>	171	<title>Running the program with GDB</title>
163	<body>	172	<body>
164		173
165	<p>	174	<p>
166	Let's say we have a program here called "bad_code" (I know, it's sort of cheesy	175	Let's say we have a program here called "bad_code". Some person claims that the
167	but..). Some person claims he can break the code and provides an example. You go	176	program crashes and provides an example. You go ahead and test it out:
168	ahead and test it out:
169	</p>	177	</p>
170		178
171	<pre caption="Breaking The Program">	179	<pre caption="Breaking The Program">
172	$ <i>./bad_code `perl -e 'print Ax100'`</i>	180	$ <i>./bad_code `perl -e 'print Ax100'`</i>
173	Segmentation fault	181	Segmentation fault
…		…
189	Type "show copying" to see the conditions.	197	Type "show copying" to see the conditions.
190	There is absolutely no warranty for GDB. Type "show warranty" for details.	198	There is absolutely no warranty for GDB. Type "show warranty" for details.
191	This GDB was configured as "i686-pc-linux-gnu"...Using host libthread_db library "/lib/libthread_db.so.1".	199	This GDB was configured as "i686-pc-linux-gnu"...Using host libthread_db library "/lib/libthread_db.so.1".
192	</pre>	200	</pre>
193		201
		202	<note>
		203	One can also debug with core dumps. These core files contain the same
		204	information that the program would produce when run with gdb. In order to debug
		205	with a core file with bad_code, you would run <c>gdb ./bad_code core</c> where
		206	core is the name of the core file.
		207	</note>
		208
194	<p>	209	<p>
195	You should see a small terminal like setup after that which says "(gdb)" and	210	You should see a prompt that says "(gdb)" and waits for input. First, we have to
196	waits for input. First, we have to run the program. We type in <c>run</c> at the	211	run the program. We type in <c>run</c> at the command and receive a notice like:
197	command and receive a notice like:
198	</p>	212	</p>
199		213
200	<pre caption="Running the program in GDB">	214	<pre caption="Running the program in GDB">
201	(gdb) run	215	(gdb) <i>run</i>
202	Starting program: /home/chris/bad_code	216	Starting program: /home/chris/bad_code
203		217
204	Program received signal SIGSEGV, Segmentation fault.	218	Program received signal SIGSEGV, Segmentation fault.
205	0xb7ec6dc0 in strcpy () from /lib/libc.so.6	219	0xb7ec6dc0 in strcpy () from /lib/libc.so.6
206	</pre>	220	</pre>
…		…
217	up on the backtrace. To get a backtrace, at the (gdb) prompt, type in <c>bt</c>.	231	up on the backtrace. To get a backtrace, at the (gdb) prompt, type in <c>bt</c>.
218	You will get something like this:	232	You will get something like this:
219	</p>	233	</p>
220		234
221	<pre caption="Program backtrace">	235	<pre caption="Program backtrace">
222	(gdb) bt	236	(gdb) <i>bt</i>
223	#0 0xb7ec6dc0 in strcpy () from /lib/libc.so.6	237	#0 0xb7ec6dc0 in strcpy () from /lib/libc.so.6
224	#1 0x0804838c in run_it ()	238	#1 0x0804838c in run_it ()
225	#2 0x080483ba in main ()	239	#2 0x080483ba in main ()
226	</pre>	240	</pre>
227		241
228	<p>	242	<p>
229	So as we see here, first main() was run, then run_it(), and somewhere in run_it	243	You can notice the trace pattern clearly. main() is called first, followed by
230	lies the strcpy() at fault. Things such as this help developers narrow things	244	run_it(), and somewhere in run_it() lies the strcpy() at fault. Things such as
231	down. Now, there are a few exceptions to the output. First off is forgetting	245	this help developers narrow down problems. There are a few exceptions to the
232	to disable debug symbols with FEATURES="nostrip". With debug symbols stripped,	246	output. First off is forgetting to enable debug symbols with
233	output looks something like this:	247	<c>FEATURES="nostrip"</c>. With debug symbols stripped, the output looks something
		248	like this:
234	</p>	249	</p>
235		250
236	<pre caption="Program backtrace With debug symbols stripped">	251	<pre caption="Program backtrace With debug symbols stripped">
237	(gdb) bt	252	(gdb) <i>bt</i>
238	#0 0xb7e2cdc0 in strcpy () from /lib/libc.so.6	253	#0 0xb7e2cdc0 in strcpy () from /lib/libc.so.6
239	#1 0x0804838c in ?? ()	254	#1 0x0804838c in ?? ()
240	#2 0xbfd19510 in ?? ()	255	#2 0xbfd19510 in ?? ()
241	#3 0x00000000 in ?? ()	256	#3 0x00000000 in ?? ()
242	#4 0x00000000 in ?? ()	257	#4 0x00000000 in ?? ()
…		…
262	#24 0x08048201 in ?? ()	277	#24 0x08048201 in ?? ()
263	</pre>	278	</pre>
264		279
265	<p>	280	<p>
266	This backtrace contains a large number of ?? marks. This is because without	281	This backtrace contains a large number of ?? marks. This is because without
267	debug symbols, <c>gdb</c> doesn't know how the program was ran. Hence, it is	282	debug symbols, <c>gdb</c> doesn't know how the program was run. Hence, it is
268	crucial that debug symbols are <e>not</e> stripped. Now remember a while ago I	283	crucial that debug symbols are <e>not</e> stripped. Now remember a while ago we
269	told you about the -g flag. Let's see what the output looks like with that:	284	mentioned the -ggdb flag. Let's see what the output looks like with the flag
		285	enabled:
270	</p>	286	</p>
271		287
272	<pre caption="Program backtrace with -g flag">	288	<pre caption="Program backtrace with -ggdb3">
273	(gdb) bt	289	(gdb) <i>bt</i>
274	#0 0xb7e4bdc0 in strcpy () from /lib/libc.so.6	290	#0 0xb7e4bdc0 in strcpy () from /lib/libc.so.6
275	#1 0x0804838c in run_it (input=0x0) at bad_code.c:7	291	#1 0x0804838c in run_it (input=0x0) at bad_code.c:7
276	#2 0x080483ba in main (argc=1, argv=0xbfd3a434) at bad_code.c:12	292	#2 0x080483ba in main (argc=1, argv=0xbfd3a434) at bad_code.c:12
277	</pre>	293	</pre>
278		294
279	<p>	295	<p>
280	Here we see that a lot more information is available for developers. Not only is	296	Here we see that a lot more information is available for developers. Not only is
281	variable information displayed, but even the exact line numbers of the source	297	function information displayed, but even the exact line numbers of the source
282	files. This method is the most preferred if you can spare the extra space.	298	files. This method is the most preferred if you can spare the extra space.
283	Here's how much the file size varies between debug, strip, and -g programs.	299	Here's how much the file size varies between debug, strip, and -ggdb enabled
		300	programs.
284	</p>	301	</p>
285		302
286	<pre caption="Filesize differences With -g flag">	303	<pre caption="Filesize differences With -ggdb flag">
287	<comment>(debug symbols stripped)</comment>	304	<comment>(debug symbols stripped)</comment>
288	-rwxr-xr-x 1 chris users 3140 6/28 13:11 bad_code	305	-rwxr-xr-x 1 chris users 3140 6/28 13:11 bad_code
289	<comment>(debug symbols enabled)</comment>	306	<comment>(debug symbols enabled)</comment>
290	-rwxr-xr-x 1 chris users 6374 6/28 13:10 bad_code	307	-rwxr-xr-x 1 chris users 6374 6/28 13:10 bad_code
291	<comment>(-g flag enabled)</comment>	308	<comment>(-ggdb flag enabled)</comment>
292	-rwxr-xr-x 1 chris users 7366 6/28 13:11 bad_code	309	-rwxr-xr-x 1 chris users 19552 6/28 13:11 bad_code
293	</pre>	310	</pre>
294		311
295	<p>	312	<p>
296	As you can see, -g adds about a 1000 more bytes to the file size over the one	313	As you can see, -ggdb adds about <e>13178</e> more bytes to the file size over the one
297	with debugging symbols. However, as shown above, this increase in file size can	314	with debugging symbols. However, as shown above, this increase in file size can
298	be worth it if presenting debug information to developers. Now that we have	315	be worth it if presenting debug information to developers. The backtrace can be
299	obtained the backtrace, we can save it to a file by copying and pasting from the	316	saved to a file by copying and pasting from the terminal (if it's a non-x based
300	terminal (if it's a non-x based terminal, you can use gpm. To keep this doc	317	terminal, you can use gpm. To keep this doc simple, I recommend you read up on
301	simple, I recommend you read up on the documentation for gpm to see how to copy	318	the documentation for gpm to see how to copy and paste with it). Now that we're
302	and paste with it). Now that we're done with <c>gdb</c>, we can quit.	319	done with <c>gdb</c>, we can quit.
303	</p>	320	</p>
304		321
305	<pre caption="Quitting GDB">	322	<pre caption="Quitting GDB">
306	(gdb) quit	323	(gdb) <i>quit</i>
307	The program is running. Exit anyway? (y or n) y	324	The program is running. Exit anyway? (y or n) <i>y</i>
308	$	325	$
309	</pre>	326	</pre>
310		327
311	<p>	328	<p>
312	This ends the walk-through of <c>gdb</c>. Using <c>gdb</c>, I hope that you will	329	This ends the walk-through of <c>gdb</c>. Using <c>gdb</c>, we hope that you will
313	be able to use it to create better bug reports. However, there are other types	330	be able to use it to create better bug reports. However, there are other types
314	of errors that can cause a program to fail during run time. One of the other	331	of errors that can cause a program to fail during run time. One of the other
315	ways is through improper file access. We can find those using a nifty little	332	ways is through improper file access. We can find those using a nifty little
316	tool called <c>strace</c>.	333	tool called <c>strace</c>.
317	</p>	334	</p>
…		…
325	<section>	342	<section>
326	<title>Introduction</title>	343	<title>Introduction</title>
327	<body>	344	<body>
328		345
329	<p>	346	<p>
330	Programs often use files to get configuration information, get access	347	Programs often use files to fetch configuration information, access hardware or
331	to hardware, or write logs. Often a program attempts to reach such files	348	write logs. Sometimes, a program attempts to reach such files incorrectly. A
332	incorrectly. A program called <c>strace</c> was created to help deal with	349	tool called <c>strace</c> was created to help deal with this. <c>strace</c>
333	this. <c>strace</c> traces system calls (hence the name) which include	350	traces system calls (hence the name) which include calls that use the memory and
334	calls that use the memory and files. For our example, we're going to take a	351	files. For our example, we're going to take a program foobar2. This is an
335	program foobar2. This is an updated version of foobar. However, during the	352	updated version of foobar. However, during the change over to foobar2, you notice
336	changeover to foobar2, you notice all your configurations are missing! In	353	all your configurations are missing! In foobar version 1, you had it setup to
337	foobar version 1, you had it setup to say "foo", but now it's using the default	354	say "foo", but now it's using the default "bar".
338	"bar".
339	</p>	355	</p>
340		356
341	<pre caption="Foobar2 With an invalid configuration">	357	<pre caption="Foobar2 With an invalid configuration">
342	$ <i>./foobar2</i>	358	$ <i>./foobar2</i>
343	Configuration says: bar	359	Configuration says: bar
…		…
353	<section>	369	<section>
354	<title>Using strace to track the issue</title>	370	<title>Using strace to track the issue</title>
355	<body>	371	<body>
356		372
357	<p>	373	<p>
358	Let's have <c>strace</c> log the results of the system calls. To do this, we run	374	We make <c>strace</c> log the results of the system calls. To do this, we run
359	<c>strace</c> with the -o[file] arguments. Let's use it on foobar2 like so:	375	<c>strace</c> with the -o[file] arguments. Let's use it on foobar2 as shown.
360	</p>	376	</p>
361		377
362	<pre caption="Running foobar2 through strace">	378	<pre caption="Running foobar2 through strace">
363	# <i>strace -ostrace.log ./foobar2</i>	379	# <i>strace -ostrace.log ./foobar2</i>
364	</pre>	380	</pre>
365		381
366	<p>	382	<p>
367	This creates a file called strace.log in the current directory. We check the	383	This creates a file called <path>strace.log</path> in the current directory. We
368	file, and shown below are the relevant parts from the file.	384	check the file, and shown below are the relevant parts from the file.
369	</p>	385	</p>
370		386
371	<pre caption="A Look At the strace Log">	387	<pre caption="A Look At the strace Log">
372	open(".foobar2/config", O_RDONLY) = 3	388	open(".foobar2/config", O_RDONLY) = 3
373	read(3, "bar", 3) = 3	389	read(3, "bar", 3) = 3
…		…
376	<p>	392	<p>
377	Aha! So There's the problem. Someone moved the configuration directory to	393	Aha! So There's the problem. Someone moved the configuration directory to
378	<path>.foobar2</path> instead of <path>.foobar</path>. We also see the program	394	<path>.foobar2</path> instead of <path>.foobar</path>. We also see the program
379	reading in "bar" as it should. In this case, we can recommend the ebuild	395	reading in "bar" as it should. In this case, we can recommend the ebuild
380	maintainer to put a warning about it. For now though, we can copy over the	396	maintainer to put a warning about it. For now though, we can copy over the
381	config file from .foobar and modify it to produce the correct results.	397	config file from <path>.foobar</path> and modify it to produce the correct
		398	results.
382	</p>	399	</p>
383		400
384	</body>	401	</body>
385	</section>	402	</section>
386	<section>	403	<section>
…		…
388	<body>	405	<body>
389		406
390	<p>	407	<p>
391	Now we've taken care of finding run time bugs. These bugs prove to be	408	Now we've taken care of finding run time bugs. These bugs prove to be
392	problematic when you try and run your programs. However, run time errors are	409	problematic when you try and run your programs. However, run time errors are
393	the least of your concern if your program won't compile at all. Let's take a	410	the least of your concerns if your program won't compile at all. Let's take a
394	look at how to address <c>emerge</c> compile errors.	411	look at how to address <c>emerge</c> compile errors.
395	</p>	412	</p>
396		413
397	</body>	414	</body>
398	</section>	415	</section>
…		…
404	<title>Introduction</title>	421	<title>Introduction</title>
405	<body>	422	<body>
406		423
407	<p>	424	<p>
408	<c>emerge</c> errors, such as the one displayed earlier, can be a major cause	425	<c>emerge</c> errors, such as the one displayed earlier, can be a major cause
409	of frustration for users. Reporting them is considered crucial for helping	426	of frustration for users. Reporting them is considered crucial for maintaining
410	maintain the health of Gentoo. Let's take a look at a sample ebuild, foobar2,	427	the health of Gentoo. Let's take a look at a sample ebuild, foobar2, which
411	which contains some build errors.	428	contains some build errors.
412	</p>	429	</p>
413		430
414	</body>	431	</body>
415	</section>	432	</section>
416	<section id="emerge_error">	433	<section id="emerge_error">
…		…
434	!!! Make failed!	451	!!! Make failed!
435	!!! If you need support, post the topmost build error, NOT this status message	452	!!! If you need support, post the topmost build error, NOT this status message
436	</pre>	453	</pre>
437		454
438	<p>	455	<p>
439	The compile is going smoothly, then it stops and presents an error message. This	456	The program is compiling smoothly when it suddenly stops and presents an error message. This
440	particular error can be split into 3 different sections, The compile, the build	457	particular error can be split into 3 different sections, The compile messages, the build
441	error, and the emerge error message as shown below.	458	error, and the emerge error message as shown below.
442	</p>	459	</p>
443		460
444	<pre caption="Parts of the error">	461	<pre caption="Parts of the error">
445	<comment>(Compile Error)</comment>	462	<comment>(Compilation Messages)</comment>
446	gcc -D__TEST__ -D__GNU__ -D__LINUX__ -L/usr/lib -I/usr/include -L/usr/lib/nspr/ -I/usr/include/fmod -c -o foobar2-7.o foobar2-7.c	463	gcc -D__TEST__ -D__GNU__ -D__LINUX__ -L/usr/lib -I/usr/include -L/usr/lib/nspr/ -I/usr/include/fmod -c -o foobar2-7.o foobar2-7.c
447	gcc -D__TEST__ -D__GNU__ -D__LINUX__ -L/usr/lib -I/usr/include -L/usr/lib/nspr/ -I/usr/include/fmod -c -o foobar2-8.o foobar2-8.c	464	gcc -D__TEST__ -D__GNU__ -D__LINUX__ -L/usr/lib -I/usr/include -L/usr/lib/nspr/ -I/usr/include/fmod -c -o foobar2-8.o foobar2-8.c
448	gcc -D__TEST__ -D__GNU__ -D__LINUX__ -L/usr/lib -I/usr/include -L/usr/lib/nspr/ -I/usr/include/fmod -c -o foobar2-9.o foobar2-9.c	465	gcc -D__TEST__ -D__GNU__ -D__LINUX__ -L/usr/lib -I/usr/include -L/usr/lib/nspr/ -I/usr/include/fmod -c -o foobar2-9.o foobar2-9.c
449	gcc -D__TEST__ -D__GNU__ -D__LINUX__ -L/usr/lib -I/usr/include -L/usr/lib/nspr/ -I/usr/include/fmod -c -o foobar2.o foobar2.c	466	gcc -D__TEST__ -D__GNU__ -D__LINUX__ -L/usr/lib -I/usr/include -L/usr/lib/nspr/ -I/usr/include/fmod -c -o foobar2.o foobar2.c
450		467
451	<comment>(Build Error)</comment>	468	<comment>(Build Error)</comment>
452	foobar2.c:1:17: ogg.h: No such file or directory	469	foobar2.c:1:17: ogg.h: No such file or directory
453	make: *** [foobar2.o] Error 1	470	make: *** [foobar2.o] Error 1
454		471
455	<comment>(Emerge Error)</comment>	472	<comment>(emerge Error)</comment>
456	!!! ERROR: sys-apps/foobar2-1.0 failed.	473	!!! ERROR: sys-apps/foobar2-1.0 failed.
457	!!! Function src_compile, Line 19, Exitcode 2	474	!!! Function src_compile, Line 19, Exitcode 2
458	!!! Make failed!	475	!!! Make failed!
459	!!! If you need support, post the topmost build error, NOT this status message	476	!!! If you need support, post the topmost build error, NOT this status message
460	</pre>	477	</pre>
461		478
462	<p>	479	<p>
463	The compile is what leads up to the error. Most often, it's good to at least	480	The compilation messages are what lead up to the error. Most often, it's good to
464	include 10 lines of compile information so that the developer knows where the	481	at least include 10 lines of compile information so that the developer knows
465	compile is at. Make errors are the actual error, and the information the	482	where the compilation was at when the error occurred.
466	developer needs. When you see "make: ***", this is often where the error has	483	</p>
467	occurred. Normally, you can copy and paste 10 lines above it and the developer	484
468	will be able to address the issue. However, this may not always work and we'll	485	<p>
469	take a look at an alternative shortly. The emerge error is what <c>emerge</c>	486	Make errors are the actual error and the information the developer needs. When
470	can address from the make error. Often people make the mistake of posting the	487	you see "make: ***", this is often where the error has occurred. Normally, you
		488	can copy and paste 10 lines above it and the developer will be able to address
		489	the issue. However, this may not always work and we'll take a look at an
		490	alternative shortly.
		491	</p>
		492
		493	<p>
		494	The emerge error is what <c>emerge</c> throws out as an error. Sometimes, this
		495	might also contain some important information. Often people make the mistake of
471	emerge error and that's all. This is useless by itself, but with make error and	496	posting the emerge error and that's all. This is useless by itself, but with
472	compile information, a developer can get what application and what version of	497	make error and compile information, a developer can get what application and
473	the package is failing. As a side note, make is commonly used as the build	498	what version of the package is failing. As a side note, make is commonly used as
474	process for programs (<b>but not always</b>). If you can't find a "make: ***"	499	the build process for programs (<b>but not always</b>). If you can't find a
475	error anywhere, then simply copy and paste 20 lines before the emerge error.	500	"make: ***" error anywhere, then simply copy and paste 20 lines before the
476	This should take care of most all build system error messages. Now let's say the	501	emerge error. This should take care of most all build system error messages. Now
477	errors seem to be quite large. 10 lines won't be enough to catch everything.	502	let's say the errors seem to be quite large. 10 lines won't be enough to catch
478	That's where PORT_LOGDIR comes into play.	503	everything. That's where PORT_LOGDIR comes into play.
479	</p>	504	</p>
480		505
481	</body>	506	</body>
482	</section>	507	</section>
483	<section>	508	<section>
484	<title>emerge and PORT_LOGDIR</title>	509	<title>emerge and PORT_LOGDIR</title>
485	<body>	510	<body>
486		511
487	<p>	512	<p>
488	PORT_LOGDIR is a portage variable that sets up a log directory for separate	513	PORT_LOGDIR is a portage variable that sets up a log directory for separate
489	emerge logs. Let's take a look and see what that entails. First, run your emerge	514	emerge logs. Let's take a look and see what that entails. First, run your
490	with PORT_LOGDIR set to your favorite log location. Let's say we have a	515	emerge with PORT_LOGDIR set to your favorite log location. Let's say we have a
491	location /var/log/portage. We'll use that for our log directory:	516	location <path>/var/log/portage</path>. We'll use that for our log directory:
492	</p>	517	</p>
493		518
494	<note>	519	<note>
495	In the default setup, /var/log/portage does not exist, and you will most likely	520	In the default setup, <path>/var/log/portage</path> does not exist, and you will
496	have to create it. If you do not, portage will fail to write the logs.	521	most likely have to create it. If you do not, portage will fail to write the
		522	logs.
497	</note>	523	</note>
498		524
499	<pre caption="emerge-ing With PORT_LOGDIR">	525	<pre caption="emerge-ing With PORT_LOGDIR">
500	# <i>PORT_LOGDIR=/var/log/portage emerge foobar2</i>	526	# <i>PORT_LOGDIR=/var/log/portage emerge foobar2</i>
501	</pre>	527	</pre>
…		…
536		562
537	<p>	563	<p>
538	<uri link="http://www.bugzilla.org">Bugzilla</uri> is what we at Gentoo use to	564	<uri link="http://www.bugzilla.org">Bugzilla</uri> is what we at Gentoo use to
539	handle bugs. Gentoo's Bugzilla is reachable by HTTPS and HTTP. HTTPS is	565	handle bugs. Gentoo's Bugzilla is reachable by HTTPS and HTTP. HTTPS is
540	available for those on insecure networks or simply paranoid :). For the sake of	566	available for those on insecure networks or simply paranoid :). For the sake of
541	consistency, I will be using the HTTPS version in the examples to follow. Head	567	consistency, we will be using the HTTPS version in the examples to follow. Head
542	over to <uri link="https://bugs.gentoo.org">Gentoo Bugs</uri> to see how it	568	over to <uri link="https://bugs.gentoo.org">Gentoo Bugs</uri> to see how it
543	looks.	569	looks.
544	</p>	570	</p>
545		571
546	<p>	572	<p>
547	One of the most frustrating thing for developers and bug-wranglers is finding	573	One of the most frustrating things for developers and bug-wranglers is finding
548	duplicate bug reports. This causes them valuable time they could be using to	574	duplicate bug reports. These cost them valuable time that they could otherwise
549	find new and more important bugs. Often, this can be prevented by a few simple	575	use to work on more important bugs. Often, this can be prevented by a few simple
550	search methods. So we're going to see how to search for bugs and find out if	576	search methods. So we're going to see how to search for bugs and find out if
551	you have one that's similar. For this example, we're going to use the xclass	577	you have one that's similar. For this example, we're going to use the xclass
552	emerge error that was used earlier:	578	emerge error that was used earlier.
553	</p>	579	</p>
554		580
555	<pre caption="xclass emerge error">	581	<pre caption="xclass emerge error">
556	/usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.2/include/g++-v3/backward/backward_warning.h:32:2:	582	/usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.2/include/g++-v3/backward/backward_warning.h:32:2:
557	warning: #warning This file includes at least one deprecated or antiquated	583	warning: #warning This file includes at least one deprecated or antiquated
…		…
582	!!! 'emake shared' failed	608	!!! 'emake shared' failed
583	</pre>	609	</pre>
584		610
585	<p>	611	<p>
586	So to begin searching, we head over to the <uri	612	So to begin searching, we head over to the <uri
587	link="http://bugs.gentoo.org/">Bugzilla Homepage</uri>.	613	link="https://bugs.gentoo.org/">Bugzilla Homepage</uri>.
588	</p>	614	</p>
589		615
590	<figure link="/images/docs/bugzie-homepage.png" caption="Bugzilla Homepage"/>	616	<figure link="/images/docs/bugzie-homepage.png" caption="Bugzilla Homepage"/>
591		617
592	<p>	618	<p>
593	In order to begin our search, we'll click on "Query Existing bug reports". The	619	We'll click on "Query Existing bug reports". The reason why we choose this
594	reason why we choose this versus the basic bug search is because the basic bug	620	over the basic bug search is because the basic bug search tends to give vague
595	search tends to give vague results and often hinders users from looking	621	results and often hinders users from looking through the results and finding the
596	through the results and finding the duplicate bug. Once we click on the query	622	duplicate bug. Once we click on the query screen, we reach the next page:
597	screen, we reach the next page:
598	</p>	623	</p>
599		624
600	<figure link="/images/docs/bugzie-search.png" caption="Bugzilla Search Page"/>	625	<figure link="/images/docs/bugzie-search.png" caption="Bugzilla Search Page"/>
601		626
602	<note>	627	<note>
603	If you've used the Advanced Search before, you'll most likely see that screen	628	If you've used the Advanced Search before, you'll most likely see that screen
604	instead.	629	instead.
605	</note>	630	</note>
606		631
607	<p>	632	<p>
608	Proceed on clicking the "Advanced Search" link to bring up the Advanced	633	Proceed by clicking on the "Advanced Search" link to bring up the Advanced
609	Search page:	634	Search page.
610	</p>	635	</p>
611		636
612	<figure link="/images/docs/bugzie-adv-search.png" caption="Advanced Search Page"/>	637	<figure link="/images/docs/bugzie-adv-search.png" caption="Advanced Search Page"/>
613		638
614	<p>	639	<p>
615	This is how the Advanced Search Page looks like. While it may seem overwhelming	640	This is how the Advanced Search Page looks like. While it may seem overwhelming
616	at first, we're going to look at a few simple areas to narrow down the rather	641	at first, we're going to look at a few simple areas to narrow down the rather
617	vague searches bugzilla returns.	642	vague searches bugzilla returns.
618	</p>	643	</p>
619		644
620	<figure link="/images/docs/bugzie-content.png" caption="Content"/>	645	<figure link="/images/docs/bugzie-content.png" caption="Content"/>
621		646
622	<p>	647	<p>
623	The first field is the summary of the bug. Here we're simply going to put the	648	The first field is the summary of the bug. Here we're simply going to put the
624	name of the package that's crashing. If you still don't get results, try	649	name of the package that's crashing. If bugzie doesn't return results, try
625	removing the package name, just in case someone didn't put that in the summary	650	removing the package name, just in case someone didn't put that in the summary
626	(highly unlikely, but I've seen my fair share of strange bug reports).	651	(highly unlikely, but we've seen a fair share of strange bug reports).
627	</p>	652	</p>
628		653
629	<p>	654	<p>
630	Product, Component, and Version should all be set to the default. This	655	Product, Component, and Version should all be set to the default. This
631	prevents us from being too specific and missing all the bugs.	656	prevents us from being too specific and missing all the bugs.
632	</p>	657	</p>
633		658
634	<p>	659	<p>
635	Comment is the important part. Use comment to list what appears to be a	660	Comment is the important part. Use the comment field to list what appears to be a
636	specific instance of the error. Basically, don't use anything like the	661	specific instance of the error. Basically, don't use anything like the
637	beginning of the build error, find a line that's before it stating a true	662	beginning of the build error, find a line that's before it stating a true
638	error. Also, you'll want to filter out any punctuation to prevent bugzilla	663	error. Also, you'll want to filter out any punctuation to prevent bugzilla
639	from interpreting the results the comment the wrong way. Example from the xclass	664	from interpreting the results the comment the wrong way. Example from the xclass
640	emerge error:	665	emerge error:
…		…
651	other xclass compile failure candidates.	676	other xclass compile failure candidates.
652	</p>	677	</p>
653		678
654	<p>	679	<p>
655	URI, Whiteboard, and Keywords can all be left alone. What we've entered so far	680	URI, Whiteboard, and Keywords can all be left alone. What we've entered so far
656	should be enough to find our bug. Let's see what we have filled out:	681	should be enough to find our bug. Let's take a look at what we have filled out.
657	</p>	682	</p>
658		683
659	<figure link="/images/docs/bugzie-comp-search.png" caption="Completed Search Form"/>	684	<figure link="/images/docs/bugzie-comp-search.png" caption="Completed Search Form"/>
660		685
661	<p>	686	<p>
662	Now we click on the Search button and here come the results:	687	Now we click on the Search button and here come the results...
663	</p>	688	</p>
664		689
665	<figure link="/images/docs/bugzie-search-result.png" caption="Search Results"/>	690	<figure link="/images/docs/bugzie-search-result.png" caption="Search Results"/>
666		691
667	<p>	692	<p>
668	Only 2 bugs! That's a lot easier to deal with. We click on the first one to	693	Only 2 bugs! That's a lot easier to deal with. We click on the first one to
669	check, and sure enough it's the one we're looking for:	694	check, and sure enough it's the one we're looking for.
670	</p>	695	</p>
671		696
672	<figure link="/images/docs/bugzie-located.png" caption="Bug Located"/>	697	<figure link="/images/docs/bugzie-located.png" caption="Bug Located"/>
673		698
674	<p>	699	<p>
675	Not only is it the one we want, but it has also been resolved. By checking the	700	Not only is it the one we want, but it has also been resolved. By checking the
676	last comment we see the solution and know what to do in order to resolve it.	701	last comment we see the solution and know what to do in order to resolve it.
677	Now, let's see what would have happened if we had not used the advanced search:	702	Now, let's see what would have happened if we had not used the advanced search.
678	</p>	703	</p>
679		704
680	<figure link="/images/docs/bugzie-basic-search-result.png" caption="Basic Search Results"/>	705	<figure link="/images/docs/bugzie-basic-search-result.png" caption="Basic Search Results"/>
681		706
682	<p>	707	<p>
…		…
713	</p>	738	</p>
714		739
715	<figure link="/images/docs/bugzie-homepage.png" caption="Bugzilla Homepage"/>	740	<figure link="/images/docs/bugzie-homepage.png" caption="Bugzilla Homepage"/>
716		741
717	<p>	742	<p>
718	Go ahead and click on "Report a Bug - Using the Guided format".	743	Click on "Report a Bug - Using the guided format".
719	</p>	744	</p>
720		745
721	<figure link="/images/docs/bugzie-prod-select.png" caption="Product Selection"/>	746	<figure link="/images/docs/bugzie-prod-select.png" caption="Product Selection"/>
722		747
723	<p>	748	<p>
724	As you can see, <b>major</b> emphasis has been placed on putting your bug in the	749	As you can see, <b>major</b> emphasis has been placed on putting your bug in the
725	right place. Gentoo Linux is where a large majority of bugs go. Despite this,	750	right place. Gentoo Linux is where a large majority of bugs go.
		751	</p>
		752
		753	<p>
726	some people will file ebuild bugs in portage development (assumption that	754	Despite this, some people will file ebuild bugs in portage development
727	portage team handles the portage tree) or infra (assumption that infra has	755	(assumption that portage team handles the portage tree) or infra (assumption
728	access to mirrors and rsync and can fix it directly). This is simply not how	756	that infra has access to mirrors and rsync and can fix it directly). This is
729	things work. Our bug goes in Gentoo Linux, as it's an ebuild bug. We head over	757	simply not how things work.
730	there and are presented with the multi-step bug reporting process.	758	</p>
		759
		760	<p>
		761	Another common misconception occurs with our Documentation bugs. For example, a
		762	user finds a bug with the <uri link="/proj/en/releng/catalyst/">Catalyst
		763	Docs</uri>. The general tendency is to file a bug under Docs-user, which gets
		764	assigned to the <uri link="http://gdp.gentoo.org">GDP</uri>, when it should
		765	actually go to a member of the <uri link="/proj/en/releng/">Release
		766	Engineering</uri> team. As a rule of thumb, only documentation under
		767	<path>http://www.gentoo.org/doc/*</path> is under the GDP. Anything under
		768	<path>http://www.gentoo.org/proj/*</path> is under the respective teams.
731	</p>	769	</p>
732		770
733	<note>	771	<note>
734	We would rather see a non-Gentoo Linux bug filed in Gentoo Linux than a Gentoo	772	We would rather see a bug whose product was not supposed to be Gentoo Linux but
735	Linux bug filed in non-Gentoo Linux projects! While neither is preferred, the	773	has been filed under the same rather than seeing a bug which belongs the Gentoo
		774	Linux product and filed elsewhere. While neither is preferred, the former is more
736	former is more acceptable and understandable (except website bugs.. we might	775	acceptable and understandable (except website bugs.. we might have an issue with
737	have an issue with that...)	776	that...).
738	</note>	777	</note>
739		778
740	<p>	779	<p>
741	Let us now proceed with Step 1...	780	Our bug goes in Gentoo Linux as it's an ebuild bug. We head over there and are
		781	presented with the multi-step bug reporting process. Let us now proceed with
		782	Step 1...
742	</p>	783	</p>
743		784
744	<figure link="/images/docs/bugzie-guide-step1.png" caption="Guided Format Step 1"/>	785	<figure link="/images/docs/bugzie-guide-step1.png" caption="Guided Format Step 1"/>
745		786
746	<p>	787	<p>
747	The first step here is really important (as the red text tells you). This is	788	The first step here is really important (as the red text tells you). This is
748	where you search to see that someone else hasn't hit the same bug you have, yet.	789	where you search to see that someone else hasn't hit the same bug you have, yet.
749	If you do skip this step, and a bug like yours already exists, it will be marked	790	If you do skip this step and a bug like yours already exists, it will be marked
750	as a DUPLICATE thus wasting a large amount of QA effort. To give you an idea,	791	as a DUPLICATE thus wasting a large amount of QA effort. To give you an idea,
751	the numbers that are struck out above are duplicate bugs. Now comes step 2,	792	the bug numbers that are struck out above are duplicate bugs. Now comes step 2,
752	where we give the information.	793	where we give the information.
753	</p>	794	</p>
754		795
755	</body>	796	</body>
756	</section>	797	</section>
757	<section>	798	<section>
758	<title>Required Information</title>	799	<title>Required Information</title>
759	<body>	800	<body>
760		801
761	<p>
762	Onto Step 2, let's see what we have here.
763	</p>
764
765	<figure link="/images/docs/bugzie-basic.png" caption="Basic Information"/>	802	<figure link="/images/docs/bugzie-basic.png" caption="Basic Information"/>
766		803
767	<p>	804	<p>
768	First, there's the product. This is Gentoo Linux, which we selected earlier.	805	Let us take a closer look at what's what.
769	Component is where the problem occurs. We use this to help us sort out the	806	</p>
770	severity of the issue (i.e. baselayout and core systems will be more important	807
771	than new ebuilds or application bugs). Here we select Application, as it is an	808	<ul>
772	application at fault. Hardware platform is what architecture you're running.	809	<li>
773	If you were running SPARC, you would set it to SPARC. For this example, we know	810	First, there's the Product. The product will narrow down the bug to a
774	this error can occur on multiple architectures, so we'll select All. Operating	811	specific area of Gentoo like Bugzilla (for bugs relating to
775	System is what Operating System you're using. Because Gentoo is considered a	812	bugs.gentoo.org), Docs-user(for User Documentation) or Gentoo Linux (for
		813	ebuilds and the like).
		814	</li>
		815	<li>
		816	Component is where exactly the problem occurs, more specifically which part
		817	of selected product the bug comes under. This makes classification easier.
		818	</li>
		819	<li>
		820	Hardware platform is what architecture you're running. If you were running
		821	SPARC, you would set it to SPARC.
		822	</li>
		823	<li>
		824	Operating System is what Operating System you're using. Because Gentoo is
776	"Meta-distribution", it can run on other operating systems beside Linux.	825	considered a "Meta-distribution", it can run on other operating systems
777	Examples are Gentoo on MacOS, Gentoo on FreeBSD, etc. For this example,	826	beside Linux.
778	we'll select All, as this can occur on all types of systems. Build Identifier	827	</li>
779	is what is being used to report the bugs (for logging purposes). You can just	828	</ul>
780	leave this as is. Let's see what we have so far:	829
781	</p>	830	<p>
		831	So, for our example bug, we have :
		832	</p>
		833
		834	<ul>
		835	<li>Product - Gentoo Linux (Since it is an ebuild issue)</li>
		836	<li>Component - Application (It is an application at fault, foobar2)</li>
		837	<li>Hardware Platform - All (This error could occur across architectures)</li>
		838	<li>Operation System - All (It could occur on all types of systems)</li>
		839	</ul>
782		840
783	<figure link="/images/docs/bugzie-basic-comp.png" caption="Completed Basic Information"/>	841	<figure link="/images/docs/bugzie-basic-comp.png" caption="Completed Basic Information"/>
784		842
785	<p>	843	<ul>
786	That does look good, so we'll begin with the actual report. In this instance,	844	<li>
787	we'll use the foobar2 bug as our example. URL is used to point to errors on a	845	Build Identifier is basically the User Agent of the browser that is being
		846	used to report the bugs (for logging purposes). You can just leave this as
		847	is.
		848	</li>
		849	<li>
		850	URL is optional and is used to point to errors on a site someplace
788	site someplace (pastebin, etc.). However, doing it inside the bug allows the	851	(pastebin, etc.). However, doing it inside the bug allows the developers be
789	developers be able to reference to it at any time and is preferred. Then we have	852	able to reference to it at any time and is preferred.
		853	</li>
		854	<li>
790	the summary. In the summary, you should put the package category, name, and	855	In the Summary, you should put the package category, name, and number.
		856	</li>
		857	</ul>
		858
		859	<p>
791	number. Not including the category really isn't too bad, but it's recommended.	860	Not including the category in the summary really isn't too bad, but it's
792	If you don't include the package name, however, we won't know what you're	861	recommended. If you don't include the package name, however, we won't know what
793	filling a bug for, and will have to ask you about it later. The version number	862	you're filling a bug for, and will have to ask you about it later. The version
794	is important for people searching for bugs. If 20 people filed bugs and not one	863	number is important for people searching for bugs. If 20 people filed bugs and
795	put a version number, how would people looking for similar bugs be able to tell	864	not one put a version number, how would people looking for similar bugs be able
796	if one was there's? They'd have to look through every single bug, which isn't	865	to tell if one was there's? They'd have to look through every single bug, which
797	too hard, but if there are say, 200 bugs.. it's not that easy. After all the	866	isn't too hard, but if there are say, 200 bugs.. it's not that easy. After all
798	package information, you'll want to include a small description of the incident.	867	the package information, you'll want to include a small description of the
799	Here's an example:	868	incident. Here's an example:
800	</p>	869	</p>
801		870
802	<figure link="/images/docs/bugzie-summary.png" caption="Summary"/>	871	<figure link="/images/docs/bugzie-summary.png" caption="Summary"/>
803		872
804	<p>	873	<p>
805	Just these simple rules can make handling bugs a lot easier. Next are the	874	These simple rules can make handling bugs a lot easier. Next are the details.
806	details. Here we put in the information about the bug. We'll demonstrate with an	875	Here we put in the information about the bug. We'll demonstrate with an example:
807	example:
808	</p>	876	</p>
809		877
810	<figure link="/images/docs/bugzie-details.png" caption="Details"/>	878	<figure link="/images/docs/bugzie-details.png" caption="Details"/>
811		879
812	<p>	880	<p>
813	So now the developer knows why we're filing the bug. They can then try to	881	Now the developer knows why we're filing the bug. They can then try to
814	reproduce it. Reproducibility tells us how often we were able to make the	882	reproduce it. Reproducibility tells us how often we were able to make the
815	problem recur. In this example, we can reproduce it any time simply by running	883	problem recur. In this example, we can reproduce it any time simply by running
816	foobar2. Let's put that information in:	884	foobar2. Let's put that information in.
817	</p>	885	</p>
818		886
819	<figure link="/images/docs/bugzie-reprod.png" caption="Reproduction"/>	887	<figure link="/images/docs/bugzie-reprod.png" caption="Reproduction"/>
820		888
821	<p>	889	<p>
822	So now we have explained how we found the bug. Next we want to explain what	890	We have explained how we found the bug. The next step is to explain what were
823	the results were and what we think they should be:	891	the results we got and what we think they should actually be.
824	</p>	892	</p>
825		893
826	<figure link="/images/docs/bugzie-results.png" caption="Results"/>	894	<figure link="/images/docs/bugzie-results.png" caption="Results"/>
827		895
828	<p>	896	<p>
829	Next we put additional information. This can be things such as stack traces,	897	We could then provide additional information. This could be things such as
		898	stack traces, <b>sections</b> (since the whole log is usually big and of not
830	<b>sections</b> of strace logs, but most importantly, your emerge --info output.	899	much use) of strace logs, but most importantly, your <c>emerge --info</c>
831	Here's an example:	900	output. Here's an example.
832	</p>	901	</p>
833		902
834	<figure link="/images/docs/bugzie-addl-info.png" caption="Additional Information"/>	903	<figure link="/images/docs/bugzie-addl-info.png" caption="Additional Information"/>
835		904
836	<p>	905	<p>
837	Lastly we select the severity of the bug. Please look this over carefully. In	906	Lastly we select the severity of the bug. Please look this over carefully. In
838	most cases it's OK to leave it as is and someone will raise/lower it for you.	907	most cases it's OK to leave it as is and someone will raise/lower it for you.
839	However, if you raise the severity of the bug, please make sure you read it over	908	However, if you raise the severity of the bug, please make sure you read it over
840	carefully and make sure you're not making a mistake. Here we will set it to the	909	carefully and make sure you're not making a mistake. A run down of the various
841	default of normal:	910	levels is given below.
842	</p>	911	</p>
		912
		913	<ul>
		914	<li>
		915	Blocker - The program just plain doesn't want to emerge or is a major
		916	hinderance to the system. For example a <c>baselayout</c> issue which
		917	prevents a system from booting up would be a sure candidate to be labelled
		918	blocker.
		919	</li>
		920	<li>
		921	Critical - The program has loss of data or severe memory leaks during
		922	runtime. Again, an important program like say <c>net-tools</c> failing to
		923	compile could be labelled critical. It won't prevent the system from
		924	starting up, but is quite essential for day to day stuff.
		925	</li>
		926	<li>
		927	Major - The program crashes, but nothing that causes your system severe
		928	damage or information loss.
		929	</li>
		930	<li>
		931	Minor - Your program crashes here and there with apparent workarounds.
		932	</li>
		933	<li>
		934	Normal - The default. If you're not sure leave it here unless it's a new
		935	build or cosmetic change, then read below for more information.
		936	</li>
		937	<li>Trivial - Things such as a mispelled word or whitespace clean up. </li>
		938	<li>
		939	Enhancement - A request to enable a new feature in a program, or more
		940	specifically <e>new ebuilds</e>.
		941	</li>
		942	</ul>
843		943
844	<figure link="/images/docs/bugzie-sev.png" caption="Severity"/>	944	<figure link="/images/docs/bugzie-sev.png" caption="Severity"/>
		945
		946	<p>
		947	Here, we'll set it to Normal.
		948	</p>
845		949
846	<p>	950	<p>
847	Now we can submit the bug report by clicking on the Submit Bug Report box. You	951	Now we can submit the bug report by clicking on the Submit Bug Report box. You
848	will now see your new bug come up. See <uri	952	will now see your new bug come up. See <uri
849	link="https://bugs.gentoo.org/show_bug.cgi?id=97265">Bug 97561</uri> for what	953	link="https://bugs.gentoo.org/show_bug.cgi?id=97265">Bug 97561</uri> for what
…		…
859	<title>Working With Your Bug</title>	963	<title>Working With Your Bug</title>
860	<section>	964	<section>
861	<body>	965	<body>
862		966
863	<p>	967	<p>
864	looking at the bug, we see the information we provided earlier:	968	Looking at the bug, we see the information we provided earlier. You will notice
		969	that the bug has been assigned to bug-wranglers@gentoo.org. This is the default
		970	location for Application component bugs.
865	</p>	971	</p>
866		972
867	<figure link="/images/docs/bugzie-new-basic.png" caption="New Bug Basic Information"/>	973	<figure link="/images/docs/bugzie-new-basic.png" caption="New Bug Basic Information"/>
868		974
869	<p>	975	<p>
870	And our details are there too:	976	The details we entered about the bug are available as well.
871	</p>	977	</p>
872		978
873	<figure link="/images/docs/bugzie-new-details.png" caption="New Bug Details"/>	979	<figure link="/images/docs/bugzie-new-details.png" caption="New Bug Details"/>
874		980
875	<p>	981	<p>
876	Now as you can see, the bug has been assigned to bug-wranglers@gentoo.org. This	982	However, bug-wranglers (usually) won't fix our bugs, so we'll reassign it to
877	is the default location for Application component bugs. However, bug-wranglers	983	someone that can (you can let bug-wranglers re-assign it for you as well). For
878	(usually) won't fix our bugs, so we'll reassign it to someone that can (you can
879	let bug-wranglers re-assign it for you as well). For this we use the package's
880	metadata.xml. You can normally find them in	984	this we use the package's metadata.xml. You can normally find them in
881	/usr/portage/category/package/metadata.xml. Here's one I've made up for foobar2:	985	<path>/usr/portage/category/package/metadata.xml</path>. Here's one I've made up
		986	for foobar2.
882	</p>	987	</p>
		988
		989	<note>
		990	You have to be the reporter of the bug or a member of certain Gentoo Bugzilla
		991	groups (like Gentoo Developers) to be able to reassign bugs.
		992	</note>
883		993
884	<pre caption="metadata.xml">	994	<pre caption="metadata.xml">
885	<?xml version="1.0" encoding="UTF-8"?>	995	<?xml version="1.0" encoding="UTF-8"?>
886	<!DOCTYPE pkgmetadata SYSTEM "http://www.gentoo.org/dtd/metadata.dtd">	996	<!DOCTYPE pkgmetadata SYSTEM "http://www.gentoo.org/dtd/metadata.dtd">
887	<pkgmetadata>	997	<pkgmetadata>
…		…
895	</longdescription>	1005	</longdescription>
896	</pkgmetadata>	1006	</pkgmetadata>
897	</pre>	1007	</pre>
898		1008
899	<p>	1009	<p>
900	Notice the maintainer section.T his lists the maintainer of the package, which	1010	Notice the maintainer section. This lists the maintainer of the package, which
901	in this case is myself, Chris White. The email listed is chriswhite@gentoo.org.	1011	in this case is myself, Chris White. The email listed is chriswhite@gentoo.org.
902	We will use this to re-assign the bug to the proper person. To do this, click	1012	We will use this to re-assign the bug to the proper person. To do this, click
903	the bubble next to Reassign bug to, then fill in the email:	1013	the bubble next to Reassign bug to, then fill in the email.
904	</p>	1014	</p>
905		1015
906	<note>	1016	<note>
907	A package without a metadata.xml file should be re-assigned to	1017	A bug for a package without a metadata.xml file should be re-assigned to
908	maintainer-needed@gentoo.org.	1018	maintainer-needed@gentoo.org and a package that needs a Gentoo Developer to
		1019	maintain should be assigned to maintainer-wanted@gentoo.org.
909	</note>	1020	</note>
910		1021
911	<figure link="/images/docs/bugzie-reassign.png" caption="Bug Reassignment"/>	1022	<figure link="/images/docs/bugzie-reassign.png" caption="Bug Reassignment"/>
912		1023
913	<p>	1024	<p>
…		…
920	</p>	1031	</p>
921		1032
922	<figure link="/images/docs/bugzie-new-attach.png" caption="New Attachment"/>	1033	<figure link="/images/docs/bugzie-new-attach.png" caption="New Attachment"/>
923		1034
924	<p>	1035	<p>
925	Now we have to attach the log. We click on the "Browse..." button under "File"	1036	Now we have to attach the log. Let's go throught it step wise.
926	and select the strace log. For Description, we'll put "strace log". Content-Type	1037	</p>
927	is what kind of file we're attaching. A common mistake is to set it to	1038
928	auto. In most cases it's best to manually set it. Our log file is a	1039	<ul>
929	plain text file, so we select "plain text (text/plain)". Obsoletes are for when	1040	<li>
930	you are attaching a revision to a previously attached file. You can simply click	1041	File - This is the location of the file in your machine. In this example,
931	a check box next to the old file and Bugzilla will cross it out in the bug,	1042	the location of <path>strace.log</path>. You can use the "Browse..." button
932	indicating that the attachment has been obsoleted. Reassignment means you want	1043	to select the file, or enter the path directly in the text field.
933	to take the bug yourself. I rarely tend to use this.. and I don't think you will	1044	</li>
934	need to at some point (unless you create great patches and we don't care about	1045	<li>
935	you taking our bugs ;). Comments are for leaving comments about the file you're	1046	Description - A short one liner, or a few wors describing the attachment.
936	posting. We'll put "Here is the strace file you requested". Now we have	1047	We'll just enter strace.log here, since that's quite self-explanatory.
937	something like this:	1048	</li>
		1049	<li>
		1050	Content Type - This is the type of the file we're attaching to the bug.
		1051	</li>
		1052	<li>
		1053	Obsoletes - If there were attachements submitted to the bug before the
		1054	current one, you have an option of declaring them obsoleted by yours. Since
		1055	we have no prior attachments to this bug, we need not bother.
		1056	</li>
		1057	<li>
		1058	Comment - Enter comments that will be visible along with the attachments.
		1059	You could elaborate on the attachment here, if needed.
		1060	</li>
		1061	</ul>
		1062
938	</p>	1063	<p>
		1064	With respect to Content Type, here are a few more details. You can check the
		1065	"patch" checkbox if you're submitting a patch. Otherwise, you could ask
		1066	Bugzilla to "auto-detect" the file type (not advisable). The other options are
		1067	"select from list", which is most frequently used. Use plain text (text/plain)
		1068	for <e>most</e> attachments except binary files like images (which can use
		1069	image/gif, image/jpeg or image/png depending on type) or compressed files like
		1070	.tar.bz2 which would use application/octet-stream as content type.
		1071	</p>
		1072
939		1073
940	<figure link="/images/docs/bugzie-new-attach-comp.png" caption="New Attachment Completed"/>	1074	<figure link="/images/docs/bugzie-new-attach-comp.png" caption="New Attachment Completed"/>
941		1075
942	<p>	1076	<p>
943	We Submit the patch and it is now reflected on the bug report.	1077	We submit <path>strace.log</path> and it is reflected on the bug report.
944	</p>	1078	</p>
945		1079
946	<figure link="/images/docs/bugzie-strace.png" caption="Attached strace log"/>	1080	<figure link="/images/docs/bugzie-strace.png" caption="Attached strace log"/>
947		1081
948	<p>	1082	<p>
949	Now, while we're waiting another person notices your bug during step 1 of the	1083	We've mentioned before that sometimes ebuilds will tell you to attach a file in
950	Guided Format. This person wants to see the status of the bug as well. He or	1084	the emerge error. An example can be seen below.
951	she may do so by putting their email in the Add CC field like so:	1085	</p>
		1086
		1087	<pre caption="Example File Attachment Request">
		1088	configure: error: PNG support requires ZLIB. Use --with-zlib-dir=<DIR>
		1089
		1090	!!! Please attach the config.log to your bug report:
		1091	!!! /var/tmp/portage/php-5.0.3-r1/work/php-5.0.3/config.log
		1092
		1093	!!! ERROR: dev-php/php-5.0.3-r1 failed.
		1094	!!! Function econf, Line 485, Exitcode 0
		1095	!!! econf failed
		1096	!!! If you need support, post the topmost build error, NOT this status message.
		1097	</pre>
		1098
		1099	<p>
		1100	Please attach any file mentioned like this to your bug report.
		1101	</p>
		1102
		1103	<p>
		1104	While we're doing all this, suppose another person finds your bug by searching
		1105	through bugzilla and is curious to keep track of the bug, they may do so by
		1106	putting their email in the Add CC field of the bug as shown below. You could
		1107	also keep track of other bugs by following the same method.
952	</p>	1108	</p>
953		1109
954	<figure link="/images/docs/bugzie-add-email.png" caption="Adding Email To CC:"/>	1110	<figure link="/images/docs/bugzie-add-email.png" caption="Adding Email To CC:"/>
955		1111
956	<note>	1112	<note>
957	Email addresses must be registered with bugzilla. In order to CC multiple	1113	Email addresses must be registered with Gentoo Bugzilla. In order to CC multiple
958	addresses, simply separate them with commas or spaces.	1114	addresses, simply separate them with commas or spaces.
959	</note>	1115	</note>
960		1116
961	<p>	1117	<p>
962	After all this work, the bug can undergo various status markings. Here's a few:	1118	After all this work, the bug can undergo various status markings. This is
		1119	usually done by the Gentoo Developers and sometimes by the reporter. The
		1120	following are the various possible states a bug may go through during its
		1121	lifetime.
963	</p>	1122	</p>
964		1123
965	<ul>	1124	<ul>
966	<li>	1125	<li>
967	UNCONFIRMED - You're generally not going to see this too often. This	1126	UNCONFIRMED - You're generally not going to see this too often. This means
968	means that a bug reporter has opened a bug using the advanced method and is	1127	that a bug reporter has opened a bug using the advanced method and is
969	uncertain his or her bug is an actual bug.	1128	uncertain his or her bug is an actual bug.
970	</li>
971	<li>	1129	</li>
972	NEW - Bugs that are first opened are considered new.	1130	<li>NEW - Bugs that are first opened are considered new.</li>
973	</li>
974	<li>	1131	<li>
975	ASSIGNED - When the person you've assigned the bug too validates your	1132	ASSIGNED - When the person you've assigned the bug too validates your bug,
976	bug, it will often receive ASSIGNED status while they figure out the issue.	1133	it will often receive ASSIGNED status while they figure out the issue.
977	This let's you know that they've accepted your bug as a real bug.	1134	This lets you know that they've accepted your bug as a real bug.
978	</li>
979	<li>	1135	</li>
		1136	<li>
980	REOPENED - Someone has resolved a bug and you think the solution is not	1137	REOPENED - Someone has resolved a bug and you think the solution is not
981	feasible or the problem still persists. At this point, you may re-open the	1138	feasible or the problem still persists. At this point, you may re-open the
982	bug. However <b>please do not abuse this</b>. If a developer closes the bug a	1139	bug. Please <b>do not abuse this</b>. If a developer closes the bug a
983	second or even third time, chances are that your bug is closed.	1140	second or third time, chances are that your bug is closed.
984	</li>	1141	</li>
		1142	<li>
		1143	RESOLVED - A firm decision has been taken on the bug. Usually goes onto
		1144	FIXED to indicate the bug is solved and the matter closed although various
		1145	other resolutions are possible. We'll look into those a little later.
		1146	</li>
		1147	<li>
		1148	VERIFIED - The steps take to work the bug are correct. This is usually a QA
		1149	thing.
		1150	</li>
		1151	<li>
		1152	CLOSED - Basically means RIP for the bug and it's buried under the never
		1153	ending flow of new bugs.
		1154	</li>
985	</ul>	1155	</ul>
986		1156
987	<p>	1157	<p>
988	Now shortly afterward, I find the error in the strace log. I resolve the bug	1158	Now shortly afterward, we find the error in the strace log and fix the bug and
989	as RESOLVED FIXED and say that there was a change in the location of	1159	mark it as RESOLVED FIXED and mention that there was a change in the location
990	configuration files, and that I will update the ebuild with a warning about it.	1160	of configuration files, and that I will update the ebuild with a warning about
991	The bug now becomes resolved, and you are displayed with this:	1161	it. The bug now becomes resolved, and you are shown the following.
992	</p>	1162	</p>
993		1163
994	<figure link="/images/docs/bugzie-reso.png" caption="Resolved Bug"/>	1164	<figure link="/images/docs/bugzie-reso.png" caption="Resolved Bug"/>
995		1165
996	<p>	1166	<p>
997	Also note the section here:	1167	A little below, you'll see the following:
998	</p>	1168	</p>
999		1169
1000	<figure link="/images/docs/bugzie-options.png" caption="Bug Options"/>	1170	<figure link="/images/docs/bugzie-options.png" caption="Bug Options"/>
1001		1171
1002	<p>	1172	<p>
1003	This gives you the option of Reopening the bug if you wish to (i.e. the developer	1173	This gives you the option of Reopening the bug if you wish to (i.e. the
1004	thinks it's resolved but it's really not to your standards). Now our bug is	1174	developer thinks it's resolved but it's really not to your standards). Now our
1005	fixed! However, different resolutions can occur. Here's a small list:	1175	bug is fixed! However, different resolutions can occur. Here's a small list:
1006	</p>	1176	</p>
1007		1177
1008	<ul>	1178	<ul>
1009	<li>	1179	<li>
1010	FIXED - The bug is fixed, follow the instructions to resolve your	1180	FIXED - The bug is fixed, follow the instructions to resolve your issue.
1011	issue.
1012	</li>
1013	<li>	1181	</li>
		1182	<li>
1014	INVALID - You did not do something specifically documented, causing the	1183	INVALID - You did not do something specifically documented, causing the
1015	bug	1184	bug.
1016	</li>
1017	<li>	1185	</li>
1018	DUPLICATE - You didn't use this guide and reported a duplicate bug	1186	<li>DUPLICATE - You didn't use this guide and reported a duplicate bug.</li>
1019	:)	1187	<li>
1020	</li>	1188	WORKSFORME - Developer/person assigned the bug cannot reproduce your error.
1021	<li>	1189	</li>
1022	WORKSFORME - Developer/person assigned the bug cannot reproduce your	1190	<li>
1023	error	1191	CANTFIX - Somehow the bug cannot be solved because of certain
		1192	circumstances. These circumstances will be noted by the person taking the
		1193	bug.
1024	</li>	1194	</li>
		1195	<li>
		1196	WONTFIX - This is usually applied to new ebuilds or feature requests.
		1197	Basically the developer does not want to add a certain feature because it
		1198	is not needed, a better alternative exists, or it's just plain broken.
		1199	Sometimes you may be given a solution to get said issue resolved.
		1200	</li>
		1201	<li>
		1202	UPSTREAM - The bug cannot be fixed by the Gentoo development team, and have
		1203	requested you take the problem upstream (the people that actually made the
		1204	program) for review. Upstream has a few ways of handling bugs. These
		1205	include mailing lists, irc channels, and even bug reporting systems. If
		1206	you're not sure how to contact them, ask in the bug and someone will point
		1207	you to the right direction.
		1208	</li>
1025	</ul>	1209	</ul>
1026		1210
		1211	<p>
		1212	Sometimes, before the bug can be resolved, a developer may request that you
		1213	test an updated ebulid. In the next chapter we'll take a look at testing
		1214	ebuilds.
		1215	</p>
		1216
1027	</body>	1217	</body>
		1218	</section>
		1219	</chapter>
		1220
		1221	<chapter>
		1222	<title>Testing Ebuilds</title>
1028	</section>	1223	<section>
		1224	<title>Getting The Files</title>
		1225	<body>
1029		1226
		1227	<p>
		1228	Let's say that you reported a bug for the foobar2 compile fix from earlier. Now
		1229	developers might find out what the problem is and might need you to test the
		1230	ebuild for them to be sure it works for you as well:
		1231	</p>
		1232
		1233	<figure link="/images/docs/bugzie-ebuild-request.png" caption="Ebuild Test Request"/>
		1234
		1235	<p>
		1236	Some rather confusing vocabulary is used here. First off, let's see what an
		1237	overlay is. An overlay is a special directory like <path>/usr/portage</path>,
		1238	the difference being that when you <c>emerge sync</c>, files contained within it
		1239	will not be deleted. Luckily, a special <path>/usr/local/portage</path>
		1240	directory is created for that purpose. Let's go ahead and set our portage
		1241	overlay in<path>/etc/make.conf</path>. Open make.conf up in your favorite editor
		1242	and add this towards the end.
		1243	</p>
		1244
		1245	<pre caption="Setting Up PORTDIR_OVERLAY">
		1246	PORTDIR_OVERLAY="/usr/local/portage"
		1247	</pre>
		1248
		1249	<p>
		1250	Now we'll want to create the appropriate directories to put our test ebuild
		1251	files in. In this case, we're supposed to put them in sys-apps/foobar2. You'll
		1252	notice that the second comment asks for a files directory for the patch. The
		1253	files directory holds the digests (md5sums of files for a particular version of
		1254	a package) and any other required files that aren't included with the standard
		1255	source archive (patches, init.d scripts, etc). This is a subdir in the package
		1256	directory called files. Go ahead and create these directories:
		1257	</p>
		1258
		1259	<pre caption="Setting Up The Category And Package Directories">
		1260	# <i>mkdir -p /usr/local/portage/sys-apps/foobar2/files</i>
		1261	</pre>
		1262
		1263	<note>
		1264	The -p in mkdir creates not only the directory you want but also any missing
		1265	parent directories as well (sys-apps and foobar2 in this case).
		1266	</note>
		1267
		1268	<p>
		1269	Ok now, we can go ahead and download the files. First, download the ebuild
		1270	into <path>/usr/local/portage/sys-apps/foobar2</path>, and then add the patch
		1271	to <path>/usr/local/portage/sys-apps/foobar2/files</path>. Now that we have the
		1272	files, we can begin working on testing the ebuild.
		1273	</p>
		1274
		1275	</body>
		1276	</section>
		1277	<section>
		1278	<title>Testing The ebuild</title>
		1279	<body>
		1280
		1281	<p>
		1282	The process to create an ebuild that can be used by emerge is fairly simple. You
		1283	must create a Manifest and a digest file for the ebuild. This can be done with
		1284	the ebuild command. Run it as shown.
		1285	</p>
		1286
		1287	<pre caption="Creating the Manifest and digest files">
		1288	# <i>ebuild foobar2-1.0.ebuild digest</i>
		1289	>>> Generating digest file...
		1290	<<< foobar2-1.0.tar.bz2
		1291	>>> Generating manifest file...
		1292	<<< foobar2-1.0.ebuild
		1293	<<< files/digest-foobar2-1.0
		1294	<<< files/foobar2-1.0-Makefile.patch
		1295	>>> Computed message digests.
		1296	</pre>
		1297
		1298	<p>
		1299	Now let's test to see if it works as it should.
		1300	</p>
		1301
		1302	<pre caption="Testing With emerge -pv">
		1303	# <i>emerge -pv foobar2</i>
		1304
		1305	These are the packages that I would merge, in order:
		1306
		1307	Calculating dependencies ...done!
		1308	[ebuild N ] sys-apps/foobar2-1.0 0 kB [1]
		1309
		1310	Total size of downloads: 0 kB
		1311	Portage overlays:
		1312	[1] /usr/local/portage
		1313	</pre>
		1314
		1315	<p>
		1316	It does seem to have worked! You'll notice the [1] next to the [ebuild] line.
		1317	That points to <path>/usr/local/portage</path>, which is the overlay we created
		1318	earlier. Now we go ahead and emerge the package.
		1319	</p>
		1320
		1321	<pre caption="Emerge Result">
		1322	# <i>emerge foobar2</i>
		1323	Calculating dependencies ...done!
		1324	<comment>(compile info snipped)</comment>
		1325	>>> Unpacking foobar2-1.0.tar.bz2 to /var/tmp/portage/foobar2-1.0/work
		1326	* Applying foobar2-1.0-Makefile.patch ... [ ok ]
		1327	<comment>(compile info snipped)</comment>
		1328	>>> Merging sys-apps/foobar2-1.0 to /
		1329	>>> chris +sandbox(preinst)
		1330	--- /usr/
		1331	--- /usr/bin/
		1332	>>> /usr/bin/foobar2
		1333	</pre>
		1334
		1335	<p>
		1336	In the first section we see that the emerge started off as it should. The second
		1337	section shows our patch being applied successfully by the "[ ok ]" status
		1338	message to the right. The last section tells us the program compiled ok. The
		1339	patch works! Now we can go and let the developer know that their patch works
		1340	fine, and that they can commit the fix to portage.
		1341	</p>
		1342
		1343	</body>
		1344	</section>
1030	<section>	1345	<section>
1031	<title>Conclusion</title>	1346	<title>Conclusion</title>
1032	<body>	1347	<body>
1033		1348
1034	<p>	1349	<p>
1035	This concludes the howto on working with Bugzilla. I hope you find this useful.	1350	This concludes the howto on working with Bugzilla. I hope you find this useful.
1036	If you have any questions, comments, or ideas regarding this document, please	1351	If you have any questions, comments, or ideas regarding this document, please
1037	send them to me at <mail	1352	send them to me at <mail>chriswhite@gentoo.org</mail>. Special thanks go to
1038	link="chriswhite@gentoo.org">chriswhite@gentoo.org</mail>. Special
1039	thanks go to moreon for his notes on -g flags and compile errors, the people at	1353	moreon for his notes on -g flags and compile errors, the people at #gentoo-bugs
1040	#gentoo-bugs for helping out with bug-wrangling, and Griffon26 for his notes on	1354	for helping out with bug-wrangling, Griffon26 for his notes on
1041	maintainer-needed.	1355	maintainer-needed, robbat2 for general suggestions and fox2mike for fixing up
		1356	the doc and adding stuff as needed.
1042	</p>	1357	</p>
1043		1358
1044	</body>	1359	</body>
1045	</section>	1360	</section>
1046	</chapter>	1361	</chapter>

 Legend:



Removed from v.1.1
 


changed lines


 
Added in v.1.6
 Legend:



Removed from v.1.1
 


changed lines


 
Added in v.1.6
-Removed from v.1.1
+Added in v.1.6

	ViewVC Help
Powered by ViewVC 1.1.20