en/overlays/devguide.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE guide SYSTEM "/dtd/guide.dtd">

<guide link="/proj/en/overlays/devguide.xml">
<title>Gentoo Overlays: Developers' Guide</title>

<author title="Author">
  <mail link="stuart">Stuart Herbert</mail>
</author>
<author title="Author">
  <mail link="jokey">Markus Ullmann</mail>
</author>
<author title="Author">
  <mail link="robbat2">Robin H. Johnson</mail>
</author>

<abstract>This guide helps developers understand how to use the Gentoo
Overlays service.</abstract>

<license/>

<version>2.3</version>
<date>2009-01-18</date>

<chapter>
<title>Introduction</title>

<section>
<title>Audience</title>
<body>
<p>This document has been written for Gentoo developers and Gentoo staff members.  If you are a Gentoo user, or you just want to start downloading and using overlays, please see <uri link="/proj/en/overlays/userguide.xml">the Gentoo Overlays User Guide</uri> instead.</p>
</body>
</section>

<section>
<title>Who Can Use overlays.gentoo.org?</title>
<body>
<p>Any Gentoo project, or Gentoo developer, can have their own overlay hosted on (git.)overlays.gentoo.org, with the RSS feed from the changelog included on <uri link="http://overlays.gentoo.org">the overlays.gentoo.org planet</uri>.</p>

<p>Any User can download and use the contents of any hosted
overlay.  If you choose, you can also give users write access to your
overlay.</p>
</body>
</section>

<section>
<title>What Does overlays.gentoo.org Give Me?</title>
<body>
<p>The (git.)overlays.gentoo.org service currently provides:</p>
<ul>
<li><uri link="http://trac.edgewall.com">Trac</uri> (a wiki w/ integrated subversion browser), for quickly creating and maintaining documentation about your Subversion based overlay</li>
<li>Publishing the changelog for your overlay on <uri link="http://overlays.gentoo.org/">the o.g.o homepage</uri>, so that everyone who's interested can see what's going on</li>
<li><uri link="http://git.or.cz/gitwiki/Gitweb">gitweb</uri> - provides full-fledged web interface for viewing Git repositories.</li>
</ul>

<p>... all hosted on secure, backed-up Gentoo infrastructure, administered by
<uri link="/proj/en/infrastructure">the Gentoo Infrastructure team</uri> (hardware / base OS) and <uri link="/proj/en/overlays">the Gentoo Overlays team</uri>
(wiki / VCS / ACLs).</p>

<p>Each overlay has separate authentication lists for Trac, Subversion and Git.
It's no problem at all to give someone write access to just Trac (e.g. for
writing documentation) without giving them write access to Subversion.</p>
</body>
</section>

<section>
<title>Why Should We Use overlays.gentoo.org?</title>
<body>
<p>You don't have to.  You don't have to have an overlay at all, and if you do
have one, you are absolutely free to host your own overlay somewhere else.
You don't have to host an overlay on o.g.o for it to be considered
"official".</p>

<p>The advantage of using overlays.gentoo.org is that we already have everything
setup for you.  You don't need to admin your own server, or worry about
software upgrades.  We take care of all of that for you.</p>
</body>
</section>

<section>
<title>When Should We Not Use overlays.gentoo.org?</title>
<body>
<p>The purpose of o.g.o is to help bridge the gap between developers and users.
Gentoo is a community-based distribution, and we believe that our users are
just as important a part of that community as developers are.</p>

<p>All of the overlays hosted on o.g.o are there for all users to download and
use.  It's for users to make their own decision about what software they
install on their computers - and that includes choosing to use your overlay.
Some users will make bad decisions, and end up breaking their computer.  They
may even end up blaming Gentoo as a result.  That's okay; these people
probably go around blaming everyone but themselves for their own
mistakes anyway, and there's probably nothing you can do to change that.
But that still doesn't give any of us the right to choose for them.</p>

<p>Users are free (in fact, they are encouraged) to provide constructive
feedback about anything to do with Gentoo - including all of the overlays
hosted on o.g.o.  That feedback can come via bugs.g.o, via email to your
project team or directly to you, via the forums, or via IRC.  We're not
talking about genuinely abusive users; we have no time for those, and no-one
expects you to have any time for them either.</p>

<warn>If you're not happy with users using your overlay, and / or if you don't want
users bothering you about your overlay, then don't use o.g.o to host your
overlay.</warn>

<p>o.g.o used to have restrictions of not being the $UPSTREAM for packages.
This restriction has been adjusted. We do offer hosting as the $UPSTREAM now,
but only for packages that are Gentoo-specific or important to the running of
Gentoo. Other hosting may be more suitable, some services in this vein are:
<uri link="http://www.sourceforge.net/">SourceForge.net</uri>, <uri
link="http://www.berlios.de">Berlios</uri>, or Patrick's <uri
link="http://www.gentooexperimental.org">GentooExperimental.org</uri>.</p>
</body>
</section>
</chapter>

<chapter>
<title>Requesting An Overlay</title>

<section>
<title>Introduction</title>
<body>
<p>There are two types of overlay - "project" overlays, and "developer"
overlays.  The only difference between them is responsibility and
accountability.</p>

<impo>Before requesting an overlay, please make sure that you have read our <uri link="/proj/en/overlays/policy.xml">Policy Document</uri>.  It clearly sets out what's allowed and what is not, and what your responsibilities will be.</impo>
</body>
</section>

<section>
<title>Project Overlays</title>
<body>
<p>"Project" overlays are overlays for official Gentoo projects.  An example is <uri link="http://overlays.gentoo.org/proj/php">the PHP Overlay</uri>.</p>

<p>An official Gentoo project is a project that has a project page on
www.gentoo.org, and that has an elected lead.  (This definition comes from the
metastructure document).  The project lead(s) are responsible for the project overlay, including its contents, and any problems it causes other Gentoo projects and developers.</p>

<p>To request a SVN project overlay, the project's lead just needs to pop into
#gentoo-overlays on IRC and ask for an overlay to be created.  Or, if he/she
prefers, drop an email to overlays@gentoo.org.  We'll take care of the rest,
including granting write access to all the members of your project (as listed
on your project page).</p>

<p>To request a Git project overlay, just visit the <uri
link="http://git.overlays.gentoo.org">git.overlays</uri> site, and follow the
setup instructions, emailing the completed template as directed.</p>

<p>For an SVN request, we will:</p>
<ul>
<li>create your overlay (trac site + svn)</li>
<li>add your overlay's RSS feed to the o.g.o homepage</li>
<li>create an o.g.o account for you if you don't already have one</li>
<li>give you write access to your overlay's Trac wiki and Subversion
repository</li>
<li>give write access to all project members who already have an o.g.o
account</li>
</ul>

<p>For an Git request, we will:</p>
<ul>
<li>create your overlay (git, gitweb, no trac)</li>
<li>add your overlay's RSS feed to the o.g.o homepage</li>
<li>create an git.o.g.o account for you if you don't already have one</li>
<li>give you write access to your overlays Git repository</li>
<li>give write access to all project members who already have an o.g.o account</li>
</ul>

</body>
</section>

<section>
<title>Developer Overlays</title>
<body>

<p>"Developer" overlays are overlays owned by individual Gentoo developers.  One
 example is <uri link="http://overlays.gentoo.org/dev/beandog">beandog's overlay</uri>.</p>

<p>If you have an @gentoo.org email address, and you've passed the ebuild quiz,
then you can have your own developer overlay on o.g.o.</p>

<p>To request a SVN developer overlay, just pop into #gentoo-overlays on IRC
and ask for an overlay to be created for you.  Or, if you prefer, drop an email
to overlays@gentoo.org.</p>

<p>To request a Git developer overlay, just visit the <uri
link="http://git.overlays.gentoo.org">git.overlays</uri> site, and follow the
setup instructions, emailing the completed template as directed.</p>

<p>For an SVN request, we will:</p>
<ul>
<li>create your overlay (trac site + svn)</li>
<li>add your overlay's RSS feed to the o.g.o homepage</li>
<li>create an o.g.o account for you if you don't have one already</li>
<li>give you write access to your overlay's Trac wiki and Subversion
repository</li>
</ul>

<p>For an Git request, we will:</p>
<ul>
<li>create your overlay (git, gitweb, no trac)</li>
<li>add your overlay's RSS feed to the o.g.o homepage</li>
<li>create an git.o.g.o account for you if you don't already have one</li>
<li>give you write access to your overlays Git repository</li>
</ul>
</body>
</section>

<section>
<title>A Word About Accounts</title>
<body>
<p>Because o.g.o is designed to support a blend of both Gentoo developers and
Gentoo users, we don't create 'real' system-level accounts on the o.g.o host.
</p>

<impo>You do *not* have SSH access to o.g.o.</impo>
</body>
</section>
</chapter>

<chapter>
<title>Working With Your Overlay</title>

<section>
<title>Introduction</title>
<body>
<p>You can access your overlay as soon as it has been created.  Project and
developer overlays have different URLs, so that everyone can tell one from the
other, but otherwise they are identical in every way.</p>

<p>There are *no* read restrictions on overlays or wikis.  Everyone has full
read access to all overlays and wikis.  If you need a 'secret' overlay of some
kind, then o.g.o is not for you.</p>
</body>
</section>

<section>
<title>Accessing Project Overlays</title>
<body>
<p>If your project overlay is called 'foo', your Trac wiki site will be
here: http://overlays.gentoo.org/proj/foo/</p>

<p>To checkout your Subversion repository, use:</p>
<pre caption="Checkout your project overlay">svn co https://overlays.gentoo.org/svn/proj/foo/</pre>

<p>While you may perform read-only actions via unsecure HTTP, you must perform
all commits via HTTPS. If you need to switch between modes, use:</p>
<pre caption="Switching your project overlay from HTTP to HTTPS">svn sw --relocate http://overlays.gentoo.org/svn/proj/foo/ https://overlays.gentoo.org/svn/proj/foo/</pre>

<p>We maintain <uri link="http://overlays.gentoo.org/proj/">a full list of project overlays hosted on overlays.gentoo.org</uri>.</p>
</body>
</section>

<section>
<title>Accessing Developer Overlays</title>
<body>
<p>If your Gentoo email address is 'foo@gentoo.org', your Trac wiki site will be
here: http://overlays.gentoo.org/dev/foo/</p>

<p>To checkout your Subversion repository, use:</p>
<pre caption="Checkout your developer overlay">svn co https://overlays.gentoo.org/svn/dev/foo/</pre>

<p>While you may perform read-only actions via unsecure HTTP, you must perform
all commits via HTTPS. If you need to switch between modes, use:</p>
<pre caption="Switching your developer overlay from HTTP to HTTPS">svn sw --relocate http://overlays.gentoo.org/svn/dev/foo/ https://overlays.gentoo.org/svn/dev/foo/</pre>

<p>We maintain <uri link="http://overlays.gentoo.org/dev/">a full list of developer overlays hosted on overlays.gentoo.org</uri>.</p>
</body>
</section>

<section>
<title>Getting Started With Trac</title>
<body>
<p>Your overlay comes with <uri link="http://trac.edgewall.com">Trac</uri>.  Trac is a wiki, a subversion repository browser, and a bug tracking system that's
very popular with open source developers.</p>

<p>We have disabled the bug tracking system in Trac.  Use <uri link="http://bugs.gentoo.org">Gentoo's bugzilla</uri> for bug tracking your overlay.</p>

<p>Your overlay's RSS feed - the one that is shown on <uri link="http://overlays.gentoo.org">the o.g.o homepage</uri> - comes from Trac's Timeline page or GitWeb's summary.</p>

</body>
</section>

<section>
<title>Getting Started With Subversion</title>
<body>
<p>The advantages of Subversion over CVS include real versioning of directories, full changeset support, and it's much easier to do branching if you need to.  The main disadvantage of Subversion is that it is slower than CVS, and that a local Subversion checkout requires more disk space.</p>

<p>If you have never used Subversion before, the online book is an excellent way to learn Subversion.  You can buy it in dead-tree format too if you prefer.</p>

<p>Here are some basic commands to get you started.</p>
<pre caption="Checking out your overlay">svn co https://overlays.gentoo.org/proj/php</pre>
<pre caption="Seeing which files need committing">svn status</pre>
<pre caption="Adding files to your overlay">svn add my.ebuild</pre>
<pre caption="Committing your changes">svn commit -m 'My changelog entry'</pre>

</body>
</section>

<section>
<title>Layman</title>
<body>
<p>We are telling users to use layman to download and manage your overlay.
layman is a utility written by <mail link="wrobel@gentoo.org">Gunnar Wrobel</mail> which makes it very easy for users to work with overlays.</p>

<p>
To get started with using layman, see <uri
link="http://layman.sourceforge.net/">the homepage documentation</uri>, the <uri
link="http://www.gentoo.org/news/en/gwn/20060522-newsletter.xml">Gentoo Weekly
News for 22nd May</uri> or read <c>man layman</c>.
</p>
</body>
</section>

<section>
<title>Auto-Sync From Portage</title>
<body>
<p>Your packages in the Portage tree are always at risk of being changed without
you knowing about it in advance.  Arch teams need to be able to keyword
packages (and fix arch-specific brokenness), the QA team fix perceived
standards violations, and occaisionally developers will edit packages that
they shouldn't.</p>

<p>You need to make sure that the changes made in Portage aren't lost the next
time you copy your packages from your overlay back into Portage.</p>

<p>The PHP team have solved this problem by automatically copying their packages
from Portage back into a 'portage' branch of their overlay every night.  They
can then use Subversion (or Trac's timeline) to see what has changed every day.</p>

</body>
</section>
</chapter>

<chapter>
<title>Using git on overlays</title>

<section>
<title>Initializing your overlay</title>
<body>
<p>Before uploading you need to locally create a git repository and add all items:</p>
<pre caption="go to your overlay">cd ~/my-overlay</pre>
<pre caption="create a new git repo">git init
git add .
git commit -m "populate overlay"</pre>
<p>Note that this commit was only locally, now we get the server into the game.</p>
<pre caption="tell git the url">git remote add origin git+ssh://git@git.overlays.gentoo.org/(proj or dev)/(name)</pre>
<pre caption="finally get it up there">git push origin master</pre>
<p>Source: http://www.kernel.org/pub/software/scm/git/docs/tutorial.html</p>
</body></section>

<section>
<title>Checking out the overlay with git</title>
<body>
<pre caption="clone it!">git clone git://git@git.overlays.gentoo.org/(proj or dev)/(name)/</pre>
</body>
</section>
</chapter>

<chapter>
<title>Giving Access To Your Overlay To Others</title>

<section>
<title>Introduction</title>
<body>
<p>One of the key features of o.g.o is that people who do not have write access
to the Gentoo Portage package tree can have write access to one or more
overlays.  Several Gentoo projects have found that this is a great way to train
and evaluate potential Gentoo developers in a safe environment.</p>
</body>
</section>

<section>
<title>Project Overlays: Giving Write Access To Team Members</title>
<body>
<p>Any developer listed on a team's project page on www.g.o can have write 
access to the team's overlay.  The project lead can ask on their behalf, or
the developer can come and ask for access themselves.</p>

<p>If the developer doesn't have an o.g.o account yet, he/she will need to drop
by #gentoo-overlays so that we can create an account for them.</p>
</body>
</section>

<section>
<title>Project Overlays: Giving Write Access To Other Gentoo
Developers</title>
<body>

<p>Any Gentoo developer *not* listed on a team's project page on www.g.o can
have write access to the team's overlay.  The request for write access must
come from a member of the team.  It doesn't have to come from the project
lead.</p>

<p>If the developer doesn't have an o.g.o account yet, he/she will need to drop
by #gentoo-overlays so that we can create an account for them.</p>
</body>
</section>

<section>
<title>Project Overlays: Giving Write Access To Gentoo Users</title>
<body>
<p>Any Gentoo user can have write access to the team's overlay.  The request for
write access must come from one of the project's leads.  You can request that
we give the user write access to Trac, to Subversion, or to both.  (We will
assume that the request is for write access to both, unless you say
otherwise).</p>

<p>We cannot accept these requests from anyone other than a project lead.  If
your project only has the one lead, we recommend electing a second lead.  If
your one and only lead is AWOL, consider electing a replacement :)</p>

<p>If the user doesn't have an o.g.o account yet, he/she will need to drop by
#gentoo-overlays so that we can create an account for them.</p>
</body>
</section>

<section>
<title>Developer Overlays: Giving Write Access To Gentoo Developers</title>
<body>
<p>Any Gentoo developer can have write access to your developer overlay.  The
developer can ask us directly; we will not give access until we've checked
with you.  You can also ask us to give write access to any named
developer.</p>

<p>If the developer doesn't have an o.g.o account yet, he/she will need to drop
by #gentoo-overlays so that we can create an account for them.</p>
</body>
</section>

<section>
<title>Developer Overlays: Giving Write Access To Gentoo Users</title>
<body>
<p>Any Gentoo user can have write access to your developer's overlay.  The
request for write access must come from you.  You can request that we give
the user write access just to Trac, just to Subversion, or to both.  (We will
assume that the request is for write access to both unless you say
otherwise).</p>

<p>We cannot accept these requests from anyone else other than you.  If you find
yourself giving access to a lot of other people, it might be that you should
consider setting up a new project, and transfering your work there
instead.</p>

<p>If the user doesn't have an o.g.o account yet, he/she will need to drop by
#gentoo-overlays so that we can create an account for them.</p>
</body>
</section>
</chapter>

<chapter>
<title>Accessing Someone Else's Overlay</title>

<section>
<title>Using An Overlay</title>
<body>
<p>Everyone has full read access to every overlay.  We recommend that you
use</p>
<pre caption="Install layman">
  emerge layman
  echo 'source /usr/portage/local/layman/layman.conf' &gt;&gt; /etc/make.conf
</pre>

<note>
Layman will create "/usr/portage/local/layman/make.conf" once you add
your first overlay. But if you do not plan to install an overlay
immediately you should ensure that this file actually exists and
contains the empty variable "PORTDIR_OVERLAY". Otherwise portage will
complain. You can run "echo PORTDIR_OVERLAY=\"\" >
/usr/portage/local/layman/make.conf" in order to have the file created
correctly.
</note>

<p>After that, to view the list of overlays, use</p>

<pre caption="List overlays that layman knows about">layman -L</pre>

<p>To install an overlay, use</p>
<pre caption="Install an overlay">layman -a &lt;overlay-name&gt;</pre>

<p>You can now install packages from the overlay.</p>
</body>
</section>

<section>
<title>Requesting Write Access</title>
<body>
<p>If you want write access to a project overlay, contact a member of the
project team, and ask them for access.  If they approve your request, they
will arrange for you to have write access, by contacting the Overlays
team.</p>

<p>If you want write access to a developer's overlay, contact the developer
directly, and ask them for access.  If they approve your request, they will
arrange for you to have write access, by contacting the Overlays team.</p>
</body>
</section>
</chapter>

<chapter>
<title>Frequently Asked Questions</title>

<section>
<title>o.g.o Administration</title>
<body>

<p>Q: How do I contact the o.g.o admin staff?</p>
<ul>
<li>A: You can pop into #gentoo-overlays on IRC, and talk to us there.  The
   current staff are mostly in European timezones.</li>
<li>A: You can send an email to overlays@gentoo.org.  Someone will answer you as
   soon as possible.</li>
</ul>

<p>Q: Why can't I edit the access control list directly?</p>
<ul>
<li>A: (SVN) The access control list currently lives in htpasswd format.  Only o.g.o admin staff members have ssh access to the o.g.o box.</li>
<li>A: (Git) The access control list currently lives in the gitosis-admin repository, which is writable only by the o.g.o staff.</li>
</ul>
</body>
</section>

<section>
<title>Security</title>
<body>
<p>Q: Is my overlay available via https?</p>
<ul>
<li>A: Yes, it is.</li>
</ul>
</body>
</section>

<section>
<title>Multiple Overlays</title>
<body>

<p>Q: Can I have multiple overlays?</p>
<ul>
<li>A: Yes, in a fashion.  Inside your overlay, you can create
sub-directories, and put separate package trees inside those sub-directories.
Please take a look at the PHP project overlay for an example.</li>
</ul>
</body>
</section>

<section>
<title>Importing Existing Overlays</title>
<body>
<p>Q: I already have an overlay, and I'd like to move it to o.g.o.  How do I
go about doing that?</p>
<ul>
<li>A: Create a tarball of your subversion repository, and put it somewhere
where it can be downloaded via http.  We'll download it and install it onto
o.g.o for you.</li>
</ul>
<note>Make sure you tar up your repository, and not a checkout!</note>

<p>Q: I have an overlay, but it doesn't use Subversion.  How do I go about
moving it to o.g.o?</p>
<ul>
<li>A: Ask us to create a new, empty overlay for you.  You can then use 'svn
import' to import your files into the new overlay.  You'll lose your history,
but that can't be helped.</li>
<li>A: Search the Internet, and see if there is a tool to convert from your
existing version control software to Subversion.  If there is, use that, and
then we can help you move it to o.g.o.</li>
<li>A: If your version control software is used by Trac, and it can be used
over HTTP, come and help us add
support for your version control software on o.g.o.</li>
</ul>
</body>
</section>

<section>
<title>"Official" Overlays</title>
<body>
<p>Q: When is an overlay considered "official"?</p>
<ul>
<li>A: An "official" overlay is an overlay managed by a Gentoo project (for
project overlays) or a Gentoo developer (for developer overlays).</li>
</ul>

<p>Q: Does an overlay have to be on o.g.o to be "official"?</p>
<ul>
<li>A: No.</li>
</ul>
</body>
</section>
</chapter>
</guide>