CVS Migration Tracking Page Alec Warner This page was created to track progress regarding the migration of Gentoo's CVS repositories to another versioning control system. This is being done in conjuction with a Google Summer of Code project. 1.0 2006-04-30 CVS Migration
Reasons for Migration
  • CVS does not support branching in a sane manner.
  • CVS commits are not atomic.
  • CVS has a lot of overhead when working with a remote checkout.
Milestones for success Select VCS systems for testing.We have selected Mercurial, Subversion, GIT, and our control, CVS.May 22nd, 2006May 22nd, 2006Other VCS systems may be added provided a repository snapshot is provided by June 19th 2006.Convert gentoo-x86 to each VCS System.Completed git, svn, cvsJune 5th, 2006July 14th, 2006Completed a migration to svn and git, total time ~30 days!Design a set of stress tests and analysis tools for Version Control Systems.Skipped, see notesJune 12th, 2006Completed June 19th, 2006I was hoping to have a design prior to starting, but work commitments forced me to accelerate my project a bit. This phase was moreso me scratching things out on a napkin ;)Implement a set of stress tests and analysis tools for Version Control Systems.DroppedJune 19th, 2006July 14thI basically broke down and used dstat, I got to the point where I had spent about 12 hours on the code for this tool, and then figured why spend more time when a superior tool exists. As such I decided use the better tool instead of writing a replacement.Run the stress tests on each VCS system in order to generate a useful data set.July 26th, 2006August 3rd, 2006Completed Auguest 15thThis was completed around August 3rdAnalyze the data and present this to the Gentoo Community. Attempt to have the community choose a VCS system. In the event that the community takes too long in determining their future VCS system; discuss with Lance and pick a VCS to continue the project with.StartedAug 10th, 2006September 4thThe code was ported to all three systems; isntead of choosing just one.Compose a Migration Plan to migrate to the new VCS System.Started August 21stAugust 30thSept 4thGLEP XX is currently in the submittal process.Update and author developer documentation related to the VCS system. This will include updating any tools that are focussed on VCS systems such as echangelog, repoman, and the cvs->rsync scripts.Start July 14thAug 8th, 2006Pending CompletionRepoman and echangelog been released but need thorough testing. Please see hereSet up test environment and give developers a change to use the new VCS when it is not live. This is also a chance to make sure all tools work properly.Not yet startedSept 1st, 2006Pending CompletionGLEP XX must be approved before this can start.Test in the testing environment for up to one month, ensure sufficient hardware requirements and also ensure that real world data matches data collected during the data mining.Not yet startedOct 1st, 2006Pending CompletionGLEP XX must be approved before this can startSet up the live system and migrate to it.Not yet startedNov 1st, 2006Pending CompletionGLEP XX must be approved before this can start.
Task Status Due Date Date of Completion Notes
Version Control Systems under consideration SubversionAtomic Commits, Merging, Tagging, Branching is a copy operation, Versioned Metadata, Directory Versioning, AnnotationTwice the disk spaceMigration Complete (cvs2svn) (conversion stats)17 minutes, 3 seconds.Server Usage (7.3gb) Client Usage (2.8gb) 21.8mb/s20mb per checkout server stats client stats GITAnnotation; Two, interchangeable, on-disk formats are used: An efficient, packed format that saves space and network bandwidth An unpacked format, optimized for fast writes and incremental work. Merging, tagging, branching, very fine grained control.Being a distributed VCS means it may be difficult for us to use, has high server spec requirements.Migration complete, minus the Authors file.
  • Smart Clone: 84 minutes, 58 seconds
  • Dumb Clone: 121 minutes, 52 seconds (over http)
Note: Smart clone being a checkout over a smart protocol, one that will generate the packs for you; this generally pains the server (lots of ram and cpu usage). However the cloned repo will be all ready for you to use. A Dumb clone is one over http, or rsync, where the server just tranfersfiles and the client does all the work to prep the repository. Packed (1.1gb), Unpacked (1.6gb)1.72mb/s (smart), 1.2mb/s (dumb)
  • Up to 400mb per clone (Smart clone on the server), server process was causing a high server load (~1.0 load per checkout)
  • Up to 60mb per clone (Dumb clone on the server), server process only occasionally caused a high load (spikes to 1.0, but usually around .2)
Smart clone Stats Dumb clone Stats (Server) Dumb clone Stats (Client) CVS (Stats on the current usage)Already converted, status quo, no migration, no training, does what we need 90% of the time.Sucks at branching, merging branches back in.Migration Unnecessary8 minutes, 54 seconds.Server (1.6gb) Checkout(~880Mb)13.18mb/s15mb per checkout server stats client stats
System Pros Cons Migration Full Checkout Space Considerations Bandwidth Usage Memory Usage Others?