Improving ClearCase workflow with Git

Git is the future of version control systems and maybe even more.

I started my acquaintance with SCM tools many years ago with Microsoft Visual SourceSafe (ugh!), then moved to Perforce mainly because the company I was working for (Symbian, in case anyone is curious) was using it. For my personal projects I switched to CVS, then moved to Subversion and continued to use it till earlier this year, when a friend mentioned a new system called Git (written by a self-confessed git.)

After reading about Git and watching the Google tech talk presentation by the man himself I decided to try Git out for my projects, and was hooked. It is so much better than anything else out there, at least for the time being. I’m not emotionally attached to a particular SCM system—if anything better becomes available, I’ll switch to make my life easier (I’m lazy), but at this time Git, IMHO, is the best. It makes my development work much more pleasant and doesn’t get in the way. Its branching and merging capabilities are the cream of all tools I’ve used, and it is faster than any other SCM.

Unfortunately, at the current project I have to use ClearCase. As someone said, it’s an expensive CVS. I won’t be talking about why I don’t like ClearCase here; if you are still reading, it means CC hasn’t found a devoted enthusiast in you either. Don’t despair: there is hope.

Matt Keller wrote an excellent article on how to combine ClearCase with Git and make one’s life bearable again. I’m going to present my own version of this workflow, slightly modified and enhanced for my own circumstances, namely working in Windows XP.

The advantages of this method for me are:

• no need to keep files checked out exclusively in ClearCase while I work on a new feature or fix a bug;
• no need to work in private ClearCase branch and merge using ClearCase tools;
• keeping my modifications (patches) as small as possible and always based on the latest code;
• ability to quickly branch code to test a new idea without anyone noticing it;
• ability to work disconnected (if I were to use a laptop, that would be a killer feature);
• by keeping my changes grouped and submitting them to ClearCase together I can emulate changelist feature absent in ClearCase (without this feature one can often expect broken builds due to partial commits).

OK, enough marketing talk. Here are the steps.

Installing Cygwin

First of all, you need to install Cygwin. I use Cygwin Git which is the official Windows port. You will also need other Unix tools to assist you, and the easiest way to get them is installing Cygwin. When setting up Cygwin, be sure to install git and rsync packages.

Copying ClearCase view to working directory

Let’s assume you have a ClearCase snapshot view in c:ccproject. You will not be working directly in this directory. Instead create a new directory c:gitproject which will be your working directory, and initialise a new Git repository there:

$ mkdir -p /cygdrive/c/git/project/

$ cd /cygdrive/c/git/project/

$ git init

To copy the code from ClearCase view to your working directory (which becomes your master branch in Git), use rsync. Do a dry run first using -n command line option and redirect output to a file to see if there are files or directories that should not be copied. For example, you don’t need to copy any of the ClearCase .vws and view* files. Create an .rsyncignore file in the Git working directory first, to keep excluded file globs in it (add name of the file itself to the contents, as otherwise it’ll be deleted the next time you sync):

$ cat .rsyncignore <

.rsyncignore

<Ctrl+D>

$ rsync -rvtn --progress --delete --exclude-from=.rsyncignore /cygdrive/c/cc/project/ /cygdrive/c/git/project/ > rsync.log

Study the file rsync.log, re-run the rsync command as many times as you need, adding more and more exclusion globs to .rsyncignore, until you are sure you will have only the files you need in the working directory after running rsync. When you are ready to actually copy the files, enter:

$ rsync -rvt --progress --delete --exclude-from=.rsyncignore /cygdrive/c/cc/project/ /cygdrive/c/git/project/

Creating a working branch

Your Git working directory now has the exact copy of your ClearCase snapshot view. This will be your master branch for rebasing your feature work. Add everything to the repository, then create a feature branch (you can name it anything you like, of course):

$ git add .

$ git commit -m 'Initial commit'

$ git checkout -b feature

Developing a feature

Now that you are on your feature branch in your Git directory, you can start working on your feature, bug fix or whatever. Commit your changes to the feature branch and keep working undisturbed. As Matt Keller explained, it is easy to keep your work based on the latest code in ClearCase. To achieve this, you need to:

• switch to the master branch
• update your ClearCase view using ClearCase Explorer
• mirror the latest upstream changes in your Git master branch using rsync
• switch back to Git feature branch
• rebase the feature branch on the master branch, resolving any conflicts

The commands are (you are in c:gitproject directory)

$ git checkout master

$ rsync -rvt --progress --delete --exclude-from=.rsyncignore /cygdrive/c/cc/project/ /cygdrive/c/git/project/

$ git add .

$ git commit -m 'Update from ClearCase'

$ git checkout feature

$ git rebase master

After that, keep working on your feature, repeating the steps above whenever the upstream code in ClearCase gets updated.

Packing your changes for review

If you need to get your code reviewed and want to produce a set of changed files and a diff file, you can do the following (you are on your feature branch):

$ git diff -b master > feature.diff

$ git diff master --name-only | grep FeatureDir | zip FeatureChanges.zip -@

This produces a ZIP archive containing all changed files in the FeatureDir directory and a set of unified diffs. You can send them to your reviewer. (Yay for the power of Unix command line!)

The git-diff option -b, or --ignore-space-change, ignores changes in the amount of whitespace and can be handy when working in Cygwin.

Getting changes to ClearCase

As per above, you have a set of your diffs in feature.diff file. You need to check out the affected files in ClearCase. Do it manually? Nah.

To produce a set of file names of the changed files with double backslashes as path separators (so that xargs and cleartool can process them), run the following command in Cygwin when in your Git directory on the feature branch:

$ git diff master --name-only | grep FeatureDir | sed 's!/!\\!g' > FeatureFiles.lst

Then move both the file list and the diffs to the ClearCase view directory and switch to that directory:

$ mv FeatureFiles.lst feature.diff /cygdrive/c/cc/project/

$ cd /cygdrive/c/cc/project/

Now check out multiple files with an optional comment (or use -nc instead of -c for no comment):

$ cat FeatureFiles.lst | xargs cleartool co -c 'Some comment'

This magically checks out all files you need to patch! Apply the patch, fixing line endings so that Cygwin’s patch tool works:

$ cat feature.diff | tr -d "\015" | patch -p1

The -p1 option of the patch tool strips one level of directories from the file names in the diff file so that they point to the same files in both Git and ClearCase directories.

After patching run ClearCase Explorer, find checkouts and check them in. You’re done!

I’m by no means an expert in Unix, Git or ClearCase. I would like to hear any suggestions on how to improve this workflow. As it is now the process had helped me enormously and made development much more entertaining. I hope it helps you, too.