All Unkept
Posted in: Django, Python, Software development  —  31 July 2008

Managing Django patches using Mercurial

For any slightly longlived Django patch, for instance if I want to work on something in stages before committing, or if it might never be committed to trunk, here is the pattern I started using today:

Setup §

Do this once:

In a subversion checkout:

$ hg init
$ cat > .hgignore
\.svn/
\.hgignore$
\.pyc
~$
^D

(^D == Control-D, EOF)

$ hg ci -Am "Initializing from subversion checkout"
$ hg qinit -c

NB: make sure you do 'hg qinit -c'. Your patches live in .hg/patches, and this command makes that directory into a Mercurial repository, so all your changes to the patches are versioned. You have to remember to do hg qcommit after hg qrefresh.

Usage §

Commit all work to mercurial queues, not to mercurial directly.

New patch:

$ hg qnew name_of_patch

(Hack away, using hg diff etc. to see changes)

Commit changes to the current patch:

$ hg qrefresh && hg qcommit

Pull in new subversion changes:

$ hg qpop -a
$ svn update
$ hg addremove -s 75
$ hg commit -m upstream
$ hg qpush -a

(Of course, you could put that lot in a little script)

Once done, commit to subversion, and delete patches:

$ svn commit
$ hg qdel -r qbase:qtip
$ hg qcommit -m "Pushed patches upstream"

(Your patches are still there, of course, in the mercurial repository in .hg/patches, should you need the history)

Evaluation §

I'm not yet sure if this is the best way to do this (see the other methods on the Mercurial Working with Subversion page), but it is a lot better than Subversion on its own, and it isn't too complicated.

One advantage of the system is that it is very easy to ignore it or use it as required. Most of my Django work I will just hack directly and commit. If I realise part way through that it is going to take I while, or that I'm doing two sets of changes that I want to split up, hg qrecord is just the ticket -- interactively record changes to a new patch. If I then need to work on something else, I can just do hg qnew, doing hg qpop first if necessary/desired.

It is quite nice that the patches are just simple files in .hg/patches -- you can just go and look at them with normal tools etc. If your patch has got messy (i.e. accidental whitespace changes), you can just hg qpop it, fix it up manually, hg qcommit and hg qpush it again.

One of the downsides is that patches are managed as a stack -- if you want to add a new patch, you have to put in somewhere in the series of patches (not necessarily at the 'top' i.e. most recent end, but you do need to think about where to insert it). I imagine things can get tricky if you put it in the wrong order initially, though you could always just delete it and put it somewhere else. As long as you have versioned everything, you can get at it easily.

On the other hand, given that your patches directory is a full hg repository, I imagine you can actually do very powerful things -- you can branch your patches etc. very easily. I've discovered recently that Mercurial branching is actually very powerful -- you don't have to create clones, you can do everything inside one working directory.

Anyway, hope that is useful to someone else.

Comments §

§ On 1 August 2008, Christian Wyglendowski wrote:
335 Thanks for this writeup. It seems like a nice general guide for using mqueue in combination with Subversion - I'm not a Django developer but I still found it informative. Keep up the good work.

§ On 3 August 2008, Nicola Larosa wrote:
337 That's a great workflow for when you have commit rights to a main Subversion repository.

I use hgsvn and then create a clone where I too manage patches via versioned patch queues. I don't like named branches much: hg makes clones very cheap via hardlinking, and that lets you keep distinct working copies.

It's not the case with Django, but some projects reference and use external apps via the svn:externals property. I wrote a script, hgsvnimpull, that clones and keeps up to date those too by wrapping hgsvn. It currently lives in a Pinax issue (cannot find a way to create links here, sorry):

Mercurial script to keep track of Pinax and external dependencies
http://code.google.com/p/django-hotclub/issues/detail?id=32

§ On 5 August 2008, erob wrote:
345 Nice post!

I use MQ too for managing patches with Mercurial. I think this approach is really neat.

Here's a little trick for
importing patches from a patch repo, assuming that you have done "hg qinit -c" to initialize your main Django repository:

$ cd /some/django/repo/
$ ls /path/to/bugfixes/*.patch | xargs hg qimport

The result is a full set of patches available within Mercurial.

Cheers,
Etienne

§ On 12 August 2008, roppert wrote:
349 Just for the record; One need to activate the Mercurial Queues Extension in Mercurial for this to work. I might be obvious to everybody but I thought I'd just mention it anyway.
By adding the following to your .hgrc the MqExtension (http://www.selenic.com/mercurial/wiki/index.cgi/MqExtension) is activated:
 
[extensions]
hgext.mq =

(Btw posting comments was a little too tricky to be convenient.)

§ On 21 May 2009, osimons wrote:
433 A bit late perhaps, but being the interweb no doubt my words will be read anyway...

You don't need to keep all patches in a single stack - like everything else in the .hg/patches directory the 'series' file is plain text and editable.

I manage various distinct patches and patch queues in the series file just by reorganizing the file, and commenting out the patches that are completed or on hold. Using comment lines (#) you can also document and describe what the various patches and stacks do.

Just qpop -a to get a clean repos, comment/uncomment, and qnew or qpush back what you want to work on.

Add comment

Format:

  • Javascript has to be on to get past my spam protection, and cookies, and there is a delay, sorry for any inconvenience!
  • I reserve the right to moderate comments.