Use JSON instead of YAML for Django test fixtures

by Luke Plant

Posted in:

Django

— November 11, 2010 23:27

Today I learned that JSON is much faster than YAML for use in Django fixtures. I sped up the rather slow test suite in an app of mine by nearly a factor of 2 by switching to JSON (the test suite seems to be dominated by time spent parsing fixtures). Here is a handy script I wrote to convert the fixtures:

#!/usr/bin/env python
#
# yaml2json.py

import datetime
import os
import sys
import simplejson
import yaml


class JSONEncoder(simplejson.JSONEncoder):
    """
    JSONEncoder subclass that knows how to encode date/time and decimal types.
    """

    DATE_FORMAT = "%Y-%m-%d"
    TIME_FORMAT = "%H:%M:%S"

    def default(self, o):
        if isinstance(o, datetime.datetime):
            return o.strftime("%s %s" % (self.DATE_FORMAT, self.TIME_FORMAT))
        elif isinstance(o, datetime.date):
            return o.strftime(self.DATE_FORMAT)
        elif isinstance(o, datetime.time):
            return o.strftime(self.TIME_FORMAT)
        elif isinstance(o, decimal.Decimal):
            return str(o)
        else:
            return super(JSONEncoder, self).default(o)


def main(fname):
    assert os.path.splitext(fname)[1] == ".yaml"

    with file(fname) as fp:
        d = yaml.load(fp)

    outname = os.path.splitext(fname)[0] + ".json"
    with open(outname, "wb") as fp:
        fp.write(JSONEncoder(indent='    ').encode(d))

if __name__ == '__main__':
    main(sys.argv[1])

Pass the filename as the first argument, something like this:

for TMP in $(find . -name '*.yaml'); do yaml2json.py $TMP; done

(and you can add git/hg/bzr/svn add and remove commands into that line too).

You will need to update the fixtures attribute in your tests if you specified the '.yaml' extension.

You may also like: §

Comments §