Fuzzy testing with assertNumQueries

Posted in: Django

The yet-to-be-released Django 1.3 has an assertNumQueries method which will allows you to simply specify the number of queries you expect.

Sometimes, however, specifying the exact number of queries is overkill, and makes the test too brittle:

  1. I might not actually care if some global change to my app means I get one more request on a whole bunch of non-critical views, for example.
  2. Caching of various queries might mean that the number of queries in a block of code can vary by a few. The correct way to fix this is to track down all the queries that are being cached, and to ensure the cache is populated before you start counting the queries. But this can be tedious in practice.

Sometimes, I simply want to ensure that a change to a template or view function that deals with a set of N objects hasn’t gone from O(1) queries to O(N) queries. I want to set up a test in which N is some reasonably big number, and then test that the number of queries is less than N, and reasonably close to what it currently is.

With an appropriate subclass of int, I can still use assertNumQueries to do this:

class FuzzyInt(int):
    def __new__(cls, lowest, highest):
        obj = super(FuzzyInt, cls).__new__(cls, highest)
        obj.lowest = lowest
        obj.highest = highest
        return obj

    def __eq__(self, other):
        return other >= self.lowest and other <= self.highest

    def __repr__(self):
        return "[%d..%d]" % (self.lowest, self.highest)

class MyFuncTests(TestCase):
    def test_1(self):
        with self.assertNumQueries(FuzzyInt(5,8)):

This will ensure that the number of queries is between 5 and 8 (inclusive). The error message is slightly confused if the test fails, but it still has all the info you need:

AssertionError: 10 != [5..8] : 10 queries executed, 8 expected

This trick also works with the count parameter to assertContains.

Comments §

blog comments powered by Disqus