Luke Plant's home page

When perfection is table stakes

2026-03-16T15:37:13Z

For a rewrite or a replacement of existing software, very often you will find that the minimum you must achieve is something that works at least as good as the previous system in every respect, for all your users all of the time. Perfection is table stakes.

This is quite annoying if you happen to have the job of doing the rewrite, and not understanding this is a source of agro.

Here are a couple of examples from the Open Source world:

Pulseaudio is a “sound server” for Linux. A sound server is the part of your audio system that manages the fact that multiple applications might all want to be playing at once, and the multiple sound input/output devices you might be using.

Before Pulseaudio there was a different system, and the first time my system was “upgraded” to Pulseaudio, it had poor sound quality for some apps. I’m not an audiophile, but the cracks and pops and hissing were very obvious to me.

Now, compared to the previous system, Pulseaudio also came with a bunch of new features, like the ability to have a per-application volume control, and do sound over Bluetooth more easily. But I didn’t care about those features I had been given (I didn’t need them), while I certainly cared about music sounding bad. Perfect sound quality, all the time, is the minimum you have to achieve as a sound server.

I also found that if I uninstalled Pulseaudio, then I got back to the previous state where I had ALSA which just worked, and everything sounded fine. So I was pretty annoyed at this “upgrade”.

That was some years ago, and these days Pulseaudio has now apparently been replaced by Pipewire in my current Linux system. This happened without me knowing it, and in fact this is one of the highest compliments you can give to a rewrite or replacement system: the users didn’t notice.
More ambitiously in Linux world, there is Wayland, a project started in 2008 which is attempting to replace the X Window System as the protocol for how to do a windowing system, graphics output and keyboard/mouse input (so, almost everything you consciously interact with on a computer). Unsurprisingly, this is a lot of work, and there are still lots of things that don’t work for some people, and lots of problems that people didn’t have before.

Such reports are going to be pretty frustrating for the creators of the new system, whose only reward for decades of work, usually done for free, is a long list of complaints.

But for anyone who is forced to switch, if they find any regression in anything at any level, they are going to say “this sucks”. And fair enough - why should they care about all the work you did when the end result for them is worse? Why should they care about all the new features, which they don’t yet need (or don’t know they need), when some of the existing features they definitely do need are now broken?

So, what is the lesson? Don’t bother?

Rather, I think it is this: go ahead and do the rewrite, just buckle up, and set your expectations properly.

Until it has reached 100% of what it is supposed to replace, you will mostly get complaints. Don’t expect to be thanked for all your work, whether it is paid and commercial or free Open Source development. Nobody cares about the fantastic architectural improvements (which mainly help you, not them), nor about the new features they aren’t using yet. Don’t complain about the complainers and the naysayers, or ask them to shut up, or pretend that their problems don’t exist - why shouldn’t they complain, when you made their life worse?

(Do the complainers actually have a right to complain? As a side note, when talking about this with my wife, her view was that they really don’t, and maybe they should think about paying for this free software before complaining. This is coming from the perspective of “a wife of an Open Source project maintainer who spends a lot of hours on projects for which he is not thanked (or paid)”. But however much truth there is in that perspective, people will complain, and I don’t think it is sensible to assume otherwise).

I’m writing this down partly for myself, as I start a project rewriting a working piece of software for a client. This rewrite will include a large interface overhaul as well as a lot of internal architecture changes, and I’m hoping that the interface changes will be welcome.

However, I have to prepare for the fact that initial reactions will likely be negative, or not nearly as positive as we might expect, because just by changing anything I’ve already made someone’s life worse: having to relearn how to do something they used to know is not most people’s idea of fun. To make up for that, if I want them to be positively enthusiastic, not only do I have to cater for every need and fulfil every previous expectation, the improvements have also got to be both large and obvious — it’s got to be an absolute slam dunk, from the very first time they try it.

We’ve got a tiny number of current users for this application, so compared to the other projects I mentioned, the stakes are much lower and we should be able to accommodate everyone’s needs. But getting my expectations right, having empathy with the user, is essential.

Announcing fluent-codegen

2026-03-10T12:33:49Z

This post announces fluent-codegen, a Python library for generating Python code. Not everyone needs this, but when you need it, you’ll probably know, and you’ll want probably want a decent solution that is not based on string concatenation.

The history of this project is that it started out as a codegen module in fluent-compiler, which is an implementation of Project Fluent, Mozilla’s internationalisation solution. So the word “fluent” referred to that originally, but now it refers to a set of nice APIs for building up Python expressions.

As I needed a code generation library for another purpose, I pulled out this library and fleshed it out into a relatively complete, standalone project. It has also evolved quite a bit since its earlier form, and is a pretty well rounded library now. You can check out the nice docs, which include a nice little toy example of a “SVG to Python Turtle” compiler. Enjoy!

Help my website is too small

2025-12-19T13:45:33Z

A jobs web site I belong to just emailed me, telling me that some of the links in my public profile on their site are “broken” and “thus have been removed”.

The evidence that these sites are broken? They are too small:

https://www.djangoproject.com/: response body too small (6220 bytes)

https://www.cciw.co.uk/: response body too small (3033 bytes)

The first is the home page of the Django web framework, and is, unsurprisingly, implemented using Django (see the djangoproject.com source code). The second is one of my own projects, and also implemented using Django (source also available for anyone who cares).

Checking in webdev tools on these sites gives very similar numbers to the above for the over-the-wire size of the initial HTML (though I get slightly higher figures), so this wasn’t a blip caused by downtime, as far as I can see.

Apparently, if your HTML is less than 7k, that obviously can’t be a real website, let alone something as ridiculously small as 3k. Even with compression turned up all the way, it’s clearly impossible to return more than an error message with less than at least 4k, right?

So please can Django get it sorted and add some bloat to their home page, and to their framework, and can someone also send me tips on bloating my own sites, so that my profile links can be counted as real websites? Thanks!

Breaking “provably correct” Leftpad

2025-10-03T21:00:00+01:00

I don’t know much about about formal methods, so a while back I read Hillel Wayne’s post Let’s prove Leftpad with interest. However:

I know Donald Knuth’s famous quote: “Beware of bugs in the above code; I have only proved it correct, not tried it”
I also know how it turned out that code that had been proved correct harboured a bug not found for decades.

So I thought I’d take a peek and do some testing on these Leftpad implementations that are all “provably correct”.

Methodology

I’ll pick a few, simple, perfectly ordinary inputs at random, and work out what I think the output should be. This is a pretty trivial problem so I’m expecting that all the implementations will match my output. [narrator: He is is expecting no such thing]

I’m also expecting that, even if for some reason I’ve made a mistake, all the implementations will at least match each other. [narrator: More lies] They’ve all been proved correct, right?

Here are my inputs and expected outputs. I’m going to pad to a length of 10, and use - as padding so it can be seen and counted more easily than spaces.

Item	Input	Length	Expected padding	Expected Output
1	`𝄞`	1	9	`---------𝄞`
2	`Å`	1	9	`---------Å`
3	`𓀾`	1	9	`---------𓀾`
4	`אֳֽ֑`	1	9	`---------אֳֽ֑`
5	`résumé`	6	4	`----résumé`
6	`résumé`	6	4	`----résumé`

[“ordinary”, “random” - I think my work here is done…]

I’ve used a monospace font so that the right hand side of the outputs all line up as you’d expect.

Entry 6 is not a mistake, by the way, it just does “e acute” in a different way to entry 5. Nothing to see here, move along…

Implementations

Not all of the implementations were that easy to run. In fact, many of them didn’t take any kind of “string” as input, but vectors or lists or such things, and it wasn’t obvious to me how to pass strings in. So I discounted them.

For the ones I could run, I attempted to do so by embedding the test inputs directly in the program, if possible.

Liquid Haskell

Embedding the characters directly in Haskell source code kept getting me “lexical error in string/character literal”, so I wrote a small driver program that read from a file.

Java

The leftpad function provided didn’t take a string, but a char[]. Thankfully, it’s easy to convert from String objects, using the .toCharArray() function. So I did that.

Lean4

There is a handy online playground, and the implementation had a helpful #eval block that I could modify to get output. You can play with it here.

Rust

The code here had loads of extra lines regarding specs etc. which I stripped so I could easily run it, which worked fine.

More tricky was that the code didn’t take a string, but some Vec<Thingy>. As I know nothing about Rust, I got ChatGPT to tell me how to convert from a string to that. It gave me two options, I picked the one that looked simpler and less <<angry>>. I didn’t deliberately pick the one which made Rust look even worse than all the others, out of peevish resentment for every time someone has rewritten some Python code (my go-to language) in Rust and made it a million times faster – that’s a ridiculous suggestion.

Some competition!

To make things interesting, let’s compare these provably correct implementations with one vibe-coded by ChatGPT, in some random language, like, say, um, Swift. It gave me this code:

import Foundation

func leftPad(_ string: String, length: Int, pad: Character = " ") -> String {
    let paddingCount = max(0, length - string.count)
    return String(repeating: pad, count: paddingCount) + string
}

You can play with it online here.

Results

Here are the results, green for correct and red for … less correct.

Input	Reference	Java	Haskell	Lean	Rust	Swift
𝄞	---------𝄞	--------𝄞	---------𝄞	---------𝄞	------𝄞	---------𝄞
Å	---------Å	--------Å	--------Å	--------Å	-------Å	---------Å
𓀾	---------𓀾	--------𓀾	---------𓀾	---------𓀾	------𓀾	---------𓀾
אֳֽ֑	---------אֳֽ֑	------אֳֽ֑	------אֳֽ֑	------אֳֽ֑	--אֳֽ֑	---------אֳֽ֑
résumé	----résumé	----résumé	----résumé	----résumé	--résumé	----résumé
résumé	----résumé	--résumé	--résumé	--résumé	résumé	----résumé

And pivoted the other way around so you can compare individual inputs more easily:

Language	𝄞	Å	𓀾	אֳֽ֑	résumé	résumé
Reference	---------𝄞	---------Å	---------𓀾	---------אֳֽ֑	----résumé	----résumé
Java	--------𝄞	--------Å	--------𓀾	------אֳֽ֑	----résumé	--résumé
Haskell	---------𝄞	--------Å	---------𓀾	------אֳֽ֑	----résumé	--résumé
Lean	---------𝄞	--------Å	---------𓀾	------אֳֽ֑	----résumé	--résumé
Rust	------𝄞	-------Å	------𓀾	--אֳֽ֑	--résumé	résumé
Swift	---------𝄞	---------Å	---------𓀾	---------אֳֽ֑	----résumé	----résumé

Comments

Rust, as expected, gets nul points. What can I say?

Vibe-coding with Swift: 💯

Other that, we can see:

The only item that all implementations (apart from Rust) get correct is entry 5, the first of the two résumé options.
Java is mostly consistent with the others, but it appears it doesn’t like musical notation, or Egyptian hieroglyphics (item 3 is “standing mummy”), which seems a little rude.

The score so far:

Fancy-pants languages and formal verification: 0
Vibe-coding it with ChatGPT: 1

Explanation

OK, I’ve had my fun now :-)

(The original “Let’s Prove Leftpad” project was done “because it is funny”, and this post is in the same spirit. I want to be especially clear that I’m not actually a fan of vibe-coding).

What’s actually going on here? There are two main issues, both tied up with the concept of “the length of a string”.

(If you already know enough about Unicode, or don’t care about the details, you can skip to the “What went wrong?” section to continue discussion regarding formal verification).

First:

What is a character?

Strings are composed of “characters”, but what are they?

Most modern computer languages, and all the ones I included above, use Unicode as the basis for answering this. Unicode, at its heart, is a list of “code points” (although it is bit more than this). A code point, however, is not exactly a character.

Many of the code points you use most often, like Latin Capital Letter A U+0041, are exactly what you think of as a character. But many are not. These include:

combining characters, which are used to add accents and marks to other characters. These have some complexity:
- First, whether you regard the accent as part of the character or not can be a matter of debate. For example, in é, “e acute”, you might think of this as a different letter to e, or an e plus an acute accent. In some languages, this distinction is critical e.g. in Turkish ç is not just “c with some decoration”, it’s an entirely distinct letter.
- Second, Unicode often has multiple ways of generating these accented characters, either “pre-composed” as a single code point, or as a combination of multiple code points. So, there is both:
  - é: Latin Small Letter E With Acute U+00E9
  - é: Latin Small Letter E U+0065 + Combining Acute Accent U+0301
various spacing characters
control characters such as bidi isolation characters.
and probably more

So, Unicode has another concept, called the “extended grapheme cluster”, or “user-perceived character”, which more closely maps to what you might think of as a “character”. That’s the concept I’m implicitly using for my claim of what leftpad should output.

Secondly, there is the question of:

How does a programming language handle strings?

Different languages have different fundamental answers to the question of “what is a string?”, different internal representations of them (how the data is actually stored), and different ways of exposing strings to the programmer.

Some languages, especially performance oriented ones, provide little to zero insulation from the internal representation, while others provide a fairly strong abstraction. Some languages, like Haskell, provide multiple string types (which can be used with string literals in your code with the OverloadedStrings extension).

At this point, as well as “code points”, we’ve got to consider “encodings”. If you have a code point, even a “simple” one like U+0041 (A), you have said nothing about how you are going to store or transmit that data. An encoding is a system for doing that. Two relevant ones here:

UTF-8 is probably the most common/popular one. It uses anything from 1 to 4 bytes to express code points. It has lots of useful properties, but an important one is backwards compatibility with ASCII - if you happen to be limited to the 127 characters found in ASCII, then UTF-8 encoded Unicode text is one-byte-per-codepoint, and is byte-for-byte the the same as ASCII.
UTF-16 is an encoding where most code points (specifically those in the Basic Multilingual Plane or BMP) take up 2 bytes, and the remainder can be specified using 4 bytes and a system called “surrogate pairs”.

UTF-16 exists because originally Unicode had less than 65,536 code points, meaning you could represent all points with just two bytes, and it was thought we would never need more.

In terms of languages today, with some simplification we can say the following:

In Haskell, Lean, Python, and many other languages, strings are sequences of Unicode code points.

There is little attempt to hide the idea of code points from you, although in some, like Python, the internal representation itself might be hidden (see PEP 393).
In Java and Javascript, strings are sequences of UTF-16 items.

Originally strings were intended to be sequences of code points, but when Unicode grew, to avoid breaking all existing Java code, it morphed into the UTF-16 compromise.
In Rust, strings are UTF-8 encoded Unicode code points (see String).

Rust does also have an easily accessible chars method/concept, which corresponds to a Unicode code point. I didn’t use this above - Rust would have behaved the same as Haskell/Lean if I had.
In Swift, strings are sequences of user-perceived characters.

I’m not a user of Swift, but from the docs it appears to do a pretty good job of abstracting away from the “sequence of code point” paradigm. For example, iterating over a string gets you the “characters” (i.e. extended grapheme clusters) one by one, and the .count property gives you the number of such characters. It does also have a .length property that gives you the number of code points.

Putting them together

These differences, between them, explain the differences in output above. In more detail:

the treble clef 𝄞 is a single code point, but it is outside the BMP and requires 2 UTF-16 items. So Java considers the length to be 2, and only 8 padding characters were added, in contrast to other languages.
I created the Å using two code points (although there is a pre-composed version of this character).
The אֳֽ֑ is Hebrew, and is composed an Aleph followed by multiple vowel marks and a cantillation mark, bringing it up to 4 code points.
résumé was spelled in two different ways, one with precomposed é which is one code point, the other with combining characters where each é requires two code points.
None of the inputs was wholly in the ASCII range, so encoding them as UTF-8 requires more bytes, which is why Rust (as I used it) behaved as it did.

What went wrong?

For me, the biggest issue is not the “code points” vs “characters” debate, which is responsible for most of the variation shown, but the issue that resulted in the difference in the Java output i.e. UTF-16. All of the others (if I hadn’t stitched up Rust) would have resulted in the same output at least.

Apparently, nothing in the process of doing the formal verification forced the implementations to converge, and I think it is pretty fair to conclude that that at least one of the implementations must be faulty, given that they produce different output.

So what went wrong?

Lies, damned lies and natural language

English (or any natural language) is at the heart of the problem here. We should start with the phrase “provably correct”. It has a technical definition, but I’m not convinced those English words help us. The post accompanying the LiquidHaskell entry for Leftpad puts it this way:

My eyes roll whenever I read the phrase “proved X (a function, a program) correct”.

There is no such thing as “correct”.

There are only “specifications” or “properties”, and proofs that ensures that your code matches those specifications or properties.

For these reasons, I think I’d prefer talking about “formally verified” functions – it at least prompts you to ask “what does that mean”, and maybe suggests that you should be thinking about what, specifically, has been verified.

The next bit of English to trip us up is “the length of a string”. It’s extremely easy to imagine this is an easy concept, but in the presence of Unicode it really isn’t.

Hillel’s original informal requirements don’t actually use that phrase, instead they use the pseudo-code max(n, len(str)). Looking at the implementations, it appears people have subconsciously interpreted this as “the length of the string”, and then assumed that the functions like length or size that their language provides do “the right thing”.

We could conclude this is in fact a problem with informal requirements – it was at the level of interpreting those requirements that this went wrong. Therefore, we need more formal specifications and verification, not less. But I don’t think this gets round the fact that we’ve got to translate at some point, and at that point you’ve got trouble.

What is correct?

The issue I haven’t addressed so far is whether my reference output and the Swift implementation are actually “correct”. The reality is that you can make arguments for different things.

Implicitly, I’m arguing that left pad should be used for visual alignment in a fixed width context, and the implementation that does the best at that is the best one. I think that is a pretty reasonable case. But I’m sure you could make a case for other output – there isn’t actually anything that says what left pad should be used for. It’s possible that there are use cases where “the language’s underlying concept of the length of a string, whatever that may be” is the most important thing.

In addition, I was hiding the fact that “fixed width” is yet another lie:

I was originally going to use a flag character like 🏴󠁧󠁢󠁥󠁮󠁧󠁿 as one of my inputs, which is a single “extended grapheme cluster” that uses no less then 7 code points. It also results in 14 UTF-16 units in Java. The problem was that this character, like most emojis and many other characters from wide scripts like Chinese, takes up a double width even with a monospace font.

To maintain the subterfuge of “look how these all line up neatly in the correct output”, I was forced to use other examples. In other words, the example use case I was relying on to prove that these leftpad implementations were broken, is itself a broken concept. But I would still maintain that my reference output is closer to what you would expect leftpad to do.

A big point is this: even if we argue that a give implementation is “correct” (in that it does what its specifications say it does), that doesn’t mean you are using it correctly. Are you using it for its intended purpose and context? That seems like a really hard question to answer even for leftpad, and many other real world functions are similar.

So, I’m not sure what my final conclusion is, other than “programming is hard, let’s go shopping let’s eat chocolate” (alternative suggested by my wife, that’s my plan for the evening then).

Confessions and corrections

The Swift implementation was indeed written by ChatGPT, and it got it right first time, with just the prompt “Implement leftpad in Swift”. However:
- Swift is the only language I know where an implementation that does what I wanted it to do is that simple.
- I followed up by getting ChatGPT to produce a Python version, and it had all the same problems as Haskell/Lean and similar.
- I noticed that Swift doesn’t calculate strings lengths the way I would need for my use case for some characters, such as Zero Width Space U+200B and Right-To-Left Isolate U+2067, which I would need to count for zero length.
As mentioned, the other way to use the Rust version has the same behaviour as the Haskell/Lean/etc versions. ChatGPT did actually point out to me that this way was better, and the other way (the one I used) was adequate only if the input was limited to ASCII.

I do feel slightly justified in my treatment of this solution however - the provided code didn’t give a function that actually took a string, nor did it force you to use it in a specific way. It dodged the hard part, so I punished it.

Response

Hillel was kind enough to look at this post, and had this response to add:

In general, formally verified code can “go wrong” in two ways: the proven properties that don’t match what we need, or the proof depends on assumptions that are not true in practice. This is a good example of the former. An example of the latter would be the assumption that the output string is storable in memory. None of the formally verified functions will correctly render leftpad(“-“, 1e300, “foo”). This is why we always need to be careful when talking about “proving correctness”. In formal methods, “correct” always means “conforms to a certain specification under certain assumptions”, which is very different from the colloquial use of “correct” (does what you want and doesn’t have bugs).

He also pointed out that padding/alignment functionality available in standard libraries, like Python’s Format Specification Mini-Language and Javascript’s padStart, have similar issues.

Devotions on 1 Thessalonians 1, August 2025

2025-09-13T15:03:01+01:00

A few weeks ago on CCiW Purple Camp, I led devotions one morning, speaking from 1 Thessalonians 1. A few days ago I was asked if I had a written form, and since I did prepare a full script, here it is.

I must mention, however, that 100% of the inspiration and insight from these devotions came from an excellent paper by Wayne Grudem, Pleasing God by Our Obedience. This paper has been hugely helpful and influential for me over about the past 10 years, and I highly recommend it. Also, The Hole in Our Holiness by Kevin DeYoung has several chapters that enlarge very helpfully on these subjects, and it is again very highly recommended.

We’re going to look briefly at 1 Thessalonians chapter 1, especially v2-3, and some verses from chapter 4.

2 We always thank God for all of you and continually mention you in our prayers. 3 We remember before our God and Father your work produced by faith, your labour prompted by love, and your endurance inspired by hope in our Lord Jesus Christ.

Now I cannot claim that I am as faithful in my praying for all of you as the apostle Paul was in his prayers for the Thessalonians. However, when I think of you – some of whom I know better than others – when I think of the people here I do feel the same way as Paul did. I think of people who love God and have been faithfully following him and serving him, and it brings me great joy.

The apostle Paul had confidence that these people were real believers because of the way that they accepted the message, and then changed in their behaviour - v4-10. The Holy Spirit had so obviously been active among them, and continued to be active – especially in producing faith, love and hope, which were very visible in the lives of the Thessalonians.

When I look around at the people here, I see the same things:

I see work produced by faith - God has promised to build his kingdom through the proclamation of his word, and you believe that, and that’s a huge part of why you bother with all this. I see labour prompted by love – I see people who make sacrifices in order to come on camp, giving up time and holiday and money, and I know that many of you do much more throughout the year. You do it because you love Jesus and his people and love the world who need to hear the message. And I see endurance inspired by hope in our Lord Jesus – many of you have been doing this for many years.

When we see the same graces that Paul saw in the Thessalonians, we should come to the same conclusion. As evangelicals, sometimes we are very quick to talk about all the ways we fail, but much less quick to recognise that God is at work in us, and that he is succeeding.

1 Thessalonians 4:1, a little later, says this:

As for other matters, brothers and sisters, we instructed you how to live in order to please God, as in fact you are living. Now we ask you and urge you in the Lord Jesus to do this more and more.

Paul is saying you are already living in a way that pleases God.

There are multiple ways in which God is pleased with his people. When you wake up in the morning, before you’ve done anything good or bad, you can say “God is pleased with me today, because I’m in Christ, and all his righteousness is counted to me.” That’s justification, it’s absolutely wonderful and foundational.

But that’s not what I’m talking about here. God is also pleased with us because of our behaviour. And sometimes we struggle with this a bit more. Sometimes we even say “nothing we do pleases God, all our works are like filthy rags”. That may be true for the unbeliever, but it’s not for the believer. Our good works are described in the Bible as good works, not bad works. That doesn’t mean they are perfect. There may be some sin in them, there may be mixed motives, or failure of various kinds. But they are still good works, and they bring God pleasure.

The Holy Spirit is active in us just as he was in the Thessalonians to produce things that are genuinely good, that bring pleasure to our Father in heaven. And he’s a heavenly Father who is so much better than any human father. That means for one that he’s never a nit-picker. He’s not a fault finder. He’s not hard to please.

One of the really importance consequences of this will be joy and motivation.

Imagine, for a second, the gender-stereotypical married couple, where the wife stays at home and puts a lot of effort into looking after the house and cooking meals, and the husband goes out to work. Imagine that the husband, after finishing the meal the wife has prepared, every single time, says “I want you to know that, despite the meal you have made me, I will always love you”. How’s she going to feel? Isn’t she going to be very depressed? If she thinks that with all her efforts no meal she makes ever brings him any pleasure, that he doesn’t enjoy it at all – isn’t she going to be miserable? Where will she get motivation from?

In the same way, we will live miserable Christian lives if we think our service of God is never good enough to please Him. No! We need to understand that our labour prompted by love does bring God pleasure, by God’s power we are succeeding in our aim! (Even if not perfectly). It is the normal Christian life to lie down at the end of a day and think “I’m living the way my heavenly Father wants, and he is pleased with my service today”. And then, you’ll be motivated to do the “more and more” bit.

The message is this - God is pleased with all your love and faith at work this week, it has brought a smile to his face. So enjoy that smile, and let the joy of knowing that smile motivate you to keep going, and do more and more.

Why I’m not letting the juniors use GenAI for coding

2025-07-26T21:30:00+01:00

In my current project, I am training some junior developers — some of them pretty much brand new developers — and one of the first rules I gave them was “ensure that Copilot (or any other Generative AI assistant that will write code for you in your editor) is turned off”. This post explains why. The long and short of it is this: it’s because I want the junior developers to become senior developers, and I want them to enjoy programming as well.

Other people might also be interested in my reasons, so I’m writing this as a blog post.

I’ve read many, many other posts that are for and against, or just plain depressed, and some of this is about personal preference, but as I’m making rules for other people, I feel I ought to justify those rules just a little.

I’m also attempting to write this up in a way that hopefully non-programmers can understand. I don’t want to write a whole load of posts about this increasingly tedious subject, so I’m making one slightly broader one that I can just link to if anyone asks my opinion.

Rather than talk generalities, I’ll build my case using a single very concrete example of real output from an LLM on a real task.

The problem

This example comes from a one-off problem I created when I accidentally ended up with data in two separate SQLite databases, when I wanted it one. The data wasn’t that important – it was all just test data for the project I’m currently working on – so I could have just thrown one of these databases away. But the data had some value to me, so I decided to see if I could use an LLM to quickly create a one-off script to merge the two databases.

Merging databases sounds like something so common there would be a generic tool to do it, but in reality the devil is in the details, and every database schema has specifics that mean you need a custom solution.

The specific databases in question were pretty small, with a pretty simple schema. The only big problem with the schema, common to many databases schemas, is how you deal with ID fields:

This database schema has multiple tables, and records in one table are related to records in another table using an ID column, which in my case was just a unique number starting at one: 1, 2, 3 etc. To make it concrete, let’s say my database stored a list of houses, and a list of rooms in each house (it didn’t, but it works as an example).

So you have a houses table:

id	address
1	45 Main St
2	67 Mayfair
3	2 Bag End
…	…

And a rooms table:

id	house_id	name
1	1	Dining room
2	1	Living room
3	2	The Library
…	…	…

The house_id column above links each room with a specific house from the houses table. In the example above, the rooms with ids 1 and 2 both belong to house with ID 1, aka “45 Main St”.

Due to the database merge, we’ve got multiple instances of each of these tables, and they could very easily be using the same ID values to refer to different things. When I come to merge the tables:

I’ve got to assign new values in the id columns for each table
for the rooms table, I’ve got to remember the mapping of old-to-new IDs from the houses table and apply the same mapping to the house_id column.

If I get this wrong, the data will be horribly confused and corrupted.

What I did

I used aider.chat, which is a pretty good project with a good reputation, and one that I’m used to, to the point of being reasonably competent (although I can’t claim I use any of these tools a massive amount). This was a while back, and I can’t remember which LLM model I was using, but I think it was one of the best ones from Claude, or maybe DeepSeek-R1, both of which are (or were) well regarded.

I fed it the database schema as a SQL file, then prompted it similar to below:

Write a script that takes a list of db files as command line args, and merges the contents. The output will have a single ’metadata’ table copied from one of the input files. The ’rooms’ and ’houses’ tables will be merged, being careful to update ’id’ and ’house_id’ columns correctly, so that they refer to new values.

I’m not going to spend time arguing about whether this was the best possible tool/model/prompt, that’s an endless time sink – I’m happy that I did a decent enough job on all fronts for my comments to be fair.

How it went

The results were basically great in most ways that most people would care about: the LLM created a mostly working script of a few hundred lines of code in a few minutes. If I remember correctly it was pretty much there on the first attempt.

I did actually care about the data, so I carefully checked the script, especially the code that mapped the IDs.

There was one bit of code that created the mapping, and a second bit of code that then used it. For this second part, correct code would have looked something like this:

new_id = id_mapping[old_id]

Here:

id_mapping is a mapping (dictionary) that contains the “old ID” as keys, and the “new ID” as values
old_id is a variable containing an old ID value.
the brackets syntax [old_id] does a dictionary lookup and returns the value found – in other words, given the old ID it returns the new ID.

There is a crucial question with mappings like this: what happens if, for some reason, the mapping doesn’t contain the old_id value? With the code as written above, the dictionary lookup will raise an exception, which will cause the code to abort at this point. This is a good thing – since somehow we are missing a value, there is nothing we can do with the current data, and obviously there is some serious problem with our code which means that aborting loudly is the best of our options, or at least the most sensible default.

However, what the LLM actually wrote was like this:

new_id = id_mapping.get(old_id, old_id)

This code also does a dictionary lookup, but it uses the get method to supply a default value. If the lookup fails because the value is missing, the default is returned instead of raising an exception. The default given here is old_id, which is the original value. This is, of course, disastrously wrong. If, for whatever reason, the new ID value is missing, the old ID value is not a good enough fallback – that’s just going to create horribly corrupted data. Worst of all, it will do so absolutely silently – it could just fill the output data with essentially garbage, with no warning that something had gone wrong.

We might ask where this idea came from — the LLM has written extra code to produce a much worse result, why? The answer is most likely found, in part, in the way the LLM was trained – that is, on mediocre code.

A better answer to “why” the AI wrote this is much more troubling, but I’ll come back to that.

We might also ask, “does it matter?”

Part of the answer to that is context. For the project I’m currently working on, “silently wrong output” is one of the very worst things we can do. There are projects with different priorities, and there are even a few where quality barely matters at all, for which we really wouldn’t care about things like this. There are also lots of projects where you might expect people would care about this, but they actually don’t, which is fairly sad. Anyway, I’m glad not to be currently working on any projects like that – correct output matters, and I like that.

In this case, there is a second reason you might say it doesn’t matter: the rest of the code was actually correct. This meant that the mapping dictionary was complete, so the incorrect behaviour would never get triggered – the problem I’ve found is actually hypothetical. So what am I worrying about?

The problem is that in real code, the hypothetical could become reality very quickly. For example:

Some change to the code could introduce a bug which means that the mapping is now missing entries.
A refactoring means the code gets used in a slightly different situation.
There is some change to the context in which the code is used. For example, if other processes were writing to one of these databases while the merge operation was happening, it would be entirely possible for the mapping dictionary to be missing entries.

So we find ourselves in an interesting situation:

The code, as written by the LLM, appears on the one hand to be perfectly adequate, if measured according to the metric of “does it work right now”.
On the other hand, the code is disastrously inadequate if measured by the standard of letting it go anywhere near important data, or anything you would want to live long term in your code base.

The main point

Writing correct code is hard. The difference between correct and disastrously bad can be extremely subtle. These differences are easily missed by the untrained eye, and you might not even be able to come up with an example where the bad code fails, because in its initial context it does not.

There are many, many other examples of this in programming, and there are many examples of LLMs tripping up like this. Often it is not what is present that is even the problem, but what is absent.

So what?

OK, so LLMs sometimes write horribly flawed code that appears to work. Couldn’t we say the same about junior programmers?

Yes, we could. I think the big difference comes when you think about what happens next, after this bad code is written. So, I’ll think about this under 3 scenarios.

Scenario 1

In this scenario, I, as a senior developer, am the person who got the LLM to write the code, and I’m now tasked with code review in order to find potential flaws and time-bombs like the above.

First, in this scenario, the temptation to not check carefully is very strong. The whole reason I’m tempted to use an LLM in the first place is that I don’t want to devote much time to the task. For me this happens when:

I can’t justify much time, because I consider it’s not that important - it’s just something I need to do to get back to the main thing I’m doing.
I don’t think I will enjoy spending that time.

For both of these cases, the “must go faster” mindset means it’s psychologically very hard to slow down and do the careful review needed.

So, I’m unlikely to review this code as carefully as I should. For me, assuming that the code matters at all, this is a killer problem.

Maybe someone else would review it and catch this? That’s not good enough for me – I don’t rely on other people reviewing my code. I’m a professional software developer who works on important projects. Sometimes I work alone, with no-one else doing effective review. My clients expect and deserve that I write code that actually works.

Of course I also know that I’m far from perfect, and welcome any code review I can get. But even when I think there is review going on, I treat it as a backup, an extra safety measure, and not a reason to justify being sloppy and careless.

Scenario 2

In this scenario, the code was written by a junior developer. In contrast to the previous section, I don’t expect junior developers to produce code that works. But I do expect them to learn to do so. And I hope that that I will be rightly suspicious and do a thorough review.

Like Glyph, I actually quite enjoy doing code review for junior developers, as long as they actually have both the willingness and capacity to improve (which they usually do, but not always). Code review can be a great opportunity to actually train someone.

So what happens in this scenario when I, hopefully, after careful review spot the flaw and point it out?

If I have time to properly mentor a developer, the process would be a conversation (preferably in person or a video call) that starts something like this:

When you wrote new_id = old_id_to_new_id.get(old_id, old_id), can you explain what was going through your mind? Why did you choose to write .get(old_id, old_id), rather than the simpler and shorter [old_id]?

It’s possible the reason was them thinking “doing something is better than crashing”. We can then address that disastrously wrong (but quite common) mindset. This will hopefully be a very fruitful discussion, because it will actually correct the issue at the root, and so stop it happening again, or at least make it much less likely.

In some cases, there isn’t a reason “why” they wrote the code they did – sometimes junior developers just don’t understand a lot of the code they are writing, they are just guessing until something seems to work. In that case, we can address that problem:

The developer needs to go back to basics of things like assignments and function calls etc. The developer must reach the point where they can explain and justify every last jot and tittle of code they produce. It seems pretty obvious to me that the best way to achieve that is to make them write pretty much all their code, at least at the level of choosing each “token” that they add (e.g. choosing from a list of methods with auto-complete is OK, but not more than that). If I want them to be able to justify each token choice, it is essential to make them engage their brain and choose each token.

Scenario 3

The third scenario is again that the junior developer “produced” the code, and I’m now reviewing it, but it turns out they just prompted some LLM assistant.

At this point, the first step in the root cause analysis – the question “what was your thought process in producing this code” – fails immediately.

There is no real answer to the question “why” the LLM wrote the bad code in the first place. You can’t ask it “what were you were thinking” and get a meaningful answer – it’s pointless to ask so don’t even try. It lacks the meta-cognition needed for self-improvement, and much of the time when it looks like it is reasoning, it is actually just pattern matching.

The next problem is that the LLM basically can’t learn, at least not deeply enough to make a significant difference. You can tell it “don’t do that again”, and it might partly work, but you can’t really change how it thinks, just what the current instructions are.

So we can’t address the root cause.

What about the junior developer learning from this, in this scenario – is there something they could take away which would prevent the mistake in the future?

If their own mind wasn’t engaged in making the mistake, I think it is quite unlikely that they will be able to effectively learn from the LLM’s mistakes. For the junior dev, the possible take-aways about changes to their own behaviour are:

Check the LLM’s output more carefully. But I’m not convinced they are equipped at this point to really do checking – they first need practice in the careful thinking about what correct code looks like, to gain the trained eye that can spot subtle issues.
Don’t use LLMs to write code (which is what I’m arguing here)

What about the mid-level programmer?

At what point do I suggest the junior developers should start using LLMs for some of the more boring tasks? Once they are “mid-level” (whatever that means), is it appropriate?

There is obviously a point at which you let people make their own decisions.

For myself, I’m pretty firmly convinced that the software I’ve just recently created with 25+ years’ professional experience, I simply couldn’t have created with only 15 years experience. Not because I’m a poor programmer – I think I’ve got good reason to believe I’m significantly above average (for example, I taught myself machine code and assembly as a young teenager, and I’ve been part of the core team of top open-source projects). There is just a lot to learn, and always a lot further you could go.

In this most recent project, we’ve seen a lot of success because of certain key architectural decisions that I made right at the beginning. Some of these used advanced patterns that 10 years I would not have instinctively reached for, or wouldn’t even have known well enough to attempt them.

For example, due to difficulties with manual testing of the output in my current project, we depend critically on regression tests that make heavy use of a version of the Command pattern or plan-execute pattern. In fact we use multiple levels of it, one level being essentially an embedded DSL that has both an evaluator and compiler. This code is not trivial, and it’s also quite far from the simplest thing that you would think of, but it has proven critical to success.

How did I know I needed these? Well, I can tell you one thing for sure: I would never have got the experience and judgement necessary if I hadn’t been doing a lot of the coding myself over the past 25 years.

In his video A tale of two problem solvers, youtuber 3Blue1Brown has a section that is hugely relevant here, especially from 36 minutes onwards. In it, he describes how practising mathematicians will often do hundreds of concrete examples in order to build up intuition and sharpen their skills so that they can tackle more general and harder problems. What particularly struck me was that famous mathematicians throughout history have done this – “they all have this seemingly infinite patience for doing tedious calculations”.

Computer programming may seem different, in that we deliberately don’t do the tedious calculations, but teach the computer to do that. However, there are huge similarities. Programming, like mathematics, involves a formal notation and problem solving. In the case pf programming, the formal notation is aimed at a machine that will mechanically interpret it to produce a desired behaviour.

Obviously we do avoid doing lots of long multiplication once we’ve taught the computer to do that. However, given the similarities of the mental processes involved in maths and programming, when it comes to any of the higher level things about how to structure programs, I think we are absolutely fooling ourselves if we think we can avoid doing all the “grunt work” of writing code, organising code bases, slowly improving code structure, etc. and still end up magically knowing all the patterns that we need to use, understanding their trade-offs, and all the reasons why certain architectures or patterns would or wouldn’t be appropriate.

If you ever want to progress beyond mid-level, I strongly suspect that offloading significant parts of programming to an LLM will greatly reduce your growth. You may be much faster at outputting things of a similar level to what you can currently handle, but I doubt you’ll be able to tackle fundamentally harder projects in the future.

While I’m talking about youtubers, the video The Expert Myth by Veritasium is also really helpful here. He describes how expertise requires the following ingredients:

Valid environment
Many repetitions
Timely feedback
Deliberate practice

As far as I can see, heavy use of LLMs to write code for you will destroy points 2 and 3 – neither the repetitions nor the feedback are really happening if you don’t actually write the code yourself. I doubt that the “deliberate practice and study, in an uncomfortable zone” is going to happen either if you never get happy with the manual bits of coding.

And what about the senior programmer?

To complete this post, we of course want to ask, should the “senior programmer” use LLMs to do a lot of the grunt programming? By senior, I do mean significantly more than the 5 years that many “seniors” have. Does there come a point where you have “arrived” and the arguments in the previous section no longer apply? A time when you basically don’t need to do much more learning and sharpening of skills?

My answer to that is “I hope not”. Like others, I’m still hoping that by the end of my career I could be at least 10x faster than I am now (without an LLM writing the code for me), maybe even more. Computers are mental levers, so I don’t think that’s ridiculous. I’m hoping not just to be 10x faster, but 10x better in other ways – in terms of reliability, or in terms of being able to tackle problems that would just stump me today, or to which my solutions today would be massively inferior.

Everything I know about learning says that outsourcing my actual thinking to LLMs is unlikely to produce that result.

In addition, there are at least some people who, after actively using LLMs integrated into their editor (like Copilot and Cursor), have now stopped doing so because they noticed their skills were rusting.

These arguments, plus the small amount of evidence I have, are enough that I don’t feel the need to make a guinea pig out of myself to collect more data.

So, for myself, I choose to use LLMs very rarely for actually writing code. Rarely enough that if they were to disappear completely, it would make little difference to me.

But, aside from the practical arguments regarding becoming a better programmer, one of the big reasons for this is that I simply enjoy programming. I enjoy the challenge and the process of getting the computer to do exactly what I want it to do, in a reasonable amount of time, expressing myself both precisely and concisely.

When you are programming at the optimal level of abstraction, it is actually much nicer to express yourself in code compared to English, and often much faster too. Natural language is truly horrible for times when you want precision. And in computer programming, you usually are able to create the abstractions you need, so you can often get close to that optimal level of abstraction.

Obviously there are times when there is a bad fit between the abstractions you want and the abstractions you have, resulting in a lot of redundancy. You can’t always rewrite everything to make it all as ideal as you want. But if I’m writing large amounts of code at the wrong abstraction level, that’s bad, and I don’t really want a system that helps me to write more and more code like that.

The point of this section is really for the benefit of the junior developers that I’m forcing to do things “the hard way”. What I’m saying is this: I’m not withholding some special treat that I reserve just for myself. I willingly code in exactly the same way as you, and I really enjoy it. I believe that I’m sparing you the miserable existence of never becoming good at programming, while you keep trying to cajole something that doesn’t understand what it’s doing into producing something you don’t understand either. That just doesn’t sound like my idea of fun.

Updates

March 2026:

It looks like Carson Gross, of whom I’m a big fan, agrees with this stance when it comes to his students.
Since writing the above, I have successfully used coding agents to produce significant, high quality projects where I have done extremely little of the actual coding. However, I stand by what I’ve written above: while the agent was coding away, I never relegated myself to merely managing it. I was coding away just as hard, but on a different, related bit of work. I still don’t use AI integrated into my editor.

I’m not letting my skills stagnate, and if I get an agent to work outside my area of competency, I do it consciously. For example, in the planegcs library linked above, I was quite happy to not improve my understanding of C++ and its build tools. On the other hand, I think I’m going to need to do a Rust project, and I’m intending to do all the coding myself, even though I know an agent could make much more rapid progress, because I actually want to get good at Rust.

Statically checking Python dicts for completeness

2025-06-27T11:09:03+01:00

In Python, I often have the situation where I create a dictionary, and want to ensure that it is complete – it has an entry for every valid key.

Let’s say for my (currently hypothetical) automatic squirrel-deterring water gun system, I have a number of different states the water tank can be in, defined using an enum:

from enum import StrEnum

class TankState(StrEnum):
    FULL = "FULL"
    HALF_FULL = "HALF_FULL"
    NEARLY_EMPTY = "NEARLY_EMPTY"
    EMPTY = "EMPTY"

In a separate bit of code, I define an RGB colour for each of these states, using a simple dict.

TANK_STATE_COLORS = {
    TankState.FULL: 0x00FF00,
    TankState.HALF_FULL: 0x28D728,
    TankState.NEARLY_EMPTY: 0xFF9900,
    TankState.EMPTY: 0xFF0000,
}

This is deliberately distinct from my TankState code and related definitions, because it relates to a different part of the project - the user interface. The UI concerns shouldn’t be mixed up with the core logic.

This dict is fine, and currently complete. But I’d like to ensure that if I add a new item to TankState, I don’t forget to update the TANK_STATE_COLORS dict.

With a growing ability to do static type checks in Python, some people have asked how we can ensure this using static type checks. The short answer is, we can’t (at least at the moment).

But the better question is “how can we (somehow) ensure we don’t forget?” It doesn’t have to be a static type check, as long as it’s very hard to forget, and if it preferably runs as early as possible.

Instead of shoe-horning everything into static type checks, let’s just make use of the fact that this is Python and we can write any code we want at module level. All we need to do is this:

TANK_STATE_COLORS = {
    # …
}
for val in TankState:
    assert val in TANK_STATE_COLORS, f"TANK_STATE_COLORS is missing an entry for {val}"

That’s it, that’s the whole technique. I’d argue that this is a pretty much optimal, Pythonic solution to the problem. No clever type tricks to debug later, just 2 lines of plain simple code, and it’s impossible to import your code until you fix the problem, which means you get the early checking you want. Plus you get exactly the error message you want, not some obscure compiler output, which is also really important.

It can also be extended if you want to do something more fancy (e.g. allow some values of the enum to be missing), and if it does get in your way, you can turn it off temporarily by just commenting out a couple of lines.

That’s not quite it

OK, in a project where I’m using this a lot, I did eventually get bored of this small bit of boilerplate. So, as a Pythonic extension of this Pythonic solution, I now do this:

TANK_STATE_COLORS: dict[TankState, int] = {
    TankState.FULL: 0x00FF00,
    TankState.HALF_FULL: 0x28D728,
    TankState.NEARLY_EMPTY: 0xFF9900,
    TankState.EMPTY: 0xFF0000,
}
assert_complete_enumerations_dict(TANK_STATE_COLORS)

Specifically, I’m adding:

a type hint on the constant
a call to a clever utility function that does just the right amount of Python magic.

This function needs to be “magical” because we want it to produce good error messages, like we had before. This means it needs to get hold of the name of the dict in the calling module, but functions don’t usually have access to that.

In addition, it wants to get hold of the type hint (although there would be other ways to infer it without a type hint, there are advantages this way), for which we also need the name.

The specific magic we need is:

the clever function needs to get hold of the module that called it
it then looks through the module dictionary to get the name of the object that has been passed in
then it can find type hints, and do the checking.

So, because you don’t want to write all that yourself, the code is below. It also supports:

having a tuple of Enum types as the key
allowing some items to be missing

using Literal as the key. So you can do things like this:

X: dict[typing.Literal[-1, 0, 1], str] = {
    -1: "negative",
    0: "zero",
    1: "positive",
}
assert_complete_enumerations_dict(X)

It’s got a ton of error checking, because once you get magical then you really don’t want to be debugging obscure messages.

Enjoy!

I hereby place the following code into the public domain - CC0 1.0 Universal.

import inspect
import itertools
import sys
import typing
from collections.abc import Mapping, Sequence
from enum import Enum

from frozendict import frozendict


def assert_complete_enumerations_dict[T](the_dict: Mapping[T, object], *, allowed_missing: Sequence[T] = ()):
    """
    Magically assert that the dict in the calling module has a
    value for every item in an enumeration.
    The dict object must be bound to a name in the module.
    It must be type hinted, with the key being an Enum subclass, or Literal.
    The key may also be a tuple of Enum subclasses

    If you expect some values to be missing, pass them in `allowed_missing`
    """
    assert isinstance(the_dict, Mapping), f"{the_dict!r} is not a dict or mapping, it is a {type(the_dict)}"

    frame_up = sys._getframe(1)  # type: ignore[reportPrivateUsage]
    assert frame_up is not None
    module = inspect.getmodule(frame_up)
    assert module is not None, f"Couldn't get module for frame {frame_up}"
    msg_prefix = f"In module `{module.__name__}`,"

    module_dict = frame_up.f_locals
    name: str | None = None
    # Find the object:
    names = [k for k, val in module_dict.items() if val is the_dict]
    assert names, f"{msg_prefix} there is no name for {the_dict}, please check"

    # Any name that has a type hint will do, there will usually be one.
    hints = typing.get_type_hints(module)
    hinted_names = [name for name in names if name in hints]
    assert (
        hinted_names
    ), f"{msg_prefix} no type hints were found for {', '.join(names)}, they are needed to use assert_complete_enumerations_dict"
    name = hinted_names[0]

    hint = hints[name]
    origin = typing.get_origin(hint)
    assert origin is not None, f"{msg_prefix} type hint for {name} must supply arguments"
    assert origin in (
        dict,
        typing.Mapping,
        Mapping,
        frozendict,
    ), f"{msg_prefix} type hint for {name} must be dict/frozendict/Mapping with arguments to use assert_complete_enumerations_dict, not {origin}"

    args = typing.get_args(hint)
    assert len(args) == 2, f"{msg_prefix} type hint for {name} must have two args"

    arg0, _ = args
    arg0_origin = typing.get_origin(arg0)
    if arg0_origin is tuple:
        # tuple of Enums
        enum_list = typing.get_args(arg0)
        for enum_cls in enum_list:
            assert issubclass(
                enum_cls, Enum
            ), f"{msg_prefix} type hint must be an Enum to use assert_complete_enumerations_dict, not {enum_cls}"

        items = list(itertools.product(*(list(enum_cls) for enum_cls in enum_list)))
    elif arg0_origin is typing.Literal:
        items = typing.get_args(arg0)
    else:
        assert issubclass(
            arg0, Enum
        ), f"{msg_prefix} type hint must be an Enum to use assert_complete_enumerations_dict, not {arg0}"
        items = list(arg0)

    for item in items:
        if item in allowed_missing:
            continue

        # This is the assert we actually want to do, everything else is just error checking:
        assert item in the_dict, f"{msg_prefix} {name} needs an entry for {item}"

Knowledge creates technical debt

2025-05-13T09:08:50+01:00

The term technical debt, now used widely in software circles, was coined to explain a deliberate process where you write software quickly to gain knowledge, and then you have to use that knowledge gained to improve your software.

This perspective is still helpful today when people speak of technical debt as only a negative, or only as a result of bad decisions. Martin Fowler’s Tech Debt Quadrant is a useful antidote to that.

A consequence of this perspective is that technical debt can appear at any time, apparently from nowhere, if you are unfortunate enough to gain some knowledge.

If you discover a better way to do things, the old way of doing it that is embedded in your code base is now “debt”:

you can either live with the debt, “paying interest” in the form of all the ways that it makes your code harder to work with;
or you can “pay down” the debt by fixing all the code in light of your new knowledge, which takes up front resources which could have been spent on something else, but hopefully will make sense in the long term.

This “better way” might be a different language, library, tool or pattern. In some cases, the better way has only recently been invented. It might be your own personal discovery, or something industry wide. It might be knowledge gained through the actual work of doing the current project (which was Ward Cunningham’s usage of the tem), or from somewhere else. But the end result is the same – you know more than you did, and now you have a debt.

The problem is that this doesn’t sound like a good thing. You learn something, and now you have a problem you didn’t have before, and it’s difficult to put a good spin on “I discovered a debt”.

But from another angle, maybe this perspective gives us different language to use when communicating with others and explaining why we need to address technical debt. Rather than say “we have a liability”, the knowledge we have gained can be framed as an opportunity. Failure to take the opportunity is an opportunity cost.

The “pile of technical debt” is essentially a pile of knowledge – everything we now think is bad about the code represents what we’ve learned about how to do software better. The gap between what it is and what it should be is the gap between what we used to know and what we now know.

And fixing that code is not “a debt we have to pay off”, but an investment opportunity that will reap rewards. You can refuse to take that opportunity if you want, but it’s a tragic waste of your hard-earned knowledge – a waste of the investment you previously made in learning – and eventually you’ll be losing money, and losing out to competitors who will be making the most of their knowledge.

Finally, I think phrasing it in terms of knowledge can help tame some of our more rash instincts to call everything we don’t like “tech debt”. Can I really say “we now know” that the existing code is inferior? Is it true that fixing the code is “investing my knowledge”? If it’s just a hunch, or a personal preference, or the latest fashion, maybe I can both resist the urge for unnecessary rewrites, and feel happier about it at the same time.

Christ the True and Better Frodo

2025-02-12T20:44:13Z

Inspired by our family conversation at our meal time tonight — we have three boys who are already big Lord of the Rings fans — I decided to put ChatGPT to some edifying use.

I want you to rewrite the following song, Christ the True and Better Adam, by Matt Boswell / Matt Papa / Keith Getty, which compares Jesus to Old Testament figures. I want a new version which compares instead to characters from Lord of the Rings. It should include references to the different things they did, and how Christ is the better version of them. It's important to keep the lyrics in the same meter, so it could be sung to the original tune. Here is the original song:

Christ, the true and better Adam
Son of God and Son of man
Who, when tempted in the garden
Never yielded, never sinned
He, who makes the many righteous
Brings us back to life again
Dying, He reversed the curse, then
Rising, crushed the serpent's head

Christ, the true and better Isaac
Humble son of sacrifice
Who would climb the fearful mountain
There to offer up his life
Laid with faith upon the altar
Father's joy and only son
There salvation was provided
Oh, what full and boundless love

Amen! Amen!
From beginning to end
Christ the story, His the glory
Alleluia! Amen!

Christ, the true and better Moses
Called to lead a people home
Standing bold to earthly powers
God's great glory to be known
With his arms stretched wide to heaven
See the waters part in two
See the veil is torn forever
Cleansed with blood we pass now through

Amen! Amen!
From beginning to end
Christ the story, His the glory
Alleluia! Amen!

Christ, the true and better David
Lowly shepherd, mighty king
He the champion in the battle
Where, o death, is now thy sting?
In our place He bled and conquered
Crown Him Lord of majesty
His shall be the throne forever
We shall e'er His people be

Amen! Amen!
From beginning to end
Christ the story, His the glory
Alleluia! Amen!

Here's your rewritten version comparing Christ to characters from The Lord of the Rings, keeping the same meter so it can be sung to the original tune.

Christ, the true and better Frodo
Bore the burden none could bear
Left His home to face the darkness
Took the cross to end despair
Through the fire of tribulation
Passed alone to win the fight
Crushed the power of the darkness
Bringing us into the light

Christ, the true and better Aragorn
He, the long-expected King
Left His throne to seek the hopeless
Now we rise at His redeeming
From the paths of death He called us
Broke the curse, renewed the land
By His hand we reign in glory
In His kingdom e'er to stand

Amen! Amen!
From beginning to end
Christ the story, His the glory
Alleluia! Amen!

Christ, the true and better Gandalf
Came to break the prison's seal
Faced the shadow, fell in darkness
Rose again, the doom repealed
Leading forth a ransomed people
Bringing hope where all seemed lost
By His wisdom, by His power
Every fear and foe is crushed

Christ, the true and better Samwise
Faithful till the journey’s end
Took our burden, bore it for us
Loved beyond what we could send
At our weakest, He sustains us
Never turning from the way
At His side we reach the homeland
Ever in His light to stay

Amen! Amen!
From beginning to end
Christ the story, His the glory
Alleluia! Amen!

Honestly, I think it pretty much nailed it. It exactly captures the way the original song works, and the better you know the LOTR stories the more you appreciate some of the details. Even the scanning is better than I hoped - there are just a couple of places that would need fixing if you wanted to sing this.

I particularly liked the verse about Samwise, which I wasn’t expecting — the other characters have more obvious parallels to draw on — and at points it’s really quite touching. ChatGPT presumably is drawing on a large and wonderful corpus of hymnody that helps shape its output, but it’s done a pretty good job of pulling all that together.

Anyway, hope you enjoyed!

Updates

Some fixes I would make:

v2 Instead of “Left His throne to seek the hopeless / Now we rise at His redeeming” which doesn’t fit the story of Aragorn and also doesn’t scan, maybe something like:

Passed the test and claimed his birthright

Proved by hands that healing bring

or:

Tested, tried and proven worthy,

Known by hands that healing bring
v4 Instead of “Loved beyond what we could send” which doesn’t exactly make sense, I would put:

Well deserves the name of Friend!
v1 My wife points out that Frodo doesn’t really conquer, he stumbles at the end, where Christ did not. I don’t know how to fix that.

Recursive project search in Emacs

2024-12-30T19:34:33Z

Before reading this, you might want to check it out in video form on Youtube, or with the embed below:

Video is probably a more helpful format for demonstrating the workflow I’m talking about. Otherwise read on. If you’re coming from the video to look at the Elisp code, it is found towards the bottom of this post.

“Recursive project search” is the name I’m giving to the flow where you do some kind of search to identify things that need to be done, but each of those tasks may leads you to do another search, etc. You need to complete all the sub-searches, but without losing your place in the parent searches.

This is extremely common in software development and maintenance, whether you are just trying to scope out a set of changes, or whether actually doing them. In fact just about any task can end being some form of this, and you never know when it will turn out that way.

This post is about how I use Emacs to do this, which is not rocket science but includes some tips and Elisp tweaks that can help a lot. When it comes to other editors or IDEs I’ve tried, I’ve never come close to finding a decent workflow for this, so I’ll have to leave it to other people to describe their approaches with other editors.

Example task - pyastgrep

I’m going to take as an example my pyastgrep project and a fairly simple refactoring I needed to do recently.

For background, pyastgrep is a command line program and library that allows you to grep Python code at the level of Abstract Syntax Trees rather than just string. At the heart of this is a function that takes a Python path, and converts it to AST and also XML.

The refactoring I want to do is make this function swappable, mostly so that users can apply different caching strategies to it. This is going to be a straightforward example of turning it into a parameter, or “dependency injection” if you want a fancy term. But that may involve modifying a number of functions in several layers of function calls.

The function in question is process_python_file.

Example workflow

The first step is a search, which in this case I will doing using lsp-mode. I happen to use lsp-pyright for Python, but there are other options.

So I’ll kick off by opening the file, putting my cursor on the function python_process_file, and calling M-x lsp-find-references. This returns a bunch of references. I can then step through them using M-x next-error and M-x previous-error — for which there are shortcuts defined, and I also do this so much that I have an F key for it — F2 and shift-F2 respectively.

Notice that in addition to the normal cursor, there is also a little triangle in the search results which shows the current result you are on.

In this case, after the function definition itself, and an import, there is just one real usage – that last item in the search results.

The details of doing this refactoring aren’t that important, but I’ll include some of the steps for completeness. The last result brings me to code like this:

def search_python_file(
    path: Path | BinaryIO,
    query_func: XMLQueryFunc,
    expression: str,
) -> Iterable[Match | ReadError | NonElementReturned]:

    ...

    processed_python = process_python_file(path)

So I make process_python_file a parameter:

def search_python_file(
    path: Path | BinaryIO,
    query_func: XMLQueryFunc,
    expression: str,
    *,
    python_file_processor: Callable[[Path], ProcessedPython | ReadError] = process_python_file,
) -> Iterable[Match | ReadError | NonElementReturned]:

    ...

    processed_python = python_file_processor(path)

Having done this, I now need to search for all usages of the function I just modified, search_python_file, so that I can pass the new parameter — another M-x lsp-find-references. I won’t go into the details this time, in this case it involves the following:

Wherever search_python_file is used, either:

don’t pass python_file_processor, because the default is what we want.
or do pass it, usually by similarly adding a python_file_processor parameter to the calling function, and passing that parameter into search_python_file.

This quickly gets me to search_python_files (note the s), and I find that it is imported in pyastgrep.api. There are no usages to be fixed here, but it is exported in __all__. This reminds me that the new parameter to this search_python_files function is actually intended to be a part of the publicly documented API — in fact this is the whole reason I’m doing this change. This means I now need to fix the docs. Another search is needed, but this time a string-based grep in the docs folder. For this, I use ripgrep and M-x rg-project-all-files.

So now I have another buffer of results to get through – I’m about 4 levels deep at this point.

Now comes one of the critical points in this workflow. I’ve completed the docs fix, and I’ve reached the end of that ripgrep buffer of search results:

So I’ve come to a “leaf” of my search. But there were a whole load of other searches that I only got half way through. What happens now?

All I do is kill these finished buffers – both the file I’m done with, and the the search buffer. And that puts me back to the previous search buffer, at exactly the point I left off, with the cursor in the search buffer in the expected place.

So I just continue. This process repeats itself, with any number of additional “side quests”, such as adding tests etc., until I get to the end of last search buffer, at which point I’m done.

Explanation

What I’ve basically done here is a depth-first recursive search over “everything that needs to be done to complete the task”. I started working on one thing, which lead to another and another, and for each one I went off and completed them as they came up.

Doing a search like that requires keeping track of a fair amount of state, and if I were doing that in my head, I would get lost very quickly and forget what I was doing. So what I do instead is to use Emacs buffers to maintain all of that state.

The buffers themselves form a stack, and each buffer has a cursor within it which tells me how far through the list of results I am. (This is equivalent to how recursive function calls typically work in a program - there will be a stack of function calls, each with local variables stored in a frame somehow).

In the above case I only went about 4 or 5 levels deep, and each level was fairly short. But you can go much deeper and not get lost, because the buffers are maintaining all the state you need. You can be 12 levels down, deep in the woods, and then just put it to one side and come back after lunch, or the next day, and just carry on, because the buffers are remembering everything for you, and the buffer that is in front of you tells you what you were doing last.

It doesn’t even matter if you switch between buffers and get them a bit out of order – you just have to ensure that you get to the bottom of each one.

Another important feature is that you can use different kinds of search, and mix and match them as you need, such as lsp-find-references and the ripgrep searches above. In addition to these two, you can insert other “search-like” things, like linters, static type checkers, compilers and build processes – anything that will return a list of items to check. So this is not a feature of just one mode, it’s a feature of how buffers work together.

At each step, you can also apply different kinds of fixes – e.g. instead of manually editing, you might be using M-x lsp-rename on each instance, and you might be using keyboard macros etc.

When I’ve attempted to use editors other than Emacs, this is one of the things I’ve missed most of all. Just getting them to do the most basic requirement of “do a search, without overwriting the previous search results” has been a headache or impossible – although I may have given up too soon.

I’m guessing people manage somehow – or perhaps not: I’ve sometimes noticed that I’ve been happy to take on tasks that involved this kind of workflow which appeared to be daunting to other people, and completed them without problem, which was apparently impressive to others. I also wonder whether difficulties in using search in an editor drive a reluctance to take on basic refactoring tasks, such as manual renames, or an inability to complete them correctly. If so, it would help to explain why codebases often have many basic problems (like bad names).

In any case, I don’t think I can take for granted that people can do this, so that’s why I’m bothering to post about it!

Requirements

What do you need in Emacs for this to work? Or what equivalent functionality is needed if you want to reproduce this elsewhere?

First, the search buffer must have the ability to remember your position. All the search modes I’ve seen in Emacs do this.
Second, you need an easy way to step through results, and this is provided by the really convenient M-x next-error function which does exactly what you want (see the Emacs docs for it, which describe how it works). This function is part of Compilation Mode or Compilation Minor Mode, and all the search modes I’ve seen use this correctly.
Thirdly, the search modes mustn’t re-use buffers for different searches, otherwise you’ll clobber earlier search results that you hadn’t finished processing. Some modes do this automatically, others have a habit of re-using buffers — but we can fix that:

Unique buffers for searches

If search commands re-use search buffers by default, I’ve found it’s usually pretty easy to override the behaviour so that you automatically always get a unique buffer for each new search you do.

The approach you need often varies slightly for each mode, but the basic principle is similar - different searches should get different buffer names, and you can use some Elisp Advice to insert the behaviour you want.

So here are the main ones that I override:

rg.el for ripgrep searching:

(defadvice rg-run (before rg-run-before activate)
  (rg-save-search))

This is just using the save feature built in to rg.el.

This goes in your init.el. Since I use use-package I usually put it inside the relevant (use-package :config) block.

For lsp-find-references I use the following which makes a unique buffer name based on the symbol being searched for:

(advice-add 'lsp-find-references
            :around #'my/lsp-find-references-unique-buffer)

(defun my/lsp-find-references-unique-buffer (orig-func &rest args)
  "Gives lsp-find-references a unique buffer name, to help with recursive search."
  (let
      ((xref-buffer-name (format "%s %s" xref-buffer-name (symbol-at-point))))
    (apply orig-func args)))

Then for general M-x compile or projectile-compile-project commands I use the following, which gives a unique buffer name based on the compilation command used:

(advice-add 'compilation-start
            :around #'my/compilation-unique-buffer)

(defun my/compilation-unique-buffer (orig-func &rest args)
  "Give compilation buffers a unique name so that new compilations get new
buffers. This helps with recursive search.

If a compile command is run starting from the compilation buffer,
the buffer will be re-used, but if it is started from a different
buffer a new compilation buffer will be created."
  (let* ((command (car args))
         (compilation-buffer (apply orig-func args))
         (new-buffer-name (concat (buffer-name compilation-buffer) " " command)))
      (with-current-buffer compilation-buffer
        (rename-buffer new-buffer-name t))))

I use this mode quite a lot via M-x compile or M-x projectile-compile-project to do things other than compilation – for custom search commands, like pyastgrep, for linters and static checkers.

Other tips

External tools

Many of externals tools that you might run via M-x compile already have output that is in exactly the format that Emacs compilation-mode expects, so that search results or error messages become hyperlinked as expected inside Emacs. For example, mypy has the right format by default, and most older compilers like gcc etc.

For those tools that don’t, sometimes you can tweak the options of how they print. For example, if you run ripgrep as rg --no-heading (just using normal M-x compile, without a dedicated ripgrep mode), it produces the necessary format.

Alternatively, you can make custom wrappers that fix the format. For example, I’ve got this one-liner bash script to wrap pyright:

#!/bin/sh
# Adjust ANSI colour and codes and whitespace output from pyright to make it work nicer in Emacs
pyright "$@" | sed 's/\x1b\[[0-9;]*[mGK]//g' | awk '{$1=$1};1' |  sed 's/\(.*:[0-9]*:[0-9]*\) -/\1: -/g'

Buffer order and keeping things tidy

Emacs typically manages your buffers as a stack. However, it’s very easy for things to get out of order as you are jumping around files or search buffers. It doesn’t matter too much if you go through the search buffers in the “wrong” order – as long as you get to the bottom of all of them. But to be sure I have got to the bottom, I do two things:

Before starting anything that I anticipate will be more than a few levels deep, I tidy up by closing all buffers but the one I’m working on. You can do this with crux-kill-other-buffers from crux. I have my own version that is more customised for my needs — crux-kill-other-buffers only kills buffers that are files.
At the end, when I think I’m done, I check the buffer list (C-x C-b) for anything else I missed.

Project context

In order to limit the scope of searches to my project, I’m typically leaning on projectile.el, but there are other options.

Conclusion

I hope this post has been helpful, and if you’ve got additional tips for this kind of workflow, please leave a comment!

Luke Plant's home page

When perfection is table stakes

Announcing fluent-codegen

Help my website is too small

Breaking “provably correct” Leftpad

Devotions on 1 Thessalonians 1, August 2025

Why I’m not letting the juniors use GenAI for coding

Statically checking Python dicts for completeness

That’s not quite it

Knowledge creates technical debt

Links

Christ the True and Better Frodo

Updates

Recursive project search in Emacs