The old way

It would be nice if I could give more detailed feedback about why exactly I think your contribution matches the term “slop”. In the past, when I got a pull request that wasn’t up to scratch from a new contributor, I would try to take the time to give detailed feedback, to effectively train them.

This approach was based on a few assumptions:

• First, there was a person on the other end of the interaction, who deserved to be treated like a person.

• Second, that person had already demonstrated a significant amount of effort just by the fact that they bothered to write the code, and then prepare a pull request.

• Third, there was a good chance that, equipped with some of the feedback I gave, the new contributor would get better in the future, to the point where they would be an asset to the project, and more of an asset to the world in general.

Unfortunately, use of AI in software has massively eroded all of the above. The human involvement in creating the PR may have been almost zero. Writing is no longer proof of work. Many people will not benefit at all from the detailed feedback I give, because often they just give up. Basically, the balance that Open Source relied has been destroyed by the slopocalypse, which means that I have to allocate my time in a very different way to the past.

If you are a budding Open Source contributor trying to do the right thing, trying to learn, maybe using AI or maybe not using it all but still getting lumped with those who are, this sucks for you. I’m sorry for that, and for the world that has been lost, but I’m afraid I can’t fix it, and this post is the best I can do.

I’m also sorry for the curtness of my interactions with you, and the cynical tone you might pick up from this post, but again it is a matter of necessity to avoid the burn-out that many others have suffered, and to ensure I can give a good account for how I’ve used my life (Romans 14:12).

What’s wrong with your contribution

It might be that you have used an AI to generate something, or you’ve worked hard on it yourself, and you think it looks pretty good, but it is still labelled slop. Below are some clues as to why that might be. You will have to work through these yourself and see which apply to yours.

The best summary I can give for what I’m looking for in a good quality code contribution is the following:

Every single line, word, token and character should have been written with the understanding that another human being will need to read it, understand it and potentially maintain it indefinitely. This covers:

• implementation code

• test code

• documentation

• commit messages

• descriptions and comments in pull requests

• basically, everything.

Of course, the code must actually work (that’s step zero, the computer has to understand it too), and in addition, it must be for the benefit of all current and future users, which means it mustn’t recklessly break anything, for example, and must be designed with long term needs in mind.

In short, the contribution should treat the recipients (reviewers and maintainer) as human beings whose time is extremely valuable, and the project as something valuable that needs to be protected.

Here are some of the most common problems I see with AI generated stuff:

• General “low quality”:

  • Hacky code that fixes the author’s problem, without caring about what it does to the code structure and maintainability.

  • Buggy code of all kind.

• Simply too much stuff. This includes things like:

  • Silly comments that simply repeat what you can see immediately from the code.

  • Unnecessary tests, which are not adding anything useful, and seem to demonstrate a failure to understand what the specific feature/bug/edge cases are. You should write tests like you are betting on your life.

  • a PR description or commit message which includes a load of irrelevant stuff. e.g.:

    • “100% of the test suite passes” – this is too obvious to be stated, of course it wasn’t going to be merged if it added failing tests or broke existing things, or broke other documented coding/contribution standards.

    • “3 new tests added, 2 altered” – I can see that from the diff.

    • etc. etc.

  With my own code, in common with many of my blog posts and many sermons, I often spend as much time deleting things as I do adding them.

• Not enough detail (yes, I know this is the opposite of the above).

  If the contribution breaks compatibility or has some negative behaviour or limitations, I need to know about that. You might not care about it, and the AI certainly doesn’t, but I do.

  If something looks like it might break something, but in fact doesn’t for some good reason, I want to know that. Have you considered the edge cases? Have you considered why it was the way it was?

• A failure to think about “what”.

  It is very easy to tell an AI to do a certain task or implement something, without really digging into whether that was what you should be implementing, and what the consequences of that would be.

At the end of this, you might think something like this:

Wow, this is really hard! It would probably be faster to start from scratch and write every word myself. That way, I’m only going to write the necessary stuff, because it will take too much effort to write unnecessary things. It will also force me to think about whether something really is necessary.

Then, if something looks unnecessary but actually isn’t I might even need to justify why it is needed!

If I end up actually needing a lot of text, I’m going to have to structure it extremely well and design my whole approach to this work to ensure that people’s eyes don’t just glaze over. But, I really need to avoid writing a lot of text if I possible can.

Yep, exactly, all that is true. It was always true of good coding. Doing the above is simply what is necessary when you treat the recipient as a human. A high quality PR was always a lot of work – you have to do all the work needed to make the maintainer’s job easy, because you are asking them to do something for you.

The only thing that has got harder about contributing to an Open Source project is that I’m now not telling you exactly what is wrong, you’ll have to figure that bit out for yourself, and I’m now much more likely to simply put something in the slop bin.

Your own use of AI may well have made things harder for you:

• Since it is cheap to write tests, you don’t write them like you are betting on your life. But while the cost of writing them has gone down, they still have to be read by someone else, executed in the test suite, and maintained by someone else who has to figure out whether they are just repetition or actually add value.

• An AI has no sense of boredom, so won’t care enough about repetitive edits that should have been turned into neater code (reducing repetition) or manual tasks that should have been an automated process.

• If you delete or modify code, you’ll almost certainly stop and think “hmm, I wonder why this was there in the first place, and written like that?”

To gain empathy with the recipient, you are indeed better off starting from scratch and literally typing everything in, even if you use some of the insights from the AI.

In saying this, don’t think I’m 100% against AI. I have successfully used AI to help debug things that I might never have been able to work out otherwise, and I’ve used agents in very controlled ways where I’ve been able to judge their output and the trade-offs they involve. But, for many of the reasons above, I virtually never use them to write actual code or text if I need another human to read it.

Well, that’s the end. You better believe that I typed every letter myself.