Development standards

This page exists for several purposes:

  • To document some simple best practices for development work (especially web development):

    • first, for the sake of junior web developers, and

    • second, to help those looking to hire a web developer to know what to look for and what to demand from a development company or freelancer.

  • To let potential clients know what to expect from me as a developer creating a new project or maintaining one.

  • To document what I expect from an existing project if a client wants me to take it on.

The last point covers the fact that if I take on an existing code base to do some maintenance work or add new features, I will require all or most of the points below to be addressed before I do any significant amount of work on the project. If anything is missing from this checklist, there will need to be funds for fixing these things first. I've learnt to do this from experience, since anything missing from this list will cause much bigger maintenance problems in the future, and is a huge liability for me.

This usually requires that I be allowed to inspect the existing installation or code before starting work. If that isn't possible, any agreement I come to with a client will be contingent on these things: if I discover that something is missing when I start work, I have the right to cancel any agreement.

If you are looking to hire a web developer for any custom development work (as opposed to picking something off the shelf), you should insist on all these things, even if you don't know what they mean, and even if the developer says it will cost more to add them, because in the long run these things ensure your code is maintainable by other people.

Summary

First, the essential philosophy:

Work under the assumption that you could be passing on the project to someone else very soon – or you could be maintaining it forever.

And then code for the maintainer.

I should be able to pass on any of my projects at a moment's notice, and not hae to explain anything to anyone (because it's already documented), and not need to apologise to anyone (because it's already well-written and maintained). Some companies have the opposite philosophy: they assume that they will always be the guardians of your software, and will make it difficult for other people to take it over. This actually has the result of encouraging every kind of bad practice, which makes life worse for both the developer and the client.

On the other hand, some developers do shoddy work because they have a short term view, and never think they will have to live with the consequences of poor decisions. They put off maintenance work and taking responsibility for the state of the project.

Now, to details: a checklist. It's written as a list of rules to obey, each followed by some explanation, and then possibly by notes indicating if my own policies/practices go further than these rules or are more specific.

Contents

Rules for development practices:

Rules concerning the agreement between developer and client:

Complete source code available

The developer will hand over to the client all of the source code they produce for the clients' needs, under a licence that allows the client to make further modifications.

The licence for the source code produced by the developer may or may not be exclusive (that is, the developer may or may not want the right to re-use the same source code with other clients, or for their own use), but this will be made explicit in the agreement.

Explanation:

Some of these things may seem obvious, but in practice some developers work on the assumption that only they will ever maintain the project, and don't provide full source code to the client, or don't provide it in a form that makes modification easy or legal. This means the client is locked in to that company or developer, giving them a monopoly on future work, which is obviously not good for the client, and in practice isn't good for the developer since it can encourage a poor attitude towards the work.

Often in developing for a specific project, there might be code that is more generically useful, and the developer might want to re-use it later on in a different project. It's important that client and developer both know the score on this front.

My practices:

In some cases it makes sense to release code as an Open Source licence if it is likely to be useful outside the original project, and this is something I would discuss with a client. Work for a client will often involve bug fixes or feature additions to a third party library, and in that case I will always contribute them back to the original library (and might even be required by law to do so).

Also, some of the code I include in a project may have been written for another situation, and its inclusion doesn't mean a client has an exclusive licence to it.

Dependencies documented

Where the source code has external dependencies, the project will contain an explicit and unambiguous list of all direct dependencies and their versions (and preferably indirect dependencies as well).

The licence provided with the custom source code might not extend to third party dependencies. Where it does not, the licences needed for these dependencies will be made clear.

Generic and interchangeable dependencies (such as ‘an operating system’ or ‘a web server’) do not necessarily need to be included in this list.

Explanation:

The legal concerns regarding modification of the custom source code also apply to dependencies that the developer hasn't specifically written but has chosen to use. These can have a similar constraining effect on the client – if the developer writes code that is only compatible with a specific database, for example, the client is locked into that database, and any licensing requirements there might be for that database.

Some companies might make use of internally developed libraries when building a site. If you want someone else to continue development of the site, it then becomes difficult, because the company might be unwilling to give someone else the source code for their internal code, but without it you cannot run the site.

My practices:

For Python code, I usually use a 'requirements.txt' file that can be automatically parsed by the installation tool 'pip'.

For system level components, I prefer to use hosting tools that manage these things for me.

I prefer to use Open Source dependencies as far as possible. This means the software is available for free, under a licence that allows you to run it without paying for it, and allows you to modify the software. By choosing these dependencies, I can pull together very well written software from many other people and fix bugs in it if necessary, and pass it on at no cost.

Dependencies kept up-to-date

Under normal circumstances, system components and other dependencies will be kept updated to a recent stable, secure and supported version.

The client must be prepared to pay for maintenance work due to upgrades to third party components that are required for security or due to obsolescence.

Explanation: software systems are not, unfortunately, static entities that can be expected to run forever without modification. It would be nice if they were, but in reality there are a host of reasons software will need to be continually updated (system updates needed to take advantage of new hardware, or to adjust to changes in network technology e.g. IPv6, or security bugs etc.)

It is not just the operating system – any website will be built using many layers of software, each of which may need to be kept updated. Without updates, continued maintenance becomes difficult – you are working with old versions which might not play well with the new things you want to add. This is called 'bitrot'. But updates themselves are often not just a case of installing the new version – it is possible that subtle problems may arise with the new versions, which need to be fixed.

Some components might be ‘secure’, but not ‘supported’. This means that they have no known security bugs, but if any were found, the providers of the software will not provide a fix for that version, because it is obsolete as far as they are concerned. They will only provide a fix for a more recent version. For example, you might be running Django 1.2.7 – no known vulnerabilities at the time of writing. But it is not ‘supported’ – the 1.2.X branch is not receiving any bug fixes. Running such a version is a big risk: if a vulnerability is found, you can't just upgrade to a fixed version, because other changes might break something in your software. So you need to keep updating the versions of your dependencies, even if the newer versions don't offer you anything you need.

In short, a website is like a car or house that needs to be maintained. Failure to keep up with maintenance can lead to much greater and more expensive problems, just like failure to replace a missing roof slate will cause more expensive problems in the long run due to erosion from the elements.

For simple, out-of-the-box websites, the service provider or developer may be able to cover the cost of all these under a fixed yearly fee. For custom websites, however, it's likely that there will be more significant costs to this.

Version Control System used for all sources

The developer will use a version control system (VCS) to store all source code, using best practices and careful commit logs. The repository and its complete history will be made available to the client, without cost, at any point. The client will pass it on, in its entirety, to future developers.

Exceptions: some configuration data might not be stored in the source code.

Explanation:

A VCS is vital for any software project, to allow the developer to make changes and still have copies of previous versions of the software, to consult or revert to if necessary.

The history of a software project is also often of vital importance in understanding its present state – for example in identifying redundant code. For this to work, every change to the project should appear in the project history as a logically consistent set of changes (‘changeset’) with a clear and correct message (‘commit log’) describing the change. Changesets should be as small as they can be while still making sense, and should not contain multiple unrelated changes. Failure to use VCS, or using it poorly, is simply irresponsible.

Some companies decide to put the source code for a project into a global company repository. This can become painful if you want to extract that project, and only that project, complete with its history, so this practice must be avoided.

Some people may also want to use proprietary tools for source control, or a system that requires a central server running proprietary software. This means that you may need a licence to be able to access the history, which in my opinion is clearly unacceptable, and will very often lead to subsequent developers losing access to the history of the project. In my opinion it is bad practice to use anything other than Open Source VCS tools.

Some configuration information may be of a private and and security critical nature. In some circumstances, it can be better not to store this in the main source code repository. For example, if the code is eventually open-sourced, you not only have to remove the private information from the repository, but rewrite the complete history, which can be very difficult and time-consuming, or alternatively lose the history.

My practices:

I tend to use Git, a very popular and an Open Source DVCS tool, to store projects that I'm in charge of. I'm also happy to work with existing repositories using other Open Source tools (e.g. Mercurial, Subversion, Bazaar).

Practically speaking, some older tools that are not ‘distributed’ in nature — i.e. they require a central server (e.g. CVS and Subversion) – can be very difficult to work with, and I'll either convert the repository to a newer system or use a ‘bridge’ to improve work flow. Conversions, however, must always preserve the history as much as possible.

Automated tests for non-trivial functionality

For all non-trivial custom code that is written, automated tests should accompany the code to ensure it works. Exceptions can be made for code that is hard to test, but in general the project should be written so that as much as possible is testable and tested by automated means.

Explanation:

When working on a project, especially when inheriting someone else's work, a developer needs to know that changes made do not break existing code. Without this, it becomes impossible to upgrade any component, or clean up existing code, leading to maintenance nightmares and security problems.

Manual testing of existing functionality is very error prone, and a waste of the developer's time, and in many cases impossible to do in a reliable manner. Therefore automated tests (i.e. programs that check the program) are needed.

There are various types of automated tests, such as integration tests, unit tests, load tests and static type checking. Depending on circumstances and the type of code, different strategies and levels of testing will be needed.

My practices:

This requirement is harder to pin down that others, since exhaustive testing is literally impossible. However, I won't take on a project that lacks unit tests for mission critical functionality, or I will ask to be paid to write those tests initially. Otherwise, any change made could potentially cause breakage to existing functionality, and I will have to fix it – even if the problem was with the fragility of the existing code.

Initial setup of automated tests can be one of the most time-consuming things, and if there is no working testing system in place, creating the first automated tests can be costly.

Deployment and packaging scripts

Where developers are responsible for deployment, they will create, maintain and pass on scripts that automate the deployment or packaging process so that it can be done simply and easily without error.

Without deployment scripts, developers can easily make mistakes when deploying the code, and future developers especially could make catastrophic mistakes. Much better than documenting deployment steps (and forgetting to update the instructions), a deployment script will ensure that the right thing is done.

If the system includes a database, the deployment script must include a pain-free way of running any necessary database migrations.

Relevant standards and laws followed

Developers will abide by relevant standards and laws as far as possible, and in most cases will refuse to rely on non-standard behaviour of one particular system or contravene the various standards, even if the client asks for it.

Explanation:

There are many implicit specifications that a developer should know and abide by when developing any software. The client also needs to accept the developer's word regarding things that are part of the developer's expertise!

For a web developer, it includes knowing the HTTP protocol and being able to write standards-compliant HTML etc. Ideally it would also cover coding standards, such as PEP8 for Python code etc., to make it easier for other people to take your work on.

There are also laws and legal requirements that govern what software may and may not do. It goes without saying that the developer will abide by these. Sometimes that may include laws of other countries, due to the fact that software or sites published on the internet are essentially international in nature.

My practices:

I take ethics in programming seriously, so, in addition to these things, you will find it hard to persuade me to implement something that I consider to be a dark pattern, even if it is legal.

Documentation included

Any necessary documentation will always be included with the source code, usually in the form of a README as a starting point.

Explanation:

Source code should be self-documenting as much as possible, but there will always be the need for some basic information to get a new developer started, including things like how to run the tests, how to set up a development environment, what deployment tools are being used etc., and anything that a new developer would otherwise have to spend time digging around to find out.

The previous items referred to the practices to be used for development, that I adhere to, and that I require to be present on an existing project before taking it on.

The following items concern the terms of agreement about work to be done and services to be provided:

Payment rate agreed

The developer and client will agree, before works starts, either an hourly rate or a fee for the whole project or piece of work.

My practices:

I much prefer to be paid by the hour, since making estimates for development work is notoriously difficult. However, since this depends to a large degree on trust between developer and client, usually the initial pieces of work will be done on a project basis.

Specifications and requirements provided

The client will explain requirements clearly with sufficient detail for the developer, and the developer will ask for clarification at least once if they need it.

Explanation:

Creating good specifications is very hard, and I've worked with many ‘specs’ that contain a small fraction of the information needed to actually create the system wanted.

For the developer, getting answers to specific questions can be hard, and also time-consuming and frustrating if the client doesn’t respond.

However, I don't think the answer is requiring more complete specs, since they are extremely tedious for the client, and still usually next to useless. More informal ways of working out what the client is after are usually better, in my experience, and better understanding of the business process – which sometimes leads to a complete reworking of what the client initially asks for.

This approach means that the developer is still left to guess at points, and the client mustn't confuse what they had imagined it would be like with what they had communicated it should be like. Otherwise, it becomes extremely time-consuming and expensive for the developer, who has to constantly rework things.

If a client wants to be more picky about the result, they have to be prepared to be more precise with specs. If the client chooses to be vague, the developer must be allowed to make sensible guesses, and not be asked to rework things on their own time.

Hosting and domain registration arrangements clear

If the agreement includes the developer arranging and managing web hosting for the client, the developer will communicate an estimate of yearly costs of domain registration and hosting, and may include some markup in these fees as an administration charge.

The developer may register the domain, if applicable, under their own name, but must pass on control of the domain to the client when asked to do so.

Explanation:

Hosting costs can vary according to circumstance, so it won't be possible to know exactly how much web hosting will cost in advance. The client does need to understand in this situation that there are ongoing costs for their website.

Some companies include a complete web development and hosting solution, and their terms of service usually won't give the client the right to take the site and host it elsewhere. If you are looking for someone to build a site, beware of these kind of deals. A company may provide a very competitive price for this kind of site, but you can often be locked in to their platform and services.

This is a very different type of web development from the type I'm involved in, and it may have its place, but be sure to read the fine print if you go for that kind of service!

I am happy to provide hosting where appropriate, and normally use DigitalOcean for most projects where I have the choice.