All Unkept
Posted in: Haskell, Web development  —  November 07, 2009 at 03:57 PM

Haskell blog software

by Luke Plant

I finally finished the Haskell blog project that I've been doing for a long time! You're looking at it now (unless you are reading this a few months/years after I wrote it, in which case I will probably have again re-implemented my blog software in my new language-du-jour...) [EDIT: I switched to blogofile in June 2012]

The blog software itself is not particularly interesting — fairly standard features, Atom feeds etc. It uses HDBC Sqlite for storage, and HStringTemplate for rendering (a nice library, BTW). For framework stuff, it uses my own Ella library. I didn't find a forms/validation library I could use, and ended up just using a few adhoc bits and pieces. I've used the lovely pandoc to allow reStructuredText both for my own posts and for comments, which is a nice feature IMO.

The main interest for me has been the learning process. You get a much better, rounded understanding of a language from a project like this than you do from the small code samples that people knock around.

The project nearly failed at the last hurdle. Everything was working, but when I uploaded to my server, it failed on some URLs. I realised it was a memory problem — the CGI program must have been killed for using too much memory.

At first, I thought the limits on the server must be unreasonably small. Understanding the output of +RTS -s -RTS is kind of difficult. When I eventually found out that GHC compiled programs never release any memory back to the operating system, I realised that it's the first figure—the total amount of memory allocated in the heap—that was killing me. On the bigger pages, this was over 160 Mb. At that point I stopped complaining to my web host!

By changing to ByteString instead of Data.Text for StringTemplate, and using ByteString in a few other places, I achieved a 4-5 fold reduction in memory usage, along with a significant speed up. Most pages now only use about 10-15 Mb to render, which is OK for a short running process I think. It's not ideal, especially when an additional 1k comment on a page seems to require at least 300k extra memory to render, but it's good enough for now. Profiling further will be very hard, as I suspect it will mainly be to do with the guts of HStringTemplate.

I'll be blogging about the experience of developing this over the next few days/weeks, and what I've learnt. It's certainly been enjoyable overall, although it's definitely had its pain points too!

I've put redirection in for all the old, crufty URLs, so there shouldn't be any broken links. Feed readers will likely be confused, sorry!

If you have problems getting through my spam protection, please let me know. It enforces a 10 second wait before it accepts submissions, which serves to prevent thoughtless comments as well as spam :-)

Comments §

blog comments powered by Disqus