All Unkept
Posted in: Haskell, Web development  —  July 14, 2008 at 09:38 PM

Haskell Blog Rewrite - Session 7

by Luke Plant

I am really stubborn, so I never completely gave up on my little pet project of re-writing my blog as a Haskell CGI app. I also didn't give up on using my budget host. (Partly because I have even less money than before). So, the saga continues.

Due to the following facts:

  • GHC compiles to native code
  • My dev box is quite different from my web site host box
  • I have no shell account on my host

...the binaries I produce on my dev box won't run on my host( /lib/tls/libc.so.6: version 'GLIBC_2.4' not found etc, and worse if I try to compile static binaries).

So I had to set up a VM for creating binaries. This took quite some doing, and below are the gory details. (In fact I have spared you quite a lot, this took days, but would take much less now. It's easy when you know how...). Conclusions at the bottom.

  • Installed CentOS 4.5, the OS used on my host, as a VirtualBox VM

    I did this ages ago, so I can't remember how long this took, or how I did it, should be fairly straightforward.

  • Got host networking working - this took hours of messing around, but eventually these got me there:

    I should probably just have used VMWare Player, it works well enough.

  • On the CentOS machine:

    • Turned off the firewall

    • Created a user for building, added it (him?) to 'wheel' group for sudo

    • Set up ~/.ssh/authorized_keys (remember chmod go-rwx), .inputrc, .bashrc etc so I can log in using ssh easily and everything is reasonably comfortable.

    • Update system:

      sudo yum update
      
    • I found that I had to install a custom kernel to sort out VirtualBox CPU usage (thankfully I didn't have to build this myself, other people have had the same problem):

      cd /etc/yum.repos.d
      sudo wget http://vmware.xaox.net/centos/4/VMware.repo
      # Can't figure out how to install specific version
      with yum, but this did the right thing:
      sudo yum install kernel
      
    • Installed mercurial (which I'm using for source control, and as an easy means of getting stuff onto the build box):

      wget http://i18n-zh.googlecode.com/files/mercurial-0.9.4-1.i686.CentOS-4.5.rpm
      sudo rpm -i mercurial-0.9.4-1.i686.CentOS-4.5.rpm
      
    • While doing 'hg serve' on host, pull onto build vm:

      hg clone http://10.0.0.13:8000/
      mv 10.0.0.13\:8000/ repos
      cd repos/
      
    • Later, I decided that rsync was better, since it didn't require checking really experimental stuff into the repos. It does mean I have to remember not to edit on the build box, since it is no longer a branch.:

      # on dev box
      rsync -vr --exclude='*~' --exclude='*.o' --exclude='*.hi' haskellblog build@10.0.0.14:/home/build/build/
      
    • About to run my build script on the build script — oh yeah, need to install GHC :-)

    • Installed GHC - attempt 1:

      wget http://www.haskell.org/ghc/dist/6.8.3/ghc-6.8.3-i386-unknown-linux.tar.bz2
      # untar etc
      # install:
      ./configure --prefix=/home/build/local
      checking build system type... i686-pc-linux-gnu
      checking host system type... i686-pc-linux-gnu
      checking target system type... i686-pc-linux-gnu
      Which we'll further canonicalise into: i386-unknown-linux
      checking for path to top of build tree... configure:
      error: cannot determine current directory
      

      Not very clever. It appears CentOS 4.5 is too ancient for the binaries that come with the GHC 6.8.3 installer, here's the bug: http://hackage.haskell.org/trac/ghc/ticket/2211

      So, try another approach - I found some old GHC 6.4 binary packages for CentOS 4.5 on the web, used them to bootstrap GHC 6.8:

      wget http://centos.karan.org/el4/extras/stable/i386/RPMS/ghc-6.4.1-1.el4.kb.i386.rpm
      wget http://centos.karan.org/el4/extras/stable/i386/RPMS/ghc641-6.4.1-1.el4.kb.i386.rpm
      sudo rpm -i ghc-6.4.1-1.el4.kb.i386.rpm ghc641-6.4.1-1.el4.kb.i386.rpm
      wget http://www.haskell.org/ghc/dist/6.8.3/ghc-6.8.3-src.tar.bz2
      wget http://www.haskell.org/ghc/dist/6.8.3/ghc-6.8.3-src-extralibs.tar.bz2
      tar -xjf ghc-6.8.3-src.tar.bz2
      tar -xjf ghc-6.8.3-src-extralibs.tar.bz2
      cd ghc-6.8.3
      

      Then followed the build instructions in README, with './configure --prefix=/home/build/local' so I wouldn't mess up my VM. Time to go off and do a work out...45 minutes later, it's still building...another hour...right, I'll be leaving my machine on tonight then...

      In the morning...bummer, out of disk space! The GHC build directories are using over 1 Gb of space, I didn't anticipate that, my 4 Gb VM disk wasn't large enough.

      So, time to resize the VirtualBox disks. Except that gets a bit complicated, since CentOS uses LVM, and I have no idea what I'm doing. But, hang on, LVM should make life much easier actually, and remove the need to actually resize the hard disk -- I can just add another. Once I figure out how to use it (took a while). So I:

      • Created a new 20 GB (dynamic) hard disk with the VirtualBox GUI

      • Created a single 'Linux LVM' partition in it using cfdisk (by booting from a Damn Small Linux CD ISO file, CentOS doesn't have cfdisk for some reason, I probably could have used sfdisk or something, but I'm familiar with cfdisk)

      • In CentOS, run lvm, then:

        lvm> pvcreate /dev/hdb1
        lvm> vgextend VolGroup00 /dev/hdb1
        lvm> lvextend -L+20GB /dev/VolGroup00/LogVol00
        

        (Except I didn't do just that, and in my experimentation I managed to make my VM unbootable by deleting a hard disk that I had added in the manner above. Cue another hour working out what to do, ending (happily) with booting the CentOS install DVD in 'rescue' mode, running lvm and vgreduce --removemissing VolGroup00, hooray, my VM boots again!)

      Then, having extended the logical volume that holds the root filesystem, you've got to resize the filesystem, too. But you can't, since it's mounted. (I thought LVM was supposed to make this stuff easier? This sucks...) So, back to the CentOS rescue CD. It finds the installed system, and mounts it all, so you then have to do:

      umount /mnt/sysimage/proc
      umount /mnt/sysimage/sys
      umount /mnt/sysimage/dev
      umount /mnt/sysimage/boot
      umount /mnt/sysimage/selinux
      umount /mnt/sysimage/
      e2fsck -f /dev/VolGroup00/LogVol00
      ## make ext3 into ext2, not really necessary, according to what I read later...
      tune2fs -O ^has_journal /dev/VolGroup00/LogVol00
      resize2fs /dev/VolGroup00/LogVol00
      ## re-enable the journal
      tune2fs -O has_journal /dev/VolGroup00/LogVol00
      

      Check:

      mount /dev/VolGroup00/LogVol00 /mnt/sysimage
      df -h
      ...
      /dev/VolGroup00/LogVol00 23G 3.2G 19G 15% /mnt/sysimage
      

      19GB free - woot! That ended up being much harder than anticipated, but at least I know about LVM now. Well, I know that it works, but it's not that much fun, and when I get a new machine and make my current one into a backup server, it will be an OpenSolaris box with all the ZFS goodness, and not a Linux box.

    • Soooo...try again to build GHC, hopefully I can just do 'make && make install' again and it will carry on and work...takes another hour...

  • Installed the Haskell libraries needed for my web app. I installed all the extra libs with GHC, so I didn't need that much more. The easiest way is to first get cabal-install working. URLs below are from HackageDB: http://hackage.haskell.org/packages/archive/pkg-list.html

    • First, a wrapper script to make things easier, because I'm installing to ~/local:

      $ cat > ~/bin/install_haskell_library
      #!/bin/bash
      
      if [ -f "Setup.lhs" ]
      then
              SETUP="Setup.lhs";
      elif [ -f "Setup.hs" ]
      then
              SETUP="Setup.hs"
      else
              echo "No Setup.hs or Setup.lhs
              found!" > /dev/stderr
              exit 1
      fi
      
      echo runhaskell configure $SETUP --prefix=$HOME/local
      --user
      runhaskell $SETUP configure --prefix=$HOME/local
      --user || exit 1
      echo runhaskell $SETUP build
      runhaskell $SETUP build || exit 1
      echo runhaskell $SETUP install --user
      runhaskell $SETUP install --user || exit 1
      

      Then:

      chmod +x ~/bin/install_haskell_library
      
    • Now, the libraries, in the order needed. In each case (apart from 'Cabal', read the README):

      • tar -xzf <the_package>
      • cd <the_package_directory>
      • install_haskell_library
      • cd ..
    • By the time you read this, probably all the version numbers need updating, but this is what I did, re-ordered so that it will work first time:

      wget http://hackage.haskell.org/packages/archive/HTTP/3001.0.4/HTTP-3001.0.4.tar.gz
      wget http://hackage.haskell.org/packages/archive/zlib/0.4.0.4/zlib-0.4.0.4.tar.gz
      wget http://hackage.haskell.org/packages/archive/Cabal/1.4.0.1/Cabal-1.4.0.1.tar.gz
      wget http://hackage.haskell.org/packages/archive/cabal-install/0.5.1/cabal-install-0.5.1.tar.gz
      
    • Now I can use cabal-install:

      cabal update
      cabal install --prefix=$HOME/local --user HDBC HDBC-sqlite3
      

      Oh, I need this for the last step to work:

      sudo yum install sqlite-devel
      
  • Finally, everything is setup, I run my build script for my web app, generate the binaries, upload them to my host. The moment of truth: can I create and access and SQLite database with my binaries? I go to: http://lukeplant.me.uk/cgi-bin/installdb.cgi

    Records inserted in table Test are:
    *************************************
    [(1,"Test1"),(2,"Test2"),(3,"Test3"),(4,"Test4")]
    *************************************
    
  • Hallelujah!!111!!!!

Conclusions

  • Set up for developing a Haskell CGI web app is hard. (Do I get a prize for understatement of the year? Do I? Oh, thank you!)
  • In general, you will need a VM for building binaries.
    • Even if you are developing on a Linux box, web hosts tend to use pretty old and stable software, so you might have binary incompatibilities. This is only going to get worse with 64bit etc.
    • Even if your web site host gives you shell access, they probably won't have a recent enough GHC installed, and building it takes 1.5 GB of disk space, (and lots of CPU and memory for quite a few hours), which you probably don't want to do on your hosting box. (NB: actually building GHC is usually not required, installing the binary should work, but it might not, as in my case.)
  • Make sure your VM has more than enough space.
  • VirtualBox sucks for setting up host networking, VMPlayer is much easier.
  • You have to be really stubborn. Hopefully, some idea of just how much effort it might take, and some of the pitfalls to avoid, will help you.
  • Make sure you factor in the time to do all this if you have any kind of deadline!

Comments §

blog comments powered by Disqus