All Unkept

Controlling Sozi remotely from an Android phone

2013-03-11T15:37:57Z

I'm due to give a talk next Sunday evening, and I've created a presentation using the very cool Sozi. Sozi is quite like Prezi, but open source and free, and its output is a standard SVG file (with a bit of Javascript), so it can be displayed in several web browsers.

(On the other hand, for many people Sozi is probably a lot harder to use for simple presentations than Prezi, as it works as a plugin to Inkscape, a general purpose drawing program, but, to me, it is simply unacceptable to depend on the continued existence of a company, and a subscription fee, to be able to continue to view/edit your presentations).

One problem with not using ‘standard’ software like Powerpoint is that proprietary tools like hand-held remote controls for Powerpoint won't work, or won't work easily (and I don't want to find out that it won't work when I turn up). But I've found a solution that works for me:

On my Android phone, I installed Coversal, which is an app designed for controlling media players, but allows custom commands to be executed, so actually works fine for presentations.
Within Coversal, I installed the ‘SSH Custom’ plugin.
On my Linux laptop, I installed openssh-server to allow the app to log in to my machine. To avoid putting my laptop password on the phone, I created an SSH keypair for this app, transferred to private key to the phone, and added the public key to ~/.ssh/authorized_keys.

I created a script on my laptop that can control Sozi running in Chrome:

#!/usr/bin/env python

# controlsozi

import os
import subprocess
import sys
import logging

from datetime import datetime

logging.basicConfig(level=logging.DEBUG, filename='/tmp/controlsozi.log', filemode="a")

def get_window():
    # Assumes that there is just a single Google Chrome window open, and this
    # has the presentation in it.
    windows = subprocess.check_output("DISPLAY=:0 wmctrl -l -x | egrep 'google-chrome|gnome-www-browser'", shell=True)
    return windows.strip().split()[0]

def send_text(text):
    os.system("DISPLAY=:0 xvkbd -window %s -text '%s'" % (get_window(), text))

ACTIONS = {
    'previous':      "\[Left]",
    'next':          "\[Right]",
    'fullscreen':    "\[F11]",
    'previous_fast': "\[Up]",
    'next_fast':     "\[Down]",
    'start':         "\[Home]",
    'end':           "\[End]",
    'launch':        None,
}

def log_uncaught_exceptions(*exc_info):
    logging.critical('Unhandled exception:', exc_info=exc_info)
sys.excepthook = log_uncaught_exceptions


if __name__ == '__main__':
    logging.debug("Starting...")
    if len(sys.argv) < 2:
        print "Usage: "
        for k in ACTIONS.keys():
            print "  " + k
    else:
        action = sys.argv[1]
        logging.debug("Action = %s" % action)
        if action == "launch":
            os.system("DISPLAY=:0 nohup google-chrome '%s' &" % sys.argv[2])
        else:
            text = ACTIONS[action]
            logging.debug("Text = %s" % text)
            send_text(text)

    logging.debug("Done")

This relies on command-line tools wmctrl and xvkbd. With only small modifications it should work for Firefox too. It is not 100% reliable, as it works by sending keypresses to windows, but it works fine for me.

I set up a profile in Coversal to connect to my laptop and use this script.

Coversal allows you to define the commands for certain actions — the list of actions includes things like up, right, next etc., but you can add your own. You then assign actions to buttons, as desired.

So I simply I had to set commands like "/path/to/controlsozi next", and then assign these actions to the buttons I wanted to use.

By setting the start_playback command to "/path/to/controlsozi launch '%s'", I could even use the file browser built in to Coversal to select the presentation file to view, which is pretty neat.

The one thing I was really missing was physical buttons — with a touch screen, it's so easy to hit the wrong place, and I wanted something to rest my finger or thumb on.

I then realised that my phone does have a couple of physical buttons - the volume control buttons on the side. By enabling the 'volume_up' and 'volume_down' buttons in Coversal and mapping them to the 'next' and 'previous' commands, the physical volume buttons get mapped to the same actions, and they do the job just fine. I can use the rest of the controls on the screen for the less common tasks.

For my phone to be able to connect to my laptop over SSH, there is one more piece of configuration that must be done — the IP address of the laptop has to be entered. If you have a local wifi, this is no problem — just connect the two devices to the same wifi and they will be able to talk. My phone has a more reliable option, though — it can itself operate as a wifi hotspot, creating a wifi network that the laptop can connect to.

Conclusions:

Linux is awesome.
Smart phones are awesome.
Free Android apps are awesome (and I think it is the Linux culture which encourages this).
Python is awesome.
Being a programmer and being able to plug all this stuff together is awesome.

KTimeTracker replacement - TimeCult

2012-02-24T17:57:32Z

For a long time I haven't been able to find a decent replacement or alternative to KTimeTracker. But I've now succeeded: TimeCult.

It is currently lacking an installer for Linux, but it can be made to work with Linux easily.

I also found that it uses a very easy-to-deciper XML format, and, with the help of vobject, the KTimeTracker ical files are only slightly harder to figure out, so I created a Python script that will do the conversion.

Use the script like this:

./ics_to_timecult.py ~/.kde/share/apps/ktimetracker/ktimetracker.ics > out.tmt

It's here for reference - dependencies are vobject and elementtree:

#!/usr/bin/env python
# Quick and dirty script to convert ktimetracker.ics files
# into TimeCult files.

import os.path
import sys
import time
import uuid

import elementtree.ElementTree as ET
import vobject


# Structures to store the tree of data. Use native Python data types, and
# convert to what TimeCult expects when we 'render'

class Struct(object):
    def __init__(self, **kwargs):
        self.__dict__.update(kwargs)


class TimeCult(Struct):
    # 
    # Eventual attributes:
    #  name
    #  projectTree
    #  timeLog

    def toXML(self):
        root = ET.XML('')
        root.attrib['uuid'] = str(uuid.uuid1())
        root.attrib['name'] = self.name
        pt = ET.Element('projectTree')
        for n in self.projectTree:
            pt.append(n.toXML())
        root.append(pt)
        tl = ET.Element('timeLog')
        for tr in self.timeLog:
            tl.append(tr.toXML())
        root.append(tl)
        return root

class Node(Struct):
    #  and 
    # Eventually will have these attributes:
    #  vtodo
    #  summary
    #  uid
    #  created
    #  parent
    #  children
    #  type = "project|task"
    #  finished
    def __repr__(self):
        return "%s summary=%r, parent=%r>" % (
            self.uid,
            self.summary,
            self.parent.summary if self.parent is not None else None)

    def toXML(self):
        e = ET.Element(self.type)
        e.attrib['id'] = self.id
        e.attrib['name'] = self.summary
        e.attrib['created'] = str(int(time.mktime(self.created.timetuple()) * 1000))
        if (self.type == "task"):
            assert len(self.children) == 0
            e.attrib['status'] = 'finished' if self.finished else 'inProgress'
        else:
            for c in self.children:
                e.append(c.toXML())
        return e

class TimeRec(Struct):
    # 
    # Eventual attributes:
    #  startTime  # datetime
    #  duration   # seconds
    #  task       # Node
    def toXML(self):
        e = ET.Element("timeRec")
        e.attrib['taskId'] = self.taskId
        e.attrib['startTime'] = str(int(time.mktime(self.startTime.timetuple()) * 1000))
        e.attrib['duration'] = str(int(self.duration * 1000))
        e.attrib['notes'] = ""
        return e

def convert(filename):
    f = vobject.readOne(file(filename).read())

    # First parse Todos
    nodes = {}
    for vtodo in f.vtodo_list:
        n = Node(vtodo = vtodo,
                 uid=vtodo.uid.value,
                 summary=vtodo.summary.value,
                 created=vtodo.created.value,
                 finished=vtodo.contents['percent-complete'][0].value == "100"
                 )
        nodes[n.uid] = n

    children = {}
    for uid, node in nodes.items():
        vtodo = node.vtodo
        try:
            related_uid = vtodo.contents['related-to'][0].value
            node.parent = nodes[related_uid]
            children.setdefault(related_uid, []).append(node)
        except KeyError:
            node.parent = None

    for uid, node in nodes.items():
        node.children = children.get(uid, [])
        if len(node.children) == 0:
            node.type = "task"
        else:
            node.type = "project"



    autouid = 0 # For newly created Tasks

    # Now find the time information in the ICS file
    time_log = []
    for vevent in f.vevent_list:
        related_uid = vevent.contents['related-to'][0].value
        task = nodes[related_uid]
        if task.type == "project":
            # TimeCult can only cope with time events being against 'leaf' nodes
            # in the tree. So we need to adjust and create an additional leaf
            project = task
            undefined_l = [t for t in project.children if t.uid.startswith('autouid')]
            if len(undefined_l) > 0:
                task = undefined_l[0]
            else:
                task = Node(uid='autouid-%d' % autouid,
                            summary='other',
                            parent=project,
                            type="task",
                            created=project.created,
                            finished=False,
                            children=[])
                autouid += 1
                nodes[task.uid] = task
                project.children.insert(0, task)

        dtstart = vevent.dtstart.value
        dtend = vevent.dtend.value
        if dtstart.tzinfo is None:
            dtstart = dtstart.replace(tzinfo=dtend.tzinfo)
        if dtend.tzinfo is None:
            dtend = dtend.replace(tzinfo=dtstart.tzinfo)
        time_rec = TimeRec(startTime=dtstart,
                           duration=(dtend - dtstart).total_seconds(),
                           task=task)
        time_log.append(time_rec)

    # Now need to assign IDs
    taskId = 0
    for u, n in nodes.items():
        taskId += 1
        n.id = str(taskId)

    for tr in time_log:
        tr.taskId = tr.task.id

    return TimeCult(
        name=os.path.basename(filename),
        projectTree=[n for u, n in nodes.items() if n.parent is None],
        timeLog=time_log)


if __name__ == '__main__':
    filename = sys.argv[1]
    timecult = convert(filename)
    sys.stdout.write(
        """""" +
        ET.tostring(timecult.toXML()))

My Bash prompt

2011-02-28T23:32:42Z

I like my bash prompt:

to include the current Mercurial/Git branch if applicable — otherwise I would quickly get in big trouble with committing things to the wrong branch in Django etc.
to show the full current working directory - for similar reasons to above, as I often end up lost in a maze of twisty little directories, all alike.
to be always instantaneous (tricky, when 'hg branch' can take up to a second on a bad day due to startup time)
to work well with virtualenv (which adds the environment name to the beginning of the prompt)
to show the time (so that if I have a long running job that I didn't realise was going to be long running, all the info is there to find out how long it took).
to do all the above without confusing me!

All of these together mean I need two lines, but this has advantages - it means the commands I type always start at the same column. I achieve (1) by a bash function that looks for .hg or .git recursively, and looks for '.hg/branch' where applicable. I achieve (6) by using some colours distinguish the different bits.

The end product looks like this - first without a virtualenv, then with:

The code looks like this (in ~/.bashrc)

# Prints " $branchname" if in a hg or git repo, otherwise nothing.
print_branch_name() {
    if [ -z "$1" ]
    then
        curdir=`pwd`
    else
        curdir=$1
    fi
    if [ -d "$curdir/.hg" ]
    then
        echo -n " "
        if [ -f  "$curdir/.hg/branch" ]
        then
            cat "$curdir/.hg/branch"
        else
            echo "default"
        fi
        return 0
    elif [ -d "$curdir/.git" ]
    then
        echo -n " "
        git branch --no-color 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/\1/'
    fi
    # Recurse upwards
    if [ "$curdir" == '/' ]
    then
        return 1
    else
        print_branch_name `dirname "$curdir"`
    fi
}

e=\\\033
export PS1="\[$e[1;36m\][\u@\h \t]\[$e[1;33m\]\$(print_branch_name) \[$e[0m\]\w\n\[$e[1;37m\]——> \[$e[0m\]"

player_do — control multiple media players from a single (command line) interface

2010-07-15T16:23:05Z

I tend to use several music/media players, such as shell-fm, moc and Clementine. And I often try out new ones. In order to be able to use the media buttons on my keyboard for pausing/skipping, I wrote this player_do Python script to automatically route the commands to the right one.

Quick start:

sudo easy_install playerdo

Or, if you prefer this way like me, and have $HOME/.local/bin on your $PATH:

easy_install --prefix=$HOME/.local playerdo
player_do help

Then use your window manager/desktop to setup keyboard shortcuts to player_do playpause, player_do next etc.

mpris-remote does a similar job, but only for MPRIS players, and I needed others. My script also has an MPRIS backend, so it will support XMMS2, Amarok, Clementine, Exaile, Dragon Player and various others out of the box. It also has support for shell-fm, moc, and rhythmbox.

If you have more than one running at a time, it tries to be intelligent about which one to talk to, although of course it is not always possible to read your mind correctly!

For other players that don't support MPRIS it should be quite easy to add a backend — please fork the bitbucket repository, have a look at the existing backends and send me your patches!

Leaving KDE

2010-06-17T01:57:39Z

After being a loyal fan of KDE for a long time, I've just switched to GNOME.

I'm afraid to say that since the KDE 4 series, I've been unable to recommend KDE to anyone. With KDE 4.0 to 4.3, that was due to general crashiness, brokenness and bugs that are at the level of embarrassing — when a bug is obvious enough and significant enough that a new user would be shocked to find it and I would have a hard time explaining how I could use software that is clearly inferior to whatever they were using before (usually Windows).

With KDE 4.4, most desktop instability has been ironed out, but it has been replaced by problems that mean I can no longer stomach or risk staying with KDE for myself, let alone recommend it to anyone else. The final straw was that the framework for storing PIM data (Akondai) is turning out no better than I feared and is ‘losing’ some of my data.

I found that some things added to the address book ‘never’ actually end up in the file they are supposed to be stored in. I've added an e-mail address for a contact, and it appears to have added fine in KAddressBook. The data persists even across reboots etc., so it isn't ‘lost’ in one sense. Usually, the data appears in my std.vcf after a few seconds. In this case, however, four days later it still hasn't appeared in std.vcf. That counts as close enough to ‘never’ in my book, so it is fair to say it is ‘lost’. This has happened intermittently since KDE 4.4.1.

One additional result of this is that KMail never sees that address — at the moment it seems that it looks only in std.vcf, not Akonadi (and this is probably the only reason I found the bug in the first place). That's a pain, but the more serious problem is that my data is just not where it is supposed to be. If I need to backup and restore, I imagine it would be possible for that to succeed, if the Akonadi data was backed-up and restored along with the underlying data. But if you get that bit wrong, I can only imagine a world of pain and subtle data corruption. It is completely unacceptable that my data is being stored in “my data files + completely opaque Akonadi database” rather than just “my data files”.

Another bug is that the Google contacts Akonadi plugin also loses data in a serious way, making it far worse than useless. The nature of this bug makes me think that either there is a significant bug in the Akonadi framework, or that the framework API made it is easy to get this wrong — both of which are worrying.

These bugs are on top my general concerns with the framework — it has always been painful to work with, and the fact that it is even visible is a major failing — the software layer it replaced was, after all, completely invisible, just as it should have been.

From what I can tell, migration of KMail to Akonadi is going ahead at full pace and will be mandatory soon. My contact data is modest (500 Kb), so I have been able to cope with moving it around and fixing some problems, including some data loss and mistakes caused, I think, by the above bugs, since backups are quick. But my e-mail data is 2.5 Gb, and detecting problems or errors is going to be massively harder. I'm not willing to risk my e-mail data with Akonadi, so I need a new e-mail client.

It turns out that Evolution works for me. It stores data in mbox files, which isn't my preferred (1 file per e-mail, like maildir or MH, is nicer for various purposes), but it is a standard format, and since my backup tool is 'diff' based it will still cope efficiently with it. It even integrates with Google contacts out of the box, and the calendar app integrates with Google calendar, while allowing me to work offline. This is what Akonadi is supposed to offer one day, and I wish them the best of luck with that. But for now I'm going to stick with a system that prioritises the safety of my data, and that I know works. Evolution has a few significant niggles, but I think I can live with them.

That leaves me with the choice of desktop. Using programs from one desktop suite generally is faster and easier and looks nicer. I had already dropped Konqueror for Chrome (Konqueror has fallen far too far behind these days), and GNOME equivalents to most KDE apps work well enough — like Empathy/Pidgin. That tips the balance over to GNOME — and I can still use apps like k3b for specialist work when I need them.

Most of my reasons for leaving GNOME in the first place (about 7 years ago I think) have been resolved. GNOME apps like Nautilus and Evolution are no longer as excuciatingly slow and bloated as they used to be (and I don't think it is just that my computer got faster), and actually feel quite snappy most of the time. In some cases they definitely outperform the equivalent KDE default app (e.g. Eye of GNOME is very fast and works very nicely, doing 90% of what I used Gwenview for before, but much faster). Generally GNOME is much snappier as a desktop — it takes only about 5 seconds to start after I've logged in, which is much faster than KDE.

So I'm using Ubuntu's GNOME desktop, and I no longer have plasma and all the plasmoids. To be honest, I don't think I'm missing them. I have all the panel applets I need, and they are well designed and work properly. Proliferation of plasmoids is like the proliferation of apps on Apple app store — most of them are not worth having, and having to search through to find the ones that are is an anti-feature (not to mention that the new horizontal applet picker in KDE 4.4, which replaced the vertical one in KDE 4.3, is just awful — I don't understand how such a major regression happened).

I certainly won't shed a tear over Nepomuk/Strigi. After weeks of it constant disk activity as they attempted to index my files, they never seemed to do anything useful. They have been replaced by Tracker, which works very well, and very quickly, and without being a huge resource hog. It does not update immediately when I add new files, but it is fast enough to know that when I really want to find something in a real situation, it is likely to be in the index already. And it isn't the massive resource hog that Nepomuk was.

First experience with Ubuntu's GNOME has thrown up some problems. Here are some niggles I've been able to fix/solve:

PulseAudio - this breaks lots of things, especially games, and provides no added benefit. But it can be removed. Perhaps the Ubuntu guys will remove it in future, or make it easier to remove. Lots of people and programs seem to have problems with it, it's very difficult to see what value it is adding for most people.
To turn off desktop effects quickly (for better performance games) - either: install fusion-icon (use "fusion-icon -n" as the command) - or, to do it automatically for fullscreen games, set this option to true: /apps/compiz/general/screen0/options/unredirect_fullscreen_windows
To turn off screen locking on resume - there is setting is in gconf, search for 'lock'.

And here are some that are still bothering me a bit:

Reading new e-mails with keyboard only - it's a bit buggy/confusing with Evolution.
Some options are hidden away in gconf, so it's difficult to know if it is possible to change things.
GNOME has less configurability in general, or harder to get to it, especially for creating new keyboard shortcuts.
Compiz doesn't always seem to obey or understand all metacity options (e.g. shortcut for fullscreen-ing windows), so it sometimes confusing working out what is going on.
The GNOME file dialog boxes have improved a little bit since I was last using GNOME, but haven't begun to close the gap on KDE's far superior ones.

But there are some niggles which are no longer there due to the switch from KDE. Overall, things are similar at this level.

To conclude, I think the KDE developers were far too ambitious with the KDE 4 series. The cost has been that the only supported versions of KDE are ones that you cannot recommend to anyone, and that is a critical mistake in a competitive environment. Meanwhile, GNOME has massively improved in speed and has incrementally added features that people actually need, instead of creating large frameworks that may or may not be robust and useful one day.

Linux and calendars - bad combination...

2010-03-23T13:22:49Z

So, I'd like to be able to access my sync the calendar on my computer with other devices, like my BlackBerry Curve. This is 2010 right?

I already blogged the failure to sync directly. Let's not go there.
How about just syncing KOrganizer with Google, and syncing the BlackBerry with Google? I already have some calendars on Google, and that works fine. But now for my main calender:
- I tried GCalDaemon, but found that it ignores/deletes all recurring events on your calendar. Which is pretty evil.
- There is Akonadi... Given how hopelessly buggy Akonadi is for syncing contacts to Google (like, if Google has more than one e-mail for a contact, none of them are transferred, and worse), I'm not very hopeful about this (and I'm using most recent KDE 4.4.1). But I'll try anyway...
  - The whole experience is fraught with bugs and confusion. You click 'Add' and things aren't added, you click 'Remove' and they don't disappear, or only after some indefinite period of time (I've waited 30 seconds, is it just broken or should I carry on waiting?). You close the Akonadi configuration and open it again, or restart the server, and items have appeared, and other things that you didn't ask for. A lot seems to go on asynchronously, which might be a good idea, but you get the impression you are attempting to communicate with a forgetful, deaf, sleepy, indecisive, incompetent idiot. I have eventually managed to add a Google calendar resource, but it says "Failed getting last updated event", and nothing appears on my calendar.
OK, let's try Evolution calendar. At first it didn't work, then it seemed to work a bit better, but trying to modify events returns "Unknown error", and now it refuses to show the calendar at all.
OK, let's just try using Google calendar, importing all my existing calendar data, ditching any hope of using a local calendar app. But Google doesn't like my calendar file - "Failed to import events: Processing error during upload."

Epic fail. Bad day.

Update: in the last case, I discovered after some investigation that Google doesn't like the 'VTIMEZONE' sections in my calendar file. Removing them allows the import to succeed (although it still produces an error message, and I'm guessing it might have lost some time information on summer events though, not sure).

Blackberry Curve ↔ KDE-PIM sync on Kubuntu Karmic. FAIL

2010-02-11T17:07:58Z

For the sake of anyone else who might want to know about this in advance, I'll blog my failure.

I tried with Kubuntu Karmic, and the following steps:

Install the necessary packages:

sudo aptitude install opensync-plugin-kdepim opensyncutils barrybackup-gui barry-util opensync-plugin-barry

Plug in the Blackberry, and choose "Mass storage" mode on the Blackberry.
Set up permissions to allow a non-root user to access the device. I added the file /etc/udev/rules.d/040-devices.rules, with the following contents:
```
BUS=="usb", SYSFS{idVendor}=="0fca", SYSFS{idProduct}=="8004", GROUP="admin"
```
The above line gives permissions to all users of group 'admin', which includes my normal user. The numbers "0fca" and "8004" are device numbers obtainable from lsusb:
```
lsusb | grep "Research In Motion"
```
Result:
```
Bus 002 Device 008: ID 0fca:8004 Research In Motion, Ltd.
```
You will need to unplug the Blackberry and plug back in after this step.
Check you can access the Blackberry.
```
btool -t
```
This should list a series of databases on the Blackberry
Back up the current contents:
```
barrybackup
```
Note the pin for later, and follow the prompts to backup the Blackberry data.
Back up your KDE-PIM data, in case of any corruption, duplicate elements etc. (It happened to me!).
Create a msynctool configuration for doing the synchronisation.
```
msynctool --addgroup Blackberry
msynctool --addmember Blackberry barry-sync
msynctool --addmember Blackberry kdepim-sync
msynctool --showgroup Blackberry
```
The final command should show the first member as missing configuration. Do the following
```
msynctool --configure Blackberry 1
```
and insert your pin in the relevant place.
Moment of truth. Close KOrganizer and KMail, then:
```
msynctool --sync Blackberry
```

I came across the following major bugs:

Syncing never completes, and I have to press Ctrl-C to abort.
When I try to sync a second/third time, I end up with lots of duplicates on my computer, caused by items from the Blackberry being copied back. I'm not sure if they are to do with the unclean abort, or the following symptoms: * 'All day events'. A duplicate appears with the start time set to '0:00' * Daylight savings problems - for summer events, a duplicate appears 1 hour different from the original * Items from my Google calendar (which is a read only calendar) got copied back to my main calendar.
Each time I attempt to sync, duplicate events and contacts build up on the Blackberry as well.
Using barrybackup to restore to a clean state with the backup made previously fails with lots of errors. By removing some items in ~/.barry/backup//config, I've got it to apparently succeed, but the contacts and calendar on the Blackberry are not cleared out, which is the point of the exercise. * There is a way to clear calendar/contacts on the Blackberry. Go to Calendar -> Menu -> Options. Type RSET. The calendar will be cleared (though it says nothing). Repeat for contacts. This time you will get a prompt about clearing. Clearing out the msynctool database is also a good idea to keep further attempts clean and fast.

In the end, a one-time, one-way sync is all that I have been able to achieve. That is quite disappointing, since none of the web pages I found before deciding on a Blackberry mentioned any problems like these.

I tried upgrading all the barry packages from 0.14 to 0.16, but got lots more errors - when trying to send data to the Blackberry I just got "Timeout in usb_bulk_read" blah blah and do data was actually sent.

Sources:

Python and copyright

2009-07-15T22:04:42Z

There has been lots of interesting discussion about dynamic languages and the GPL, but much of it seems to miss the main point: with many dynamic languages, the terms of the GPL are an irrelevance.

In the comments on the above page, Peter Gerdes seems to be one of the few who does get this:

@Morty Yidel

It doesn't matter what the GPL says in situations 2 & 3. What matters here is copyright law.

The GPL is a license that lets you create and distribute copies of certain software. You are only bound by the terms of that license when you need the license as a matter of copyright law.

In 2 & 3 you aren't distributing any GPLed software thus you need only abide by the GPL if it is a derivitive work of a GPLed program. As I said that's a complex question but it's one that can only be answered by looking at copyright law and precedent not the GPL. Moreover, if merely using an API makes your code a derivitive work of the library servicing the API it's hard to see how programs like WINE aren't infringing.

If someone wants to apply the terms of the GPL, they first have to demonstrate something that would otherwise be copyright infringement. As the GPL (v2) says:

5. You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Program or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Program (or any work based on the Program), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Program or works based on it.

So, if I sit down at my computer and type (for the sake of argument):

echo 'import qt' > test.py

and then distribute test.py, have I created a derivative work of Qt (or any other GPL library)? Since I could type all of that without having a drop of Qt source code on my computer, I'd argue it's pretty difficult to say that this is a derivative work. Of course, once I go further and actually start using more the API than just the name 'qt', there is the chance that someone important might decide that by virtue of my use of an API, the program constitutes a derivative work of the library. (If they did, that would have huge implications — every Linux program, not to mention Linux itself, would become a derivative of Unix itself, for example, because it uses Unix API's, and every Win32 program would be a Windows derivative...)

This, of course, is completely different from C/C++, where in order to compile an executable, the compiler #includes parts of the actual source code of the library into the executable, thereby automatically triggering the need for an exemption from copyright laws.

With Python and Javascript, nothing of the sort occurs for the user to get the developer's software. If the developer decides to test their program before distributing, they may well need copies of the library code, either in source or object form, on their machine, but that has nothing to do with their distribution of their own code.

Now, the GPL (or whatever), may have terms explicitly forbidding you from finding some sneaky way around the licence, such as dynamic languages or web interfaces etc. But it doesn't matter. The licence could require you to sacrifice your first-born child to the gods of Jupiter for even thinking of doing such a thing, but the licence is irrelevant, since you haven't done anything that would require you to accept its terms.

So, unless judges are going to rule that use of APIs constitutes copyright infringement, all software licenses of this kind are simply irrelevant in this case.

This is one of the things that makes me shy away from the GPL. I'm a really grateful beneficiary of all that it has helped to safeguard, but as it relies on copyright law which may well be completely irrelevant for dynamic languages, it seems somewhat unwise to lean on it for any of my own financial security. More and more it seems that being paid for writing code, and not for the code itself, is the way programmers should make their living. Just because copyright law may have supported certain practices in the past doesn't mean we have a right to that continuing.

MOC and last.fm

2008-06-26T22:26:11Z

Update: people are having trouble with this now, I am unable to debug what is wrong, it seems like a problem with mocp.

When not using Amarok (e.g. if I'm low on memory due to using VMs or something), moc is very nice music player (Debian/Ubuntu package 'moc', but command 'mocp'). Here is how to get it to submit tracks to last.fm in a well behaved manner (that doesn't submit tracks if you are just skipping through a playlist):

Install lastfmsubmitd: sudo aptitude install lastfmsubmitd
Add yourself to lastfm group: sudo adduser [yourusername] lastfm
Download and save my MOC - last.fm script, and make it executable.
Alter your ~/.moc/config to set OnSongChange as per the instructions at the top of the script.
Completely quit moc and restart it.

The advantage of the script I wrote over just using OnSongChange is that my script will wait for half the length of the song before submitting to last.fm. Every 5 seconds it will check you are still listening to that song, and if not, it will quit. This way it will behave much more like Amarok and other well behaved last.fm clients.

Personal wiki and desktop integration

2008-02-23T21:12:57Z

Like many people, I find it easier to find stuff on the web than on my own machine. This post is about some of my attempts to remedy that. You probably need to be a developer or that way inclined to get the most out of some of the solutions I've got here. Stuff to download is at the bottom.

Search tool §

Obviously, first of all there is a need for a decent desktop search tool. On Linux, your main options are:

Google desktop search
- Main problem: it stops it's initial index after 100,000 documents. I've got far more than that, and it means it never actually reaches my mail. Typical crippleware. This is useless for me.
Beagle
- Main problem: it will decide to thrash your disk and/or CPU and crash. For me, runaway CPU usage and crashing made this completely useless. It might be 99% of the way to being brilliant, but until these things are sorted out, it is a 100% waste of time.
Strigi
- Minor problem: the GUIs suck
- Minor problem: it indexes too much, sometimes.
- Main problem: after indexing about 300,000 documents, Strigi dies on me and refuses to do anything useful at all. I don't know what causes it to choke, but it, too, is 100% useless AFAIC. Maybe it's fixed now, but I've got bored of filing bugs and compiling from source just to find out it still doesn't work.
MetaTracker, or Tracker for short.
- It can't search zip files etc. like Strigi can, which is a shame.
- It doesn't yet index my emails (KMail) -- it only does evolution.
- But I actually find this tool useful, . It doesn't thrash my disk, it has a simple but effective GUI, as well as command line tools, it integrates reasonably well into KDE, and it also has a nice tagging facility.

Tagging and Konqueror §

Adding tags to files needs to be easy to do if it's going to happen. MetaTracker provides the tracker-tag program which works well, but it is command line. So I've written a small GUI wrapper that integrates into Konqueror via a service menu. If you tag more than one file at once (by selecting a few files in Konqueror), it is in bulk tag mode, where it pops up an input box and simple adds the space-separated tags you enter to all the files.

In single file mode, it does something slightly different. It will first query the file for any existing tags, adding them to the default text that is in the input box, so that you can remove tags as well as add new ones. It will also add a suggestion for an 'ID' tag -- based on the filename, with punctuation replaced by hyphens, and with 'ID-' prepended. The idea is that I edit this and make it into something that will uniquely identify that file. For example, a file "bible reading schedule.pdf" ends up with the tag ID-bible-reading-schedule-pdf, but "rest-quickref2.html" I will expand to "ID-restructured-text-quickref-html", and in some cases I would have to include something else that will distinguish it.

The nice thing about these tags is that MetaTracker will keep track of them even if I move or rename the file. I'm not entirely sure how it does this (I think it uses inotify). But it works, and that's all I care about for now.

Migrating §

It may be that strigi / Nepomuk will mature and become useful. When they do, I can dump the tag database from MetaTracker pretty easily, and should be able to import into strigi. Changing the tools/scripts I've developed to use a different metadata database should be pretty simple.

Personal wiki §

On top of this, there are other things that file systems are just not very helpful for organising. Hierarchy works pretty well for about half my stuff (development work in particular). But notes and things I like to jot down often get lost on my disk -- there are multiple places that some things belong, and some belong nowhere at all.

I decided that a personal wiki would be the best way forward for this. However, I then had to choose a wiki -- thankfully the WikiMatrix helps a lot. I had tried the Instiki wiki. But it turns out that Ruby on Rails does suck as much as they say -- the instiki process would sit there at about 50Mb for a while, but then decide that it would use up all my memory. (Other people have noticed the same behaviour). So I wasted some time with that, and I don't think I'll be trying Ruby on Rails soon if this is the kind of thing I can expect.

But anyway, WikiMatrix helped me find the Trac wiki. Though I've used Trac a lot before, I hadn't thought of this option, as it is a wiki that is geared towards software projects, not general things. However, I've discovered it is ideal for me:

You can very easily turn off all the stuff you don't want, and then those components just disappear from the interface, exactly like they should.
It has all the help you need built in.
Nice GUI for administering it -- you need to use the command line UI for the first few steps, and you need to use htdigest to set up a password file, which some people might find tricky, but as a developer I'm right at home with that kind of thing.
It is written in Python, and it seems to be very well written too.
It uses SQlite (can use other databases), so it easy to query from outside if necessary.
It comes with a built in server -- very little setup, easy to run as a normal user.
It has a plugin architecture with lots of good plugins, and you can easily write your own.

This has allowed me to bring lots of bits and pieces into one place, and organise them as loosely or tightly as I want. The TagsPlugin for Trac makes this particularly convenient. The wiki is actually pretty nice for composing blog posts too -- it can do all the code highlighting I want, and I can just copy the output HTML into my blog when it's ready.

Combining wiki and desktop §

The final problem is integrating wiki and other files. Obviously, I have many documents that are much better managed outside a wiki. It would be nice to be able to refer to these easily. And by writing some Trac plugins, this is actually pretty easy. I can now write things like this:

  file:ID-bible-reading-schedule-pdf

in my wiki and it turns into a file:/// link to the right document. In Konqueror, this works really well -- right clicking on the link brings up the same context menu that I would get in file browsing mode, so everything is available to me right there from the wiki page. I can also do a search:

  [[MetaTrackerSearch(programming languages,web)]]

This will returns links to all files containing the text "programming languages" and "web", nicely formatted in a table. Or I can do a tag search like this:

  [[MetaTrackerTagSearch(sermon)]]

to return all my sermons.

Further §

I've created a very simple Trac plugin for creating KioSword links, which works nicely for embedding links to Bible passages. I've got some ideas for further integrating different parts of my desktop. One thing I'd like to do is produce a database of my sermons and bible studies, the texts they are on, when and where I've preached them etc. A private Django app using the Django admin would probably work exceedingly well, and be very little work. A simple export routine could produce a wiki page that could be 'included' into my wiki (using a Trac macro). It could also export an read only ICal file which can be displayed in Kontact. So many cool ideas, so little time...

Download links §

MetaTrackerPluginMacro

tracker_tag_file
Install this by putting it in your path somewhere (e.g. ~/bin, if that is on your PATH)
It requires:
- kdialog (present on all KDE installations)
- setClipboard -- a little utility to set the clipboard with the contents of the tags just added, for convenience only, see below.

setClipboard

tagfile.desktop
Install into ~/.kde/share/apps/konqueror/servicemenus/