Luke Plant's home page (Posts about Linux)

HP Pavilion 17-e153sa on Linux Mint

2014-08-04T10:54:30+01:00

This page reports my success with installing and using Linux on an HP Pavilion 17 laptop, model e153sa.

These results apply to Linux Mint Debian Edition (2014-03) and Linux Mint 16. (These are similar to Debian and Ubuntu).

Overall status: Almost everything works perfectly.

Tests included:

Screen, keyboard, touchpad, SD card reader, VGA output, HDMI output (multi-monitor setup), CD/DVD reader/writer, 8Gb RAM, CPU (Linux reports and uses 4 cores as expected for this i5 processor), wifi, webcam, built in microphone, built in speakers, analog headphone/speaker out and microphone in, USB inputs.

Notes:

To install Linux from CD/DVD, you have to press 'Esc' during bootup to go into the BIOS settings. Change the boot order to boot from the install CD/DVD before hard drive, and disable UEFI secure boot, then exit and reboot.
The laptop monitor seems to have some kind of default brightness that is really, really low (almost black).

Thankfully, you can adjust this using the keys on the keyboard. After you've fixed it once after bootup, it can sometimes be triggered again e.g. when you go into "System Settings" on Linux Mint - the laptop screen goes completely black. This is a bit annoying, and can make you think it's not working at all, but after setting the brightness once or twice I usually don't have a problem again until I reboot, which is pretty rare.
The wifi/webcam/graphics all worked perfectly out of the box with no extra drivers needed.
WebGl in Firefox works great. I found it was disabled on Chrome by default, but this can be easily fixed by enabling “Override software rendering list” (opening a tab with the URL chrome://flags/#ignore-gpu-blacklist) and then it works fine.
Initially I found that SD cards were not recognised at all when inserted into the slot. After some Googling, I found this can be solved by having an SD card in the slot while the laptop boots, and whenever you resume from suspend. Annoying, but not too serious if you have an old SD card that you don't mind leaving in your laptop.
Very occasionally my external USB mouse stops working when I plug other devices in to the nearby USB ports. Solved by unplugging and re-plugging the USB mouse. I'm not sure if this is a hardware or Linux issue.
Multi-touch for touchpad works, at least to a basic level e.g. two finger scrolling, using the Linux Mint control panel.
I have not succeeded in getting HDMI audio output to work (using PulseAudio). Update 2015-12-19 - with Linux Mint 17 this now works fine.

Import Chrome passwords into Firefox on Linux/Gnome

2014-03-07T09:32:32Z

If you are using Linux and the Gnome/Mate/Cinnamon desktop environments, Chrome stores passwords in the Gnome password manager (also known as Seahorse).

The following script will find these passwords, and export them to an XML password file that can be imported by the Firefox "Password Exporter" extension, so that, between them, you can transfer saved passwords from Chrome to Firefox.

Please see the top of the script for instructions, item 3. (The script also has other modes, for pure export/import of Seahorse passwords). You'll need to know how to install Python and related packages - this is not a point-and-click solution, sorry!

https://github.com/spookylukey/gnome-keyring-import-export/blob/master/gnome_keyring_import_export.py

Export and import for Gnome password manager (Seahorse)

2014-03-07T09:17:44Z

Without much ado, here is a Python script that will allow you to dump password from the Gnome password manager (also known as Seahorse, and used in other desktop environments such as Cinnamon) into a JSON file.

This is useful for just viewing all your stored passwords (which are not always visible in the password manager GUI), but it will also import them again, which is especially useful when you want to pull in all your saved passwords from an old machine.

Please see the top of the script for instructions. You'll need to know how to install Python and related packages - this is not a point-and-click solution, sorry!

https://github.com/spookylukey/gnome-keyring-import-export/blob/master/gnome_keyring_import_export.py

Controlling Sozi remotely from an Android phone

2013-03-11T15:37:57Z

I'm due to give a talk next Sunday evening, and I've created a presentation using the very cool Sozi. Sozi is quite like Prezi, but open source and free, and its output is a standard SVG file (with a bit of Javascript), so it can be displayed in several web browsers.

(On the other hand, for many people Sozi is probably a lot harder to use for simple presentations than Prezi, as it works as a plugin to Inkscape, a general purpose drawing program, but, to me, it is simply unacceptable to depend on the continued existence of a company, and a subscription fee, to be able to continue to view/edit your presentations).

One problem with not using ‘standard’ software like Powerpoint is that proprietary tools like hand-held remote controls for Powerpoint won't work, or won't work easily (and I don't want to find out that it won't work when I turn up) [EDIT: it looks like Sozi now has good compatibility with many wireless presenter tools, but I don't have one anyway]. But I've found a solution that works for me:

On my Android phone, I installed Coversal, which is an app designed for controlling media players, but allows custom commands to be executed, so actually works fine for presentations.
Within Coversal, I installed the ‘SSH Custom’ plugin.
On my Linux laptop, I installed openssh-server to allow the app to log in to my machine. To avoid putting my laptop password on the phone, I created an SSH keypair for this app, transferred to private key to the phone, and added the public key to ~/.ssh/authorized_keys.

I created a script on my laptop that can control Sozi running in Chrome:

#!/usr/bin/env python

# controlsozi

import os
import subprocess
import sys
import logging

from datetime import datetime

logging.basicConfig(level=logging.DEBUG, filename='/tmp/controlsozi.log', filemode="a")

def get_window():
    # Assumes that there is just a single Google Chrome window open, and this
    # has the presentation in it.
    windows = subprocess.check_output("DISPLAY=:0 wmctrl -l -x | egrep 'google-chrome|gnome-www-browser'", shell=True)
    return windows.strip().split()[0]

def send_text(text):
    os.system("DISPLAY=:0 xvkbd -window %s -text '%s'" % (get_window(), text))

ACTIONS = {
    'previous':      "\[Left]",
    'next':          "\[Right]",
    'fullscreen':    "\[F11]",
    'previous_fast': "\[Up]",
    'next_fast':     "\[Down]",
    'start':         "\[Home]",
    'end':           "\[End]",
    'launch':        None,
}

def log_uncaught_exceptions(*exc_info):
    logging.critical('Unhandled exception:', exc_info=exc_info)
sys.excepthook = log_uncaught_exceptions


if __name__ == '__main__':
    logging.debug("Starting...")
    if len(sys.argv) < 2:
        print "Usage: "
        for k in ACTIONS.keys():
            print "  " + k
    else:
        action = sys.argv[1]
        logging.debug("Action = %s" % action)
        if action == "launch":
            os.system("DISPLAY=:0 nohup google-chrome '%s' &" % sys.argv[2])
        else:
            text = ACTIONS[action]
            logging.debug("Text = %s" % text)
            send_text(text)

    logging.debug("Done")

This relies on command-line tools wmctrl and xvkbd. With only small modifications it should work for Firefox too. It is not 100% reliable, as it works by sending keypresses to windows, but it works fine for me.

I set up a profile in Coversal to connect to my laptop and use this script.

Coversal allows you to define the commands for certain actions – the list of actions includes things like up, right, next etc., but you can add your own. You then assign actions to buttons, as desired.

So I simply I had to set commands like "/path/to/controlsozi next", and then assign these actions to the buttons I wanted to use.

By setting the start_playback command to "/path/to/controlsozi launch '%s'", I could even use the file browser built in to Coversal to select the presentation file to view, which is pretty neat.

The one thing I was really missing was physical buttons – with a touch screen, it's so easy to hit the wrong place, and I wanted something to rest my finger or thumb on.

I then realised that my phone does have a couple of physical buttons - the volume control buttons on the side. By enabling the 'volume_up' and 'volume_down' buttons in Coversal and mapping them to the 'next' and 'previous' commands, the physical volume buttons get mapped to the same actions, and they do the job just fine. I can use the rest of the controls on the screen for the less common tasks.

For my phone to be able to connect to my laptop over SSH, there is one more piece of configuration that must be done – the IP address of the laptop has to be entered. If you have a local wifi, this is no problem – just connect the two devices to the same wifi and they will be able to talk. My phone has a more reliable option, though – it can itself operate as a wifi hotspot, creating a wifi network that the laptop can connect to.

Conclusions:

Linux is awesome.
Smart phones are awesome.
Free Android apps are awesome (and I think it is the Linux culture which encourages this).
Python is awesome.
Being a programmer and being able to plug all this stuff together is awesome.

KTimeTracker replacement - TimeCult

2012-02-24T17:57:32Z

For a long time I haven't been able to find a decent replacement or alternative to KTimeTracker. But I've now succeeded: TimeCult.

It is currently lacking an installer for Linux, but it can be made to work with Linux easily.

I also found that it uses a very easy-to-deciper XML format, and, with the help of vobject, the KTimeTracker ical files are only slightly harder to figure out, so I created a Python script that will do the conversion.

Use the script like this:

./ics_to_timecult.py ~/.kde/share/apps/ktimetracker/ktimetracker.ics > out.tmt

It's here for reference - dependencies are vobject and elementtree:

#!/usr/bin/env python
# Quick and dirty script to convert ktimetracker.ics files
# into TimeCult files.

import os.path
import sys
import time
import uuid

import elementtree.ElementTree as ET
import vobject


# Structures to store the tree of data. Use native Python data types, and
# convert to what TimeCult expects when we 'render'

class Struct(object):
    def __init__(self, **kwargs):
        self.__dict__.update(kwargs)


class TimeCult(Struct):
    # <timecult>
    # Eventual attributes:
    #  name
    #  projectTree
    #  timeLog

    def toXML(self):
        root = ET.XML('<timecult appVersion="0.12" fileVersion="10" />')
        root.attrib['uuid'] = str(uuid.uuid1())
        root.attrib['name'] = self.name
        pt = ET.Element('projectTree')
        for n in self.projectTree:
            pt.append(n.toXML())
        root.append(pt)
        tl = ET.Element('timeLog')
        for tr in self.timeLog:
            tl.append(tr.toXML())
        root.append(tl)
        return root

class Node(Struct):
    # <project> and <task>
    # Eventually will have these attributes:
    #  vtodo
    #  summary
    #  uid
    #  created
    #  parent
    #  children
    #  type = "project|task"
    #  finished
    def __repr__(self):
        return "<Node: uid=%s summary=%r, parent=%r>" % (
            self.uid,
            self.summary,
            self.parent.summary if self.parent is not None else None)

    def toXML(self):
        e = ET.Element(self.type)
        e.attrib['id'] = self.id
        e.attrib['name'] = self.summary
        e.attrib['created'] = str(int(time.mktime(self.created.timetuple()) * 1000))
        if (self.type == "task"):
            assert len(self.children) == 0
            e.attrib['status'] = 'finished' if self.finished else 'inProgress'
        else:
            for c in self.children:
                e.append(c.toXML())
        return e

class TimeRec(Struct):
    # <timerec>
    # Eventual attributes:
    #  startTime  # datetime
    #  duration   # seconds
    #  task       # Node
    def toXML(self):
        e = ET.Element("timeRec")
        e.attrib['taskId'] = self.taskId
        e.attrib['startTime'] = str(int(time.mktime(self.startTime.timetuple()) * 1000))
        e.attrib['duration'] = str(int(self.duration * 1000))
        e.attrib['notes'] = ""
        return e

def convert(filename):
    f = vobject.readOne(file(filename).read())

    # First parse Todos
    nodes = {}
    for vtodo in f.vtodo_list:
        n = Node(vtodo = vtodo,
                 uid=vtodo.uid.value,
                 summary=vtodo.summary.value,
                 created=vtodo.created.value,
                 finished=vtodo.contents['percent-complete'][0].value == "100"
                 )
        nodes[n.uid] = n

    children = {}
    for uid, node in nodes.items():
        vtodo = node.vtodo
        try:
            related_uid = vtodo.contents['related-to'][0].value
            node.parent = nodes[related_uid]
            children.setdefault(related_uid, []).append(node)
        except KeyError:
            node.parent = None

    for uid, node in nodes.items():
        node.children = children.get(uid, [])
        if len(node.children) == 0:
            node.type = "task"
        else:
            node.type = "project"



    autouid = 0 # For newly created Tasks

    # Now find the time information in the ICS file
    time_log = []
    for vevent in f.vevent_list:
        related_uid = vevent.contents['related-to'][0].value
        task = nodes[related_uid]
        if task.type == "project":
            # TimeCult can only cope with time events being against 'leaf' nodes
            # in the tree. So we need to adjust and create an additional leaf
            project = task
            undefined_l = [t for t in project.children if t.uid.startswith('autouid')]
            if len(undefined_l) > 0:
                task = undefined_l[0]
            else:
                task = Node(uid='autouid-%d' % autouid,
                            summary='other',
                            parent=project,
                            type="task",
                            created=project.created,
                            finished=False,
                            children=[])
                autouid += 1
                nodes[task.uid] = task
                project.children.insert(0, task)

        dtstart = vevent.dtstart.value
        dtend = vevent.dtend.value
        if dtstart.tzinfo is None:
            dtstart = dtstart.replace(tzinfo=dtend.tzinfo)
        if dtend.tzinfo is None:
            dtend = dtend.replace(tzinfo=dtstart.tzinfo)
        time_rec = TimeRec(startTime=dtstart,
                           duration=(dtend - dtstart).total_seconds(),
                           task=task)
        time_log.append(time_rec)

    # Now need to assign IDs
    taskId = 0
    for u, n in nodes.items():
        taskId += 1
        n.id = str(taskId)

    for tr in time_log:
        tr.taskId = tr.task.id

    return TimeCult(
        name=os.path.basename(filename),
        projectTree=[n for u, n in nodes.items() if n.parent is None],
        timeLog=time_log)


if __name__ == '__main__':
    filename = sys.argv[1]
    timecult = convert(filename)
    sys.stdout.write(
        """<?xml version="1.0" encoding="UTF-8"?>""" +
        ET.tostring(timecult.toXML()))

My Bash prompt

2011-02-28T23:32:42Z

I like my bash prompt:

to include the current Mercurial/Git branch if applicable – otherwise I would quickly get in big trouble with committing things to the wrong branch in Django etc.
to show the full current working directory - for similar reasons to above, as I often end up lost in a maze of twisty little directories, all alike.
to be always instantaneous (tricky, when 'hg branch' can take up to a second on a bad day due to startup time)
to work well with virtualenv (which adds the environment name to the beginning of the prompt)
to show the time (so that if I have a long running job that I didn't realise was going to be long running, all the info is there to find out how long it took).
to do all the above without confusing me!

All of these together mean I need two lines, but this has advantages - it means the commands I type always start at the same column. I achieve (1) by a bash function that looks for .hg or .git recursively, and looks for '.hg/branch' where applicable. I achieve (6) by using some colours distinguish the different bits.

The end product looks like this - first without a virtualenv, then with:

The code looks like this (in ~/.bashrc)

# Prints " $branchname" if in a hg or git repo, otherwise nothing.
print_branch_name() {
    if [ -z "$1" ]
    then
        curdir=`pwd`
    else
        curdir=$1
    fi
    if [ -d "$curdir/.hg" ]
    then
        echo -n " "
        if [ -f  "$curdir/.hg/branch" ]
        then
            cat "$curdir/.hg/branch"
        else
            echo "default"
        fi
        return 0
    elif [ -d "$curdir/.git" ]
    then
        echo -n " "
        git branch --no-color 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/\1/'
    fi
    # Recurse upwards
    if [ "$curdir" == '/' ]
    then
        return 1
    else
        print_branch_name `dirname "$curdir"`
    fi
}

e=\\\033
export PS1="\[$e[1;36m\][\u@\h \t]\[$e[1;33m\]\$(print_branch_name) \[$e[0m\]\w\n\[$e[1;37m\]——> \[$e[0m\]"

player_do – control multiple media players from a single (command line) interface

2010-07-15T16:23:05+01:00

I tend to use several music/media players, such as shell-fm, moc and Clementine. And I often try out new ones. In order to be able to use the media buttons on my keyboard for pausing/skipping, I wrote this player_do Python script to automatically route the commands to the right one.

Quick start:

sudo easy_install playerdo

Or, if you prefer this way like me, and have $HOME/.local/bin on your $PATH:

easy_install --prefix=$HOME/.local playerdo
player_do help

Then use your window manager/desktop to setup keyboard shortcuts to player_do playpause, player_do next etc.

mpris-remote does a similar job, but only for MPRIS players, and I needed others. My script also has an MPRIS backend, so it will support XMMS2, Amarok, Clementine, Exaile, Dragon Player and various others out of the box. It also has support for shell-fm, moc, and rhythmbox.

If you have more than one running at a time, it tries to be intelligent about which one to talk to, although of course it is not always possible to read your mind correctly!

For other players that don't support MPRIS it should be quite easy to add a backend – please fork the GitHub repository, have a look at the existing backends and send me your patches!

Leaving KDE

2010-06-17T01:57:39+01:00

After being a loyal fan of KDE for a long time, I've just switched to GNOME.

I'm afraid to say that since the KDE 4 series, I've been unable to recommend KDE to anyone. With KDE 4.0 to 4.3, that was due to general crashiness, brokenness and bugs that are at the level of embarrassing – when a bug is obvious enough and significant enough that a new user would be shocked to find it and I would have a hard time explaining how I could use software that is clearly inferior to whatever they were using before (usually Windows).

With KDE 4.4, most desktop instability has been ironed out, but it has been replaced by problems that mean I can no longer stomach or risk staying with KDE for myself, let alone recommend it to anyone else. The final straw was that the framework for storing PIM data (Akondai) is turning out no better than I feared and is ‘losing’ some of my data.

I found that some things added to the address book ‘never’ actually end up in the file they are supposed to be stored in. I've added an e-mail address for a contact, and it appears to have added fine in KAddressBook. The data persists even across reboots etc., so it isn't ‘lost’ in one sense. Usually, the data appears in my std.vcf after a few seconds. In this case, however, four days later it still hasn't appeared in std.vcf. That counts as close enough to ‘never’ in my book, so it is fair to say it is ‘lost’. This has happened intermittently since KDE 4.4.1.

One additional result of this is that KMail never sees that address – at the moment it seems that it looks only in std.vcf, not Akonadi (and this is probably the only reason I found the bug in the first place). That's a pain, but the more serious problem is that my data is just not where it is supposed to be. If I need to backup and restore, I imagine it would be possible for that to succeed, if the Akonadi data was backed-up and restored along with the underlying data. But if you get that bit wrong, I can only imagine a world of pain and subtle data corruption. It is completely unacceptable that my data is being stored in “my data files + completely opaque Akonadi database” rather than just “my data files”.

Another bug is that the Google contacts Akonadi plugin also loses data in a serious way, making it far worse than useless. The nature of this bug makes me think that either there is a significant bug in the Akonadi framework, or that the framework API made it is easy to get this wrong – both of which are worrying.

These bugs are on top my general concerns with the framework – it has always been painful to work with, and the fact that it is even visible is a major failing – the software layer it replaced was, after all, completely invisible, just as it should have been.

From what I can tell, migration of KMail to Akonadi is going ahead at full pace and will be mandatory soon. My contact data is modest (500 Kb), so I have been able to cope with moving it around and fixing some problems, including some data loss and mistakes caused, I think, by the above bugs, since backups are quick. But my e-mail data is 2.5 Gb, and detecting problems or errors is going to be massively harder. I'm not willing to risk my e-mail data with Akonadi, so I need a new e-mail client.

It turns out that Evolution works for me. It stores data in mbox files, which isn't my preferred (1 file per e-mail, like maildir or MH, is nicer for various purposes), but it is a standard format, and since my backup tool is 'diff' based it will still cope efficiently with it. It even integrates with Google contacts out of the box, and the calendar app integrates with Google calendar, while allowing me to work offline. This is what Akonadi is supposed to offer one day, and I wish them the best of luck with that. But for now I'm going to stick with a system that prioritises the safety of my data, and that I know works. Evolution has a few significant niggles, but I think I can live with them.

That leaves me with the choice of desktop. Using programs from one desktop suite generally is faster and easier and looks nicer. I had already dropped Konqueror for Chrome (Konqueror has fallen far too far behind these days), and GNOME equivalents to most KDE apps work well enough – like Empathy/Pidgin. That tips the balance over to GNOME – and I can still use apps like k3b for specialist work when I need them.

Most of my reasons for leaving GNOME in the first place (about 7 years ago I think) have been resolved. GNOME apps like Nautilus and Evolution are no longer as excuciatingly slow and bloated as they used to be (and I don't think it is just that my computer got faster), and actually feel quite snappy most of the time. In some cases they definitely outperform the equivalent KDE default app (e.g. Eye of GNOME is very fast and works very nicely, doing 90% of what I used Gwenview for before, but much faster). Generally GNOME is much snappier as a desktop – it takes only about 5 seconds to start after I've logged in, which is much faster than KDE.

So I'm using Ubuntu's GNOME desktop, and I no longer have plasma and all the plasmoids. To be honest, I don't think I'm missing them. I have all the panel applets I need, and they are well designed and work properly. Proliferation of plasmoids is like the proliferation of apps on Apple app store – most of them are not worth having, and having to search through to find the ones that are is an anti-feature (not to mention that the new horizontal applet picker in KDE 4.4, which replaced the vertical one in KDE 4.3, is just awful – I don't understand how such a major regression happened).

I certainly won't shed a tear over Nepomuk/Strigi. After weeks of it constant disk activity as they attempted to index my files, they never seemed to do anything useful. They have been replaced by Tracker, which works very well, and very quickly, and without being a huge resource hog. It does not update immediately when I add new files, but it is fast enough to know that when I really want to find something in a real situation, it is likely to be in the index already. And it isn't the massive resource hog that Nepomuk was.

First experience with Ubuntu's GNOME has thrown up some problems. Here are some niggles I've been able to fix/solve:

PulseAudio - this breaks lots of things, especially games, and provides no added benefit. But it can be removed. Perhaps the Ubuntu guys will remove it in future, or make it easier to remove. Lots of people and programs seem to have problems with it, it's very difficult to see what value it is adding for most people.
To turn off desktop effects quickly (for better performance games) - either: install fusion-icon (use "fusion-icon -n" as the command) - or, to do it automatically for fullscreen games, set this option to true: /apps/compiz/general/screen0/options/unredirect_fullscreen_windows
To turn off screen locking on resume - there is setting is in gconf, search for 'lock'.

And here are some that are still bothering me a bit:

Reading new e-mails with keyboard only - it's a bit buggy/confusing with Evolution.
Some options are hidden away in gconf, so it's difficult to know if it is possible to change things.
GNOME has less configurability in general, or harder to get to it, especially for creating new keyboard shortcuts.
Compiz doesn't always seem to obey or understand all metacity options (e.g. shortcut for fullscreen-ing windows), so it sometimes confusing working out what is going on.
The GNOME file dialog boxes have improved a little bit since I was last using GNOME, but haven't begun to close the gap on KDE's far superior ones.

But there are some niggles which are no longer there due to the switch from KDE. Overall, things are similar at this level.

To conclude, I think the KDE developers were far too ambitious with the KDE 4 series. The cost has been that the only supported versions of KDE are ones that you cannot recommend to anyone, and that is a critical mistake in a competitive environment. Meanwhile, GNOME has massively improved in speed and has incrementally added features that people actually need, instead of creating large frameworks that may or may not be robust and useful one day.

Linux and calendars - bad combination...

2010-03-23T13:22:49Z

So, I'd like to be able to access my sync the calendar on my computer with other devices, like my BlackBerry Curve. This is 2010 right?

I already blogged the failure to sync directly. Let's not go there.
How about just syncing KOrganizer with Google, and syncing the BlackBerry with Google? I already have some calendars on Google, and that works fine. But now for my main calender:
- I tried GCalDaemon, but found that it ignores/deletes all recurring events on your calendar. Which is pretty evil.
- There is Akonadi... Given how hopelessly buggy Akonadi is for syncing contacts to Google (like, if Google has more than one e-mail for a contact, none of them are transferred, and worse), I'm not very hopeful about this (and I'm using most recent KDE 4.4.1). But I'll try anyway...
  - The whole experience is fraught with bugs and confusion. You click 'Add' and things aren't added, you click 'Remove' and they don't disappear, or only after some indefinite period of time (I've waited 30 seconds, is it just broken or should I carry on waiting?). You close the Akonadi configuration and open it again, or restart the server, and items have appeared, and other things that you didn't ask for. A lot seems to go on asynchronously, which might be a good idea, but you get the impression you are attempting to communicate with a forgetful, deaf, sleepy, indecisive, incompetent idiot. I have eventually managed to add a Google calendar resource, but it says "Failed getting last updated event", and nothing appears on my calendar.
OK, let's try Evolution calendar. At first it didn't work, then it seemed to work a bit better, but trying to modify events returns "Unknown error", and now it refuses to show the calendar at all.
OK, let's just try using Google calendar, importing all my existing calendar data, ditching any hope of using a local calendar app. But Google doesn't like my calendar file - "Failed to import events: Processing error during upload."

Epic fail. Bad day.

Update: in the last case, I discovered after some investigation that Google doesn't like the 'VTIMEZONE' sections in my calendar file. Removing them allows the import to succeed (although it still produces an error message, and I'm guessing it might have lost some time information on summer events though, not sure).

Blackberry Curve ↔ KDE-PIM sync on Kubuntu Karmic. FAIL

2010-02-11T17:07:58Z

For the sake of anyone else who might want to know about this in advance, I'll blog my failure.

I tried with Kubuntu Karmic, and the following steps:

Install the necessary packages:

sudo aptitude install opensync-plugin-kdepim opensyncutils barrybackup-gui barry-util opensync-plugin-barry

Plug in the Blackberry, and choose "Mass storage" mode on the Blackberry.
Set up permissions to allow a non-root user to access the device. I added the file /etc/udev/rules.d/040-devices.rules, with the following contents:
```
BUS=="usb", SYSFS{idVendor}=="0fca", SYSFS{idProduct}=="8004", GROUP="admin"
```
The above line gives permissions to all users of group 'admin', which includes my normal user. The numbers "0fca" and "8004" are device numbers obtainable from lsusb:
```
lsusb | grep "Research In Motion"
```
Result:
```
Bus 002 Device 008: ID 0fca:8004 Research In Motion, Ltd.
```
You will need to unplug the Blackberry and plug back in after this step.
Check you can access the Blackberry.
```
btool -t
```
This should list a series of databases on the Blackberry
Back up the current contents:
```
barrybackup
```
Note the pin for later, and follow the prompts to backup the Blackberry data.
Back up your KDE-PIM data, in case of any corruption, duplicate elements etc. (It happened to me!).

Create a msynctool configuration for doing the synchronisation.

msynctool --addgroup Blackberry
msynctool --addmember Blackberry barry-sync
msynctool --addmember Blackberry kdepim-sync
msynctool --showgroup Blackberry

The final command should show the first member as missing configuration. Do the following

msynctool --configure Blackberry 1

and insert your pin in the relevant place.

Moment of truth. Close KOrganizer and KMail, then:
```
msynctool --sync Blackberry
```

I came across the following major bugs:

Syncing never completes, and I have to press Ctrl-C to abort.
When I try to sync a second/third time, I end up with lots of duplicates on my computer, caused by items from the Blackberry being copied back. I'm not sure if they are to do with the unclean abort, or the following symptoms: * 'All day events'. A duplicate appears with the start time set to '0:00' * Daylight savings problems - for summer events, a duplicate appears 1 hour different from the original * Items from my Google calendar (which is a read only calendar) got copied back to my main calendar.
Each time I attempt to sync, duplicate events and contacts build up on the Blackberry as well.
Using barrybackup to restore to a clean state with the backup made previously fails with lots of errors. By removing some items in ~/.barry/backup/<pin>/config, I've got it to apparently succeed, but the contacts and calendar on the Blackberry are not cleared out, which is the point of the exercise. * There is a way to clear calendar/contacts on the Blackberry. Go to Calendar -> Menu -> Options. Type RSET. The calendar will be cleared (though it says nothing). Repeat for contacts. This time you will get a prompt about clearing. Clearing out the msynctool database is also a good idea to keep further attempts clean and fast.

In the end, a one-time, one-way sync is all that I have been able to achieve. That is quite disappointing, since none of the web pages I found before deciding on a Blackberry mentioned any problems like these.

I tried upgrading all the barry packages from 0.14 to 0.16, but got lots more errors - when trying to send data to the Blackberry I just got "Timeout in usb_bulk_read" blah blah and do data was actually sent.

Sources: