All Unkept
Posted in: Python, Software development  —  20 June 2006

Pyrex and Lisp

On a whim, I tried converting one module of Python code in my Django app to Pyrex, to see what kind of performance benefits I'd get. So far, I haven't actually succeeded in getting the C code to compile. Changing the Python code so that Pyrex could handle it was pretty straightforward, however, and it also led to a small revelation.

Pyrex can't handle some Python features such as list comprehensions, generators or closures. I had a few list comprehensions (easily changed), no generators and just one closure (I wrote the code a while ago -- it would probably have more closures if I wrote it today). Rewriting a closure is actually pretty straightforward. Instead of something like this:

def foo(x):
    def bar(y):
        return something_with(x, y)
    return bar

you have:

class Bar:
    def __init__(self, x):
        self.x = x
    def __call__(y):
        return something_with(self.x, y)

def foo(x):
    refurn Bar(x)

This was the first time I had to explicitly perform the transformation on a real bit of code, and it made it blindingly clear that classes are human-compiled closures.

Of course, some people would object that we would use classes anyway, and in different ways to the above code, because they are part of object-oriented programming. Due to increasingly negative experiences of OOP at work, and a taste of the absolutely nightmarish mess it can make of a large software project, I've become very disillusioned with that programming paradigm, and find myself believing more and more that state is the root of all evil. So what I really want is closures and functional programming -- not classes and OOP at all. (Of course, that's not to say that all OOP is hopeless -- a lot works well. But I'm less and less convinced about it in general).

This means, of course, that Paul Graham is right, again. That's the really annoying thing about smug lisp weenies -- they're always right.

Well, if you can't beat 'em, join 'em! I think the first functional programming language I'm going to learn is Haskell -- I've looked at it a little bit, and it seems wonderfully elegant, and I want a pure FP language to ensure that I really learn the functional style. Once I've got my head round Haskell and monads, and I'm craving for macros, I'll take the red pill and join the Rubber Duck brigade. When I (hopefully) achieve enlightenment, I'll try not to be too smug :-)

Comments §

§ On 24 August 2006, Anonymous Coward wrote:
90 If you want to get rid of STATE with a practical functional language, how about Erlang?

Making reliable distributed systems in the presence of sodware errors
http://www.sics.se/~joe/thesis/armstrong_thesis_2003.pdf

§ On 5 October 2006, Marcel Popescu wrote:
149 Pure declarative languages say that there is no state. The global state of the system is carried into all functions and comes out from all functions. Mechanisms (...) are used to hide state from the programmer (...)
---
Wow. Talk about... well, I can't talk about it without using insulting terms :)

First of all, the paragraph contradicts itself.

Second, the idea that all functions have access to all state at all times is, well, BASIC-80. I thought everyone knew by now *why* globals are a bad idea.

Finally, that state which does not exist but at the same time is globally accessible is hidden, sorta. Through mechanisms. Which is bad when OOPLs do it, but good when "we" (for some value) do it.

The whole thing is so stupid I am half convinced it is a parody of sorts.

§ On 6 October 2006, luke wrote:
150 Marcel:

I'm not sure why I'm having to defend someone else's article, but anyway...

That article has a simplification of how state works in functional programming langauges. The point about there being no state is simply that the output of every function depends *entirely* on it inputs. So you don't have to worry about what is the state of the system at this point, unlike imperative/OO systems when you have an object whose state you are unsure of, because you don't know how it is has been messed with so far.

Of course, programs do need to alter the real world - display things on the screen, alter databases etc. In most programming languages, you can do this at any point, anywhere in your program -- making the program very difficult to reason about, due to side effects. Having access to global state everywhere is exactly what imperative/OOPLs do all the time -- it is *these* languages that haven't progressed from the days of BASIC (the fact that global state may be wrapped in accessors helps but is far from solving the problem).

By contrast, in FPLs like Haskell, you have to explicitly pass 'the Universe' into a function, and you get back a new 'Universe'. Actually, it's more subtle than that, and much more convenient (the mechanisms that he talks about), and more powerful...but no, it's not stupid at all, you really have to try it to find out.

Add comment

Format:

  • Javascript has to be on to get past my spam protection, and cookies, and there is a delay, sorry for any inconvenience!
  • I reserve the right to moderate comments.