Tottering piles of crapware

So I’ve been playing around with Jabber recently, and I wanted a client for testing and admin purposes. I wasn’t keen on Psi because installing a pre-built package would have required upgrading the world, and building it and Qt takes for ever. So I thought I’d try CJC, the console jabber client, which is appealingly retro.

CJC is written in Python and has an associated library called PyXMPP which does the protocol end of things. (In fact this is another reason for choosing CJC - in the fullness of time I’ll want to write some glue software to hook Jabber into other stuff and PyXMPP seems like a plausible choice of foundation.) PyXMPP needs a few add-on libraries in addition to the standard Python install.

Building Python and persuading it to link to the correct libraries wasn’t too much trouble. DNSpython builds in the standard Python way without any difficulty at all. LibXML2 was also not too bad - though because it depends on the Python install to compile its Python bindings, but part of the Python install (viz. PyXMPP) depends on libxml2, I couldn’t separate it from Python properly. A bit scruffy, but shrug.

The problems are almost entirely to do with M2Crypto, which provides Python bindings for OpenSSL. M2Crypto uses SWIG to do most of the work of hooking the two together. The nice thing about SWIG is that it can be built to support loads of scripting languages without having any of them installed, so (unlike libxml2) I could divorce it from the Python install.

M2Crypto uses SWIG in a rather grotty way, resulting in lots of Warning(121): %name is deprecated. Use %rename instead. I guess that the reason for this deprecation is that the %name directive requires the SWIG interface file to include a copy of the C function’s declaration. This caused the M2Crypto build to explode when pointed at OpenSSL-0.9.7h because there have been some constness changes in the OpenSSL header files which cause the M2Crypto’s idea of some function types to be wrong. Sigh.

So I put together an evil hack to patch this bug, and got the whole thing built successfully. However when I tried to connect to jabber.org with cjc, I got a lovely stack traceback:

    File ".../pyxmpp/stream.py", line 1206, in _make_tls_connection
      ctx=SSL.Context('tlsv1')
    File ".../M2Crypto/SSL/Context.py", line 41, in __init__
      map()[self.ctx] = self
    File ".../M2Crypto/SSL/Context.py", line 20, in __setitem__
     self.map[key] = value
  TypeError: unhashable type

M2Crypto tries to keep a hash table of SSL context structures. (I don’t know why; it looks redundant to me because that bit of the code could just use self.ctx instead.) The SSL context structure is a SWIG wrapper around the corresponding OpenSSL type, and SWIG doesn’t define a tp_hash function in its wrappers, so Python bitches as you can see above.

Working this out required far too much effort. At first I thought it might be a problem with version skew, so I tried using older Pythons and older SWIGs and newer cjc/pyxmpp snapshots, and the bug remained. So I looked at the code more closely and consulted friendly Python experts and came to the conclusion that the code couldn’t possibly work. (But then how were others managing to use it?) I then thrashed the SSL context hash table to within an inch of its life by using repr(key) instead of key, so that a stringified version of the context pointer was used to index the table, thus bypassing the problem.

After that, all I was left with was a minor certificate verification problem, which was solved by teaching M2Crypto about the many benefits of SSL_CTX_set_default_verify_paths() and getting PyXMPP to invoke the new M2Crypto feature appropriately. And finding the CA certificates required to verify the signing chain of jabber.org’s dodgy certificate.

So maybe now I can make some real progress, instead of shaving yaks.