I finally got time today to work seriously on Cinemon again. Spent the time working on a remoted SNMP query server (i.e. something that can be put in a remote network to do the actual querying while the Cinemon server is outside the network). I initially tried using Twisted's jelly module (via pb), but couldn't get the dratted thing to accept that the OID class should be allowed to be linearised across the interfaces without having to alter the OID class itself (which is way over in PySNMP).

Spent far too long pouring over the documentation, and eventually the code. Finally just gave up and decided to write my own restricted pickle/unpickle module to dump the data to a string for transmission across pb's interface.

That decision really disturbs me, looking back on it, why is it that I saw the Twisted code as so foreign that I didn't want to start patching it? Sure, some of it is that I just wanted to get done tonight, instead of having to go about getting something integrated into a foreign codebase. Some of it is just design mismatch (jelly assumes that you control (and want to change) the objects you're transmitting, I don't want to alter PySNMP's core objects just to make the available through pb).

I should have jumped onto IRC with the Twisted folks and had a long discussion about the design and integration of pb, jelly and the rest, then worked out some sort of proposal to make the code work with externally-added/registered pickle/unpickle functions... maybe there would turn out to be an existing solution, even... but I just wanted to get the work done, and strange as it seems, writing a pickle-like module isn't all that hard in Python; freezer is only 400 lines, jelly is 900, pickle is 1400 (jelly and pickle do more, incidentally). Much like web frameworks, it's quite possible to sit down and work out a moderately functional and precisely tailored tool in a few hours, often less time than you'd spend on discussions for changing the behaviour of another tool.

Which is the problem. Python just makes complex tasks a little too easy... it's too easy to code new things when something's wrong with an existing module. Or maybe it's just that one shouldn't code with a "I've only got a few days to work here, can't afford to wait for other people" feeling.

Anyway, for the curious, here's how the module works:

You register a pickler and an unpickler function for each class you want to service on the Freezer instance. Each Freezer instance has a loads and dumps method, these use a Package instance to do the actual encode/decode of the object. During linearisation the Package builds a table of class to classID mappings that forms the header of the encoding, and the objects use references into that table. There's a method to register handlers for the normal builtin classes (defined in the module, but not otherwise special).

The format is not particularly inspired, it's very much a "well, I need something here, grab handy character X and toss it in as a delimiter" design, with encoded data-lengths in standard "object" headers used for most of the heavy lifting. It handles recursive objects, multiple references to the same object, all the common stuff you'd expect.

Writing unpicklers is a little crude, as the code evolved to support recursive references after I had written most of the unpicklers for basic objects. The result is that unpicklers for compound objects have to register the result object explicitly before they reconstitute their child objects.

Anyway, though I can't imagine why anyone would want it, I've put the code up in the scripts collection.