Category archives: Snaking

Discussions of programming, particularly of programming Python

RSS feed of Snaking

Parsing for fun and profit (Data importers are *fun* )


Spent the day working with SimpleParse. The easy part was defining a grammar, took maybe an hour. After that the annoying work of building the interpreter began. It was annoying because I realised that there is a fundamental flaw in my plan for this sub-project, namely, I'm not forcing the user to differentiate between objects ...

Continue reading

Oh, I'm a continuous scanner, I am... (Continuous scanner, I am, I am)


Seem to be finished the hierarchy discovery conversion (I'm sure there's still missing parts, but it seems to work). Now I get to integrate the actual steady-state scanning code (which has been running for a month or two) with it. After that I do the loading of location data, then the archiving/statistics machinery.

All of ...

Continue reading

Multi-threading with PyPgSQL can be a serious PITA (Another few hours tracking down hangs due to a missing commit...)


Full-featured databases tend to have low-level locking mechanisms such that if a particular thread or client has modified a row in one transaction, then attempts to reference that row in other transactions are blocked. When your database interface (e.g. PyPgSQL) doesn't allow for sharing transactions between threads, you can (when a problem occurs) wind up ...

Continue reading

Rather annoying cutting old code (New scanner code added to old cloth makes some poor seams...)


When you wind up back-modifying large-ish bodies of code to work in entirely new ways, you wind up with a lot of areas where the fit isn't wonderful. In this case, the ping scanner was written with the assumption that the entire hierarchy was coming from database, and so therefor every operation could rely on ...

Continue reading

Why do we still have software patents? (Have they helped foster invention? No. Then why grant them?)


Just seems insane to me that people can get patents on what are essentially just ideas. It should not be possible to patent "using a computer + network to do x", yet that's what's being done. It should not be possible to patent basic mathematics or algorithms, yet we allow it. And every time we ...

Continue reading

Extreme Twisted Goodness! (Days are more productive with extreme programming... though having a "User" would be useful)


The day's work was basically getting the hierarchy-discovery code moved into the ping-scanner framework. As a side effect, I've finally built a full-system simulator for the CMTS+Channels+modems... makes writing tests much easier, which in turn gives a better feel to the development process... a good day :) .

The actual mock CMTS + Modems are ...

Continue reading

Joy of Twisted (Well, sort of an ennui, really...)


For days now I've been working in Twisted, building a severely trimmed-down version of Cinemon. Every so often little glimmers of greatness show up, but then I have to create yet another 2 functions to deal with success/failure cases, and the glimmer fades.

As I've mentioned time and again to anyone who will listen; the ...

Continue reading

Simpler solutions make life simpler... (Abandoning chaining in favour of updating)


As mentioned a while ago, I was trying to build a chaining mechanism into TwistedSNMP's OIDStore mechanism, the idea being that you could mix-and-match OIDStores so that you could compose test-cases readily from pre-built OIDStore fragments.

Problem was that the code became hideous. It's just a graph-traversal algorithm, but the code wasn't written with graph-traversal ...

Continue reading

Chaining OIDStores in TwistedSNMP (Today's little side project...)


TwistedSNMP has a mechanism called an OIDStore, which allows you to define the values to be returned from a (server-side) agent. At the moment, these are quite simplistic creations, basically either a bisect-sorted list or a BSDDB btree database (the OIDStore must be ordered, which is why a simple dictionary wouldn't work).

That works fine ...

Continue reading

Day's slogging grinds to a close (Somewhat unsatisfactorily...)


Spent most of the day (when I wasn't sleeping (I'm obviously not getting enough sleep these days)) tracking down bugs in the data-import mechanism. Honestly, I think I'm just trying to do too much with it. It allows for altering the planned objects as you are importing the data-set, generates an arbitrarily deep number of ...

Continue reading