SNMP Engine for 250,000+ Device Networks (Scaling up...)

Wanting to scale up Cinemon's SNMP performance. We're currently in the 10-20,000 device range, which is okay, but I'd like to increase that by a factor of 5 or 10 to address some performance issues. We've got plenty of memory for that scale (though we'll need to work on the long-term storage system a bit). The big limiter right now is processor load in encoding/decoding all the SNMP in Python (and the long-term database (postgresql) storage).

Thinking I'll write a NetSNMP wrapper and wrap that into TwistedSNMP so that it can use either back-end. That should let me test the performance of the back-ends. I'm assuming NetSNMP can handle scanning 100,000+ networks without breaking a sweat on the server. It can probably handle 250,000 too, after all, 15x faster than Python isn't all that hard for C, particularly when it's all grotty, low-level stuff like SNMP parsing.

Not going to get done this week, though. Two deadlines on Friday to deal with.


  1. Justin

    Justin on 02/12/2008 11:18 p.m. #

    lol, not another wrapper! There are already a ton of them, and most of them are terrible :(<br />
    <br />
    I'd hope one of the ones mentioned at<br /><br />
    <br />
    would work for you :)

  2. Mike Fletcher

    Mike Fletcher on 02/13/2008 7:40 a.m. #

    Unfortunately, no. The GPL licensing makes it a non-starter for Cinemon (which is a commercial, closed-source product, though not for a lack of my trying to convince people to Open Source it).

  3. Eric Newton

    Eric Newton on 02/20/2008 6:33 a.m. #

    Hey Mike,<br />
    <br />
    Hopefully you will be at PyCON and we can talk a bit about this. It would be nice if we improved the ctypes wrapper I wrote for Zenoss, and if the license is an issue, we can talk about that, too.<br />
    <br />
    -Eric<br />

  4. Mike Fletcher

    Mike Fletcher on 02/20/2008 9:31 p.m. #

    I'll be at PyCon, would be happy to chat with you about it there.

Comments are closed.


Pingbacks are closed.