SNMP Engine for 250,000+ Device Networks (Scaling up...)
Written by
on
in
Knight Errant.
Wanting to scale up Cinemon's SNMP performance. We're currently in the 10-20,000 device range, which is okay, but I'd like to increase that by a factor of 5 or 10 to address some performance issues. We've got plenty of memory for that scale (though we'll need to work on the long-term storage system a bit). The big limiter right now is processor load in encoding/decoding all the SNMP in Python (and the long-term database (postgresql) storage).
Thinking I'll write a NetSNMP wrapper and wrap that into TwistedSNMP so that it can use either back-end. That should let me test the performance of the back-ends. I'm assuming NetSNMP can handle scanning 100,000+ networks without breaking a sweat on the server. It can probably handle 250,000 too, after all, 15x faster than Python isn't all that hard for C, particularly when it's all grotty, low-level stuff like SNMP parsing.
Not going to get done this week, though. Two deadlines on Friday to deal with.
Comments
Comments are closed.
Pingbacks
Pingbacks are closed.
Justin on 02/12/2008 11:18 p.m. #
lol, not another wrapper! There are already a ton of them, and most of them are terrible :(<br />
<br />
I'd hope one of the ones mentioned at<br />
http://www.dlevel.com/blogs/alex/22<br />
<br />
would work for you :)
Mike Fletcher on 02/13/2008 7:40 a.m. #
Unfortunately, no. The GPL licensing makes it a non-starter for Cinemon (which is a commercial, closed-source product, though not for a lack of my trying to convince people to Open Source it).
Eric Newton on 02/20/2008 6:33 a.m. #
Hey Mike,<br />
<br />
Hopefully you will be at PyCON and we can talk a bit about this. It would be nice if we improved the ctypes wrapper I wrote for Zenoss, and if the license is an issue, we can talk about that, too.<br />
<br />
-Eric<br />
Mike Fletcher on 02/20/2008 9:31 p.m. #
I'll be at PyCon, would be happy to chat with you about it there.