What should networking look like?
Written by on in Snaking.

I spent far too long today playing with two different networking systems.  The first was RabbitMQ and py-amqplib.  This seems pretty nice, you can send hundreds of messages in a fraction of a second... though it then seems to take *forever* to process them.  Appears to force in-order delivery, including an explicit ack from the client code... probably not suitable for low-latency (game) applications.

So for some reason I started playing with PyQNet.  It was (by contrast) processing around 3000 messages/second, though the messages in PyQNet can get loaded into a single package/packet and send together, so there would be fewer packets, though that was with 500B packages, so not all that many fewer.  Thing is, it doesn't matter how fast it is, as no-one uses it.

PyQNet has a traditional "struct" based interface.  You define message-types (structs) and the structs are linearized/delinearized on sending/receiving.  But what newbie user wants to define structs like that?  So this evening, after PyGTA, I sat down and coded up a simple experiment to use compressed json as the wire encoding for data... still preliminary, but it seems to slow down to about 1/10th of the speed, but it does feel better as an API.

Ah well, have spent far too much time playing with this stuff today, maybe some other day I'll actually have a game to write and writing a network library for it will come up again.

Comments

  1. j_king

    j_king on 06/18/2009 9:31 a.m. #

    I'm not sure what your requirements are...

    but binary pickles are a great format when you're dealing with a closed environment.

    twisted has some great frameworks, especially for games development.

    i've also used pyro (python remote objects) for p2p-based RPC systems. the agent system is really nice for propagating updates to nodes or executing long-running tasks on a network of workers.

    RabbitMQ, amPQ, etc are good enterprise messaging systems where it's important that the protocol be agnostic of any consumer/producer implementation... so you might be on the right track if that's what you need.

    I'll probably be able to make it to the next PyGTA. See you then. :)

  2. Mike C. Fletcher

    Mike C. Fletcher on 06/18/2009 10:54 a.m. #

    Looking for safe communications between machines.

    Twisted is wonderful, but it's *way* too big a dependency IMO when all you want is a single protocol.

    PyRO, AFAIK is not a low-latency protocol, low latency being one of my desires/requirements.

    RabbitMQ etceteras were just something I picked up off a random Python Reddit link. Not really a low-latency game-centric mechanism, more the thing that got me thinking about pyqnet again.

    Thanks for the suggestions.

  3. jarrod roberson

    jarrod roberson on 06/18/2009 12:13 p.m. #

    before you go and write something yourself, just stand on the shoulders of giants, http://sourceforge.net/projects/opentnl/
    personally I think a python wrapper interface to Torque Network Library would the easiest and the most power thing you could do.

  4. Richard Tew

    Richard Tew on 06/18/2009 9:39 p.m. #

    Rewriting the wheel gets dismissed a lot, but as someone who has worked on networking frameworks within games, there's a tremendous amount of value in understanding a framework from top to bottom and knowing the design decisions that were made in its development. And this isn't just limited to networking frameworks, it goes for pretty much any system you might use.

    You don't just have the ability to modify a system to suit your changing needs without worrying about drifting too far from a source project to gain any value from changes made externally to it. You also have the level of familiarity and knowledge to know how to adapt it in ways that might not otherwise be obvious.

  5. Mike C. Fletcher

    Mike C. Fletcher on 06/18/2009 10:34 p.m. #

    I'm not all that interested in maintaining a C++ wrapper to a GPL program. I *might* work on an existing open-source Python networking library that otherwise met my needs, but even then, I'm doing this mostly as a "fun" thing... which is to say the need is more to create something cool than to have something cool.

  6. Glyph Lefkowitz

    Glyph Lefkowitz on 06/20/2009 3:22 a.m. #

    How big is "too big"? Do you have a metric for what the right size of a dependency is? Do you mean in terms of on-disk footprint, memory footprint...? We've gone to some trouble to provide a slim distribution (the Twisted core tarball fits on a floppy disk) for those who need it.

    I've put a Twisted app onto a XScale embedded device with 8MB of RAM, so it doesn't seem unreasonably huge to me :). I'm not arguing that it *isn't* too big for you, though, just trying to get a sense of this objection in a realistic context. I know you've worked plenty with Twisted in the past, so this isn't some bogus off-the-cuff reaction.

  7. Glyph Lefkowitz

    Glyph Lefkowitz on 06/20/2009 3:35 a.m. #

    A word of warning about compressed JSON and similar dynamic self-defining protocol schemes: there is the potential for an attacker to confuse your code with unexpected arguments.

    There's both a security and a usability benefit to "struct"-based approaches. The security benefit is that since all your argument types are predeclared, you're never going to get an unexpected data-type in your application code. Python's laissez-faire attitude towards data-types is great as long as your code is all under the control of the "consenting adults" that python folks talk about; but with networking you're potentially letting strangers into your house and they don't necessarily deserve the same privileges as your spouse. Sure, *most* of the time you just get a harmless TypeError, if you get for example a dict where you expect an int, but the more flexible you make your serialization, and the more types you allow network code to pass around, the wider the attack surface of *all* your application code. There's also the issue of resource management. With a JSON-based protocol, "{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{..." is a perfectly valid message which can perform a resource exhaustion attack before you're even done parsing it. With a struct-based protocol, once you get beyond the expected length of the input struct, it's game over for the network peer.

    The second benefit of a struct-based approach is that you can separate out your protocol definition from your protocol implementation. If you ever plan to allow more than one client to access your server, this is a helpful resource to have. With a self-defined protocol that uses simple data structures, it's all too easy to build up messages by passing data structures around to odd corners of your codebase, and it becomes easy to forget about optional arguments or extension structures for certain messages.

    I've written about this in more depth here: http://glyph.twistedmatrix.com/2008/07/static-on-wire.html

    Unfortunately I don't have any recommendations for you, except, you know, "use Twisted" ;-), and probably AMP too.

  8. Mike C. Fletcher

    Mike C. Fletcher on 06/20/2009 11:51 a.m. #

    Currently PyQNet is about 20KB compressed, and I consider that to be a bit heavy. Once I get a presence mechanism in it will likely be close to 30KB.

Comments are closed.

Pingbacks

Pingbacks are closed.