It really seems there should be something in Twisted that does something along these lines, but I can't find anything quite right:

Create and monitor a pool of pb (twisted.spread) connections (this is to spread a CPU-intensive operation across many servers)

On disconnect or connect failure, do retries (forever), but with scaling frequency (i.e. if a server goes away for a day or two it should be picked up when it comes back online, but we shouldn't pound the machine the entire time with reconnect attempts)

Monitor the number of pending calls on the connections, provide way to return the (active) connection with the least number of pending calls

Provide API so that the pooled PB connections act like a regular "perspective" instance from client code

I've been hunting around and coding up half-arsed solutions trying to reuse bits of Twisted for the last 3 hours on this. I'm thinking maybe I'll try to code something fully-arsed instead. Not what I was planning to work on, but oh well, it'll probably come in handy some other place too.