Bug turns out to be a logic problem with parallel operation (It only waits until an op is finished, if it finishes instantly it never waits)
Written by
on
in
Snaking.
Finally tracked down the hanging Twisted code. It was a big operation that happens every 20 minutes. The first time through every operation has to wait for a network connection or 2, so there's always a deferred. Subsequent times none of the (3000+) operations has to wait, but they are very heavy in terms of processing requirements. With the parallel iteration code, those 3000+ heavy operations were all occurring in a single "time slice" as it were. That froze the reactor for 3-4 minutes.
All of this is because I haven't found a decent way to say "run this task with low priority" within the Twisted system. Maybe I should just spelunk through the library for a while to expand my higher-level toolkit.
Pingbacks
Pingbacks are closed.
Comments
Comments are closed.