I think I've just discovered an anti-pattern in my code. Unfortunately, it's a really common idiom in the code base. Basically, I've been using dictionaries initialised with all records in a tracking table to make looking up a particular record very fast.

If you're going to be looking up individual records many times, and are likely to look up all records, a good strategy. However, as it turns out, I don't do more than a single query, and I generally access approximately half of the records for any given run (originally the long-running process was going to re-use the loaded records time and time again, just caching the record values to DB periodically to allow for restarting after interruptions, but for various reasons that's no longer happening).

It's not a huge change to rewrite any particular instance of the pattern, but as I mentioned, it's been a common idiom. I've rewritten what looks like the most ugly of the situations (in terms of being able to cause weird side-effects)... it's not markedly faster than the previous version, but it's far less likely to get into pathalogical situations.