Dreaming of deterministic profilers future...

Some things it occurs to me that I'd like to see in a new Python profiler:
  • High resolution, accurate, captured data (obviously)
  • Trace every line executed (yes, every line)
  • Trace every call operation, i.e. CALL #32 of function #45 took 0.0003s and have a record of CALL #32 which details what *it* called, its timing, etc.
  • Dump to an efficient low-level binary format (we're going to dump a *lot* of data)
  • Mark each record (or stream, or whatever) with process and thread information
  • Preferably be able to dump to socket *or* file (i.e. make remote live profiling easy)
  • Provide an efficient low-level binary loader that lets us generate indexes on the side (or translate the raw capture data into indexed/queried form)
  • Be able to index such that I can find all instances of X call extremely fast, similarly be able to do subtree queries very fast (i.e. display a treemap of time spent on *this* call of this function)
  • Obviously, be able to index such that threads can be watched separately
  • Possibly provide an API to send "marks" into the profile stream

Basically I want to be able to deep-dive through a run of an application.  I want to be able to play the run through as an animation.  I want to be able to "slice out" sections of the profile so I can see what happened at time X without reference to previous or subsequent times.  I want to be to see the annotated source coloured by execution frequency. To make any of that useful, the data format and structures need to be very efficient, as it will be a *lot* of data (the old Hotspot profiler was a PITA for loading-times, for instance).

Comments

  1. panos

    panos on 10/18/2011 6:41 a.m. #

    It is not exactly what you ask for, but perhaps it could serve your needs, or maybe serve as a starting point

    http://packages.python.org/line_profiler/

Comments are closed.

Pingbacks

Pingbacks are closed.