Analyze Python cProfile stats created with pstats.dump_stats() off line.

Background

In order to improve Code Performance, I need to find functions that are good candidates for Cython implementation.

Here’s the link to the Python Docs for Code Profiling. Pay attention to cProfile.

I asked and answered this on stack overflow here.

Preamble

Previously I have essentially done the following:

import cProfile, pstats, StringIO
pr = cProfile.Profile()
pr.enable()
# ... my code did something ...
pr.disable()
s = StringIO.StringIO()
sortby = 'cumulative'
ps = pstats.Stats(pr, stream=s).sort_stats(sortby)

ps.dump_stats('stats.dmp')  # dump the stats to a file named stats.dmp

So now I have the file named ‘stats.dmp’ stored off line.

Here’s how I used pstats to analyze the ‘stats.dmp’ file for human consumption off line.

The Code

#!/usr/local/bin/python2.7
# -*- coding: UTF-8 -*-
"""analyze_dmp.py takes the file INFILEPATH [a pstats dump file] Producing OUTFILEPATH [a human readable python profile]
Usage:   analyze_dmp.py INFILEPATH  OUTFILEPATH
Example: analyze_dmp.py stats.dmp   stats.log
"""
import sys, os
import cProfile, pstats, StringIO

def analyze_dmp(myinfilepath='stats.dmp', myoutfilepath='stats.log'):
    out_stream = open(myoutfilepath, 'w')
    ps = pstats.Stats(myinfilepath, stream=out_stream)
    sortby = 'cumulative'

    ps.strip_dirs().sort_stats(sortby).print_stats(.3)  # plink around with this to get the results you need

NUM_ARGS = 2
def main():
    args = sys.argv[1:]
    if len(args) != NUM_ARGS or "-h" in args or "--help" in args:
        print __doc__
        s = raw_input('hit return to quit')
        sys.exit(2)
    analyze_dmp(myinfilepath=args[0], myoutfilepath=args[1])

if __name__ == '__main__':
    main()

I tested this with a .dmp file created on Linux & analyzed on Windows XP. It worked FINE. The above Python file is named, “analyze_dmp.py”.

An Additional Resource

I also found out how to use the Python command line to analyze stats.dmp in an interactive way here.

Here’s the command line i used on Windows XP.

ipython -m pstats stats.dmp

#code-analysis, #code-performance, #cprofile, #cython, #performance-analysis, #python