Programmatic Logging of Performance Data

An Introduction to PDH with Python for Windows

Looking for a better way to log performance data than using screen captures of the Performance Monitor? Looking for a simple way to set up administrative functions based on the values of performance criteria? Microsoft has provided you with a tool for accessing the information, the Performance Data Helper (PDH), but until now, it has meant considerable C programming, access to the MS Developers Network, and a few headaches.

Now Python for Windows, the win32 version of the popular language, has added PDH, wrapped and ready to go, to its substantial toolbox. This article shows you how to create a simplistic PDH logger using this newly wrapped tool. To run the example, you will need:

Which can be obtained from: http://www.python.org/1.5

Start the Python interpreter. 

 import win32pdhquery, string # [1]
 query = win32pdhquery.Query() # [2]

One user has reported a non-reproducible crash on the following line, 
likely due to a problem with the browser dialogue, take appropriate precautions.

 query.addcounterbybrowsing() #[3]  
 query.collectdatawhile(0.25) # [4]

The prompt immediately returns, we are logging the data in the background

 query.collectdatawhile_stop() #[5] 

Wait at least 0.25 seconds for thread to stopUse a valid filename below, the file will be overwritten without warning

 filehandle = open('c:\\temp\\testlog.log', 'w') # [6]
 if query.curpaths: # [7] press enter, then tab, then type next line...
     filehandle.write( string.join( query.curpaths,'\t')+'\n' ) # enter, enter...
 for dataset in query.curresults: # [8] enter,tab ...
     if dataset: # [9] enter, tab, tab...
          filehandle.write(string.join(map(str, dataset), '\t')+'\n') # enter, enter... 

filehandle.close() # [10] (optional) [ctrl]-z # to exit from console-mode Python, use Menu File|Exit in PythonWin

What's going on?

  1. Loads the PDHQuery module for Python for Windows and the standard Python string-handling module.
  2. Create a new, empty PDH Query, the wrapped interface to the PDH system.
  3. Open the standard PDH Counters Dialogue. Within this dialogue (which you may have used in the NT Performance Monitor), select and click on Add for each counter you would like to log. If you have a large number of windows open, the dialogue may show up behind some of them, use Alt-tab to find it (it uses the default windows Icon).
  4. Start a background thread which will log the performance data every 0.25 seconds
  5. Signal the background thread to stop and make its results available
  6. Open the file in which to store the results. We are going to store them in tab-delimited format for easy import into your favorite spreadsheet or database.
  7. Assuming you selected some counters in step 3, write the counter "paths" to the file, connect them with tabs and finish the record with a new line.
  8. Loop over each object in the query's list of current results.
  9. map(str, dataset) gets string-representations of the (numeric) data, otherwise identical to 7
  10. You only need to close the file if you are going to open it in another application while the interpreter is running (file-locks are released automatically when the interpreter is closed)

You can now open the output file in your favorite spreadsheet and analyze or annotate it as you wish. Of course, far more complex examples could be created, and both Python and Python for Windows extensive libraries make it possible to create more significant tools for administering your NT system, but that's another article. For more information, open the win32pdhquery.py module in the win32/lib subdirectory of your Python Directory. The module is heavily documented, and points you to other online sources.