In this post I’m going to cover adding performance counters to your automated tests.
Performance counters can be invaluable in analysing your application’s behaviour while running tests. If you are lucky enough to use Visual Studio Enterprise you can add counters to your load test with just a few clicks.
Since I’m using a different test runner I have to add them manually as part of my test run, I’m going to use PowerShell and LogMan to do this. You can create the data collectors entirely in PowerShell but there are loads of properties you can set on a data collector and it quickly becomes unmanageable. Instead I’m going to create the data collectors manually and use LogMan to export them and then import them on the test machine as part of the test sequence.
Step 1
Create your data collectors using Perfmon just like you normally do, take note of the names and optionally specify the path to store the counter file.
When you are done, right click on each data collector, select Export Template and save it to a file, keep the convention of CounterName.xml file, you will later see how this makes the script simpler.
Step 2
In my environment the VMs are reset every day so I scripted the import of the data collector templates at as part of my VM rebuild. I gave each exported XML file the same name as its data collector, if you follow this convention it makes it very easy to add new ones by adding it to the $counters array.
$counters = @("CPU","ASP.NET","Disk","Memory","SQL")
foreach ($counter in $counters)
{
Write-Host "Creating Counter $counter"
$create = "logman.exe import -name $counter -xml $counter.xml"
Invoke-Expression $create
}Step 3
After the previous step the data collectors are created but not started yet. I want to start, stop and retrieve the data collectors for each set of tests I run. At the start of a test run I clear out the data collectors folder and start the data collectors.
Remove-Item c:\logs\perfmon\* -recurse -force
$counters = @("CPU","ASP.NET","Disk","Memory", "SQL")
foreach ($counter in $counters)
{
Write-Host "Starting Counter $counter"
$start = "logman.exe start $counter"
Invoke-Expression $start
}Step 4
At the end of a test run I stop the data collectors and collect the files to include them as artefacts in my build system. You have to stop the data collectors before you attempt to copy or even view the reports of the data collector.
$counters = @("CPU","ASP.NET","Disk","Memory", "SQL")
foreach ($counter in $counters)
{
Write-Host "Stopping Counter $counter"
$stop = "logman.exe stop $counter"
Invoke-Expression $stop
}I had a problem copying the files due to the permissions Windows assigned to the data collector folders when it created them, I used the following script to assign permissions to the appropriate user group.
$Path = "C:\Logs\Perfmon\"
$ACL = (Get-Item $Path).GetAccessControl("Access")
$ACE = New-Object System.Security.AccessControl.FileSystemAccessRule("YourUserGroup", "FullControl", "ContainerInherit,ObjectInherit", "None", "Allow")
$ACL.AddAccessRule($ACE)
ForEach($_ In Get-ChildItem $Path -Recurse)
{
Set-Acl -ACLObject $ACL $_.FullName
}Now you can view the counter files using PerfMon. In my case I added them to the artefacts of my build step in TeamCity.
Tip: There is actually a lot more functionality to data collectors than most people know, you can for instance start and stop them using various triggers including a schedule. You can manage the growth and archiving of the data collector files and even create html reports. You can for instance reset the counters daily, archive the log files, generate a report and email it. These settings are controlled from the Data Manager.
Analysing Counters
There are loads of counters available and it is sometimes difficult to determine if the values are indicative of a problem. Luckily there is are tools to help you. I use PAL tool. It is fully scriptable and generates HTML reports so you can include the analysis of the counters in your build step.
