When modeling a Web Application’s responsiveness with an increasing User Load, you usually configure a Load Test Mix with a Step based Load Pattern, run the test until the Response Time goes above a certain response time, and then manually stop the test. This requires some of your time monitoring the test progress and having to intervene to stop the test.

Wouldn’t be nice if this was automated? Fortunately this can be easily achieved through the use of a custom Load Test Plugin (ILoadTestPlugin) and a configured Threshold. (Custom load test plugins were improved in VS2008SP1 and are also available in VS 2010, used in the creation of this blog post.) This will save you (the Tester) the trouble of having to manually monitor the load test and stop it when the threshold is exceeded.

Let’s take a look at how this can be implemented in code:

Creating a Load Test Plug-in that stops the Load Test when a threshold is exceeded

Load Test Plug-ins allow custom code to be executed when a load test starts, ends, or when a threshold was exceeded, among other events. We’ll create a Custom Load Test Plug-in that attaches itself to the ThresholdExceeded event and stops the test.

1. Create a Class Library project.
2. Add a reference to the Microsoft.VisualStudio.QualityTools.LoadTestFramework assembly.
3. Create a class named CustomLoadTestPlugin and complete it with the following code (VS2008 and VS2010):

   using System;
   using Microsoft.VisualStudio.TestTools.LoadTesting;
   
   public class CustomLoadTestPlugin : ILoadTestPlugin
   {
       private LoadTest test;
   
       public void Initialize(LoadTest loadTest)
       {
           if (loadTest == null)
           {
               throw new ArgumentNullException("loadTest");
           }
   
           this.test = loadTest;
   
           loadTest.ThresholdExceeded += new EventHandler<ThresholdExceededEventArgs>(OnThresholdExceeded);
       }
   
       void OnThresholdExceeded(object sender, ThresholdExceededEventArgs e)
       {
           if (e.ThresholdResult == ThresholdRuleResult.Critical
               && e.CounterValue != null
               && e.CounterValue.CounterName == "Avg. Test Time")
           {
               this.test.Scenarios[0].CurrentLoad = 0;
               this.test.Abort();
           }
       }
   }

4. Build the Load Test Plug-in solution

Note: If you’d like to get more information about creating a Load Test Plug-in, see the following link.

Using the Custom Load Test Plug-in

1. Open the Load Test Configuration (.loadtest) you’d like to modify.
2. Using Solution Explorer, add a reference to the Load Test Plug-In (either to the Load Test Plug-In project or to the compiled assembly).
3. Add a Compare Constant threshold rule on the Avg. Test Time counter. To do this, Expand the Counter Sets | LoadTest | Counter Categories | LoadTest:Test element, right click on the Avg. Test Time counter and select Add Threshold Rule.
4. Add the Load Test Plug-In to the test run. To do this, right click on the Load Test Name (the top element) and choose Add Load Test Plug-in. On the window that opens, select CustomLoadTestPlugin and hit OK.

   

5. That’s it, you should now run the Load Test and see if it stops after your configured threshold.

Hope this saves you time. If you’d like to learn more ways to extend Visual Studio Load Testing, check Bill Barnett’s  article.

Related Articles

• Visual Studio 2010- How to create a Performance Data Collector

Happy testing!

Let’s assume you often run Performance Tests and your application outputs a log file that you manually backup on every test run. You can automate the process using a Collector, a new feature that ships with Visual Studio 2010.

Note: For a great introduction on Data Collectors in Visual Studio 2010 Beta1, please check Amit’s blog.

This post is meant to answer the question: How can I create my own Data Collector in Visual Studio 2010?

Steps to create a Performance Collector

Let’s create a Performance Collector that generates a log file and copies it to the Test Results folder:

1. Create a Class Library Project (C# 4.0).
2. Add a reference to Microsoft.VisualStudio.PerformanceTools.DataCollection and Microsoft.VisualStudio.QualityTools.ExecutionCommon. Both assemblies can be found in the %DevEnvDir%\PrivateAssemblies folder, and should only be used for extending Visual Studio.
   
3. Create a new class. Assign it a name, like CustomLogDataCollector.
4. Add the following using declarations above the class name.
   
5. Make the class inherit from DataCollector (Microsoft.VisualStudio.PerformanceTools.DataCollection namespace).
6. Decorate the class name with the DataCollectorFriendlyName, DataCollectorTypeUri and DataCollectorDescription attributes. The Uri needs to be unique for Visual Studio to identify the Collector, it is usually created in a Company/CollectorName/Version hierarchy, as shown below.
   
7. Add a property of type IDataCollectorSink and implement the DoInitialize method to capture the IDataCollectorSink value. An IDataCollectorSink allows the Collector to interact with the Test Results folder.
   
8. Override the SessionEnd method and invoke the IDataCollectionSync’s SendFileAsync method to copy the log file contents into the Test Results folder. In the example below I’m simulating a log file by creating a HelloWorld.txt file with a log line. It’s very easy to adapt this to your own environment!
    
9. Build the project. Copy the project’s DLL into the %DevEnvDir%\PrivateAssemblies\DataCollectors folder.

That’s it! Now it’s time to enjoy your first Custom Collector. 

Using the custom Data Collector in your Test Project

To use the Data Collector for your Test Project you just need to:

1. In Solution Explorer, double click on the “.testsettings” file (the project’s test run configuration) to open the configuration editor.
    
2. Select the Execution Criteria configuration from the list on the left. In the Collectors section on the right, scroll down until you see “Your custom collector1″. Make sure the Enabled checkbox is selected and click Apply and then Close.
3. Start a Load Test run.
   
4. Wait until the Load Test run is completed. On the Test Results window, click the Test run completed link.
   
5. Verify that there is a file named HelloWorld.txt in the Collected Files section, as shown below. Clicking the link will open the file in your configured text editor.
   

Next steps

This was a quick introduction on how to create custom Collectors for your Test Projects. Now that you have it working, you can augment it by creating:

• An installer that copies the DLL file to PrivateAssemblies\DataCollectors folder
• A class to hold the Collector’s configuration
• A visual editor for the Data Collector’s options

I hope you can find a good use for this, and your feedback is greatly appreciated.