Hi everyone,

Recently I’ve been working with Web Performance and Load Tests in Visual Studio 2010, with the objective of measuring some metrics of a web site, like the average page time and the average first byte time. To do this, I had to record several web tests to serve as the input of the Load test, which will finally show the metrics we want to measure. In this case, the web tests are based on the System Test Cases developed for this web site, but they also could be based on the use cases in a case where the test cases are more granular.

In a first section in this post, we will see how to easily capture the web requests of a test case using Fiddler, and then export them as Visual Studio Web Tests, so we can manipulate their settings, such as Request Timeout, Credentials, etc., using Visual Studio.

In a second section, we will see how to create a Load Test from the scratch using Visual Studio and feed it with the web tests already recorded, how we can easily manipulate its properties, how to execute it focusing on counters such as Average Page Time and Average First Byte Time and how to interpret the results obtained.

Prerequisites

• Fiddler Web Debugger
• Visual Studio 2010 Ultimate
• A Test Rig (Controller and Agents) properly configured to run Load Tests

Section 1 – Web Performance Tests

Recording Web Performance Tests using Fiddler
Once Fiddler is opened and before recording any request, we will ensure that it’s capturing the Web traffic by checking the Capturing message at the bottom left corner of the window.

Now, we are ready to start recording all the web requests of a Test Case. Open a browser, and log in to your web site (with your credentials if necessary).

Once in it, follow the flow of the test case that you want to record. When finished, go back to Fiddler and you will see all the web requests recorded.

Click Capturing at the bottom left corner of the window to stop capturing the web traffic. You will notice that the label Capturing disappears.

Now that we have our requests recorded, we will proceed to export them as Visual Studio Web Tests to open them in Visual Studio and manipulate their properties. To do this, go to File | Export Sessions | All Sessions. A new window will open to select the Export Format.

In the dropdown list, select Visual Studio Web Test and click Next.

After that, you will be requested to select the folder where you want the web test to be saved and to provide a name for it.

Finally, you have your web test ready to be included in your Test Project in Visual Studio.

Manipulating Web Tests in Visual Studio

The next step will be to modify some properties of the web test already recorded with Fiddler, so it’s ready to be executed, using Visual Studio. In this case, we will focus on modifying the Timeout property of all the requests in the web tests to match a Performance Requirement of 4 seconds, and then we will add the necessary credentials for the requests to properly access the web site.

First, we will include the new web test to our Test Project in Visual Studio. To do this, right click your test project in Visual Studio and select Add | Existing Item. Browse to your .webtest file and click Add.

The web test is added to the project.

Double-click the Web Test that you have already added. You will see the list of recorded requests performed in Fiddler.

Editing the Timeout property

In the web test tree, right click the first request and select Properties. All the properties for that request will be displayed, and among them we will see the Timeout, which is the one we will modify to match the Performance Requirements.

In the Timeout option, insert the required amount of seconds (in this case, 4).

Now, do the same for all the other requests.

Note: To speed up this process of modifying the Timeout for each request, you can press CTRL + H to open the “Find and Replace” dialog, and replace each Timeout=”60” for Timeout=”4” in the entire solution.

Adding Credentials

The last thing to do will be to add the credentials for the web test. To do this, click Set Credentials in the web test Menu.

Now, insert the Name and Password used when recording the test in Fiddler and click OK. Save the web test.

Finally, the web test is ready to be executed.

Updating the Site URL

In case you have to update all your web tests because the URL of the site under test has changed, you can simply press CTRL + H, and replace all the old URLs with the new one.

Note: some web tests can fail if the data used to execute the test wasn’t previously uploaded to the site in the test initialization.

Executing a Web Test

In order to execute the web test, click Run Test in the web test Menu.

Verify that the web test executed successfully. You can check some information about the test execution (Request Time is one of the important metrics).

Section 2 – Load Tests

Creating a New Load Test

In order to create a new Load Test, right-click your test project and select Add | New Test.

In the Add New Test window, select Load Test, provide a name for it and then click OK.

After that, a wizard will be displayed, where you will configure your load test. Click Next in the Welcome section.

The next step will be to configure the scenario where the load test will be executed. You will be requested to insert a name for the scenario and the Think time profile that you want to use. Provide a name, leave the think time profile by default to use the recorded think times in the requests and click Next.

Note: Think time is the time spent by a user perusing a Web page, including viewing the page and determining the next action. Think time does not include the time that is required for a page to load.

In the Load Pattern step, we will have to specify if we want a constant load of users during all the simulation, or if we want to increase the amount of users in steps. The difference between both of them is the duration of the simulation.

By choosing the first one, we should specify a warm up time for the load test, until the user count reaches the inserted value. This will cause the whole simulation to last less time than the one with a step load, because the amount of users will increase lineally until the maximum specified count, with no major complications for the processor of the machine that is executing the test.

By choosing the second one, we will have to specify the start user count, the step duration, the step user count and the maximum user count values. In this case, the duration of the simulation will have to be bigger than the other one, because if we choose a big amount of users per step, the processor of the machine that is executing the tests can start to fail, and the simulation results could not be entirely correct. We also should specify a short warm-up time at the beginning until the start user count is reached.

Note: When you specify a warm-up time, you avoid over stressing your site by having hundreds of new user sessions hitting the site at the same time. Also, the results obtained during this time will not be taken into account for the final results.

After choosing the load pattern, click Next.

The next thing to do will be to choose the Test Mix Model that we want to use during the execution of the load test. This model specifies the probability of a virtual user running a given test in a load test scenario. This lets you simulate load more realistically.

Choose the Based on the total number of tests mix model, so we can select the distribution for each web test that we want to run in the load test in the next step, and click Next.

In the Test Mix section we will be able to add the web tests that the load test will use in the simulation to hit the web site. Also we will choose the distribution of the web tests, to indicate which ones we want to occur more times than others.

To add the web tests, click Add.

In the Add Tests window, choose the web tests included in the current project that you want to be executed during the simulation and press >. Click OK to close the Add Tests window.

Back in the Test Mix section, choose the distribution for each web test and click Next.

The next step will be to choose the Network Mix, which is a combination of two factors: the selection of networks that are contained within the scenario, and the distribution of those networks within the scenario.

Add the networks that you would like the load test to mix, select their distribution and click Next. By default, we will only use LAN.

Another thing to specify is the Browser mix. This let us send the requests through different types and versions of browsers. We can add all the browsers that we want the load test to use and their distribution. By default, we will only use Internet Explorer 9.0. After selecting it, click Next.

Note: you will notice that Internet Explorer 9.0 is not loaded by default in Visual Studio. In order to add other browsers to test, take a look at this article.

In the Counter Sets section, we will add the controller computer and agents that will execute all the web tests in the load test. These ones will be available according to the configuration of the Test Rig that is required to execute Load Tests. In this case, we choose only one agent (which is the only one configured in the machine). Click Next.

The final step in the creation of a Load Test is to specify the Run Settings. You will be able to choose between Load test duration, where a warm-up time and a run duration will be required, and Test iterations, where you can select the number of times to run the test. By default, we will choose Load test duration. Provide a long warm-up time if you have chosen a constant load of users in the Load Pattern section of the wizard, or a short warm-up time if you have chosen a Step load of users. After providing also the duration time, click Finish.

Our Load Test is finished and added to our test project. Now we are ready to execute it.

The recommendation for a proper simulation is to choose a constant load of users (the maximum available in Visual Studio is 250 without license, and packages of 1000 users per license), a long warm-up time until the maximum specified amount of users is reached, and almost 15 minutes of simulation with the system stressed with the maximum load.

Manipulating Load Tests in Visual Studio

Once we have our load test created, we can take a look at its properties, and modify them as we wish. This is useful to, for example, edit the total duration of the simulation, edit the warm-up time, add new web tests to the Test Mix, select Initialize tests and Terminate Tests, etc.

Updating the Scenario

In the load test tree, right-click the scenario and select Properties. The Properties pane should display.

Select Test Mix Type and click the Ellipsis next to the text mix type. The Edit Test Mix window should appear.

Here we can change the text mix model by selecting one of them in the dropdown list, add more web tests to our set by clicking Add, and also change their distribution.

Note: an alternative way to add new web tests to the test mix is by right-clicking Test Mix under the scenario in the load test tree, and selecting Add Tests. Analogously, the same can be done to add more browsers to the Browser Mix, or networks to the Network Mix.

Apart from that, we can set an Initialize test and a Terminate test for all the web tests in the list. This means that before every web test that a user will execute in the simulation, the Initialize test will execute, and after the execution of all the web tests, the Terminate test will do so.

At the beginning we thought that this could be a good way to initialize the environment where we were working on in order to isolate the tests execution, but then we found a disadvantage that forced us to discard it:

• Only one web test can be selected to run as Initialize test

What we really needed was to execute several web tests to initialize the environment before the execution of each web test in the load test.

Finally, another thing to consider is that we can change the Load Pattern from the load test tree. Just right-click the load pattern and select Properties. Here you can select the pattern you want from the dropdown list, and also manage its properties.

Updating the Run Settings

In the Run Settings section, you can basically edit the duration of the simulation and the warm-up time, among other things. Right-click the Run Settings in the load test tree and select Properties. You will be able to edit them in that pane.

Executing a Load Test

Now it’s time to execute our load test. To do so, click Run Test in the load test Menu.

The test will start to run, showing different graphs with the progress of the simulation. Those graphs show the current selected counters (you can find them in the Counters pane at the left side of the screen after expanding the nodes), such as User Load, Pages/Sec, Avg. Page Time, etc.

An important thing to mention is that all the counters offered by Visual Studio are recorded during the simulation. We choose only the ones we need to show in the graphs, tables, etc., but we can also add new ones to the report after the simulation is finished.

Interpreting Load Test results

When the load test finishes, all the results are stored in the database, and Visual Studio generates a general report. In this section, we will review most of the important things on it.

Summary View

Once the results are stored in the database, a Summary of the simulation is displayed.

An important section in this summary is the Overall Results, where some of the important counters are displayed. You can check the other ones in the Tables view.

Note: you can change the layout of the reports in the Load test menu. The different views are Summary, Graphs and Tables.

Graphs View

In this view, you can select the graphs that you want to display. The tables below them will update automatically.

The Key Indicators graph shows the behavior of the selected counters for this simulation, and it shows metrics such as minimums, maximums and average of each counter. If you compare this graph with the one shown in the Executing a Load Test section of this post, you will notice that the results obtained during the warm-up time (5 minutes) were removed.

Note: the Range column in the Key Indicators table shows the axis range of that counter in the graph.

Counters

• Pages/Sec: it measures the hard page faults per second. A hard page fault occurs when a memory page is not in the immediate memory and needs to be fetched from the disk.
• Avg. Page Time: is a measure of the average time to download the page and all of its dependent requests, such as images, css, js, etc.
• Avg. Response Time: is a measure of the average time that took to give a response after a request.
• Avg. First Byte Time: is a measure of the average time that took to start receiving data back from the server.

Note: the Average First Byte Time is not displayed by default. To display this counter in the table and the graph (as shown in Figure 41), you have to expand the Overall node in the Counters section at the left side of the screen, then expand the Request node, and then double-click Avg. First Byte Time.

The Controller and Agents graph shows the performance of the processor and the memory of the machine that is executing the load test. This is an important thing to consider, because if the processor starts to fail more often, the metrics showed in the Key Indicators graph are not entirely correct. So, this indicates the time where the simulation performed correctly, and the time where the results are not entirely reliable (if the processor failed during a considerable time). When giving a report, we should only consider the first case.

Tables View

Finally, you can check the errors and the statistics of how many times each web test failed or passed within the execution of the load test in the Tables view. By following the links of the failures, you will get more information about the errors.

Sources

• How to: Visual Studio web performance tests – not using Internet Explorer
• How to: Create a New Load Test Using the New Load Test Wizard
• Configuring Test Controllers and Test Agents for Load Testing
• How to: Create Run Settings in the New Load Test Wizard
• Considerations for Load Tests
• How to: Edit the Text Mix Model Using the Load Test Editor (to add Initialize and Terminate tests)

I hope you can find all of this useful!

Thanks!