Web Performance Test Using Visual Studio- Part IVisual Studio is one of the tools used for Performance Test. Visual Studio Test edition or Visual Studio 2010 Ultimate provides the support for test automation. This article describes the web test feature available in visual studio.

Introduction

Performance of an application is very important for a multi-user application. Performance is not only the speed of execution; it includes the load and concurrency aspects. Performance Test is an answer for the following questions

How can we ensure that our new application will support the expected user load? How to avoid the issues which will arise only in real load conditions? How to find the response time? How to plan the capacity of the servers?

Visual Studio is one of the tools used for Performance Test. Visual Studio Test edition or Visual Studio 2010 Ultimate provides the support for test automation.

This will be the first part of a series of articles.

Part I: Web Test and Customization Part II: Load Test, RIG and Load test terminologies. Part III: Performance Counters, Load Test Result Analysis.

VS 2010 Test Types

Visual studio support various test types to perform the test automation. Figure 1 shows the main Test types available in VSTS 2010.

Basic Unit Test, Unit Test and Unit Test Wizard helps in creating new Unit test for a class file. This will be helpful for both developers and testers to perform unit level testing.

Coded UI test is used for recording the UI activities of a manual test. 

Database Unit Test used for testing stored procedures and functions.

Generic Test is for wrapping an executable as a test method. You can wrap the executable using generic Test and can include the same in test automation.

Ordered test is for executing multiple test scripts in a particular order. 

Web Performance Test used for recording the URLs and generating the code for performance testing.

Web Test

Web Performance Test or Web test allows performing the web application test. We will create a test project by selecting the Test project template.

Once you created the test project right click on the project and add new Item. Select the web performance test, which will open the recorder in an internet explorer and start recording the navigation URLs and associated parameters.

Recording

Following figure shows the Web test recorder, records the Google search operation.

In this example, we are recording the search feature of Google. Once the recording is over, stop recording using the Stop button in Web test recorder. This will generate the recorded URLs as a web test and add dynamic correlations to.

Correlation

Correlation is nothing but linking the response of one web request to the next web request.

For example, when you login to the site it generates a SID for tracking the session. This SID is passed to the client after the login request. Along with the next request, the stored SID will be send to server. When you record this operation using Web test, it records the values and saves as hardcoded value. SID will be different for the next run.

For avoiding such situation, correlate the SID value from the response of the login request to the next request parameters.  First we will extract the SID values and save in a context parameter, which will be passed to the next request as a parameter. 

Context Parameter

Context Parameters is just like to global variables. If you want to refer one parameter across all URLs, declare the same as Context Parameter.

For example, we need to run the script in multiple environments. Instead of recording the script for each environment separately, define the context parameter as ‘Webserver’ and use the same for URL formation. Context Parameters should be referred inside double curly brackets like

{{Webserver}}. All the URLs should be modified using the same context parameter. When you want to run the script in another environment, modify the value of the context parameter, which will automatically take care of the URLs.

The query string we used for searching is recorded as part of the URL recording.

In test environment the input value will differ depends on the positive testing, negative testing, boundary testing, etc. For passing multiple values to the parameters, we will do the parameterization.

Parameterization

For parameterizing the arguments, add the data source from where we can pick the values. Right click on the Web test -> Select Add Data Source option.

Select the type of data source. Data Source can be a Database like SQl Server, Oracle and Excel, or data can be fetched from CSV or XML files too.

Select the Data file or the database and table contains the input data. Preview of the data will be displayed on the wizard.

Click on the query string and move to the properties of the querystring. Change the value using the Data Source mapping as below.

Data source binding will be displayed as the query string value

Extraction Rule

Extraction rules are used for extracting the data from the response of a request. We have the following options for extracting the values; extract the form fields, extract HTTP header, etc. Extracted values can be used as part of the next web request or can be used for making any business decisions.

In following example, I used Extract Text option to extract an id passed from the server. By analyzing the html output of the request, you can form the Starts With and Ends With frames. The HTML response is displayed as part of the result window(will discuss soon).

Validation Rule

 Validation rules are to enforce you are in the correct page only. After SignIn or Login In, you may be expecting a page with SignOut text. Following validation rule validates, whether the current response contains the text “SignOut’ or not.

We can form the validation rules using any of the following options. Visual Studio 2010 and 2008 automatically add the Response URL validations to the recorded test.

Transaction

 Transactions are a set of operations or round-trips required to perform a particular operation. For example, the process of buying a Book consist of steps for select a Book, Add to cart, Check out and Payment.

Defining transactions will be helpful for analyzing the results. Normally the response time, Response Byte, etc. will be displayed for each URL separately. Once you define the transactions, the response time and all other measures will be displayed for the transaction level.

 

Run Test

When you run the test, you will get the following screen, where you can see the status of each URL, how the result appear in Web Browser, what are the parameters passed as part of the request, HTML response, context parameters and the details.

 

This is related to single run. If you want to check the parameterization to want to run the test multiple times, click on the Edit run Settings option displayed on top the screen.

Here, you can specify the number of times the test needs to be run. Each run will pick one record from the Parameter data source and run the test. Along with run count, we can specify the browser types too; which simulates how the site appears in different browser.

After setting the run settings, select the Click here to run again option specified on top of the screen.

Generate Code

 Generate code option allows you to create the code corresponding to the script.

This will generate the C# code corresponding to the web test and also create a separate Test itself. We can modify the code project without affecting the web test and vice versa.

We can use the C# capabilities to automate or customize the coded web test. Can use loop for iterating one operation or ADO.Net to connect to database and pull some data for request, enforce think times (will discuss in Part II) and process WCF services.

Conclusion

Web Performance Test can be used for recording the website navigations and generating the Code. We can use this test or the generated coded test as an input to the performance test. Customization of the test can be done using

                Parameterization – makes the user inputs dynamic

                Correlation – Extract the Server generated data for subsequent requests.

In second part, we will look into the Load test aspects and on third part of this article series is for analyzing the performance test results and understanding the major performance counters.

Web Performance

Introduction

Performance of an application is very important for a multi-user application. Performance is not only the speed of execution; it includes the load and concurrency aspects. Performance Test is an answer for the following questions

How can we ensure that our new application will support the expected user load? How to avoid the issues which will arise only in real load conditions? How to find the response time? How to plan the capacity of the servers?

Visual Studio is one of the tools used for Performance Test. Visual Studio Test edition or Visual Studio 2010 Ultimate provides the support for test automation.   This will be the second part of a series of articles.

Part I:  Web Test and Customization Part II: Load Test, RIG and Load test terminologies. Part III: Performance Counters, Load Test Result Analysis. 

Load Test

Load test is used for performing load test on an application. It simulates multiple users as virtual users and executes the test scripts to simulate the real user load on different servers like Application Server, Database server and Web server. Load test can be used with any of the test scripts.

Different Terminologies

 

Think Time

Think time is the time taken between two requests. This can be the time taken by the user to fill a form, view a page or reading some text etc. Think times are used for simulating the real user scenario; how the system will work with a real user.

Constant Load

Constant load means same number of users hit the site from the starting of the test till the end. It is like 25 users are using the system for a period of 1hr. All 25 users are hitting the system continuously. This type of testing is mainly used for stress testing.

Step Load

In step load, users will join in a step manner. This is the same as different users hitting the system in various times and the number of users in a system is not constant. Following parameter needs to be specified in step load pattern

Start user count :- How many user should hit the system at the time of test start Step duration :- After how many seconds the next users should join the system Step user count :- How many user needs to join the system after step duration Maximum user count :- what is the maximum user count.

Test Mix

Test Mix specifies how different scenarios are executed or used in the system. Different test mixes are formed by studying the system’s usage.

For example, if we are doing a load testing on online marketing site; around 60% people will search for different products, 30% will buy few products and 10% bookmark the products. From this usage information, we can form a test mix as 60% simulated users will execute the script for search, 30% will execute the script for buying a product and remaining 10% will execute the script for bookmarking a product.

Load test Creation

Right click on the project and select Add -> New Test -> Load test template. It will open the New Load Test Wizard

Next, specify the scenario name like booking the ticket, searching a book, etc. Also specify the Think Time profile. We can either use the think times recorded as part of the web test recording or can use a normal think time distribution. We can also avoid the think times recorded in the web test using the third option.

Next, specify the Load Pattern. Specify whether we are going to use a constant load or a step load pattern. In constant load specify the user count. In case of step load specify the start user count, step duration, step user count and maximum user count.  

Next, specify the test mix model. The different options are [Following definitions are from Visual Studio]

Based on the total number of tests This model of test mix determines which test is run when a virtual user starts test iteration. At the end of the load test, the number of times that a particular test was run matches the assigned test distribution. Follow this model when you are basing the test mix on transaction percentages in an US log or in production data. 

 Based on the number of virtual users This model of test mix determines the percentage of virtual users who will run a particular test. At any point in the load test, the number of users who are running a particular test matches the assigned distribution. Follow this model when you are basing the test mix on the percentage of users running a particular test

Based on user pace Over the course of the load test, each test is run the specified number of times per user per hour. Follow this model when you want virtual users to run tests at a certain pace throughout the load test.

Based on sequential test order Each virtual user runs the tests in the order that the tests are defined in the scenario. The virtual user continues cycling through the tests in this order until the load test is complete. 

Next, specify the Test Mix. In Test Mix, we are adding different web test scripts to the load test. If we have only one script, all the users are going to perform the same task. If we have 2 or more scripts, we can specify, how many user needs to perform a specific task.

For example, we have two scripts, one for searching a book and another for buying a book. In my load test, I can specify that 70% of users to do search and remaining 30% do the other operation.

Test mix is also for simulating the real user experience. Some of the features are used by many users and others may not be used that much frequently. Depends on the feature usage, we can test the system, which will give a real performance result. 

In the following example, we selected two web test scripts and specified the test mix as 65% of the users will execute Webtest1 and 35% will execute Webtest2. 

Next, specify the network Mix. Here we can simulate the network like LAN, WAN or internet and compare the performance of the system in different networks. 

Next, specify the Browser mix. This will be useful to find how the system is performing in different browsers. 

Next, add the systems used as web server, application server and database server for collecting the performance counters. Add the required performance counters for each system. 

Next, specify the duration of the test or how many iterations are required. If a warm-up period is set, the load test will automatically increase the load gradually during the warm-up period. 

Once, you finish the creation of the load test, it will create a .loadtest file like below. We can edit and add all the settings directly from the below screen itself.

Select the constant Load Pattern and select the corresponding properties. From the property grid, we can change the load pattern as Constant, Step or Goal based.

Goal based load test is used to find the maximum user load which satisfies the condition. Here, we are setting the goal like average response time should be 8min; once the system reaches the goal, it stops execution. Same way we can use goal based testing to find the maximum load which support a processor utilization of 70%. 

 

 

From the Run Settings properties, we can alter the run duration, web test connection pool size, web test connection model, warm-up duration, etc. 

 

We can specify the threshold value for counters. Once it reaches the warning level, it will start displaying warning. When it reaches the Critical threshold value, the requests will start failing. 

 

You can add custom counters and counter sets to the Counter Sets collection.

From the Scenario properties, we can alter the Think time profile and associated think time.

 

Execute Load Test

Execute the load test by selecting Run test. 

Once the test started, you can observe different performance counters and test status in the following screen. You can drag the counters from left side and drop it in the graph area, which will display the graph corresponding to the counter. Requests summary and tests summary can be observed under Overview section. 

We can observe the requests, errors, pages, transactions and the details like how many failed, response time, content length from tables option.

You can change the graph display options to only one graph, two vertical graphs, four vertical panels, etc.

Result

Once the load test execution completes, it display the summary report as follows.

 

  We can export the report to Excel using Create Excel Report option. It creates an excel report with multiple sheets representing average response time, runs, page time, etc. We can create two types of reports.                 Trend – create an excel report with the trends in the selected run result.                 Comparison - create an excel report by comparing two or more run results.

  Detail option will allow us to study the result in granular format. We can select the time period from the bottom area and study the trend in that particular period. 

Rig

Sometimes, in performance test, we need to test the system with very high load.  Each machine is having limitation in generating the load. From one system we can go a maximum of 500 to 800 user load. If you need more load, we need to add multiple systems to perform the load test. If we are running it from multiple systems, then the analysis of test results and creation of a combined report will be difficult.

Rig is the solution for running the load test from multiple client systems. We need to install the Load controller on one system and Load agent on other systems. VSTS Rig means a group of systems, which consist of one controller and one or more agents. Controller will distribute the work among the agents and will collect the data from all agents and create a single report.

Load Controller and Load Agent installation and configuration, please refer http://blogs.msdn.com/b/edglas/archive/2007/02/17/load-agent-and-load-controller-installation-and-configuration-guide.aspx 

Conclusion

Load test can be used for performing different performance tests on a system. We can test the system by setting different test mixes, user loads, stress conditions, network mixes, browsers and load patterns. Load test will generate a detailed report, which can be queried from the LoadTest or LoadTest2010 database under SQLExpress. Also, from the result we can create an Excel Trend report or Comparison report. Summary report can be saved as HTML file.

In third part, we will look into the important performance counters and analysis of the performance test result.