silk performer presentation v1
TRANSCRIPT
Silk Performer – Day 1
Types of Load Testing
Performance testing
Capacity testing
Endurance testing
Volume testing
Stress testing
Regression testing
Performance testing
The load is gradually increased during the test by adding more and more concurrent virtual users.
The following parameters should be monitored throughout the test and compared for different test phases:
• Web application response time;• Number of HTTP requests or application-specific transactions
processed per second;• Percentage of failed requests (error rate).
Capacity testing
Capacity tests are executed to find out how many concurrent users the application can handle without degradation of quality.
Virtual users are added gradually during the test, but in this case we know the quality criteria in advance and just need to check that they are observed.
Stress testing
Every system has a capacity limit. When the load goes beyond it, the application starts responding very slowly and produces errors.
The goals of stress testing are:
• Find the capacity limit;• Check that when it is reached, the application handles the stress
correctly: produces graceful overload notifications and does not crash;
• When the load is reduced back to regular level, the application should return to normal operation retaining all its performance characteristics.
Volume testing
Volume tests are targeted at loading the application with significant amount of data and maximizing the complexity of each transaction.
For example, if the application can upload files, try using very large ones. If it has a search function, try complex keyword combinations and queries producing a very long list of results.
Endurance testing
This type of testing (also called “soak testing”) is used to check that the system can stand the load for a long time or a large number of transactions.
It usually reveals various types of resource allocation problems. For example, a small memory leak will not be evident from a quick test even with a high load.
For endurance testing it is recommended to use changing periodic load and long duration.
Regression testing
Make the load testing a part of your regular development process by creating regression load tests and applying them to every new version of your application.
Overview of Silk Performer
Silk Performer, a Micro focus product enables you to create powerful, realistic load tests for users across a range of application environments, including the latest web and mobile technologies.
It simulates peak-loads of any size from multiple geographies without you having to invest in load testing hardware and setup.
Silk Performer is used to assess the performance of Internet -servers, database-servers, distributed-applications and middleware, both before and after they are fully developed.
Silk Performer helps you to quickly and cost-effectively produce reliable, high-quality applicationapplications.
Load Testing with Silk Performer
Silk-Performer reduces the personnel requirements by replacing human users with virtual users or Vusers. These Vusers emulate the behavior of real users—operating real application
Because numerous Vusers can run on a single computer, Silk Performer reduces the hardware requirements
The Silk Performer Work Load allows you to easily and effectively control all the Vusers—from a single point of control
Silk Performer Features
General It Is a powerful, flexible, load/performance testing tool with a user-friendly GUI Simulates modifiable virtual users Requires minimum hardware resources to simulate hundreds of concurrent users
Recording Covers the entire functionality needed to record all supported protocols using a single recorder Provides a high-level script summary Generates scripts
Reporting Presents information in tabular and graphical form Provides predefined charts Uses results analysis that is drag-and-drop enabled Allows for report customization Combines repository (Silk Central Test Manager or SCTM) and time series data Allows for data export for third-party reporting tools integration
Silk Performer Features (Cont.)
Simulation Supports multiple protocols and modem speeds including DSL and cable modem Supports agents located inside or outside a firewall Provides automatic load distribution to all participating agents Supports six models of load tests Provides for multiple scripts per agent Allows for verification under load conditions Creates client/server log and visual log
Monitoring Runs in real time Provides Agent Health Control Monitoring Supports integrated server monitoring Provides instantaneous generation of real-time data charts
What’s New in Silk Performer Ver. 15.5
CloudBurst VPN
Silk Performer CloudBurst VPN allows you to comfortably load test a machine or an application on the machine that is part of your company network using cloud agents
Since company networks are usually protected by a firewall, cloud agents that are supposed to put load on a company machine are not able to connect to the system under test
The incoming traffic is blocked by firewall, Configuring a CloudBurst VPN resolves this problem
Optimize Web Pages with Page Speed
The Page-speed integration helps you identify areas for potential performance optimization by comparing your web pages against proven best practices for web site development
TrueLog Explorer displays an additional tab, containing the PageSpeed score rating along with recommendations about how the performance of the web page in focus can be optimized
What’s New in Silk Performer Ver. 15.5 (Cont.)
Trying Out Agents
Silk Performer 15.5 introduces the new Try Agents functionality, which allows you to verify thatyour test scripts work on all agents you are planning to use for your load test.
This new feature resembles the Try Script functionality, which is used to verify that a recorded orwritten script works correctly during replay, whereas Try Agents is used to ensure that the scriptworks on a variety of local and cloud agents.
The Try Agents feature is typically executed before you start the actual load test to ensure that your test environment is setup correctly
Graceful Ramp Down
In Silk Performer 15.5, you can define Ramp Down parameters, which allows you to gradually stop the running virtual users.
Consequently, the virtual users' end transactions are not executed all at once, but in a staggered fashion
What’s New in Silk Performer Ver. 15.5 (Cont.)
Web-Sockets
The Silk Performer 15.5 you can now also test connections that make use of the Web Socket protocol
The Web-Socket protocol allows to establish a full-duplex (bidirectional) connection between client and server and is an alternative to communication models such as polling and long-polling
Browser Driven Web Load Testing Enhancements
Silk Performer offers a new BDL API function (BrowserWaitForNewWindow), which waits until a new browser window is created and then returns the handle to this window. This function is especially useful when you record web applications with asynchronous communication
The core engine for browser-driven projects has been upgraded to the latest Silk Test web tech-domain
What’s New in Silk Performer Ver. 15.5 (Cont.)
JavaScript Pop-ups in TrueLog Explorer no longer visibleo When navigating from node to node in TrueLog Explorer, occasionally JavaScript messages pop
up. As these messages are of no use in this case, Silk Performer15.5 suppresses them. This is true for the messages caused by the JavaScript events on before unload, alert, prompt, input, and confirm
Automatically Remove Used Data from a Data Fileo You can now let Silk Performer remove used data from a data or CSV file. The Parameter Wizard
allows you to enable this option or you can manually script the BDL function FileGetNextUniqueRow and set the new parameter bRemoveUsedRow to true.
Rule-Based Content Rendering in TrueLog Explorero For web protocol-based TrueLogs, TrueLog Explorer now allows rule-based content filtering, for
example for displaying HTML fragments
Improved Script Generation Workflow for AMF3 and Granite DS Application Typeso When recording AMF3 or Granite DS applications, the captured traffic is stored in a capture file.
Using the offline script generation approach, Silk Performer now helps you to adjust Java related settings correctly, so that fully readable and customizable scripts can be generated
Questions That Silk Performer Can Help You Answer
How many simultaneous users can my server support?
What response times do my users experience during peak hours?
Which hardware and software products do I need to ensure optimum performance from my server?
Which components are the bottlenecks in my system?
What is the performance impact on my system of employing security technology?
Which areas of my application perform adequately, and which areas contain bottlenecks in the forms of business transactions, objects, and operations that can be evaluated?
Which factors affect performance? What effects do they have? And at what point do such factors impact service levels?
Silk Performer Scripting Process
BDL (Benchmark Description Language)
Record – Generate a Silk-Performer BDF Script
Replay – Replay a generated Silk-Performer BDF Script
Parameterization – Defining the Parameters necessary for a complete set of relevant
Specifications of a Model
Correlation – Observed Relationships between instances of two events
[For Example: To Handle the Dynamic-Data (SessionID)]
Applying Transactions - Different set of User Transactions
Try Script – A Try Script run determines if a script accurately recreates the actions that you
recorded with the Recorder
Silk Performer Protocols
Protocol is a set of rules or procedures for transmitting data between computers
Silk Performer has various types of Protocols
Each type of Protocol is designed to handle different aspects of today’s system
architectures and Business Technology
In general Protocol called as Vuser or Virtual Users
Silk Performer Protocols - List of Vuser Types
E-business
•Web (HTTP,HTML), LDAP, POP3, COM/DCOM, Jolt, LDAP, POP3 , FTP protocols,•[CORBA – Common Object Request Broken Architecture],• [FTP – File Transfer Protocol]
Middleware•Jolt, and Tuxedo(6.0, 6.3) protocols
ERP•SAP, Baan, Oracle NCA, Peoplesoft (Tuxedo or Web) and Siebel protocols
Client/Server
•Informix, MSSQLServer, ODBC, Oracle (2-tier), Sybase Ctlib, Sybase Dblib, and Windows Sockets protocols
Legacy
•Terminal Emulation (RTE) Ex: Mainframe Applications
General
•Supporting C template, Java template, and Windows Sockets type scripts
Types of Protocols
WEB, JAVA, .NET PROTOCOLS
ERP/CRM SYSTEMS PROTOCOLS
Silk Performer – Definition of Vusers, Scripts & Transactions
In the Business scenario, Silk Performer replaces human users with virtual users or Vusers. When you run a Business scenario. Vusers emulate the actions of human users— submitting input to the serverWhile a workstation accommodates only a single human user, many Vusers can run concurrently on a single workstation. In fact, a scenario can contain tens, hundreds, or even thousands of Vusers
The actions that a Vuser performs during the Business scenario are described in a script. When you run a Business scenario, each Vuser executes a script, The scripts include functions that measure and record the performance of the server during the scenario
To measure the performance of the server, you define transactions. A transaction represents an action or a set of actions that you are interested in measuring, You define transactions within your script by enclosing the appropriate sections of the script with start and end transaction statements
Silk Performer Components
• The Silk Performer infrastructure consists of the Multi-Machine Controller [MMC], Agents, SCTM[Silk Central Test Manager] and a license server.
• Silk Performer Workbench, the MMC is used for script development, load test management, results analysis and root cause analysis.
• The MMC includes the following Four components:
Recorder, Script Editor and Compiler
True-Log Explorer
Performance Explorer
Server -Analysi
sModule
Silk Performer
Silk Performer Components (Cont.)
Recorder, script editor, and compilero Silk Performer Recorder captures and records a representative amount of real traffic between a
client application and the server to be tested, The script is written in Silk Performer's proprietary scripting language called as Benchmark Description Language (BDL)
True Log Explorero True Log Explorer open in the form of Tree-Mode, It Supports a framework from which you can
customize test scripts and workflow bar, enabling you to augment tests with session handling, verification checks, and parameterized input data
Performance Explorero Performance Explorer allows you to view measurements obtained through Real-Time
Monitoring and to analyze results of past load tests.
Server Analysis Moduleo Server Analysis module generates server-side results that you can archive for future viewing and
comparing. Monitoring also reveals, locates, resolves & server bottlenecks
Silk Performer Testing Process
Planning the Test
Creating Vuser Scripts
Creating the Scenario
Running the Scenario
Analyzing Test Results
Silk Performer – Project Plan, Test Plan, Project Outline
Develop a Project Plan
• Establish a test team • Obtain and review
Business Requirements
• Establish thresholds and boundaries [Threshold – Waiting Time]
Create Test plans -
• Define one or more scenarios
• Define user groups• Define action sets• Define profile settings• Determine which
variables to randomize• Define the workload
configuration • Define the type of test• Define the duration
Outline the Project in SilkPerformer
• Enter the project name
• Enter the description• Enter the application
type
Silk Performer's UI features four main areas
Exercises
Workflow Outline Project – Enter Name of Project on workflow
Select Protocol from the Filter Types,To be given the Project Name & Descriptions
Protocol Selection
Click Model Script from Workflow Bar
1) Select Application Profile from Lists [Ex: Internet Explorer , chrome (or) Firefox] 2) To be given the web based URL :
Select Profile & Specify Web based URL Link
Project Creation on Silk PerformerShopIT Project Created Successfully on Silk Performer Workbench
Silk Performer – Day 2
Recorder Settings
Recorder Button
Recorder captures Internet traffic generated by most browsers or Web applications
Recorder creates measures during a recording session to time actions
Recorder generates a script and log files
Recorder facilitates creation of realistic load tests
Two ways to launch the Recorder Record Menu and Select Recorder
Or Click Model Script on the Workflow bar
Button Tooltip Using the Button
Start Application Click to start the application from the selection list
Add or Edit Application Profiles
Click to add or edit profiles for the applications whose traffic you record
Pause Recording Click to pause a recording session
Stop Recording Click to stop a recording session, generate a test script based on the recoded traffic and save the script
Edit Record Settings Click to view and edit the settings for the current recordings in the active profile
Button Tooltip Using the Button
New Transaction A transaction is a logical group of action statements. Click to stop current transaction and create a new transaction
New Timer Session Click to stop the current timer and create a new timer.Tip: Besides timing actions group different action sets (as an alternative to entering a comment)
Start Timer Click to start a new timer within the current transaction
Stop Timer Click to end a user-defined measurement period. Note: You can also set timers manually in the script
Transaction & Timer Control Buttons
Recorder Toolbar ButtonsMain Recorder Control Buttons
Recorder Toolbar Buttons(Cont.)
Button Tooltip Using the Button
Insert Comment Click to add a comment to the script and log at the current location
Generate Script Click to create a Benchmark Description Language (BDL) script based on the recorded traffic before saving the file
Clear Click to delete all log data and associated script data. You can clear log and temporary script files while maintaining the active recording session
Edit Script Settings Click to view and edit the current script settings in the active profile
Change GUI Size Click to expand and collapse the Recorder
Keep on Top Click to toggle between having the Silkperformer Recorder window always visible or not
Script Control Buttons
Recorder Settings Application Profile The Recorder has several settings that affect its behaviour and impact the appearance of the script , Access
these settings by using the Settings menu Or Edit Application Profiles
Recorder Web Settings Methodo On the Settings menu, Select System Settings-Workbench dialogo Click the Recorder icon, See the System Settings-Recorder dialogo Select an application profile o Click Edit, See the Application Profile dialogo Click Web Settings, The Web Settings dialog appears, Choosing Winsock mode (or) Proxy mode
Recorder - Web Settings from Application Profile WinSock Mode: o Winsock – To hook into the Winsock function library of the client application, wrap all the traffic that the application
produces in the SOCKS protocol and redirect the traffic to a Recorder SOCKS proxy, When using this method,
select the dynamic link library that the Recorder hooks into for recording traffic
o Focuses on traffic generated by dynamic link libraries (DLLs)
o Requires Read/Write access privileges to the registry
o Does not require specific browser settings Winsock mode works best for testing the Web-based applications
Recorder - Web Settings from Application Profile (Cont.)
Proxy Mode:
o Proxy – This mode is used to redirect all the traffic from a browser to the Recorder proxies
o Intercepts and records traffic on the port level
o Requires specific proxy settings for the client application
Proxy mode works best for testing an application when the browser using Automatic Configuration Script that is not Web-based applications
Application Profile IE Settings
Application Profiles tab to create a new profile, All Type of Web Applications Recorded using the Browsers like MS-Internet Explorer, Firefox (or) Chrome whose traffic is to be captured by theRecorder
Recorder Settings - Browser Emulation Level
To access the settings choices for the browser emulation level, select (Edit Record Settings) from the Recorder Button
Go to Edit Profile Click on Web-Protocol-Level Recorder Button
Browser emulation levels are used to:
Capture the traffic during the recording session Determine what type of script commands to use to emulate calls to the application
Browser Emulation has Three Types:
Page based browser-level API (HTML/HTTP) Browser-level API (HTTP) TCP/IP-level (TCP/IP/HTTP)
Recorder Settings - Browser Emulation Level (Cont.)Browser Emulation Level
Recorder Settings – Page-based Browser Level API
Page-based browser-level API to record traffic at the page level [HTML/HTTP] :During page-based browser emulation, Silk Performer parses HTML and requests all embedded
objects
PARSING: Parsing functions are used to parse content returned from a Web server and check if the values meet testing criteria. Basically check the occurrence of a specified input value, the parsing functions parse the content of a server response and return the parsed value in a BDL variable
For EX: Parse session IDs and replace static session IDs in the script with parsed, dynamic session IDs to maintain state information in Web applications. This is one of the main fields of application of parsing functions
Advantages of using the Page-based browser-level API:
o Easy-to-read scripts
o Automatically handle session management
o When Web pages contain dynamically embedded documents & objects
o When your load balancer redirects the client to different servers dynamically
o When you use links, rather than URLs, to advance to a different page
Recorder Settings – Browser-level API
Browser-level API to record traffic sent and received from the browser and to generate scripts Containing easily customized browser-level API calls. During browser-level emulation, Silk Performer does not parse HTML; it hard-codes every embedded object (such as GIF files & style sheets), For EX: Wireless Access Protocol (WAP) applicationsrequire browser-level emulation.
Advantages of using the browser-level API [HTTP]:
o To maximize the number of virtual users on an agent (no parsing required)
o When an application is not Web browser based; for example, multimedia clients that use
HTTP for communications, but do not use HTML
o When you use a hard-coded URL path to advance to a different page
o For verification of attachments
o TCP/IP-level API to record Internet traffic at a low level or to capture proprietary protocols
carried by TCP/IP
Try Script
Try Script Options
Four Ways To Launch TRY SCRIPT
Click Try Script on the Workflow bar
Click Try Script option from the Run menu
Click Try Script on the Toolbar
Press the function key F8
Advantages of Using the Try Script on the Workflow Bar from Silk-Performer Workbench
Compile and save Test Script to determine whether there any problems on the Scripting side
Debug the script
Validate successful execution prior to running against many virtual users
Try Script Dialog and Workflow Bar
When you click Run in the Try Script dialog, Silk-Performer:
Limits the try to one script, one user group, and one virtual user
Runs the script as a stress test (ignore think times) unless Enable think times is selected
Generates a report file, an output file, a log file, and a True Log file
Launches the True Log Explorer if you select the Animated Run option
WORKFLOW BAR
Perform Trial RunRun the Verification Test Script with one user
Monitor Window
After running Try Script, the monitor window shows an overview of the test execution results:
It provides the ability to monitor information at three levels:
User summary Agent summary Individual user detail or Detailed output
Monitor Window
The Monitor Virtual User Output Window
The Virtual User tab displays detailed information about the selected user such as:
o User and Agent nameso Associated line number in the scripto Timestampo Type of function executed at Line numbero Text and information associated with the function
Virtual User Output Window
Summary Details WindowThe Overall Summary details section of the Monitor window given the below output details includingof [All users, Agents, & User Groups]:
Summary of All Users, Specific Script or Agent
Overall Status of load test
Total number of users
Number of users created, executing, and failed
Percentage of CPU utilization on Agent
Percentage of memory utilization on Agent
Percentage of responsiveness on Agent
Total Transaction and Transactions Busy time
Overall progress
Total number of errors
Report Files
Virtual User Report File: Has an .rpt extension
Contains statistics such as number of transactions, amount of data received, and response times
Lists errors and corresponding descriptions
Can be viewed in the HTML screen of the True Log Explorer by selecting the root of the transaction
tree
Can help determine optimum response times and bandwidth requirements for a single user
Explore True Log File: Has an .xlg extension
Contains an XML representation of the pages received by the user
Generates both a Tree view and an HTML view of the documents received by the user
May be used to examine the execution path of the user and to quickly determine failed actions
Can be viewed in the rendered screen of True Log Explorer
Results Profile Settings Specifies the types of result files to be generated when running a test
Creates a separate file for each user in the current agent
Allows you to hide passwords in log files
TrueLog Explorer
TrueLog Explorer is a powerful support tool that simplifies the script customization and root cause analysis of system failures.
Provides a visual representation of the actions executed by the Test script
It is used primarily when debugging Test scripts
Can display more than one TrueLog at a time
Has a Workflow bar for advanced features that allow visual analysis
The TrueLog Explorer interface contains the following components:
– Menu bar– Toolbar– Workflow bar– Tree List Window on the left, – Source Window (showing 4 tabs) – Information Window
True Log Explorer (Cont.)
The following is the List of Details that can viewed on the True Log Explorer Window The Further Details explained on the Next Slides :
– Tree List Window– Source Window– Information Window
Workflow Bar
Tree List Window
The Tree List Window:
The Tree List Window is located on the left side of TrueLog Explorer and highlighted with a red border for the previous slide:
Allows the user to expand and collapse TrueLog data downloaded during trial tests
Displays both TrueLog and TrueLog On Error files
Shows links to all relevant API nodes, to see all API nodes expand relevant API nodes
The nodes include root nodes that indicate the File Names, API Transactions, and Timers
Source Window
The Source Window:
The Source Window contains multiple tab views of all received data Tabs that can appear on the Source window when testing Web applications include the following:
Rendered (HTML and XML only) displays the rendered response from the server for the selected API node in the Tree List view.
Source (HTML, XML, TCP/IP, and UDP only) displays HTML code used to generate Web content
Source Differences - Enabled when comparing result files Post Data (HTML and XML only) displays out-data (data sent by an application to a server) in a
rendered format
Information Window
Below is a list of the default views of information window:
Info ‒ General information about the loaded TrueLog file and the selected API node, including Script files name, function, line number, time, duration, absolute URL, and completion status.
BDL ‒ The benchmark description language script that corresponds to the loaded TrueLog. The BDL script is automatically positioned to the line of the selected API node.
References ‒ Source reference files (frames, embedded documents, forms, hyperlinks, etc.).
Form Data ‒ Allows for visual customization of contextless Web API calls.
Out-hdr (HTTP only) ‒ Contains the exact HTTP header that the application sends to the server.
Out-data ‒ Contains information sent to server if applicable.
In-hdr (HTTP only) ‒ Contains the exact HTTP header that the server sends to the application. In-data (HTTP, TCP/IP, and UDP only) ‒ Contains data received by the application from the Server. The data is
presented in raw format as it is received from the server. Statistics ‒ Contains data received by the application from the server. The data is presented in raw format as it is
received from the server.
Workflow Bar on TrueLog Explorer
Tool Bar Button Names and its Usage on TrueLog Explorer
Button Names Purpose of the Workflow Bar Button
Analyzing a test involves examining the outcome of a previous test run. It focuses on reviewing a Virtual User Summary Report, locating replay errors, and comparing a replay session to the original recording session
Customizing session handling automates the process of identifying static outdated session data in the script, and replacing it with dynamic data
Customizing user data helps to simulate real-world conditions during load testing by varying the virtual user’s actions
Add Verifications after customizing the session handling and user data. Verifications allows you to build functions into the script that check the test returns accurate data
Analyze TrueLog On Error provides a complete history of all the transactions uncovered during testing. It also enables the user to drill down through real content to perform root cause analysis of system and faults
Introduction to BDL Scripting
BenchMark Description Language
Data Types & Variables
Data Type Bytes Descriptions
Boolean 4 True or False
Number 4 Integral number between −2147483648 and 2147483647
Float 8 Floating-point number with maximum value of +/− 1.7E308 with 15 significant digits
String N+1 Character data or binary data
Array Length * Type size
Ordered list of data of the same data type, Elements in the array are addressed starting with the number 1
List Length * Type size
size A container of variables of same type, where the data types are Boolean, Number, Float, or String
Data Types & Symbols
Data types are applied to project attributes, constants, global variables, and local variables
Project Attributes: Project attributes are global values that can be used in more than one script
Constants: It can be declared and initialized in the const section of the BDL script
Variables: It allow for randomization of data used in forms, as well as the timing and execution of transactions
• Declared in the VAR section of the script • Can be referenced anywhere in the script • Are generally values calculated within a transaction and then used in a form
Global Variables
• Declared in the VAR sub-section of a transaction within the script • Can be bound to a transaction or function• Are generally values that are loop counters or other variables used to help
perform calculations within a transaction or function
Local Variable
s
Silk performer Keywords
Keyword Explanations
Const Defining Constants of Integer Type, Boolean, Float or String
DclRand Introducing a Section Name “DCLRAND” on the scripting for declaring random variables
DclParam Introducing a Section Name “DCLPARAM” on the scripting for declaring the parameter Types [From Silk Performer Tool to Create the Visual New Parameter Wizard]
DclFunc Introducing a Section name “DCLFUNC” on the scripting for defining functions on the script
DclForm Introducing a Section name “DCLFORM” on the scripting that can be called by the Transactions, Webforms are the most interface to exchange data within the web server
Transactions
Keywords Explanations
Transaction Keyword Transaction introducing in the dcltrans section and referenced in the dcluser section
Tinit Initialization for all of the Transactions
Tmain Default Transactions Containing all the recorded API Functions
Var Declare variables local to the transaction
Begin Marks the beginning of a Transaction Block
End Marks the End of a Transaction Block
User Groups
Simulation for the Transaction Order Example:
Silk Performer executes the transaction in the following order:
The TInit transaction marked with "begin" executes first, and it executes 1 time at the beginning of the entire simulation
The NewLogin transaction executes next, and it executes 1 time The SearchAndAdd transaction executes next, and for 1 time The CheckOut transaction executes next, and it executes 1 time. Steps 2 thru 4 will repeat until the
simulation is completed The CloseFiles transaction that is marked with "end" executes 1 time at the end of the simulation or when
the tester manually stops the test
Exercises
Silk Performer – Day 3
Customize Script
Customize Session Handling
CUSTOMIZATION
o Enhance Test scripts with Randomized user data
o Handle session-specific data
o Insert verification functionso Silk Performer is able to detect the session ID, but if it is not able to and reports say as an error, then we can use
the Customize Session Handling Wizard to fix the problem
What is Customize Session Handling?
Web server applications often generate information at runtime that is necessary to identify further client requests.
In the response to the browser, the server may include a unique string, commonly known as the Session ID
Customize Session Handling(cont.)The Customize Session Handling Wizard takes the following steps when you click Find differences:
1. Compares the most recent trial run with the recorded TrueLog file, focusing on Session IDs and Other unique information that identifies a user’s session
2. Identifies relevant differences in the Source Window via display message icons:
Correlation Techniques
Manual Correlation, To identify the dynamic value and capture it from the response of previous request. Replace dynamic value with parameter name manually everywhere in the script, Manual Correlation can be performed during script editing
Manual Correlation
Automated Correlation works with Pre-Defined Rules, Automatic Data Correlation Performed during test generation
Automated Correlation
Correlation is not only done for dynamic values, which changing every time but also for data returned by server for different users, To identify such data record, use two scripts with different users [login credentials] keeping user input and steps same
Correlate Using Parsing Functions
Correlation function WebParseDataBoundEx is used for capturing the value for correlate on Silk Performer
For example Session-ids are used by server engines such as ASP, ASP.NET, JSP and PHP to manage sessions,These session ids will change each time the page is loaded
Click “Customize Session Handling” Button from Toolbar getting “FIND DIFFERENCE” link to identify the dynamic values for Record & Replay Scripts on the Silkperformer TrueLog Explorer Tree View
When two parameters that need to be correlated have different names, automated data correlation fails to recognize that the two parameters are related.
Parsing Functions and Usage
Function Name Explanations
WebParseDataBoundEx [sResult, nMaxResultLen, sLeftBoundary, nLeftOccurance, sRightBoundary, nOptions, nDocNum, nBytesParsed)
sResult String variable that receives the string between the specified boundary strings
nMaxResultLen (Optional) Maximum length of the string to return
sLeftBoundary Left boundary of the HTML content to compare
nLeftOccurrence (Optional) The sLeftBoundary has to be found nLeftOccurrence times before the copy process starts
sRightBoundary Right boundary of the HTML content to compare
nOptions (Optional) See Help file to apply different parsing flags
nDocNum DocNum = (Optional Specifies the document to parse
nBytesParsed (Optional) Variable receiving the number of bytes actually parsed
The WebParse function must be immediately before the request WebPageUrl, WebPageSubmit, and WebPageLink that generates the response, Place any Print statements or Write statements after the request for validation
The WebParseDataBoundEx function parses the HTML body for information such as hidden fields or dynamic data. It populates sResult with the text that follows the sLeftBoundary and before the sRightBoundary but looking for nLeftOccurances
Lists of Parsing Functions on Silk Performer
Data parsing functions can be applied to response data returned by servers. In cases where HTML documents are returned from servers, this includes the complete source code of documents. These parsing functions are applied on the TrueLog Explorer Source page, The Different Types of PARSING Functions listed below on the Silk Performer:
WebParseHtmlBound and WebParseHtmlBoundEx
WebParseHtmlBoundArray (not supported by TrueLog Explorer)
WebParseResponseHeader (not supported by TrueLog Explorer)
WebParseHtmlTitle
WebParseTable
WebParseResponseTag (not supported by TrueLog Explorer)
WebParseResponseTagContent (not supported by TrueLog Explorer)
Example for WebparsedataboundEX
Example:
transaction Tmainbegin
WebPageLink(“main menu", "ShopIt - Main menu (#2)"); // Link 1WebParseDataBoundEx (gsSParseDataVar1, STRING_COMPLETE, "name=\"", 5, "\"",
WEB_FLAG_IGNORE_WHITE_SPACE, 1);ThinkTime(7.8);WebPageLink("Check out", "ShopIt - Check Out"); // Link 3Print("Parsed \"sParseDataVar1\", result: \"" + gsSParseDataVar1 + "\""); // Was "87545033" when recordingend TMain;
dclform Payment SHOPITV60_KINDOFPAYMENT_ASP003: "choice" := "CreditCard", "price" := "125.8", "sid" := gsSParseDataVar1; // value: "87545033“End dclform
Session Handling Methods
Session ID’s are Sent to Clients in a number of Ways, Most often find them included for the following Ways:
Cookies
Inside HTML as part of Hyperlink URL’s (or) URL’s of Embedded Objects
Hidden HTML Form Fields
Examples:
1) Session Information inside Cookies [Set-Cookie : SessionID=HELLOWORLD;]
2) Session information included in the URL’s of HTML Links:< html >
< ahref = “/<<Company>>-soft / acknowledge.asp?SessionID=EmployeeName<<Company>>Software”
</ahref> </html>
3) Session information in hidden form fields:<html> <Form Action=“kindofpayment.asp” method=“post”>
<Input Type=“hidden” name=“sessionid” value =“AMAZONWORLD”<Input Type=“text” name=“person-name” value=“JACK”><Input Type=“Submit” name=“payment-button” value=“Submit”>
</Form></html>
TrueLog Explorer – Parameterization
The Customize User Data Wizard allows you to:
Customize the user input data entered into forms during the test runs
Replace recorded user input data with randomized, parameterized or controlled user data
Use the Parameter Wizard for data type selection
PARAMETERIZATION:
Select the ‘Customize Value’ option from the pop-up list.
The Parameter Wizard opens to guide you through the parameterization. With it you can modify script values in one of two ways:
1) Use an existing parameter that’s already defined in your script
2) Create a new parameter
For the coming Slides explained how to create a New Parameter from Parameter Wizard
Parameter Wizard
1) Option to Select Create New Parameter 2) Option to Select Parameter from Random Variable
Parameter WizardThe Strings from file type generates data strings that can be selected randomly or sequentially from a specified random file.
Create new file by choosing the New Button from File Option Menu and Select Random option in the Order group box list.
Parameter Wizard – Random Variable Declarations
Choose anyone of three options as per the System Requirements:
Transaction Mode - Per Usage, Per Transaction and Per Test
Transactions Usage:
Per Usage: It instructs the Vuser to use a new value for each occurrence of the parameter
Per Transaction: It Instructs the Vusers to use a new value for each iterations
Per Test: It Instructs the Vusers to update the Value of the Parameter is same for the Entire Test
Exercises
Correlate - Sample BDF Script
When we Run the Try script for below Sample BDF Script getting Timeout Error on the Virtual User Output File,Identified needs to be Correlate on this script
Silk TrueLog Explorer WindowFrom Silk TrueLog Explorer Window Using Find Differences Button for compare the Record and Replay Scripts,Seeing the differences on the Session ID, To Use Customize Session Handling for handle this type of errors
Insert Parsing FunctionsScript Modified successfully apply the Insert parsing functions:
Summary Details
Try Script Run successfully
Silk Performer – Day 4
Syntax of Load Testing Scripts
The various sections of a Silk Performer load testing script must be declared using the following
syntax:
External functions contained in a dynamic link library that are called from a script must be declared
using the keyword dll
Include statements in a script include one or more files during the compilation of a script. Declare
include files with the keyword use
Constants are defined using the keyword const. Wherever you can use an integer constant, a
floating-point constant, a boolean constant or a string constant, you can likewise use a named
constant declared in this section. You do not need to specify the data type when declaring a
constant
Variables are defined using the keyword var. Global variables have global scope and therefore can
be used in every transaction and as bind or INTO variables in every SQL command. You can also
define variables with local scope for any transaction
Syntax of Load Testing Scripts (Cont.)
Parameters are variables, which are re-used by the visual script customization wizards
Parameters are (automatically) defined using the keyword dclparam
Random variables are defined using the keyword dclrand
Functions used in a script are defined using the keyword dclfunc
Users to be simulated are defined using the keyword dcluser
Transactions executed by the users specified in the workload section of a script are defined
using the keyword dcltrans
Web forms used in transactions in a script are defined using the keyword dclform.
SQL commands that are called by transactions in a script are defined using the keyword dclsql
Handler functions are defined using the keyword dclevent
Sample Load Testing Scripts
benchmark Sample use "Kernel.bdh" dll "kernel32.dll" "WaitForSingleObject" function WaitForSingleObject(in number, in number): number;const NUMBER_CONST := 1; STRING_CONST := "ABCD"; BOOLEAN_CONST := true;
var nNumber : number; sString : string; bBoolean : boolean;
dclparam nNumParam : number; sStringParam : string; bBoolParam : boolean; dclrand rNumber: RndUniN(1..6);
dclfunc function Function1 begin end Function1;
dclevent handler ErrorHandler <EVENT_RAISE_ERROR> begin end ErrorHandler;dcluser user User1 transactions TMain : 1; dcltrans transaction TMain begin end TMain; dclform Form1: "elem1" := "Value1", "elem2" := "Value2“;
Silk Performer FunctionsGeneral Functions - Overview
FUNCTIONS1 FUNCTIONS2
Data Structure Functions Synchronization Functions
Date Functions Random Functions
Error-Handling Functions Random Types
File Functions Runtime Functions
List Functions Simulation Setting Functions
Load Test Control Functions String Functions
Logging Functions Overview Regular Expressions
Measurement Functions System Information Functions
Memory Functions Attributes Manipulation Functions
Process Control Functions XML Functions
Variable Initialization - Example
dcltrans transaction TMain;
var nNumber : number init 10; fFloat : float init 123.456; bBoolean : boolean init true; sString : string init "Hello world!";
begin write("number = "); write(nNumber); writeln; write("float = "); write(fFloat); writeln; if bBoolean then write("boolean = true"); writeln else write("boolean = false"); Writeln
end; write("string = "); write(sString); writeln;
end Tmain;
Outputnumber = 10 float = 123.456 boolean = true string = Hello world!
Find Baseline
Baseline Test
What is Baseline Test?
A baseline test can serve as a trial run of the performance test script, It helps to verify the following:
A customized test is run with just one user per user type.
The script can accurately and fully reproduce the interaction between the client application and the server Baseline tests ignore the Random thinking time option [Makes it easier to for comparing the results of different
baseline tests because the think times remain constant between the different test iterations
Baseline Test Settings
The Settings Below for Baseline report file is automatically created
The Random thinking time option is disabled
The Load test description text box is set to Baseline Test
The Stop virtual users after simulation time (Queuing Workload) option is enabled
The Display All Errors Of All Users option in the Monitor window is enabled
The Virtual user output files (.wrt) option is enabled
The Virtual user report files (.rpt) option is enabled
Find Baseline
Find Baseline (Cont.)
Performance baseline testing is called Benchmark test. This is the first test to Compares the performance of a new or unknown server to a known reference standard, such as measurements.
The baseline runs one user of every user type.
The Baseline:
– Allows you to establish the performance benchmark
– Runs a load test for selected combinations of user groups and profiles known as user types
– Generates the following files:
o Time Series Data (.tsd)
o Report (.rpt)
o Output (.wrt)
o Project Settings (.xml)
o Baseline Report (.brp)
Confirm Baseline
The Baseline Test Summary Page displayed the following sections:
Quick Summary – Duration of Testing Executions, Users Count, Errors
Available User Types – Graphical Display of Network Bandwidth filtered by script and user profile
High Impact Measurements – A pair of donut-looking graphs that show page size and page slowness
Next Steps – A set of actions the user can take based on the Baseline Test Summary
Help Topics – A Quick Link to the Help section on the current topic of Baselines
Baseline Test Results
Summary Sheet:
Workload Models
Adjust Workload on Silkperformer
Silk Performer provides a wizard to set up anyone of the workload models listed below:
Different Types of Workloads on Silk Performer Workbench
Increasing : At the beginning of a load test, Silk Performer does not simulate the total number of users defined. Instead, it simulates only a specified part of them. Step by step, the workload increases until all the users specified in the user list are running
Example: This workload model is especially useful when you want to find out at which load level your system crashes or does not respond within acceptable response times or error thresholds
Steady State: In this model, the same number of virtual users is employed throughout the test. Every virtual user executes the transactions defined in the load-testing script. When work is finished, the virtual user starts again with executing the transactions. No delay occurs between transactions, and the test completes when the specified simulation time is reached
Example: This workload model is especially useful when you want to find out about the behaviour of your tested system at a specific load level
Dynamic: You can manually change the number of virtual users in the test while it runs. After the maximum number of virtual users is set, the number can be increased or decreased within this limit at any time during the test. No simulation time is specified. You must finish the test manually
Example:This workload model is especially useful when you want to experiment with different load levels and to have the control over the load level during a load test
Different Types of Workloads on Silk Performer Workbench(cont.)
All Day : This workload model allows you to define the distribution of your load in a flexible manner. You can assign different numbers of virtual users to any interval of the load test, and each user type can use a different load distribution. Therefore, you can design complex workload scenarios, such as workday workloads and weekly workloads. You can also adjust the load level during a load test for intervals that have not started executing
Example: This workload model is especially useful when you want to model complex, long lasting workload scenarios in the most realistic way possible.
Queuing :– This workload model is especially useful when you want to simulate workloads that use queuing mechanisms to handle multiple concurrent requests. Typically, application servers like servlet engines or transaction servers, which are receiving their requests from Web servers and not from end users, can be accurately tested by using the queuing model.
Example: if you specify a simulation time of 3,000 seconds and want to execute 100 transactions, then you observe an average transaction arrival rate of 30 seconds.
Verification: – A verification test run is especially useful when combined with the extended verification functionality. This combination can then be used for regression tests of Web-based applications. A verification test run always runs a single user of a specific user type on a specified agent computer.
Example: This workload is especially useful when you want to automate the verification of Web applications and when you want to start the verification test from the command line interface
Workload Configuration
Select Adjust Workload from the Workflow Bar
Common Workload Configuration Settings
The Below listed settings are common for all of the Workloads
Adjust Think Time
Think Time is the Time Delay between the two iterations
Go To Settings menu Select Active Profile Getting Simulation Dialog Diagram Below
Think Time PacingPACING: Pacing is the wait time between the Action Items.
Static: To reach the goal session time, the think times in the script are modified by a constant & static factor Dynamic: To reach the goal session time, the think times are modified by a dynamic think time factor that is
recalculated after each session
Silk Performer – Day 5
Result Analysis Silk Performer Workbench
Results Tab
The Results tab displays a Tree view containing all test results of the active projects
Baseline report
Recent Try Script runs
Time series files
True Log files (contains True Log files for each virtual user)
User results (contains user profiles, which list all result files per virtual user)
Silk Central Test Manager Integration Results
Results Viewing Overall on Silk Performer Workbench
Performance Explorer: This is the primary area for viewing test results. A fully Comprehensive array of graphic features displays the results‒primarily in user-defined Graphs‒with as many elements as are required. The results of different tests can be compared. There are extensive features for server monitoring. A comprehensive HTML based overview report that combines user type statistics with time series test result information is also available.
True Log on Error: Silk Performer provides full visual verification under load capabilities for Web applications. It allows combining extensive content verification checks With full error drill-down analysis during load tests.
Virtual User Report files: Details of the measurements for each individual user are Presented in tabular form. Virtual User Output files: These files contain the output of Write statements used in Test scripts, if their
generation is enabled.
Viewing Baseline Reports: A detailed XML/XSL-based report that provides you with a Summary table, transaction response-time details, timers for all accessed HTML pages, Web forms & errors that occurred.
Silk Central Reports: Silk Performer projects and results can be integrated into Silk Central Test Manager’s projects and test containers. Once integrated, each Silk Performer Project can be executed directly within the Silk Central Web interface
Performance Explorer
The Performance Explorer generates an overview report that provides performance and throughput analysis, utilizes time series data files, and includes standardized graphs and reports
Performance explorer InterfacesPE Workflow
Overview Report – Administrative Information
The Performance Explorer’s overview report includes the most important test results in tabular and graphical form and combines all TSD information collected from the load test
TSD[Time Series Data]:
Project name [Description of the Project]
Load test number
Date of the load test
Duration of the simulation
Number of agent computers
Number of virtual users
Report description
General project settings:
Application type
Workload type
Workload name
Overview Report - Charts Display
General graphs
Active Users in [ Transaction/second or Errors/second ]
Throughput/Concurrency in [KB/sec or Hits/sec]
Response Times Timers
Response Times Transactions
Response Times Page Timers
Throughput:
The amount of data sent to and received from the server; this includes header and body content information,
all TCP/IP-related traffic (HTTP, native TCP/IP, IIOP, POP3, SMTP, FTP, LDAP and WAP), and secure traffic over
SSL/TLS. This measurement does not include data overhead caused by SSL/TLS encryption and WTLS encryption
in case of WAP
Virtual user Report – Baseline Test
Summary tables The Summary Table Containing Summarized measurements on a global level, Summary
are
sub-divided into three Sections:
Summary General – A general summary count of all transactions
Summary Internet – A summary of all internet-related metrics
Summary Web – A summary of Web or HTTP-related metrics
Transactions Transactions are the starting and ending point of any user actions and the transactions measurement
group contains response time information for all of the transactions.
Custom timers Calculating the Amount of time between a set of Actions , this timers displayed only user-defined
actions, and the Custom Timer measurement group contains the timers that are defined in the load-testing script with
the MeasureStart and the MeasureStop functions
Virtual user Report - Baseline Test
Page and Action timers The Time Elapsed between Two Events and the page timer measurements group contains
measurements related to Web pages that are downloaded by a virtual user. Response times for Web Pages are
subdivided into server-busy times, document-downloading times & Round-trip times
Web forms The Web form measurement group provides response time measurements and throughput rates for
form submissions with the POST, GET, and HEAD methods
Summary Table Report
Number of Transactions - The Number of Silk Performer Transactions per second
Request data sent - The Amount of data sent to the Server
Request Data received - The Amount of data received from the Server
Throughput information - The Total amount of Request data sent and Response data received from the server; this includes
header and body content information, all TCP/IP-related traffic (HTTP, native TCP/IP, IIOP, POP3, SMTP, FTP, LDAP and WAP),
and secure traffic over SSL/TLS
Errors[if any]
Overall MetricsVolume, capacity, and rate metrics are also frequently requested by stakeholders, even though the implications of these metrics are often more challenging to interpret. For this reason, it is important to report these metrics in relation to specific performance criteria or a specific performance issue.
Bandwidth consumed
Throughput
Transactions per second
Hits per second
Number of supported registered users
Number of records/items able to be stored in the database
Response TimesEnd-user Response Times
End-user response time is by far the most commonly requested and reported metric in performance testing. If you have captured goals and requirements effectively, this is a measure of presumed user satisfaction with the performance characteristics of the system or application. Stakeholders are interested in end-user response times to judge the degree to which users will be satisfied with the application. Technical team members are interested because they want to know if they are achieving the overall performance goals from a user’s perspective, and if not, in what areas those goals not being met.
Metrics Goals
Metrics:
Sample Test EnvironmentTest Environment
Advantages of Performance Testing
Performance Testing Advantages