Testing Tools usingVisual Studio

Randy Pagels Sr. Developer Technology SpecialistMicrosoft Corporation

Production

Development Collaboration

BACKLOG

REQUIREMENTS Monitor + Learn

Plan

Develop + Test Release

Application lifecycle management

Agenda• Web Performance Testing• Load Testing• Unit Testing• Manual Testing• Exploratory Testing• Coded UI Testing• Lab Management

Microsoft Testing Tools

• Coded UI Tests• Web Performance

Tests• Load Tests• Generic Tests• Ordered Tests• Developer Unit Tests• Database Unit Tests

• Test Plan Management• Test Case Management• Shared Steps Management• Exploratory Testing• Test Configuration

Management• Test Fast Forwarding• Test Data Collection• Test Lab Management

Test ManagerManual Testing Tools

Visual StudioAutomated Testing Tools

Developer Focused Unit Test ExperienceRed-Green

Bar

Most important

tests shown first

Timings

Shows tests from any

framework

Search

Run Details

New Test Runner• Tests run in background• Run automatically on build• Support for multiple unit testing

frameworks: MS Test, xUnit, nUnit ,and

more!• Deep integration in the IDE• Supports native C++ code• Multiple run options

Faile, not-run, and all tests• Easy code coverage access

Unit Testing with Code Coverage• Determine what

proportion of your project's code is actually being tested. Guard effectively against bugs by exercising a large proportion of your code. Code coverage analysis can be applied to both managed (CLI) and unmanaged (native) code.

Manual Testing• Manual testing is the process of

manually testing software for defects. Microsoft Test Manager allows you to work with test plans, author and organize manual test cases, execute test cases, file bugs, and post results back to Team Foundation Server.

Exploratory Testing• Exploratory testing is the

testing of an application without a set of tests defined in advance. During an exploratory test session, you are not restricted to a script or a set of predetermined steps. Some improvements have been made to the exploratory testing in Microsoft Test Manager.

Coded UI Testing• Coded UI tests provide a way

to create fully automated tests to validate the functionality and behavior of your application’s user interface. You can create coded UI tests directly from existing action recordings. Action recordings contain the steps taken during manual testing of an application.

Web Performance Testing• Any new Web performance and

load testing efforts you create in Visual Studio are now created in a Web performance and load test project type instead of a test project. A Web performance test consists of a series of HTTP requests. Web performance tests work at the protocol layer by issuing HTTP requests.

Load Testing• Load tests consist of a series of

Web performance tests or unit tests which operate under multiple simulated users over a period of time. Load tests can also include automated coded UI tests. Coded UI tests should be included only under specific circumstances. All the scenarios that use coded UI tests in load tests involve using the coded UI tests as performance tests.

Load Testing in the cloudCloud-based load testing Use existing load tests No need to setup servers or

virtual machines Use Visual Studio Ultimate

to create "Web Performance and Load Test" types

Run and analyze your load test

View Performance and Throughput graphs

View all your load test runs at any time via explorer

Download reports to view performance counter data, threshold violations, and error information

Get performance data from Application Insights

Load Testing in the cloudAI Integration View the

performance data from Application Insights

Application counters are correlated with the user load

Understand what issues may be causing any performance problems

Easily go to Application Insights if you need to do a more detailed analysis

Load Test Manager

View Load Test Runs Use from

Visual Studio Ultimate.

Lists all the test run against the service.

Same look and feel as web access, but hosted inside Visual Studio.

Testing Tools using Visual Studio

Demo

Work Item ChartingReporting Enable a

spectrum from very simple charting up to very powerful BI capabilities

Agile Tools Tagging REST APIs for tags Query on Tag Tags in Visual Studio Tag Permissions Exclude non-working days Cumulative Flow Start

Dates Pinning of Work Item

Charts

Test: Test Case Management & MSDN Benefits

Enhanced test management• Support for non-Windows

operating systems with enhanced web-based test case management

• Create or manage Test Suites and Plans using web interface

Flexible on-premises or cloud dev and test MSDN subscriptions now including

dev/test support in the cloud.

Test ManagementTest Export Export your test plan to HTML Choose the level of detail you

want

Test Management Test Parameters Create sets of test data Reference by various test

cases Easily bind to automated

tests

User and Security ManagementPermissions Reference• default groups, permissions, and roles• configure to control access to the resources• restrict access in TFS with allow or

deny permissions

Permissions by Groups

Permissions by Individual

TFS Alerts Set alerts, get notified when changes occur• Set alerts just for

yourself• Set alerts for your

team

Lab Management• Testing a complex multi-

tiered application and creating different configurations in a user-flow can be challenging. Lab Management provides teams a powerful and flexible development and test lab environment to enable scalable build-deploy-test workflow and continuous integration through environment templates.

Testing Architecture

VM Template Library SCVMM Environment Standard Environment

VM Template 2VM Template 1

Test Agent Test Agent Test AgentTest AgentVMMAgent

Web ServerVM

Database Server

VM

VM Host Web Server Database Server

Test AgentTest Agent

Team Foundation Server

System Center Virtual Machine Manager (SCVMM)

Team Build

Test Controller

Build-Deploy-Test

SCVMM Environment

Test AgentTest AgentVMMAgent

Web ServerVM

Database Server

VM

VM Host

Team Foundation Server

Test Controller

Team Build

(1) Get Source Code (2) Compile projects

(7) Send testresults

(6) Executeautomated

tests

(3) Copy build to runningenvironment

(4) Run deployment scripts for

each machine

(5) Create environment snapshot

(8) Publish results toTeam Foundation Server

Related Contenthttp://msdn.microsoft.com/en-us/library/tfs/bb385901%28v=vs.110%29

http://blogs.msdn.com/b/visualstudioalm/archive/2012/06/04/getting-started-with-load-testing-in-visual-studio-2012.aspx

http://msdn.microsoft.com/en-us/library/tfs/dd293547%28v=vs.110%29

http://blogs.msdn.com/b/visualstudioalm/archive/2012/07/09/javascript-unit-tests-on-team-foundation-service-with-chutzpah.aspx

http://blogs.msdn.com/b/visualstudioalm/archive/2012/07/31/writing-a-visual-studio-2012-unit-test-adapter.aspx

http://msdn.microsoft.com/en-us/library/tfs/ms182539%28v=vs.110%29.aspx

Visit Team System Café Today!The door is open 24/7 at the café! Stop by for a cup of joe and delicious tidbits on AppDev topics!

help & how tootrainingbooksFAQs

downloadsevidencedocumentsslides

most popularitems

freshly brewed news

http://www.teamsystemcafe.net

blog’sblogroll

traininglinks

eventsin personwebinars

FAQsTFSALM

user groups

labmanagementlinks

azure & windows 8resource links

What's New for Application Lifecycle Management in Visual Studio 2012http://bit.ly/baSMz5

What's new for Microsoft Test Manager 2012http://bit.ly/wN2JrH

Quick Start Guide for Manual Testing using Microsoft Test Manager 2012http://bit.ly/d4l9qN

Test Scribe for Visual Studio 2012http://bit.ly/Nim0gH

What is new in Coded UI Tests 2012http://bit.ly/zWIbVf

Platforms for Coded UI Tests and Action Recordingshttp://bit.ly/b5m32u

Resources

© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.