testing in dot net

8/14/2019 Testing in Dot Net

1/31

1

Automated Testing in

.NET:

What you need to know about

Integrating Testing &Development

Mary Romero SweeneySammamish Software Services

Copyright Sammamish Software Services 2008; All rights Reserved


2/31

2

Former developer both embedded andbusiness systems for the Boeing

Company

Published Visual Basic for Testers

Apress Aug, 2001

Published A Testers Guide to .NETProgramming Apress, Feb 2006

Quick Bio


3/31

3

Todays talk

This talk is notintended only for Microsoft

customers; we want to talk about a newapproach to test development

In this talk I will provide only an overview

of the features available in Visual StudiosTeam Test software in order to give you afeel for its power and capabilities forcomparison to your own effortsWe will notbe covering the entire feature set of

this new addition; this is not a Microsoft-sponsored presentation!


4/31

4

Test Techniques Timeline

Note: All along many have had support from white box testers with codingskills enabling companies to create their own unique automated tests

Manual Testing

Automated Testing using3rd party tools

Automated TestingUsing Open Source Tools

Test and Development IntegrationInto Development platform software


5/31

5

Traditional Automation Software vs

Integrated Dev and Automated Test

ApplicationSoftware

Automation

Software

Integrated

Development andAutomated Test


6/316

Problems with the status quo

Tool bundlingThis occurs because vendor tools often dont have all the

capability needed

Decreases communication due to the lack of integration

Adding diverse tools from different makers makescommunication transfer difficult. Many vendors add to this

issue by acquiring new features that are just acquisitionof other companies tools

Expensive vendor tools vs. risky open source tools is a

tough choiceDo it yourself test harnessesLack of commonality and lack of enough available talent


7/317

Microsofts Team Edition for Testers

With the release of Visual Studio 2005,Microsoft has integrated a significant amountof test capability into Visual Studio with itsVisual Studio Team Test software

The idea behind the Team Test software is

the integration of testing tools into thesoftware development environment.Developers create code and then create Unit Tests

and Test Engineers use these unit test shells tocreate more sophisticated system tests

OR, Testers can create the Unit Tests fromdevelopment code


8/318

Cross Organization communication

Intent is to create a setting whereinformation can be shared across

allthe groups involved in softwaredevelopment.

Tests and their results can then beshared and managed across theorganizationfrom management to

the development and test teams,

and back again.

Test Results can be published by thesoftware into reports and graphs

Testdevelopment/

publication

Dev

Mgt

Test


9/319

Team Edition for Testers

There are a variety of test typesto

address different organizationalprocesses and softwaredevelopment methodologies,

including Agile, Extreme, and TestFirst testing methods, as well as thetraditional software lifecycle model.

Types of Tests:

Web,

Load,

Ordered,

Manual,

Unit,

Generic


10/3110

Test Types

Web:

Verify the functionality of web applications. Navigate through the site,clicking on different buttons, links, etc. and all web traffic gets saved andcan be played back

Load:

Take other tests (except manual tests) and run them simultaneouslyunder simulated load using virtual users. Allows you to set thresholdsand to see where performance points have failed. Resulting performancedata can be collected and reported.

Ordered: Place a set of existing tests in an ordered container.

Continued


11/3111

Test Types, continued

Manual:Describe the steps for a tester to follow manually. Provides a

way to keep track of your manual tests along with the othertests in your project.

Unit:

Create code-generated tests for your components. You canalso create data-driven unit tests that use a database to readand store data.

Generic:

The generic test type allows you to take an existing programyou have and wrap it to function as a test that can beexecuted inside Visual Studio. Harness existing nUnit teststhis way.


12/3112

Test Team support for Component Testing

But, Unit (component) testing is traditionallyconsidered the domain of the development

team!?

Not so in Agile and Test First methodologiesNo one is suggesting that developers stop doing

unit testing; however, a test professional can add

great value to the project by incorporating

professional test expertise into the unit testingprocess.


13/3113

The Process Developer creates component Depending on methodology employed the

skeleton test harness for the component iscreated by either the developer or a tester

Test professional enhances the harness with

proper test expertise Including equivalence classing, boundary analysis, etc.

Test Team executes and provides results to

management and developer. Developer uses integrated environment to

directly and quickly address test issues

Warning! Code on next page..


14/3114

Creating a Unit Test in an Integrated

EnvironmentHuman error

sometimes accountsfor missed methodsand classes

Code generation candevelop the skeletonof your unit tests

Namespaces

Classes

Methods


15/3115

'''A test for NewInvoiceNumber()

''' _Public Sub NewInvoiceNumberTestMethod()Dim CustID As Integer = 0

Dim target As Invoice = New Invoice(CustID)Dim expected As Integer = 75Dim actual As Integeractual = target.NewInvoiceNumber

Assert.AreEqual(expected, actual, _"InvoiceLib.Invoice.NewInvoiceNumber did not return the

expected value.")End Sub

Generated Test Code


16/3116

Whats the effect on best practices

for development and test? The developer often does not have the time and

certainly does not always have the correctmindset to perform thorough Quality Assurancetesting

In fact, in many organizations whats being done is morealong the lines of Unit Exercisingor simply ensuring basicfunctionality not full professional testing

Integration of test with development allows formore detail support from the test team and cross-department communication of metrics

Eases the introduction of test support earlier in thesoftware develo ment rocess


17/31

17

Problems and advantages

Downside:

Only the VS Team Edition for SoftwareDevelopers (VSTESD) gives you the fullcapability including reporting

Expensive! Due to the cost, many .NETcustomer companies will stick with nUnit andother automated testing tool options


18/31

18

Upside:

Gets test and development not just thinkingabout but doingtesting MUCH earlier in thesoftware process

Increases the role and capability of the TestEngineer

Provides far more visibility to management on

test development and progressProvides for more quick and direct reporting and

resolution for certain kinds of bugs

Problems and advantages


19/31

19

How new integrated testing compares

to traditional and current practices

Traditional: black box & manual testing

If your organization separates test anddevelopment and provides little or no white box

test support this will involve a major culture change

Test First and Agile methods

If you are a Windows/Microsoft project then this

provides more specific and convenient supportIf you are notWindows/Microsoft, look for more

support along these lines from your own platform

vendors


20/31

20


21/31

21

Test Manager window

The Test Manager window allows you to organizeand manage your tests and test runs (suites)


22/31

22

Test View window

Create, run, andorganize tests from

Test View


23/31

23

Test Results Window

Test results window provides integratedresolution of test and the opportunity to view

more info, debug and rerun


24/31

24

Code Coverage Support

Do you pay for a code coverage tool? Its built-inhere:


25/31

25

Test Run Configuration window

Tests and test runs are configurable


26/31

26

Load Test Settings


27/31

27

Load Testing Results


28/31

28

Summary/Conclusions

Now using .NET we can test using the

development software; this produces manyadvantages in cross-departmentcommunication development

Larger software development companies can

make effective use of these integrated toolsto save a lot of coding time for automatedtests in Visual Studio.

The expense of this Visual Studio versionmay not be necessary for smallerorganizations and is prohibitive for individualtesters who want to write quick and simpletests.


29/31

29

Finally a word to the Microsoft users:

If you havent upgraded to VS TeamSystem, should you?If you do white box testing and coding

currently then this could save you hundredsof hours in test development

Product is fairly stable

If you dont do white box testing and coding

you need to analyze your current talent anddetermine if this is a good fit; it may be costprohibitive for anything but the larger shops -- at this time

Can you do without it? Sure.


30/31

30

The future

Prediction: Integrated test and developmentplatforms will be a standard in the future

Remember when integrating a debugger into thedevelopment platform was a new thing? (OK Im old)

Well be saying the same thing about having code

coverage and test benches in the platform as well. Isnt the cost prohibitive? Not really compared to

purchasing multiple tools; the advantage over open

source is the same vendor/seamless integration.Like most things cost will go down (?)

How does this affect major test tool vendors? Not goodnews for them.

Are youready for this next phase?????


31/31

Resources

Testers Guide to .NET Programming by Mary Romero Sweeney and Randal Root (Apress, Feb 2006)

Pro Visual Studio Team System by Jeff Levinson (Apress, May 2006)

Microsoft (of course) http://msdn.microsoft.com/vstudio/teamsystem

Lots of Users groups and white papers

Product comparison website: http://msdn.microsoft.com/vstudio/products/compare/default.aspx
http://msdn.microsoft.com/vstudio/teamsystemhttp://msdn.microsoft.com/vstudio/products/compare/default.aspxhttp://msdn.microsoft.com/vstudio/products/compare/default.aspxhttp://msdn.microsoft.com/vstudio/teamsystem

testing in dot net

Documents