functional testing with the java stack test runner
DESCRIPTION
Functional Testing with the Java Stack Test Runner. Disclaimer. This is a training not a presentation. Please be prepared to: Learn Ask questions Participate in the lab Prerequisites: Knowledge of Maven (required) Java Stack ALM Training (recommended). What we’ll cover…. - PowerPoint PPT PresentationTRANSCRIPT
Functional Testingwith the
Java Stack Test Runner
DisclaimerThis is a training not a presentation.
Please be prepared to: Learn Ask questions Participate in the lab
Prerequisites:
Knowledge of Maven (required) Java Stack ALM Training (recommended)
What we’ll cover…In this training, we will discuss
What is functional testing?
How is it unique from other forms of testing?
What is the Java Stack Test Runner?
How does it integrate with Maven and AnthillPro?
How do I invoke Test Runner from my IDE?
Where do I find my functional test results?
What we won’t cover…Topics that we will not have time to discuss are:
Testing frameworks and APIs• i.e. TestNG, Selenium, Spock, etc.Testing methodologiesWriting testsTest case management tools
First, some background…
Stages of TestingWhat When
1. Unit Testing After code compilation
2. Integration testing After the build artifact is packaged.
3. Functional testing* After the build artifact is deployed and started.
4. Load testing After deployment to staging environment
5 End-user testing After code is “feature complete”.
* We will discuss functional testing and how it differs from unit and integration testing.
Unit Testing Scope• Narrow (single class or method)
Purpose• Validates small units of code
Responsible Party• Code developer
Failure:• Indicates code is broken• Typically fails the build
* Has direct dependencies on application code
Integration Testing Scope• Medium (spans two integrated systems)
Purpose• Validate code that integrates with another system.
(e.g. a database or web service) Responsible Party• Code developer.
Failure:• Indicates failure of systems to communicate as
expected.• Does not typically fail the build.
* Has direct dependencies on application code.
Functional Testing Scope• Broad (end-to-end feature testing)
Purpose• Validate functional requirements and/or user stories
Responsible Party• QA resource or feature developer
Failure:• Indicates failure to meet user story requirements• Results recorded in test case management and/or bug
tracker
* Has little or no direct dependencies on application code.
Functional Testingand
Application Life-cycle Management
(ALM)
ALM Build ToolsMaven
• Project definition
• Dependency management
• Plug-in configuration and management
• Build profiles
AnthillPro
• Manages build processes Creation of build artifact(s) Deployment to server
environment Functional test execution
• Maintains build life history
• Provides ability to re-execute a process on any build life
ALM Tools: Maven Life-cycles
The default Maven build life-cycle (simplified):
compile test (unit tests) package integration-test (integration tests) install (to local Maven repository) deploy (to remote Maven
repository)
Where do we plug in functional testing?
ALM Tools: Maven Life-cycles
The woes of having “one build to rule them all”:
Functional Testing ≠ Integration TestingConfuses build with server deploymentMust re-build to re-deploy or re-testComplicates build managementLess productive use of build agents
ALM Tools: Maven Life-cycles
The Java Stack ALM build life-cycle (simplified):
alm-db (run database scripts) alm-deploy (deploy application to
server) alm-test (run functional tests) alm-promote (run all previous phases)
This life-cycle is available to the artifact produced by the “alm” module of a Java Stack project.
ALM Tools: Maven Life-cycle
Advantages of a second build life-cycle: Clean separation of configuration:• One configuration for build• One configuration for deployment and/or functional
testing Separate processes:
• One process for building artifact• One process for deployment and/or functional testing
Simplified build management More productive use of build agents
ALM Tools: Test RunnerIn Stack 3.2, as part of its ALM tooling, the Java Stack introduced the “test-bundle” artifact as a part of its suite of testing plug-ins, collectively known as “Test Runner”:
Why another artifact?
Why another tool?
ALM Tools: The Test Bundle
The “test” jar
Not an executable jar
Difficult to execute without help from Maven.
Relied on maven to supply its dependencies.
Difficult to execute tests from past code revisions.
The “test-bundle” jar
An executable jar
Can be executed with or without Maven.
Comes packaged with all its dependencies.
Each test artifact represents a snapshot of the current code revision.
ALM Tools: Test Runner
The “failsafe:test” goal
Supports TestNG
Supports test suites
Supports environment configuration
Produces JUnit and TestNG reports.
Executes tests from the target/classes folder
The “stack-test:functional-test” goal
Supports TestNG
Supports test suites
Supports environment configuration
Produces JUnit and TestNG reports.
Executes “test-bundle” artifact.
ALM Tools: Functional Test Goal
Common configuration for stack-test:functional-test:• testBundle – groupId and artifactId of QA test bundle• testEnv – the test environment properties to load• skipFTs – whether to skip functional tests• suiteXmlResources – TestNG suites to load• includes – test class patterns (e.g. **/*FT.class)• systemPropertyVariables – custom system properties• argLine – JVM arguments (e.g. memory options, debugger,
etc.)
For the additional options, see:• http://code.lds.org/maven-sites/stack/modules/test/
ALM Tools: The Test BundleSimple to upgrade existing QA projects.1. Upgrade the project to Stack 3.2
Then change the QA pom’s packaging type from
to
2. Add QA module as a dependency to the ALM module.
<packaging>jar</packaging> <packaging>test-bundle</packaging>
<dependency> <groupId>${project.groupId}</groupId> <artifactId>example-qa</artifactId> <version>${project.version}</version> <type>test-bundle</type></dependency>
ALM Tools: The Test Bundle3. Bind the functional-test goal to the alm-test phase:
<plugin> <groupId>org.lds.stack.test</groupId> <artifactId>stack-test-maven-plugin</artifactId> <executions> <execution> <id>test</id> <goals><goal>functional-test</goal></goals> <phase>alm-test</phase> <configuration> <testBundle> <groupId>org.lds.training.testrunner.lab1</groupId> <artifactId>testrunner-lab1-qa</artifactId> </testBundle> </configuration> </execution> </executions></plugin>
ALM Tools: The Test Bundle4. Modify build profiles to fit your needs
<profile> <id>test</id> <properties> <skipFTs>false</skipFTs> <testEnv>test</testEnv> </properties></profile>
Lab #1A Simple QA Test Bundle Project
https://tech.lds.org/wiki/Functional_Testing_with_the_Java_Stack_Test_Runner
Lab #2Running Functional Testswith Maven and your IDE
https://tech.lds.org/wiki/Functional_Testing_with_the_Java_Stack_Test_Runner
Lab #3Running Functional Tests
in AnthillProhttps://tech.lds.org/wiki/Functional_Testing_with_the_Java_Stack_Test_Runn
er
CreditsThe Apache Maven Project:
http://maven.apache.org/
Urban Code, for it’s AnthillPro documentation:
http://www.urbancode.com/html/products/anthillpro/
The Dark Lord Sauron, for the quote, “One ring to rule them all”. (The Lord of the Rings, by J.R.R. Tolkien)
<code snippet block>