“the future of software testing” test environments, · test environments, the future...
TRANSCRIPT
-
Test Environments, the Future Achilles’
HeelMichiel Vroon, Quaboo,
The Netherlands
Europe’s Premier Software Testing Event
World Forum Convention Centre, The Hague, Netherlands
WWW.QUALTECHCONFERENCES.COM
“The Future of Software Testing”
-
Test environment
Achilles heel of the future?
Michiel Vroon
-
Test environment
-
Sounds familiar?
• I thought we solved this defect before?
• This part worked fine some moment ago!
• Who knows the correct configuration?
• What do you mean, first production??!!
• This looks different from production!
• Do you know a user-id we can use?
• What is in the current release?
• I cannot reproduce the defect!
-
Some causes
• No dedicated competence
• Difficult to define requirements for test
• There are no processes, plan or strategy
• Design is not clear
• Test data is not maintained
• Too many people involved
• Responsibilities are vague
• Manageable versus flexible
• …
-
Some solutions
• Define single point of contact
• Set up processes
• Arrange test data
• Work with environment types
• Use master/copy
• Install shared environments
• Have an authorization strategy
-
Define single point of contact
• Test environment coordinator
– Responsible for setting up and maintaining environment
– Single gateway to all parties involved
– Combination of technical and testing skills
– Control function
– Communicator and project leader
– In bigger organizations:
dedicated department of
test environments
-
Set up processes - 1
• Define what to deliver when and by whom
• Connection with other processes: design, build,
test and implementation
• Simple process:
1. Specifying the infrastructure
2. Realising the infrastructure
3. Specifying the infrastructure intake
4. Intake of the infrastructure
5. Maintaining the infrastructure
6. Preserving the infrastructure
2 3
5
Start
4
1
6
End
-
Set up processes - 2
Maintaining the environment
– Configuration management
– Change management
– Release management
Wish Use
Dev Test Acc Prod
Configuration management
Change
Release
-
Arrange test data
• Define test data
– Production data (real or scrambled)
– Test data (regular system functions or separate)
• Naming test data
– Real names
– Functional names
– Test related names
• Maintain test data
– Cumulative
– Periodic restore
– Several versions
-
Work with environment types
• DTAP –Model, 4 types of environments
– Development
– Test
– Acceptance
– Production
• Not a technical solution!
• Each type has it’s own characteristics on:
– Maintenance and ownership
– Authorization
– Flexibility
-
Example
Local development
STenvironment
UATenvironment
Development Test Acceptance Production
Environment in project
Environment type acc. to DTAP
Central development
PATenvironment
Productionenvironment
Shadowenvironment
-
Use master/copy
• Principle for setting up environments
Master
CopyCopy
Copy
• What is the master?
• What is in the master?
• Who owns the master?
-
Install shared environments
• An end-to-end environment
• Consistent end-to-end test data
• Shared use by different projects
• Well defined user processes
• Always up and running
• Continuous maintenance
-
Have an authorization strategy
• How to deal with authorization in the
environment?
• Test users or real life users?
• Impersonal users (roles like “guest”) or
personal users?
-
The case!
ITO ProjectIntegrale Test Omgeving(Integral Test Environment)
-
The situation
• Dutch bank
• Department Environments
– Responsible for all test environments
-
Single point of contact
Department environments
Coordinator environments
Maintenance&
Exploitation
Outsourceparties
Projects
External partners
Deliveryunit
Unix
Local
Central
Test data
-
Processes
-
Current situation
• Availability environments
– Demand > supply
– Temporary solutions
• Maintainability environments
– Versions
• Test data environments
– End-to-end consistency
• Complexity
– Replacement old client systems (OLI > SIEBEL)
– Connections
-
The environments
OLI (25)
X (25)
Y (20)
Z (15)
A (10)
B (5)
Siebel (2)
-
Replacement old client systems
OLI (25)
X (25)
Y (20)
Z (15)
A (10)
B (5)
Siebel (25)
-
end-to-end shared environments
OLI (5)
X (5)
Y (5)
Z (5)
A (5)
B (5)
Siebel (5) Connected systemsConsistent test dataMaintainedShared useRules
-
Goals
• Reduction of systems
• Transparency
– You know what’s in the environment (data, software,
hardware, connections etc)
– It’s consistent
– All users are known
• Shorten setup time
– It’s already there
– Only adjustments of connections
-
Results
-
Quaboo
Ludwigstraat 39
4701 NE Roosendaal
The Netherlands
M. +31(0)6 21 489 775
I. www.quaboo.eu