software testing in the cloud · •system migration stitc (software testing in the cloud)...
TRANSCRIPT
![Page 2: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/2.jpg)
About Me
• University Department Chair, Software
Engineering, Keiser University
• PhD, CS, Florida Institute of Technology
• MBA, University of Central Florida
• ISTQB Certified Tester (Foundation, Manager, Agile)
• Independent consultant and trainer
• Software Quality Engineer/ Tester
• Yahoo!, Sabre, Wikimedia, …
![Page 3: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/3.jpg)
About Me…
Vice President & COO
![Page 4: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/4.jpg)
Motivation
• Cloud computing is here to stay
• Changing the way we engineer software
• Leverage the power of cloud in testing
![Page 5: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/5.jpg)
Objectives
What is the cloud about?
How is it different from traditional computing?
What is testing in the cloud?
What does it mean for testers?
Where to start?
![Page 6: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/6.jpg)
What is Cloud Computing?
An old idea whose time has finally come
If computers of the kind I have advocated become the computers of the future, then computing may someday be organized as a public utility just as the telephone system is a public utility. The computer
utility could become the basis of a new and important industry.
John McCarthy, 1961
![Page 7: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/7.jpg)
What is Cloud Computing?
Virtualized hardware
SOA, On-demand, utility model
Rapid elasticity
Resource abstraction techniques
Multi-tenancy
Self-healing
Not in your control (mostly) / things are taken care of
![Page 8: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/8.jpg)
Cloud Services
![Page 9: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/9.jpg)
Cloud Services
![Page 10: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/10.jpg)
Cloud Services
![Page 11: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/11.jpg)
Cloud Delivery Models
Public • Services offered by a provider that are publicly available, can be used with a
contractual agreement
Private• Virtualized resources available as a service owned by individual organization
• Client managed, customization, high efficiency, privacy and security policies
Hybrid
• Mix of public and private
• Extend private cloud via VPN
• Good candidate for incremental migration
![Page 12: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/12.jpg)
Cloud Benefits
Faster, cheaper, better
Flexible
• Multiple test beds for multiple release testing
• Instant setup, tear down
Scalable
• Virtualized environments
• Scalable resources
On-demand
• Pay-per-use
![Page 13: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/13.jpg)
![Page 14: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/14.jpg)
What Does it Mean for Testers?
How would you test this?
![Page 15: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/15.jpg)
Software Testing in the Cloud
Lies at the intersection of three key areas
• Software Testing
• Cloud Computing
• System Migration
STITC (Software Testing in the Cloud)
• Testing in the cloud
• Testing of the cloud
• Migrating testing to the cloud
http://www.stitc.org/
![Page 16: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/16.jpg)
Traditional Testing
Low asset utilization
Scalability: Long time to increase capacity
Long time to build datacenters
Difficult to manage
Duplicate test systems
Creates unnecessary waste
Testing in the Cloud
Improved asset utilization
Less time (instantaneous) increase and reduction in capacity
Purchased as a service from cloud providers
Better management and increased productivity
Aggregated system
Cleaner, greener testing
![Page 17: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/17.jpg)
Case Studies
Case Study #1: Concurrent execution of JUnit test cases
Case Study #2: Concurrent execution of GUI testing of iOS applications
• It can take a large number of test cases to cover all the variations that users could experience
• Different configurations
• OS versions (iOS ?)
• Device Orientations (Portrait, Landscape)
• Screen sizes (3.5", 4")
• Form Factors (iPhone, iPad, Universal)
![Page 18: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/18.jpg)
Case Study #2
![Page 19: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/19.jpg)
Software Testing in the Cloud
![Page 20: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/20.jpg)
Software Testing in the Cloud
Management
• Process
• Technology
• People
Cloud testing strategy
Roles and responsibilities
Can’t be achieved as an execution-only
activity
![Page 21: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/21.jpg)
Management: Process
Strengths
•Faster
•Cheaper
•Better
Weakness
•Network connectivity
•Security
•Standards
•Availability
•Performance
•Project management
Opportunities
•Faster execution
•Faster feedback
•Efficient testing process
•Value proposition
Threats
•Complexity
•Hype
•Vendor lock
•Tool chain
•Education
•Training
SWOTANAYSIS
![Page 22: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/22.jpg)
Management: Process
Migration Strategy
• Where are you now?
• Where do you go next?
SMART-T: A decision framework to migrate testing to the cloud
Desirable
Faster Cheaper Better
Business Drivers
Feasible
YES
YES
Types of
Applications
Technical Factors
Types of
Testing
Test
Execution
Environment
Acceptable
NO
NO
YES
NO
Stakeholders
WorkshopPilot Study
Evaluation
Results
Trial Migration
Adjust
Estimates
Document
Guidelines
Implement
Migration
Plan
Actual Migration
MAYBE
MAYBE
MAYBE
Operational Results
![Page 23: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/23.jpg)
Management: Process
Evaluate current overlapping activities
• Agile development and testing
• Test early, test often
• TDD, Unit testing
• JUnit in the cloud
• Google Test in the cloud
• HadoopUnit
• Faster feedback
• Automation
• Regression testing
![Page 24: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/24.jpg)
Management: Process
Cost model
• How much does it cost you to test now?
• Do you have a test lab? Machines for every project?
• How often do you configure the machines?
• How often do you upgrade them?
ABC: Activity Based Cost model – Identify and put $$ to all the big and small activities
![Page 25: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/25.jpg)
Management: Technology
![Page 26: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/26.jpg)
Management: Technology
Testing as a Service
Distributed processing technologies
Test execution environments
![Page 27: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/27.jpg)
Management: Technology
Amazon Web Services (AWS)
![Page 28: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/28.jpg)
Management: People
Will your testers be able to adapt to cloud testing?
• Testing at many levels – User Interface, Services, Governance, Process, Security, Integration
Programmers?
• Beyond unit testing
System administrators?
• Cloud service providers
Training
Continuing education
Testing skills, programming skills
Communication skills
![Page 29: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/29.jpg)
Testing in the Cloud: Strategy
Does testing in the change the foundation of testing?
Does it involve new technique, technology, and domain?
How is it going to effect your current tasks? (good/bad)
Do you need new skills?
Are there different types of testing?
Are there tools to help you?
Risks associated with cloud testing
Assumptions, constraints
![Page 30: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/30.jpg)
Testing in the Cloud: Strategy
Critical Success Factors (CSF)
What are the CSF for testing in the cloud?
• Automation?
• Skilled testers?
• Test lab/environment?
How do your measure them?
• Post release evaluation?
• Number of bugs for the release
• Test lab setup and tear down?
![Page 31: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/31.jpg)
Testing in the Cloud: Strategy
![Page 32: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/32.jpg)
Roles and Responsibilities
Developers: Unit testing, Integration testing
Test Analyst: Implement testing in the cloud strategy
Technical Test Analyst: Testers
Customers: Acceptance testing
IT department: POC to cloud provider, monitoring, availability
Cloud providers: SLA, services testing
![Page 33: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/33.jpg)
Lessons Learned
![Page 34: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/34.jpg)
Lessons Learned
Testing Environment is a legacy system
Acceptability of partial migrations
Selecting/Preparing test execution environment
Configuration in the cloud is challenging
The nature of the test cases
False negatives
![Page 35: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/35.jpg)
What Does it Mean for Testers?
How would you test this?
![Page 36: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/36.jpg)
Key Points
Software testing is challenging activity
SOA, virtualization, computational grid, and cloud computing bring efficiency in development process/consumer need
Leverage these technologies in software testing
Plan, start early, know your goals
Understand the holistic system architecture
Understand the migrated test data
Establish communications with the vendor
![Page 37: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/37.jpg)
Key Points
Cloud computing is not hype!!!
Provides the infrastructure to process Big Data
It’s changing the testing landscape
It’s changing the testing profession
We have had similar shifts in the past
Cloud computing is here to stay
Brace for change in testing!!!
![Page 38: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/38.jpg)
Food for thought!!!
WSJ (Nov 6, 2015)
![Page 39: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/39.jpg)
![Page 40: Software Testing in the Cloud · •System Migration STITC (Software Testing in the Cloud) •Testing in the cloud ... •Form Factors (iPhone, iPad, Universal) Case Study #2. Software](https://reader030.vdocuments.net/reader030/viewer/2022040320/5e4ba8656bc6d245184de1f3/html5/thumbnails/40.jpg)
Thank [email protected]
References:• Jamsa: Cloud Computing: SaaS, PaaS, IaaS, Virtualization, Business Models, Mobile, Security• Hausman: Cloud Essentials