tariq m. king
DESCRIPTION
Towards a Self-Testing Cloud Computing Infrastructure. Tariq M. King. STITC 2011. Markham, Ontario, Canada. Co-located with. Outline. Introduction Research Questions Cloud Testability Test Support as-a-Service Vision: Self-Testing in the Cloud Summary Open Discussion. Introduction. - PowerPoint PPT PresentationTRANSCRIPT
NORTH DAKOTA STATE UNIVERSITY DEPARTMENT OF COMPUTER SCIENCE
© NDSU SOFTWARE TESTING RESEARCH GROUP
Tariq M. King
STITC 2011Co-located with
Towards a Self-Testing Cloud Computing Infrastructure
Markham, Ontario, Canada
© NDSU SOFTWARE TESTING RESEARCH GROUP 2
Outline Introduction Research Questions Cloud Testability Test Support as-a-Service Vision: Self-Testing in the Cloud Summary Open Discussion
© NDSU SOFTWARE TESTING RESEARCH GROUP
Cloud provides ubiquitous, on-demand access to computing resources “as services” via the Internet
Operational Perspective Availability, Scalability, Efficiency, …
Development Perspective Reusable Components, Info. Hiding Highly Complex Local Infrastructure
Testing Perspective?
Introduction
3
CLOUD
© NDSU SOFTWARE TESTING RESEARCH GROUP
Cloud brings new testing challenges and benefits Software Testing In the Cloud (STITC)
STITC 2010 Workshop focused on three areas: Testing IN the Cloud – leveraging the cloud for
test execution and testing in a virtualized environment
Testing OF the Cloud – validating applications that are hosted and deployed cloud
Testing TO the Cloud – moving the testing process and other assets to the cloud
Introduction
4
STITC
© NDSU SOFTWARE TESTING RESEARCH GROUP 5
Services in cloud computing typically fall intothree categories:
Software-as-a-Service | Google Docs, Office Web Apps
Platform-as-a-Service | Google App Engine, Windows Azure
Infrastructure-as-a-Service | Amazon EC2, Rackspace
Introduction CLOUD SERVICES
© NDSU SOFTWARE TESTING RESEARCH GROUP 6
Many cloud platforms offer Virtualization support Creating abstract computing resources from
more powerful physical resources
Hosted Architecture
Introduction VIRTUALIZATION
Hypervisor Architecture
© NDSU SOFTWARE TESTING RESEARCH GROUP 7
Introduction Cloud Computing shares characteristics with
Autonomic Computing (AC), Grid, Utility, …
Central theme of AC is self-management to reduce burden of human administration and integration
King et. al 2008 introduce animplicit self-test characteristic
Regardless of self-* featuresimplemented, system shouldvalidate adaptations at runtime
AUTONOMIC COMPUTING
Self-Configure
Self-Optimize
Self-Heal
Self-Protect
Self-Test
© NDSU SOFTWARE TESTING RESEARCH GROUP
“Migrating Autonomic Self-Testing to the Cloud” Tariq M. King and Annaji S. Ganti | STITC 2010 High-level position on how a self-testing
approach for autonomic software can be applied to the cloud
In today’s talk, we highlight our recent progress and developments from the above perspective Extended version of King et al. | CASCON 2011 Summarizes contents of book chapter submitted
to STITC: Perspectives on an Emerging Discipline
Introduction
8
ABOUT THIS TALK
© NDSU SOFTWARE TESTING RESEARCH GROUP 9
How does cloud computing affect our ability to test applications that are either built or deployed in a cloud environment, or depend on cloud services?
Research Question #1
Testability
Investigates impact of cloud development
model on software testability
(Positive, Negative)
© NDSU SOFTWARE TESTING RESEARCH GROUP 10
How can the test artifacts and tools used to validate a cloud application service during development, be reused to help test derivative applications?
Research Question #2
Test Reuse
Investigates ways to reuse outputs of pre-deployment
testing effort
TEST (Artifacts, Tooling)
© NDSU SOFTWARE TESTING RESEARCH GROUP 11
How can built-in cloud computing technologies such as virtualization be used to improve the process of testing cloud applications?
Research Question #3
Cloud Leverage
Investigates ways in whichthe cloud infrastructure
itself can be harnessed for Testing OF the Cloud
(Power, Virtualization)
© NDSU SOFTWARE TESTING RESEARCH GROUP 12
Cloud Testability: The Negative
Complexity
Remoteness
InformationHiding
Autonomy Adaptiveness
Statefulness
High Performance
Criteria
High Dependabili
tyCriteria
(Paradigm Infancy)
© NDSU SOFTWARE TESTING RESEARCH GROUP 13
Computational Power
Storage
Testing requires lots of resources and the cloud is certainly powerful enough to handle it
Virtualization
Cloud Testability: The Positive
© NDSU SOFTWARE TESTING RESEARCH GROUP 14
As a step towards Research Questions 2 and 3, we investigate a novel approach to software testing of the cloud, in the cloud | King et al., CASCON ‘11
Scenario: One host (Provider B) is developing an application service that will extend functionality of a service offered by another host (Provider A)
Testing of the Cloud, In the Cloud
© NDSU SOFTWARE TESTING RESEARCH GROUP 15
Even with its limited scope, our scenario reveals many cloud integration testing challenges: Difficult for Provider B to Setup Some Tests
for his application since he does not have full control and observation of Service A.
Developing Accurate Stubs is Hard since Provider B has no knowledge of Service A’s implementation
Provider B Limited to Black Box Testing of those parts of his application implemented by Service A
Service B can Change w/o Provider B’s Knowledge i.e., Update to Service A Need to Retest Service B
Integration Testing Challenges
© NDSU SOFTWARE TESTING RESEARCH GROUP 16
Our proposed solution introduces the notion of Test Support as-a-Service (TSaaS) in the cloud
Prior to the deployment of Service A, Provider A would have tested it, and created: Test Artifacts – Test Drivers, Test Stubs, Test
Data Local Test Bed – Hardware/Software
Environment
TSaaS seeks to reuse existing test artifacts, tools and configurations of the local test bed, to aid the rapid development and delivery of test support services for collaborating partners
Proposed Solution
© NDSU SOFTWARE TESTING RESEARCH GROUP 17
Provider B has full access to a test copy of Service A Test hooks, instrumentation, and scaffolding
reuse Test data (non-confidential) can be made
available Testing does not interrupt service in
production
Concerns: Maintenance, Scalability, Security
Test Support as-a-Service (TSaaS)
Provider A exposes TSaaS to Provider B
© NDSU SOFTWARE TESTING RESEARCH GROUP 18
Hypervisor-Based Virtualization and Secure Access
TSaaS Architecture
© NDSU SOFTWARE TESTING RESEARCH GROUP 19
TSaaS Control Flow
Control Flow When Realizing Test Support Services
© NDSU SOFTWARE TESTING RESEARCH GROUP 20
Using Existing Test Automation to Guide Development
TSaaS Development Method
© NDSU SOFTWARE TESTING RESEARCH GROUP 21
Designed with goal of making its components easily deployable to Windows Azure®
Web & Worker Roles (ASP.NET, IIS7, WCF Services)
VM Role (Windows 2008 R2 Server Image)
Application Domain: Credit Reporting/Banking
TSaaS Prototype DETAILED DESIGN
© NDSU SOFTWARE TESTING RESEARCH GROUP 22
We were able to develop and expose a cross section of test support operations via the TSaaS API Test Specification – user defines fixture
operations
Test Execution – runs specified fixture operations
Code Coverage – analyzes branch and lines covered
Server-Side Diagnostics – facilitates configuring web server, including performing a reset after failures
Runtime Virtualization – create, start, stop, delete, take snapshot, and update VHD configuration
TSaaS PrototypePROGRAMMING INTERFACE
© NDSU SOFTWARE TESTING RESEARCH GROUP 23
TSaaS PrototypeGRAPHICAL USER INTERFACE
© NDSU SOFTWARE TESTING RESEARCH GROUP 24
Vision: Self-Testing In The Cloud
Service A (Production Env.)
Monitor
Analyze Plan
Execute
Knowledge
Autonomic Test Manager
TSaaS
SensorProvider A
Provider B
Test Harness
Service B(Test Environment)
Provider A builds a
sensor for Service A
Provider B monitors Service A
for changes
If a change occurs, Provider B generates a plan for regression testing of Service BProvider B reruns
tests on latest build of Service B, which is set to use new
version of Service A via TSaaS
© NDSU SOFTWARE TESTING RESEARCH GROUP 25
Test Support as-a-Service Improved Testability and Confidence
(Partner) Control/Observation, White Box Testing,
Metrics
Improved Product (Provider) Partners test your component, bug
reporting/fixes
Virtualization Benefits Snapshots: Speed Up, Bug
Replication/Debugging Scalability, Ease of Maintenance, Hardware
Savings, and Environment Testing (OS, Applications)
Foreseen Benefits
© NDSU SOFTWARE TESTING RESEARCH GROUP 26
Self-Testing in the Cloud via TSaaS Automatic Regression Testing of changes
due to: Software maintenance and evolution Dynamic adaptation and updating
Improved Autonomic Decision-Making Use of validation to determine which service
may be the best candidate to select during adaptation
As a proactive form of self-healing (fault detection through formalized testing techniques)
Foreseen Benefits (cont.)
© NDSU SOFTWARE TESTING RESEARCH GROUP 27
Standardization Providing standardized interfaces and
guidelines for performing testing in the cloud, and developing test support infrastructures
Evaluation Providing convincing evidence that these
approaches are not only feasible to implement but can have real value to practitioners
Validation Showing virtualization is consistent with
production
Key Challenges
© NDSU SOFTWARE TESTING RESEARCH GROUP 28
Described our progress towards our grand vision of migrating autonomic self-testing in the cloud Proposed an approach to Test Support as-a-
Service,which uses the cloud to tackle testing of the cloud
Described the design and implementation of a a prototype of TSaaS
Discussed benefits and key challenges associated with TSaaS and use for self-testing OF/IN the cloud
In terms of our progress, we have barely scratched the surface of the research problem under investigation
Summary
© NDSU SOFTWARE TESTING RESEARCH GROUP 29
Acknowledgements Annaji Ganti | PhD Student,
NDSU/Microsoft
Neha Kale | MS Student, NDSU
David Froslie | Test Architect, Microsoft
Alisson Sol | Knowledge Engineer, Microsoft
Dr. Scott Tilley | Professor, Florida Tech
Participants and Reviewers STITC 2010 CASCON 2011
© NDSU SOFTWARE TESTING RESEARCH GROUP 30
Thank You! Questions?
¿Preguntas?問題Sawwalвопросы質問domandeερωτήσεις