cerberus : framework for manual and automated testing (web application)
TRANSCRIPT
![Page 1: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/1.jpg)
Cerberus
![Page 2: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/2.jpg)
What is it ?Web Application .
Developed by La Redoute since 2011, then published in Open source.
Source code and documentation available on github and sourceforge :https://github.com/vertigo17/CerberusAlmost 2600 commits by 10 contributeurs
Centralized test repository.Automate multi-technologies. (Web, Application Mobile, Heavy Client, SOAP, Rest, SQL…)
Multi-environment (DEV, QA, UAT, PROD….)Multi-langages (FR, UK, RU, IT, PL, …)
![Page 3: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/3.jpg)
Business
Development TeamTest Team
Where does it come from ?• From the observation of the limits of a traditional organization.
• Silo Organisation.• Test definition during or after development phase.
The possible interpretation of a functional specification can lead to a number of significant go back .
The test case, correctly described in earlier in the development phase , is the most effective way to prevent the poor quality .
Need for a common test repository to the various players , fed earlier in the development phase.
![Page 4: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/4.jpg)
Where does it come from ?• From the experimentation of new organizations : Agile / Continuous Integration /
Devops
time
risk
timeShort iteration
Long iteration
Need for test automation
No automated Tests = No quick feedback = No short iteration = Increasing risk to production deploy.Regression risk evolution vs delay between each deploy
DevOps is an effort to create business value by involving all stakeholders in the software development ( from design to production , including the developers ) through short iterations and recognize the value closely , consists of experimentation and rewind.
![Page 5: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/5.jpg)
Where does it come from ?• From the experience in test automation.
To guarantee test maintenability.
a.Avoid code duplication Use step library.
b.Guaranty functional test coverage. Functional test coverage should be defined by business/MOA team.
c. Deploy one unique repository. Link functional information with technical implementation to allow to easily maintain one unique
repository.
Need for a test framework, including functional and technical information.
Login
Product
Basket
![Page 6: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/6.jpg)
Test Automation
Web (Selenium)IOS and Android Apps (Appium)
Heavy Client / GUI (sikuli)Web-services (xml unit)
Databases (sql connectors - jdbc)
Why do it ?• A common test repository to the various players , fed earlier in the development
phase.• Test automation whatever the technology.• Test framework, including functional and technical information.
Test Repository
Functional description ACTION / CONTROLES / DATATest centralisationStep libraryTest description standardisation.
Testing Management
Testcase execution Dashboard.Performance metrics.
Log (screenshot).Standardize test reporting.
Link to ticketing system.
![Page 7: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/7.jpg)
Cerberus Usage: In the context of the project.
![Page 8: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/8.jpg)
Business
Dev TeamQA Team
Business
Dev TeamQA Team
Cerberus usage : in the context of the project
Team exchanges around a centralized tool.
Become
![Page 9: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/9.jpg)
Cerberus usage : in the context of the project1 – Test specification allowing to guaranty the quality of the project
• Who ? Business, AMOA.
• Where ?In the description fields.
• How ?Following the frame (Action/Control).Using step library and data library.
• When?As soon as possible in the project.Ideally during functional specification.
![Page 10: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/10.jpg)
Cerberus usage : in the context of the project2 – Test Automation
• Who ? Developers, QA Team.
• Where ?In the script fields.
• How ?Following the frame.
• When?During development phase.When user interface are designed.
![Page 11: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/11.jpg)
Cerberus usage : in the context of the project3 – Test Execution
• Who ? All project actors.
• Where ?Manual & Automated. In the test execution page.
• How ?Manual: Allow to define status for each action.Automated : Launched manually or for continuous integration chain.
• When ?During the test phase.
Manual execution
![Page 12: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/12.jpg)
Cerberus usage : in the context of the project4 – Test project management
• Who? Every project actors.
• Where ?Reporting page
• How ?Follow execution status.Interface with several ticketing tools.
• When?During the test phase.
![Page 13: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/13.jpg)
Cerberus usage : in the context of the project
Synthesis:
1 Test specification.• Suivant le cadre défini (Data/Actions/Contrôles)• En utilisant les données et étapes de libraires
Test automation (20/80)• Pour le périmètre qui n’est pas en librairie / données
manquantes• Permettant de valider les cas passant et quelques cas
bloquant
2
Project test execution (manual)3
Business
Dev TeamQA Team
3
GO/NO GO on common reporting
Project testing become regression testing for next projects.
3 Project test execution (automated)
Regression test execution (automated)
444
5
![Page 14: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/14.jpg)
Regression Testing : key figures
La Redoute :For the perimeter www.laredoute.xx (10 countries) and m.laredoute.xx (10 countries) : 3500 tests launched twice a day (UAT and PreProd). 4 production deploy for each application. Speed of test creation increasing, due to library usage.
Evolution du périmètre des TNRs
Aout 2014180 Tests
Aout 20152000 Tests
Janvier 20162900 Tests
Avril 2016 3500 Tests
![Page 15: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/15.jpg)
Another usage: Functional Monitoring
![Page 16: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/16.jpg)
Functional monitoring Run functional to guaranty .
1
23
1
Functional test execution(Example : 10 scenarii every 5 minutes)
Technical data records• Response time per action• Network Trafic (HAR file)
2
Data exploitation• Services allowing to raise alert on
non OK status.• Publish data into Elastic Search
/Kibana• Business Activity Monitoring
3
![Page 17: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/17.jpg)
Functional monitoring : key figures
Follow the availability of the key scenarios:Homepage / Authenticate / Account creation / Navigation / Product List / Product Page / Comparison / Basket / Delivery / Payment
At a pertinent frequency:Every 5 minutes, 21h a day
>> 24.000.000 step every year for1 application
Decorelate the cost of the monitoring from the execution frequancy allow to avoid to opposite quality with cost of the necessary solution.
![Page 18: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/18.jpg)
Integration into IT service
![Page 19: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/19.jpg)
Cerberus in an Information Technology service
Cerberus can be interfaced with a lot of popular tools already in place in most of the IT service
Oracle SQLMySQLPostGreSQLDB2Microsoft SQLServerSSAS
Can be launched by any task scheduler.• Via 1 api REST• $U, cron, Jenkins….• Execute 1 test or even a campaign
Push execution status.• OK/KO for a go/no go status
Push into ES/Kibana
Automatic ticket opening.• In Mantis / Redmine / ….• Forward testcase information
Can be intefaced with Centreon
Allow to Read/Write any Database.
![Page 20: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/20.jpg)
Architecture
Application serverActif / Actif(Glassfish / Tomcat / Jboss)
DatabaseActif/Passif(MySQL / MariaDB)
Robots Selenium / Sikuli(GUI Testing)
Front
Back
Robot
Core
![Page 21: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/21.jpg)
Architecture : For a POC
Technical information:1 VM (Linux)4 vCPU8 GB RAM50 Go Disk Space
Application :MySQL (v5.6.xx)Glassfish 4.1.1
Technical information:3 VM (Linux)2 vCPU4 GB RAM10 Go Disk Space
Application :Selenium serverFirefox
Front
Back
Robot
Core
![Page 22: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/22.jpg)
RoadMap 2016
![Page 23: Cerberus : Framework for Manual and Automated Testing (Web Application)](https://reader035.vdocuments.net/reader035/viewer/2022062522/589ed9551a28ab39498b57c5/html5/thumbnails/23.jpg)
RoadMap 2016
https://github.com/vertigo17/Cerberus/issues
• GUI Refactoring (70% Done)• Bootstrap• Interface Internationalisation
• Interfacing with Jmeter• Load test scenario centralisation• JMX generation (in study)• Trigger Execution
• TestCase Versionning
• Plugin management• Add action and control as plugin