scalable test automation tty -testauspäivä 2016 · scalable test automation tty -testauspäivä...

Company ConfidentialCompany Confidential

Scalable Test AutomationTTY - Testauspäivä 2016

Sakari Hoisko

Company Confidential

WE ARE SYMBIO

Symbio is a global BPO and R&D technology development company offering solutions and services for end-to-end customer care. We help clients develop technologies and customer care services that enable them to manage their business and customer relationships more efficiently. Our vision is to become the world’s leading provider of R&D Engineering and BPO services. Together with our partners, we design, build, implement and support innovative customer engagement platforms by seamlessly integrating our robust customer insights, technology and world class business services.

2


WHY AUTOMATION?Helping you focus what is most important!

3


4

TOOLS


ROBOT FRAMEWORK Compact sheet: https://public.centerdevice.de/0db44f0e-c485-41be-ba4a-85218b9b0542

› Test Automation Framework with html and XML result output

› Include tagging and basic setup/teardown tree functionality to suites and testcases

› Has a flexible keyword-driven approach to describe and implement tests.

› Keywords can be stored in Resource-Files to separate implementation details from tests.

› Main programming languages for implementing test functionality are Java (JYBOT) and Python (PYBOT).

› There are ready-made test libraries available for a lot of technologies.

› Implementing tests is supported by the Robot Framework IDE(RIDE).


DOCKER –> PACK –> SHIP –> RUNCheat sheet: https://github.com/wsargent/docker-cheat-sheet

› PACK software to docker container image

- Defined by using docker file whitch is pure text file

- Include just SW and it dependencies, nothing else.

- Built from docker file

- Minimize maintain of platform / security / updates

› SHIP software by using

- Container image docker file

- Container image by using private or public registry

› RUN docker container

- Isolated user space, shared kernel space

- Running application by single “docker run” command from “docker image” template

- Is like lightweight “virtual machine” without OS and HW emulation

- Include HW resources allocation, clustering, HA, networks, port mapping, etc…


ROBOT TEST SUITES / CASES DESIGN

› Unique independed test suites- Test cases are executed parallel so there should not

be dependency between test cases which effects to other testcase in any phase/step of test case execution

- Test data should be unique

- All actions inside of test case should be unique (=if same table is droped and added in different testcases we have a problem).

› Timings- All timings should be solved by loops / triggers /

polling (=not sleeps/etc ugly hacks)

- Is service up&running? Test it by using “run keyword until” keyword functionality.

Keep parallel execution in mind while TA developing.


PABOT VS (PY/JY)BOTDetails: https://github.com/mkorpela/pabot

› Parallel executor for Robot Framework test suites by using pybot / jybot

› With Pabot you can split one execution into many and save test execution time.

› Implemented as library to Robot Framework

› Speed up

- CI pipeline

- Test automation

- Developer feedback

› Require

- multiple robot host instances with same library/dependency contents

- Instances are defined to valueset.dat whitch is provided to pabot call as argument

- pabot.PabotLib to test suite(s)/test case(s) (=provides keywords that will help communication and data sharing between the executor processes).

› Include locks that could be used to sync test suites/cases between each other inside of pabot cluster

VS.VS.

J

Y

B

O

T

P

Y

B

O

T

PABOT


9

TOOLS TOGETHER => SCALABLE TEST AUTOMATION


ROBOT FRAMEWORK IN DOCKER CONTAINERSame container everywhere.

› TA container image include:

- Pybot, Jybot and Pabot

- Required Robot Framework Libraries

- Tools (=browsers, RIDE, firebot, Zap, etc…)

- SSH server with known account (FYI: pabot doesn’t support key

based auth)

- All of those dependencies with correct order and required versions (=standalone installation means gray hairs to them

who made this without docker)

› Same version of different components are in use both in developer machine as in pipeline (if test suite works in developer

machine then it works in CI TA)

› GUIs could be forwarded outside of container by using X11 forwarding (=cool selling demo to managers when selenium2library is used)

TA

Cluster

TA

Cluster

TA developmentTA development


11

EXAMPLE FROM DEMO –TEST AUTOMATION FRAMEWORK DOCKER FILE


ROBOT FW CONTAINER CLUSTERSame container everywhere.

› Master container

- Started by CI / developer

- Shut down by CI / developer

- All phases are executed in here

- All test results are collected in here by pabot

› Phase 1:

- Input: folder where suites located

- Calculate amount of test suites

- Start own TA container for every test suite (=start TA cluster containers)

- Generate parameter file where TA cluster is defined

- Build up platform to sandbox

- Generate parameter file where platform interfaces are defined

› Phase 2:

- Input: cluster and platform definition file

- Start PABOT (test suites) execution by parallel against platform sandbox

› Phase 3:

- Input: cluster and platform definition file

- Teardown shutdown platform sandbox and TA cluster

TA cluster containers


POOR MAN’S PERFORMANCE TESTCould be part of CI testing pipe.

› Start also system monitoring in phase 1.

› Pick up dedicated test case(s) that loads platform system as widely as you can.- Log in is usually good start some load authentication, DB,

frontend, backend, etc…

› Create test case(s) to keyword resource file

› Call same keyword(s) with uniq test data from multiple test suites parallel by using pabot

› Monitor platform simultaneously when test set is under execution

› Stop system monitorin in phase 3.


14

DEMO TIME - SCALABLE TEST AUTOMATION


DOCKERIZING A ROBOT FRAMEWORKTo get official robot framework docker image to https://hub.docker.com/explore/

› Community based Open Source development

› Figuring out:

- What is base content?

• Execution tools?

• Libraries?

• R&D tools?

• Example test suites/cases (=Demo)?

- Reusable in different use cases?

- How to get content to be more configurable?

- Split configuration between:

• build time configuration (inside of image)

• run time configuration (deploy container from image)

› Contribute your view/points:https://github.com/symbionext/DockerizedRobotFramework


16

FRIENDLY ASK QUESTIONS?


FINAL TIP => CD-PIPELINE WITH DOCKERJenkins integration - Poor man’s perf behind of jenkins in Continuous Delivery pipeline

- Jenkins could start docker containers as slaves dynamically

• With docker plugin: https://wiki.jenkins-ci.org/display/JENKINS/Docker+Plugin

• Master container could be integrated to Jenkins as dynamic slave

- Jenkins could be installed to system by using docker container

• Jenkins official docker container: https://hub.docker.com/_/jenkins/

What else you need to build up test automation to your system?

Jenkins Slave

Container


CONTACT INFOhttp://www.symbio.com/contact-us/


19

THANKS.

scalable test automation tty -testauspäivä 2016 · scalable test automation tty -testauspäivä...

Documents