EASY TEST AUTOMATION of

INFRARED DEVICES & APPLICATIONS

for TV middlewares, Smart TV apps & devices.

by

Increase ARPU, by increasing time-to-market of new services

Decrease the number of unwanted calls to contact center, by discovering low probability issues early

Empower engineering team to confidently undertake risky refactoring's by providing means to rapidly test multiple configurations and device platforms.

BUSINESSBENEFITS

CHALLENGES

HOW TO MAKE SURE TESTING IS EFFECTIVE?

■ being confident in what has been tested

■ identifying problems of low probability (race conditions, app crashes, etc.)

HOW TO REDUCE TEST CAMPAIGN DURATION?

■ testing using actual target environments

■ being able to reproduce found problems

■ 24/7 testing

HOW TO CONDUCT RISKY REFACTORING'S?

LAYER UPON LAYER OF DEPENDENCIES

Complex end-userGUI application

SDK

Browser

Operating System

Firmware

STB/TV device hardware

TENS TO HUNDREDS OF THOUSANDS OF LINES

OF COMPLEX APP CODE

COMPARISON of ALTERNATIVES

- Ridiculous amount of clicking after each change in code

- Accuracy relative to tester

- Time is limited to working hours

- QA engineer training usually takes 3-6 months

+ Superior intelligence

+ Easy to create and manage test portfolio through single web portal

+ Tests not sensitive to UI changes

+ Provides steps to reproduce the problem

+ Works 24/7, unlimited number of test devices

+ Same test can be used in parallel on different devices/platforms

+ Start effective testing from day 1!

- Not able to test video related quality of experience

- Setup, training and test portfolio build-up take about 1 month, maintenance overhead very high

- Tests very sensitive to UI changes

+ Provides steps to reproduce the problem

+ Works 24/7, unlimited number of test devices

+ Able to test video related quality of experience

MANUALTESTING

SIMOTEPLATFORM

OCR BASEDTOOLS(WITBE, TAKT ENGINE, STB TESTER)

��× ∞

Fix problems, repeat tests to verify

HOW SIMOTE WORKS

Uses human-recorded tests on chosen application/configuration versions’ in mass-volumes to establish stability, performance and functional quality metrics.

Record tests with standard IR remote control

Connect test devices to Simote, play recorded tests

“Steps to Reproduce the Issue” are provided for all problems identified.

1 2 3

SOLUTION HIGHLIGHTS

■ Create and manage tests with just standard remote control – no scripting required!

■ Test portfolio not sensitive to UI changes!

■ Functional testing

■ Performance testing

■ Stability testing

■ Randomizer aka “Monkey” test

■ Use unlimited number of parallel test devices (incl. use the same test in parallel on LG, Samsung, Motorola, etc.)

■ Steps to reproduce errors shown

■ Comprehensive reporting & analysis

■ Automatic log collection per found error from devices

EASY TO USE! MINIMAL TEST MAINTENANCE

OVERHEAD!

SETUP LESSTHAN 8H

Coming soon: DOM structure analysis & JIRA integration

Monkey test – probabilistic key press automation:

1. Define desired key presses

2. Configure probability of each key press relative to each other

3. Start testing (let Monkey out of the cage)

WHAT IS “MONKEY” TEST?

Captures the same quality metricsas any custom test case portfolio.

DEVICE RESTARTS/CRASHES■ State transitions that led to restart■ Timestamp■ Associated device

APPLICATION ERRORS ■ State transitions that led to restart■ Timestamp■ Associated device■ Error message, associated file and line number

APPLICATION PERFORMANCE METRICS■ Minimum, Maximum, Average and count of GUI view’s covered by test run

ANYTHING ELSE THROUGH SCRIPTABLE API

WHAT EVENTSSIMOTE CAPTURES?

MOTOROLA/ARRIS DEVICE LOG LINKED TO ERRORS/RESTARTS

EXAMPLE REPORTS

MOTOROLA/ARRIS DEVICE LOG LINKED TO ERRORS/RESTARTS

DEVICE RESTARTS

APPLICATION ERRORS

PERFORMANCE

CASE STUDIES

■ 170 000 IPTV customers■ 3 different middleware

applications■ 5 platforms: Amino, Motorola,

Arris, Samsung and LG■ Multi-location development

team

SIMOTE USED FOR:

■ 24/7 regression testing on Motorola, Arris, LG and Samsung platforms during development

■ Developers use to discover app memory leaks

■ Release candidate end-to-end testing by QA

■ 125 000 IPTV customers■ 1 middleware application■ 2 platforms: Motorola, Arris■ Multi-location development

team

SIMOTE USED FOR:

■ 24/7 regression testing on Arris platform during development

■ Release candidate end-to-end testing by QA

END TO MANUAL TESTING AS WE KNOW IT

See what Simote can do for you!��Contact for demo:�

Siim Puskai�

siim@stagnationlab.com�+372 51 53385�skype: siimpuskai