Image Based Testing- application technology

independent automation

Girish KolapkarSAS R&D

(India)

Agenda:• What exactly IS Image-Based Testing (IBT)?• OBT vs IBT• Thinking and Testing Differently with IBT• Developing IBT Tests and App Maps• SAFS Image-Based Recognition Syntax• How to automate using IBT• Image Manager Tool• Sample Application Map And Script

Agenda:• Dealing with Variations in the IBT Environment• Dealing with NLS Testing in an IBT Environment• Challenges• Demo• Q & A

What exactly IS Image-Based Testing?

• Strictly “What You See is What You Get” (WYSIWYG)• Interaction with the visual display of the screen• Testing based on finding and interacting with graphics

on the screen• Automation is driven through keyboard and pointer

events• Test technologies partially or not supported by existing

toolso E.g Legacy Applications (Xterm application),Flash

Applications, GUI on Unix

What exactly IS Image-Based Testing?

• An alternative when items are not recognized at all by existing tools

• Independent of any underlying application technology

• The antonym or opposite of Object-Based Testing.

What exactly IS Image-Based Testing?

Image Based Testing Tool

Operating System

Application Under Test

DisplayBuffer

Mouse pointer events/ keyboard events queue

OBT vs IBT

Object-Based Testing• Ideal for automating PC

hosts • Performance: slower than

IBT• Can automate supported

technologies only• Can produce reliable &

robust scripts(time tested)• Wide range of functions

available to use

Image-Based Testing• Ideal for Applications which

OBT can’t support or limited support e.g. GUI on non PC hosts, legacy applications

• Performance :faster than OBT

• Can automate virtually anything that can be displayed on the screen

• Needs to be proved\improved after extensive usage

• Limited verifications

Thinking and Testing Differently for IBT

• Object-Based Testing deals with real components.• Image-Based Testing deals with images.• IBT doesn’t look for any object hierarchy • Application Map or Object Map is defined by images

not by properties• All possible Image variations needs to be stored

SAFS Image-Based Recognition Syntax

• SAFS keeps concept of Components in Windows

• A “component” can be: an image inside another image.

an image inside the bounds of other images.

Component as an Image inside another Image

Finding the right Arrow in a Sea of Matching Arrows• Locate the Unique Anchor that contains the Target• Locate the Target within the Anchor

Component Function call:BulletsItem, ArrowButton, Click

• A “component” can be: an image inside another image.

an image inside the bounds of other images.

BulletsItem ArrowButton

Component as an Image inside the bounds defined by other Images

Finding a Target inside the bounds from a single image Component Function call:OfficeWin, NewSlide, Click

• A “component” can be: an image inside another image.

an image inside the bounds of other images.

OfficeWin NewSlide

Component as an Image inside the bounds defined by other Images

Find a Target inside the bounds defined by multiple imagesComponent Function call:OfficeWin, CenterText, Click

• A “component” can be: an image inside another image.

an image inside the bounds of other images.

OfficeWin CenterText

SAFS Image-Based Recognition Syntax

• Component is an image inside another image. [BulletsItem]

BulletsItem =“Image=pathTo\BulletsItemImages” ArrowButton=“Image=pathTo\ArrowButtonImages”

• Component is inside bounds defined by other images. [OfficeWin]

OfficeWin =“Image=pathTo\OfficeAnchorImages; _ImageRight=pathTo\OfficeCloseIcons”

CenterText=“Image=pathTo\CenterTextImages”

BulletsItem

ArrowButton

OfficeWin

CenterText

How to automate using IBT• Test Development for IBT is the same

Fewer Component Functions supported. Tests and commands look the same as OBT.

• App Map development is mostly the same Recognition strings are VERY different.

It’s all about images, not components.e.g.IExplorer App Map Entry:[IExplorer]IExplorer="Image=<imagepath>"

How to automate using IBT

• Capture images in supported formats o BMP, GIF,JPEG,PNG,TIFF etc.

• Map logical names for the "window" and the "component" in the application map

• Write test records\scripts same as OBT• Execute records using SAFSDRIVER• Review the results

Image Manager Tool

• Tool to facilitate image captureso Command to launch

java org.safs.image.ImageManager

Enhancements

BitTolerance|BT=Optional. Specifies the integer percentage (1-100) of image bits or pixels that must match for an image to be considered a successful match. The default is, of course, 100. This means ALL pixels must match unless some other BitTolerance is specified.Samples:IExplorer="Image=<imagepath>;BitTolerance=70"IExplorer="Image=<imagepath>;ImageR=<imagepath>;BT=75"

Sample Application Map

[SampleApplication]SampleApplication="Image=c:\Images\AnchorImage.bmp;ImageR=c:\Images\CloseIcon.bmp"ButtonMinimize="Image=c:\Images\MinIcon.bmp"ButtonMaximize="Image=c:\Images\MaxIcon.bmp"ButtonClose="Image=c:\Images\CloseIcon.bmp"

Sample Test Records

C SetApplicationMap Demo.MAPC LaunchApplication SampleApplication "c:\safs\samples\Dotnet\DotNetApp\WinDemo.exe" C WaitForGUI SampleApplication SampleApplication 15 T SampleApplication SampleApplication GetGUIImage c:\OutputImage1.jpg T SampleApplication SampleApplication RightClick T SampleApp SampleApp InputKeys "x" T SampleApplication SampleApplication GetGUIImage c:\OutputImage2.jpg T SampleApplication ButtonClose Click

Dealing with Variations in the IBT Environment

• 32-Bit, 24-Bit, 16-Bit Color Depth and Resolution• Themes and Schemes• Platform and Version Image and Icon Variations •  Multiple images are necessary if the target image is

different in different environments• Identify and Store Item Variations in a Directory

Dealing with NLS Testing in an IBT Environment

• Same as any Recognition String handling for NLS:Item1=pathTo\{^SHARED_DIR}\targetImages

Item2=pathTo\{^LANG_DIR}\targetImages

• Images will have to be captured and transferred if there is a language dependency

Challenges

• It’s all about imageso Require to capture all possible image variationso Precise\Reliable\Unique Imageso Display configurationso Time consuming test developmento Verification

Image verification only Can’t verify component properties Can’t extract data from application under test

• Themes and Schemes• Platform and Version Image and Icon Variations • Identify and Store Item Variations in a Directory

Demo

• What the IBT demo doeso Launch Sample Applicationo Perform few interactionso Closeo Review the results

Q&A

Thanks

• References :o SAFS Image-Based Recognition

http://safsdev.sourceforge.net/sqabasic2000/SAFSImageBasedRecognition.htm