EPAM SYSTEMS

UI Automation/White/CodedUI.

common problems & tricks

Dmitry Hes, Tsimafei Avilin, EPAM Sys.

4/6/2015

Summary: The page describes some of the problems in QA Automation that occur

when using UIAutomation (CodedUI, White) to automated desktop applications

and methods for their solution on the example of the project developed with

DevExpress library.

Dmitry Hes, Tsimafei Avilin, EPAM Sys.

1

At the beginning was … UI Automation

Using Automation UI

Using White framework.

Using Visual Studio 2010 Coded UI.

o Working with White

Some tricks to automate unsupported controls.

o UI Automation properties

o Using patterns, on SelectionPatternIdentifiers example

o DevExpress.Xpf.Core.ClearAutomationEventsHelper.IsEnabled

property

o When simple way ends…

Use LegacyIAccessible

Using COM-wrapper

Use Win32 API

Call Invoke()

Use BoundingRectangle

Data caching

Summary: The page describes some of the problems in QA Automation that occur

when using UIAutomation (CodedUI, White) to automated desktop applications

and methods for their solution on the example of the project developed with

DevExpress library.

At the beginning was … UI Automation

Using Automation UI

This framework provides access to user interface elements Win32, Windows

Forms or WPF applications with the possibility to get or set the element property,

emulate user inputs (the official documentation is available here). Note that it is

base for some popular frameworks such as CodedUI and White.

In the UI Automation UI elements are represented as a tree structure of

AutomationElement objects with desktop as a root. The searching of element are

carried out through the tree by conditions specify in PropertyCondition. Some

controls implement patterns (e.g., SelectionItemPattern, SelectionPatternIdentifiers

the pattern for ListBox, CheckBox, GroupBox).

Dmitry Hes, Tsimafei Avilin, EPAM Sys.

2

Control patterns provide a way to categorize and expose a control's functionality

independent of the control type or the appearance of the control.

UI Automation uses control patterns to represent common control behaviors. For

example, the Invoke control pattern uses for controls that can be invoked (such as

buttons) and the Scroll control pattern for controls that have scroll bars (such as list

boxes, list views, or combo boxes). Because each control pattern represents a

separate functionality, their combination describes the full set of functionality

supported by a particular control.

Some advantages of UI Automation:

- free;

- support Win32, WinForms, WPF;

- included in .Net framework.

Drawback:

- the framework works well standard UI controls, but to support custom and

complex controls it’s necessary to make additional implementations;

- a lot of code for controls searching and processing.

Using White framework.

Based on the UI Automation this framework has convenient access to the controls

and their properties. He has the same features as UI Automation. The framework

can be used to test the Win32, WPF, WinForm, SWT and Silverlight applications.

White framework allows you to write tests simpler and more readable.

You can use a ready BDDfy framework (read here), which allows to easily

implement BDD testing with White.

Dmitry Hes, Tsimafei Avilin, EPAM Sys.

3

Using Visual Studio 2010 Coded UI.

Coded UI — Microsoft’s solution based of the same UI Automation, appeared in

Visual Studio 2010 (you can read here). Under this link you can find useful article

how to use AutomationPeer to automate WPF/Silverlight custom controls.

Profits:

- recorder is available, recording user actions for autogenerating tests;

- Visual Studio in-box;

- support from Microsoft, integration in TFS;

- frequently recurring test steps can easily be re-used. It’s up to you how fine-

grained you want to record your test steps. Calling a step (consisting of one or

more actions) is just a matter of a single function call;

- generates C# and XML code by default;

- features fuzzy matching of UI elements. This seems to make the tests more

robust when updating the user interface;

- you can test many different kinds of user interfaces, not just the web.

Shortcomings:

- the same as that UI Automation;

- proprietary software: Coded UI available in more expensive Visual studio

editions: Ultimate, Premium;

- test steps are stored in an XML file (called UIMap) which in turn compiles to

C# code. The files is big and clunky and there currently is no editor or

documentation for it. So if you want to make some changes, things can get

complicated unless you want to re-record an entire test step.

- creating very simple assertions (such as “look for the string “foo” on this web

page”) is a bit clumsy and requires too many mouse clicks. An IE accelerator

would be great!

Dmitry Hes, Tsimafei Avilin, EPAM Sys.

4

- while fuzzy matching works great in most cases, it can get in the way in

others.

Based on the above choice was made in favor of the White framework, since it is

free and makes writing tests easier and makes them look intuitive.

Working with White

It is easy to start working with the White for people who has not did. Even it does

not have such a good guide to use as MSDN, but for beginning White has enough

resources here http://teststack.azurewebsites.net/white/index.html.

To find the required control by AutomatedId property it’s enough to execute Get

method parameterized of control type, and as a function parameter to specify the

desired AutomatedId. For example, for Edit field with AutomatedId = "txtCity":

TextBox button = window.Get< TextBox>("txtCity");

Dmitry Hes, Tsimafei Avilin, EPAM Sys.

5

To set other search criteria, such as text, you can use the SearchCriteria class.

They can be combined with the methods And…(). So to find the button by the text

"OK":

Button button = window.Get<Button>(SearchCriteria.ByText("OK"));

To find the controls in the application, as well as getting its properties are useful

Inspectors like as Inspect (download - Inspect.exe) supplied with MS win8 SDK,

or ACorns.Hawkeye (https://hawkeye.codeplex.com/). Moreover, Hawkeye shows

the actual type of the object, as well as an object implementation library.

Some tricks to automate unsupported controls.

It was noted that White is a wrapper of UI Automation, and thus it implements the

same basic controls. However, the application under test uses set of custom

controls implemented by e.g. DevExpress library. Some troubles in processing

such controls and their solutions were found out.

UI Automation properties

The first trouble is that White does not process custom DateEdit field. The

framework does not treat it neither like any DateTimePicker, nor Edit. As the

DataGrid contains MaskBox field, to get the control values you can get the value

of control through UI Automation properties:

Dmitry Hes, Tsimafei Avilin, EPAM Sys.

6

//Get value from whiteElement.AutomationElement

private string GetDate( AutomationElement element)

{

AutomationElement dateEdit = element.FindFirst(

TreeScope.Children, new PropertyCondition(

AutomationElement.ControlTypeProperty, ControlType.Edit));

object valuePattern = null;

dateEdit.TryGetCurrentPattern( ValuePattern.Pattern, out valuePattern);

return ((ValuePattern)valuePattern).Current.Value;

}

To set values use typing emulation:

//Type text @dateString into @element

private void SetDataToField( AutomationElement element, string dateString)

{

AutomationElement dateEdit

= element.FindFirst(TreeScope.Children, new

PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Edit));

dateEdit.SetFocus();

System.Windows.Forms.SendKeys.SendWait(dateString);

}

Dmitry Hes, Tsimafei Avilin, EPAM Sys.

7

Using patterns, on SelectionPatternIdentifiers example

The next problem is the processing RadioGroup - getting the value of the selected

item. Applying White isSelected property for element RadioButton, always return

False value.

So UI Automation is useful. The implementation of custom function helps us. This

function refers to the parental control (RadioGroup) and takes the value of the

selected item, using the pattern SelectionPatternIdentifiers.

public bool isSelected()

{

string selectedElement =

GetSelectedElementName( _element.AutomationElement);

return selectedElement.ToLower().Equals(_element.Name.ToLower());

}

//Gets selected element name

private string GetSelectedElementName(AutomationElement element)

{

AutomationElement parentElement = Utils.Util.GetParentElement(element);

var selection = parentElement.GetCurrentPropertyValue(

SelectionPatternIdentifiers.SelectionProperty);

AutomationElement[] selectedElements = selection as AutomationElement[];

if (selectedElements.Length == 0) { return ""; }

return selectedElements[0].Current.Name;

}

Dmitry Hes, Tsimafei Avilin, EPAM Sys.

8

DevExpress.Xpf.Core.ClearAutomationEventsHelper.IsEnabled property

ComboBox controls refer to DevExpress XtraEditors.LookUpEdit class library and

represent as an Edit field combined with a Button, clicking on which opens a drop-

down list. However, the list is a UI Automation Window element, is a direct

descendant (child) of the main form.

In addition, when working with a library DevExpress a problem with dynamically

created controls or dynamically fills tables was spotted. UI Automation does not

see and does not find such controls and child elements of the table even searching

Dmitry Hes, Tsimafei Avilin, EPAM Sys.

9

methods in TreeScope. The solution to this problem is setting to disable static

cleaning event queue property in the application under test:

DevExpress.Xpf.Core.ClearAutomationEventsHelper.IsEnabled = false

However, this may reduce the performance of the application (more info here).

When simple way ends…

White is not so quite good for processing Tables. Especially when tables using a

complex structure as a set of data in the collapsing combined group, nested tables,

any controls instead of text in cells.

After reading variety of websites and forums, solutions for the processing of

custom controls were found out.

Dmitry Hes, Tsimafei Avilin, EPAM Sys.

10

Use LegacyIAccessible

To get cell values use UI Automation. To identify the type of line (collapsing

group element / data line) read the value LegacyIAccessible.State. To do this, you

should use IUIAutomationLegacyIAccessiblePattern, which provides the interface

ILegacyIAccessibleProvider. However, LegacyIAccessible pattern is not available

for control based on UI Automation, because it is not implemented neither UI

Automation nor White

if ((

bool) child.GetCurrentPropertyValue(AutomationElementIdentifiers.IsLegacyIAccessiblePatternAvailableProperty))

{ var pattern = ((LegacyIAccessiblePattern) child.GetCurrentPattern(LegacyIAccessiblePattern.Pattern));

var state = pattern.GetIAccessible().accState;

// Do something with state...

Using COM-wrapper

Using COM-wrapper library UIAComWrapper.dll distributed with Windows Kits

is one of the way to solution. It have implementation for Inspect.exe of

ILegacyIAccessibleProvider for getting LegacyIAccessible properties.

Use Win32 API

In addition, you can use Win32 API to send window messages and thus get

control properties or invoke events.

Here is an example to get DevExpress panel Caption.

Dmitry Hes, Tsimafei Avilin, EPAM Sys.

11

using System.Runtime.InteropServices;

using System.Text;

using System.Threading.Tasks;

using System.Windows.Automation;

using System.Windows.Automation.Text;

[System.Runtime.InteropServices.DllImport("user32.dll", EntryPoint =

"SendMessage", CharSet = System.Runtime.InteropServices.CharSet.Auto)]

private static extern bool SendMessage(IntPtr hWnd, uint Msg, int wParam, StringBuilder

lParam);

[System.Runtime.InteropServices.DllImport("user32.dll", SetLastError = true)]

private static extern IntPtr SendMessage(int hWnd, int Msg, int wparam, int lparam);

//Get Text property of UI element using Win32 API

public static string GetText(AutomationElement element)

{

const int WM_GETTEXT = 0x000D;

const int WM_GETTEXTLENGTH = 0x000E;

//Get handle property of the control

int wndHandle =

(int)element.GetCurrentPropertyValue(AutomationElement.NativeWindowHandleProperty);

StringBuilder title = new StringBuilder();

// Get the size of the string required to hold the window title.

Int32 size = SendMessage(wndHandle, WM_GETTEXTLENGTH, 0, 0).ToInt32();

// If the return is 0, there is no title.

if (size > 0)

{

title = new StringBuilder(size + 1);

SendMessage( new IntPtr(wndHandle), ( int)WM_GETTEXT, title.Capacity, title);

}

return title.ToString();

}

Dmitry Hes, Tsimafei Avilin, EPAM Sys.

12

Call Invoke()

Furthermore you can call Invoke() method to get control properties. This method

implements InvokePattern, which provides support operation and invoking

unambiguous action of controls that do not maintain state by activation.

Use BoundingRectangle

Worst case to use BoundingRectangle property to get control layout. Then use

mouse clicks via SendInput expanding/collapsing/ the table.

Data caching

For processing controls which consist volume data (lists, tables, trees), as well as

getting properties and control patterns in one operation it is necessary to use data

caching. It allows improving performance and, in some cases, avoiding prolonged

searching or exceeding time-out interval through the frequent accessing to

elements. It may lead to test failures.

In UI Automation, caching means pre-fetching of data. Caching occurs when the

application activates a CacheRequest object and then uses any method or property

that returns an AutomationElement, for example, FindFirst, FindAll. The methods

of the TreeWalker class are an exception; caching is only done if a CacheRequest

is specified as a parameter (for example,

TreeWalker.GetFirstChild(AutomationElement, CacheRequest)). Caching data

helps to avoid direct access to date. Any changes made to the CacheRequest after

you subscribe to the event have no effect. More information see here.