techtalk: wind tunnel, personas, and testing real ux

Wind Tunnel, Personas and Testing Real UX

Partners Tech Talk Webinar

David Broerman

Partner Enablement Manager

Email: [email protected]

2

Agenda

• Intro to Perfecto • Who is Perfecto• Basic Architecture/Setup

• Digital Challenges• Overview of the Perfecto CQ Lab and Wind Tunnel• Wind Tunnel and Personas• What is a Persona? Existing and Custom Personas

• Adding Wind Tunnel capabilities to your test• Easily add a Persona right into your script

• Reporting and Digital Zoom• Sample Test and Demo that includes Wind Tunnel• Eclipse/Java/Selenium or Appium

© 2016, Perfecto Mobile Ltd. All Rights Reserved.

Intro to Perfecto

Perfecto Offers:

A Cloud Based Continuous Quality Lab

enabling testing and monitoring of all digital platforms

on real devices and real end user conditions,

for enterprises adopting agile delivery processes.

3

Intro to Perfecto

Dominant leader & visionary

Strong Ecosystem

Perfecto @ a glance

Working with Enterprises and Partners to deliver exceptional digital experiences.

“Perfecto Mobile marks shift in mobile apps QA with Continuous Quality Lab”

4

Setting the bar for market success

• Founded in late 2006, Business HQ in Boston • Over 300 employees globally• Fortune 500 customer base

Cloud Based Quality Platform For The Digital Era

05/02/2023 5© 2017, Perfecto Mobile Ltd. All Rights Reserved.

Open & Integrated into Dev/Test/Ops

Any test tool

Any CI server

Any test framework

Any IDE

Any Location

Code/Debug Functional Test

Performance Test

Production Insight

One Lab for all Digital Platforms

Reporting & Analytics

Wind Tunnel Layer

Customer Support

Browsers Desktops Mobile Devices IoT

Architected for Digital Quality at High Velocity


Maximize Digital TestCoverage

Maximize % of

Automation

1. Cloud-first architecture total reliability, unlimited scale, secure, global and available 24/7

2. Platform connectivity layer™ (PCL) architecture

• Making open source enterprise grade• Enables high test automation coverage

3. One Script, One Lab for all digital platforms

4. Embedded into your toolchain to streamline activities

5. WindTunnel™ Layer persona based user environment testing for true digital coverage

6. DigitalZoom™ Reporting provides cross platform visibility and fast drill down

Quality Across Dev-QA-Ops

Fast Feedback Loop

FOUR PILLARS OF SUCCESS

Perfecto PCL™ - Architectural Workflow


Standard Open Source

Platform Under Test

Standard Appium Server/ Selenium Grid

Standard OSS Test Code In Dev IDE’s

Platform Under Test

Standard OSS Test Code In Dev IDE’s

Standard OSS

Standard Appium Server/ Selenium Grid

Perfecto PCL™

BenefitsStandard

Open Source

With Perfecto

PCL™

Test across platforms

Same day support for any platform using any test framework

Enhanced test coverage• Full system level control• Handle interruptions: text msg, system msg • Test audio (e.g. voice commands)• Test image (e.g. check deposit)• Test environment conditions (Wind Tunnel™)

Visual validation of screen layout

Always On lab

8

Datacenter and Device Setup/Configuration

05/02/2023

9

Global Data Centers Network

05/02/2023

10

Mobile Experience Quality Challenge

The Mobile ExperienceQuality Challenge

1) Building common user environment condition profiles into test suites

2) Assembling multiple solutions to execute tests across profiles

3) Seamlessly integrating user condition testing within Continuous integration pipelines

4) Analyzing app performance blind spots

5) How to set up a lab and these various conditions?


Digital Presents Several Challenges for Testing Teams

11

Issue Negative Consequence

1 Separate tools and labs for web and mobile teams.

• Silo-ed test methods result in redundant investments

• Differing tools, differing expertise

2 Timely access to required lab resources in a dynamic market.

• Release schedules slowed down• Quality risks go unmitigated due to inadequate

coverage

3 Lack of easy visibility to user experience across all browsers and devices

• No strategic quality of UX management• Inefficient root cause analysis





2 Timely access to required lab resources in a dynamic market.

• Release schedules slowed down• Quality risks go unmitigated due to inadequate

coverage





Perfecto Continuous Quality LabOne Script, One Lab for all Digital Platforms


13

User Centric Quality Lab with End User Conditions

Using a Mobile App

1 2

Network QualityCellular, Wi-Fi

3

Varied Device Conditions

4

Multiple devicesSpecific profiles

5

Location

6

Resource Conflicts

7

Interacting with sensorsPeak time -

Millions of transactions

8

Simple Functional Testing Is Not Enough

14

Overview of Perfecto Wind Tunnel

Wind Tunnel

Profile Selector

Usage Patterns1. Login2. Search3. Select4. Payment5. Logout

Unchanged Test FlowsExample: User Journey

on Banking/Financial App

Existing Flows

Analyze & Optimize

• Lives in NY• iPhone 6S • Verizon Customer• Constant WhatsApp msg

Georgia

12

3 4

PersonaPoints of Interest

Wind Tunnel Testing

Wind Tunnel Report


15

The Personas

• The Perfecto Wind Tunnel personas define the test environment

• A Persona is defined by a set of capabilities• There are 5 available pre-defined personas• Personas can be created and edited

• Georgia• 47 year old high income, corporate road warrior living in Manhattan

traveling to San Fran & London• Traits

• Network: Constantly connected, frequent network switches across various WiFi and cellular networks

• Brands of choice: Apple & Verizon• Device(s): iPhone 6S, Smartwatch & iPad Pro• App loyalty: Intense social user, WhatsApp, Mobile Banking and PO &

Expense Report approvals

Location

Carrier & Brands

Device coverage Background

apps


5 Pre-Defined Personas

16© 2016, Perfecto Mobile Ltd. All Rights Reserved.

Wind Tunnel – Languages & Frameworks

17© 2016, Perfecto Mobile Ltd. All Rights Reserved.

18

Using & Tailoring Personas

• Set a persona using the windTunnelPersona capability

• Tailor an existing persona by defining new capabilities

capabilities.setCapability("windTunnelPersona", "Georgia");

capabilities.setCapability("windTunnelPersona", "Georgia");//Add location either by an address or lat,long coordinatescapabilities.setCapability("windTunnelLocation", "New York City,NY");//Set orientation to landscape and portraitcapabilities.setCapability("windTunnelOrientation", "portrait");//list the apps running in the backgroundcapabilities.setCapability("windTunnelBackgroundRunningApps", "Candy Crush Saga");


19

Persona JSON file• Example persona JSON file

{ "settings":{ "windTunnelLocationAddress":"New York, NY", "windTunnelBackgroundRunningApps":"United,Twitter,Outlook,Marriott,BofA", "windTunnelVNetwork":"4G LTE Poor", "windTunnelOrientation":"landscape" }, "properties":{ "description":"Corporate Road Warrior, Age 47", "name":"Georgia", "image":"PUBLIC:Personas/Georgia.png"}, "device":{ "model":"iPhone-6S||iPad Mini 4" }}


capabilities.setCapability("windTunnelPersona", "Georgia");

Example of Reference Persona – remove Georgia

20

Points of Interest

•The focus points that are part of your Wind Tunnel execution and reports•The app issues that are being tested in real user

conditionsExamples:• Purchase transaction that takes too long• CPU consumption of the running Facebook app


21

Adding Points of Interest to the test

•Java Example• Following code adds a Point of Interest that will add

an indication of a successful page load.•Will be selectable in the wind tunnel report

...// Wind Tunnel: Add success point of interest to the Wind Tunnel reportWindTunnelUtils.pointOfInterest(driver, "Point Of Interest 1: Page load","Success");


22

How to Create a Wind Tunnel Script

• Determine IDE and language • Download Perfecto Plug In • Create a new Perfecto Lab Project • Add sample code to your project (include persona for wind

tunnel)• Run the test• Review the run results/report


Reporting and Digital Zoom

Reporting - Usage and Flow

Link

Execution

Capability Download PDF: PerfectoLabUtils.downloadReport(driver, "pdf", "C:\\test\\report");

Link to STR - String reportURL = (String)(driver.getCapabilities().getCapability(WindTunnelUtils.SINGLE_TEST_REPORT_URL_CAPABILITY

Reporting Tab

Capability – Get Grid URL & create testName

Reporting Tab

Grid View/Dashboard

Legacy Test Report

Single Test Report

12

3

4

5

25

Single Test Report

The Single Test Report timeline includes:

• Points of interest• Wind Tunnel events• Checkpoints

(validation)


DigitalZoom™ Reports - Visibility, Traceability, Fast Analysis


• One cloud BI Reporting system f From executive level to root cause analysis for engineers• Full Flexibility to view any granularity • One click drill down with full traceability between reporting levels• Out-of-the-box digital relevant filters + Build any custom filters• Collaboration capabilities for remote joint analysis

• Digital View• Fast analysis • Fast feedback

loop

Question/Answer & Wrap-UpThank You for attending!

David BroermanPartner Enablement ManagerEmail: [email protected]

techtalk: wind tunnel, personas, and testing real ux

Technology