hitchhiker's guide to functional testing

Hitchhiker's guide to Functional Testing

Ali Derbane Wiebe Elsinga

Senior Android Developer

Speaker

Ali Derbane Wiebe Elsinga

Android Technical Lead

GDG Dutch Android User Group

Speaker

iWhatWhat is functional testing

WhyWhy do we want to use it

i

Functional testing

What is Functional testing

+

Functional testing

Functional testing is a quality assurance (QA) process and a type of black box testing that bases its test cases on the specifications of the software component under test.



i

Functional testing

Check functionality

Why do we want to use it

+

Check functionality

Check quality

Why do we want to use it

+



i

Functional testing

Tested Frameworks+

Frameworks

iRobotium

iSelendroid

iRobolectric

iCalabash

iEspresso

Show exampleShow example

Example

Tested Frameworks+

Frameworks

iRobotium

iSelendroid

iRobolectric

iCalabash

iEspresso

Robotium

Sort of WYSWYGS

Java BasedJ

Robotium

assertTrue(mSolo.waitForText("42"));

new Solo(getInstrumentation(), getActivity());

Setup

Checking result

mSolo.assertCurrentActivity("Started", MainActivity.class); mSolo.clickOnText("7"); mSolo.clickOnText("×"); mSolo.clickOnText("6"); mSolo.clickOnText("=");

The Test

Robotium

Run./gradlew connectedAndroidTest

Result

Robotium

APIs are easy to use

Not the fastest

Extensive documentation and examples

Tested Frameworks+

Frameworks

iRobotium

iSelendroid

iRobolectric

iCalabash

iEspresso

Selendroid

Sort of WYSWYGS

Java BasedJ

Selendroid

@BeforeClass public static void startSelendroidServer() throws Exception { SelendroidConfiguration config = new SelendroidConfiguration(); config.addSupportedApp("apk/resigned-SampleApp.apk");

mSelendroidServer = new SelendroidLauncher(config); mSelendroidServer.launchSelendroid();

SelendroidCapabilities caps = new SelendroidCapabilities(“droidcon.nl.calculator:");

mSDriver = new SelendroidDriver(caps); }

Setup

Selendroid

@AfterClass public static void stopSelendroidServer() {

if(mSDriver != null) { mSDriver.quit();

}

if(mSelendroidServer != null) { mSelendroidServer.stopSelendroid(); }

}

Tear down

Selendroid

WebElement resText = mSDriver.findElement(By.id(“resText"));

Assert.assertEquals(resText.getText(), "42");

mSDriver.findElement(By.partialLinkText("CL")).click(); mSDriver.findElement(By.id("num_7")).click(); mSDriver.findElement(By.xpath(“//Button[@id=‘op_mul’]")) .click(); mSDriver.findElement(By.linkText("6")).click(); mSDriver.findElement(By.linkText("=")).click();

The Test

Checking result

Selendroid

RunJUnit

Result

Selendroid

Works well with JUnit

Only MDPI devices/emulators

Debuggable

WYSIWYG Editor

Tested Frameworks+

Frameworks

iRobotium

iSelendroid

iRobolectric

iCalabash

iEspresso

Robolectric

Own visual UI simulatorO

Java BasedJ

Robolectric

@RunWith(RobolectricTestRunner.class) @Config(emulateSdk = 18) public class MainActivityRobolectricTest {

@Before public void setUp() throws Exception { mTestAct = Robolectric.buildActivity(MainActivity.class) .create().get();

}

Setup

Robolectric

EditText resultText = (EditText) mTestAct.findViewById(R.id.resText); assertEquals("42", resultText.getText().toString());

Checking result

mTestAct.findViewById(R.id.num_7).performClick(); mTestAct.findViewById(R.id.op_mul).performClick(); mTestAct.findViewById(R.id.num_6).performClick(); mTestAct.findViewById(R.id.op_equ).performClick();

The Test

Robolectric

./gradlew clean test

Run

Result

Robolectric

Does not require emulator or device

Extends JUnit framework

Does not run on actual device

Great performance

Tested Frameworks+

Frameworks

iRobotium

iSelendroid

iRobolectric

iCalabash

iEspresso

Calabash

Gherkin (Natural language)G

WYSWYGW

Calabash

Setup

Feature: Hitchhiker's Guide Scenario: Answer to the Ultimate Question of Life

the Universe and Everything

Calabash

Feature: Hitchhiker's Guide Scenario: Answer to the Ultimate Question of Life

the Universe and Everything

Given My app is running When I press the "7" button And I press the "×" button And I press the "6" button And I press the "=" button Then I should see "42"

The Test & Checking result

Calabash

Runcalabash-android run apk/resigned-SampleApp.apk features/ --format html --out result.html

Result

Calabash

Features can be written/read by anyone

Custom steps need Ruby knowledge

Available for iOS, Android and hybrids

Occasionally returns false negatives

Tested Frameworks+

Frameworks

iRobotium

iSelendroid

iRobolectric

iCalabash

iEspresso

Espresso

Google approvedG

Java BasedJ

Espresso

onView(withId(R.id.resText)).check(matches(withText("42")));

None

Setup

Checking result

onView(withText(“7")).perform(click()); onView(withText("×")).perform(click()); onView(withText("6")).perform(click()); onView(withText("=")).perform(click());

The Test

Espresso

Run./gradlew connectedAndroidTest

Result

Espresso

Easy to use

Google support

Great performance

Tested Frameworks+

Frameworks

iRobotium

iSelendroid

iRobolectric

iCalabash

iEspresso

New framework

There seems to be a new framework available. Do you want to know more?

NO YES

Android Testing Support Library


EspressoW

AndroidJUnitRunnerJ

IntentoE


Intentointended(allOf( hasAction(Intent.ACTION_CALL),

toPackage("com.android.phone"), hasData(allOf(

hasSchemaSpecificPart("tel", “123-345-6789")))), times(1));

Tested Frameworks+

Frameworks

iRobotium

iSelendroid

iRobolectric

iCalabash

iEspresso

Convince your bosses that testing can save

money and start testing!

Fan of Calabash and curious about Selendroid

Calabash is portable to non Android, but is

a bit flaky

Big fan of Espresso

DevFest Istanbul+

Want to learn more

Q & A

[email protected] Elsinga

+WiebeElsingaG

welsingaT

[email protected] Derbane

+AliDerbaneG

hitchhiker's guide to functional testing

Technology