testing web applications willem visser rw334. overview testing theory testing web apps tools

Testing Web Applications

Willem VisserRW334

Overview

• Testing Theory• Testing Web Apps• Tools

The Basics• A test consists of

– Input– Expected output (also called the Oracle)

• White Box Testing– Considering the code when coming up with tests

• Black Box Testing– Only consider the input-output interface

• Automated Testing is the Holy Grail

Types of Testing• Unit Testing• Integration Testing • Systems Testing• Regression Testing• Acceptance Testing

– Alpha/Beta Testing– Usability Testing

• Performance Testing• Robustness Testing• Security Testing

Lower Level Code Testing• Unit Testing

– Test individual components of the code– Can be as little as one function– White box

• Integration Testing– Interface between components– White Box– Can be as much as the whole system (which case it

becomes more black box)

• Code Coverage as Test Adequacy Measure

Functional Testing

• Testing that the systems meets the requirements

• Systems Testing– In the deployed environment

• Unlike Integration testing

– Always black box

Cross Cutting

• Regression Testing– When code changes (some) old tests must still

pass– Could be at unit, integration or systems level

• Very high cost involved in regression testing– Regression failure has high cost to fix

Customer Facing• Acceptance Testing

– Is the customer happy with the product?

• Alpha/Beta Testing– Let the system out to a few of your customers and

see how they feel about it

• Usability Testing– Typically for GUI/Web/Mobile to not just check

that the system is correct but also easy to use– Harder to create an Oracle

Non-Functional Testing

• Performance– Load

• See how the system behaves at peak load

– Stress• Push the system beyond its limits to see how far it will

survive

• Security Testing– Check whether there are vulnerabilities that might

lead to loss of privacy or other security issues

Web App Testing

• Full Gambit of Features• Front End GUI

– Usability issues– Hard to test– Browser Compatibility issues

• Server Side (including storage)– Performance issues

• Security issues

Unit Testing• Utterly important!• Finding bugs early saves money• Makes regression testing much more effective

– Write once, run often

• What do you test?– Business logic!– No need to test simple code or interactions with 3rd party

libraries• Until they fail of course!

• Use Coverage tools to help you decide if you have tested enough

Stubs or Mocks• In unit testing you are interested in local

behavior and assume other things you might be using behave correctly

• Most unit testing frameworks provides stubs for these 3rd party components

• Good example is datastore and memcache stubs provided by GAE

• Only problem is that sometimes these stubs don’t respect the behavior of the real thing!

Unit Testing Frameworks• Java – JUnit

– The most famous of them all

• PHP – PHPUnit• Python – PyUnit

– Actually just “import unittest”

• And many more XUnit tools for language X– For example GAEUnit, but it looks “dead” now

• Use coverage tools along side, examples…– Coverage.py for Python– (Ecl)Emma for (Eclipse) Java

Unit Testing and GUIs

• When doing Unit Testing you try and stay well clear of the GUI– Clicking and entering text through a GUI is not

automated, although we will see later that with Replay Technology it can also be done

• Isolating the GUI to allow more efficient testing is the reason to use Model-View-Presenter rather than Model-View-Controller design pattern

Webapp2 example

import unittestimport webapp2

# from the app main.pyimport main

class TestHandlers(unittest.TestCase): def test_hello(self): #input request = webapp2.Request.blank('/’) #output response = request.get_response(main.app) #oracle self.assertEqual(response.status_int, 200) self.assertEqual(response.body, 'Hello, world!’)

import webapp2

class HelloHandler(webapp2.RequestHandler): def get(self): self.response.write('Hello, world!')

app = webapp2.WSGIapplication([('/', HelloHandler)])

Code to Test

Test

Integration Testing• Thin line between unit and integration• Strictly speaking when more than one

component is used you are doing integration testing– For example if your web app uses a datastore then

the test on the previous slide that came in via a GET request could be an integration test not a unit test

• This is not worth worrying about– Unit/Integration Testing is fine

System Testing

• Now the full round-trip is being tested, including the Browser component

• Unit/Integration Testing can be done on a local environment, but System Testing need to be in the deployed environment

• State-of-the-Practice– Record and replay tests

Seleniumhttp://docs.seleniumhq.org/

• The #1 tool in Web System Testing• “automates browsers. That’s it!”

– Much like the webapp2 example earlier, just many more options, including browser specific drivers

• IDE for record and replay– With scripts that can be edited

• WebDriver that allows one to run without a browser at all

Selenium Examplefrom selenium import webdriverfrom selenium.common.exceptions import TimeoutExceptionfrom selenium.webdriver.support.ui import WebDriverWait # available since 2.4.0from selenium.webdriver.support import expected_conditions as EC # available since 2.26.0

# Create a new instance of the Firefox driverdriver = webdriver.Firefox()

# go to the google home pagedriver.get("http://www.google.com")

# find the element that's name attribute is q (the google search box)inputElement = driver.find_element_by_name("q")

# type in the searchinputElement.send_keys("cheese!")

inputElement.submit()

try: # we have to wait for the page to refresh, the last thing that seems to be updated is the title WebDriverWait(driver, 10).until(EC.title_contains("cheese!"))

# You should see "cheese! - Google Search" print driver.title

finally: driver.quit()

Java Exampleimport com.thoughtworks.selenium.*;// This is the driver's import. You'll use this for instantiating a// browser and making it do what you need.

public class NewTest extends SeleneseTestCase {

public void setUp() throws Exception { setUp("http://www.google.com/", "*firefox"); }

public void testNew() throws Exception { selenium.open("/"); selenium.type("q", "selenium rc"); selenium.click("btnG"); selenium.waitForPageToLoad("30000"); assertTrue(selenium.isTextPresent("Results * for selenium rc")); }}

Usability Testing

• Based on user opinions• Manual according to a list of tasks• Observer records behavior• Could use things like eye-tracking for more

precise results• Automated Usability Testing is still a research

topic

Robustness Testing

• Try anything and see if something breaks• Most famous of these is Android Monkey

– Sends random keystrokes to your Android app

• gremlins.js does the same thing for web apps

<script src="path/to/gremlins.min.js"></script><script>gremlins.createHorde().unleash();</script>

Performance Testing• System performance under specific loads

– Concurrent users doing a certain number of transactions for a certain duration

• This is a very hard kind of testing to do– Needs lots of infrastructure – Often the performance bottleneck is your testing framework

and not the system under test

• In the end we know where the problem is– THE DATABASE!– More generally where something has to wait for something

else to finish

Web Performance Testing

• Measure throughput or transaction rate• Server response time• Rendering, but that might need additional

scripts on the client side• You might not know what is the expected

performance, so often you profile the performance– You will quickly notice bad performance

Web front-end Performance Tools

• Google Pagespeed tools• www.webpagetest.org• Google Chrome Developer Tools• Nice summary of tools related to performance

can be found at http://samsaffron.com/archive/2012/03/23/sam-s-ultimate-web-performance-tools-and-resources

JMeter

• Load and Performance Testing• Server side• Not a browser, but can simulate some actions

typically done by a browser– HTTP, etc.– No javascript execution

Security Testing• This is HUGE!

– A whole course or even degree can be devoted to it

• Nice checklist of things to do at https://www.owasp.org/index.php/Web_Application_Security_Testing_Cheat_Sheet

• Large part of security testing is manual• Penetration Testing

– Find security vulnerabilities that can be exploited

• Tools to find buffer overflows is one of the most effective automatic approaches

• Fuzz Testing– Semi-structured random testing

Zed Attack Proxy (ZAP)

• Automated Penetration Testing– Point it to URL and it does the rest

• Includes Fuzzing and many more • Find it here

https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project