web services testing tools dinakar

Symphony Services

Last changed: 12/14/2007

Document Revision Number: Draft Web-Services’ Testing Tools

Symphony-Lawson GOC Page 1 of 10

Web-Services’ Testing Tools

By

Dinakar Adari

Symphony Services




Table of Contents

TABLE OF CONTENTS ............................................................................................................................................................ 2

1. ABSTRACT ........................................................................................................................................................................ 3

2. AUDIENCE ......................................................................................................................................................................... 3

3. CURRENT CHALLENGES ............................................................................................................................................... 3

4. PREMISE ............................................................................................................................................................................ 3

5. AVAILABLE TOOLS......................................................................................................................................................... 4

5.1. ALTOVA MISSIONKIT .................................................................................................................. 4

5.2. PARASOFT SOATEST .................................................................................................................. 4

5.3. QUASAR .................................................................................................................................... 5

5.4. XAB TOOLKIT ............................................................................................................................ 5

5.5 ANTEATER ................................................................................................................................. 5

5.6 HTTPCLIENT, JUNIT, XMLUNIT ................................................................................................... 6

5.7 TESTING FROM BROWSER ............................................................................................................ 6

5.7.1 TRADITIONAL APPROACH ............................................................................................................ 6

5.7.2 MODERN APPROACH ................................................................................................................... 7

5.8 SOAPUI ..................................................................................................................................... 7

6. PROPOSED SOLUTION .................................................................................................................................................... 8

7. RESULTS / CONCLUSION ............................................................................................................................................... 9

8. REFERENCES .................................................................................................................................................................. 10

Symphony Services




1. Abstract In this white paper, an attempt is made to discuss various web-services’ testing tools available and suggest the best possible solution.

2. Audience Developers/Testers working on web-services’ projects

3. Current Challenges Testing of web services is time consuming if test cases are executed manually, then comparing the response with the benchmark responses to ensure the functional correction of the web-services. The task would worsen exponentially as the number of web services pile up. So, a tool which would build the service requests, send the requests across to the host server where the web-service is deployed and give us the response to the request would be helpful.

4. Premise In the context of SOA(Service Oriented Architecture), following are some of the activities that happen during lifetime of a service: a) Service Analysis b) Service Development c) Service Testing d) Service Provisioning e) Service Operation f) Service Consumption g) Service Change Management h) Service Decommission After a business process is identified for web service development, development of web service is done using a web services stack, IDE (Stylus Studio, XMLSpy, Netbeans, etc.) and with a choice of programming language (Java, .NET, C++, etc.). In the testing phase, unit, smoke, functional, load testing is done to ensure that all the scenarios envisioned in the analysis phase are met. Wide variety of software tools is available for testing but few provide with the complete set for web-service testing. Testing of web services is time consuming if test cases are executed manually, then comparing the response with the benchmark responses to ensure the functional correction of the web-services. The task would further worsen as the number of web services pile up. So, a tool which would build the service requests, send the requests across to the host server where the web-service is deployed and give us the response to the request would be helpful.

Symphony Services




5. Available tools Following are some of the testing tools available for web-service testing:

a) Commercial ware:

5.1 Quasar Soap Test Client(Shareware) 5.2 Altova MissionKit 5.3 Parasoft SOAtest

b) Free or Open-Source Software: 5.4 XAB toolkit 5.5 HttpClient, JUnit, XMLUnit 5.6 Anteater 5.7 Testing from browser 5.8 soapUI

With each of the above tools, the underlying communications with the web service hosting server can be captured from one of the many monitoring tools like TCPmon, ethereal/wireshark, MS Network Monitor, snoop, Kismet, Netstumbler, etc. Let’s take a deeper look into each of the tools.

5.1. Altova Missionkit

The Altova MissionKit includes complete Web services development capabilities. This is a combination of two tools – XMLSpy and MapForce. Following are some of its features:

1) Graphical XML Schema editor for embedding schema in WSDL files 2) Visual, drag & drop WSDL design 3) Automatic SOAP message generation from WSDL 4) Universal SOAP client & SOAP debugger 5) Implementing Web services visually 6) Support for XML, database, flat file, EDI, & Web services data sources 7) Mapping of data sources to WSDL operations 8) Consuming existing Web services in data integration mapping projects

5.2. Parasoft SOATest

SOAtest is a comprehensive, collaborative test and analysis tool suite designed specifically for test and validation of Service Oriented Architectures. It streamlines the process of rapidly constructing robust regression suites. Some of the features of this tool include:

1) WSDL schema and semantic verification and compliance to WS-I 2) SOAP, PoX (Plain XML) REST, JSON, and BPEL support 3) Asynchronous testing 4) Various WS-* standards support 5) MTOM(XOP)/MIME/DIME Attachment support 6) UDDI support: query verification, validation, and load testing 7) Data-driven testing through data sources 8) Fully reusable functional tests for load testing

Symphony Services




9) Apply expected QoS (Quality of Service) metrics to load tests 10) Automate load test execution and track performance metrics throughout the SDLC 11) Security penetration testing

5.3. Quasar

Quasar is a SOA testing tool, allowing for easy monitoring and generation of SOA events. Quasar is a light weight tool for unit testing a variety of SOA components. The tool comes with an embedded XML editor. Some of the key features of it include:

1) Make SOAP requests (also available through command line interface) 2) Reply to SOAP requests 3) Unit test your SOAP services 4) Regression test your SOAP services 5) Stress / Performance test your JMS components

*Disadvantage with the commercial ware is i) Any changes whatever required in future to accommodate new features cannot be done ii) Bug fixing, maintenance costs extra iii) Requires client approval before implementing within the project

5.4. XAB toolkit

The XAB program is an application to build 3 tier applications with Mozilla, JavaScript, SOAP, PHP and MySQL. It can be used to build database applications with a rich user interface around Ajax in Mozilla. It will automate the building of most or all of an application producing PHP and JavaScript code which can be installed on a server. It uses Mozilla and DHTML to render its user interface, SOAP as a data transport method, php for application logic and MySQL as data storage. It will build a set of PHP classes, JavaScript wrappers, DHTML forms, as well as the SOAP requests. Disadvantage: The product is not maintained anymore and even with the existing one, in order to use it you will have to edit the files to fill in default test values and assertions to test. You will also need to create scripts to call the tests you want to run.

5.5 Anteater

Anteater is a testing framework designed around Ant. It provides an easy way to write tests for checking the functionality of a Web application or of an XML Web service. Following is a typical scenario which can be executed using Anteater

• Send a HTTP/HTTPS request to a Web server. When the response comes back, test that it meets certain criteria. You can check for HTTP headers and response codes, and validate the response body with regular expression, XPath, Relax NG, or contentEquals tests, plus some binary formats. New tests can be easily added.

The ability to wait for incoming HTTP messages is something unique to Anteater, which makes it especially useful when building tests for applications that use high level SOAP-based communication, like ebXML or BizTalk. Applications written using these protocols usually receive SOAP messages, and

Symphony Services




send back a meaningless response. It is only later when they inform the client, using an HTTP request on the client, about the results of the processing. These are the so-called asynchronous SOAP messages, and are the heart of many high-level protocols based on SOAP or XML messages.

5.6 HttpClient, JUnit, XMLUnit

This is a combination of tools which needs a bit of homework to be done. The process involves the following steps -

1) Create a simple Web service 2) Use HttpClient to invoke a Web service 3) Compare the expected response and actual response using XMLUnit

In this process, JUnit can be used to invoke HttpClient and then use XMLUnit to compare the expected and actual responses. Disadvantage: Anteater, (HttpClient, JUnit, XMLUnit) combination needs much of the add-on features, customization needed for a full-fledged web-service testing tool.

5.7 Testing from browser

5.7.1 Traditional approach

With Mozilla 1.0/Netscape 7.0x /Firefox, it is possible for the browser to directly communicate with web services using its low-level SOAP implementation through JavaScript. The underlying Gecko layout engine (second most popular layout engine used by Firefox, Mozilla, Netscape, Opera, flock, etc) uses JavaScript interface for establishing SOAP calls. This is a low-level API which requires building the SOAP envelope using several SOAP-specific JavaScript objects. However, using JavaScript to talk to web services is subject to the same security policies as other scripts in terms of cross domain access. Therefore, accessing web services on a server other than the one where the JavaScript lives will violate the cross-domain policy. However, this can be circumvented for testing purpose in the following manner:

1) Run the script from your local drive. Save the code locally to your hard drive. The cross-domain security model does not affect code that is run from the local hard drive.

2) Enable Cross Domain Access Cross-domain check can be bypassed by setting a preference in about:config page in Firefox or in prefs.js file located in the local profile folder in general and in adding a JavaScript command to request overriding of the cross-domain check(using UniversalBrowserRead privilege variable). Testing the web-service from the browser using the low-level API involves following steps:

1) Create a SOAPCall object 2) Encode the function call with list of headers and regular parameters, 3) Invoke the call

a) If the call is successful, the response (SOAPResponse) contains the results of the service call, output parameters and/or header blocks.

Symphony Services




b) If unsuccessful, fault element of the response contains the error. If the call is invoked asynchronously, then a function is supplied by the caller which receives the response when it arrives from the remote service.

5.7.2 Modern approach

Using SOAP has been somewhat tedious, requiring manual construction and delivery of the SOAP envelope the web service expects. SOAP-based response also has need to be parsed manually for the information required. As of Netscape 7.1/Mozilla 1.4, Gecko supports WSDL 1.1 proxying. Using the WSDL file, "script" the web services as if it were a native object, hiding the SOAP and XML aspect. For example, after creating a proxy instance of a web service using WSDL, call methods on the proxy object like any JavaScript object Ex:- proxyWS.getTranslation("en_hi", "Hello World from a Web Service") The testing process involves the following steps:

a) Create WSDL proxy b) Use WSDL and fill in the parameters required for the web-service call c) Make call to the web service and handle the response Disadvantage: ‘Testing from browser’ can be used only when testing is required on a small scale and scalability is another big problem with this approach.

5.8 soapUI

soapUI is a free and open source desktop application for inspecting, invoking, developing, simulating/mocking and functional/load/compliance testing of web services over HTTP. Functional and Load-Testing can be done interactively or within an automated build/integration process using the soapUI command-line tools. Key features:

1) Besides just sending SOAP requests, groovy scripts can also be executed with which state of the database can be prepared, so that a full and clean test cycle is created.

2) soapUI supports functional testing of Web Services by providing a TestCase metaphor where a number of TestSteps can be executed in sequence. There are currently six types of TestSteps available. TestCases are further organized into TestSuites of which an arbitrary number can be created within each project.

3) Opt for default parameters which are set as parameters for the service. During importing the WSDL, tick the checkbox asking to create a SOAP request for each operation, if you want one. If you double click the request you can see the content of the SOAP request in the main window

Results of the call can be seen in the adjoining window. To make a good test tool we need to create a new test suite in our project and add test cases to it. With these test cases you can send your custom SOAP requests and check the response with assertions. This can be done by right-clicking the web-service created and choosing 'Generate Test Suite' option. This generates a test suite with one test case for every operation. Expand the Test Steps node and double click at the operation to see the simple SOAP request in the main window. Assertions can be added at web-service or operation level.

Symphony Services




This helps in not only checking if the service is reached but also checks if the content of the response is what you would expect to be returned by the service. Disadvantage: This product suits most of the needs but a bit of customization in terms of giving inputs to the tool would make this a perfect one.

6. Proposed Solution

soapUI++ Considering the approaches discussed above, soapUI++ fits the bill perfectly. soapUI++ is an extension to soapUI making it specific for Lawson CAPI project. As soapUI itself is developed in java and it’s API publicly available, extra features can be added to the existing tool (which in itself facilitates for manual testing of web-services). One of the best things about soapUI is that it provides tools for running test cases. These tools can be used in association with the APIs provided with the application to create application specific tools. ‘SoapUITestCaseRunner’ is one such tool which runs specified TestCases and reports/exports results as configured. soapUI++ runs as a batch of operations. It requires 2 kinds of inputs - 1) A batch file, containing the necessary web-services/individual operation of web-service. The

operations specified in this file are executed in sequence. 2) Input values for each operation is specified in another file which is used in populating the SOAP

request(used in step 2 specified below) Following steps are automated using soapUI++: 1) Creation of SOAP request 2) Populating the SOAP request 3) Sending the request to the server on which web-services are deployed 4) Receiving the response 5) Comparing the response with that of the benchmarked response 6) Logging the result of comparison, other important events during this process for analysis purpose.

soapUI++ gives the flexibility to run requests at operation/web-service level and in test/benchmark mode. In benchmark mode, the output is stored, against which the test cases are compared.

Symphony Services




7. Results / Conclusion Benefits:

1) Minimal manual intervention 2) Speedy execution of test cases 3) Saved time 4) Scalable for any number of web-service/operations 5) Reduced effort for future execution

Technical Advantage: This framework is currently used for ‘Functional Testing’ of web services over HTTP. It’s best used during configuration/environment changes of the server and that when the same set of web-services are to be checked for new configurations/environments. Ex: Deploying of web-services developed in older versions in new versions requires the testing of all developed web-services against the new environment. Furthermore, the framework’s functionality can be extended for

1) Load, Compliance testing 2) Inspecting, invoking, simulating/mock testing of web services.

Cost Savings:

1) Man days: Though, the exact savings in man days is not captured, ratio of automation to manual testing is 30:1 per web-service operation.

2) Cost of software: Using free and open source software reduces a. Cost of procurement of the product b. As the source code of the final product is available, maintenance costs if any can be

internally borne.

Symphony Services




8. References 1) SOA Testing: https://www6.software.ibm.com/developerworks/education/ws-soa-autotest2/section2.html 2) Web Services in Mozilla: http://developer.mozilla.org/en/docs/Accessing_Web_Services_in_Mozilla_Using_WSDL_Proxying 3) SOAP scripts in Mozilla: http://lxr.mozilla.org/mozilla/source/extensions/webservices/docs/Soap_Scripts_in_Mozilla.html 4) soapUI vs. JUnit: http://www.myarch.com/soapui-vs-junit 5) soapUI: http://www.soapui.org/ 6) Wireshark: http://wiki.wireshark.org/ 7) Gecko engine: http://en.wikipedia.org/wiki/Gecko_%28layout_engine%29 8) Security Restrictions on cross-domain access: http://developer.mozilla.org/en/docs/Bypassing_Security_Restrictions_and_Signing_Code 9) Calling SOAP Servers from JS in Mozilla: http://www.onlamp.com/lpt/a/5981 10) Signed scripts in Mozilla: http://www.mozilla.org/projects/security/components/signed-scripts.html 11) SOAP in Gecko browsers: http://developer.mozilla.org/en/docs/SOAP_in_Gecko-based_Browsers 12) Using Mozilla SOAP API: http://www.oreillynet.com/lpt/a/2677 13) Quasar: http://www.integrationcentral.com/product.html?gclid=CLWXmZjX_Y8CFUaPOAodBw0_MA

web services testing tools dinakar

Documents