Functional TestingFunctional Testing

MontpellierMontpellierOctober 2006October 2006perfSONAR MeetingperfSONAR MeetingMichalis Michael (Cynet)Michalis Michael (Cynet)

Functional TestingFunctional Testing

• Service Oriented TestingService Oriented Testing

• Black Box TestingBlack Box Testing

• Behavioural TestingBehavioural Testing

• Automated TestingAutomated Testing

Functional Testing is Functional Testing is necessary to:necessary to:

• Make sure that the service actually Make sure that the service actually worksworks• This is accomplished by a wide range This is accomplished by a wide range of testsof tests• If the tests are successful the service If the tests are successful the service is functioning inside the specified is functioning inside the specified parametersparameters

Functional Testing processFunctional Testing process

• A wide range of requests is fed to the A wide range of requests is fed to the serviceservice

Requests that are well formatted containing valid Requests that are well formatted containing valid informationinformation

Requests that are malformedRequests that are malformed Requests containing invalid or non sufficient Requests containing invalid or non sufficient

informationinformation

• Responses are collected and checked for Responses are collected and checked for validityvalidity

• Responses information are cross-checked Responses information are cross-checked with information inside the service databasewith information inside the service database

•ProcessProcessCreate RequestsCreate Requests

Feed requests into ServiceFeed requests into Service

Log responsesLog responses

Check for validity/Correlate responsesCheck for validity/Correlate responses

against actual DB dataagainst actual DB data

Test PassedTest Passed Test FailedTest Failed

LS Functional TestsLS Functional Tests

• Focus on checking LS response to :Focus on checking LS response to : Well formatted requests containing valid Well formatted requests containing valid

informationinformation Requests that are not well formatted with or with Requests that are not well formatted with or with

out valid informationout valid information Requests that do not contain enough informationRequests that do not contain enough information• Requests with excess information also maybe fedRequests with excess information also maybe fed• Suggestions:Suggestions: Check LSCleanup (will do if time is enough)Check LSCleanup (will do if time is enough) Check the RRD MA against the LSCheck the RRD MA against the LS

A LSRegisterRequest exampleA LSRegisterRequest example

• The request is missing the Data element . The LS should return an error response

LS Functional Test – Parts neededLS Functional Test – Parts needed

• A request building script-class.A request building script-class.

• A script-class that feeds the requests to the A script-class that feeds the requests to the LS and stores the responses from the LS.LS and stores the responses from the LS.

(Requests are send via the SOAPClient class that is (Requests are send via the SOAPClient class that is already available in the release)already available in the release)

• A script-class that processes the responses A script-class that processes the responses (parser) and correlates the results with the actual (parser) and correlates the results with the actual data stored in the database (eXist). Here is data stored in the database (eXist). Here is decided whether the test has succeeded or failed.decided whether the test has succeeded or failed.

What has been done so farWhat has been done so far

• A document was send to the perf-dev A document was send to the perf-dev mailing list, listing the tests that are going to mailing list, listing the tests that are going to be done, in order to get feedback from the be done, in order to get feedback from the developersdevelopers

• Started working on the request generating Started working on the request generating class. Class for the LSRegisterRequest has class. Class for the LSRegisterRequest has been developedbeen developed

TO DOTO DO

• Finish the developing of the classes for Finish the developing of the classes for generating requests for the other types of generating requests for the other types of requestsrequests

• Make sure that the communication with the Make sure that the communication with the LS via the SOAPClient class is workingLS via the SOAPClient class is working

• Last ( but trickier ) the development of the Last ( but trickier ) the development of the script-class for parsing and checking the script-class for parsing and checking the responses from the LSresponses from the LS

Help NeededHelp Needed

• More like RFC documentation ( An accurate More like RFC documentation ( An accurate and detailed documentation is needed for and detailed documentation is needed for the tester to understand the way a service the tester to understand the way a service works and responds).works and responds).

• Maybe some information on how Maybe some information on how independent components of the services independent components of the services work, like the databases i.e.work, like the databases i.e.

• Suggestions for enriching and improving the Suggestions for enriching and improving the scheduled tests. scheduled tests.

An example – RRD MA An example – RRD MA

• Metadata Key RequestMetadata Key Request<?xml version='1.0' encoding='UTF-8'?><nmwg:message type="MetadataKeyRequest" id="mdrq1" …   <nmwg:parameters id="msgparam1"> <nmwg:parameter name="authToken">PIONIER-Public</nmwg:parameter> <nmwg:parameter name="timeValue">1127250495</nmwg:parameter> <nmwg:parameter name="timeType">unix</nmwg:parameter> </nmwg:parameters>

  <nmwg:metadata id="meta1"> <netutil:subject id="subj1"> <nmwgt:interface> <nmwgt:ifAddress type="ipv4">212.191.224.106</nmwgt:ifAddress> <nmwgt:ifName>ge-2/1/0.102</mwgt:ifName> <nmwgt:direction>out</nmwgt:direction> </nmwgt:interface> </netutil:subject> <nmwg:eventType>utilization</nmwg:eventType> </nmwg:metadata>

  <nmwg:data id="1" metadataIdRef="meta1"/>   </nmwg:message>

Test categories•Message Format•Message Values•Combination

Both correct and incorrectmessages are fed

An example – RRD MA An example – RRD MA

• Tested messagesTested messages• MetaData Key RequestMetaData Key Request• SetupData RequestSetupData Request• StoreRequestStoreRequest• All examples found inAll examples found in

• http://wiki.perfsonar.net/jra1-wiki/index.php/Schemata_and_Instanceshttp://wiki.perfsonar.net/jra1-wiki/index.php/Schemata_and_Instances

• Helped produce a number of testsHelped produce a number of tests

• Interaction with the LS will also be Interaction with the LS will also be tested. tested.

Current StatusCurrent Status

RRDMA Tests statusRRDMA Tests status1.1. Infrastructure completeInfrastructure complete

The main script - CoordinatorThe main script - Coordinator ExecutesExecutes

The creator script The creator script Creates the requests in xmlCreates the requests in xml filesfiles Notes the validity of the requests Notes the validity of the requests

The request scriptThe request script Actually used the SoapClient ClassActually used the SoapClient Class Feeds the xml requests to the service and outputs the responses to xml Feeds the xml requests to the service and outputs the responses to xml

filesfiles The parser scriptThe parser script

Checks the validity of the responsesChecks the validity of the responses Double-checks the responses with the actual data in the databaseDouble-checks the responses with the actual data in the database

Current Status 2Current Status 2

2.2. Creator ScriptCreator ScriptThe creator script is almost done.The creator script is almost done.Actually different creator scripts were created for different message Actually different creator scripts were created for different message

types.types.Minor changes are to be made when everything is completedMinor changes are to be made when everything is completed

3.3. Request ScriptRequest ScriptDoneDone

4.4. Parser ScriptParser ScriptWork in progress.Work in progress.So far 2 test types are recognised.So far 2 test types are recognised.Work is needed also on the double-checking mechanism.Work is needed also on the double-checking mechanism.

Current Status 3Current Status 3

RRDMA service has changed since RRDMA service has changed since perfSONAR 1.0. Major changes:perfSONAR 1.0. Major changes:

– 1.0 Uses Error Messages1.0 Uses Error Messages– 1.0 will use Error Codes1.0 will use Error CodesAn RRDMA 1.1 server should be able to interact An RRDMA 1.1 server should be able to interact

natively with a 1.0 clientnatively with a 1.0 clientThe functional testing team will check 1.1 version The functional testing team will check 1.1 version

against 1.1 and 1.0 tailored tests. against 1.1 and 1.0 tailored tests.

Q & AQ & A

• QuestionsQuestions

• CommentsComments