1

March 3, 2011

Automating SCA Testing with Spectra CX

INTRODUCTION

Spectra CX is a model-driven development tool that forms the backbone of the Software Communications Architecture (SCA) development process.

Spectra CX (SCX) reduces development risk, accelerates development and improves the software quality of SCA applications and platforms through SCA validation and automatic generation of SCA artefacts.

The SCA Test feature facilitates a key element in this development process by providing industrial strength SCA component test code generation, execution of tests on host or target systems, and detailed reporting of test results.

2

AGENDA

Overview of the SCA Test feature

Benefits of SCA Test

How it works

How you use SCA TestWorkflow

Demonstration

Customization and Extensibility

3

Validation versus Testing

XML Descriptors

Binaries

Validation(Static)

Test(Dynamic)

ValidationReport

TestCases

ValidationRules

TestReport

4

SCA Model

Spectra CX – SCA Test

Model-based testing framework for the SCAPlug-in feature for Spectra CX 3.2.X

Provides radio developers with a standalone capability for unit testing SCA components

On-host or on-target using TargetLoader Implementation independent, black box, testing using JUnits and CORBA

Includes test cases for the Component Lifecycle interfaceSupports user-defined test cases

5

Tests – Base Application Interfaces

CF::Port Test connectPort succeeds for a valid connectionTest connectPort fails with an InvalidPort exception (error code 1) when called with an invalid object referenceTest connectPort fails with an InvalidPort exception (error code 1) when called with an object reference which can't be narrowed to the correct typeTest connectPort succeeds where number of connections <= maximum (where maximum is finite)Test connectPort fails with an OccupiedPort exception when the number of connections exceeds the maximum (where maximum is finite)Test disconnectPort succeeds for a valid connectionTest disconnectPort fails with an InvalidPort exception (error code 2) when the connection ID is invalidTest disconnectPort fails with an InvalidPort exception (error code 2) when attempting to disconnect an already disconnected port

CF::LifeCycle Test initialize succeedsNote: No negative tests as these are implementation dependant

Test releaseObject succeeds

CF::TestableObject Test runTest fails with an UnknownTest exception when trying to run an unknown testTest runTest fails with an UnknownProperties exception when invalid properties are passed to runTestTest runTest succeeds when valid properties are passed to a valid test

CF::PortSupplier Test getPort succeeds for a valid portTest getPort fails with an UnknownPort exception for an invalid port

CF::PropertySet Test configure of invalid simple propertyTest query of invalid simple propertyTest configure of valid simple propertyTest configure of valid simple property with invalid typesTest query of valid simple propertyTest query of exec paramTest configure then query of valid simple propertyTest query returns all properties if no property specifiedTest query returns only specified properties (1 property)Test query of external allocation properties.Test configure of properties with range (< min, min, max, > max)

CF::Resource Test identifier matches value expectedTest start completes successfullyTest stop completes successfullyTest configure works when resource is stoppedTest query works when resource is stoppedTest start works when resource is stopped

CF::ResourceFactory Test identifier matches value expectedTest createResource succeeds for a valid resourceTest createResource returns a reference to the same resource when a resource is created twiceTest releaseResource succeeds for a valid resourceTest releaseResource fails for an invalid resourceTest releaseResource only releases resources with multiple references when the reference count is 1Test shutdown succeeds

Note CF::Port tests are currently generated by the same script which generates tests for CF::PortSupplier

Note: Similar tests could also be written for SimpleSequence, Struct and StructSequence properties

6

Tests – Base Device Interfaces

CF::Device Test usageStage attribute is initially set to IDLETest adminState attribute is initially set to UNLOCKEDTest softwareProfile attribute contains the correct valueTest label attribute contains the correct valueTest allocateCapacity with an invalid property IDTest deallocateCapacity with an invalid property IDTest allocateCapacity with invalid typesTest deallocateCapacity with invalidTypesTest releaseObject succeeds (including checking relevant state changes are published to a mock IDM, sub devices released, device manager de-registration)

CF::LoadableDevice Test load succeeds for a valid mock executableTest load fails when called with invalid LoadType for device (multiple tests)Test multiple load of an executable succeedsTest load executable fails when called with an invalid filenameTest unload succeeds when given a valid filenameTest unload succeeds after unloading multiple loaded executables

Test unload with duplicate valid file names (should fail the 2nd time)

CF::ExecutableDevice Test execute succeeds for a valid mock executableTest execute succeeds for a valid executable with valid propertiesTest execute succeeds for a valid executable with valid optionsTest execute fails when invoked with an invalid nameTest execute fails when invoked with invalid propertiesTest execute fails when invoked with an invalid optionsTest execute fails when invoked with an executable which hasn't been loaded firstTest terminate succeeds with valid PIDTest terminate fails with invalid PIDTest terminate fails with PID of process which has terminatedTest double terminate of PID

CF::AggregateDevice Test addDevice succeeds for a valid deviceTest removeDevice succeeds for a valid deviceTest behaviour of addDevice when adding a duplicate deviceTest removeDevice fails for a device which has not been addedTest addDevice for a a nil device failsTest removeDevice for a nil device failsTest addDevice failure results in a FAILURE_ALARM log record being generated if appropriateTest removeDevice failure results in a FAILURE_ALARM log record being generated if appropriateTest child devices are released when releaseObject is called (will require a mock device)

7

Benefits of Unit Testing

By automating the process of testing SCA components and applications, SCA Test significantly accelerates the testing cycle, improves user productivity and establishes high quality software from the very beginning of the development cycle

SCA Test can be used to test the compliance of software that may have been hand written, generated automatically using a Spectra CX Code Generator, or generated using a 3rd party tool.

Testing early on and throughout the development cycle reduces the risks that affect time to market, cost and software quality.

By exercising components at the beginning of the development cycle in a runtime environment, and by continuing to test as components are refined, developers can be assured continuous SCA compliance up to development cycle completion.

8

Benefits of SCA Test

SDR/SCA-specific model-based testingProvides a test capability that is optimized for SDR/SCA

Testing based on the current design and implementation

Completely integrated with the Spectra CX development environment, All SCA Test compile, build and execute actions can be initiated from the Spectra CX

environment.

Users can prepare the runtime environment for execution and observe their code running on host and/or target.

Developers benefit from the simplicity of using only one tool for modeling, validation, code generation, execution, and test.

Tests can be executed on host or targetUnit testing can even begin with only a basic CORBA connection to the component

allowing testing to begin before a full platform with a live SCA operating environment is available.

Multiple implementations of a component can be tested to ensure proper operation on all defined implementation platforms.

9

How it Works 10

SCA TestFeature

Eclipse JUnitFramework (Java)

SCA TestTarget Loader (C++)

ComponentUnder Test

Communicates withvia CORBA

Instantiates

GeneratesJUnits

Instantiates& Terminates

Component

Host

Host or Target

TargetLoader

In an SCA system typically the operating environment and more specifically the platform elements take care of instantiating components and or devices. Since it is desirable to test components before this may be available or without the overhead of the SCA operating environment we provide TargetLoader software.

TargetLoader is responsible for loading the component under test into the appropriate environment.

Typically there will be one TargetLoader for each target environment

A TargetLoader can be referenced by multiple component implementations -- don’t have to model the TargetLoader for each component implementation Once a TargetLoader is running, it can be reused for multiple test executions

11

SCA Test Workflow

Generate

Compile

Execute

Evaluate

12

PrepareFor each

component to be tested

Create test project(s)/model(s)

Build the TargetLoader(s)

Run the TargetLoader(s)

Model the TargetLoader(s)

Generating Component Tests

SCA Test generates a JUnit test for a component’s LifeCycle, PortSupplier, PropertySet, ResourceFactory, Resource and Device interfaces

Any component implementation can be designated for automatic testing.

A comprehensive suite of tests will be automatically generated for each component.

The generated test harness includes all the code needed for connecting to and exercising, through CORBA, the resource interface on the component under test -- an SCA Core Framework is not required.

13

Compiling Component Tests

The target may be an actual SDR platform or the host development environment, depending on the compiler and CORBA tooling that is available for the test harness.

The generated testing software can be fully integrated into the project’s build process.

Spectra CX can be directed to automatically build the generated test harness into an executable, ready to run on host or target.

14

Executing Component Tests

SCA Test generates the JUnits which interact with the Target Loader

Target Loader deploys and instantiates the component implementation and test harness binaries on the host or target platform

Users simply initiate the test from Spectra CXOnce initiated, execution of the tests defined for SCA resource compliance is carried out by the JUnit through the CORBA environment, without need for an SCA Core Framework to be present

Results of the tests are recorded by the JUnit

15

Evaluating Test Results

Upon completion of the tests, a unified report is presented in the SCX user interface

This report can then be used to correct any issues identified and/or prove compliance with the SCA standard for components that support the LifeCycle, PortSupplier, PropertySet, ResourceFactory, Resource or Device interfaces

A standard JUnit report (xml) is also generatedThis can be converted into HTML and other formats

16

DEMONSTRATION

17

Customization and Extensibility

Spectra CX uses open standards to translate model information to test code and gives users full access to the code template.

Users can modify testsEngineering teams can optimize the template to accommodate their environment if they need to.

Provide customer extensible framework for unit testing a component

Generate empty tests for all provides ports methodsCustomers can add their own JUnit tests

Generate Target Loaders from the tool using build configurations

Provides support for the customer’s target platform(s)

18

Adding Unit Tests for an SCA Component

1. Create a new Java Project

2. Create a new class for the test cases

3. Configure the build path for the project

4. Add a JUnit library

5. Add a default constructor to the test class

6. Add test cases

19

Create a new Java Project

Select ‘File > New >Java Project’ while in the Java Perspective.

In the ‘New Java Project’ wizard set the ‘Project Name’. Leave the other values as their defaults. Click the ‘Next’ button.

20

Set the project parameters

On the ‘Java Settings’ page switch to the ‘Project’ page.

Click the ‘Add…’ button and add the project that contains your generated test code.

21

Create a new class for the test cases

Right-click on the “src” folder in the new Java project and select ‘New > Class’.

22

Set the class fields

In the ‘New Java Class’ wizard set the ‘Package’, ‘Name’ and ‘Superclass’ fields.

The ‘Superclass’ field should be set to the implementation test class from the generated test code.

23

Configure the build path for the project

Configure the build path for the project so that the class will not have compilation errors.

Select the new Java project and right-click, then select ‘Build Path > Configure Build Path…’

24

Add a JUnit Library

Switch to the ‘Libraries’ page and click on ‘Add Library…’.

Select the ‘JUnit’ library

Make sure that the ‘JUnit Library Version’ is set to JUnit 3.

25

Add a default constructor to your class

For example, the following shows a default constructor that makes use of the default behavior and changes the path to the component under test binary. If the component is generated by Spectra CX then your path will resemble this pattern “../<generated src project name>/” + hostFileName.

public EncryptorUnitTests() throws Exception {super();hostFileName = "../EncryptDecrypt_SRC/" + hostFileName;

}

Test out your extension using Run As > JUnit Test

26

SUMMARY

Spectra CX 3.2 includes the SCA Test feature which provides for:Standalone, unit testing of componentsImplementation independent, black box, testing using JUnitsOn-host or on-target using TargetLoader in place of a Core FrameworkBasic component lifecycle interface testing -- 32 test casesThe addition of user-defined test cases (JUnits)

Future releases will expand the test coverage with cases forComplete Lifecycle interface testing – 129+ test casesComponents, Assemblies, and ApplicationsPlatforms: Core Framework, Services, …

We will also improve the testing capability withSCA-specific JUnit framework – easier to create test casesSupport testing profiles -- standard (e.g. SCA, SCA-Next) and customSupport for test hardware

27

Additional Information 28

For additional information on PrismTech’s Spectra products and services:

E-mail:

info@prismtech.com

Website:

http://www.prismtech.com

THANK YOU

Mike WilliamsSpectra CX, Product Management

mike.williams@prismtech.com

29