it's all about software testing...6 sut to be stimulated during test execution by different...
Post on 30-May-2020
4 Views
Preview:
TRANSCRIPT
V1.0 | 2018-11-06
Concepts for Unit and Functional System Tests
It's All About Software Testing
2
Situation: Perspective of Unit Testing and Functional System Testing
3
Goal: Joined Perspective of Testing
4
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Reuse in the test
design
Flexibility in the
chosen test design
method
Consideration of
both DUT and test
variants
Adequate test
coverage
Analysis and
reporting during
and after test run
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Traceability and
Continuous
Integration
5
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Flexibility in the
chosen test design
method
Consideration of
both DUT and test
variants
Adequate test
coverage
Analysis and
reporting during
and after test run
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Traceability and
Continuous
Integration
6
SUT to be stimulated during test execution by different kind of “sources”:
Sequential test input to stimulate and check the SUT
System environment, e.g. remaining bus simulation with other ECUs
The “real world”, e.g. a road course
Mapping the asynchronous behavior of the system environment and “real world” to a sequential test script is hardly possible
Challenges with Testing
SUT
Test Script
7
Challenges with Testing
Test Script
8
Model
• plant model
Challenges with Testing
Model
• system environment
Test Script
9
Challenges with Testing
SUT
Data Flow
Control Flow
Model
• plant model
Model
• system environment
Test Script
10
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Flexibility in the
chosen test design
method
Consideration of
both DUT and test
variants
Adequate test
coverage
Analysis and
reporting during
and after test run
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Traceability and
Continuous
Integration
11
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Flexibility in the
chosen test design
method
Consideration of
both DUT and test
variants
Adequate test
coverage
Analysis and
reporting during
and after test run
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Traceability and
Continuous
Integration
12
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Separation of test
logic,
implementation
and test data
Flexibility in the
chosen test design
method
Consideration of
both DUT and test
variants
Adequate test
coverage
Analysis and
reporting during
and after test run
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Traceability and
Continuous
Integration
13
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Separation of test
logic,
implementation
and test data
Flexibility in the
chosen test design
method
Consideration of
both DUT and test
variants
Adequate test
coverage
Analysis and
reporting during
and after test run
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Traceability and
Continuous
Integration
int a
int a
float b
int a
float b
int64 x
int a
int a
float b
int64 x
14
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Separation of test
logic,
implementation
and test data
Flexibility in the
chosen test design
method
Consideration of
both DUT and test
variants
Adequate test
coverage
Analysis and
reporting during
and after test run
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Traceability and
Continuous
Integration
15
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Separation of test
logic,
implementation
and test data
Flexibility in the
chosen test design
method
Integration of
different test
notations in one
tool (graphical,
tabular, coding)
Consideration of
both DUT and test
variants
Adequate test
coverage
Analysis and
reporting during
and after test run
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Traceability and
Continuous
Integration
16
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Separation of test
logic,
implementation
and test data
Flexibility in the
chosen test design
method
Integration of
different test
notations in one
tool (graphical,
tabular, coding)
Consideration of
both DUT and test
variants
Adequate test
coverage
Analysis and
reporting during
and after test run
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Traceability and
Continuous
Integration
int a
int a
float b
int a
int a
float b
int64 x
int a
float b
float b
int64 x
int64 x
17
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Separation of test
logic,
implementation
and test data
Flexibility in the
chosen test design
method
Integration of
different test
notations in one
tool (graphical,
tabular, coding)
Consideration of
both DUT and test
variants
Adequate test
coverage
Analysis and
reporting during
and after test run
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Traceability and
Continuous
Integration
18
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Separation of test
logic,
implementation
and test data
Flexibility in the
chosen test design
method
Integration of
different test
notations in one
tool (graphical,
tabular, coding)
Consideration of
both DUT and test
variants
Variant handling
for test structure,
test logic and
parameters
Adequate test
coverage
Analysis and
reporting during
and after test run
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Traceability and
Continuous
Integration
19
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Separation of test
logic,
implementation
and test data
Flexibility in the
chosen test design
method
Integration of
different test
notations in one
tool (graphical,
tabular, coding)
Consideration of
both DUT and test
variants
Variant handling
for test structure,
test logic and
parameters
Adequate test
coverage
Analysis and
reporting during
and after test run
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Traceability and
Continuous
Integration
#ifdef
20
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Separation of test
logic,
implementation
and test data
Flexibility in the
chosen test design
method
Integration of
different test
notations in one
tool (graphical,
tabular, coding)
Consideration of
both DUT and test
variants
Variant handling
for test structure,
test logic and
parameters
Adequate test
coverage
Analysis and
reporting during
and after test run
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Traceability and
Continuous
Integration
21
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Separation of test
logic,
implementation
and test data
Flexibility in the
chosen test design
method
Integration of
different test
notations in one
tool (graphical,
tabular, coding)
Consideration of
both DUT and test
variants
Variant handling
for test structure,
test logic and
parameters
Adequate test
coverage
Design features
for definition of
relevant test case
data and
generation of test
cases
Analysis and
reporting during
and after test run
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Traceability and
Continuous
Integration
22
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Separation of test
logic,
implementation
and test data
Flexibility in the
chosen test design
method
Integration of
different test
notations in one
tool (graphical,
tabular, coding)
Consideration of
both DUT and test
variants
Variant handling
for test structure,
test logic and
parameters
Adequate test
coverage
Design features
for definition of
relevant test case
data and
generation of test
cases
Analysis and
reporting during
and after test run
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Traceability and
Continuous
Integration
int a float b int64 x
23
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Separation of test
logic,
implementation
and test data
Flexibility in the
chosen test design
method
Integration of
different test
notations in one
tool (graphical,
tabular, coding)
Consideration of
both DUT and test
variants
Variant handling
for test structure,
test logic and
parameters
Adequate test
coverage
Design features
for definition of
relevant test case
data and
generation of test
cases
Analysis and
reporting during
and after test run
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Traceability and
Continuous
Integration
24
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Separation of test
logic,
implementation
and test data
Flexibility in the
chosen test design
method
Integration of
different test
notations in one
tool (graphical,
tabular, coding)
Consideration of
both DUT and test
variants
Variant handling
for test structure,
test logic and
parameters
Adequate test
coverage
Design features
for definition of
relevant test case
data and
generation of test
cases
Analysis and
reporting during
and after test run
Analysis windows,
e.g. for
visualization of
value charts
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Traceability and
Continuous
Integration
25
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Separation of test
logic,
implementation
and test data
Flexibility in the
chosen test design
method
Integration of
different test
notations in one
tool (graphical,
tabular, coding)
Consideration of
both DUT and test
variants
Variant handling
for test structure,
test logic and
parameters
Adequate test
coverage
Design features
for definition of
relevant test case
data and
generation of test
cases
Analysis and
reporting during
and after test run
Analysis windows,
e.g. for
visualization of
value charts
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Traceability and
Continuous
Integration
float b
int a
int64 x
int a
26
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Separation of test
logic,
implementation
and test data
Flexibility in the
chosen test design
method
Integration of
different test
notations in one
tool (graphical,
tabular, coding)
Consideration of
both DUT and test
variants
Variant handling
for test structure,
test logic and
parameters
Adequate test
coverage
Design features
for definition of
relevant test case
data and
generation of test
cases
Analysis and
reporting during
and after test run
Analysis windows,
e.g. for
visualization of
value charts
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Traceability and
Continuous
Integration
27
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Separation of test
logic,
implementation
and test data
Flexibility in the
chosen test design
method
Integration of
different test
notations in one
tool (graphical,
tabular, coding)
Consideration of
both DUT and test
variants
Variant handling
for test structure,
test logic and
parameters
Adequate test
coverage
Design features
for definition of
relevant test case
data and
generation of test
cases
Analysis and
reporting during
and after test run
Analysis windows,
e.g. for
visualization of
value charts
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Measure code
coverage both
with unit and
system tests
Traceability and
Continuous
Integration
28
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Separation of test
logic,
implementation
and test data
Flexibility in the
chosen test design
method
Integration of
different test
notations in one
tool (graphical,
tabular, coding)
Consideration of
both DUT and test
variants
Variant handling
for test structure,
test logic and
parameters
Adequate test
coverage
Design features
for definition of
relevant test case
data and
generation of test
cases
Analysis and
reporting during
and after test run
Analysis windows,
e.g. for
visualization of
value charts
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Measure code
coverage both
with unit and
system tests
Traceability and
Continuous
Integration
Set LightIntensity to 30
Wait 500ms
Check LightState is on
Call x = GetLightState(intensity = 30)
Check x == 1
29
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Separation of test
logic,
implementation
and test data
Flexibility in the
chosen test design
method
Integration of
different test
notations in one
tool (graphical,
tabular, coding)
Consideration of
both DUT and test
variants
Variant handling
for test structure,
test logic and
parameters
Adequate test
coverage
Design features
for definition of
relevant test case
data and
generation of test
cases
Analysis and
reporting during
and after test run
Analysis windows,
e.g. for
visualization of
value charts
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Measure code
coverage both
with unit and
system tests
Traceability and
Continuous
Integration
30
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Separation of test
logic,
implementation
and test data
Flexibility in the
chosen test design
method
Integration of
different test
notations in one
tool (graphical,
tabular, coding)
Consideration of
both DUT and test
variants
Variant handling
for test structure,
test logic and
parameters
Adequate test
coverage
Design features
for definition of
relevant test case
data and
generation of test
cases
Analysis and
reporting during
and after test run
Analysis windows,
e.g. for
visualization of
value charts
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Measure code
coverage both
with unit and
system tests
Traceability and
Continuous
Integration
Connection to test
management tools
and Jenkins,
changed-based
testing
31
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Separation of test
logic,
implementation
and test data
Flexibility in the
chosen test design
method
Integration of
different test
notations in one
tool (graphical,
tabular, coding)
Consideration of
both DUT and test
variants
Variant handling
for test structure,
test logic and
parameters
Adequate test
coverage
Design features
for definition of
relevant test case
data and
generation of test
cases
Analysis and
reporting during
and after test run
Analysis windows,
e.g. for
visualization of
value charts
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Measure code
coverage both
with unit and
system tests
Traceability and
Continuous
Integration
Connection to test
management tools
and Jenkins,
changed-based
testing
Req1:
Pressing the button switches the light on.
Req1:
Passing function parameter value ‚1‘ leads to return value ‚true‘.
32
Challenges with Testing
Early test of
components,
independent of
availability of
other components
Usage of models
Reuse in the test
design
Separation of test
logic,
implementation
and test data
Flexibility in the
chosen test design
method
Integration of
different test
notations in one
tool (graphical,
tabular, coding)
Consideration of
both DUT and test
variants
Variant handling
for test structure,
test logic and
parameters
Adequate test
coverage
Design features
for definition of
relevant test case
data and
generation of test
cases
Analysis and
reporting during
and after test run
Analysis windows,
e.g. for
visualization of
value charts
Code coverage to
meet standards as
ISO26262,
IEC62304,
DO-178B
Measure code
coverage both
with unit and
system tests
Traceability and
Continuous
Integration
Connection to test
management tools
and Jenkins,
changed-based
testing
33
Conclusion
The requirements and challenges on test environments of HIL system tests, SW system tests, integration tests and unit tests are widely the same
Corresponding test techniques are applicable to all kind of tests
Vector strategy: provide a comprehensive test solution for all kind of tests based on well established tools
Combine the strength of vTESTstudio and CANoe for system tests with the strength of VectorCAST for code based tests
34 © 2018. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. V1.0 | 2018-11-06
Tariq Javaid Vector UK
For more information about Vectorand our products please visit
www.vector.com
top related