three steps to avoid software obsolescence in your avionic
TRANSCRIPT
Advanced military avionic software is regularly updated in response to changing theatre requirements. A key challenge this raises for developers is how new features can be added without degrading performance to the point where expensive hardware upgrades are required. The answer is a three step process: measure timing, identify optimizations and evaluate the results.
White Paper
Three steps to avoid software obsolescence in
your avionic systems
On-target software verification solutions
Three steps to avoid software obsolescence in your avionic systems | page 2
Contents
Introduction – the obsolescence problem in legacy systems 3
1. Collecting timing information 5
2. Deciding what to optimize 8
3. Evaluating the results 10
Optimization studies by Rapita Systems 11
Conclusions – breathing new life into your legacy system 13
Rapita Verification Suite (RVS) 15
Free trial version 16
Three steps to avoid software obsolescence in your avionic systems | page 3
Introduction – the obsolescence problem in legacy systems Unlike most critical real-time embedded systems, military avionics systems are
regularly upgraded through software updates over an operational life stretching to
decades. New and emerging capability requirements mean developers are compelled
to keep pace with military demands which can lead to frequent, expensive hardware
upgrades. This is where the “obsolescence problem” begins. What is this problem
and what can you do about it?
Do you really need more software?Through-life software updates gradually
increase the demands placed on the
underlying computing platform, e.g. CPU
utilization. This can lead to decreased
performance capability and intermittent
failures due to timing overruns.
These timing failures almost always go
unnoticed until development is nearly
complete and final tests are underway.
Worse still, timing issue failures may
even fail to be detected during system
testing and go on to cause operational
problems post-deployment.
When timing failures are spotted,
considerable effort is usually spent to
identify and then resolve the cause of
the problem, often to no avail. System
developers then face the difficult choice
of either abandoning planned new
features, leading to capability decay,
or replacing hardware at considerable
cost because of early obsolescence.
A third option would be to increase the
effectiveness of optimization.
Software optimizationWhen avionics software is incrementally
upgraded it suffers from two problems:
1. The original architecture degrades –
the long timescales involved
in avionics projects mean that
changes are inevitably made by
engineers who were not involved in
the original design and may not fully
understand the rationale behind the
original design decisions.
2. As additional capabilities are
introduced, the amount of code
the application needs to execute
increases.
The consequence of 1 and 2 is that the
execution time of tasks or partitions
increases. Eventually the execution time
exceeds the time budget allocated.
When a set of enhancements are
planned, a practical first step is to
create room for those enhancements
by optimizing the performance of the
existing software. This results in the
ability to add new functionality without
hardware upgrades.
In this paper, we
show that a viable
solution to the
“obsolescence
problem” is to
optimize the
performance of
the software as
updates are made.
Three steps to avoid software obsolescence in your avionic systems | page 4
Improving the effectiveness of optimizationSoftware optimization efforts are often
far less productive than they could
be. Sometimes the most obvious
optimizations make no difference to
overall performance. While this has
no effect on the longest execution
time and doesn’t create room for
new functionality, it takes up valuable
engineering time, impacts timescales
and can have a detrimental effect on
software clarity and maintainability.
The key to improving the effectiveness of
optimization lies in targeting the places
where optimization can have the biggest
impact. This can be achieved by taking
detailed timing measurements when the
software is run on the target hardware
and analyzing the measurements.
The steps to improving optimization are:
1. Collecting timing information
2. Deciding what to optimize
3. Evaluating the results
“The key to improving
the effectiveness of
optimization lies in
targeting the places
where optimization can
have the biggest impact.”
Three steps to avoid software obsolescence in your avionic systems | page 5
Safety critical systems and timing confidence
1. Collecting timing information
Measuring execution times
The time an embedded real-time system
takes to respond to some event comes
from several sources, including physical
components (actuators, for example)
and electronic sub-systems. However,
the biggest variable is often the
execution time of the software itself.
Not all real-time systems treat timing
requirements in the same way. On some
systems it may be the case that deadlines
must always be met, on others average
CPU utilization may be more important.
For example, the developers of
safety critical systems such as flight
controls must be able to show to a
very high degree of confidence that
the software will always execute within
its time constraints. In other systems,
if the software occasionally misses
a deadline, then this may not be
disastrous; however, it may still
impact on performance, robustness,
and the customer’s perceptions of
product quality.
Engineers typically measure the
execution time of major software
components such as partitions and
tasks during system tests. Ideally, these
measurements should reveal how
often, if at all, execution time budgets
are exceeded, giving some degree
of confidence that the timing behavior of
the system will be as expected
once deployed.
To obtain these measurements,
the standard approach is to place
instrumentation points at the start and
end of the sections of code being
measured. These instrumentation points
record the elapsed time, either by
toggling an output port monitored via
an oscilloscope or logic analyzer or by
reading an on-chip timer and recording
the resulting timestamps in memory.
Recording end-to-end execution time
measurements in this way allows
engineers to see the distribution of
execution times for each major software
component and also the high-water
mark: the longest execution time
observed. The distribution of execution
times and the high-water mark provide
evidence of, and some confidence in,
the timing behavior of the system.
Three steps to avoid software obsolescence in your avionic systems | page 6
Going beyond the high-water mark
Why more detailed timing information is required
60 (f3)
10
10
10
50 (f2)20 (f1)
5 (f4)
110 140 85
Figure 1: Execution Paths
Adding more features to software may
produce high-water mark execution
times which show software taking too
long to execute. Unfortunately these
measurements cannot identify those
parts of the code which contribute
most to the overall execution time and
therefore offer no guidance on the
code to optimize.
This is a significant problem, because
optimizing code that is not on the
worst-case path does not address
timing issues, and may even
encourage costly attempts to re-write
large volumes of code which risks
introducing bugs into the software.
Many systems will benefit from
detailed timing information which
offers confidence in timing correctness
and accurately targets software
optimization at code which is always
on the worst-case path.
Unfortunately, there are some
fundamental problems inherent in the
simple end-to-end measurements
used to obtain execution time high-
water marks. The lack of detail from
an end-to-end measurement conceals
heavy consumers of execution time.
The high-water marks may not reflect
the longest time that the code could
take to execute. This happens when
the longest path through the code has
not been exercised by the tests. While
code coverage tools can be used to
check whether the available tests cover
all of the code, for example statements,
conditions, decisions and MC/DC, it is
possible, and in fact is often the case,
that the longest overall path has still
not been executed, as illustrated by the
example in Figure 1.
Let us assume that two tests are run,
represented by the green path and the
blue path. This is sufficient to give 100%
statement coverage, 100% decision
coverage, and, depending on the
complexity of the conditions within the
two decisions, 100% MC/DC coverage.
The observed execution
times from the two tests
are 110 and 85.
Of course there is another
valid path through the
code, the red path, which
has an execution time of
140. Because adequate
coverage is achieved with
the green and blue paths, it
isn’t necessary to execute
the red path. This illustrates
why code coverage
analysis alone won’t show
whether the longest path
has been exercised.
Three steps to avoid software obsolescence in your avionic systems | page 7
Adding instrumentation points
Instrumenting commercial scale code
by hand is technically feasible, albeit
extremely laborious. Attempting to
manually correlate trace data with
program structural information increases
the effort requirements by orders of
magnitude beyond that.
Fortunately, the tasks of program
instrumentation, trace processing,
combining trace data with program
structural information, and data mining/
presentation are all amenable to
automation.
Commercial tools such as RapiTime
facilitate:
» Automatic instrumentation of the source
code at various levels of abstraction.
» Automatic combination of structural
information (obtained by static
analysis of the source code at
instrumentation time) with trace data
(obtained by running instrumented
code on the target).
» Reductions in the bandwidth and/
or storage of traces by automatically
limiting the range of values required
to distinguish IDs uniquely.
What information can you gather?TEST COVERAGE INFORMATION
» Placing an instrumentation point at the start of each function
reveals which functions were executed during testing.
» Instrumentation can provide even richer measurements of
coverage, right up to MC/DC.
» Learn how many times each sub-path was executed during
testing and whether you need to address any gaps in testing.
» Determine/check whether the expected maximum number of
iterations of each loop are seen or even exceeded in practice.
EXECUTION TIME INFORMATION
» Determine the worst-case execution time of the software, even if
the worst-case path is not actually executed during testing, but
all the individual sub-paths are.
» Find out which lines of code are on the worst-case path and which
ones are not, identifying or ruling out candidates for optimization.
» See how the end-to-end execution time of each function varies
over the different tests, revealing the correlation between specific
test cases and long execution times.
» Obtain average, minimum and maximum execution times as well
as other useful information, including the number of times each
sub-program was executed by the tests and the number of lines
of code in each sub-program.
Taking advantage of automation developments
An effective way of obtaining
detailed timing information is to add
instrumentation points at each decision
point in the code. Each of these points
needs to have an identifier (ID) that allows
it to be distinguished from the others.
Whenever an instrumentation point is
executed, its ID and a timestamp are
captured, hence running a series of tests
on the system results in the creation of a
timing trace containing instrumentation
point ID – timestamp pairs.
Three steps to avoid software obsolescence in your avionic systems | page 8
2. Deciding what to optimize
Don’t guess – measure!While the temptation to guess where
the the biggest contributors to WCET
(Worst-Case Execution Time) are,
optimize the code and then assess the
results must be strong, you should resist
such an urge. Extensive experience
of software optimization across many
different avionics, telecommunications
and automotive electronics projects tells
us that even highly experienced software
engineers, when unable to access
detailed timing information, struggle to
identify significant contributors to the
worst-case execution time. Without this
information they cannot pinpoint the best
candidates for optimization.
Detailed execution time measurements
taken during extensive testing can
highlight problems where certain
software components overrun or have
the potential to overrun their budgets.
Such problems can occur during initial
system development, and/or as a result
of adding new functionality during
part of an upgrade. Only a systematic
and scientific approach which reveals
accurate timing information about all of
the software components in the system
can resolve the problem. The answer is
simple: don’t guess, measure!
Find the significant contributors to the WCETFigure 2 illustrates how different
sub-programs contribute significantly
different amounts to the worst-case
execution time. This illustration is based
on timing data provided by RapiTime.
For each sub-program, RapiTime
determines the contribution of the code
in that sub-program to the worst-case
execution time.
By inspecting contribution data (see
Figures 3 and 4), engineers can easily
identify a relatively small number of
sub-programs where optimization could
potentially have a large impact on the
overall worst-case execution time of the
software component.
We note that the largest contributors
to the worst-case execution time
often come from code that is seldom Figure 2: Illustration of cumulative contribution of sub-programs to the overall WCET
100%
Cum
ulat
ive
cont
ribut
ion
to th
e lo
nges
t exe
cutio
n tim
e
Number of sub-programs All (100%)0
(1) Most sub-programs contribute nothing to the longest execution time (they are not on the worst-case path)
(2) Many sub-programs contribute a small amount to the longest execution time
(3) A small number of sub-programs contribute a large fraction of the longest execution time
executed and so is not highlighted by
conventional profiling tools that assess
average case performance.
Three steps to avoid software obsolescence in your avionic systems | page 9
WCET optimizations
Figure 3: This shows how
RapiTime presents the
contributions of specific
subprograms to worst-
case execution times
(W-SelfCT%) – the longest
predicted execution time –
and to the high water mark
(H-SelfCT%) – the longest
observed execution time.
Figure 4: Going further it is possible to identify specific blocks of code within these sub-
programs, showing as for this example that 4 lines of code (Ipoint 11: lines 154-156 and
Ipoint 12: line 158) are responsible for 30.4% of WCET (W-OverCT%) and 31.4% of HWM
(H-OverCT%).
The key to any optimization strategy is
to prioritize those optimizations where
the minimum effort, and the minimum
amount of compromise in other factors,
is required to gain the maximum benefit
in terms of execution time reduction.
The best way to meet this objective is
to identify the level at which to perform
optimization. Optimizations can be
classified at three levels: design-level,
sub-program-level and low-level.
Design-level optimizations:
Optimizations at the design-level refer to
changes in the overall design of the system,
for example changes to the API of
a subsystem or other structural changes.
Sub-program-level optimizations:
At this level, the focus is on changes
within a single sub-program, or a set of
tightly coupled sub-programs, without
changing the specification of those
components. Examples here include
substituting one algorithm for another.
Low-level optimizations: Low-level
optimizations focus on the generated
machine code. These optimizations
aim to use the most efficient machine
instructions available for performing
particular operations.
Three steps to avoid software obsolescence in your avionic systems | page 10
3. Evaluating the resultsOnce optimizations have been made to the code, good engineering
practice suggests that we should repeat the measurements to see
whether the desired results have been achieved.
Figure 5: Process for optimization
Even more than in the first step,
measurement automation is a
key benefit to us. Rerunning the
measurements is largely a case of
repeating the tests. Once a process for
collecting and analyzing measurements
has been set up, the incremental cost
of performing the measurements again
is not much higher than the cost of
repeating the tests.
This low cost of repeating measurements
means that it is possible to consider
several candidate optimizations, and to
iterate through a process of evaluating
the effectiveness of each.
Identify Optimization Candidate(s)
Assess Potential
Gains
Sensitivity Analysis
Implement Potential
Optimizations
Evaluate
Three steps to avoid software obsolescence in your avionic systems | page 11
Optimization studies by Rapita Systems
In this section we show the level
of improvements in worst-case
execution times that can be obtained
through a simple process of software
optimization. These results were
achieved using RapiTime to provide
detailed timing information.
The results are from avionics projects
located in the UK and Europe. The results
are given on a per software partition
basis and the partitions are referred to by
the characters X, Y, Z etc. The partition,
project and company names are not
used for confidentiality reasons.
Software Optimization results valuable 10% reductions in the overall
execution time of partitions V and X.
In the case of partition X, the software had
been optimized repeatedly over a number
of years. Despite this and to the surprise
of the engineers involved, it was possible
to reduce the worst-case execution time
by over 10% via a set of simple changes
to low level code. This would not have
been possible without the focus on worst-
case hotspots by RapiTime.
Partition Lang. Notes Optimization Level Percentage Improvement
U Ada High data throughput Design level 60%
V Ada Spark exception Low level 10%
W Ada Efficient block copy Sub-program 40%
X Ada Multiple simple optimizations Low level 10%
Y Ada Efficient block copy Design level 50%
Z Ada Loop variables Sub-program 15%
Table 1: Reduction in WCET via Software Optimization using RapiTime
Improving worst-case execution times
The data in Table 1 shows how a
reduction in worst-case execution time
is made possible by targeted software
optimizations. Where design level
modifications could be made, these lead
to substantial improvements in execution
time, more than halving the overall
execution time of partitions U and Y. Sub-
program optimizations to partitions W and
Z reduced worst-case execution times by
15-40%, while relatively minor low-level
optimizations resulted in significant and
Three steps to avoid software obsolescence in your avionic systems | page 12
It is important to note that all of the
optimizations made to achieve the
gains shown in Table 1 were relatively
straightforward and had little impact
on code maintainability/quality. It is
notable that in the right places, simple
modifications brought significant
rewards. In all cases, the amount of
code in sub-programs highlighted
Manual or automatic approach to
as worst-case hotspots represented
just a few percent of the total for the
partition. In other words, the volume of
code inspected and optimized was a
small fraction of the total, varying from
0.5% to 2% for the low level and sub-
program optimizations, to a maximum of
approximately 10% for more extensive
design level optimizations.
sub-system represented by a partition.
Further, there was no clear information on
which components actually contributed
to the overall worst-case execution time.
Our solution was a systematic process
of automated timing analysis and tightly
focused software optimization which
created significant headroom and
facilitated the addition of new functionality
without the need for hardware upgrades
or capability reduction.
optimization? Manual identification of optimization
opportunities in large systems such as
these is a daunting, if not impossible
task. The systems were developed
over a number of years by large
teams of engineers. The sheer size
and complexity made it difficult, if not
impossible for a single engineer to gain
an in-depth understanding of the entire
Three steps to avoid software obsolescence in your avionic systems | page 13
We conclude with an explanation of the key advantages offered by detailed
timing information for the development of avionics systems, and illustrate how
automated technology can produce a significant return on investment.
Conclusions – breathing new life into your legacy system
» Engineers take a systematic and
scientific approach to obtaining
confidence in the timing behavior of
the system before it is deployed.
» Detailed information about the
contribution of sub-programs and
blocks of code to the worst-case
execution time enables quick
identification of worst-case hotspots
and candidates for optimization.
» ”What-if?” analysis quantifies the
maximum performance gains
obtainable by optimizing selected
software components. Together, with
hotspot analysis, the engineer is
guided to apply optimization effort
where it will have the maximum
benefit in terms of reducing the
overall execution time, and ensuring
correct timing behavior of the system.
» Timing analysis of prototype
optimizations allows their impact
to be quantified, managing the
process of including only the most
effective optimizations.
» Finally, on-target code coverage
analysis can be achieved using the
same instrumentation and trace
capture process as performance
measurement. On-target code
coverage analysis has the
advantage of increasing the quality
of testing, and hence reducing the
number of functional and timing
problems that make it through into
deployed systems.
The effectiveness of RapiTime as a means of obtaining
detailed timing information on complex avionics systems
has been demonstrated in a number of avionics projects
with significant reductions in the overall execution time
achieved via simple modifications to a small fraction of the
code.
The key advantages
Three steps to avoid software obsolescence in your avionic systems | page 14
Return on investmentFigure 6 illustrates the costs and
benefits of obtaining different levels of
timing information manually and using
an automated approach.
The cost-benefit curve for a manual
approach to timing analysis is
represented by the dashed line. Here
the cost and benefit of doing nothing
is zero. The cost of obtaining minimal
timing information e.g. just high-water
mark execution times is relatively low,
and the benefit of doing so exceeds
the cost. In fact obtaining this level of
timing information probably gives the
best return on investment for the
manual approach.
The solid line represents the cost-benefit
curve for an automated approach
to timing analysis, supported by
sophisticated measurement-based
execution time analysis tools such as
RapiTime and trace data capture devices.
Investing in this technology has a
set-up cost, and so it is probably not
cost effective if all that is required is
some simple high-water mark values.
However, once set up, the cost of
obtaining detailed timing information,
on-target code coverage, and worst-
case hotspot analysis is relatively small,
as are the costs involved in using this
highly focused information to improve
the system.
With automated tool support, the benefit
of obtaining this information greatly
exceeds the costs and offers a much
larger return on investment than the
manual approach.
Figure 6: Cost-Benefit curves
for manual and automated
approaches to timing analysis
3
2 2
Benefit
Positive return on investment
=
;
Basic timing info
On-target code coverage
Detailed timing data
Hotspot analysis
Rolmanualapproach
Rolautomatedapproach
Negative return on investment
11
3
4 4
1
2
3
4
Cost
Three steps to avoid software obsolescence in your avionic systems | page 15
ConclusionsUsing RapiTime technology for systematic timing measurement
and software optimization:
» Provides the capability to reduce the
frequency of obsolescence-induced
upgrades, and hence the costs of
re-establishing system qualification
status after hardware upgrade.
» Has significant lifecycle cost benefits.
» Allows you to track the spare
processing capacity available for
new software features and manage
change so as to delay the point
at which this spare capacity is
exhausted.
» Can be applied to both new systems
and retrospectively to existing
deployed systems thus maximizing
the through life cost benefit.
Rapita Verification Suite (RVS)RVS tools offer on-target timing verification and optimization for critical real-time
embedded systems (RapiTime) and code coverage measurement for system and
integration testing processes (RapiCover). Together these tools provide powerful
timing verification and code coverage support capabilities for engineers and
designers working on 8, 16 and 32-bit microcontrollers with C, C++ and Ada.
Free trial versionRapita Systems is offering a free trial version of RVS, giving you the opportunity to
assess the features and benefits of the latest product for yourself.
The trial provides a ready-to-go
version of RVS that allows you to
evaluate execution time measurement,
conduct code coverage and undertake
performance analysis.
The RVS trial version includes:
» RVS. This includes both command line
tools and graphical user interface. This
version is limited to 30 days use with
small applications only.
» Full documentation.
» Example embedded real-time
applications and tutorial running in
the Eclipse IDE.
» ARM9 Simulator to allow example
applications to run without an
embedded target.
» ARM9 Compiler to build example
applications.
Contact Rapita Systems to request a
trial or visit:
www.rapitasystems.com/free-trial
We will provide you with a website
address and a unique download code.
Tel (UK/International):
+44 (0)1904 413945Rapita Systems Inc.41131 Vincenti Ct.
Novi, MI 48375
Rapita Systems Ltd.Atlas House, Osbaldwick Link Road
York , YO10 3JB
Registered in England & Wales: 5011090
Tel (USA):
+1 248-957-9801
Email: [email protected] | Website: www.rapitasystems.comDocument ID: MC-WP-003 Three steps to avoid software obsolescence v3LR