department of computer engineering propane an environment for examining the propagation of errors in...

Department of Computer Engineering

PROPANEPROPANEAn Environment for Examining theAn Environment for Examining thePropagation of Errors in SoftwarePropagation of Errors in Software

Martin HillerMartin Hiller, Arshad Jhumka, Neeraj Suri, Arshad Jhumka, Neeraj Suri

Chalmers University of TechnologyChalmers University of Technology

Göteborg, SwedenGöteborg, Sweden

{{hillerhiller, arshad, suri}, arshad, suri}@[email protected]


July 22, 2002Martin Hiller

PROPANE: An Environment for Examining the Propagation of Errors in SoftwareISSTA 2002

2

Overall ObjectivesOverall Objectives

A

D

C

E

B

F

Software reliability can be provided/increased Software reliability can be provided/increased by adding mechanisms that detect and correct by adding mechanisms that detect and correct data errors data errors wrappers, assertions, etc. wrappers, assertions, etc. (especially for black-box software)(especially for black-box software)

Given a software system and limited resources one would like to Given a software system and limited resources one would like to concentrate work on the most vulnerable/exposed parts of the concentrate work on the most vulnerable/exposed parts of the software, i.e. …software, i.e. …

Where do upcoming errors propagate?Where do upcoming errors propagate?




3

OutlineOutline

• Error Propagation & Basic ApproachError Propagation & Basic Approach• Examples of results obtained by using PROPANEExamples of results obtained by using PROPANE

– Aircraft arrestment systemAircraft arrestment system• Overview of PROPANEOverview of PROPANE

– Tool suiteTool suite– Requirements & limitationsRequirements & limitations

• Conclusions & some future directionsConclusions & some future directions




4

Error PropagationError Propagation

B

F

ErrorError = A system state which is = A system state which is different from the state in a ”correct” different from the state in a ”correct” execution of the system (i.e, execution of the system (i.e, notnot mutations or software defects)mutations or software defects)

Modules in a software system have different levels of exposure and Modules in a software system have different levels of exposure and different ”ability” to break error propagation (i.e., different levels of different ”ability” to break error propagation (i.e., different levels of error containment) – examining error propagation gives the developer error containment) – examining error propagation gives the developer a picture/profile of these levels.a picture/profile of these levels.

D

C

E

A




5

Basic Approach Basic Approach Error Injection Error Injection

A

D

C

E

B

F

1.1. Generate Generate Golden RunGolden Run, i.e., an error free , i.e., an error free reference runreference run

2.2. Generate Generate Injection RunInjection Run, i.e., a run in which , i.e., a run in which an error (i.e. erroneous system state) is an error (i.e. erroneous system state) is injectedinjected

3.3. Compare Compare Injection Run Injection Run with with Golden Run Golden Run to see which parts of the system were to see which parts of the system were corrupted by the injected errorcorrupted by the injected error

PROPANE PROPANE – Propagation Analysis Environment– Propagation Analysis Environment By instrumenting the target software, PROPANE can, during execution, log By instrumenting the target software, PROPANE can, during execution, log individual variables and events and inject errors into individual variables.individual variables and events and inject errors into individual variables.

= probe= probe= injection = injection locationlocation




6

Example: Aircraft Arrestment SystemExample: Aircraft Arrestment System

CLOCK

CALC

DIST_S

PRES_AV_REGPRES_S

ms_slot_nbr i

mscntpulscnt

slow_speedstopped

IsValue

OutValue TOC2ADC

TCNTTIC1PACNT

SetValueRotationsensor

Pressuresensor

Pressurevalve

Computer

Pressurevalve

Tape drum(original)

Tape drum(mirror)

Cable

Target system overviewTarget system overview Target software overviewTarget software overview




7

Results Generated by PROPANEResults Generated by PROPANE

• From low level to high level informationFrom low level to high level information– Signal/Variable PlotsSignal/Variable Plots– Propagation SignaturesPropagation Signatures– Propagation GraphsPropagation Graphs– Propagation SummariesPropagation Summaries

• Data compilations that can be used for further Data compilations that can be used for further analysisanalysis




8

Signal/Variable PlotsSignal/Variable Plots

PACNT

0

50

100

150

200

250

300

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000t (ms)

va

lue

Golden RunInjection Run

SetValue

0

500

1000

1500

2000

2500

3000

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000t (ms)

va

lue

Golden RunInjection Run

One-cycle single-bit upset at t = 1500 msOne-cycle single-bit upset at t = 1500 ms SetValue erroneous at t = 1539 msSetValue erroneous at t = 1539 ms




9

Propagation SignaturesPropagation Signatures

• Each injected error generates a Propagation Each injected error generates a Propagation SignatureSignature showing when and where the error showing when and where the error propagatedpropagated

1490 1500 1510 1520 1530 1540 1550 1690 1700t (ms)

PACNTpu

lscnt

i SetVal

ueO

utVal

ueTOC2ADC

IsVal

ue




10

Propagation GraphsPropagation Graphs

The Propagation Graph is generated in three different formatsThe Propagation Graph is generated in three different formats– PROPANE native formatPROPANE native format– GML (Graph Markup Language)GML (Graph Markup Language)– dot dot (part of the (part of the graphvizgraphviz tool suite from AT&T Research) tool suite from AT&T Research)

ProbedProbedlocationlocation

Incoming errors from locations earlier in the Incoming errors from locations earlier in the propagation pathpropagation path

Outgoing errors to locations later in the Outgoing errors to locations later in the propagation pathpropagation path

error counterror countttmin min / t/ tavg avg / t/ tmaxmax




11

Propagation Graph for PACNTPropagation Graph for PACNTStarting point

Propagation pathEach arc carries information regarding number of propagated Each arc carries information regarding number of propagated errors and propagation timeerrors and propagation time

Variables along the propagation trajectory




12

Propagation Summary for PACNTPropagation Summary for PACNT

SignalSignal Error Error countcount

Error Error raterate ttminmin ttavgavg ttmaxmax

PACNTPACNT 18401840 1.0001.000 00 00 00

pulscntpulscnt 18401840 1.0001.000 00 00 2020

ii 18111811 .984.984 00 44 2020

SetValueSetValue 12751275 .693.693 11 613613 41594159

OutValueOutValue 12751275 .693.693 11 613613 41594159

TOC2TOC2 12751275 .693.693 33 615615 41614161

ADCADC 12651265 .688.688 1010 629629 41684168

IsValueIsValue 12021202 .653.653 155155 682682 34673467

slow_speedslow_speed 769769 .418.418 00 20042004 58905890

mscntmscnt 11841184 .643.643 476476 29822982 62016201

ms_slot_nbrms_slot_nbr 11841184 .643.643 476476 29822982 62016201

TCNTTCNT 11841184 .643.643 476476 29822982 62016201

TIC1TIC1 11841184 .643.643 476476 29822982 62016201

Probed locationProbed location(variable)(variable)

For all incoming arcs to one node:For all incoming arcs to one node:

• Total error count and error rateTotal error count and error rate

• Combined tCombined tmin min / t/ tavg avg / t/ tmaxmax

Propagation Graphs and Propagation Summaries indicate which parts of the Propagation Graphs and Propagation Summaries indicate which parts of the observed system state that are observed system state that are most vulnerable most vulnerable to the injected errorsto the injected errors perhaps a perhaps a good location for error detection/recovery mechanismsgood location for error detection/recovery mechanisms??




13

Error Propagation from PACNTError Propagation from PACNT

Highest Error RateHighest Error Rate

Lowest Error RateLowest Error Rate

ms_slot_nbr i

mscnt

pulscnt

slow_speed

stopped

IsValue

OutValueTOC2ADC

TCNT

TIC1

PACNT

SetValue

CLOCK

PRES_S V_REG PRES_A

CALC

DIST_S




14

OverviewOverview




15

PLPLPROPANE LibraryPROPANE Library

• Static C Library providing the injection/logging APIStatic C Library providing the injection/logging API– propane_inject() propane_inject() /* Injects an error *//* Injects an error */– propane_log_var() propane_log_var() /* Variable probe *//* Variable probe */– propane_log_event() propane_log_event() /* Event probe *//* Event probe */

• Target must be instrumented with these functions and linked Target must be instrumented with these functions and linked with the librarywith the library

• Experiments are then set up using description filesExperiments are then set up using description files– Error types Error types (what errors to inject, e.g., bit-flips)(what errors to inject, e.g., bit-flips)– Error triggers Error triggers (when to inject errors, e.g., once/periodically)(when to inject errors, e.g., once/periodically)– Active probes Active probes (perhaps not all instrumented variables/events have to be (perhaps not all instrumented variables/events have to be

logged)logged)




16

Instrumentation ExampleInstrumentation Example

int spherical_volume( double radius ){ double volume;

volume = 4.0 * (PI * pow(radius, 3.0)) / 3.0;

return volume;}

int spherical_volume( double radius ){ double volume;

/* Injection location for radius */ propane_inject( IL_SPHERE_VOL, &radius, PROPANE_DOUBLE );

/* Probe the value of radius */ propane_log_var( P_RADIUS, &radius );

volume = 4.0 * (PI * pow(radius, 3.0)) / 3.0;

/* Probe the value of volume */ propane_log_var( P_VOLUME, &volume );

return volume;}

Original codeOriginal code Instrumented codeInstrumented code

At this point, instrumentation is At this point, instrumentation is unfortunately still a manual task. unfortunately still a manual task. However, automation is the next However, automation is the next step in the development of step in the development of PROPANE. PROPANE.




17

PSCPSCPROPANE Setup CreatorPROPANE Setup Creator

• Once the target is instrumented, experiments can Once the target is instrumented, experiments can be set upbe set up

• Given information on injection locations, probes, Given information on injection locations, probes, error types and occurrences, PSC will createerror types and occurrences, PSC will create– description files for running experimentsdescription files for running experiments– description files for data extraction and analysisdescription files for data extraction and analysis




18

PCDPCDPROPANE Campaign DriverPROPANE Campaign Driver

• After instrumentation and setup, the PCD runs the After instrumentation and setup, the PCD runs the experimentsexperiments– Invokes instrumented target executable containing the PL and all links to Invokes instrumented target executable containing the PL and all links to

external modules, e.g., environment simulatorexternal modules, e.g., environment simulator

• Provides continuous information on experiment status and Provides continuous information on experiment status and remaining workremaining work

• User controlUser control– PausePause– AbortAbort– SkipSkip




19

PDEPDEPROPANE Data ExtractorPROPANE Data Extractor

• Extracts and analyses data from the raw readouts produced Extracts and analyses data from the raw readouts produced during experimentsduring experiments– Perform Perform Golden Run ComparisonsGolden Run Comparisons, i.e., compare an injection run with a , i.e., compare an injection run with a

”clean” reference run to trace errors, generating a ”clean” reference run to trace errors, generating a Propagation SignaturePropagation Signature

– Collapse multiple propagation signatures into Collapse multiple propagation signatures into Propagation Graphs Propagation Graphs and and Propagation SummariesPropagation Summaries

– Prepare/compile data for further external analysis (using e.g. MatLab)Prepare/compile data for further external analysis (using e.g. MatLab)

1490 1500 1510 1520 1530 1540 1550 1690 1700t (ms)

PACNTpu

lscnt

i SetVal

ueO

utVal

ueTOC2ADC

IsVal

ue




20

Requirements & LimitationsRequirements & Limitations

• Programming languageProgramming language– The library itself is written in C The library itself is written in C the language in the target system must the language in the target system must

be able to interface with C-librariesbe able to interface with C-libraries• Environment simulatorEnvironment simulator

– Stimuli to the target system must be provided by an environment simulatorStimuli to the target system must be provided by an environment simulator– PROPANE provides basic interfacing functionality such that control of the PROPANE provides basic interfacing functionality such that control of the

EnvSim can be made part of the experiment setup (~plug-in’s)EnvSim can be made part of the experiment setup (~plug-in’s)• Target instrumentationTarget instrumentation

– Difficulties with real-time applications that cannot be made to run in Difficulties with real-time applications that cannot be made to run in simulated timesimulated time

• Current implementation focused towards single-node SWCurrent implementation focused towards single-node SW– Distributed SW not considered at this pointDistributed SW not considered at this point




21

ConclusionsConclusions

• PROPANE enables experimental evaluation of error PROPANE enables experimental evaluation of error propagation in SWpropagation in SW– Propagation profiles indicate which parts of system state that Propagation profiles indicate which parts of system state that

are vulnerable are vulnerable aid for equipping SW with error detection aid for equipping SW with error detection and recovery mechanismsand recovery mechanisms

• We envision that PROPANE can be used as a We envision that PROPANE can be used as a design stage aid for analysis of single node SWdesign stage aid for analysis of single node SW

• Limitations apply for some apsects of real-time and Limitations apply for some apsects of real-time and distributed SWdistributed SW




22

Some Future DirectionsSome Future Directions

• Increased automationIncreased automation– E.g. automatic instrumentation of target softwareE.g. automatic instrumentation of target software

• Extended analysis capabilitiesExtended analysis capabilities– E.g. provide your own analysis plug-insE.g. provide your own analysis plug-ins

• Open sourceOpen source– Possibly in the near futurePossibly in the near future




23

Further InformationFurther Information(and future download area)(and future download area)

www.ce.chalmers.se/LDC/www.ce.chalmers.se/LDC/DEEDSDEEDS

department of computer engineering propane an environment for examining the propagation of errors in...

Documents

software system

propagation of errors

software issta

software martin hiller

martin hiller propane

error propagation b

upcoming errors

software defects modules