SACS — A Pattern Language for Safe Adaptive ControlSoftware

André A. HaugeDepartment of ICT Risk and Dependability

Institute for energy technology, Halden, NorwayDepartment of InformaticsUniversity of Oslo, Norwayandre.hauge@hrp.no

Ketil StølenDepartment of Networked Systems and Services

Sintef ICT, Oslo, NorwayDepartment of InformaticsUniversity of Oslo, Norwayketil.stolen@sintef.no

ABSTRACTThis article puts forward a pattern language for Safe Adap-tive Control Software named SACS. We interpret the term“pattern language” in line with Alexander in [1] where a setof patterns, the interconnections between them, and howthese are intended to be used make up a language. Thepattern language consists of three basic types of patterns,namely the Requirement Pattern, the Design Pattern, andthe Safety Case Pattern and a composite pattern type. Thepatterns are intended to be used in the context of safety re-lated and safety critical systems, thus the safety aspect is aprincipal concern. The pattern language may be used as atool for e.g. safety engineers and system developers to in-crease effectiveness during conceptual design and facilitateeffective evaluation of alternative adaptive design solutionswith respect to utilisation in a safety related application.

Categories and Subject DescriptorsD.2.10 [Software Engineering]: Design—Representation

General TermsDesign, Documentation

Keywordspattern language, adaptive control software, safety assur-ance

1. INTRODUCTIONAdding adaptive behaviour to the features of a control sys-tem increases the complexity as well as the potential risk oferroneous behaviour. As it is reasonable to assume that theobjective for adding adaptive behaviour is to yield a positivenet effect and at the same time uphold the safety integritycompared to a traditional control system, we may derive thefollowing general requirements:

i) the adaptive control feature shall provide a level ofperformance (or increased level of safety with respectto events that require a resilient handling) which maynot be feasible with conventional control techniques;

ii) the adaptive control feature shall have no negative ef-fect on safety (or decrease level of safety with respectto events which is normally mitigated by conventionalcontrol techniques).

The requirements (i) and (ii) may be satisfied by differentmeans and strategies. While requirement (i) put constraintson the provision of a justification for introducing adaptivityand on providing the confidence that a higher level of perfor-mance may be obtained and by what means. Requirement(ii) put constraints on the need to provide a safety demon-stration. In this article we will focus on the safety aspect.Three possible approaches for satisfying (ii) may be:

I) to constrain the adaptation mechanism such that theadaptable controller may not evolve into an unsafe con-figuration;

II) to evaluate at runtime that any adaptation is safe be-fore committing a modification of the control software;

III) to constrain the set of operational states for which theadaptive controller is granted control privileges ratherthan constrain the adaptation mechanism.

In this article, we address the challenges associated with theapproach outlined in alternative (III). A means to uphold aminimal level of performance and assure safe control couldbe provided by having a backup controller that is grantedcontrol privileges in those scenarios where an adaptable con-troller may not be trusted to provide sufficient safety mar-gins.

We specify three complementary types of patterns and howthey may be combined in order to augment a pattern lan-guage solution. The patterns represent an initial set of pat-terns intended as parts of an extensible pattern language.

An overview of the SACS language is given in Section 2.Section 3 presents a set of SACS patterns. Section 4 de-scribes the use of the presented patterns in the context ofan example railway interlock system. Section 5 concludesand describes future work.

Figure 1: The Basic Elements of SACS

2. THE SACS PATTERN LANGUAGEThe UML class diagram [5] in Figure 1 illustrates the mainelements available in SACS.

There are three basic pattern types available in SACS. TheRequirement Pattern type is inspired by the problem framesapproach [3]. The problem frames approach is a means tospecify the problem domains and interaction phenomena be-tween problem domains in order to derive requirements forthe system which is intended to be built. The Design Pat-tern type is inspired by the well-known GoF1 [2] approach todescribe design solutions, whereas the Safety Case Patterntype focuses on providing solutions on how to demonstratethat a system is sufficiently safe for its intended purpose.The goal of the Safety Case Pattern is to provide an ap-proach to the structure of claims such that it may be logi-cally deduced and concluded that the system, built accord-ing to a SACS pattern, is sufficiently safe.

The semantics of the combinators may be summarised into:

satisfies — a Design Pattern shall satisfy a Require-ment Pattern. More concretely, the Design Pattern isbound to the Requirement Set contained in a Require-ment Pattern. The Requirement Set acts as a param-eter for the satisfies combinator and thus details therelationship by specifying the requirements that shallbe satisfied by different parts of the design. A DesignPattern shall thus satisfy the Requirement Set of a Re-quirement Pattern. Although a generic RequirementSet is provided, a concrete Requirement Set is givenonce the Requirement Pattern is instantiated.

enables — a Design Pattern enables the use of a SafetyCase Pattern as a means to demonstrate safety. More

1The GoF abbreviation is short for Gang of Four and refersto the four authors of the book [2].

concretely, the Safety Case Pattern is bound to theDesign Pattern through the System Specification con-tained in a Design Pattern. The System Specifica-tion acts as a parameter for the enables combinatorand provides the context for the Safety Demonstra-tion. The System Specification that is contained ina Design Pattern is abstract and generically defined,a concrete System Specification is obtained once theDesign Pattern is instantiated.

includes — a pattern may rely on other patterns aspart of its specification. The combinator does not takea parameter in this case, this is shown in Figure 1 bythe default null parameter. An includes relationshipshould be interpreted such that there is a dependencyto the included pattern.

contains — a pattern embodies an artefact that isgenerically defined and intended to be refined by theuser when instantiating a pattern. A pattern artefactthat is provided by one pattern may act as a parameterin another pattern. The contains combinator are usedto denote the distinct part of a source pattern thatused as a parameter in a target pattern. The differentpattern types holds different pattern artefacts where aRequirement Pattern holds a Requirement Set, a De-sign Pattern holds a System Specification and a SafetyCase Pattern holds a Safety Demonstration.

Where there is a one-to-many relationship between patterns(e.g. a Design Pattern has a satisfies relationship to severalRequirement Patterns), the following rules may be used inorder to provide additional detail:

or — relationship between a source and two or moretargets defines that the targets represent alternativeswhere only one is required for completeness;

Figure 2: Pattern Integration and Binding

and — relationship between a source and two or moretargets defines that all the target patterns are requiredfor completeness.

In this article, one-to-many relationships and the includescombinator is not used, thus no further details are given.All other basic elements of SACS as illustrated in Figure 1are addressed and will be explained in the following sections.

3. A SACS COMPOSITE PATTERNSection 3.1 provides an overview of the Trusted BackupComposite Pattern and illustrates by example one possibleinstance of a Composite Pattern and how a set of basic pat-terns may be bound to each other. The Composite Patternis the result of choosing a set of patterns in order to solve aspecific task and combine them by applying the rules intro-duced in Section 2. The Composite Pattern is composed ofthree basic patterns, one for each of the three available pat-tern types Requirement Pattern, Design Pattern and SafetyCase Pattern. The basic patterns are explained individuallyin the Sections 3.2, 3.3 and 3.4.

3.1 Pattern Integration and BindingA pattern in SACS may be a basic pattern or any compo-sition of patterns framed within a Composite Pattern de-scription. We illustrate the relationships between patternsin Figure 2 by adapting the UML [5] collaboration notationto express a SACS Composite Pattern.

A Composite Pattern is defined by a user as a means to de-tail the set of patterns that will be used to solve a particular

development problem. The Composite Pattern in Figure 2 isa triplet of basic pattern types and consists of a RequirementPattern, a Design Pattern and a Safety Case Pattern. A wellformed Composite Pattern consists of patterns addressingall the three aspects emphasised in SACS namely require-ment, design and safety demonstration issues. As a patternin SACS may be a basic pattern or any composition of pat-terns, including composition of Composite Patterns, thereare syntactic rules for specification of well formed Compos-ite Patterns. Here we will describe the rules by exampleof the Composite Pattern illustrated in Figure 2 which is asimple and well formed Composite Pattern.

The Trusted Backup Composite Pattern is well formed inthat the composite contains patterns from each of the threebasic pattern types and that there is a match between thecontent a pattern artefact (contained in a source pattern)and the content of a pattern that is bound to the patternartefact.

The Trusted Backup Composite Pattern takes Operator andPlant as parameters. The specification of parameters atthe Composite Pattern level is the result of extracting theset of parameters specified for its constituent RequirementPattern. The basis for defining parameters of the TrustedBackup Composite Pattern then is the parameters of theRequirement Pattern named State Space Subset. These pa-rameters represent the parts of the world that the systemunder development will interact with and is given or detailedonce a user decides upon the context for which the patternis applied. The Operator and Plant parameters representthe human operator interacting with the adaptive system

and the process or equipment to be controlled by means ofadaptive control respectively.

The Trusted Backup design pattern has a satisfies relation-ship to the Sate Space Subset Requirement Pattern, and aenables relationship to the Adapting Within a ConstrainedState Space Safety Case Pattern. In Figure 1, we see thatthe satisfies and the enables combinators take parameters,we thus show the relationship between the patterns by theirparameters in Figure 2. The parameters, or more concretelythe binding classes, are provided by the patterns for whichthere are defined a contains relationship. In Figure 2, a re-lationship between a pattern and a specific pattern artefact(i.e the anonymous classes of type Requirement Set, SystemSpecification and Safety Demonstration) is denoted with thecontains keyword whereas the parametric use of the patternartefact is denoted with either the satisfies or enables key-word.

The basic integration idea may be summarised as follows:

The Requirement Pattern is parameterised in order toelaborate upon the problem of adaptive control withrespect to a specific context. The specific context of in-terest with respect to the State Space Subset Require-ment Pattern are given by the parameters Operatorand Plant.

The Requirement Set, defined as a part of the StateSpace Subset Requirement Pattern, contains a Require-ment Set for each named designed problem domainpart of the pattern. Each Requirement Set associatedwith a named part contains a set of requirements ad-dressing the required function or constraints associ-ated with the respective named part. These namedparts represent parts that are assumed to be repre-sented in any system solution that intends to satisfythe Requirement Pattern. Examples of named partsspecified in the State Space Subset pattern are theAdaptive Controller, the Trusted Controller and theControl Delegator. As a result of instantiating therequirement pattern, the named parts may be givenmore descriptive names and the generic requirementsmay be concretised and further detailed. In order toprovide detailed linking, a common naming schemeare used where that is appropriate. As an example,the Adaptable Controller entity appearing in the StateSpace subset Requirement Pattern (Section 3.2) is in-tended to be linked with the Adaptable Controller en-tity as described in the Trusted Backup Design Pattern(Section 3.3). These entities, appearing in differentpatterns, represent the same entity but are addressedfrom different perspectives. The Adaptable Controllerentity represents a problem domain when appearing inthe Requirement Pattern whereas it represents a sys-tem part when appearing in the Design Pattern. TheAdaptable Controller also appears in the Safety CasePattern (Section 3.4), but then as a contextual ele-ment.

The Design Pattern named Trusted Backup specifiesthe structure or behaviour of parts that corresponds inname to the named Requirement Sets. Thus a named

part in a Design Pattern should satisfy the require-ments of the correspondingly named Requirement Set(or otherwise linked).

The Design Pattern provides an abstract System Spec-ification. The instantiation of the Design Pattern pro-vides a concrete System Specification. It is assumedthat the System Specification obtained from instanti-ating the Design Pattern will contain important infor-mation regarding e.g. operating context and what isacceptably safe system behaviour within its intendedcontext, besides pure design issues.

The Safety Case Pattern uses the System Specifica-tion as an input to put the parts of the pattern in acontext. A linking is provided through the differentpattern artefacts and pattern parameters such that asafety demonstration is outlined. The concrete bind-ing is provided by the user when instantiating the pat-terns.

3.2 State Space Subset: Requirement PatternName: State Space Subset.

Intent: The intent is to address the basic problem domainsassociated with a control system which employ a strategywhere an adaptable controller is only allowed to operate ina limited part of the operational state space. The strategyinvolves dividing the state space of what is to be controlledinto a set of overlapping operational regions and use redun-dant and alternative means of control in order to reduce riskassociated with adaptive controller failure.

Motivation: The motivation for applying the pattern iswhen the adaptive control feature may provide an optimi-sation effect in a limited part of the operational state spaceand where there exist alternative means of control whichmay be applied in order to protect against or mitigate erro-neous adaptive behaviour. Given such a problem, the statespace may be divided into control regions where safety mustbe guaranteed and regions where an unconstrained adaptivecontroller will not negatively affect safety and thus may beallowed to operate. An example of such a problem may bean adaptive flight control system such that the flight perfor-mance of an aeroplane might be improved or provided withresilience to unanticipated events. A representative case ispresented in [6]. A set of limit values on the pitch, roll andyaw rotation along their respective axis with respect to theplane’s centre of mass may be used to delimit the state spacefor which adaptive control operations may be applied. A re-gion of the state space could then be specified by defining aset of rules which detail the values for the pitch, roll and yawrotation respectively for which adaptive control may be al-lowed. In the remaining part of the overall operating region,adaptive control would not be allowed such that a standardflight control system should also be available.

Applicability: The pattern is suitable to apply in the fol-lowing situations:

• when it is assumed that an operational state space maybe divided into diverse operational regions;

Figure 3: Problem Diagram - State Space Subset

• when it is assumed that adaptive control may provideoptimisation of operations within a region of an oper-ational state space;

• when it is assumed that a safe fall-back system maymitigate unwanted operations occurring as an effect oferroneous adaptive controller.

Related Patterns: The Trusted Backup Design Patternhas a satisfies relationship to the State Space Subset Re-quirement Pattern which details the main requirements whichmust be satisfied by design.

Safety Classification: General requirements.

Problem Context: The Problem Diagram (denoted bythe PD abbrevation in the upper left corner of the frame)provided in Figure 3 illustrates the main problem domainswhich play a part in the pattern. The different diagramentities are defined according to the problem frames notation[3] and may be interpreted as follows:

Operator — represents a given biddable problem do-main in the form of a human operator which interactswith the system;

Adaptable Controller — represents a designed part ofthe system (designed problem domain) which providescontrol functionality and which incorporates ability toadapt;

Trusted Controller — represents a designed or givenpart (may be a given system such as a legacy systembut is here represented as a designed problem domain)of the system which provides control functionality;

Control Delegator - represents a designed part of asystem which is responsible for delegation of control;

Monitor — represents a designed part of a systemwhich is responsible for monitoring;

Plant — represents the process or equipment to becontrolled, defined as a given problem domain;

Operating Regions Delegation Scheme — representsthe problem of specifying the diversification of the op-erational state space in a manner which allow spe-cialised controllers to operate in different regions ofthe operational state space. This is represented as adesigned description and is associated with the ControlDelegator.

Problem Frame: Figure 3 illustrates and adaptation ofthe problem frames notation [3] for expressing SACS pat-terns and specifies the main problem domains related to thedivision of an operational state space and identifies the needto specify requirements (with reference to the requirementelements in Figure 3) addressing the problems of: (i) cap-ture the main intent of the pattern by a system level speci-fication of requirements (ii) accounting for controller limita-tions; (iii) accounting for monitoring and detection limita-tions; and (iv) accounting for the hazards related to controloperations that must be avoided. The letters on the arcsof Figure 3 denote the interfaces between problem domains.The descriptive names associated with these letters, as pro-vided in the lower frame, represent shared phenomena (e.g.events) which interact between problem domains.

Based on an identification of relevant problem domains, theirinterfaces and shared phenomenon we may derive the generic

requirements which must be addressed when designing a so-lution. With respect to capturing the pattern intent identi-fied by problem (i) above, the main aspects to be resolvedare:

a) to provide a system solution that optimise service andguarantee safety by the use of a diverse set of con-trollers;

b) to partition the state space such that for each state ofthe operational state space, at least one controller (inthe set of controllers) may provide service;

c) to partition the state space such that for each stateof the operational state space, the system shall alwaysprovide a safe service;

d) to delegate control in such a manner that for each stateof the operational state space, the adaptable controlleris utilised given that safety integrity of the system maynot compromised by the adaptive control feature orotherwise do not represent an inappropriate means ofcontrol.

The main aspects which influence the state space partition-ing with respect to problem (ii) are:

a) the ability to delimit AC operations in such a mannerthat there is still a benefit of having an AC;

b) to assure that suboptimal AC behaviour does not neg-atively affect safety;

c) to assure that erroneous AC behaviour may be miti-gated;

d) the ability of a TC to guarantee safe behaviour;

e) the ability of a TC to mitigate erroneous AC actions.

The main aspects which influence the state space partition-ing with respect to problem (iii) are:

a) the ability to detect erroneous AC behaviour;

b) the ability to detect and analyse the need to adapt ACin order to meet stated goals;

c) the ability to detect system component events whichmight have a negative effect on safety;

d) the ability to detect Plant and environment eventswhich might have a negative effect on safety.

The main aspects which influence the state space partition-ing with respect to problem (iv) are:

a) the hazards associated with the regions of the statespace for which adaptable control feature are sought;

b) the failure modes or performance fluctuations that mightbe associated with the AC;

c) the ability to provide sufficient safety margins in theAC controlled part of the state space such that hazardsassociated with erroneous AC behaviour may be safelyhandled.

Requirements: This section specifies the generic require-ments and capture the main specification challenges whenapproaching the problem as defined in the intent section.The requirements are derived from the description of theproblem as contained in the sections Problem Context andProblem Frame.

The requirements are contained within the Requirements Setidentified as State Space Subset Requirement Set. The setcontains Requirement Sets for each designed domain identi-fied as the Adaptable Controller Requirement Set (require-ments denoted with the letters AC for short), Trusted Con-troller Requirement Set (TC), Control Delegator Require-ment Set (CD) and Monitor Requirement Set (M). In addi-tion a Requirement Set is defined that confine system spe-cific requirements, requirements belonging to this set aredenoted with the letter (S) for short. For each requirementthere is given a short ID (e.g. AC or TC identifying therespective Requirement Set) and a running number, the IDidentify the main subject of the requirement. In additiona note paragraph giving additional informative guidance isassociated with each requirement. The note paragraph alsogives reference to the associated problem description andparagraph for which the requirement originated as a meansto provide traceability.

The State Space Subset Requirement Set is defined as:

S.1 The System shall by the use of a set of controllersprovide capability according to a safety preserving ap-plication of control means in Operating Region.

NOTE: Refer to problem (i), paragraph a). Here ca-pability may refer to the implementation of the rulesfor delegation of control in some sort of a delegationscheme, algorithm or other mechanism that may be de-termined to be safety preserving. The set of controllermay be any configuration of controller means, e.g. aset consisting of several different adaptable controllers,conventional programmable controllers and human con-trollers.

S.2 The System shall by appropriate means incorporate adiversification of the Operating Region such that a di-verse set of controllers may operate in different parts ofthe operational domain specified by controller specificController Operating Region

NOTE: Refer to problem (i), paragraph b). If the Op-erating Region represents the state space for which thesystem is intended to provide service, then a ControllerOperating Region needs to be specified for each con-troller, specifying the partition of the state space forwhich it is intended to provide service. As it is assumedthat different controllers may not necessarily share adefined Controller Operating Region (depends on thecontext), it must be assured that there is at least onemeans of control for each possible state in the Operat-ing Region.

S.3 The System shall ensure that at least one safe means ofcontrol for every state specified by Operating Region.

NOTE: Refer to problem (i), paragraph c). In the pat-tern, TC is assumed to be an inherently safe meansof control. The ability of the AC to be a safe meansof control depends on the mitigations that may be pro-vided by the system and the delimitation AC controldefined by the AC specific Controller Operating Region.Safe means of control may also refer to the ability touse a human operator, any last resort function or pro-tection system in order to assure safe operations orotherwise graceful degradation of service.

S.4 The System shall delegate control to the most suitablecontroller with respect to the goal of providing highperformance and assure safe control for every state de-limited by the specification of Operating Region.

NOTE: Refer to problem (i), paragraph d). Severalcontrollers may qualify to provide service in a specificregion of the state space (in the event of overlappingController Operating Regions). For each operationalstate, the most suitable controller with respect to theability to achieve performance goals shall be chosenunder the assumption that the sample of controllersto choose from may be safely utilised in the scenario.Potential harmful controllers for a specific operationalscenario shall always be suppressed.

AC.1 The AC part of the System shall/should provide capa-bility within Adaptable Controller Operating Region.

NOTE: Refer to problem (ii), paragraph a). The capa-bility that are sought needs to be specified together witha description of the assumed trade-offs in order to pro-vide a convincing argument that the adaptive featurewill yield a positive net effect in the associated Adapt-able Controller Operating Region. The Adaptable Con-troller Operating Region represents the specification ofthe delimitation of AC operations.

AC.2 Suboptimal AC behaviour with negative effect on safetyshall be mitigated at the system level or sub-systemlevel by mitigations.

NOTE: Refer to problem (ii), paragraph b). Subopti-mal behaviour and their potential negative effects onsafety needs to be specified such that proper mitiga-tions may be defined. Mitigations here refer to theset of means applied on any system level such that nonon-acceptable system failure due to suboptimal AC be-haviour may be experienced.

AC.3 Erroneous AC behaviour with negative effect on safetyshall be mitigated at the system level or sub-systemlevel by mitigations.

NOTE: Refer to problem (ii), paragraph c). Erroneousbehaviour and their potential negative effects on safetyneeds to be specified such that proper mitigations maybe defined. Mitigations here refer to the set of meansapplied on any system level such that no non-acceptablesystem failure due to erroneous AC behaviour may beexperienced.

TC.1 The TC part of the System shall guarantee capabilityin Trusted Controller Operating Region.

NOTE: Refer to problem (ii), paragraph d). Capabil-ity here refers to functionality that are intended to beprovided by the TC. The Trusted Controller OperatingRegion represents the specification of the part of thestate space for which TC is intended to be applied.

TC.2 The TC part of the System shall guarantee capabilityto mitigate erroneous AC behaviour.

NOTE: Refer to problem (ii), paragraph e). Capabilityto mitigate here refers to the ability of the TC, in theevent of erroneous AC behaviour, to “takeover” controland mitigate negative system effects. The problem maybe associated with the challenge of providing ByzantineFault Tolerance.

M.1 The M part of the System shall detect erroneous ACbehaviour by capability.

NOTE: Refer to problem (iii), paragraph a). Capa-bility here refer to the set of means which provide theability to detect erroneous behaviour of the AC.

M.2 The M part of the System shall detect and analyse theneed to adapt AC by capability.

NOTE: Refer to problem (iii), paragraph b). The fea-ture of detection and analysis of the need to adapt mayas an alternative be a part of the adaptation logic ofthe AC. Then the requirement should be defined in thecontext of the AC Requirement Set.

M.3 The M part of the System shall by capability detectsub-system and component events and failures thatmay lead to negative effects on safety.

NOTE: Refer to problem (iii), paragraph c). Capabil-ity here refers to the sum of monitoring means, e.g.cross-monitoring and self-monitoring mechanisms inthe system, and the ability to accumulate informationin a monitor part of the system such that the internalhealth of the system may be determined.

M.4 The M part of the System shall by capability detectPlant and environment effects which may lead to neg-ative effects on safety.

NOTE: Refer to problem (iii), paragraph d). Capabil-ity here refers to the sum of monitoring means, e.g.sensor inputs or external monitoring systems, and theaccumulation of information to a monitor part of thesystem such that the health of the system to be con-trolled (e.g. appliance damage) and its environment(e.g. detect radiation leakage to environment) may bedetermined.

CD.1 The CD part of the System shall by capability dele-gate control according to a safety preserving schemefor application of control means.

NOTE: Refer to problem (iv), paragraph a). Here ca-pability may refer to the implementation of the rulesfor delegation of control in a Delegation Scheme, algo-rithm or other mechanism that may be determined tobe safety preserving.

CD.2 The CD part of the System shall by capability delegatecontrol to appropriate means in the event of hazardousAC performance fluctuations.

NOTE: Refer to problem (iv), paragraph b). Capabilityhere may refer to the implementation of functionalityto accommodate unexpected instability, fluctuations orother destabilising behaviour of the AC due to adapta-tion.

CD.3 The CD part of the System shall by capability incorpo-rate sufficient safety margins such that erroneous ACbehaviour may not propagate to the system level andinvalidate the ability to provide safe service.

NOTE: Refer to problem (iv), paragraph c). Safetymargins may here be interpreted to represent the set ofmeans applied in order to provide fault containment.It may not be possible to guarantee that an AdaptableController of arbitrary assurance level fail in an un-predictable manner, but one may demonstrate that anarbitrary failure to such a component is detected andmitigated or otherwise not possible to be manifested asa system failure.

As can be seen from the specification of requirements, theState Space Subset Requirements Set consists 5 RequirementSets containing a total of 16 requirements. The requirementsare divided among the sets in the following manner: 4 Sys-tem requirements, 3 Adaptable Controller requirements, 2Trusted Controller requirements, 4 Monitor requirementsand 3 Control Delegator requirements.

3.3 Trusted Backup: Design PatternName: Trusted Backup.

Intent: The intent of the pattern is to allow an adaptablecontroller, that might be of an arbitrary assurance level, tocontrol a safety related plant. The adaptable controller ishere introduced in order to accommodate changes in theplant as a means to uphold service despite changes (e.g.adjust to plant degradation) or improve service over time(e.g. better fitting of control law). The intent is to dividethe operational state space associated with some controlledplant into regions. The adaptable controller is in this controlscheme not a trusted component thus it is only granted con-trol privileges in those regions, called safety regions, wherethere are no negative effects of controller failure. In or-der to provide service in those regions which are not safetyregions or take over control if the adaptable controller ex-periences failure, a redundant trusted controller is grantedcontrol privileges. The controllers are developed accordingto different principles in which a Trusted Controller guaran-tees safe control but is less resilient to changes in the plantwhereas an Adaptable Controller accommodate changes andcan optimise accordingly but does not guarantee safe con-trol. The System controls the plant in a manner which em-phasizes both optimal and safe control by delegating controlprivileges to the most suitable controller depending on thestate of the plant.

Motivation: The pattern employ a strategy where the po-tential benefit of utilising adaptable means of control is pro-vided, but where effort required to demonstrate that the

Adaptable Controller is safe is to a large degree evaded. TheAdaptable Controller is intended to provide an increasedlevel of performance compared to a conventional softwaresystem. Any erroneous operation of the Adaptable Con-troller shall not negatively impact safety thus the AdaptableController may be of an arbitrary safety assurance level.

An example of such a problem may be an adaptable flightcontrol system where adaptiveness is used to improve flightperformance of an aircraft. We assume here that adaptivebehaviour may be introduced in order to improve roll axisrotation control with the effect of improving passenger com-fort during flight. With respect to passenger comfort, wefurther assume that the optimal roll angle should be suchthat the plane is stabilised. When the plane is in a reason-ably stable state, an erroneous actuation of control meansaffecting the roll angle negatively with respect to the optimalcomfort level will not have any immediate negative effect onsafety, it will have an effect on the flight controller’s abilityto provide optimal comfort level for its passengers. Giventhat the adaptable controller applies its actuators in such amanner that the boundary with respect to roll rotation an-gle is reached, the adaptable controller may be deactivatedand the conventional autopilot or the pilot may take overcontrol.

Applicability: The Trusted Backup Design Pattern is suit-able to apply in the following situations:

• in control scenarios where there is a need for a high de-gree of exploration of alternative controller solutions,e.g. prototyping or upgrade scenarios, as the safetyof the system is not ensured by the adaptable con-troller component and thus the adaptation logic maybe rapidly changed;

• when there is an existing trusted system. Adaptablecontrol functionality may then be provided by addingthe Adaptable Controller, Monitor and the ControlDelegator functionality.

Related Patterns: The Trusted Backup Design Pattern isrelated to the other patterns in the following manner:

satisfies the Requirement Set associated with the StateSpace Subset Requirement Pattern.

enables the use of Adapting Within a ConstrainedState Space Safety Case Pattern as instantiation of thedesign pattern provide a suitable System Specificationwith respect to the main lines of the Safety Demon-stration.

Structure: Figure 4 illustrates the main structure of partsand their interfaces, annotated in UML notation [5], thatplay a part in obtaining a Trusted Backup design.

It assumed that the Requirement Set parameter, in the eventof instantiation, is a Requirement Set containing require-ments specifying the required functionality and constraintswith respect to the given parts of Figure 4. Each named

Figure 4: Structure

part of the Trusted Backup System of Figure 4 may thusbe expected to be associated with a dedicated RequirementSet contained as element of the parameter generically namedRequirement Set (denoted in the upper right part of Figure4).

Collaborations: Delegation of control is governed by theControl Delegator part which based on received/requestedSystem and Plant state information delegate control to oneof its delegates. Given that there are no detected failures toany of its delegates, the Control Delegator delegates controlaccording to a defined Delegation Scheme.

Controller adaptation is initiated in response to the result ofthe detection and analysis of change functionality providedby the Monitor. Once the need to adapt is identified, theAdaptation Logic adapts the Control Logic accordingly. Inorder to assure a safe control process, the Adaptable Con-troller is deactivated from providing control operations inthe time period when it is modified. The Trusted Controllerwill then continuously provide service and thus safety is as-sured.

Participants: The following participants and their respec-tive responsibilities are represented in the Trusted Backuppattern: Plant — represents the process or equipment tobe controlled. The Adaptable Controller is here decom-posed into an Adaptation Logic part and a Control Logicpart. Control Logic — responsible for providing control sig-nals based upon the current plant state and reference datapoints. Adaptation of the Control Logic is here provided by

an external adaptation mechanism (external to the respec-tive controller) identified as Adaptation Logic. AdaptationLogic — is responsible for accommodating changes to theplant or the environment in which the plant operates in or-der to uphold or improve performance. It is here assumedthat the absence of side-effects due to adaptation may notbe adequately evaluated thus a high level of trust may notbe awarded the Adaptable Controller’s Control Logic northe Adaptation Logic. Trusted Controller — responsible forproviding control signals based upon the current plant stateand reference data points. It is assumed that it may bedemonstrated that the Trusted Controller always operate ina correct and sufficiently safe manner. Control Delegator —responsible for delegation of control such that control priv-ileges is granted to the most suitable controller of a set ofalternative controllers in such a manner that plant safetyis always assured and secondly that the most optimal con-troller for a given situation is always sought. All controllersoperate in parallel where a set of delegation rules (capturedin the Delegation Scheme description of the Control Dele-gator) govern which controller is granted control privilegesin each state, the remaining controllers being suppressed.Monitor — responsible for monitoring of the plant, perfor-mance monitoring of the system, and identification of theneed to adapt the adaptable controller to plant and/or en-vironmental changes.

Known Uses: Not currently identified.

Safety Features: An important part with respect to safetyassurance in the pattern is provided by the Control Dele-

gator. It must be assured that it grants control privilegesto the most suitable controller at all times. The patternis suitable to apply when there are no means of revertingto a previously stored plant state given a failure scenario,thus it would be important to support a forward recoverystrategy. Given that an erroneous control signal is givenby the Adaptable Controller, the potential danger this sig-nal inflicts must be mitigated from the current executionpoint. A means for achieving a forward recovery feature isprovided here by redundant controllers, health monitoringand conservative control delegation rules. Potentially haz-ardous control behaviour may arise if the Control Delegatorgrants control privileges to a non-trusted control compo-nent in a plant state where a worst case erroneous controlsequence may not be mitigated in successive control loopsby the Trusted Controller. With respect to providing guar-antee for safe delegation of control, the following must beknown: (1) the potential hazards associated with the plant;(2) worst case failure scenario and the failure modes of theadaptive controller which may lead to hazardous control and(3) the capability of the Trusted Controller to stabilise thePlant.

Adaptability: Software adaptations are intended to be lim-ited to the Adaptable Controller only. The objective of theAdaptable Controller is to optimise control with respect tosome performance goal, how this is achieved is of minuteconcern although a component named Adaptation Logic isassumed to contain the rules for how to adapt the controlleraccording to some identified need to adapt. The identifica-tion of the need to adapt is assumed to be provided by aMonitor component as a result of monitoring and successiveanalysis of the plant and its environment.

Variability: Variability of the system is isolated to theAdaptable Controller part, more specific, the Control Logic.The Control Logic part might exist in various versions overtime as the Adaptation Logic modifies the control function inresponse to plant or environmental events. A specificationof the potential for system variability thus may be providedonce the adaptation logic is specified.

There will be no variability of the system service with re-spect to the region of the operating state space for whichonly the Trusted Controller may operate as the Trusted Con-troller is assumed to be a deterministic controller. Variabil-ity of the system service will be experienced in the region ofthe operating state space for which the Adaptable Controllermay operate and in those scenarios it is granted control priv-ileges as different version of the Control Logic may providedifferent output when acting on the same input.

3.4 Adapting Within a Constrained State Space:Safety Case Pattern

Name: Adapting Within a Constrained State Space

Intent: The intent is to provide an argumentation structurefor how to demonstrate that an adaptable control system issafe when the Adaptable Controller part of the system is:(1) adapted by an unconstrained adaptation mechanism; (2)limited to operate in a part of the total operating state spacefor which there is sufficiently large safety margins to com-pensate for erroneous behaviour; (3) granted control priv-

ileges by a mechanism for delegating control according toa safe delegation scheme; and (4) backed up by a TrustedController which is guaranteed to provide safe behaviour inthe total operating state space.

Motivation: The motivation is to allow a freedom in howan Adaptable Controller is adapted by not assigning anytrust to this part of the of the system but rely on othermeans in order to assure safety. Whilst the Adaptable Con-troller is assumed to provide an increase in performance ina restricted part of operation domain, safety is assured bya Trusted Controller, a Control Delegator and a Monitormechanism.

Applicability: The Adapting Within a Constrained StateSpace pattern is suitable to apply in combination with sys-tems that:

• makes use of monitoring as a means to detect plantstates, to identify the need to accommodate changesby adaptations, and to detect component failure;

• makes use of a redundant set of controllers where adapt-able behaviour is applied to a subset of the controllers;

• is associated with an operational state space that maydivided such that adaptable control may be limited toa safe subset of this state space.

Related Patterns: Adapting Within a Constrained StateSpace safety case pattern is related to other patterns in thefollowing manner:

enables safety demonstration of system designs basedon the Trusted Backup Design Pattern

Safe Adaptation: The main claims which is argued by thissafety case pattern with respect to safe adaptation rely onthe assumptions that:

the objective for introducing adaptiveness is related toa demand for increasing performance, thus the maingoal of the argument is to demonstrate that there areno negative effects of the adaptive feature;

the adaptive controller operate in a constrained partof a larger operational state space where there existssafety margins such that if the adaptive controller failsto operate correctly, the potential negative effects onoperations may be mitigated by a backup controller.

Argument Structure: Figure 5 illustrates the top levelargument specified in an adapted version of the GSN [4]notation in order to express SACS patterns and providesreferences to a set of diagrams which jointly represent a de-composition of the argument. Due to space restrictions, onlythe part of the decomposition related to diversification of theOperating Region, represented by Figure 6, is provided.

It assumed that the System Specification parameter, con-tains and sufficiently specify information that may be asso-ciated with the contextual elements of Figure 5 and Figure

Figure 5: Main Safety Argument

6. Each named contextual element in the safety case dia-grams (e.g. C1 - System, C2 - Operating Region and C3 -Acceptably Safe (System level) of Figure 5) is thus expectedto be described by or referred from System Specification.

4. USE OF THE LANGUAGEThe use of the SACS pattern language will be explained withrespect to how one may apply the Trusted Backup Compos-ite Pattern, described in Section 3, on a railway applicationintroduced in Section 4.1.

4.1 Railway Interlock System ExampleA railway interlock system controls a set of signal apparatusand appliances in order to prevent conflicting train move-ments. A typical interlock system will in the situation illus-trated in Figure 7, detect that a train occupy a section of thetrack and signal proceed if the destined track section at thetrain station is free and that all other conditions associatedwith this train movement are satisfied.

Given that the correct position of a train may be obtained,as opposed to the more rough track section detection, it isassumed that a more optimal traffic flow is obtained if theproceed signal is given as late as possible. In this scenariothe signal will instead of indicating proceed as soon as theconditions for this are satisfied, continue to signal stop (stopis the default signal) until a point where the train should ei-ther: (1) slow down in order to stop before a designated stop

point; or (2) proceed and thus avoid a slow down and lossof kinetic energy. The difference between the points in timewhen the train could at its earliest be given proceed, and thepoint in time when a signal is given according to a optimallylate signalling strategy, may be allocated for other interlockoperations. The system will then not reserve the designatedtrain station track section for the train until it is absolutelynecessary, thus the very same piece of the track is availablefor other operations. If we assume that the interlock systemhas planned many operations at the train station at approx-imately the same time, a smart system is able to optimiseexecution of planned train movements at the station whileour train is approaching and thus an opportunity to exploitavailable capacity.

4.2 Applying The Composite PatternWe will detail how the composite pattern illustrated in Fig-ure 2 may be applied with respect to the railway interlock ex-ample by elaborating upon the application of its constituentpatterns.

The Requirement Pattern named State Space Subset de-tailed in Section 3.2 may be instantiated as follows withrespect to the interlocking example.

We may associate the problem domains specified by Fig-ure 3 such that the Plant represents the signalling appli-ances. The system functionality which we want to derive

Figure 6: Diversification of Operating Domain Argument

requirements for is identified by the decomposed machine il-lustrated as the dotted machine element which encapsulatesthe designed domains Adaptable Controller, Trusted Con-troller, etc. In the railway scenario, problem domains suchas the Adaptable Controller may represent an adaptive in-terlock system and the Trusted Controller may represent astandard interlock system. Although all these problem do-mains may be parts of a larger system we are developing, thefunctionality which they represent are explicitly identified asproblem domains. The problem of specifying the OperatingRegions is in the railway context represented by the prob-lem of diversifying the set of states for which the interlocksystem shall provide operations. If we assume one Adapt-able Interlock Controller (or system) and one Trusted In-terlock Controller, the Trusted Interlock Controller may begranted control privileges in any operational state for whichthe total system shall provide interlock operations, whereasthe Adaptable Interlock Controller may be granted control

privileges only when a train is approaching the train stationand when the conditions for giving “proceed” are satisfied.

The main challenge when applying the pattern is to assure asafe strategy for delegation of control by specifying operatingregions and constraints in accordance with the capability formonitoring the current state and the capability of the differ-ent controllers to enforce safe control within their dedicatedoperating regions. The pattern assumes that a minimumlevel of performance and safe operations are provided by aTrusted Interlock Controller. In our scenario this may be byemploying a track detection signalling scheme such that ifthe conditions for giving a proceed signal are satisfied, thisis given once the train is detected on the approaching tracksection. An Adaptive Interlock Controller might use sophis-ticated algorithms in order to optimise train traffic controlwithout negatively affecting safety as:

Figure 7: Railway Interlocking Adaptive Control Example

• the Adaptable Interlock Controller provide an optimi-sation effect (yields the possibility to increase the num-ber of train station movements) in this limited part ofthe operational state space;

• default signal is “stop” which is a safety preserving sig-nal as the train must stop if “proceed” is not given. De-laying the “proceed” signal then enforce an even moreconservative signalling regime with respect to safetythan the Trusted Interlock Controller;

• given that the Adaptable Interlock Controller providesa non-optimal estimate, in the sense signals “proceed”earlier than optimal, the effect is less time to optimisetrain station movements. There are no negative safetyeffects, only negative effect on optimisation;

• given that the Adaptable Interlock Controller providesa non-optimal estimate, in the sense signals “proceed”later than optimal, then there might be negative ef-fects. The Monitor part of the System should detectthat the train has moved to a point where emergencybreaking is the remaining option. In such a scenario,conditions for delegation of control should be specifiedsuch that the Trusted Interlock Controller is grantedcontrol privileges. Any Adaptable Interlock Controllerissued signal is suppressed as the train moves beyonda point where the train must be stopped immediatelyif the conditions for approaching the destined tracksection are changed (e.g destined track is occupied).

If we assume that all the states that are intended to be con-trolled by the Interlock System represents one OperatingRegion and that this region is governed by the Trusted In-terlock Controller, then the problem of specifying regions ismainly to specify the Adaptive Interlock Controller Operat-ing Region. The premises for granting control privileges tothe Adaptive Interlock Controller may be when:

• a train is detected on the approaching track section;

• conditions for giving the proceed signal is satisfied (thiscould be verified by reading the Trusted Interlock Con-troller signal, which should be “proceed” but which isin this control scenario is suppressed);

• the train has not moved beyond a point on the trackwhere emergency braking is the only option in orderto stop the train before a designated stop point.

In our example the objective of the Adaptable Interlock Con-troller is twofold as the intention is to give proceed signalas late as possible in order to: (i) increase time for per-forming other train station interlock movements; and (ii)signal sufficiently early in order to avoid a slow down ofthe train by breaking and thus preserve kinetic energy andprovide energy efficient control. The optimal point on thetrack to give a “proceed” signal is expected to be at suffi-cient distance to the point where immediate breaking mustbe considered. Thus the unwanted effect that the AdaptableInterlock Controller pushes the optimal point towards thisboundary, providing control operations where trains may ex-perience emergency braking, has a sufficient low probability.Although the emergency breaking preserves the safety of thetrain and the infrastructure, it is an unwanted use of controlmeans intended for emergency use only.

The Design Pattern named Trusted Backup detailed inSection 3.3 may be instantiated as follows with respect tothe interlock example.

If we assume that the Trusted controller illustrated in Figure4 can be considered sufficiently safe for the intended controlpurpose (e.g. a formally verified conventional programmedlogic based Trusted Interlock Controller), then safe controloperations should be guaranteed by this part alone. TheMonitor part provides, as in any current interlock system,information on the state of the different appliances involvedin the control scenario to the controller. The increased sys-tem complexity, compared to a conventional control systems,is provided by the Control Delegator part which is respon-sible for delegation of control, and the Adaptable InterlockController.

In order to assure safe delegation of control, the ControlDelegator might be implemented by some programmed logicas a set of formally verifiable rules according to the princi-ples outlined with respect to application of the requirementpattern. As we argued that the outlined delegation schemewould be safety preserving, we will focus on the functionalityfor optimising the signalling behaviour.

One alternative implementation of the Control Logic partof the Adaptive Interlock Controller may be by a controllaw represented as a mathematical formula that considersseveral parameters (e.g. train type, load, speed) in orderto estimate an optimal late signalling point. The formulamay be obtained by the Adaptation Logic part by means of

combining statistics over past events (e.g. effect of differentweather conditions on breaking time), data on the differentbreaking patterns of trains under different load conditions,etc. in order to update the control law as more data is ob-tained or better algorithms are developed. The complexityof the algorithms used to obtain the control law or the com-plexity of the control law does not provide an assessmentproblem, the result is still a controller which issues either a“proceed” or a “stop” signal. The scenarios in which thesecommands may be issued are handled with sufficient deter-minism by the Control Delegator part, thus the AdaptableInterlock Controller may be of arbitrary assurance level. Itis of minute concern how often the control law is updated assafe control is sustained by the Trusted Interlock Controllerduring the update.

The Safety Case Pattern named Adapting Within a Con-strained State Space detailed in Section 3.4 may be instan-tiated as follows with respect to the interlock example.

The intent of the Safety Case Pattern is to provide a genericargument structure in order to outline how claims, argu-ments and evidences may be structured systematically inorder to be able to demonstrate that a particular type ofsystem is sufficiently safe. In the context of our example asprovided in Section 4.1, this means to demonstrate that theoverall interlock system is safe.

When instantiating the pattern, the context entities as rep-resented by e.g. C1, C2 and C3 of Figure 5 (context enti-ties in GSN notation) must be given a real meaning. Withrespect to our railway example, the System may be repre-sented by the overall interlock system. The Operating Re-gion may be represented by a specification of the railwayinfrastructure and the environment and the states for whichthe interlock system shall operate. What is Acceptably Safeat the system level may be provided by applicable laws, reg-ulations and standards.

The goals G1 to G5 are detailed in separate diagrams (notprovided here due to space considerations), and if fulfilledsupport the overall claim G0 that the system is acceptablysafe. The justification for the completeness of G0 by thegoals G1 to G5 should be specified in J-Sub-G0. An outlineof a possible instantiation of the pattern with respect to themain goals may be as follows:

G1 The Operating Domain represents the set of all statesfor which interlock operations should be given. We ar-gued earlier that the operational state space for whichthe adaptable controller may be allowed to operatiecould be specified with a small set of rules. The G1goal would in our context be concerned with demon-strating that the set of rules provides an adequate par-titioning of the state space.

G2 The goal is concerned with demonstrating that a partof the system correctly identifies or detects the opera-tional state at any given time.

G3 The goal is concerned with demonstrating the correctenforcement of the rules for delegation of control. Thismeans to demonstrate that the adaptive controller is

only delegated control privileges in the scenarios de-veloped in G1 and that failure events which may in-validate the design intent are detected and handled orotherwise are absent or acceptable.

G4 The goal is concerned with demonstrating that anyaction performed by the Adaptable Controller is ac-ceptably safe. We stated that obtaining a non-optimalcontrol law might give the proceed signal earlier thanintended, later than intended or otherwise at a pointwhere the train must employ emergency break down ifproceed is not given. All these scenarios do not affectsafety but rather performance if the mitigations pro-posed may be implemented. Potential failure scenariosarising in the adaptable controller may be acceptableif proper mitigation is provided by other componentssuch that the failures are not allowed to propagate tothe system level.

G5 The goal is concerned with demonstrating that theTrusted Interlock Controller, which is represented bya conventional interlock controller, is sufficiently safeand may mitigate Adaptable Interlock Controller fail-ure. In our context this means to emphasise the pointthat if the Adaptable Interlock Controller erroneouslylate (with respect to optimisation) delay giving a “pro-ceed”, then “stop” is continuously signalled. This ishardly hazardous but more conservative with respectto safety. If the inability of control is meant send-ing erroneously early (with respect to optimisation)a “proceed” signal then this is in harmony with theTrusted Interlock Controller. If it is meant sending an-other signal than “proceed” or “stop” then this shouldbe easily detected by the monitor. If there are somecircumstance which alter the conditions such that a“stop” signal should be given, then the premises forgranting control privileges to the Adaptable InterlockController are not satisfied.

5. CONCLUSIONThe patterns are intended to be used in the early stagesof design, typically by system architects, software designersand safety engineers, as a means to efficiently outline the re-quirements applicable to an adaptable system, a conceptualmodel of a system as well as outlining the safety demonstra-tion challenges.

The SACS pattern language facilitates a complementary view-point to the problem of deriving safe adaptive software. Thecomplementary viewpoint is given by the possibility to in-tegrate a set of patterns, from the set of different patterntypes, into a composition such that the requirements thatmay be associated with a particular problem, design so-lutions for the problem and safety demonstration aspectsmay be conveniently explored. The SACS approach pro-vides a more nuanced means for evaluating the suitabilityof a particular design than provided by the GoF [2] styleof expressing patterns. The difference is that SACS doesnot only emphasise design pattern solutions, detailed guid-ance are also provided on the issues of capturing relevantrequirements and safety demonstration. When developingsafety critical or safety related systems, the increase in costcompared to systems not important to safety is to a large

degree related to the need to demonstrate that these sys-tems are adequately safe. By coupling patterns dedicatedto requirements elicitation, design specification and safetydemonstration, a nuanced pattern approach is provided.

Future work includes expanding the set of basic patternsand detailing the syntax of the pattern language. The in-tention is to expand the set of patterns such that at least thealternative strategies provided in the introduction (those in-troduced but not described in this article) are covered. Oneof these alternative strategies was to constrain the adapta-tion mechanism and the other alternative was to incorporatea runtime evaluation mechanism which constrains the com-mitting of modifications to only safe modifications. Futurework also includes evaluation of the SACS language throughcase studies and other suitable means in order to provide theability to conclude on SACS providing a utility or not forits intended users.

AcknowledgmentThis work has been conducted and funded within the OECDHalden Reactor Project, a nuclear research project managedby Institute for energy technology (IFE), Halden, Norway.

6. REFERENCES[1] C. Alexander, S. Ishikawa, and M. Silverstein. A

Pattern Language: Towns, Buildings, Construction.Oxford University Press, 1977.

[2] E. Gamma, R. Helm, R. E. Johnson, and J. Vlissides.Design Patterns: Elements of Reusable Object-OrientedSoftware. Addison-Wesley, 1995.

[3] M. Jackson. Problem Frames: Analysing andStructuring Software Development Problems.Addison-Wesley, 2001.

[4] T. Kelly and R. Weaver. The Goal StructuringNotation - A Safety Argument Notation. In Proc. ofDependable Systems and Networks 2004 Workshop onAssurance Cases, 2004.

[5] Object Management Group (OMG). Unified modelinglanguage specification, 2010. Version 2.3.

[6] J. Schumann and Y. E. Liu. Applications of NeuralNetworks in High Assurance Systems. Springer, 2010.