sok: attacks on industrial control logic and formal ... · dustrial control systems. a vulnerable...
TRANSCRIPT
SoK: Attacks on Industrial Control Logic and Formal Verification-BasedDefenses
Ruimin SunNortheastern [email protected]
Alejandro MeraNortheastern [email protected]
Long LuNortheastern University
David ChoffnesNortheastern University
Abstract—Programmable Logic Controllers (PLCs) play acritical role in the industrial control systems. Vulnerabilitiesin PLC programs might lead to attacks causing devastatingconsequences to the critical infrastructure, as shown inStuxnet and similar attacks. In recent years, we have seenan exponential increase in vulnerabilities reported for PLCcontrol logic. Looking back on past research, we foundextensive studies explored control logic modification attacks,as well as formal verification-based security solutions.
We performed systematization on these studies, andfound attacks that can compromise a full chain of controland evade detection. However, the majority of the formalverification research investigated ad-hoc techniques targetingPLC programs. We discovered challenges in every aspect offormal verification, including (1) the ever-expanding attacksurface from evolved system design, (2) the real-time con-straint during the program execution, and (3) the barrierin security evaluation given proprietary and vendor-specificdependencies on different techniques. Based on the knowl-edge systematization, we provide a set of recommendationsfor future research directions.
Index Terms—PLC, attack, formal verification
1. Introduction
Industrial control systems (ICS) are subject to attackssabotaging the physical processes, as shown in Stuxnet[34], Havex [47], TRITON [33], Black Energy [10], andthe German Steel Mill [60]. PLCs are the last line incontrolling and defending for these critical ICS systems.
However, in our analysis of Common Vulnerabilitiesand Exposures (CVE)s related to control logic, we haveseen a fast growth of vulnerabilities in recent years (Figure2). These vulnerabilities are distributed across vendors anddomains, and their severeness remains high (Figures 4 and5 in the Appendix show the details). A closer look at thesevulnerabilities reveals that the weaknesses behind themare not novel. As Figure 1 shows, multiple weaknessesare repeating across different industrial domains, such asstack-based buffer overflow and improper input validation.We want to understand how these weaknesses have beenused in different attacks, and how existing solutions de-fend against the attacks.
Among various attacks, control logic modification at-tacks cause the most critical damages. Such attacks lever-age the flaws in the PLC program to produce undesiredstates. As a principled approach detecting flaws in pro-grams, formal verification has long been used to defend
Critica
l Man
ufactu
ring
Energy
Was
tewate
r Sys
tems
Food a
nd A
gricu
lture
Commerc
ial Fac
ilities
Chemica
l
Transp
ortati
on
Health
care
Govern
ment F
acilit
ies
Inform
ation
Techn
ology
Unrestricted Upload Dangerous Type File CWE-434Uncontrolled Resource Consumption CWE-400
Improper Authentication CWE-287Hard-coded Credentials CWE-798
Cross-site Scripting CWE-79Out-of-bounds Write CWE-787
Improper Access Control CWE-284Out-of-bounds Read CWE-125
Stack-based Buffer Overflow CWE-121Improper Input Validation CWE-20
3 3 1 1 4 1 4 2 2 28 3 4 1 1 2 1 14 5 3 2 2 2 2 1 1 26 4 3 3 3 3 5 1 1 18 5 4 3 3 3 3 1 1 16 7 6 3 4 3 35 7 7 6 1 5 3 26 8 7 5 5 4 417 15 15 4 3 4 312 19 11 13 2 13 5 3
0
5
10
15
Figure 1: The reported common weaknesses and the af-fected industrial sectors. The notation denotes the numberof CVEs.
control logic modification attacks [26], [27]. It benefitsfrom several advantages that other security solutions failto provide. First, PLCs have to strictly meet the real-time constraints in controlling the physical processes. Thismakes it impractical for heavyweight solutions performinga large amount of dynamic analysis. Second, the physicalprocesses are often safety-critical, meaning false positivesare intolerable. Formal verification is lightweight, accu-rate, and suitable for graphical languages, as many PLCprograms are developed with.
Over the years, there have been extensive studies in-vestigating control logic modification attacks, and formalverification based defenses. To understand the currentresearch progress in these areas, and to identify openproblems for future research directions, we performed asystematization of current studies.
Scope of the paper. We considered studies present-ing control logic modification attacks through modifyingthe source code or the binary, or feeding special inputdata to trigger existed design flaws. We also consideredstudies presenting formal verification techniques for PLCprograms, including behavior modeling, state reduction,specification generation, and verification. We selected theliterature based on three criteria: (1) the study investigatescontrol logic modification attacks or formal verificationbased defenses, (2) the study is impactful considering itsnumber of citations, or (3) the study discovers a newdirection for future research.
Systematization methodology. Our systematizationwas based on the following aspects.
• Threat model assumptions: this refers to the require-ments needed to perform the attack/defense.
• Attack goal: this refers to the goal to achieve byattacks and to prevent by defenses.
arX
iv:2
006.
0480
6v2
[cs
.CR
] 1
1 N
ov 2
020
• weakness: this refers to the flaw that triggers theattacks.
• Detection to evade: this refers to the detection thatfails to capture the attacks.
• Challenges: this refers to the challenges in defendingsuch attacks.
• Formal verification aspect: this refers to the researchtopic in formal verification.
We found that control logic modification attacks couldhappen under every threat model and considered variousevasive techniques. The attacks have been fast evolvingwith the system design, through input channels from thesensors, the engineering stations, and other connectedPLCs. The attacks could also evade dynamic state es-timations and verification techniques through leveragingimplicitly specified properties. Multiple attacks [51], [61],[80] even deceived engineering stations with fake behav-iors.
We also found that applying formal verification hasmade great progress in improving code quality [93].However, the majority of the studies investigated ad-hoc formal verification research targeting PLC programs.These studies face challenges in many aspects of formalverification, during program modeling, state reduction,specification generation, and verification. We found manystudies manually define domain-specific safety properties,and verify them based on a few simple test cases. Despitethe limitation of test cases, the implicitness of propertieswere not well explored, even though such properties havebeen used to conduct input manipulation attacks [65]–[67]. Besides implicit properties, specification generationhas seen challenges in catching up with program model-ing, to support semantics and rules from new attack sur-faces. In addition, the real-time constraint limited runtimeverification in supporting temporal features, event-drivenfeatures, and multitasks. The dependency on proprietaryand vendor-specific techniques resulted in ad-hoc studies.The lack of open source toolings impeded thorough eval-uation across models, frameworks, and real programs inindustry complexity.
As a call for solutions to address these challenges,we provide a set of recommendations for future researchdirections. We recommend future research to pay attentionto plant modeling and to defend against input manipu-lation attacks. We recommend the collaboration betweenstate reduction and stealthy attack detection. We highlightthe need for an automate domain-specific specificationgeneration and incremental specification generation. Wealso encourage more exploration in real-time verification,together with more support in open source tooling, andthorough performance and security evaluation.
Our study makes the following contributions:• Systematization of control logic modification attacks
and formal verification-based defenses in the lastthirty years.
• Identifying the challenges in defending control logicmodification attacks, and barriers existed in currentformal verification research.
• Pointing out future research directions.The rest of the paper is organized as follows. Section 2
briefly describes the background knowledge of PLCs andformal verification. Section 3 and Section 4 systematize
2008 2010 2012 2014 2016 2018 2020Year
0
25
50
75
100
125
150
175
Num
ber o
f CV
Es
Figure 2: Yearly reported control logic related CVEs.
Figure 3: The architecture of a PLC.
existing studies on control logic modification attacks, andformal verification based defenses— categorized on threatmodels, and attack/defense methodologies. Section 5 pro-vides recommendations for future research directions tocounter existing challenges. Section 6 concludes the paper.
2. Background
2.1. PLC program
2.1.1. Programming Languages. IEC-61131 [83] de-fined five types of languages for PLC programs:
• Ladder diagram (LD),• Structured text (ST),• Function block diagram (FBD),• Sequential function chart (SFC),• Instruction list (IL).Among them, LD, FBD, and SFC are graph-based
languages. IL was deprecated in 2013. PLC programsare developed in engineering stations, which providestandard-compliant or vendor-specific Integrated Devel-opment Environments (IDEs) and compilers. Some high-end PLCs also support computer-compatible languages(e.g., C, BASIC, and assembly), special high-level lan-guages (e.g., Siemens GRAPH5 [5]), and boolean logiclanguages [64].
2.1.2. Program binary. Unlike conventional softwarethat follows well documented formats, such as Executableand Linkable Format (ELF) for Linux and Portable Exe-cutable (PE) for Windows, the format of PLC binaries isoften proprietary and unknown. Therefore, further explo-ration requires reverse engineering.
2.1.3. Scan cycle. Unlike conventional software, a PLCprogram executes by infinitely repeating a scan cycle thatconsists of three steps (as Figure 3 shows). First, theinput scan reads the inputs from the connected sensorsand saves them to a data table. Then, the logic executionfeeds the input data to the program and executes the logic.
2
Finally, the output scan produces the output to the physicalprocesses based on the execution result.
The scan cycle has to strictly follow a predefinedmaximum cycle time to enforce the real-time execution.The I/O operations are the critical part in meeting thecycle time.
2.1.4. Hardware support. PLCs adopt a hierarchicalmemory, with predefined addressing scheme associatedwith physical hardware locations. PLC vendors maychoose different schemes for I/O addressing, memoryorganization, and instruction sets, making it hard for thesame code to be compatible across vendors, or evenmodels within the same vendor.
2.2. PLC program security
PLCs interact with a broad set of components. Theyare connected to sensors and actuators to interact withand control the physical world. They are connected tosupervisory human interfaces (e.g. the engineering station)to update the program and receive operator commands.They may also be connected to each other in a subset.These interactions expose PLCs to various attacks. Forexample, communication between the engineering stationand the PLC may be insecure, the sensors might becompromised, and the PLC firmware can be vulnerable.
2.2.1. Control logic modification. Our study considerscontrol logic modification attacks, which we define as at-tacks performing program payload modification and inputmanipulation. The payload modification can be applied toprogram source code or binary. The input manipulationcan craft input data to trigger existed design flaws in theprogram to produce undesired states. The input may comefrom any interacting components mentioned earlier.
Control logic modification attacks can lead to manyundesired states, including (1) potentially unsafe behaviorsto the physical processes, (2) stealthy and monitor secreteinformation, or (3) spread to cause denial-of-service at-tacks.
Defending against these attacks is challenging. As wementioned earlier, PLCs have to strictly maintain the scancycle time to control the physical world in real-time.This rules out heavyweight security solutions perform-ing a large amount of dynamic analysis. Moreover, thesecurity solution has to be accurate, since the controlledphysical processes are critical in the industry, making falsepositives less tolerable.
2.2.2. Formal verification. Formal verification is alightweight and accurate defense solution, which is oftentailored for graphical languages. This makes it suitable todefend control logic modification attacks.
Formal verification is a method proves or disprovesif a program/algorithm meets its specifications or desiredproperties based a on certain form of logic [1]. Thespecification may contain security requirements and safetyrequirements. Commonly used mathematical models to doformal verification include finite state machines, labeledtransition systems, vector addition systems, Petri nets,timed automata, hybrid automata, process algebra, and for-mal semantics of programming languages, e.g. operational
semantics, denotational semantics, axiomatic semantics,and Hoare logic. In general, there are two types of for-malization: model checking and theorem proving [46].Model checking uses temporal logic to describe specifica-tions, and efficient search methods to check whether thespecifications hold for a given system. Theorem provingdescribes the system with a series of logical formulae. Itproves the formulae imply the property via deduction withinference rules provided by the logical system. It usuallyrequires more background knowledge and nontrivial man-ual efforts.
Table 2 details a list of commonly used frameworksand tools for formal verification.
3. Systematization of Attacks
This section systematizes PLC attack methodologiesunder the categorization of threat models. Within eachcategory, we discuss the goals of the attacks and theunderlying weaknesses. We summarize the challenges ofattack mitigation in the end.
3.1. Threat Model Assumptions
Existing studies assumes the following requirementsin performing attacks or defenses.
• T1: The source code of the program [65], [67], [80].• T2: The PLC hardware setting and the program bi-
nary, e.g. intercepted during transmission to the PLC,or extracted from the PLC.
• T3: The program, the PLC hardware, and the runtimeI/O traces.
These assumptions are practical since (1) insiders withina company can compromise PLC source code to injectstealthy and malicious code, (2) the communications be-tween the PLCs and other components are insufficientlyprotected, and (3) within the same domain, the generalsettings of the infrastructure layout are similar, and someinfrastructures are publicly accessible [6], [44], [66].
3.2. Weaknesses
We analyzed PLC program vulnerabilities to under-stand the weaknesses behind them, and we obtained re-sults across industries. As Figure 1 shows, although thevulnerabilities are distributed evenly across domains, themost common weaknesses are focused on improper inputvalidation, stack-based buffer overflow, and out-of-boundsread and write. Looking deeper into these weaknesses, wefound the following weak points rooted in the program andits execution.
• W1: Duplicated assignment for output variables. Thiscould cause race conditions that are hard to be no-ticed.
• W2: Uninitialized or unused variables. Reusing thesevariables by attackers will be stealthy.
• W3: Hidden jumpers bypassing a portion of theprogram. This could trigger malware by condition.
• W4: Improper runtime input. This could lead tounsafe behaviors.
• W5: Predefined memory layout of the PLC hardware.This could be leveraged by attackers during runtimeexecution.
3
• W6: Real-time constraints. This could cause syn-chronization issues in event-driven, multitasking pro-grams.
3.3. Attack Goals
Existing attacks aim at the following goals.• G1: Causing unsafe behaviors, e.g. maximizing dam-
ages to the physical processes [13], [55], [82].• G2: Being stealthy while monitoring PLC behaviors.• G3: Spreading as a worm, or causing denial of ser-
vice attacks.
3.4. Attack Methodologies
Given the exposed threat models, the following de-scribes the attack methodologies of existing studies inachieving one of the attack goals. Figure 1 summarizesthese studies.
3.4.1. T1. At the source code level, the code injectionor modification have to be stealthy, in a way that noobservable changes would be introduced to the majorfunctionality of the program, but masked as novice pro-grammer mistakes. In other words, the attacks could bedisguised as unintentional bad coding practices.
Existing studies [81], [84] mainly discussed attacks ongraphical languages, e.g. LD, since small changes on suchprograms could not be easily noticed.
Serhane et.al [81] focused on the weak points on LDprograms that could be exploited by malicious attacks.Targeting G1 to cause unsafe behaviors, attackers couldgenerate uncertainly fluctuating output variables, for ex-ample, intentionally introduce two timers to control thesame output variable, so that race condition may happen.This could damage devices, similar with Stuxnet [34],but unpredictably. Attackers could also bypass certainfunctions, attackers could manually force the values ofcertain operands, or apply empty branches or jumps.
Targeting G2 to stay stealthy while spying the pro-gram, attackers could use array instructions or user-defined instructions, to log critical parameters and values.Targeting G3 to generate DoS attacks, attackers couldapply an infinite loop via jumps, and use nest timers andjumps to only trigger the attack at a certain time. Thisattack could slow down or crash the PLC in a severematter.
Because PLC programmers often leave unused vari-ables and operators, both the spying program and the DoSprogram could leverage unused variables or operands. Us-ing them will be stealthy as they were already predefined.
These attacks leveraged weaknesses W1-W4, and fo-cused on single ladder program. To extend the attacks tomulti-ladder programs, Valentine et.al [84] further pre-sented attacks that could install a jump to subroutinecommand, and modify the interaction between two ormore ladders in a program. This could be disguised asan erroneous use of scope and linkage for a novice pro-grammer.
In addition to code injection and modification,McLaughlin et.al [66] presented input manipulation at-tacks to cause unsafe behaviors. This study analyzed thecode to obtain the relationship between the input and
output variables, and deducted the desired range for outputvariables. Attackers could craft inputs that could lead toundesired outputs for the program. The crafted inputs haveto evade the state estimation detection of the PLC. Sincethe input manipulation happens in T3, and more studiesdiscussed input manipulation attacks without using sourcecode, we will elaborate on these attacks in T3.
3.4.2. T2. Studies at this threat model mainly investigatedprogram reverse engineering, and program modificationattacks. Instead of disguising as bad coding practices, likethose in T1, the injection at the program binary aimed atevading behavior detectors.
To design an attack prototype, McLaughlin et.al [67]tested on a train interlocking program. The program wasreverse engineered using a format library. With the decom-piled program, they extracted the fieldbus ID that indicatedthe PLC vendor and model, and then obtained cluesabout the process structure and operations. To generateunsafe behaviors, such as, causing conflict states for thetrain signals, they targeted timing-sensitive signals andswitches. To evade safety property detection, they adoptedan existed solution [35] to find the implicit properties ofthe behavior detectors. For example, variable r depends onp and q, so a property may define the relationship betweenp and q, as a method to protect r. However, attackers candirectly change the value of r without affecting p and q,and the change will not alarm the detector. In this way,they automatically searched for all the Boolean equations,and could generate malicious payloads based on that.
Based on this prototype, SABOT [65] was imple-mented. SABOT required high-level description of thephysical process, such as, “the plant contains two ingredi-ent valves and one drain valve”. Such information couldbe acquired from public channels, and are similar forprocesses in the same industrial sector. With this informa-tion, SABOT generated a behavioral specification for thephysical processes and used incremental model checkingto search for a mapping between a variable within theprogram, and a specified physical process. Using this map,SABOT compiled a dynamic payload customized for thephysical process.
Both studies were limited to Siemens devices, with-out revealing many details on reverse engineering. Toprovide more information, and to support CodeSys-basedprograms, Keliris et.al [52] implemented an open-sourcedecompiler, ICSREF, which could automatically reverseengineer CodeSys-based programs, and generate mali-cious payloads. ICSREF targeted PID controller functionsand manipulated the parameters such as the setpoints,proportional/integral/derivative gains, initial values, etc.ICSREF could inference the physical characteristics of thecontrolled process, so that modified binaries could deploymeaningful and impactful attacks.
3.4.3. T3. At this level, existing studies investigated twotypes of attacks: the program modification attack, andthe program input manipulation attack. The input of theprogram could either come from the communication be-tween the PLC and the engineering station, or the sensorreadings.
• Program modification attack. this requires reverseengineering and payload injection, similar with studies
4
TABLE 1: The studies investigating control logic modification attacks.
Threat Model Weakness Paper Attack Goal Detection to EvadeG1 G2 G3 Programmer State Verification Engineering Station Others
T1W1-W3 Serhane’18 [81] X X XW1-W3, W6 Valentine’13 [84] X X XW4 McLaughlin’11 [67] X X
T2W4 ICSREF [52] XW4 SABOT [65] XW4 McLaughlin’11 [67] X X
T3
W5 PLCInject [55] XW5 PLC-Blaster [82] X XW4 Senthivel’18 [80] X XW4 CLIK [51] X XW4,W5 Beresford’11 [13] XW4,W5 Lim’17 [61] X X XW4 Xiao’16 [88] X XW4 Abbasi’16 X XW5 Yoo’19 [91] X XW4,W6 LLB X XW4 CaFDI [66] X XW4,W5 HARVEY X X X
in T2. The difference is that, given the PLC memorylayout available, and the features supported by the PLC,the design of payload becomes more targeted. Throughinjecting malicious payload to the code, PLCInject [55]and PLC-Blaster [82] presented the widespread impactof the malicious payload. PLCInject crafted a payloadwith a scanner and proxy. Due to the predefined memorylayout of Siemens Simatic PLCs, PLCInject injected thispayload at the first organization block (OB) to change theinitialization of the system. This attack turned the PLCinto a gateway of the network of PLCs. Using PLCIn-ject, Spenneberg [82] implemented a worm, PLC-Blaster,that can spread among the PLCs. PLC-Blaster spread byreplicating itself and modifying the target PLCs to executeit along with the already installed programs. PLC-Blasteradopted several anti-detection mechanisms, such as avoid-ing the anti-replay byte, storing at a less used block, andmeeting the scan cycle limit. PLCInject and PLC-Blasterachieved G3 and demonstrated the widespread impact ofprogram injection attacks.
In addition to that, Senthivel et.al [80] introducedseveral malicious payloads that could deceive the engi-neering station. Since the engineering station periodicallychecked the running program from the PLC, the attackerscould deceive it by providing an uninfected program whilekeep executing the infected program in the PLC. Sen-thivel achieved this through a self-developed decompiler(laddis) for LD programs. Senthivel also introduced threestrategies to achieve this denial of engineering operationattacks.
In a similar setting, Kalle et.al [51] presented CLIK.After payload injection, CLIK implemented a virtual PLC,which simulated the network traffic of the uninfected PLC,and fed this traffic to the engineering station to deceivethe operators. These two works employed a full chain ofvulnerabilities at the network level, without accessing theengineering station nor the PLCs.
• Input manipulation through the network. severalstudies [13], [61] hijacked certain network packets be-tween the engineering station and the PLC. Beresfordet.al [13] exploited a packet (e.g. ISO-TSAP) between thePLC and the engineering station. These packets providedprogram information, such as variable names, data block
names, and also the PLC vendor and model. Attackerscould modify these variables to cause undesired behavior.With memory probing techniques, attackers could get amapping between these names to the variables in the PLC.This would allow them to modify the program based onneeds. This attack could cause damages to the physicalprocesses. However, the chance for successful mappingof the variables through memory probing is small. In anuclear power plant setting, Lim et.al [61] interceptedand modified the command-37 packets sent between theengineering station and the PLC. This packet providedinput to an industrial-grade PLC consisted of redundantmodules for recovery. The attack caused common-modefailures for all the modules.
These attacks made the entry point through the net-work traffic. However, they ignored the fact that securitysolutions could have enabled deep packet inspection (DPI)between the PLC and the engineering station. Modifiedpackets with malicious code or input data could have beendetected before reaching the PLC. To evade DPI, Yoo et.al[90] presented stealthy malware transmission, by splittingthe malware into small fragments and transmitting onebyte per packet with a large size of noises. This was basedon the fact that DPI merges packets for detection and thuswas not able to detect small size payload. On the PLCside, Yoo leveraged a vulnerability to control the receivedmalicious payload, discard the padded noises, and config-ure the start address for execution. Although dependenton multiple vulnerabilities, this study provided insight forstealthy program modification and input manipulation atthe network level.
• Input manipulation through sensor. existing studies[6], [44], [66], [88] explored different approaches to evadevarious behavior detection, and to achieve G1.
Govil et.al [44] presented Ladder logic bombs (LLB),which was a combination of program injection and inputmanipulation attacks. The malicious payload was injectedinto the existing LD program, as a subroutine, and couldbe triggered by a certain condition. Once triggered, thismalware could replace legitimate sensor readings withmanipulated values. LLB was designed to evade manualinspection, by naming instructions similar to commonlyused ones.
5
LLB did not consider behavior detection, such as stateverification, or state estimation. To counter Buchi au-tomaton based state estimation, CaFDI [66] was presentedto introduce controller-aware false data injection attacks.CaFDI required high-level information of the physicalprocesses, and monitored I/O traces of the program. Itfirst constructed a Buchi automaton model of the programbased on its I/O traces, and then searched for a set ofinputs that may cause the model to produce the desiredmalicious behavior. CaFDI calculated the Cartesian prod-uct of the safe model and the unsafe model, and recur-sively searched for a path that could satisfy the unsafemodel in the formalization. The resulting path of inputwould be used as the sensor readings for the program. Tostay stealthy, CaFDI avoided noticeable inputs, such as anLED indicator. Xiao [88] fine tuned the undesired modelto evade existing sequence-based fault detection [54]. Anattacker could first construct a discrete event model fromthe collected fault-free I/O traces using non-deterministicautonomous automation with output (NDAAO), and thenbuild a word set of NDAAO sequences, and finally, searchfor the undetectable false sequences from the word set toinject into the compromised sensors. Similarly, by com-bining the control flow of the program, Abbasi et.al [6]presented configuration manipulation attacks by exploitingcertain pin control operations, leveraging the absence ofhardware interrupts associated to the pins.
To evade the general engineering operations, Garcia[38] developed HARVEY, a PLC rootkit at the firmwarelevel that can evade operators viewing the HMI. HAR-VEY faked sensor input to the control logic to generateadversarial commands, while simulated the legitimate con-trol commands that an operator would expect to see. Inthis way, HARVEY could maximize the damage to thephysical power equipment and cause large-scale failures,without the notice of the operators. HARVEY assumedaccess to the PLC firmware, which was less monitoredcompared with the control logic program.
These studies make it practical to inject malicious pay-loads either through a compromised network or insecuresensor configurations. Together with the stealthiness, itremains challenging to design security solutions to counterthem. The following details the challenges.
3.5. Challenges
• Expanded attack input surfaces. The attack inputsurfaces for PLC programs are expanding. The aforemen-tioned studies have shown input sources including (1) thecommunication from the engineering station, with certainpackets intercepted and hijacked, (2) internet faced PLCsin the same subnet, and (3) compromised sensors andfirmware. It becomes challenging for defense solutions toscope an appropriate threat model, since any componentalong the chain of control could be compromised.
• Predefined hierarchical memory layout. Multiplestudies leveraged this weakness to perform the attacks.However, traditional defense solutions [24] have seenmany changes: (1) the address space layout randomization(ASLR) would be too heavy to meet the scan cycle re-quirements for the PLCs, and would still suffer from code-reuse attacks, (2) the control flow integrity based solutions
require a substantial amount of memory, and would behard to detect in real-time, or to mitigate the attacks, and(3) the hierarchical memory layout is vendor-specific, andthe attacks targeting them are product-driven, for example,Siemens Simatic S7 [13], [82] targeted. It is challengingto design a lightweight and generalized defense solution.
• Confidentiality and integrity of the program I/O.The majority of the studies depended on the program I/Oto perform attacks, either to extract information of thephysical processes, and possible detection methods, or tomanipulate input to produce unsafe behaviors. ProtectingI/O is challenging in that (1) the input surfaces of the pro-grams are expanding, (2) sensors and physical processescould be public infrastructure, and (3) the I/O has to beupdated frequently to meet the scan cycle requirement.
• Stealthy attack detection. We have mentioned manystealthiness strategies based on different threat models, in-cluding (1) disguising malicious code as human errors, (2)code obfuscation with fragmentation and noise padding toevade DPI, (3) crafting input to evade state estimationand verification algorithms, (4) using specific memoryblock or configuration of the PLC, and (5) deceivingthe engineering station with faked legit behaviors. It ischallenging for a defense solution to capture these stealthyattacks.
• Implicit or incomplete specifications. Multiple stud-ies have shown crafted attacks using the implicit properties[65]–[67]. The difficulties of defining precise and com-plete specifications lie in that (1) product requirementsmay change over time thus requiring update of semanticson inputs and outputs, (2) limited expressiveness can leadto incompleteness, while over expressiveness may lead toimplicitness, and (3) domain-specific knowledge is usuallyneeded. It is challenging to maintain specifications toovercome these difficulties.
4. Formal Verification based Defenses
A large body of research has been performed usingformal verification for PLC safety and security, as Table3 shows. This study mainly focused on the followingaspects.
• Behavior Modeling: Modeling the behavior of theprogram as a state-based, flow-oriented, or time-dependent representation.
• State Reduction: Reducing the state space to im-prove search efficiency.
• Specification Generation: Generating the specifi-cation with desired properties as a temporal logicformula.
• Verification: Applying model checking or theoremproving algorithms to verify the security or safety ofthe PLC program.
Based on these aspects, the following discusses de-fense methodologies. We use the same threat models,attack goals, and weaknesses as mentioned in the lastsection §3.
4.1. Behavior Modeling
The goal of behavior modeling is to obtain a for-mal representation of the PLC program behavior, so that
6
given a specification, a formal verification framework canunderstand and verify. The following discusses behaviormodeling, based on different threat models.
4.1.1. T1. At the source code level, a line of studies [7],[32], [42], [73] have investigated the formal modeling ofgeneric program behaviors. The majority of them trans-lated programs to automata and Petri nets, since they werewell supported by most formal verification frameworks[7]. These translations usually consider each program unitas an automaton, including the main program, functions,and function block instances. Each variable defined inthe program unit was translated as a corresponding vari-able in the automaton. Input variables are assigned non-deterministically at the beginning of each PLC cycle. Thewhole program was modeled as a network of automata,where a transition represents the changes of variablevalues in different execution cycles, and a synchronizationpair represents synchronized transitions of function calls.In a similar modeling method, Newell et.al [73] translatedFBD programs to Prototype Verification System (PVS)models, since certain nuclear power generating station canonly support such representation.
These studies could formally model most PLC behav-iors, especially the internal logic within the PLC code.However, with only source code available, behavior mod-eling lacks the interaction with the PLC hardware, and thephysical processes, which might cause unsafe or maliciousbehaviors to bypass later formal verification. The follow-ing discusses behavior modeling with more informationavailable.
4.1.2. T2. Fewer studies have investigated behavior mod-eling at the program binary level. The challenges lie inreverse engineering. As mentioned in existing works [68],[96], several PLC features are not supported in the nor-mal instruction sets. PLCs are designed with hierarchicaladdressing using a dedicated memory area for the inputand output buffers. The function blocks use a parameterlist with fixed entry and exit points. PLCs also supporttimers that act differently between bit-logic instructionsand arithmetic instructions.
Thanks to an open-source library [57], which candissemble Siemense PLC binary programs into STL (ILequivalent for Siemense) programs, several works [23],[68], [89], [96] studied modeling Siemens binary pro-grams. Based on the STL program, TSV [68] leveragedan intermediate language, ILIL, to allow more completeinstruction modeling. With concolic execution, TSV ob-tained the information flow from the system registersand the memory. After executing multiple scan cycles,a temporal execution graph was constructed to representthe states of the controller code. After TSV, Zonouz et.al[96] adopted the same modeling. Chang et.al [23] and Xieet.al [89] constructed control flow graphs with similar exe-cutable paths. Chang deduced the output states of the timerbased on the existing output state transition relationships,while Xie used constraints to model the program.
Combined with studies at T1, these studies couldhandle more temporal features, such as varied scan cyclelengths, and enabled input dependent behavior model-ing. With control flow based representation, nested logicand pointers could also be supported. However, without
concrete execution of the programs, the drawbacks areobvious: (1) the input vectors were either random or haveto be manually chosen, (2) the amount of symbolic stateslimited the program sizes, (3) the temporal informationfurther increased resource consumption. Next, we discussbehavior modeling with runtime information.
4.1.3. T3. With runtime information, existing research[21], [49], [62], [94], [95] modeled programs consideringits interactions with the physical processes, the supervisor,and the operator tasks. This allowed more realistic model-ing for timing sensitive instructions, and domain-specificbehavior modeling.
Automated frameworks [87], [95] were presented tomodel PLC behaviors with interrupt scheduling, functioncalls, and IO traces. Zhou et.al [95] adopted an environ-ment module for the inputs and outputs, an interruptionmodule for time-based instructions, and a coordinatormodule to schedule these two modules with the mainprogram. Wang et.al [87] automated a BIP (Behavior,Interaction, Priority) framework to formalize the scanningmode, the interrupt scheduler, and the function calls. Mesliet.al [69] presented a component-based modeling for thewhole control-command chain, with each component de-scribed as timed automata.
To automate modeling of domain-specific event behav-ior, VetPLC [94] generated timed event causality graphs(TECG) from the program, and the runtime data traces.The TECG maintained temporal dependencies constrainedby machine operations.
These studies removed the barrier from modelingevent-driven and domain-specific behaviors. They couldmitigate attacks violating security and safety requirementsvia special sequences of valid logic.
4.1.4. Challenges.
• Lack of modeling evaluation. The majority of thestudies only adopted one modeling method to obtain aprogram representation. We understood the representationis compatible with the formal verification framework.However, there were no scientific comparisons betweenmodels from different studies, except some high-leveldescriptions. Within one model, only a few studies [22],[66], [94] evaluated the number of states in their rep-resentations. It is even more difficult to understand theperformance of the model from the security perspective.
• State explosion.. The aforementioned studies havealready adopted an efficient representation that transformsa program unit as a state automaton, and formalize thestate transition between the current cycle and the next cy-cle. A less efficient model representation transforms eachvariable of a program as a state, and formalize the transi-tion between the states. Even though such representationcan benefit PLC programs in any languages, it produceslarge size models containing too many states to be verified,even for small and medium-sized programs. Therefore, inpractice, most of the programs are modeled in the formerefficient representation. For large size programs, however,both representations will produce large amounts of statecombinations, causing the state explosion problem. Thefollowing describes research works in state reduction.
7
4.2. State Reduction
The goal of state reduction is to improve the scalabilityand complexity of PLC program formalization. There aretwo common steps involved. First, we have to determinethe meaningful states related to safety and security prop-erties. Then, we trim the less meaningful states.
4.2.1. T1. At the source code level, a line of studies [28],[31], [43], [76] performed state reduction. Gourcuff et.al[43] considered the meaningful states as those related tothe input and output variables, since they directly controlthe behavior of the physical processes. To obtain thedependency relations of the input and output variables,Gourcuff conducted static code analysis to get variabledependencies in a ST program, and found a large portionof the unrelated states. Even though this method signifi-cantly reduced the state search space, it also skipped muchof the original code for the following verification.
To improve the code coverage of the formalization,Pavlovic et.al [76] presented a general solution for FBDprograms. They first transformed the graphical programto textual statements in textFBD, and further substitutedthe circuit variables to tFBD. This approach removedthe unnecessary assignments connecting continuous state-ments and merged them into one. On top of this ap-proach, Darvas et.alfine tuned the reduction heuristicswith a more complete representation [28], [31]. Besidesunnecessary variable or logic elimination, this heuristicsadopted the Cone of influence (COI)-based reduction, andthe rule-based reduction. The COI-based reduction firstremoved unconditional states that all possible executionsgo through them. It then removed variables that do notinfluence the evaluation of the specification. The rule-based reduction could be specified based on the safety re-quirements of the application domain. Additionally, mathmodels were also used to abstract different components.Newell et.al [73] defined additional structure, attributemaps, graphs, and block groups to reduce the state spaceof their PVS code.
These studies successfully reduced the size of programstates. They were limited, however, to basic Boolean rep-resentation reduction. For programs with complex time-related variables, function blocks, or multitasks, thesestudies were insufficient. It was also unclear whether thereduction could undermine program security. The fol-lowing discusses other reduction techniques with moreinformation available.
4.2.2. T2. Studies at the binary level mostly adopted sym-bolic execution combined with flow-based representation.This demonstrated that meaningful states lead to differentsymbolic output vectors. TSV [68] merged the input statesthat could all lead to the same output values. It alsoabstracted the temporal execution graphs, by removingthe symbolic variables based on their product with thevaluations of the LTL properties.
To further reduce the unrelated states, Chang et.al[23] reduced the overlapping output states of the samescan cycle, and removed the output states that have beenanalyzed in previous cycles. To reduce the overhead oftimer modeling, they employed a deduction method for theoutput states of timers, through the analysis of the existing
output state transition relationships These reductions didnot undermine the goal of detecting malicious behaviorsspanning multiple cycles.
These studies mainly targeted the reduction from ran-dom inputs in symbolic execution. Compared with T1,these studies were more interested in preserving temporalfeatures. Without undermining the temporal feature mod-eling, and given the lack of real inputs, the reduction ofinput states was inefficient. The following discusses thereduction techniques when runtime inputs are available.
4.2.3. T3. With runtime information, we could gain abetter understanding of the real meaningful states. Theseinclude the knowledge from event scheduling for subrou-tine and interrupts, and the real inputs and outputs fromthe domain-specific processes.
Existing studies [49], [62], [94], [95] presented statereduction in different approaches. To reduce the scale ofthe model, Zhou et.al [95] modeled timers inline with themain program instead of a separate automata, since theirmodel had considered the real environment traces, theinterruptions, and the scheduling between them. Similarly,Wang et.al [87] compressed segments without jump andcall instructions into one transition.
Besides merging unnecessary states, the real inputsand domain-specific knowledge could narrow down therange for modeling numerical and float variables. InZhang’s study [94], continuous timing behavior was dis-cretized to multiple time slices with a constant interval.Since the application-specific IO traces are available, thetime interval were narrowed to a range balancing betweenefficiency and precision.
Compared with studies at T1 and T2, state reductionat T3 was more powerful, not only with more realistictemporal and event-driven features supported, but alsohelped to extract more meaningful states with domain-specific runtime information.
4.2.4. Challenges.• Lack of “ground truth” for continuous behavior.
We discussed that runtime traces helped determining a“realistic” granularity for continuous behaviors. However,choosing the granularity was still experience-based andad-hoc. In fact, a too coarse granularity would fail todetect actual attacks, while a too fine granularity expectedinfeasible attack scenarios [37]. Abstracting a “groundtruth” model for continuous behavior remains challenging.
• Implicitness and stealthy attacks from reduction.Although existing studies have considered property preser-vation, the reduced “unrelated” states may undermine PLCsecurity. We mentioned in an earlier section (§3) thatimplicit specification had led to attacks. The reduced statesmay cause the implicit mapping between the variables inthe program and its specification, or they may containstealthy behaviors that were simply omitted by the speci-fication. The following discusses research on specificationgeneration.
4.3. Specification Generation
The goal of these studies is to generate safety andsecurity specifications with formal semantics. Specify-ing precise and complete desired properties is difficult.
8
Existing studies focused on two aspects: (1) process-independent properties that describe the overall require-ments for a control system, and (2) domain-specific prop-erties that require domain expertise.
4.3.1. T1. At the source code level, a line of studies [15],[29], [30], [42], [48] investigated specification generationwith process-independent properties. These properties in-clude avoiding variable locks, avoiding unreachable oper-ating modes, operating modes are mutually exclusive, andavoiding irrelevant logic [78].
Existing studies [7], [12], [18], [71], [78] usuallyadopted CTL or LTL-based formulas to express theseproperties. LTL describes the future of paths, e.g., a con-dition will eventually be true, a condition will be true untilanother fact becomes true, etc. CTL describes invarianceand reachability, e.g., the system never leave the set ofstates, the system be able to reach the set of states,respectively. Other variants included ACTL, adopted byRawlings [78], and ptLTL, adopted by Biallas [14].
Besides CTL and LTL-based formulas, a proof assis-tant was also investigated to assist the development of for-mal proofs. To formally define the syntax and semantics,Biha et.al [15] used a type theory based proof assistant,Coq, to define the safety properties for IL programs. Thesemantics concentrated on the formalization of on-delaytimers, using discrete time with a fixed interval. BesidesCoq, K framework [48] was also adopted to provide aformal operational semantics for ST programs. K is arewriting-based semantic framework that has been appliedin defining the semantics for C and Java. Compared withCoq, K is less formal but lighter and easier to read andunderstand. The trade-off is that manual effort is requiredto ensure the formalism of the definition.
These studies limited specification generation to cer-tain program models. To enable formal semantics for state-based, data-flow-oriented, and time-dependent programmodels, Darvas et.al [29] presented PLCspecif to supportvarious models.
These studies provided opportunities for engineerslacking formalism expertise to generate formal and preciserequirements. The proof assistant frameworks even al-lowed generating directly executable programs, e.g. C pro-gram. Nevertheless, only process-independent propertiescould be automated, the following discusses specificationgeneration with more information available.
4.3.2. T2. As mentioned earlier, symbolic execution al-lowed these studies to support program modeling with nu-meric and float variables. These variables provided moreroom for property definitions in the specification. TSV[68] defined properties bounding the numerical deviceparameters, such as the maximum drive velocity and ac-celeration. Others et.al [23], [89], [96] defined propertiesto detect malicious code injection, parameter tamperingattacks. Xie et.al [89] expanded the properties to detectstealthy attacks, and denial of service attacks.
Similar to studies at T1, these studies all adopted LTL-based formalism, and could automate process-independentproperty generation. To accommodate certain attack strate-gies, the specification generation was manually defined.
4.3.3. T3. With runtime information available, specifica-tion generation considered more on domain-specific prop-
erties. In a waste water treatment plant setting, Luccariniet.al [62] applied artificial neural networks to extractqualitative patterns from the continuous signals of thewater, such as the pH and the dissolved oxygen. Thesequalitative patterns were then mapped to the control eventsin the physical processes. The mapping was logged usingXML and translated into formal rules for the specification.This approach considered the collected input and outputtraces as ground truth for security and safety properties,and removed the dependencies on domain expertise.
In reality, the runtime traces might be polluted, orcontain incomplete properties for verification. To ensurethe correctness and completeness of domain-specific rules,existing studies [37], [94] also considered semi-automatedapproaches, which combined automated data mining andmanual domain expertise. VetPLC [94] formally definedthe safety properties, through automatic data mining andevent extraction, aided with domain expertise in craftingsafety specifications. VetPLC adopted timed propositionaltemporal logic (TPTL), which was more suitable to quan-titatively express safety specifications.
Besides (semi)-automated specification generation,Mesli et.al [69] manually defined a set of rules for theinteraction between each component along the chain ofcontrol. The requirements are also written in CTL tempo-ral logic. To assist domain experts in developing formalrules, Wang et.al [87] formalized the semantics for aBIP model for all types of PLC programs. It automatedprocess-independent rules for interrupts, such as, follow-ing the first come first serve principle.
These studies enabled specification generation withdomain-specific knowledge. They thus expanded securityresearch with more concentration on safety requirements.
4.3.4. Challenges.• Lack of specification-refined programming. Since
these studies already assumed the existence of the PLCprograms (source code or binary), the generated specifi-cation could help refine the programming and programmodeling. We have mentioned earlier that state reductionconsidered property preserving, and removed “irrelevantlogic” from program modeling. However, generated prop-erties did not provide direct feedback to the programsource code. In fact, program refinement in a similarapproach of state reduction is promising in eliminatingirrelevant stealthy code from the source.
• Ad-hoc and unverified specification translations.Despite the availability of formal semantics and proofassistants, such as Coq, PVS, HOL, existing requirementsare informally defined in high-level languages, and var-ied across industrial domains. Existing studies translat-ing these requirements encountered many challenges: (1)tradeoff between an automated but unverified approach,or a formal but manual rewriting, (2) the dependencies onprogram language (many studies were based on IL [48],deprecated in IEC 61131-3 since 2013), (3) the rules werebased on sample programs without the complexity of thereal industry.
• Barrier for automated domain-specific propertygeneration. Although Luccarini [62] presented a promis-ing approach, it was based on two unrealistic assump-tions: (1) the trace collected from the physical processes
9
was complete and could be trusted, and (2) the learningalgorithm extracted the rules completely and accurately.Without further proofs (manual domain expertise) to liftthese two assumptions, the extracted properties wouldbe an incomplete“white list” which may also containimplicitness, leading to false positives or true negativesin the verification or detection.
• Specification generation with evolved system de-sign. Increasing requirements were laid on PLC programs,considering the interactions from new components. In thebehavior modeling, we have observed studies formalizingthe behaviors of new interactions, on top of existed mod-els, for example, adding a scheduler module combing anexisted program with a new component. Compared withthat, we saw fewer studies investigating incremental spec-ification generation, based on existed properties. It wasstill challenging to define the properties to synchronizePLC programs with various components, especially in atiming-sensitive fashion.
4.4. Verification
We already discussed the modeling of program be-havior, and specification generation. With these, a line ofstudies [11], [12], [18], [19], [22], [71], [72], [74], [77],[78], [92] applied model checking and theorem provingto verify the safety and security of the programs.
These studies applied several formal verificationframeworks, summarized in Table 2. The majority of themapplied Uppaal and Cadence SMV. Uppaal was used forreal-time verification represented by a network of timedautomata extended with integer variables, structured datatypes, and channel synchronization. Cadence SMV wasused for untimed verification.
4.4.1. T1. At the source code level, formal verificationstudies aimed at verifying weaknesses W1-W4, to defendagainst general safety problems. They had been appliedby programs from different industries.
To defend G1, Bender et.al [12] adopted model check-ing for LD programs modeled as timed Petri net. Theyapplied model checkers in the Tina toolkit to verify againstLTL properties. Bauer et.al [11] adopted Cadence SMVand Uppaal, to verify untimed modeling and timed mod-eling of the SFC programs, respectively. They identifiederrors from three reactors. Similarly, Niang et.al [74] veri-fied a circuit breaker program in SFC using Uppaal, basedon a recipe book specification. To defend G2, Hailesellasieet.al [45] applied Uppaal and compared two formallygenerated attributed graphs, the Golden Model with theproperties, and a random model formalized from a PLCprogram. The verification is based on the comparison ofnodes and edges of the graphs. They detected stealthycode injections.
Instead of adopting existing tools, several studiesdeveloped their own frameworks for verification. Ar-cade.PLC [14] supported model checking with CTL andLTL based properties for all type of PLC programs.PLCverif [30] supported programs from all five SiemensPLC languages. NuDE 2.0 [53] provided formal-method-based software development, verification and safety anal-ysis for nuclear industries. Rawlings et.al [78] applied
symbolic model checking tools st2smv and SynthSMV toverify and falsify a ST program controlling batch reactorsystems. They automatically verified process-independentproperties, rooted in W1-W4.
Besides model checking, existing studies [73] alsoadopted PVS theorem proving to verify the safety prop-erties described in tabular expressions in a railway inter-locking system.
These studies limited to general safety requirementsverification. To defend G2 and G3, more information willbe needed, as discussed in the following.
4.4.2. T2. This line of studies [23], [68], [87], [89], [96]allowed us to detect binary tampering attacks.
TSV [68] combined symbolic execution and modelchecking. It fed the model checker with an abstractedtemporal execution graph, with its manually crafted LTL-based safety property. Due to its support for random timervalues within one cycle, TSV was limited by checking thecode with timer operations, and still suffered from stateexplosion problem. Xie et.al [89] mitigated this problemwith the use of constraints in verifying random inputsignals. Xie used nuXmv model checker. Chang et.al [23]applied a less formal verification based on the number ofstates.
These studies successfully detected malicious parame-ter tempering attacks, based on sample programs control-ling traffic lights, elevator, water tank, stirrer, and sewageinjector.
4.4.3. T3. With runtime information, existing studiescould verify domain-specific safety and security issues,namely all the weaknesses and attacks goals discussed inan earlier section §3.
To defend G1 by considering the interactions to theprogram, Carlsson et.al [20] applied NuSMV to verifythe interaction between the Open Platform Communica-tions (OPC) interface and the program, using propertiesdefined as server/client states. They detected synchroniza-tion problems, such as jitter, delay, race condition, andslow sampling caused by the OPC interface. Mesli [69]applied UPPAAL to multi-layer timed automata, based ona set of safety and usability properties written in CTL.They detected synchronization errors between the controlprograms and the supervision interfaces.
To fully leverage the knowledge from the physicalprocesses, VetPLC [94] combined runtime traces, andapplied BUILDTSEQS to verify security properties de-fined in timed propositional temporal logic. HyPLC [39]applied Theorem prover KeYmaera X to verify the prop-erties defined in differential dynamic logic. Different fromVetPLC, HyPLC aimed at a bi-directional verificationbetween the physical processes, and the PLC program,to detect safety violations.
These studies either assumed an offline verification,or vaguely mentioned using a supervisory componentfor online verification. To provide an online verificationframework, Garcia et.al [41] presented an on-device run-time solution to detect control logic corruption. Theyleveraged an embedded hypervisor within the PLC, withmore computational power and integration of direct libraryfunction calls. The hypervisor overcame the difficulties
10
of strict timing requirements and limited resources, andallowed verification to be enforced within each scan cycle.
4.4.4. Challenges.
• Lack of benchmarks for formal verification. Similarto the challenges in behavior modeling, an ideal evaluationshould be multi-dimensional: across modeling methods,across verification methods, and based on a set of bench-mark programs. Existing evaluations, if performed, werelimited to one dimension and based on at most a few sam-ple programs. These programs were often vendor-specific,test case driven, and failed to reflect the real industry com-plexity. Without a representative benchmark and concreteevaluation, the security solution design would still be ad-hoc.
• Open-source automated verification frameworks.Existing studies have presented several open-sourceframeworks taking a PLC program as input, and automat-ically output the formal verification result over genericproperties. These frameworks (e.g. Arcade.PLC, st2smvand the SynSMV) lowered the bar for security analy-sis using formal verification. However, over the years,such frameworks were no longer supported. No compara-ble replacement emerged, except PLCverif [27] targetingSiemens programs.
• High demand for runtime verification. The chal-lenges include (1) expanded attack landscapes due toincreasingly complex networking, (2) tradeoff betweenlimited available resources on the PLC and real-timeconstraints, (3) runtime injected stealthy attacks due toinsecure communication, and (4) runtime denial of serviceattacks omitted by existing studies.
5. Recommendations
We have highlighted the security challenges in defend-ing PLC program attacks, and formal verification-basedprotection. Next, we offer recommendations to overcomethese challenges.
5.1. Program Modeling
5.1.1. Plant Modeling. Very few studies have consideredplant model formalization in addition to PLC programmodeling. However, Galvao et.al [37] have pointed outthe importance of plant models in formal verification. Werecommend more research in plant modeling. Specifically,future research should consider refinement techniques todefine the granularity and level of abstraction for the plantmodels and the properties to verify. This is challenging es-pecially for symbolic verification. Future research shouldinvestigate the avoidance of state explosion by distillingfeasible conditions of the plant that can trigger propertyviolations in the program.
5.1.2. Input manipulation verification. Plant modelingnot only helps foramlize more accurate and completeprogram behaviors, but also helps in mitigating programinput manipulation attacks. We found input manipulationis widely adopted by the attackers, as discussed in Section3. To detect such attacks, we recommend Orpheus [25],
a prototype to counter a similar problem. Although nottargeting the setting in PLC, Orpheus has demonstratedto detect input manipulation attacks through event consis-tency checking between the program model and the plantmodel. Specifically, to allow runtime checking, futureresearch could consider program instrumentation on theinput and output of different program units. We encouragefuture research to consider behavior modeling for boththe PLC program and the physical processes, and exploreevent consistency to detect input manipulation attacks.
5.2. State Reduction
Under T1, we discussed code level attacks that coulddisguise themselves as bad coding practice, and are hard tobe noticed. During the state reduction, based on an existedspecification, “unrelated” states are trimmed to avoid stateexplosion problem. However, existing studies failed toinvestigate the relationship between the “unrelated” statesand the original program. It could be hidden jumps witha stealthy logger to leak program critical information.Since the specification might only consider the noticeableunsafe behaviors, with the physical process disturbed,states from the stealthy code would be highly recognizedas “unrelated”. We, therefore, recommend future researchto investigate the security validation of unrelated code,and consider automatic program cleaning for the stealthycode.
5.3. Specification Generation
5.3.1. Domain-specific property definition. Domain-specific properties are often manually defined and causeimplicitness. This can be lifted by formal plant model-ing. In many industries, the physical process behaviorscan be formalized as a hybrid program using differentialdynamic logic, which could then be verified with a soundproof calculus. HyPLC [39] has formalized the semanticsbridging the dynamic logic model with ST programs.This enabled domain-specific property generation in anautomatic and formal way. Although promising, existingstudies are limited in supporting arithmetic operations, andmultitask programs. We recommend future research to ex-plore semantics formulation on arithmetic operations andmultitask programs. We believe such research will benefitthe automation of domain-specific property definition.
5.3.2. Incremental specification generation. We dis-cussed attacks using a full chain of vulnerabilities, modi-fying program payload, compromising the sensor, the PLCcommunication, and deceiving the engineering station. Wealso discussed the challenges, given the fast-evolving sys-tem design. This leads us to think about incremental spec-ification generation, with a full chain of behaviors, andupdate in a dynamic spectrum. Incremental specificationgeneration [8] has been designed for interactive systems.In the PLC chain of control, interactions should considerboth the physical process changes, and the inclusion of theengineering station. Modeled behaviors from these newinteractions should be compatible with existing properties.To update in a dynamic spectrum, the behavior changesfrom each interactive component should support automatic
11
TABLE 2: Common frameworks for formal verification
Frameworks Modeling Languages Property Languages/Prover Supported Verification TechniquesNuSMV/nuXMV SMV input language (BDD) CTL, LTL SMT, Model checking, fairness requirements
UPPAAL Timed automata with clock and data variables TCTL subset Time-related, and probability related propertiesCadence SMV SMV input language (BDD) CTL, LTL Temporal logic properties of finite state systems
SPIN Promela LTL Model checkingUMC UML UCTL Functional properties of service-oriented systemsCoq Gallina (Calculus of Inductive Constructions) Vernacular Theorem proof assistantPVS PVS language (typed higher-order logic) Primitive inference procedures Formal specification, verification and theorem proverZ3 SMT-LIB2 SMT-LIB2 Theories Theorem prover
TABLE 3: Existing studies using formal verification to detect control logic attacks
Defense Goal Formal VerificationThreat Model Paper G1 G2 G3 Behavior Modeling State Reduction Specification Generation VerificationAdiego’15 [7] X X XBauer’04 [11] X XBender’08 [12] X X XBiallas’12 [14] X X XBiha’11 [15] X X
Brinksma’00 [18] X XDarvas’14 [28] X XDarvas’15 [29] X X
Darvas’16-1 [30] X X XDarvas’16-2 [31] X XDarvas’17 [32] X XGiese’06 [42] X X X
Gourcuff’06 [43] X XHailesellasie’18 [45] X X X
Huang’19 [48] X XKim’17 [53] X X
Moon’94 [71] X XNewell’18 [73] X X XNiang’17 [74] X X
Pavlovic’10 [76] X XRawlings’18 [78] X X X
Mader’00 [63] X XOvatman’16 [75] X X X
Moon’92 [72] X X X X XBohlender’18 [16] X XKuzmin’13 [58] X XBonfe’03 [17] X X
Chadwick’18 [22] X X XFrey’00 [36] X XYoo’09 [92] X X X X X
Lamperiere’99 [59] X XKottler’17 [56] X X X X XYounis’03 [93] X X
T1
Canet’00 [19] X X X X XChang’18 [23] X X X X X
McLaughlin’14 [68] X X X X XXie’20 [89] X X X X X X
T2
Zonouz’14 [96] X X X XCarlsson’12 [20] X XCengic’06 [21] X XGalvao’18 [37] X X
Garcia’19-Towards [40] X XGarcia’16 [41] X XJanicke’15 [49] X X X
Luccarini’10 [62] X X X XMesli-Kesraoui’16 [69] X X X X
Wang’13 [87] X XX XZhang’19 [94] X X X X X XZhou’09 [95] X X X
Vyatkin’99 [85] X XRossi’00 [79] X XWan’09 [86] X X X
Garcia’19-HyPLC [39] X XMokadem’10 [70] X X
Cheng’17 [25] X X
T3
Ait’98 [8] X X
generation and comparison. This requires automatic trans-lations between the behavior models of each component.The closest study is HyPLC [39], which supported au-
tomatic translation between the PLC program, and thephysical plant model. However, incremental specificationgeneration was not considered. We encourage future re-
12
search to investigate this direction, and seek for mutualrefinement in an interactive fashion.
5.4. Verification
5.4.1. Real-time attack detection. Section §3 havedemonstrated several attacks targeted to deceive or evadethe runtime detection from the engineering station. How-ever, few studies supported runtime verification beyond ahigh-level prototype, let alone a compromised engineeringstation. Defending such attacks are challenging, sincemany security solutions are designed to be integrated intothe engineering station, which have already supported richfeatures outside the scope of security, and are thereforeexposed to various vulnerabilities.
It is important for security researchers to consider acompromised engineering station, and stay isolated fromthe PLC and its interacting components. In practice, thisrepresents the scenarios for many PLC users, who areonly exposed to a simple interface to access the PLC.We recommend future research to explore the verificationbetween the PLC and the other interacting components,including the engineering station. Such solution shouldeliminate the resource constraints within a PLC, and allowthe program to meet the strict limit of a scan cycle. Werecommend future research to investigate such solutionsby learning from the attack studies [38], [51], and existingbump-in-the-wire security solutions [68].
5.4.2. Open-source toolings and evaluation. As wementioned earlier, the lack of evaluation led to ad-hocdesign at each aspect of the formal verification. Despitethe large body of studies, we found it hard to see the trade-off between different behavior models, different propertydefinitions, and different verification techniques.
At a high-level, Ovatman et.al [75] surveyed exist-ing model checking studies considering application area,system size (number of states), performance (approximatetime for model checking), and automation level. They alsofound these information missing for many of the studies.We believe the lack of evaluation, for both performanceand security purpose, were due to the lack of open sourceframeworks and benchmarks. Although existing studieshave presented open source formal verification framework,such as PLCverif [27], and collected public PLC programsin binary [52], they are vendor-specific and test casedriven. We recommend future studies to remove the bar-rier by creating benchmark programs to support concreteevaluations. These benchmarks should represent industrialcomplexities, and consider IEC-61131 compliant and becompatible with open source PLCs [9]. We believe thebenchmarks will benefit security research with runtimeanalysis.
5.4.3. Multitasks Verification. Existing attacks haveshown to use PLC multitasks to perform a denial-of-service attacks, and spread stealthy worms, e.g. PLCIn-ject [55] and PLC-Bluster [82]. Based on the hardwarecharacteristics, the structure of multitask control logicincludes temporal and spatial parallelism. Several studies[70] have explored temporal paralleled multitask pro-grams, in particular, to check the reaction time betweentasks. For spatial paralleled multitask control logic [40],
formal verification is a new and challenging research area.We recommend future research to investigate this areaconsidering the BIP model, with task behavior, integrity,and priority.
6. Conclusion
Control logic vulnerabilities have seen fast growthin recent years. These vulnerabilities could lead to at-tacks causing severe damages to the critical infrastructure.Looking back on past research, over the years, extensivestudies have investigated these attacks and formal verifi-cation based defense solutions.
This paper provided a systematization of knowledgebased on these studies. We categorized the attacks anddefenses based on different threat models, attack goals,and underlying weaknesses. We discussed the techniquesand approaches applied by these studies. Our systematiza-tion showed that control logic modification attacks havebeen evolved with the system design. Advanced attackscould compromise the whole chain of control, and inthe meantime evade various security detection methods.To counter these attacks, the majority of the researchinvestigated ad-hoc formal verification techniques, and thebarriers exist in every aspect of formal verification.
To overcome these barriers, we suggest a full chain ofprotection and we encourage future research to investigatethe following: (1) formalize plant behaviors to defendinput manipulation attacks, (2) explore stealthy attackdetection with state reduction techniques, (3) automatedomain-specific specification generation and incrementalspecification generation, and (4) explore real-time veri-fication with more support in open-source tooling, andthorough evaluation.
References
[1] Formal Verification. https://en.wikipedia.org/wiki/Formalverification.
[2] ICS CERT. https://www.us-cert.gov/ics.
[3] National Vulnerability Database. https://nvd.nist.gov/vuln/data-feeds.
[4] Offensive Security. https://www.exploit-db.com.
[5] Simatic S5 PLC. https://en.wikipedia.org/wiki/Simatic S5 PLC.
[6] Ali Abbasi and Majid Hashemi. Ghost in the PLC designing anundetectable programmable logic controller rootkit via pin controlattack. Black Hat Europe, 2016:1–35, 2016.
[7] Borja Fernandez Adiego, Daniel Darvas, Enrique Blanco Vinuela,Jean-Charles Tournier, Simon Bliudze, Jan Olaf Blech, and VıctorManuel Gonzalez Suarez. Applying model checking to industrial-sized PLC programs. 11(6):1400–1410, 2015.
[8] Yamine Aıt-Ameur, Patrick Girard, and Francis Jambon. Using theb formal approach for incremental specification design of interac-tive systems. In IFIP International Conference on Engineering forHuman-Computer Interaction, pages 91–109. Springer, 1998.
[9] Thiago Alves and Thomas Morris. Openplc: An iec 61,131–3 compliant open source industrial controller for cyber securityresearch. Computers & Security, 78:364–379, 2018.
[10] Michael J Assante. Confirmation of a coordinated attack on theukrainian power grid. SANS Industrial Control Systems SecurityBlog, 207, 2016.
[11] Nanette Bauer, Sebastian Engell, Ralf Huuck, Sven Lohmann, BenLukoschus, Manuel Remelhe, and Olaf Stursberg. Verification ofPLC programs given as sequential function charts. In Integration ofsoftware specification techniques for applications in Engineering,pages 517–540. Springer, 2004.
13
[12] Darlam Fabio Bender, Benoıt Combemale, Xavier Cregut,Jean Marie Farines, Bernard Berthomieu, and Francois Vernadat.Ladder metamodeling and PLC program validation through timePetri nets. In European Conference on Model Driven Architecture-Foundations and Applications, pages 121–136. Springer, 2008.
[13] Dillon Beresford. Exploiting siemens simatic s7 plcs. Black HatUSA, 16(2):723–733, 2011.
[14] Sebastian Biallas, Jorg Brauer, and Stefan Kowalewski. Arcade.PLC: A verification platform for programmable logic controllers.In 2012 Proceedings of the 27th IEEE/ACM International Confer-ence on Automated Software Engineering, pages 338–341. IEEE,2012.
[15] Sidi Ould Biha. A formal semantics of PLC programs in Coq.In 2011 IEEE 35th Annual Computer Software and ApplicationsConference, pages 118–127. IEEE, 2011.
[16] Dimitri Bohlender and Stefan Kowalewski. Compositional verifi-cation of PLC software using horn clauses and mode abstraction.IFAC-PapersOnLine, 51(7):428–433, 2018.
[17] Marcello Bonfe and Cesare Fantuzzi. Design and verification ofmechatronic object-oriented models for industrial control systems.In EFTA 2003. 2003 IEEE Conference on Emerging Technologiesand Factory Automation. Proceedings (Cat. No. 03TH8696), vol-ume 2, pages 253–260. IEEE, 2003.
[18] Ed Brinksma and Angelika Mader. Verification and optimizationof a PLC control schedule. In International SPIN Workshop onModel Checking of Software, pages 73–92. Springer, 2000.
[19] Geraud Canet, Sandrine Couffin, J-J Lesage, Antoine Petit, andPhilippe Schnoebelen. Towards the automatic verification of PLCprograms written in Instruction List. volume 4, pages 2449–2454.IEEE, 2000.
[20] Henrik Carlsson, Bo Svensson, Fredrik Danielsson, and BengtLennartson. Methods for reliable simulation-based PLC code ver-ification. IEEE Transactions on Industrial Informatics, 8(2):267–278, 2012.
[21] Goran Cengic, Oscar Ljungkrantz, and Knut Akesson. Formalmodeling of function block applications running in IEC 61499execution runtime. In 2006 IEEE Conference on Emerging Tech-nologies and Factory Automation, pages 1269–1276. IEEE, 2006.
[22] Simon Chadwick, Phillip James, Markus Roggenbach, and TomWetner. Formal Methods for Industrial Interlocking Verification. In2018 International Conference on Intelligent Rail Transportation(ICIRT), pages 1–5. IEEE, 2018.
[23] Tianyou Chang, Qiang Wei, Wenwen Liu, and Yangyang Geng.Detecting plc program malicious behaviors based on state verifica-tion. volume 11067 of Lecture Notes in Computer Science, pages241–255, Cham, 2018. Springer International Publishing.
[24] Eyasu Getahun Chekole, Sudipta Chattopadhyay, Martın Ochoa,Huaqun Guo, and Unnikrishnan Cheramangalath. Cima: Compiler-enforced resilience against memory safety attacks in cyber-physicalsystems. Computers & Security, page 101832, 2020.
[25] Long Cheng, Ke Tian, and Danfeng Yao. Orpheus: Enforcingcyber-physical execution semantics to defend against data-orientedattacks. In Proceedings of the 33rd Annual Computer SecurityApplications Conference, pages 315–326, 2017.
[26] Stephen Chong, Joshua Guttman, Anupam Datta, Andrew Myers,Benjamin Pierce, Patrick Schaumont, Tim Sherwood, and NickolaiZeldovich. Report on the NSF workshop on formal methods forsecurity. arXiv preprint arXiv:1608.00678, 2016.
[27] D Darvas, E Blanco, and Switzerland V Molnar. PLCverif Re-engineered: An Open Platform for the Formal Analysis of PLCPrograms. ICALEPCS.
[28] Daniel Darvas, Borja Fernandez Adiego, Andras Voros, TamasBartha, Enrique Blanco Vinuela, and Vıctor M Gonzalez Suarez.Formal verification of complex properties on PLC programs. InInternational Conference on Formal Techniques for DistributedObjects, Components, and Systems, pages 284–299. Springer, 2014.
[29] Daniel Darvas, Enrique Blanco Vinuela, and Istvan Majzik. Aformal specification method for PLC-based applications. 2015.
[30] Daniel Darvas, Istvan Majzik, and Enrique Blanco Vinuela.Generic representation of PLC programming languages for formalverification. In Proc. of the 23rd PhD Mini-Symposium, pages 6–9.
[31] Daniel Darvas, Istvan Majzik, and Enrique Blanco Vinuela. Formalverification of safety PLC based control software. In Interna-tional Conference on Integrated Formal Methods, pages 508–522.Springer, 2016.
[32] Daniel Darvas, Istvan Majzik, and Enrique Blanco Vinuela. PLCprogram translation for verification purposes. Periodica Polytech-nica Electrical Engineering and Computer Science, 61(2):151–165,2017.
[33] Alessandro Di Pinto, Younes Dragoni, and Andrea Carcano. Triton:The first ics cyber attack on safety instrument systems. In Proc.Black Hat USA, pages 1–26, 2018.
[34] Nicolas Falliere, Liam O Murchu, and Eric Chien. W32. stuxnetdossier. White paper, Symantec Corp., Security Response, 5(6):29,2011.
[35] Alessio Ferrari, Gianluca Magnani, Daniele Grasso, and Alessan-dro Fantechi. Model checking interlocking control tables. InFORMS/FORMAT 2010, pages 107–115. Springer, 2011.
[36] Georg Frey and Lothar Litz. Formal methods in PLC program-ming. In Smc 2000 conference proceedings. 2000 ieee interna-tional conference on systems, man and cybernetics.’cyberneticsevolving to systems, humans, organizations, and their complexinteractions’(cat. no. 0, volume 4, pages 2431–2436. IEEE, 2000.
[37] Joel Galvao, Cedrico Oliveira, Helena Lopes, and Laura Tiainen.Formal verification: Focused on the verification using a plantmodel. In International Conference on Innovation, Engineeringand Entrepreneurship, pages 124–131. Springer, 2018.
[38] Luis Garcia, Ferdinand Brasser, Mehmet Hazar Cintuglu, Ahmad-Reza Sadeghi, Osama A Mohammed, and Saman A Zonouz. Hey,My Malware Knows Physics! Attacking PLCs with Physical ModelAware Rootkit. In NDSS, 2017.
[39] Luis Garcia, Stefan Mitsch, and Andre Platzer. HyPLC: Hybridprogrammable logic controller program translation for verification.In Proceedings of the 10th ACM/IEEE International Conferenceon Cyber-Physical Systems, pages 47–56, 2019.
[40] Luis Garcia, Stefan Mitsch, and Andre Platzer. Toward multi-task support and security analyses in plc program translation forverification. In Proceedings of the 10th ACM/IEEE InternationalConference on Cyber-Physical Systems, pages 348–349, 2019.
[41] Luis Garcia, Saman Zonouz, Dong Wei, and Leandro PflegerDe Aguiar. Detecting PLC control corruption via on-device runtimeverification. In 2016 Resilience Week (RWS), pages 67–72. IEEE,2016.
[42] Holger Giese, Sabine Glesner, Johannes Leitner, Wilhelm Schafer,and Robert Wagner. Towards verified model transformations. InProc. of the 3rd International Workshop on Model Development,Validation and Verification (MoDeV 2a), Genova, Italy, pages 78–93. Citeseer, 2006.
[43] Vincent Gourcuff, Olivier De Smet, and J-M Faure. Efficientrepresentation for formal verification of PLC programs. In 20068th International Workshop on Discrete Event Systems, pages 182–187. IEEE, 2006.
[44] Naman Govil, Anand Agrawal, and Nils Ole Tippenhauer. Onladder logic bombs in industrial control systems. In ComputerSecurity, pages 110–126. Springer, 2017.
[45] Muluken Hailesellasie and Syed Rafay Hasan. Intrusion Detectionin PLC-Based Industrial Control Systems Using Formal Verifica-tion Approach in Conjunction with Graphs. Journal of Hardwareand Systems Security, 2(1):1–14, 2018.
[46] Joseph Y Halpern and Moshe Y Vardi. Model checking vs. theoremproving: a manifesto. Artificial intelligence and mathematicaltheory of computation, 212:151–176, 1991.
[47] Daavid Hentunen and Antti Tikkanen. Havex hunts for ics/scadasystems. In F-Secure. 2014.
[48] Yanhong Huang, Xiangxing Bu, Gang Zhu, Xin Ye, Xiaoran Zhu,and Jianqi Shi. KST: Executable Formal Semantics of IEC 61131-3 Structured Text for Verification. IEEE Access, 7:14593–14602,2019.
14
[49] Helge Janicke, Andrew Nicholson, Stuart Webber, and AntonioCau. Runtime-monitoring for industrial control systems. Electron-ics, 4(4):995–1017, 2015.
[50] Fredrik Johansson. Attacking the Manufacturing Execution Sys-tem: Leveraging a Programmable Logic Controller on the ShopFloor, 2019.
[51] Sushma Kalle, Nehal Ameen, Hyunguk Yoo, and Irfan Ahmed.CLIK on PLCs! Attacking control logic with decompilation andvirtual PLC. In Binary Analysis Research (BAR) Workshop, Net-work and Distributed System Security Symposium (NDSS), 2019.
[52] Anastasis Keliris and Michail Maniatakos. ICSREF: A frameworkfor automated reverse engineering of industrial control systemsbinaries. In 26th Annual Network and Distributed System SecuritySymposium, NDSS 2019. The Internet Society, 2019.
[53] Eui-Sub Kim, Dong-Ah Lee, Sejin Jung, Junbeom Yoo, Jong-Gyun Choi, and Jang-Soo Lee. NuDE 2.0: A Formal Method-based Software Development, Verification and Safety AnalysisEnvironment for Digital I&Cs in NPPs. Journal of ComputingScience and Engineering, 11(1):9–23, 2017.
[54] Stephane Klein, Lothar Litz, and Jean-Jacques Lesage. Fault de-tection of discrete event systems using an identification approach.IFAC Proceedings Volumes, 38(1):92–97, 2005.
[55] Johannes Klick, Stephan Lau, Daniel Marzin, Jan-Ole Malchow,and Volker Roth. Internet-facing PLCs-a new back orifice. BlackhatUSA, pages 22–26, 2015.
[56] Sam Kottler, Mehdy Khayamy, Syed Rafay Hasan, and OmarElkeelany. Formal verification of ladder logic programs usingNuSMV. In SoutheastCon 2017, pages 1–5. IEEE, 2017.
[57] J Kuhner. Dotnetsiemensplctoolboxlibrary.https://github.com/jogibear9988/DotNetSiemensPLCToolBoxLibrary.
[58] Egor Vladimirovich Kuzmin, AA Shipov, and Dmitrii Aleksan-drovich Ryabukhin. Construction and verification of PLC programsby LTL specification. In 2013 Tools & Methods of ProgramAnalysis, pages 15–22. IEEE, 2013.
[59] Sandrine Lamperiere-Couffin, Olivier Rossi, J-M Roussel, and J-JLesage. Formal validation of PLC programs: a survey. In 1999European Control Conference (ECC), pages 2170–2175. IEEE,1999.
[60] Robert M Lee, Michael J Assante, and Tim Conway. German steelmill cyber attack. Industrial Control Systems, 30:62, 2014.
[61] Bernard Lim, Daniel Chen, Yongkyu An, Zbigniew Kalbarczyk,and Ravishankar Iyer. Attack induced common-mode failureson plc-based safety system in a nuclear power plant: Practicalexperience report. In 2017 IEEE 22nd Pacific Rim InternationalSymposium on Dependable Computing (PRDC), pages 205–210.IEEE, 2017.
[62] Luca Luccarini, Gianni Luigi Bragadin, Gabriele Colombini, Mau-rizio Mancini, Paola Mello, Marco Montali, and Davide Sottara.Formal verification of wastewater treatment processes using eventsdetected from continuous signals by means of artificial neuralnetworks. Case study: SBR plant. Environmental Modelling &Software, 25(5):648–660, 2010.
[63] Angelika Mader. A classification of PLC models and applications.In Discrete Event Systems, pages 239–246. Springer, 2000.
[64] PLC Manual. Basic Guide to PLCs: PLC Programming. https://www.plcmanual.com/plc-programming.
[65] Stephen McLaughlin and Patrick McDaniel. SABOT: specification-based payload generation for programmable logic controllers. InProceedings of the 2012 ACM conference on Computer and com-munications security, pages 439–449, 2012.
[66] Stephen McLaughlin and Saman Zonouz. Controller-aware falsedata injection against programmable logic controllers. In 2014IEEE International Conference on Smart Grid Communications(SmartGridComm), pages 848–853. IEEE, 2014.
[67] Stephen E McLaughlin. On Dynamic Malware Payloads Aimed atProgrammable Logic Controllers. In HotSec, 2011.
[68] Stephen E McLaughlin, Saman A Zonouz, Devin J Pohly, andPatrick D McDaniel. A Trusted Safety Verifier for Process Con-troller Code. In NDSS, volume 14, 2014.
[69] S Mesli-Kesraoui, A Toguyeni, A Bignon, F Oquendo, D Kesraoui,and P Berruet. Formal and joint verification of control programsand supervision interfaces for socio-technical systems components.IFAC-PapersOnLine, 49(19):426–431, 2016.
[70] Houda Bel Mokadem, Beatrice Berard, Vincent Gourcuff, OlivierDe Smet, and Jean-Marc Roussel. Verification of a timed multitasksystem with UPPAAL. IEEE Transactions on Automation Scienceand Engineering, 7(4):921–932, 2010.
[71] Il Moon. Modeling programmable logic controllers for logicverification. IEEE Control Systems Magazine, 14(2):53–59, 1994.
[72] Il Moon, Gary J Powers, Jerry R Burch, and Edmund M Clarke.Automatic verification of sequential control systems using temporallogic. AIChE Journal, 38(1):67–75, 1992.
[73] Josh Newell, Linna Pang, David Tremaine, Alan Wassyng, andMark Lawford. Translation of IEC 61131-3 function block di-agrams to PVS for formal verification with real-time nuclearapplication. Journal of Automated Reasoning, 60(1):63–84, 2018.
[74] Mohamed Niang, Alexandre Philippot, Francois Gellot, RaphaelCoupat, Bernard Riera, and Sebastien Lefebvre. Formal Verifi-cation for Validation of PSEEL’s PLC Program. In ICINCO (1),pages 567–574, 2017.
[75] Tolga Ovatman, Atakan Aral, Davut Polat, and Ali Osman Unver.An overview of model checking practices on verification of PLCsoftware. Software & Systems Modeling, 15(4):937–960, 2016.
[76] Olivera Pavlovic and Hans-Dieter Ehrich. Model checking PLCsoftware written in function block diagram. In 2010 Third Interna-tional Conference on Software Testing, Verification and Validation,pages 439–448. IEEE, 2010.
[77] Mathias Rausch and Bruce H Krogh. Formal verification ofPLC programs. In Proceedings of the 1998 American ControlConference. ACC (IEEE Cat. No. 98CH36207), volume 1, pages234–238. IEEE, 1998.
[78] Blake C Rawlings, John M Wassick, and B Erik Ydstie. Applica-tion of formal verification and falsification to large-scale chemicalplant automation systems. Computers & Chemical Engineering,114:211–220, 2018.
[79] Olivier Rossi and Philippe Schnoebelen. Formal modeling of timedfunction blocks for the automatic verification of Ladder Diagramprograms. In Proceedings of the 4th International Conference onAutomation of Mixed Processes: Hybrid Dynamic Systems (ADPM2000), pages 177–182. Citeseer, 2000.
[80] Saranyan Senthivel, Shrey Dhungana, Hyunguk Yoo, Irfan Ahmed,and Vassil Roussev. Denial of engineering operations attacks inindustrial control systems. In Proceedings of the Eighth ACMConference on Data and Application Security and Privacy, pages319–329, 2018.
[81] Abraham Serhane, Mohamad Raad, Raad Raad, and Willy Susilo.PLC code-level vulnerabilities. In 2018 International Conferenceon Computer and Applications (ICCA), pages 348–352. IEEE,2018.
[82] Ralf Spenneberg, Maik Bruggemann, and Hendrik Schwartke. Plc-blaster: A worm living solely in the plc. Black Hat Asia, MarinaBay Sands, Singapore, 2016.
[83] Michael Tiegelkamp and Karl-Heinz John. IEC 61131-3: Program-ming industrial automation systems. Springer, 1995.
[84] Sidney E Valentine Jr. Plc code vulnerabilities through scadasystems. 2013.
[85] Valeriy Vyatkin and H-M Hanisch. A modeling approach forverification of IEC1499 function blocks using net condition/eventsystems. In 1999 7th IEEE International Conference on EmergingTechnologies and Factory Automation. Proceedings ETFA’99 (Cat.No. 99TH8467), volume 1, pages 261–270. IEEE, 1999.
[86] Hai Wan, Gang Chen, Xiaoyu Song, and Ming Gu. Formalizationand verification of PLC timers in Coq. In 2009 33rd Annual IEEEInternational Computer Software and Applications Conference,volume 1, pages 315–323. IEEE, 2009.
[87] Rui Wang, Yong Guan, Liming Luo, Xiaoyu Song, and Jie Zhang.Formal modelling of PLC systems by BIP components. In 2013IEEE 37th Annual Computer Software and Applications Confer-ence, pages 512–518. IEEE, 2013.
15
[88] Min Xiao, Jing Wu, Chengnian Long, and Shaoyuan Li. Construc-tion of false sequence attack against PLC based power controlsystem. In 2016 35th Chinese Control Conference (CCC), pages10090–10095. IEEE, 2016.
[89] Yaobin Xie, Rui Chang, and Liehui Jiang. A malware detectionmethod using satisfiability modulo theory model checking for theprogrammable logic controller system. Concurrency and Compu-tation: Practice and Experience, n/a(n/a):e5724.
[90] Hyunguk Yoo and Irfan Ahmed. Control logic injection attacks onindustrial control systems. In IFIP International Conference on ICTSystems Security and Privacy Protection, pages 33–48. Springer,2019.
[91] Hyunguk Yoo, Sushma Kalle, Jared Smith, and Irfan Ahmed.Overshadow plc to detect remote control-logic injection attacks. InInternational Conference on Detection of Intrusions and Malware,and Vulnerability Assessment, pages 109–132. Springer, 2019.
[92] Junbeom Yoo, Eunkyoung Jee, and Sungdeok Cha. Formal mod-eling and verification of safety-critical software. IEEE software,26(3):42–49, 2009.
[93] M Bani Younis, Georg Frey, et al. Formalization of existing PLCprograms: A survey. In Proceedings of CESA, pages 0234–0239,2003.
[94] Mu Zhang, Chien-Ying Chen, Bin-Chou Kao, Yassine Qamsane,Yuru Shao, Yikai Lin, Elaine Shi, Sibin Mohan, Kira Barton, JamesMoyne, et al. Towards Automated Safety Vetting of PLC Code inReal-World Plants. In 2019 IEEE Symposium on Security andPrivacy (SP), pages 522–538. IEEE, 2019.
[95] Min Zhou, Fei He, Ming Gu, and Xiaoyu Song. Translation-basedmodel checking for PLC programs. In 2009 33rd Annual IEEEInternational Computer Software and Applications Conference,volume 1, pages 553–562. IEEE, 2009.
[96] Saman Zonouz, Julian Rrushi, and Stephen McLaughlin. Detectingindustrial control malware using automated PLC code analytics.IEEE Security & Privacy, 12(6):40–47, 2014.
Appendix
1. Vulnerability Databases
This analysis is based on some interesting information,i.e. the reported Common Vulnerabilities and Exposures(CVE)s, their corresponding Common Weakness Enumer-ation (CWE)s, the impacted vendors and industrial sectors,the complexity of the CVEs, and their public exploits.We obtain the information from several sources: the ICS-CERT [2], the National Vulnerability Database (NVD)[3], and the Exploit Database [4] created by OffensiveSecurity. The ICS-CERT reports timely security issuesand vulnerabilities specifically in the industrial controlsystems. The NVD dataset contains general reports fromall types of vulnerabilities, without details of affectedindustrial sectors and mitigation methods, as does by ICS-CERT. Nevertheless, the NVD dataset is less likely to missdata because of its popularity and has longer history (since2002), while ICS-CERT provides data since 2010. TheExploit Database is a CVE compliant archive of publicexploits, developed by penetration testers and vulnerabilityresearchers.
2. Analysis Framework
We developed an analysis framework to first crawland download data from the above sources, then extractinteresting information on control logic related vulner-abilities, combine the results from all the datasets, andfinally generate statistical reports from these information.
2008 2010 2012 2014 2016 2018 2020Year
0
2
4
6
8
10
CV
SS
Sco
re
Mean CVSS score for PLC-related CVEs per yearCVSS V2CVSS V3
Figure 4: The complexity of control logic related CVEs,depicted with the mean and the standard deviation ofCVSS scores. Some points are missing as they are notreported in the databases.
0 20 40 60 80Amount of CVEs
Siemens
Schneider
Rockwell
Omron
LCDS
Advan
Fuji
Prima
General-electric
3S-Smart
Ven
dors
200820092010201120122013201420152016201720182019
Figure 5: Notable PLC vendors and number of relatedcontrol logic vulnerabilities reported per year.
The extraction combines filtering notable PLC vendors[50], matching general keywords, such as PLC, controllogic, and combining specific keywords such as HMI andremote code execution, meaning such vulnerability in theHMI can lead to control logic execution in the PLC. Thevulnerability may exist in the following places: (1) PLC;(2) upper level components that affect running code onthe PLCs (e.g. HMI, SCADA, engineering station); (3)the communication between a PLC and such componentsor among PLCs. We randomly choose 100 CVEs fromour collected datasets. We manually label them as controllogic related or not, by going through the descriptionof the vulnerability and checking online documentationof the affected products. We consider these labels asthe ground truth. Then we run our automatic extractionframework and record control logic related CVEs. Bycomparing this record and the ground truth, we obtain theaccuracy of our automatic extraction, with false positiverate of 1% and false negative rate of 4%.
3. Analysis Result
Figure 4 shows the CVSS scores from both version 2and version 3. For these reported CVEs, we also analyzedthe top ten most affected vendors, as Figure 5 shows.
16
0 10 20 30 40 50Amount of CVEs
Improper Restriction of Operations CWE-119
Improper Input Validation CWE-20
Stack-based Buffer Overflow CWE-121
Cross-site Scripting CWE-79
Uncontrolled Resource Consumption CWE-400
NVD-CWE-noinfo
Improper Authentication CWE-287
Uncontrolled Search Path Element CWE-427
Improper Access Control CWE-284
Heap-based Buffer Overflow CWE-122
Out-of-bounds Write CWE-787
Use of Hard-coded Credentials CWE-798
Path Traversal CWE-22
Out-of-bounds Read CWE-125
NVD-CWE-Other
Missing Authentication for Critical Function CWE-306
SQL Injection CWE-89
Exposure of Sensitive Information CWE-200
Improper Check for Unusual or Exceptional Conditions CWE-754
Code Injection CWE-94
CW
Es
200820092010201120122013201420152016201720182019
Figure 6: The type of CWEs and their correspondingnumber reported per year.
17