runtime verification for the web (rv 2010 tutorial)

Sylvain Hallé

Sylvain Hallé and Roger Villemaire

Runtime Verification for the Web

A Tutorial Introduction to Interface Contractsin Web Applications

Université du Québec à ChicoutimiCANADA

NOSHOW

Université du Québec à MontréalCANADA

Fonds de recherchesur la natureet les technologies

CRSNGNSERC

Sylvain Hallé

Web applications and cloud computing: a growing part of computing systems

Very simple protocols: no state, only basic type checking, the rest is up to the developers

Loose couplingg of components: nice but comes with problems!

Few works on verification / enforcement of web applications

A ‘‘call to arms’’ to the community: interesting opportunities for application of RV

Why this tutorial?

Sylvain Hallé

Part One: The basics of web applications

?What is a web application?An example: the Beep StoreConstraints and problems

Part Two: Interface contracts in web applications

Characterizing constraintsFormalizing constraintsMonitoring constraintsDoing this for real: back to the Beep StoreWhat’s next?

Tutorial overview

Sylvain Hallé

Part One

The basics of web applications

Sylvain Hallé

Desktop computing

Sylvain Hallé

Desktop computing

Sylvain Hallé

Cloud computingCloud computing

Sylvain Hallé

Network connection

Sylvain Hallé

Cloud computingA static web site

Sylvain Hallé

Beatles

Caravan

Sylvain Hallé

Beatles

Caravan

beatles.html

Sylvain Hallé

t is th

Ask fo

Versio

beatles.html

Sylvain Hallé

t is th

Ask fo

Versio

beatles.html

<html>

</html>

...Results for

Beatles...

Sylvain Hallé

t is th

Ask fo

Versio

beatles.html

<html>

</html>

...Results for

Beatles...

COnly page rendering instructions are sent

Sylvain Hallé

Cloud computingA dynamic web site

Beatles

Caravan

Sylvain Hallé

Cloud computing

Beatles

Caravan

page.php?artist beatles=

A dynamic web site

Sylvain Hallé

artist beatles=

Sylvain Hallé

artist beatles=

Sylvain Hallé

artist beatles=

Sylvain Hallé

artist beatles=

t is th

Ask fo

Versio

Sylvain Hallé

CContent is generated programatically based on user input

artist beatles=

t is th

Ask fo

Versio

Sylvain Hallé

Cloud computingAjax web application

Sylvain Hallé

JavaScript

Sylvain Hallé

Beatles

Caravan

Sylvain Hallé

Beatles

Caravan

"javascript: findBand(’ ’)"Beatles

Sylvain Hallé

findBand(’Beatles’)

Sylvain Hallé

artist beatles=

Sylvain Hallé

artist beatles=

Sylvain Hallé

document.innerHTML = findBand(’Beatles’)

artist beatles=

Sylvain Hallé

document.innerHTML = findBand(’Beatles’)

artist beatles=

CPage is updated, not reloaded

Sylvain Hallé

artist beatles=

CServer response only provides updated contents

document.innerHTML =

Sylvain Hallé

Ajax web applications: examples

Microsoft Office Live

Sylvain Hallé

Chrome OS

Sylvain Hallé

Does not needto be a URL

Does not needto be HTML

Sylvain Hallé

</Search>

beatles

</Artist>

Sylvain Hallé

</SearchResults>

The Beatles Rubber Soul ...

<Item>

</Item>

</Artist><Title>

</Title>

</Search>

beatles

</Artist>

Sylvain Hallé

Cloud computingAjax web application<Search>

</Search>

beatles

</Artist> XMLThe eXtensible Markup

Language

?Nested collection ofelements

?Input/output data issemi-structured

</SearchResults>

The Beatles Rubber Soul ...

<Item>

</Item>

</Artist><Title>

</Title>

Sylvain Hallé

Cloud computingConceptually...

Sylvain Hallé

Web serviceWeb client

Sylvain Hallé

An example: the Beep Store

? tutorial application

?Fake CD catalog + web service+ web client

?Functionalities typical of applications we studied

?Examples:

Session login/logout Shopping cart operations

Purpose-built

SQLite PHPJavaScript

real-world

TheBeepStore

Sign in or register

What is this?

Ask for account

Contact us

Fault parameters

Search: Your Cart

Search results for ‘Beatles’

Rubber SoulThe Beatles

Yellow SubmarineThe Beatles

Sylvain Hallé

Main issue

Possible between messages sent and messages expected

mismatch

Not like traditional programming: all input-output is exchanged unverified!

Sylvain Hallé

Defining message formats

Sylvain Hallé

</ItemSearch> beatles<Artist> </object>

Sylvain Hallé

</Items></ItemSearchResponse>

Help!The Beatles

<Item>

</Item> ...

Sylvain Hallé

Help!The Beatles

<Item>

</Item> ...

XML request

XML response

Sylvain Hallé

ItemSearch[ [string]]Artist

Help!The Beatles

<Item>

</Item> ...

XML request

XML response

Sylvain Hallé

ItemSearchResponse[ [ Item[ Title[string], Artist[string] ]{0,¥} ]]

Items<ItemSearchResponse> <Items>

Help!The Beatles

<Item>

</Item> ...

XML request

XML response

Sylvain Hallé

ItemSearchResponse[ [ Item[ Title[string], Artist[string] ]{0,¥} ]]

Sylvain Hallé

WSDL: Web Service Description Language

CartCreate[ [int], [int], [ Item[ Title[string], Artist[string] ]{0,¥} ]]

ItemsSessionKeyItems

ItemSearchResponse[ [ Item[ Title[string], Artist[string], ]{0,¥} ]]

CartCreateResponse[ [int], [int], [ Item[ Title[string], Artist[string] ]{0,¥} ]]

SessionKeyCartIdItems

Sylvain Hallé

http://webservices.amazon.com/AWSECommerceService/AWSECommerceService.wsdl

https://www.paypal.com/wsdl/PayPalSvc.wsdl

http://api.google.com/GoogleSearch.wsdl

WSDLs for real world web services

Sylvain Hallé

</ItemSearch>

beatles 1234<Artist> </Artist><Bizbiz> </Bizbiz>

Sylvain Hallé

</ItemSearch>

ItemSearch[ [string]]Artist vs.?

Sylvain Hallé

</ItemSearch>

Sylvain Hallé

</ItemSearch>

Sylvain Hallé

Sylvain Hallé

What happened?

Sylvain Hallé

What happened?

Sylvain Hallé

What happened?

Sylvain Hallé

What happened?

Sylvain Hallé

What happened?

Sylvain Hallé

2What happened?

Sylvain Hallé

What happened?

Sylvain Hallé

What happened?

Sylvain Hallé

What happened?

Sylvain Hallé

Interface contracts

All messages comply with the WSDL but...

Sylvain Hallé

Interface contracts

You cannot add the same itemtwice to the shopping cart

Sylvain Hallé

Interface contracts

You cannot add the same itemtwice to the shopping cart

Sylvain Hallé

Interface contracts

Sylvain Hallé

Interface contracts

Sylvain Hallé

Free-form messages

Stateful interactions, stateless protocols

No uniform contract notation

Constraints at message level

XML, but that’s about it. No assumptions on nesting,degree, etc.

HTTP / SOAP define only message structureNo protocol enforces sequential constraints

Plain-text documentation... but OWL, RDF, ...

Components are black boxes (e.g. Amazon)

What are the issues?

Sylvain Hallé

The big question

Prevent contract

violations

Sylvain Hallé

1. A priori certification

A trustworthy authority assesses the client’s compliance to the contract...

A first solution

Testing, staticverificationetc.

Sylvain Hallé

A trustworthy authority assesses the client’s compliance to the contract...

...and grants a digital certificate

A first solution

Sylvain Hallé

The service needs a certificate to start an exchange with a client

A first solution

Sylvain Hallé

The service needs a certificate to start an exchange with a client

Example: iPhone app certification

A first solution

Sylvain Hallé

Problem: the client can change after certification

iPhone jailbreaking,Javascript prototype hijacking, ...

A first solution

Sylvain Hallé

Proposed approach

2. Client-side RuntimeMonitoring

A separate process checks each message...

CONTRACT

Sylvain Hallé

CONTRACT

Proposed approach

Sylvain Hallé

The message is relayed to the web service proper when it complies with the contract

Proposed approach

Sylvain Hallé

...and is discarded when it violates the contract

Proposed approach

Sylvain Hallé

A web service interacts with a web client through the exchange of semi-structured XML documents called

The service and client are generally designed by

No verification is done on the incoming and outgoing messages: possible between sent and expected messages (in both directions)

A priori checking of a client for compliance isvery hard, if not impossible

Runtime monitoring is a possible solution

messages

different organisations

mismatch

Summary (I)

Sylvain Hallé

NOSHOW

Part Two

Interface contracts inweb applications

Sylvain Hallé

Interface contracts

All possible sequences of all possibles messages with all possible values

Sylvain Hallé

Interface contracts

Constraintson individualmessages

Sylvain Hallé

Interface contracts

Constraintson sequencesConstraints

on individualmessages

Sylvain Hallé

Interface contracts

Constraintson sequences

Data-awaresequential constraints

Sylvain Hallé

Interface contracts

Interface contract =valid (error-free) interactions

Constraintson sequences

Data-awaresequential constraints

Sylvain Hallé

Interface contracts

As a tutorial tool, the Beep Store’s JavaScript client can be told to ‘‘forget’’ elements of the service’s interface contract

TheBeepStore

Sign in or register

What is this?

Ask for account

Contact us

Fault parameters

Search: Your Cart

Fault parameters

Don’t check Results’s typeIn the detailed search form, sends an ItemSearch message withoutchecking that the Results element is an integer.

"Add to cart" enabled if item present in cartMakes the "Add to cart" button available for items that are already in theuser's cart.

Message schemas

Cart manipulations

Highlightsdocumentation

Disables theverification

Sylvain Hallé

NOSHOW

Interface contracts

Dave, my mindis going...

As a tutorial tool, the Beep Store’s JavaScript client can be told to ‘‘forget’’ elements of the service’s interface contract

Sylvain Hallé

Constraints on individual messages

Examples:

Three types of constraints (I)

SHOWSHOW

<Message> <Action>ItemSearch</Action> <Results>5</Results> <Keyword>beatles</Keyword> <Page>1</Page></Message>

Sylvain Hallé

Examples:

SHOWSHOW

1. The element must be an integer between 1 and 20.Page "/M

Sylvain Hallé

SHOWSHOW

1. The element must be an integer between 1 and 20.

2. The element is mandatory only if is present,otherwise it is forbidden.

Page Results

Examples:

Sylvain Hallé

Expressing data constraints

Simple XPathFetches portions of an XML document according to aquery path = sequence of tags

: set of messages: set of XML query paths: set of atomic values

: ́ ® 2

Examples:(‘‘/a/b/c’’, m) = {1,2,4}(‘‘/a/b/d’’, m) = Æ

Sylvain Hallé

XPath termExpresses properties over values fetched by XPath expressions

For some message Î , path Î ,

" x : j(x) Û j(v) for every Î ( , )

$ x : j(x) Û j(v) for some Î ( , )

Examples:" x : x < 5/a/b/c

$ x :/a/b

$ x : " y : y £ x/a/b/c /a/b/c

Sylvain Hallé

Page Results

Sylvain Hallé

1. " x : x > 0 Ù x < 21/Message/Page

Page Results

Sylvain Hallé

1. " x : x > 0 Ù x < 21/Message/Page

2. $ x : Û $ y : /Message/Page /Message/Results

Sylvain Hallé

Constraints on message sequences

Examples:

<Message> <Action> Login </Action> ...</Message>

<Message> <Action> LoginResponse </Action> ...</Message>

<Message> <Action> CartCreate </Action> ...</Message>

Three types of constraints (II)

Sylvain Hallé

Examples:

3. The request cannot be resent if its response is

successful..

Login "/

Sylvain Hallé

Examples:

successful..

4. must follow a successful LoginResponse.

CartCreate

Sylvain Hallé

Linear Temporal Logic

Alphabet (A)Set of possible messages

Trace (A*)Sequence of messages

Sylvain Hallé

LTL formula = assertion on the of states in a tracesequence

a "always a" a "a in the next" a "eventually a"

a b "a until b"

G (a ® b)X (d cÚ e) WØFALSE TRUE

. . .A A EC CDB B

Sylvain Hallé

Well-known results:

1. For every LTL formula j, there exists a Büchi automaton Asuch that for every (infinite) trace s:

i.e. LTL describes languages

2. The alphabet symbols can be generalized to finite sets ofBoolean propositions

w-regular

Þ Let’s use XPath terms as our Boolean propositions

s |= j Û s Î L(A )j

Sylvain Hallé

Examples:

successful..

CartCreate

Sylvain Hallé

Examples:

3. (" a : a = LoginResponse ®/Message/Action

( " a’ : a’ ¹ Login))/Message/Action.

CartCreate "/

Sylvain Hallé

Examples:

( " a’ : a’ ¹ Login))/Message/Action.

CartCreate "/

Xpath terms

Sylvain Hallé

Examples:

( " a’ : a’ ¹ Login))/Message/Action

4. (" a : a ¹ CartCreate)/Message/Action

(" a’ : a’ = LoginResponse)/Message/Action

Xpath terms

Sylvain Hallé

The verification can be separated in two steps

1. Temporal stepDetermine termporal relationships to current message

2. Data stepEvaluate relevant XPath terms on message

Sylvain Hallé

Runtime monitoring

Gerth, Peled, Vardi, Wolper (PSTV 1995): construction of a Büchi automaton from a given LTL formula j

Benefit:

" ": automaton states are built as thetrace is readon-the-fly

Sylvain Hallé

Runtime monitoring

Benefit:

Sylvain Hallé

Runtime monitoring

Benefit:

Sylvain Hallé

Runtime monitoring

Benefit:

Sylvain Hallé

Runtime monitoring

Benefit:

Sylvain Hallé

Runtime monitoring

s = ab

Benefit:

Sylvain Hallé

Runtime monitoring

s = ab

Benefit:

Sylvain Hallé

Runtime monitoring

s = aba

Benefit:

Sylvain Hallé

Runtime monitoring

s = aba

Benefit:

Sylvain Hallé

Benefit:

" ": automaton states are built as thetrace is read

Dead end: formula is false

on-the-fly

Runtime monitoring

s = aba

Sylvain Hallé

Runtime monitoring

Algorithm overview:

1. An LTL formula is decomposed into nodes of the form

sub-formulas thatmust be true now

sub-formulas that mustbe true in the next state

Sylvain Hallé

Algorithm overview:

1. An LTL formula is decomposed into nodes of the form

Example:

sub-formulas thatmust be true now

sub-formulas that mustbe true in the next state

Runtime monitoring

Sylvain Hallé

2. Negations pushed inside (classical identities + dual of U = V)

3. At the leaves, G contains atoms + negations of atoms:we evaluate them

Verdict:

! All leaves contain : formula is false! A leaf is : formula is true! Otherwise:

4. Next event: D copied into G and we continue

FALSEempty

Runtime monitoring

Sylvain Hallé

Example: G (a ® b)X

Runtime monitoring

Sylvain Hallé

G (a ® b)X ?

Runtime monitoring

Sylvain Hallé

G (a ® b)X ?

a ® bX G (a ® b)X?

Runtime monitoring

Sylvain Hallé

G (a ® b)X ?

Øa G (a ® b)X?

a ® bX G (a ® b)X?

Runtime monitoring

Sylvain Hallé

G (a ® b)X ?

a, X b G (a ® b)X?Øa G (a ® b)X?

a ® bX G (a ® b)X?

Runtime monitoring

Sylvain Hallé

G (a ® b)X ?

a, X b G (a ® b)X?

a G (a ® b), bX?

Øa G (a ® b)X?

a ® bX G (a ® b)X?

Runtime monitoring

Sylvain Hallé

a G (a ® b), bX?

Øa G (a ® b)X?

Runtime monitoring

Sylvain Hallé

a G (a ® b), bX?

Øa G (a ® b)X?

Runtime monitoring

Sylvain Hallé

a G (a ® b), bX?

Øa G (a ® b)X?

Runtime monitoring

Sylvain Hallé

a G (a ® b), bX?

Runtime monitoring

Sylvain Hallé

G (a ® b), bX?

Runtime monitoring

Sylvain Hallé

?G (a ® b), bX

G (a ® b), bX?

Runtime monitoring

Sylvain Hallé

a, X b, b G (a ® b)X?

a, b G (a ® b), bX?

Øa, b G (a ® b)X?

a ® b, bX G (a ® b)X?

?G (a ® b), bX

Runtime monitoring

Sylvain Hallé

a, b G (a ® b), bX?

Øa, b G (a ® b)X?

Runtime monitoring

Sylvain Hallé

a, b G (a ® b), bX?

Øa, b G (a ® b)X?

Runtime monitoring

Sylvain Hallé

Øa, b G (a ® b)X?

Runtime monitoring

Sylvain Hallé

s = ac

Øa, b G (a ® b)X?

Runtime monitoring

Sylvain Hallé

s = ac

Øa, b G (a ® b)X?

Runtime monitoring

Sylvain Hallé

s = ac

No way to extend the trace:formula is false

Runtime monitoring

Sylvain Hallé

Data-aware sequential constraints

Examples:

5. There can be at most one active cart ID per session key."/

Three types of constraints (III)

Sylvain Hallé

Examples:

5. (" k : " c : /Message/SessionKey /Message/CartId

(" k’ : " c’ : /Message/SessionKey /Message/CartId

k = k’ ® c = c’))

Sylvain Hallé

Examples:

k = k’ ® c = c’))

Sylvain Hallé

k = k’ ® c = c’))

Sylvain Hallé

·XPath terms and temporal operators aremixed

·Not just ‘‘LTL with syntactical sugar’’.

·Not just a pathological case

k = k’ ® c = c’))

Sylvain Hallé

Examples:

6. You cannot add the same item twice to the shopping cart."/

<Message> <Action>CartAdd</Action> <Items> <Item> <ItemId>567</ItemId> ...

Sylvain Hallé

Examples:

6. (" a : a = CartAdd ®/Message/Action

" i : (" a’ :/Message/ItemId /Message/Action

a’ = CartAdd ® " i’ : i ¹ i’ ))/Message/ItemId

<Message> <Action>CartAdd</Action> <Items> <Item> <ItemId>567</ItemId> ...

Sylvain Hallé

Quantification must be relative to the values in the current message, and not the whole set V of possible values!

Example: ‘‘In every message, the a parameter must equal the b parameter’’. Suppose V = {1,2}, and classical first-order quantification.

Runtime monitoring

" x : " y : x = ya b

(" y : 1 = y) Ù (" y : 1 = y)b b

( 1 = 1) Ù ( 1 = 2) Ù ( 1 = 1) Ù ( 1 = 2)

Contradiction

G G G G

Sylvain Hallé

LTL-FO+

current

(Hallé & Villemaire, EDOC 2008)Extension of LTL with (limited) first-order quantification on message elements

·Boolean and LTL operators keep their original meaning·An XPath term is always meant to refer to the

message in the trace

Runtime monitoring

Sylvain Hallé

Adaptation of the runtime monitoring algorithm to handle LTL-FO+:

1. Atoms become equality tests

2. Decomposition rules for quantifiers

(and vice versa)

Runtime monitoring

Sylvain Hallé

Six constraints for the Beep Store

Data-aware constraints

Sylvain Hallé

Page Results

Sylvain Hallé

3. The request cannot be resent if its response issuccessful.

Page Results

CartCreate

Sylvain Hallé

3. The request cannot be resent if its response issuccessful.

5. There can be at most one active cart ID per session key.

6. You cannot add the same item twice to the shopping cart.

Page Results

CartCreate

Sylvain Hallé

Why are web service contracts special?

1. Presence of data-aware constraints

·Cannot separate data part from temporal partin specification AND enforcement

2. Complex messages

·Arbitrary nested structure·Cannot say ‘ ItemId’’:

there are many!·Rules out languages that

merely freeze a value in avariable

‘the

<Message> <Action>CartAdd</Action> <Items> <Item> <ItemId>567</ItemId> ... </Item> <Item> <ItemId>789</ItemId> ... </Item> ... </Items></Message>

Sylvain Hallé

Enforcing interface contracts at runtime

XMLHttpRequest

·JavaScript object·Provided by the browser·All communications to monitor

already centralized: ‘‘no’’instrumentation

Sylvain Hallé

XMLHttpRequestBB

Sylvain Hallé

XMLHttpRequestBB

XMLHttpRequestLTL-FO+algorithm

·Wrapper around original·Provides same methods·Checks messages before

relaying them

Sylvain Hallé

Add BeepBeep to an application

myapplication.html

</head><body>

</body></html>

My Application

text/javascriptmyapplication.js

�Include BeepBeep

Copy BeepBeep in the application's directoryhttp://beepbeep.sourceforge.net

Sylvain Hallé

myapplication.html

</head><body>

</body></html>

My Application

�Include BeepBeep

Sylvain Hallé

myapplication.html myapplication.js

</head><body>

</body></html>

My Application

// Initializations

req XMLHttpRequest

... req. some_message

function

�Include BeepBeep

Sylvain Hallé

beepstore.html beepstore.js

</head><body>

</body></html>

My Application

// Initializations

... req. some_message

function

XMLHttpRequestBB

Include BeepBeep

Sylvain Hallé

Create a with LTL-FO+ formulascontract file ?

# -------------------------------------------------------# BeepBeep contract file for the Beep Store# -------------------------------------------------------

% The element Page must be an integer between 1 and 20.

% The element Page is mandatory only if Results is present, otherwise it is forbidden.

% The Login request cannot be resent if its response is successful.

; ( p /Message/Page (((p) > ({0})) ((p) < ({21}))))

; ( a /Message/Action (((a) = ({ItemSearch})) ( (( r /Message/Results ({TRUE}))

( p /Message/Page ({TRUE}))) (( p /Message/Page ({TRUE})) ( r /Message/Results ({TRUE}))))))

; ( a /Message/Action (((a) ({LoginResponse})) ( ( ( b /Message/Action ( ((b) ({Login}))))))))

[ ]< >< >

< >< >

[ ][ ]

= ->! =

Caption: usedwhen violationsare discovered

Plain-textLTL-FO+(automaticallyparsed)

Sylvain Hallé

When loading the application, BeepBeep starts as a smallJava applet inside the page

TheBeepStore

Sign in or register

What is this?

Ask for account

Contact us

Fault parameters

Search: Your Cart

?/?/?/?/?/?:0:0

Sylvain Hallé

When loading the application, BeepBeep starts as a smallJava applet inside the page

TheBeepStore

Sign in or register

What is this?

Ask for account

Contact us

Fault parameters

Search: Your Cart

?/?/?/?/?/?:0:0

Sylvain Hallé

BeepBeep’s visible interface

?/?/?/?/?/?:0:0

Current state of monitorfor each property

Number ofmessagesprocessed

Cumulativeprocessingtime (in ms)

T: last message made it truet: is trueF: last message made it falsef: is false?: not yet true/false

Sylvain Hallé

An interface contract provides constraints cover the of each XML message, their and their

An extension of Linear Temporal Logic including a limited form of quantification over message elements specifies them

of these constraints can be doneefficiently, even with quantification

BeepBeep is a tool that allows it with on real applications

formatcontents ordering

Runtime monitoring

minimal modifications

http://beepbeep.sourceforge.net/

Summary (II)

Sylvain Hallé

Bounded-memory fragments of LTL

The forward-only fragment of LTL(Hallé & Villemaire, SAC 2009)

Applications to runtime monitoring of Java programs

Java-MOP plugin under construction

Symbolic (rather than explicit) handling of quantification

LTL with past operators

Standard web service mechanism for interface contracts?

Open issues and interesting questions

Sylvain Hallé

In client-side monitoring...

Sylvain Hallé

In client-side monitoring...

...the server has no guarantee that monitoring actually takes place

Sylvain Hallé

In server-side monitoring...

Sylvain Hallé

In server-side monitoring...

Too many clients may overwhelm the server’s verification process

Sylvain Hallé

Processing savings ofclient-side monitoring

Guarantees of server-sidemonitoring

Sylvain Hallé

Processing savings ofclient-side monitoring

COOPERATIVERUNTIME MONITORING

Best paper award

S. Hallé, Cooperative runtime monitoringof LTL Interface Contracts. Proc. EDOC 2010.Guarantees of server-side

monitoring

COOPERATIVERUNTIME MONITORING

runtime verification for the web (rv 2010 tutorial)

Technology

polylarva technology agnostic runtime verification

runtime verification, inc. business plan

path-aware time-triggered runtime verification

runtime verification: a computer architecture perspective

a runtime verification system for software defined networks

managing state explosion through runtime verification sharad...

course 8 adrian iftene...

interactive runtime verification - when interactive ... ·...

radiocommunication service testers procedure_rst-430.pdf ·...

runtime verification of executable...

runtime verification based on executable models: on-the-fly...

1020 - code execution and runtime verification

runtime verification of the ariac competition: can a robot

portable runtime verification with smartphones and optical...

rv : a runtime verification framework for monitoring,...

challenges in high-assurance runtime verification

leveraging dtrace for runtime verification - forside ·...

a survey of challenges for runtime verification from ...of...

haki: a runtime verification tool javascript mvc web

roster verification rv presentation to school administrators...