rhle: automatic verification of -hyperproperties · rhle forms the basis for an algo-rithm capable...

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

RHLE: Automatic Verification of ∀∃-HyperpropertiesRob DickersonPurdue University

[email protected]

Qianchuan YePurdue [email protected]

Benjamin DelawarePurdue [email protected]

AbstractSpecifications of program behavior typically consider sin-gle executions of a program, usually requiring that everyexecution never reaches a bad state (a safety property) orthat every execution can eventually produce some goodstate (a liveness property). Many desirable behaviors, how-ever, including refinement and non-interference, range overmultiple executions of a program. These sorts of behaviorsare instead expressible as a combination of k-safety and k-liveness hyperproperties. Relational program logics allow forreasoning about the validity of hyperproperties, but, justas Floyd-Hoare logics focus on axiomatic reasoning aboutsafety, existing relational logics have focused on proving k-safety properties. Such relational logics are unfortunately notsuitable for verifying more general combinations of k-safetyand k-liveness hyperproperties.

This paper presents RHLE, a relational program logic forreasoning about a class of such hyperproperties that weterm ∀∃-hyperproperties. RHLE forms the basis for an algo-rithm capable of automatically verifying this class of hyper-properties. We present an implementation of this algorithmwhich we have used to automatically verify a number of ∀∃-hyperproperties, including refinement and non-interferenceproperties, on a corpus of representative programs.

Keywords relational program logic, k-liveness, hyperprop-erties

1 IntroductionThe goal of program verification is to prove that a programis correct with respect to some specification of its intendedbehavior, e.g. that an implementation of quicksort alwaysreturns a sorted list or that a program is free of any divide-by-zero errors. The formal methods community has madegreat strides in the realm of automatic program verification,to the point that it is now possible to certify the correctnessof a wide variety of real-world systems [1, 2, 5]. The vastmajority of these techniques have focused on specificationsof the individual executions of a program.Many desirable program behaviors fall outside this class,

however, requiring specifications that can reference multipleprogram executions. Examples of such relational properties,or hyperproperties [13], include observational equivalenceof two programs, the non-leaking of sensitive informationvia program outputs, and the refinement of one program

PL’18, January 01–03, 2018, New York, NY, USA2018.

by another. Observational equivalence is an example of ak-safety hyperproperty, as it requires k executions to neverresult a “bad” state. Here, the “bad state” is one in whichthe two programs output different values after starting inthe same state. Program refinement, on the other hand, isan example of a k-liveness hyperproperty, as it requires kexecutions to possibly end in some “good” state. In this case,the “good state” is one where the original program outputsthe same value as the refined program.Two broad categories of analyses have been developed

to reason about hyperproperties axiomatically: relationalprogram logics [3, 9, 10, 21], which directly reason aboutthe executions of multiple programs and product program-based approaches [6, 7, 17], which attempt to reduce theproblem of reasoning about multiple programs to reasoningabout a single program which approximates their behaviors.Section 7 compares our proposed program logic to these ex-isting solutions in more detail, but existing approaches havelargely focused exclusively on the verification of k-safetyproperties. One exception, by Barthe et al. [8], develops aset of necessary conditions for a product program to cap-ture hyperproperties beyond k-safety, but does not discuss apractical means of constructing such product programs.Several interesting program behaviors fall outside this

class of k-safety hyperproperties, however. In particular, thispaper considers a class of behaviors of the form: for all ex-ecutions of programs in set S1, there exists a execution forevery program in a set S2 such that the final states of bothexecutions satisfies some property. This paper addresses thegap in reasoning about these ∀∃-hyperproperties by introduc-ing RHLE1, a relational logic capable of proving these kindsof hyperliveness properties. We have realized this logic in anautomated tool, called ORHLE, which is capable of automat-ically verifying ∀∃-hyperproperties, and we have appliedORHLE to analyze a variety of program behaviors encodedas ∀∃-hyperproperties.

Motivating Example Consider a program that shuffles(randomly permutes) a list of integers and adds three to eachelement:Compare this to a program that sorts the list of integersbefore adding three to each element:

Because Sort will always return a list that Shuffle couldhave returned, when given the same input list, SortPlusThree’spossible outputs are a subset of ShufflePlusThree’s. Whenone program’s potential behaviors are a subset of another’s,

1Pronounced “really”1

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

PL’18, January 01–03, 2018, New York, NY, USA Rob Dickerson, Qianchuan Ye, and Benjamin Delaware

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

Algorithm 1: ShufflePlusThreeInput :list , a list of integersbegin

shu f f led ← Shuffle(list) ;for i ← 0 to Length(list) do

shu f f led[i] ← shu f f led[i] + 3 ;return shu f f led ;

Algorithm 2: SortPlusThreeInput :list , a list of integersbegin

sorted ← Sort(list) ;for i ← 0 to Length(list) do

sorted[i] ← sorted[i] + 3 ;return sorted ;

we will say that the first program refines the second. Inthis sense, Sort refines Shuffle and SortPlusThree refinesShufflePlusThree.Refinement is an interesting property because it tells us

that we can replace one program or procedure with anotherwithout expanding the set of possible end states. A safetyproperty that applies to a program also applies to its refine-ments. To be a valid refinement, for all executions of therefined program, there must exist an execution of the originalprogram with the same behavior. The prescription of somedesirable execution makes this a liveness property, and thefact that we must ensure the existence of at least one satisfy-ing execution of the original program for all executions ofthe refinement furthermore makes this a ∀∃-hyperproperty.

1.1 Our ContributionsIn summary, this paper makes the following contributions:

• We introduceHLE, a sound program logic for verifyingliveness properties (Section 3).• We extend HLE to the relational setting to build asound program logic for verifying ∀∃-hyperproperties(Section 4).• We present ORHLE, a tool based on RHLE capableof automatically verifying these ∀∃-hyperproperties(Section 5).• We apply ORHLE to analyze a variety of relationalprogram behaviors (Section 6).

The anonomized supplementary material includes the imple-mentation of ORHLE, the complete set of benchmarks fromSection 6, and a Coq formalization of the logics described inSections 2-4.

n ∈ N Constantsx ∈ Var Variablesf ∈ FunName Function Names

c ::= skip | x := A | x := f (A)Statements

| c ; c | if c then c else c

| while c do c end

a ::= n | x | a +AArithmetic Expressions

| a − a | a ∗ a

b ::= ⊤ | ⊥ | a = a | a < aBoolean Expressions

| ¬b | b ∧ b

F ::= def f (x) {c; return a} Function Definitions

Figure 1. Syntax of FunIMP

def f (x) {s; return e} ∈ I

Σ, I ⊢ σ ,a ⇓ v Σ, I ⊢ x 7→ v, s ⇓ σ ′

Σ, I ⊢ σ ′, e ⇓ v ′

Σ, I ⊢ σ , x := f (a) ⇓ [x 7→ v ′]σECallImpl

f < I Σ(f ) = (P,Q)

Σ, I ⊢ σ ,a ⇓ v P(v) Q(v,v)

Σ, I ⊢ σ , x := f (a) ⇓ [x 7→ v ′]σECallSpec

Figure 2. Semantics of function calls.

2 Preliminaries: Syntax and SemanticsWe begin by defining FunIMP, a core imperative languagewith function calls, and then consider a program logic forreasoning about 1-safety properties in this language. Figure 1presents the syntax of FunIMP, a standard IMP languageplus syntax for function calls and definitions. Each func-tion has a fixed arity and returns a single value (althoughORHLE relaxes the latter restriction.) The semantics of thislanguage is given as a big-step reduction relation, which isparameterized over two contexts used to evaluate functioncalls. The first, Σ ∈ FunName → Assert ×Assert , is a totalmapping from function names to assertions, or predicateson states, representing pre- and post-conditions. The sec-ond, I ∈ FunName ⇀ F , is a partial mapping from functionnames to function definitions. The semantics for statementsother than function calls is the same as standard presen-tations of IMP. The evaluation of function calls, shown inFigure 2, depends on whether or not an implementation off is available in I . If so, the function is evaluated accordingto its definition per ECallImpl. If not, the function can re-turn any value consistent with the postcondition given for

2

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

Short Title PL’18, January 01–03, 2018, New York, NY, USA

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

f in Σ, provided the parameters satisfy the correspondingprecondition.

FunIMP Example As an example, given the specificationcontext Σ ≜ RandBnd 7→ (⊤, 0 ≤ ret < x), the assignmenty := RandBnd(20) can evaluate to any final state where thevalue of y is non-negative and less than 20. When a specifi-cation’s precondition fails to hold or its postcondition is notsatisfiable, evaluation will fail to produce a result. For exam-ple, evaluatingy := RandBnd(0)with the above specificationcontext gets stuck since there is no ret such that 0 ≤ ret < 0.Modifying the specification context to include a precondi-tion that x be sufficiently large, say Σ ≜ RandBnd(x) 7→(0 < x, 0 ≤ ret < x), will also result in a stuck evaluationof y := RandBnd(0), but this time because the preconditionis not met. Finally, if the function definition context con-tains RandBnd(x) as I ≜ RandBnd(x) 7→ return 5, theny := RandBnd(20) will always evaluate to final state wherey = 5.

By equipping function symbols with appropriate pre- andpostconditions, FunIMP programs are allowed to nondeter-ministically evaluate to multiple final values. This seman-tics also allows us to precisely define what it means to linkin “valid” implementations of function specifications. Wesay that a function definition implements a specificationif, whenever its body is run in a starting state satisfyingthe precondition, every final state it produces satisfies thepostcondition:

Σ, I , def f (x){s; return e} |= (P,Q) ≡

∀v . P(v) ∧ Σ, I ⊢ x 7→ v, s ⇓ σ ′ ∧ Σ, I ⊢ σ ′, e ⇓ v ′

→ Q(v ′,v)

We can straightforwardly lift this property to define validcontexts:

I |= Σ ≡ ∀f ∈ I .Σ, I , I (f ) |= Σ(f )

Theorem 2.1. When executed in a safe initial state (i.e. onein which the preconditions of all function calls are satisfied)under a valid context, a program must produce a subset of thebehaviors resulting from running the program with an emptyimplementation context:

∀I Σ p σ σ ′. I |= Σ ∧ Safe σ p ∧ Σ, I ⊢ σ ,p ⇓ σ ′

→ Σ, · ⊢ σ ,p ⇓ σ ′

We can straightforwardly construct a sound and relativelycomplete program logic for FunIMP by augmenting the stan-dard Hoare logic for IMPwith the following rule for function

calls2:

Σ ⊢

PreΣf (x) ∧ ∀v .Post

Σf (v, x)

=⇒ Ψ[v/y]

y := f (x) {Ψ}

HSpec

The HSpec rule (and thus the logic itself) reasons aboutfunction calls strictly via their specifications in Σ. Whencombined with Theorem 2.1, however, the logic admits amodular reasoning principle for FunIMP programs:

Theorem 2.2. To show that a program p behaves correctly inan environment of function definitions I and specifications Σ,it suffices to prove that p is functionally correct with respect toΣ and that the functions in I meet their specifications:

∀I Σ p P Q . I |= Σ

∧ Σ ⊢ {P} p {Q}

∧ (∀σ . P(σ ) → Safe σ p) →

∀σ σ ′. P(σ ) ∧ Σ, I ⊢ σ ,p ⇓ σ ′→ Q(σ )

A direct corollary of this theorem is that function imple-mentations can also be verified using our program logic.

3 Existential ReasoningBefore progressing to the relational setting, we begin with adiscussion of what a “1-liveness” property looks like. Con-sider the following program:

hand := card1 + card2;while hand < 21 docard3 := drawCard();hand := hand + card3

end

(p1)

This simple program plays a round of blackjack by exam-ining the two cards it is dealt and then continuously drawingcards until it either hits 21 or busts. This rather aggressivestrategy is not safe in general, as the player is guaranteed togo bust some of the time. Particularly caviller players maynot care about that however, as long as this strategy is anon-losing one, i.e. it is guaranteed to work at least some ofthe time, regardless of the hand they were dealt.We can formally encode this behavior via the following

semantic existential Hoare triple:

Σ, · |= [1 ≤ card1 ≤ 11 ∧ 1 ≤ card2 ≤ 10] p1 [hand = 21]∃

where drawCard() is specified to return a card at random:

Σ (drawCard) = (⊤, 1 ≤ ret ≤ 11) (1)

In other words, for any initial state containing valid valuesfor cards card1 and card2, p1 could yield a winning hand.

2For convenience, we define the PreΣf and ProjΣf projection functions forretrieving the pre- and post-conditions for a function symbol f in thecontext Σ.

3

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385


386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

436

437

438

439

440

More formally, our semantic existential Hoare triples aredefined as:

Σ, I |= [P] p [Q]∃ ≡ ∀σ . P(σ ) → ∃σ′. Σ, I ⊢ σ ,p ⇓ σ ′ ∧Q(σ ′)

These triples encode liveness properties, in that they guar-antee a program could enter some desirable final state. Notethat this definition forces the existence of a terminating ex-ecution satisfying the postcondition. Standard (universal)Hoare triples bound the results of all possible executions,so their semantics also covers non-terminating runs. Ourexistential triples ensure the existence of a single executionhowever, so allowing nonterminating executions to satisfythe triple would considerably weaken its guarantees.

Existential Hoare Logic We now turn to the question ofhow to prove the validity of an existential Hoare triple. Giventhat Section 2 presented a sound and relatively completelogic for universal Hoare triples, it is reasonable to ask ifexistential triples can be encoded as equivalent universaltriples. Consider the following variation on our previousblackjack player, which only draws a single card:

if card1 + card2 < 21 thencard3 := drawCard();

else card3 := 0;hand := card1 + card2 +

card3;

(p2)

Showing that p2 can avoid going bust is equivalent to show-ing that the following universal Hoare triple is not valid.

Σ, · |= {1 ≤ card1 ≤ 11 ∧ 1 ≤ card2 ≤ 10} p2 {hand > 21}

The intuition is that if the final hand does not always exceed21, there must be some execution where it does not bust. Thisintuition does not hold in general, however:

Σ, · |= [P] c [Q]∃ ⇐⇒ ¬(Σ, · |= {P} c {¬Q})

To see why, consider the following variation on p2:

if card1 + card2 < 21 thencard3 := drawCard();

else card3 := 21;h := card1 + card2 + card3;

(p3)

Our universal Hoare triple remains invalid for p3, but thecorresponding existential Hoare triple does not hold:

¬(Σ, · |= [1 ≤ card1 ≤ 11 ∧ 1 ≤ card2 ≤ 10] p3 [h ≤ 21]∃)

The intuition is that the universal Hoare triple is invalid aslong as there is a single initial state which produces an finalstate that invalidates the postcondition. The postconditioncould hold for initial states which induce a different controlflow path, e.g. states leading to the else branch of p3.

We have therefore developed a program logic for reason-ing about liveness properties. This logic, called HLE, is pre-sented in Figure 3. The proof rules are the same as thosefor the previous universal logic, with the exception of the

Σ∃ ⊢

PreΣ∃f (o, x)

∧ ∃v .PostΣ∃f (v,o, x)

∧ ∀v .PostΣ∃f (v,o, x)

=⇒ Ψ[v/y]

y := f (x) [Ψ]∃

∃Spec

⊨ Φ =⇒ Φ′

⊨ Ψ′ =⇒ Ψ Σ∃ ⊢ [Φ′] c [Ψ′]∃

Σ∃ ⊢ [Φ] c [Ψ]∃∃Conseq

Σ∃ ⊢ [Φ] skip [Φ]∃∃Skip

Σ∃ ⊢ [Φ[a/x]] x := a [Φ]∃∃Assgn

Σ∃ ⊢ [Φ] c1 [Φ′]∃ Σ∃ ⊢ [Φ

′] c2 [Ψ]∃

Σ∃ ⊢ [Φ] c1; c2 [Ψ]∃∃Seq

Σ∃ ⊢ [Φ ∧ b] c1 [Ψ]∃ Σ∃ ⊢ [Φ ∧ ¬b] c2 [Ψ]∃

Σ∃ ⊢ [Φ] if b then c1 else c2 [Ψ]∃∃Cond

Σ∃ ⊢ [Φ ∧ b ∧ am = n] S [Φ ∧ am < n]∃

Σ∃ ⊢ [Φ ∧ am > 0] while c do S end [Φ ∧ ¬b]∃∃While

Figure 3. Proof rules for existential Hoare logic (HLE)

rule for function calls, ∃Spec. To understand this rule, wefirst consider how the existential behaviors of a function isspecified.To ground this discussion, consider the universal specifi-

cation for drawCard from Equation 1: (⊤, 1≤ret≤11). Thereare two immediate interpretations for this specification exis-tentially: a valid implementation of drawCardmust have thepotential to return every integer meeting the postcondition,or a valid implementation of drawCard must be able to re-turn some integer which meets the postcondition. (Note thatneither interpretation gives an exhaustive characterization ofthe return values of a valid implementation; implementationsare free to return other values as well.) Both interpretationshave their advantages: the second admits strictly more imple-mentations, while the first provides more guarantees aboutthe set of values an implementation can produce.

HLE attempts to split the difference by extending existen-tial function specifications to include an additional set ofarguments, which we call template parameters, which thepre- and postconditions can reference. The underlying in-tuition is that that each instantiation of these parameterscorresponds to a single required existential behavior. HLEthus uses a separate context of existential specifications in

4

441

442

443

444

445

446

447

448

449

450

451

452

453

454

455

456

457

458

459

460

461

462

463

464

465

466

467

468

469

470

471

472

473

474

475

476

477

478

479

480

481

482

483

484

485

486

487

488

489

490

491

492

493

494

495


496

497

498

499

500

501

502

503

504

505

506

507

508

509

510

511

512

513

514

515

516

517

518

519

520

521

522

523

524

525

526

527

528

529

530

531

532

533

534

535

536

537

538

539

540

541

542

543

544

545

546

547

548

549

550

P ∧ c1 + c2 < 21 =⇒ R Σ∃ ⊢ [R] c3 := drawCard() [c1 + c2 + c3 ≤ 21]∃∃Spec

Σ∃ ⊢ [P ∧ c1 + c2 < 21] c3 := drawCard() [c1 + c2 + c3 ≤ 21]∃∃Conseq

····················

P ∧ c1 + c2 ≥ 21 =⇒ c1 + c2 ≤ 21 Σ∃ ⊢ [c1 + c2 ≤ 21] c3 := 0 [c1 + c2 + c3 ≤ 21]∃∃Assgn

Σ∃ ⊢ [P ∧ c1 + c2 ≥ 21] c3 := 0 [c1 + c2 + c3 ≤ 21]∃∃Conseq

·····

Σ∃ ⊢ [P] if . . . end [c1 + c2 + c3 ≤ 21]∃∃Cond

Σ∃ ⊢ [c1 + c2 + c3 ≤ 21] h := c1 + c2 + c3 [h ≤ 21]∃∃Assgn

Σ∃ ⊢ [P] if c1 + c2 < 21 then c3 := drawCard() else c3 := 0 end;h := c1 + c2 + c3 [h ≤ 21]∃∃Seq

Figure 4. HLE proof of Σ∃ ⊢ [1 ≤ card1 ≤ 11 ∧ 1 ≤ card2 ≤ 10] p2 [hand ≤ 21]∃ in the context Σ∃(drawCard) =(t, 1 ≤ t ≤ 11, ret = t). We abbreviate cardi as ci , and hand as h. P is the precondition 1 ≤ card1 ≤ 11 ∧ 1 ≤ card2 ≤ 10, and Ris 1 ≤ 1 ≤ 11 ∧ ∃v .v = 1 ∧ ∀v .v = 1 =⇒ c1 + c2 +v ≤ 21.

order to reason about function calls: Σ∃ ∈ FunName →Var ×Assert ×Assert .

Example Existential Specifications Consider a functionrandBnd which should output a natural number less thanits argument x . One possible existential specification forrandBnd (which does not use any template variables) is:

Σ∃ (randBnd) = (·, 0 < x, 0 ≤ ret < x) (2)

A valid implementation of this specification must be ableto return at least one value less than x , but not necessarilyevery value in the range. Two valid implementations arethus randBnd(x) = x − 1 and randBnd(x) = 0. While not par-ticularly random, these two implementations indeed meetthe existential specification. Perhaps surprisingly, anothervalid implementation strategy is to non-deterministically re-turn either 0 or x . Since our existential specifications permitmore behaviors than those that meet the postcondition, eventhough this implementation could output a value outsidethe specified range, it could also output the in-bounds value0. As long as at least one possible output meets the post-condition, the function is considered to meet its existentialspecification.However, we may want to specify that an implementa-

tion of randBnd must be able to return every number in therange. To do so, we can utilize a template parameter in theexistential specification like so:

Σ∃ (randBnd) = (t, 0 < x ∧ 0 ≤ t < x, ret = t) (3)

Intuitively, this specification says randBnd must be able toreturn every t less than the argument x . Observe that thisrules out all the implementations given above; they all havevalues in this range that can never be returned. Assuming afunction rand capable of returning every integer, one possi-ble valid implementation is:

def randBnd(x) {r := rand();

if r < 0 or x ≤ r then ret = 0 else ret = r end}

Statistically, this randBnd is not very random as its outputis not uniformly distributed, but it is indeed able to returnany value in the range. Again, randBnd(x) = rand() is also avalid implementation, although it exhibits a superset of thespecified behaviors.Specifications with fewer template parameters permit

more implementations at the cost of weaker guarantees. Thetemplate parameters in (3) allows us to prove triples that (2)does not, such as:

∀I . Σ, I |= [⊤] y := randBnd(10) [y = 2]∃

This triple is invalid in the context (2) because a I might havea valid implementation of randBnd that never returns 2.To see why it is useful to allow valid implementation to

exhibit more behaviors, consider the function randPrime thatreturns a random prime number. A reasonable existentialspecification for randPrime is:

Σ∃ (randPrime) = (t, t = 2 ∨ t = 3 ∨ t = 5, ret = t)

This is good enough if a client program behavior only de-pends on randPrime being able to return certain prime num-bers at different call sites. An actual implementation coulduse randBnd(5), without needing to filter out composite num-bers.

Formally we say that a function definition is a valid imple-mentation of an existential specification if, for every instan-tiation of its template parameters, it can evaluate to a finalstate satisfying the postcondition from any starting state

5

551

552

553

554

555

556

557

558

559

560

561

562

563

564

565

566

567

568

569

570

571

572

573

574

575

576

577

578

579

580

581

582

583

584

585

586

587

588

589

590

591

592

593

594

595

596

597

598

599

600

601

602

603

604

605


606

607

608

609

610

611

612

613

614

615

616

617

618

619

620

621

622

623

624

625

626

627

628

629

630

631

632

633

634

635

636

637

638

639

640

641

642

643

644

645

646

647

648

649

650

651

652

653

654

655

656

657

658

659

660

meeting the precondition:

Σ, I , def f (x){s; return e} |=∃ (x, P,Q) ≡

∀x v . P(x,v)

→∃σ ′. Σ, I ⊢ x 7→ v, s ⇓ σ ′

∧ Σ, I ⊢ σ ′, e ⇓ v ′

∧Q(v ′,o,v)

The key insight behind how ∃Spec uses the existentialcontext is that it allows a proof derivation to freely choseany instantiation of template parameters, o, that verify theprogram. To demonstrate how this works in practice, con-sider the derivation tree, shown in Figure 4, of the existentialtriple about the blackjack player p2:

Σ∃ ⊢ [1 ≤ card1 ≤ 11 ∧ 1 ≤ card2 ≤ 10] p2 [h ≤ 21]∃

The application of ∃Spec on the top is particularly notewor-thy. By applying ∃Spec with the template parameter t = 1,it is straightforward to verify that the implication in thesubsequent application ∃Conseq rule holds.

HLE admits a modular reasoning theorem similar to The-orem 2.2:

Theorem 3.1. To show that a program p can produce a de-sired state from a given set of initial states when run in anenvironment of function definitions I and specifications Σ, itsuffices to prove that p can results in such a state in existentialcontext Σ∃, as long I is consistent with Σ∃, and Σ and Σ∃ areconsistent:

∀I Σ Σ∃ p P Q . I |=∃ Σ∃

∧ Σ∃ ⊢ [P] p [Q]∃

∧ Consistent Σ Σ∃

→ Σ, I |= [P] c [Q]∃

In order to connect the existential context used in HLEwith the universal context used in the semantics of FunIMP,this theorem requires that they are consistent, i.e. everybehavior required by Σ∃ meets the corresponding safetycondition in Σ:

Consistent Σ Σ∃ ≡ ∀ f P Q o P∃ Q∃.

Σ(f ) = (P,Q) ∧ Σ∃(f ) = (o, P∃,Q∃) →

(∀x σ .P∃(x,σ ) → P(σ ))

∧ ∀x v σ .Q∃(x,v,σ ) → Q(v,σ )

4 RHLE: A program logic for∀∃-hyperproperties

When k = 1, k-liveness properties are for the most part anintellectual curiosity. They become much more useful in therelational setting (where k > 2) or when considered in thecontext of universally quantified program executions. As asimple example of the latter, we might want to verify that

our blackjack player p1 has a winning strategy when playingagainst p2. This can be expressed as the following (semantic)relational Hoare triple:

Σ, Σ∃, · |=

⟨ 1 ≤ card11 ≤ 11∧ 1 ≤ card21 ≤ 10∧ 1 ≤ card22 ≤ 11∧ 1 ≤ card22 ≤ 10

⟩p2 ∼∃ p1

⟨ (hand2 > 21∨ hand2 < hand1

)∧ hand1 ≤ 21

⟩Following Sousa and Dillig [21], the pre- and post-conditionsof this triple can reference specific variables via a numericindex: e.g. hand2 in the postcondition refers to the value ofhand in the second program (p2 in this case). Executions ofevery program to the left of ∼

∃are universally quantified,

while those to the right are existentially quantified. Formally,the definition of this triple is:

Σ, Σ∃, I |= ⟨P⟩ p∀ ∼∃ p∃ ⟨Q⟩ ≡

∀σ∀ σ∃ σ′∀. P(σ∀,σ∃) ∧ σ∀,p∀ ⇓ σ

′∀→

∃σ ′∃. Σ, I ⊢ σ∃,p ⇓ σ

′∃∧Q(σ ′

∀,σ ′∃)

Figure 5 gives examples of other interesting hyperproper-ties encoded using these relational Hoare triples, includinga generalization of our winning strategy property and therefinement property discussed in Section 1. We also give anencoding of a Generalized Non-Interference property basedon [18] which says that given any two program executionswith potentially different High security inputs, there exists athird execution whose High security inputs are the same asthe first program, but whose Low security outputs are thesame as the second. (This guarantees that no unprivilegedoutput is ever serving as a way to distinguish between twosets of privileged state.) Lastly, we encode a property whichsays every input parameter to a program is meaningful inthe sense that the program can always reach at least twodifferent end states when all other parameters are fixed.RHLE is a relational program logic for reasoning about

the sorts of hyperproperties encoded by these semantic re-lational Hoare triples3. Figure 6 gives a complete listing ofthe proof rules of the logic. The first three rules are used toend derivations: the SkipIntro∀ and SkipIntro∃ rules areused to append a final skip statement to the end of universaland existential programs, respectively. These final skips actas flags that allow the Skip axiom to be applied to end aderivation after the proof has fully considered the behaviorof each program. The Step∀ and Step∃ rules both mimic thestandard sequence rule, first leveraging the logic for theirrespective (single) program logics to prove the topmost com-mand of one of the programs meets some postcondition Φ′,which is then used as the precondition for further relationalreasoning.

3To avoid clashing with FunIMP’s sequence operator, the rules use ⊛ toseparate programs on either side of ∼

∃.

6

661

662

663

664

665

666

667

668

669

670

671

672

673

674

675

676

677

678

679

680

681

682

683

684

685

686

687

688

689

690

691

692

693

694

695

696

697

698

699

700

701

702

703

704

705

706

707

708

709

710

711

712

713

714

715


716

717

718

719

720

721

722

723

724

725

726

727

728

729

730

731

732

733

734

735

736

737

738

739

740

741

742

743

744

745

746

747

748

749

750

751

752

753

754

755

756

757

758

759

760

761

762

763

764

765

766

767

768

769

770

∀ ∃ Precondition PostconditionWinning Strategy p1, . . . ,pn pn+1 Varies ∀i ≤ n. Objective(pi ) < Objective(pn+1)

Refinement p1 p2 input1 = input2 return1 = return2

Noninterference p1,p2 p3Low1 = Low2 ∧ Low2 = Low3 Low2 = Low3

∧ Hiдh1 = Hiдh3

Parameter Usage — p1, . . . ,pn,p ′1, . . . ,p

′n

∧1≤i≤n

parami ,i , param′i ,i ∧ returni , return

′i∧

1≤j≤n, j,iparami , j = param

′i , j

Figure 5. Relational encodings of various ∀∃-hyperproperties

Σ, Σ∃ ⊢ ⟨Φ⟩ skip ∼∃ skip ⟨Φ⟩Skip

Σ, Σ∃ ⊢ ⟨Φ⟩ χ1 ⊛ S ; skip⊛ χ2 ∼∃ χ3 ⟨Ψ⟩

Σ, Σ∃ ⊢ ⟨Φ⟩ χ1 ⊛ S ⊛ χ2 ∼∃ χ3 ⟨Ψ⟩SkipIntro∀

Σ, Σ∃ ⊢ ⟨Φ⟩ χ1 ∼∃ χ2 ⊛ S ; skip⊛ χ3 ⟨Ψ⟩

Σ, Σ∃ ⊢ ⟨Φ⟩ χ1 ∼∃ χ2 ⊛ S ⊛ χ3 ⟨Ψ⟩SkipIntro∃

Σ ⊢ {Φ} S1 {Φ′}

Σ, Σ∃ ⊢ ⟨Φ′⟩ χ1 ⊛ S2 ⊛ χ2 ∼∃∼∃ χ3 ⟨Ψ⟩

Σ, Σ∃ ⊢ ⟨Φ⟩ χ1 ⊛ S1; S2 ⊛ χ2 ∼∃ χ3 ⟨Ψ⟩Step∀

Σ∃ ⊢ [Φ] S1 [Φ′]∃

Σ, Σ∃ ⊢ ⟨Φ′⟩ χ1 ∼∃ χ2 ⊛ S2 ⊛ χ3 ⟨Ψ⟩

Σ, Σ∃ ⊢ ⟨Φ⟩ χ1 ∼∃ χ2 ⊛ S1; S2 ⊛ χ3 ⟨Ψ⟩Step∃

Figure 6. Existential relational Hoare logic inference rules.

As the step rules which drive verification delegate reason-ing to single program logics, the soundness of RHLE followsstraightforwardly from the soundness of those logics:

Theorem 4.1.

∀Σ Σ∃ p∀ p∃ P Q . ∧ Consistent Σ Σ∃

∧ (∀p ∈ p∀ σ . P(σ ) → Safe σ p)

Σ, Σ∃ ⊢ ⟨P⟩ p∀ ∼∃ p∃ ⟨Q⟩

→ Σ, Σ∃, · |= ⟨P⟩ p∀ ∼∃ p∃ ⟨Q⟩

5 VerificationThis section presents an algorithm based on RHLE for au-tomatically verifying ∀∃-hyperproperties. This algorithm isparameterized over the underlying program logic as well asa context of universal and existential function specifications.The general approach is to generate verification conditionsby reasoning backward from the relational postcondition byapplying Step∀ and Step∃ until all program statements havebeen processed. A final formula is then constructed which

ensures that the accumulated verification conditions are im-plied by the initial relational precondition. This formula canthen be checked using a solver for the underlying assertionlogic.

The procedure for generating verification conditions hastwo components; StmtVC, presented in Algorithm 3, for rea-soning about non-relational Hoare triples and RhleVC, pre-sented in Algorithm 4, for stepping through relational triples.Note that StmtVC handles both universal and existentialHoare triples by taking a quant parameter that indicateswhether the universal or existential logic should be used.(This parameter is only used for function calls and loops.)RhleVC operates by applying the Step rules to single outprogram statements that can be given to StmtVC, which willgenerate the appropriate verification conditions for the cho-sen statement. The recursion terminates when all statementshave been considered, at which point the output formula canbe discharged by a solver for the underlying theory.

For a ∀∃-hyperproperty over k programs, each applicationof RhleVC can choose k possible targets for Step. Differentselection strategies can affect the performance of the verifica-tion algorithm, with sophisticated strategies potentially be-ing able to discard irrelevant subprograms altogether. Ratherthan hardcode this strategy, RhleVC is parameterized overa ChooseStep function. This function accepts a collectionof quantified programs and returns the next statement tostep over, which quantifier group the statement came from,and the collection of programs in their post-step form. It isimportant to note that, because verification conditions forexistential executions may include existential quantifiers, inthe worst case it is necessary to completely step through allthe existential programs before targeting universal programs.This ensures existential quantifiers occur inside the scopeof the quantifiers from the universal executions, preservingthe “forall-exists” semantics of RHLE triples. This, however,does not preclude ChooseStep from selecting a universalstatement before an existential one in all cases, e.g. whenthe existential statement is deterministic.

5.1 Example: Verifying a Program Refinement

7

771

772

773

774

775

776

777

778

779

780

781

782

783

784

785

786

787

788

789

790

791

792

793

794

795

796

797

798

799

800

801

802

803

804

805

806

807

808

809

810

811

812

813

814

815

816

817

818

819

820

821

822

823

824

825


826

827

828

829

830

831

832

833

834

835

836

837

838

839

840

841

842

843

844

845

846

847

848

849

850

851

852

853

854

855

856

857

858

859

860

861

862

863

864

865

866

867

868

869

870

871

872

873

874

875

876

877

878

879

880

Algorithm 3: StmtVC(quant, stmt,Ψ)Inputs :quant , quantifier in which stmt appears

smnt , a FunIMP program statementΨ, a relational postcondition

Output :verification condition for stmt to meet Ψ inthe given quant context

beginmatch stmt :

case skip doreturn Ψ

case x := a doreturn Ψ[x 7→ a]

case y := f (x) doif quant = ∀ then

ψpre ← PreΣf (x)

ψpost ← PostΣf (y, x) =⇒ Ψ

returnψpre ∧ψpostelse

ψpre ← PreΣ∃f (o, x)

ψpost ← PostΣ∃f (v,o, x)

ψend ← ∃v .ψpostψsuf ← ∀v .(ψpost =⇒ Ψ[y 7→ v])

return ∃o.(ψpre ∧ψend ∧ψsuf )

case s1; s2 doΨ′← StmtVC(quant, s2,Ψ)return StmtVC(quant, s1,Ψ

′)

case if c then st else sf doψt ← StmtVC(quant, st ,Ψ)

ψf ← StmtVC(quant, sf ,Ψ)

return c =⇒ ψt ∧ ¬c =⇒ ψf

case while c do s end, I ,M dov ← FreeVars(stmt)

v ′← Fresh(v)

if quant = ∀ thenψbody ← StmtVC(quant, s, I)

elseψmeas ← M < M[v 7→ v ′]

ψbody ← StmtVC(quant, s, I ∧ψmeas)

ψloop ← ∀v′.(c ∧ I =⇒ ψbody )[v 7→ v ′]

ψend ← ∀v .(¬c ∧ I =⇒ Ψ)

return I ∧ψloop ∧ψend

To see how this algorithm works in practice, considerthe variation of the refinement example from Section 1 pre-sented in Figure 7. To formally verify that SortPlusThreeis a refinement of ShufflePlusThree, we need to check the

Algorithm 4: RhleVCInputs :Φ, a relational precondition

χ∀, universally quantified programsχ∃, existentially quantified programsΨ, a relational postcondition

Output :verification condition for ⟨Φ⟩ χ∀ ∼∃ χ∃ ⟨Ψ⟩begin

if empty(χ∀) ∧ empty(χ∃) thenreturn Φ =⇒ Ψ

else(quant, stmt, χ ′

∀, χ ′∃) ← ChooseStep(χ∀, χ∃)

Ψ′← StmtVC (quant, stmt,Ψ)

return RhleVC(Φ, χ ′∀, χ ′∃,Ψ′)

forall: refinement;exists: original;

prog original(list):shuffled[3] := call shuffle(list);ret_0 := shuffled_0 + 3;ret_1 := shuffled_1 + 3;ret_2 := shuffled_2 + 3;

endp

prog refinement(list):sorted[3] := call sort(list);ret_0 := sorted_0 + 3;ret_1 := sorted_1 + 3;ret_2 := sorted_2 + 3;

endp

Figure 7. ORHLE input listing for a simplification of the pro-gram refinement example from Section 1 with function spec-ifications and pre- and post-conditions elided. Since sort isa refinement of shuffle, the program verifies.

following ∀∃-hyperproperty:

Σ, Σ∃ ⊢ ⟨listsor t = listshuf ⟩

SortPlusThree ∼∃ShufflePlusThree

⟨returnsor t = returnshuf ⟩

where listsor t and listshuf are the inputs to SortPlusThreeand ShufflePlusThree respectively, and returnsor t andreturnshuf are the respective outputs.

The specification for Sort requires the output to be apermutation in ascending order:

Presor t (lin, lout ) 7→ ⊤

Postsor t (lin, lout ) 7→ permutation(lin, lout )

∧ ∀i ∈ {1 . . . size(lin)}.lout [i − 1] ≤ lout [i]

8

881

882

883

884

885

886

887

888

889

890

891

892

893

894

895

896

897

898

899

900

901

902

903

904

905

906

907

908

909

910

911

912

913

914

915

916

917

918

919

920

921

922

923

924

925

926

927

928

929

930

931

932

933

934

935


936

937

938

939

940

941

942

943

944

945

946

947

948

949

950

951

952

953

954

955

956

957

958

959

960

961

962

963

964

965

966

967

968

969

970

971

972

973

974

975

976

977

978

979

980

981

982

983

984

985

986

987

988

989

990

while the specification for Shuffle simply requires the out-put to be any permutation:

∀lt .Preshuf f le (lin, lout ) 7→ permutation(lin, lt )

Postshuf f le (lin, lout ) 7→ lout = lt

Note that since Shuffle appears in an existentially quan-tified program, its specification is given in terms of a uni-versally quantified template variable lt . This specificationshould be read as, “for any list lt , if lt is a permutation oflin , then the output lout of Shuffle can be lt .” A goal of theverification algorithm will be to ensure the existence of atleast one such permutation lt that meets whatever otherconstraints are imposed on it by the rest of the programanalysis.If we implement ChooseStep as a simple sequential step

through the existential program followed by the universalprogram, RhleVC will proceed by first stepping backwardthrough ShufflePlusThree. This will dispatch two sets ofcalls to StmtVC with quant = ∃; one for the addition state-ments and one for the call to Shuffle. The full verificationcondition is omitted here for brevity, but note that the ver-ification condition at this point is of the form ∃lt . . ., in-dicating verification will succeed only when the templatevariable lt can be suitably instantiated. RhleVC then pro-ceeds to step back through SortPlusThree, again trigger-ing two sets of calls to StmtVC to handle the additions fol-lowed by the function call. After these steps, no programstatements are left, and we are ready to predicate the re-sulting verification condition on the overall precondition:listsor t = listshuf . This gives the final verification conditionthe form (listsor t = listshuf ) =⇒ . . . =⇒ (∃lt . . . .). Withfree variables implicitly universally quantified, this final ver-ification condition has the expected ∀ . . . ∃ . . . form, and isready to be validated by a solver.

6 Implementation and EvaluationORHLE is an automated verifier for ∀∃-hyperproperty

based on Algorithm 4. It is implemented in Haskell and usesZ3 as its backend solver. ORHLE specifications are writtenin SMT-LIB2, and the tool operates over FunIMP programs.Input programs must be annotated with loop invariants andfunction specifications.When a set of programs fails to verify,ORHLE outputs a falsifying model for the verification con-ditions. ORHLE implementation for ChooseStep currentlysteps through each existential execution in order followedby each universal execution, a strategy that has proved suffi-cient for our benchmark programs.

We evaluated ORHLE on a set of representative programsthat exhibit or fail to exhibit certain ∀∃-hyperpropertiesdrawn from Figure 5. (See Figure 8 for an example ORHLEinput listing.) Our experimental results are summarized inFigure 9. Timings were taken using an 8th Generation In-tel Core i7-8750H CPU with 6 2.2GHz cores. The examples

// A program which sometimes (but not// always) leaks secure state.

forall: leakA[1], leakA[2];exists: leakE[3];

pre: (and(= leakA!1!high leakE!3!high)(= leakA!1!low leakA!2!low)(= leakA!2!low leakE!3!low));

post: (= leakA!2!ret leakE!3!ret);

expected: invalid;

prog leakA(high, low):r := call randInt()

@pre { true }@post { (and (>= r 0) (< r 100)) };

if (r == 50) thenret := high + low; // leak!

elseret := low;

endendp

prog leakE(high, low):r := call randInt()

@templateVars { n }@pre { (and (>= n 0) (< n 100)) }@post { (= r n) };

if (r == 50) thenret := high + low; // leak!

elseret := low;

endendp

Figure 8. ORHLE input listing for a generalized non-interference verification. The program has an execution paththat leaks secure state and so fails to verify.

included a mix of programs that were expected to verify orfail to verify, and ORHLE returned the expected result inall cases. Verification across this example set was reason-ably quick, with most verification tasks completing in under300ms.

7 Related WorkThis section discusses related work.

Hyperproperties The concept of a hyperproperty was orig-inally introduced by Clarkson and Schneider[13], buildingoff of earlier work by Terauchi and Aiken [22]. Clarkson andSchneider define hyperproperties as finite properties over a

9

991

992

993

994

995

996

997

998

999

1000

1001

1002

1003

1004

1005

1006

1007

1008

1009

1010

1011

1012

1013

1014

1015

1016

1017

1018

1019

1020

1021

1022

1023

1024

1025

1026

1027

1028

1029

1030

1031

1032

1033

1034

1035

1036

1037

1038

1039

1040

1041

1042

1043

1044

1045


1046

1047

1048

1049

1050

1051

1052

1053

1054

1055

1056

1057

1058

1059

1060

1061

1062

1063

1064

1065

1066

1067

1068

1069

1070

1071

1072

1073

1074

1075

1076

1077

1078

1079

1080

1081

1082

1083

1084

1085

1086

1087

1088

1089

1090

1091

1092

1093

1094

1095

1096

1097

1098

1099

1100

Time (s) VerifiedProgram RefinementAdd3Shuffled 0.23 ✗Add3Sorted 0.22 ✓ConditionalNonRefinement 0.21 ✗ConditionalRefinement 0.21 ✓LoopNonRefinement 0.21 ✗LoopRefinement 0.21 ✓PermInvRefinement 0.25 ✓SimpleNonRefinement 0.22 ✗SimpleRefinement 0.21 ✓

GNIDenning1[15] 0.23 ✓Denning2[15] 0.23 ✗NonDetLeak 7.01 ✗NonDetNonLeak 0.22 ✓SimpleLeak 0.20 ✗SimpleNonLeak 0.22 ✓Smith1[20] 0.22 ✗

Parameter UsageCompletelyUnused 0.20 ✗NonDetUnused 0.23 ✗NonDetUsed 0.28 ✓SemanticallyUnused 0.20 ✗ThreeUsed 0.19 ✓

Figure 9. ORHLE results.

potentially infinite set of execution traces, and demonstratethat every trace hyperproperty is an intersection of safetyand liveness hyperproperties. They further identify the classof k-safety hyperproperties as those in which the a bad statecannot involve more than k traces. While they discuss verifi-cation, they do not offer a verification algorithm.

Relational Program Logics Relational program logics area common approach to verifying relational specifications.Relational Hoare Logic[10] (RHLP) was one of the first ex-amples of these logics. Originally developed by Benton, RHLprovides an axiomatic system for proving 2-safety proper-ties of imperative programs. The logic is shown to be sound,but no automated verification algorithm is given. RelationalHigher-order Logic[3] is a higher-order relational logic forreasoning about higher-order functional programs expressedin a simply-typed λ-calculus. Probabilistic RHL [9] is a logicfor reasoning about probabilistic programs, with the goal ofproving security properties of cryptographic schemes likeHashed ElGamal encryption and the Cramer-Shoup cryp-tosystem. The relational logic closest to RHLE is CartesianHoare Logic[21] (CHL) developed by Sousa and Dillig. Thislogic which provides an axiomatic system for reasoningabout k-safety hyperproperties along with an automatic ver-ification algorithm. ORHLE can be though of as an extension

of CHL for reasoning about a larger class of hyperpropertiesvia an existentially quantified Hoare logic.

Product Programs Product programs are another com-mon technique for verification of relational properties. Thebasic technique is to transform multiple programs with arelational property into an equivalent single program with anon-relational property[7]. Standard verification tools maythen be used to verify the resulting product program. Theadvantage to this approach is it can leverage existing non-relational verification tools and techniques, but large statespace of product programs makes verification difficult inpractice. Product programs have been used to verify k-safetyproperties and reason about non-interference and secureinformation flow[6, 17]. The most closely related work inproduct programs was that of Barthe et al. [8], which devel-ops a set of necessary conditions for “left-product programs”.These product programs can be used to verify hyperprop-erties outside of k-safety, including the ∀∃-hyperpropertiesRHLE deals with, but the work does not address how toconstruct left-product programs.

Temporal Logics Temporal logics like LTL and CTL arepopular systems for verifying liveness properties, and vari-ants like HyperLTL and HyperCTL can be used to verifyliveness hyperproperties. (See [11, 12] for surveys.) A recentpaper by Coenen et al.[14] examines verification and synthe-sis of computational models using HyperLTL formulas withalternating quantifiers, leading to analysis of the kinds of∀∃-hyperproperties considered here. While these temporallogics are capable of reasoning about these hyperlivenessproperties, they operate over finite state models of softwaresystems instead of reasoning directly over program syntaxin the way RHLE does.

Dynamic Logic First-order dynamic logic[19] is a reinter-pretation of Hoare logic in first order, multi-modal logic. Dy-namic logic programs are constructed by regular expressionoperators extended by other operators like testing and as-signment. The more conventional programming constructs,such as the ones presented in this work, can be encoded interms of the regular operators. For each program S , there aremodal operators [S] and ⟨S⟩. A modal formula [S]P meansthat after executing S , all end states satisfy P , and the dual⟨S⟩P means that after executing S , there exists an end statesatisfying P . The axiomatic proof rules for dynamic logicclosely correspond to the weakest precondition transformer,in that [S]P ⇐⇒ wlp(S, P), where wlp is the weakest lib-eral precondition, and ⟨S⟩P ⇐⇒ wp∃(S, P), wherewp∃ isthe weakest precondition for existential Hoare logic. There-fore, the universal Hoare triple ⊢ {P}S{Q} corresponds toP =⇒ [S]Q , and the existential Hoare triple ⊢ [P]S[Q]∃corresponds to P =⇒ ⟨S⟩Q . In contrast to dynamic logic,we present a more conventional imperative language, andallow function calls, potentially without implementations.

10

1101

1102

1103

1104

1105

1106

1107

1108

1109

1110

1111

1112

1113

1114

1115

1116

1117

1118

1119

1120

1121

1122

1123

1124

1125

1126

1127

1128

1129

1130

1131

1132

1133

1134

1135

1136

1137

1138

1139

1140

1141

1142

1143

1144

1145

1146

1147

1148

1149

1150

1151

1152

1153

1154

1155


1156

1157

1158

1159

1160

1161

1162

1163

1164

1165

1166

1167

1168

1169

1170

1171

1172

1173

1174

1175

1176

1177

1178

1179

1180

1181

1182

1183

1184

1185

1186

1187

1188

1189

1190

1191

1192

1193

1194

1195

1196

1197

1198

1199

1200

1201

1202

1203

1204

1205

1206

1207

1208

1209

1210

The verification algorithm presented in this work also al-low the users to specify loop invariants, making the systemmore practical. In addition, we focus on the verification ofhyperproperties.

8 Future Work and ConclusionEfficiency Improvements CurrentlyORHLE uses a simpleimplementation of the ChooseStep subroutine Algorithm 4which steps sequentially through all existential programsbefore their universal counterparts. This approach builds aformula where each an existential trace needs to be exhibitedfor each universal trace, resulting in a potentially expensivecall to Z3. Although this was not an issue in our experimentalevaluation, we believe that a smarter selection strategy couldresult in a more efficient (and thus scalable) verification algo-rithm. More sophisticated stepping may also be able to elideverification of irrelevant program segments altogether usingstrategies similar to those employed in CHL[21]. Opportuni-ties to prune existentially quantified subprograms could beparticularly fruitful, as the verifier only needs to establishthe existence of one satisfying execution. Once a satisfyingexecution is established, the behavior of different executionpaths becomes irrelevant and do not need to be considered.

Existential Loop Invariant Inference ORHLE requiresthe user to specify all loop invariants. There is a wide varietyof existing work on loop invariant inference (see section 5 of[16] for a survey) that ORHLE should be able to apply to itsuniversally quantified programs, but to our knowledge thereis no work on inferring loop invariants in an existentiallyquantified context. We believe this is an interesting problemto investigate: since only one suitable execution path inthe loop body needs to preserve the invariant, inferring avalid invariant for an existentially quantified loop may besubstantially easier than doing the same in a universallyquantified context.

Existential Specification Inference ORHLE requires theuser to give full specifications for each function call, but forsome applications it may be useful to require only universalfunctional specifications from users and have the verifica-tion tool report the most permissive satisfying existentialspecification. If ORHLE were capable of proving that, e.g.,any function implementation that can return 2 and 6 resultsin the desired program behavior, it should become much eas-ier to prove function implementations meet their existentialspecifications. Albarghouthi et al.[4] have described a tech-nique for abducing maximal function specifications that maybe applicable here; by replacing existential functions withabducibles, the verification conditions generated by ORHLEcould form the basis of a useful multiabduction query.

In summary, this paper described RHLE, a program logicfor reasoning axiomatically about ∀∃-hyperproperties, aswell as an algorithm for automatically verifying this class of

hyperproperties. This verification algorithm has been usedto build ORHLE, a verification tool that operates over anarbitrary number of ∀∃-quantified program executions. Wehave used ORHLE to successfully verify or falsify a varietyof representative ∀∃-hyperproperties.

References[1] [n. d.]. How AWS uses automated reason-

ing to help you achieve security at scale.https://aws.amazon.com/blogs/security/protect-sensitive-data-in-the-cloud-with-automated-reasoning-zelkova/.

[2] [n. d.]. POPL 2019 Most Influential PaperAward for research that led to Facebook Infer.https://research.fb.com/popl-2019-most-influential-paper-award-for-research-that-led-to-facebook-infer/.

[3] Alejandro Aguirre, Gilles Barthe, Marco Gaboardi,Deepak Garg, and Pierre-Yves Strub. 2017. A RelationalLogic for Higher-order Programs. Proc. ACM Program.Lang. 1, ICFP, Article 21 (Aug. 2017), 29 pages.

[4] Aws Albarghouthi, Isil Dillig, and Gurfi. 2016. Maximalspecification synthesis. ACM SIGPLAN Notices 51, 1(2016), 789–801.

[5] Thomas Ball, Byron Cook, Vladimir Levin, and Sriram KRajamani. 2004. SLAM and Static Driver Verifier: Tech-nology transfer of formal methods inside Microsoft. InInternational Conference on Integrated Formal Methods.Springer, 1–20.

[6] Gilles Barth, Pedro R. D’Argenio, and Tamara Rezk.2011. Secure information flow by self-composition.Mathematical Structures in Computer Science 21, 6 (2011),1207–1252.

[7] Gilles Barthe, Juan Manuel Crespo, and César Kunz.2011. Relational Verification Using Product Programs.In FM 2011: Formal Methods, Michael Butler and Wol-fram Schulte (Eds.). Springer Berlin Heidelberg, Berlin,Heidelberg, 200–214.

[8] Gilles Barthe, Juan Manuel Crespo, and César Kunz.2013. Beyond 2-safety: Asymmetric product programsfor relational program verification. In International Sym-posium on Logical Foundations of Computer Science.Springer, 29–43.

[9] Gilles Barthe, Benjamin Grégoire, and SantiagoZanella Béguelin. 2009. Formal Certification of Code-based Cryptographic Proofs. SIGPLAN Not. 44, 1 (Jan.2009), 90–101.

[10] Nick Benton. 2004. Simple Relational CorrectnessProofs for Static Analyses and Program Transforma-tions. In Proceedings of the 31st ACM SIGPLAN-SIGACTSymposium on Principles of Programming Languages(POPL ’04). ACM, New York, NY, USA, 14–25.

[11] E. Clarke, O. Grumberg, and D. Long. 1994. Verificationtools for finite-state concurrent systems. In A Decade of

11

1211

1212

1213

1214

1215

1216

1217

1218

1219

1220

1221

1222

1223

1224

1225

1226

1227

1228

1229

1230

1231

1232

1233

1234

1235

1236

1237

1238

1239

1240

1241

1242

1243

1244

1245

1246

1247

1248

1249

1250

1251

1252

1253

1254

1255

1256

1257

1258

1259

1260

1261

1262

1263

1264

1265


1266

1267

1268

1269

1270

1271

1272

1273

1274

1275

1276

1277

1278

1279

1280

1281

1282

1283

1284

1285

1286

1287

1288

1289

1290

1291

1292

1293

1294

1295

1296

1297

1298

1299

1300

1301

1302

1303

1304

1305

1306

1307

1308

1309

1310

1311

1312

1313

1314

1315

1316

1317

1318

1319

1320

Concurrency Reflections and Perspectives, J. W. de Bakker,W. P. de Roever, andG. Rozenberg (Eds.). Springer BerlinHeidelberg, Berlin, Heidelberg, 124–175.

[12] Michael R Clarkson, Bernd Finkbeiner, Masoud Koleini,Kristopher K Micinski, Markus N Rabe, and CésarSánchez. 2014. Temporal logics for hyperproperties.In International Conference on Principles of Security andTrust. Springer, 265–284.

[13] Michael R. Clarkson and Fred B. Schneider. 2010. Hy-perproperties. J. Comput. Secur. 18, 6 (Sept. 2010), 1157–1210.

[14] Norine Coenen, Bernd Finkbeiner, César Sánchez,and Leander Tentrup. 2019. Verifying Hyperlive-ness. (07 2019), 121–139. https://doi.org/10.1007/978-3-030-25540-4_7

[15] Dorothy Denning and Peter Denning. [n. d.]. LectureNotes: Certification of Programs for Secure InformationFlow. https://people.cs.vt.edu/ ryder/6304/lectures/10-Denning&Denning-InfoFlow-CACM7-97-KrishnanMurali.pdf.

[16] Carlo A Furia, Bertrand Meyer, and Sergey Velder. 2012.A survey of loop invariants. CoRR (2012).

[17] Máté Kovács, Helmut Seidl, and Bernd Finkbeiner. 2013.Relational abstract interpretation for the verificationof 2-hypersafety properties. Proceedings of the ACMConference on Computer and Communications Security,211–222.

[18] John Mclean. 1996. A general theory of composition fora class of “possibilistic” properties. Software Engineering,IEEE Transactions on 22 (02 1996), 53 – 67. https://doi.org/10.1109/32.481534

[19] Vaughan R Pratt. 1976. Semantical consideration onfloyo-hoare logic. In 17th Annual Symposium on Foun-dations of Computer Science (sfcs 1976). IEEE, 109–121.

[20] Geoffrey Smith. 2007. Principles of Secure InformationFlow Analysis. In Malware Detection, Mihai Christodor-escu, Somesh Jha, Douglas Maughan, Dawn Song, andCliff Wang (Eds.). Springer US, Boston, MA, 291–307.

[21] Marcelo Sousa and Isil Dillig. 2016. Cartesian HoareLogic for Verifying K-safety Properties. In Proceedingsof the 37th ACM SIGPLAN Conference on ProgrammingLanguage Design and Implementation (PLDI ’16). ACM,New York, NY, USA, 57–69.

[22] Tachio Terauchi and Alex Aiken. 2005. Secure Infor-mation Flow as a Safety Problem. In Static Analysis,Chris Hankin and Igor Siveroni (Eds.). Springer BerlinHeidelberg, Berlin, Heidelberg, 352–367.

12

https://doi.org/10.1007/978-3-030-25540-4_7

https://doi.org/10.1007/978-3-030-25540-4_7

https://doi.org/10.1109/32.481534

https://doi.org/10.1109/32.481534

rhle: automatic verification of -hyperproperties · rhle forms the basis for an algo-rithm capable...

Documents