CMV: Automatic Verification of Complete Mediation for Java Virtual Machines

A. Prasad Sistla, V.N. Venkatakrishnan, M. Zhou, H. Branske

University of Illinois at Chicago

Introduction

Java language and Java Platform Java Language: With powerful features, gained

considerable success Java Platform: Enable users to write and run applications

written in Java languages directly on top it

Java Applications and Platforms are Used Everywhere

Almost every browser runs JVM and supports running Java Applets

Java Applet: A Java program that is run from inside a web browser

OS (e.g. SolarisTM, Linux, Windows)

Java Virtual Machine (such as HotSpot, Harmony)

Java libraries (including java.io, java.net, jave.lang and other packages)

Java compiler and other tools

Java applications

Java Platform

Internet Explorer browses a webpage containing a Java Applet

Webpage content:<APPLET

NAME="QuizMaster" CODE="QuizMaster.

class" ….>…. </APPLET>

An Example of Java Applet

- From http://www.realapplets.com/applets/quizmaster/

Risks

What can happen if a Java applet/Java program is allowed to access any system resources? Compromise the security of the local host

Disclose the confidential information: Transmit information

Destroy system files Display annoying pictures on a user screen

How the JVM prevents this from happening?

Java Fine-grained Access Control Model

Security Policies Map code to sets of permissions to access

security sensitive resources grant codeBase “URL” { permission java.io.FilePermission “/home/tmp/", "read"; }

Runtime Monitoring in JVM Through a call to library class

SecurityManager, before accessing any sensitive resources

A Typical Use of SecurityManagerpublic FileInputStream(File file) throws FileNotFoundException {

String name = (file != null ? file.getPath() : null);

// Existing Solutions SecurityManager security = System.getSecurityManager(); //

checks for SM

if (security != null) {security.checkRead(name); permission

check }

…… open(name); sensitive operation implemented as a

native method}

Potential Problem:Without checking the permissions, any file specified

by the application can be opened, leading to potential confidential information leaks.

Challenge: Ensure that the SecurityManager is consulted on all

paths that lead to sensitive operations

Our Goal

Ensure Java Standard Libraries are trustworthy They satisfy Complete Mediation Property:

SecurityManager is consulted on all paths that lead to sensitive operations

This Assurance will benefit millions of Java users

Sensitive Operations Implemented as native methods

Code that is specific to a hardware and operating system platform and are written in other languages, such as C and C++. Called by Java code.

Challenges to Our Goal:• Large code base - Thousands of classes in Java libraries• Existing Model Checking tools (e.g. SLAM, MOPS) are general purpose oriented, while our goal is specialized to verify complete mediation property. These generic tool may not be scalable to a large code base such as the Java libraries

Our Solution

Security Property Verifier

NoAll paths from public methods to sensitive operations are guarded by security checks.

YesSome paths from some public methods to sensitive operations are missing security checks.

Input:Java Standard Libraries

Output:

An automated static analysis technique.Sound. Not Complete

Our Contributions

A scalable automated model checking technique that verifies satisfaction of complete mediation in open systems

A tool, Complete Mediation Verifier (CMV) Efficiently analyzes JVM library classes

Experimental results in two shrink-wrapped JVM implementations -- HotSpot and Harmony VMs.

A Code Example to Walk through Various Verification Approaches

public void X () {

// do some operations

Y();

//sensitive-operationFileRd1();

}

private void Y() { if (….){ SecurityManager sm =

System.getSecurityManager();

if (sm != null) { sm.checkPermission(

FilePermission(pathname”read”)); }

// sensitive-operation FileRd2(); } else { // other operations }}

A branch w/o a security check

Risky

Analysis : Naive First Approach - private void Y() {0. if (….){1. SecurityManager sm =

System.getSecurityManager();2. if (sm != null) {3. sm.checkPermission(

FilePermission(pathname,

”read”)); } // sensitive-operation4. FileRd2(); }

else {5. // other operations }}

Y1

Y3

Y4

Y0

Y2

Y5

RET

safe

First Approach (Contd.)- Presence of Method Calls Expand and analyze CFG inline to make Extended

CFGs(ECFG)

Inline Expansion Drawbacks: Recursion Size of ECFG(M) – can be exponential

public void X(int x) {0: // do some operations

1: Y();

//sensitive-operation2: FileRd1();}

X0

X2

RET

X1

Invokes Y

CFG(X)

RET

Expand To

X0

X2

Y1

Y3

Y4

Y0

Y2

Y5

Y

At risk

Summarization

All_Path_Secure No paths to return node(s) w/o

security check. Otherwise, Insecure_Path

Good No paths to unguarded

sensitive operations. Otherwise, Bad

We store negations of these two properties

Method Summary A 2-tuple to store

<Insecure_Path, Bad> 4 kinds of values

<insecure_path, bad> <insecure_path, ┴> <┴, bad> <┴, ┴>

EN RT

EN

High-level idea: Analyze methods once; “summarize” and reuse the results

Observations to Compute Method Summary

A node that Invokes a bad

method M1

A sensitive node

equivalent

Invokes a All_Path-Secure and Good method

M1

A security check node

To compute Bad attribute:

equivalent

Invokes a Insecure_Path

method M1

a neutral node

To compute Insecure_path attribute:

equivalent

Second Approach –Computing Summaries of Methods

Meth. X

Meth. Y

Step 1: Construct call graph

Step 2: Find out Y needs to be analyzed first;The summary of the method computed first can then be reused.In presence of multiple methods, list the methods in reverse topological order

Given two methods X & Y. X invokes Y.

CFG(Y)Summary(Y) = <InSecure_Path, ┴ >

Step 3: Analyze CFG(Y) to Compute Summary(Y)

Y1

Y3

Y4

Y0

Y2

Y5

RET

safe

Second Approach (Contd.)

Step 4: - Compute Summary(X)

X0

X2

RET

X1

Invokes Y, Summary(Y) = <InSecure_Path, ┴ >

CFG(X)

invocation Node X1 is equivalent to a neutral node

X0

X2

RET

CFG(X)Summary(X) = <InSecure_Path, Bad>

At risk

Second Approach - Algorithm

Algorithm in summary:Construct call Graph;Get a list of methods in Reverse Topological

Order(RTO);For each method M in the list ordered in RTO

Compute Summary(M), reusing the method summaries of those invocation nodes;

Runtime complexity: O(ΣM) ΣM the sum of the sizes of the CFGs of all the

methods.

Expanded Second Approach- For Recursion Let us say M1 and M2 are mutually recursive methods

Analysis based on reverse topological order will not work

Solution: Repeat the previous algorithm, until all method

summaries reach a fix point. Worst case run time:

O(N* ΣM). Quadratic N the number of methods analyzed

Quadratic algorithms still problematic for a large code base such as libraries

How can we make this more efficient?

Meth. M1

Meth. M2A cycle exists in the call graph

Main Idea: Compute method summaries for all methods, at

the same time For Insecure_Path summaries, process all

the methods at the same time Start with entry nodes in a queue, and keep adding

successors to the queue as you explore

Linear run time complexity: O(ΣM)

ΣM sum of sizes of the CFGs of all methods

public void X(int x) {0: // do some operations

1: Z();

//sensitive-operation2: FileRd1();}

Our Approach – A New Efficient Solution (Part 1 – Compute Insecure_Path)

private void Z() {0: SecurityManager sm =

System.getSecurityManager();1: sm.checkPermission( FilePermission(pathname, ”read”)); //sensitive-operation2: FileRd2();}

X0

X1

RET

X1

Invokes ZZ0

Z1

RET

Z1

X0QStep 1: Add X0, Z0 to Q Z0

Step 2: X0 is a neutral node. Add its successor X1 to Q Z0Q X1

Step 3: Z0 is a neutral node. Add its successor Z1 to Q X1Q Z1

Step 4: X1 is a method invocation node. Add X1 to WQ(Z).

Z1Q

X1WQ(Z)

Step 5: Z1 is a security check. Stop exploring the path.

Q

X1WQ(Z)

Step 6: Q is empty. Terminate the algorithm. Both X and Z are All_Path_Secure.

Main Idea (Similar to the computing of Insecure_Path ) process all the methods at the same time.

Different in processing of each node in the queue Q for a sensitive operation, label method summary as

bad for a node invoking a method M’,

If Insecure_Path(M’) is true, put all successors to the Q

Put the node to a waiting queue of M’ Whenever the bad summary of M’ is computed, process

the node based on summary of M’

Linear run time complexity: O(ΣM) O(ΣM) sum of sizes of the CFGs of all methods

public void X(int x) {0: // do some operations

1: Z();

//sensitive-operation2: FileRd1();}

Our Approach - A New Efficient Solution (Part 2 – Compute Bad Summaries)

private void Z() {0: SecurityManager sm =

System.getSecurityManager();1: sm.checkPermission( FilePermission(pathname, ”read”)); //sensitive-operation2: FileRd2();}

X0

X1

RET

X1

Invokes ZZ0

Z1

RET

Z1

X0QStep 1: Add X0, Z0 to Q Z0

Step 2: X0 is a neutral node. Add its successor X1 to Q Z0Q X1

Step 3: Z0 is a neutral node. Add its successor Z1 to Q X1Q Z1

Step 4: X1 is a method invocation node. Insecure_Path(Z) is false. Add Method X to WQ(Z).

Z1Q

XWQ(Z)

Step 5: Z1 is a security check. Stop exploring the path.

Q

XWQ(Z)

Step 6: Q is empty. Terminate the algorithm. Both X and Z are Good.

public void Z() {0: // do some operations

//sensitive-operation1: FileRd1();

2: Y(); // Y is a non-public method}

Our Improvement to the New Approach

Observations Our goal is to identify paths from public

methods to sensitive operations without security checks.

To achieve such a goal, sometimes there is no need to compute some method summaries.

Z0

Z1

RET

Z2

Invokes Y

CFG(Z)

At risk

Current solution:Constructs and explores CFG(Z) and CFG(Y)Computes Summary(Y) and Summary(Z).

However, to identify a risky path in Z,No need to explore the CFG(Y) and compute Summary(Y)No need to compute Insecure_Path attribute of Z

How to make the algorithm even more efficient?

Our Approach: On-the-fly Approach - A Even More Efficient Solution Start to process all public methods first

By putting their entry nodes of CFGs to a queue While exploring the CFGs, if a method invocation

node is encountered and is never visited before, add its entry node to the queue to explore its CFG. Whenever its Bad attribute or Insecure_Path attribute is

computed, replace the node with a non-method-invocation node based on its summary

For a return node, label the method as Insecure_Path

For a sensitive operation, label the method as Bad For a security check, stop exploring that path

public void Z() {0: // do some operations

//sensitive-operation1: FileRd1();

2: Y(); // Y is a non-public method}

An Example to Illustrate the On-the-fly Approach

Z0

Z1

RET

Z2

Invokes non public method

Y

CFG(Z)

At risk

Step 3: Z1 is sensitive, label method Z as Bad

Q

Z1

Step 2: Z0 is a neutral node. Add its successors to Q Q

Z0Q

Step 1: Add Z0 to Q

Step 4: Q is empty. Terminate the algorithm

Runtime of On-the-fly Approach

In comparison to the non-on-the-fly approach: No need to construct the Call Graphs to get all

methods! Construct the CFG and compute the method

summary of each method on demand. In worst case, linear run time

complexity O(ΣM)

ΣM sum of sizes of the CFGs of all methods

Witness Generation & Analysis Witness - A call chain to show a counter example, i.e.

An unguarded path to a sensitive operation, or An insecure path.

java.lang.Class: java.lang.ClassforName0(String, boolean, ClassLoader)

<java.lang.Class: java.lang.Class forName(String)

java.lang.Class: java.lang.Class forName(String,boolean, ClassLoader)

java.net.URL: java.net.URLStreamHandler getURLStreamHandler(String)

java.io.ObjectInputStream: java.lang.Class resolveClass(ObjectStreamClass)

java.net.DatagramSocket: void createImpl()

Method format: <Declaring class Name>:<Return Type>:<MethodName(param list)>

Bold highlights risky methods

java.net.URL: void <init>(URL, String)

java.io.ObjectInputStream: java.lang.Object readObject

represents other risky methods

Experiment & Results

Risky method: A public and bad method.Real Risky: a risky method that has at least one feasible bad path in practice.

HotSpot VM from SUN Microsystems, java.io.*; java.net.*; java.lang.Class;

Total JVM Class

# of Meth. In JVM class Concrete

Total Meth. LOBC

# of Risky Meth.

Real Risky

22 775 703 1520 23,394 61 0             

Harmony VM from Apache Software Foundation, java.io.*; java.net.*; java.lang.Class

Total JVM Class

# of Meth. In JVM class Concrete

Total Meth LOBC

# of Risky Meth.

Real Risky

21 748 689 3928 65,362 0 0

Experiment & Results (Contd.) The average time taken by CMV to analyze

each class was 74 seconds The bulk of the time is spent on Call Graph construction

Preloaded method summary database storing summaries

Risky methods: Need further analysis In HotSpot VM, only 61 methods to be analyzed,

vs. code review on 1520 methods, a reduction in two orders of magnitude!

Automated witness analysis: further reduce human review efforts

Future Work #1 - Method Overriding A method in base class can be overridden by

child classes For a method invocation node, method

summaries from base and all child classes should be considered.

Our Solution Transform the code of base class by adding the

n-way branch statement before the first statement, to make a call to each overriding method of child classes.

Method summary in each branch is computed

Future Work #2- Exceptions Code can throw exceptions. Control flow is

changed Issues

Throw statement If catch block exists, add an edge from throw to

catch block Add an edge from throw to return node

Method invocation node Similar to Throw statement

Rethrow statement, i.e. a throw in a catch block Replace the node with a return node

Solutions: Add new edges

Future Work #3 - Check Non-native Sensitive Operations Sensitive operations are not implemented as

native methods. For examples: Setting a Sensitive Private Data Member Returning a Sensitive Private Data Member Returning a Password Returning Network IP Address

Solutions Identify these sensitive operations, and then apply

the same approach (to the handle of sensitive native methods), to compute method summaries.

Future Work #4 - Automated Sensitive Operation Detection

To automate the detections of sensitive operations in Java libraries.

Challenges- Hard to identify them due to: Some native methods are non-sensitive

They do not access sensitive system resources

The parameters passed to a sensitive operation: another factor (in addition to the method name) to determine if a method is a sensitive operation and the sensitive type (e.g. read or write a file)

Future Work #5 - Automated Permission Checks Detection Determine the types of permission

checks for each sensitive operation. Challenges

Needs to first identify the sensitive operations in JVM.

Should also determine the permission checks for those sensitive operations performed in Java libraries, but without permission checks, due to an undetected defect.

Future Work #6 - Certification Given a method say M0 that is claimed

to be Good or All_Path_Secure, verification is needed to ascertain its correctness.

Our Solution We have initiated a novel and scalable

verification technique to certify the correctness of method summaries.

Experiments are ongoing, and we will report our work in the future.

Related work

Model checking Several general purpose: MOPS, SLAM, Bandera, C-Wolf

Ours is specialized for this problem; property-specific customization makes it scalable

Jensen et al [IEEE S&P99] Algorithms for verification of closed systems

Our approach checks open systems (e.g., libraries) Static analysis

Bug-finding [METAL, CQUAL] Checking complete mediation [Zhang et al [Security’03], Fraser

[PLAS06] These techniques require non-trivial transformations to Java-like libraries

E.g. security checks and sensitive ops in different methods Retrofitting code for authorization

Naccio, SASI, Ganapathy et al [IEEE S&P06] One requires retrofitting only when verification reports unsafe methods

Tool & Paper

Tool: Complete Mediation Verifier (CMV) Developed. To be released by Dec. 2008

Paper: CMV: Automatic Verification of Complete Mediation for Java Virtual Machines Accepted by ACM Symposium on Information,

Computer and Communications Security (ASIACCS’08), to be held in Tokyo, Japan. March 18-20, 2008

Collaborators: Hilary Branske Prof. Prasad Sistla Prof. V.N.Venkatakrishnan

Thank you!