averroes: letting go of the library!karim ali averroes: letting go of the library! partial-program...

Averroes: Letting go of the library!

Karim Ali

Workshop on WALA - PLDI ‘15 June 13th, 2015

Karim Ali Averroes: Letting go of the library! 2

public class HelloWorld { public static void main(String[] args) { System.out.println("Hello, World!");}

}

Karim Ali Averroes: Letting go of the library!

• ~ 8 seconds

• ~ 600 MB of memory

• > 900 reachable methods

• > 1,700 call edges3

public class HelloWorld { public static void main(String[] args) { System.out.println("Hello, World!");}

}


Hello, World!

4


Partial-Program Analysis

5



6

I'd like to ignore library code

what about callbacks?

this would be unsound but better than nothing

ignore non-application program elements (e.g., system libraries)?

whole-program analysis always pulls in the world for completeness.

The problem is that the world is fairly large

I am NOT interested in those



7

?


Ideal Call Graph


Ideal Call Graph

Whole-Program Call Graph


Ideal Call Graph


Incomplete Call Graph (unsound)


Ideal Call Graph



Conservative Call Graph

(highly imprecise)


Analyzed Code

?m(){ ...

}

c.center = new Point(0,0);

call method

create objectmodify field

Conservative Assumptions


Ideal Call Graph



Conservative Call Graph

(highly imprecise)

Partial-Program Call Graph

13


The Separate Compilation Assumption

14


The Separate Compilation Assumption

All of the library classes can be compiled in the absence of the application classes.

15


Constraints

1. Class Hierarchy

2. Class Instantiation

3. Local Variables

4. Method Calls

16

5. Field Access

6. Array Access

7. Static Initialization

8. Exception Handling


Constraints

1. Class Hierarchy

2. Class Instantiation

3. Local Variables

4. Method Calls

17

5. Field Access

6. Array Access

7. Static Initialization

8. Exception Handling


Library Points-to Set (LPT)

18

Application Library

pt(v1) = o1 o3

pt(v2) = o2 o3

pt(v3) = o1 o4

LPT =o1

o2 o3 o5

[Tip & Palsberg OOPSLA’00]


Library Callbacks

19

Application Library

class C { m(); }

class B extends L { m(); }

class A extends L { m(); } calls

class L { m(); }

1

LPT = A C

2


[K. Ali and O. Lhoták, ECOOP ’13]


JAR

Placeholder Library

SCA

JAR

21


REF

REF

Placeholder Library

SCA

REF

averroes.Library

Application


averroes.Library


public class averroes.Library {

public static Object libraryPointsTo;public static void doItAll() {// Class instantiation

// Library callbacks

// Field writes

// Array element writes

// Exception Handling}

}


Evaluated for SOOT and DOOP

✓ 300x smaller library!

✓ Up to 15x faster analysis

✓ Up to 8x less memory

✓ Precise

✓ Sound

25


Implementation Tweaks

26


Class Loaders in WALA

27

ClassLoaderReference.Extension

ClassLoaderReference.Primordial

ClassLoaderReference.Application


Problem 1: App.methods()

28




public class Library {

public static void doItAll() {// Class instantiationA a = new A();

// Library callbacksa.foo();

}}


Problem 1: App.methods()

29







}}


Solution 1: App.methods()

30







}}


Problem 2: doItAll()

31





public static void doItAll() {...

}}

public class PrintStream {

public void println(String s) {...

Library.doItAll();...

}}


Solution

32


public class AbstractLibrary {public static AbstractLibrary instance;public abstract void doItAll();

}

public class Library extends AbstractLibrary {static {

AbstractLibrary.instance = new Library();}

public void doItAll() {...A a = new A();

}}

public class PrintStream {public void println(String s) {

... AbstractLibrary.instance.doItAll();

...}

}






}




}}



...}

}






}




}}



...}

}




35


Evaluation

36


Evaluation

• What are the performance gains of using the placeholder library?

• How precise is Averroes?

• Does using the placeholder library affect the soundness of WALA?

37


Library Size

38

SCA

Original Library ~ 25 MB

Placeholder Library ~ 80 KB

300xSMALLER

ON AVG


Execution Time

WALAOverhead WALAAVEAnalysis WALAAVE

OverheadWALAAnalysis AVERROES

Exec

utio

n Ti

me

(s)

0

5

10

15

20

25

30

antlr bloat chart hsqldb luindex lusearch pmd xalan compress db jack javac jess raytrace

2xFASTER

ON AVG


WALA WALAAVE

Memory UsageM

emor

y Us

age

(GB)

0.00.10.20.30.40.50.60.70.80.91.01.1

antlr bloat chart hsqldb luindex lusearch pmd xalan compress db jack javac jess raytrace

2xLESS

ON AVG


Precision of Averroes

41


|WALAAVE \ WALA| |WALA|

ANTLR 2.06%BLOAT 19.66%CHART 22.76%HSQLDB 32.56%LUINDEX 4.35%LUSEARCH 10.45%

PMD 7.78%XALAN 33,100.00%

COMPRESS 0.00%DB 7.04%JACK 3.63%JAVAC 1.21%JESS 8.70%

RAYTRACE 0.00%42

Precision of Averroes

8%ON AVG


Imprecise Library Callbacks

Application Library

JDBC Implementation

JDBC Interface

Client Code

WALAAVE \ WALA

HSQLDB org.hsqldb.jdbc.* 577 (70%)


Sound(i?)ness

44


Unsoundness w.r.t. Dynamic Call Graphs|DYNAMIC \ WALA|

|DYNAMIC||DYNAMIC \ WALAAVE|

|DYNAMIC|ANTLR - -BLOAT - -CHART 6% 6%HSQLDB 4% 4%LUINDEX 1% -LUSEARCH 1% -

PMD 3% -XALAN 99% -

COMPRESS - -DB - -

JACK - -JAVAC - -JESS - -

RAYTRACE - -


Reasons for Unsoundness

46

public void notifyListeners(Event event) { Object[] ls = this.listenerList.getListenerList(); for (int i = ls.length - 2; i >= 0; i -= 2) { if (ls[i] == Listener.class) { ((Listener)ls[(i + 1)]).changed(event); } } }


Correctness ProofBased on Featherweight Java

47


FJ…

m

…

m’


FJ FJAVE…

m

…

m’

…

m

…

m’


Evaluation Summary

✓ 300x smaller library!

✓ 2x faster

✓ 2x less memory

✓ Precise

✓ Sound

50


What’s Next?

51


Automatic Library Detection

What’s Next?Analyzing Large Frameworks

SCALACGSCA

DOOPSCA

Application Library

The Past The Future

Incremental JAR Generation

Better Reflection Support

Integration with WALA

52


Analyzing Large Frameworks

SCALACG

DOOP

The Future

53

• library is much larger than application • event-based systems => many callbacks • modeling application lifecycle is not trivial


Better Reflection Support

SCALACG

DOOP

The Future

54

better support for reflection

automatic library detection

incremental JAR generation


WALA Averroes Analysis Scope

WALA Analysis Scope

Integration with WALA

SCALACG

DOOP

The Future

55

Averroes: Letting go of the library! Karim Ali

Technische Universität Darmstadt karimali.ca

https://github.com/karimhamdanali/averroes

http://karimali.ca

averroes: letting go of the library!karim ali averroes: letting go of the library! partial-program...

Documents