dynamic assignment of scoped memory regions in the...

Distributed Object Computing LaboratoryWashington University

St. Louis, MO

Dynamic Assignment of Scoped Memory Regions in the Translation

of Java to Real-Time JavaMorgan Deters

Advisor: Dr. Ron K. Cytron

M.S. Thesis Defense10 March 2003

Copyright © 2003 Morgan Deters

Research funded by DARPA under contract F33615-00-C-1697

Washington UniversitySt. Louis

Dynamic Assignment of Scoped Memory RegionsMorgan Deters

110 March 2003

Overview

Dynamic AnalysisMemory Regions

JavaReal-Time

Garbage Collection Avoidance

Scoped Memory

Aspect-Oriented Programming



210 March 2003

Big Picture

Java

JVM

OS

RT-Java

RTJVM

RTOS



310 March 2003

Java vs. Real-Time

• High-level• Popular• Object-oriented• Garbage collection• Reusable components• Flexible, adaptable• Scalable• Portable• Optimized average-case• Memory use varies• Coarse control of

concurrency

• Low-level• Specialized• Manually-optimized• Specialized allocators• Tailored components• Hard-coded behavior• May not scale• Hardware-dependent• Reasonable worst-case• Footprint, latency critical• Fine control over

concurrency



410 March 2003

Real-Time Java ?

• Real-Time Specification for Java (RTSJ, JSR001) brings infrastructure for real-time accountability to Java§ Threads with real-time guarantees§ Real-time schedulability§ High-precision timers§ Asynchronous transfer of control§ “Physical” memory access§ Choice of memory allocators and garbage collection§ Memory reclamation without garbage collection



510 March 2003

Real-Time Java Memory Hierarchy

MemoryArea

HeapMemoryScopedMemoryImmortalMemory



610 March 2003

Rules for Scope References

From the Real-Time Specification for Java v1.0

Thou shalt not reference any object whose lifetime could be shorter than thine own.



710 March 2003

Inter-Object Reference Rules

• Objects may only refer to objects in§ ancestor memory regions§ garbage-collected heap

Child Region

Parent Region

An object cannot refer to an objectfrom a shorter-lived memory region!



810 March 2003

Nested ScopeMemory References

X YZ

A

B

C

FGD

E

Threads create scopes and objectsThreads can enter existing memory scopesMore scope and object creationObject references must point outwardReferences pointing inward are illegalThreads recede causing collection en masseInward-pointing references can “dangle”



910 March 2003

A Stack of Regions



1010 March 2003

Using Real-Time Java Scoped Memory

• Consider class C:

class C {void foo() {bar();// ...

}void bar() {// generate lots of garbage

}}



1110 March 2003

Using Real-Time Java Scoped Memory

• foo() calls bar() in a separate regionclass C {void foo() {ScopedMemory sc = new ScopedMemory(1024,1024);sc.enter(new Runnable() {

public void run() {bar();

}});

}void bar() {// generate lots of garbage

}}



1210 March 2003

Using Real-Time Java Scoped Memoryclass C {Object barResult; // assume single-threadedvoid foo() {ScopedMemory sc = new ScopedMemory(1024,1024);sc.enter(new Runnable() {

public void run() {barResult = bar();

}});

// now use the value in barResult...}Object bar() {// generate lots of garbage// ... new MyObject() ...

}}

Illegalreference

Allocated inscoped memory lt



1310 March 2003

RT-Java Memory Summary

• Assign memory regions to execution scopes• new operations allocate from current region

• Introduces memory management dependencies between caller and target

• Hurts reusability of code• Integrates concerns of program• Introduces interdependence of libraries



1410 March 2003

High-Level View

PP + = PPReal-Time JavaJava

DynamicAnalysisDynamicAnalysis

ProgramInstrumentation




1510 March 2003

Valley ofStatic Analysis

Valley ofDynamic Analysis

THE TRUTHTHE TRUTH



1610 March 2003

Stack of Regions == Call Stack

M()

N()

O()

P()

Q()

R()

S()



1710 March 2003

Goal:AP : OP ? FP

OP: objects of program PFP: call-stack frames of program P

Given program P …



1810 March 2003

Approach

• Run program P…• Observe referencing behavior• Observe object lifetimes• Come up with scoping hierarchy that does this



1910 March 2003

Determine Referencing andObject Lifetimes

• Keep a doesReference graph

• Annotate it with object death frames

A C

D E

B

F

G1

2

0 2

222Represents all legal scoping hierarchies



2010 March 2003

Solving the doesReference graph

• Decompose into strongly-connected components

A C

D E

B

F

G1

2

0 2

222A B C



2110 March 2003

Solution to doesReference graph

• Propagate smaller numbers through the graph

D

A B C

F

G1

2

0 2

20

0

E



2210 March 2003

How to recognize objects from one run

to another?Q:

A:Use an object’s

allocation site as a discriminator



2310 March 2003

Object Behavior Categories

• How to map objects observed in previous analyses to objects observed in current program execution?§ Class-based behavior

• All instances of a class behave similarly

§ “New site”-based behavior• All instances of a class instantiated at the same source

code location behave similarly

§ Instance-based behavior• Instances of a class behave differently



2410 March 2003

Impact of Simplification

• Consider a method that acts as a Factory:class LinkedList {public Iterator iterator() {return new LLIterator();

}}

Iterator i = list.iterator();while(i.hasNext()) {// ...

}

list.iterator

()

list.iterator

()



2510 March 2003

Revised Goal:

AP : SP ? ?FP

SP: allocation sites of program PFP : call-stack frames of program P?FP : object lifetimes in # of frames

Given program P …



2610 March 2003

Determine Referencing andObject Lifetimes

A C

D E

B

F

G2,1

2,2

0,0 2,2

2,22,22,2

birth frame, collection frame



2710 March 2003

Solving the Graph

• Liveness constraints:For each vertex vScope(v) = max oi – oc

• Reference constraints:For each edge (u,v)Scope(v) = Scope(u) + max oi – min oi

o ? Objects(v) o ? Objects(u)

o ? Objects(v)

oi = object allocation frameoc = object collection frame



2810 March 2003

Solving the Graph

D E F

G2,1

2,2

0,0 2,2

A B C2,2

0

0

01

0 ?

Scope(ABC) = Scope(E) + max oi – min oiABC E

Reference Constraint:

birth frame, collection frame



2910 March 2003

Solution to doesReference graph

D E F

G

A B C

2,1

2,2

0,0 2,2

2,2

0

0

01

2



3010 March 2003

Deriving the Reference Constraint

• Scope(v) = Scope(u) + max oi – min oio ? Objects(v) o ? Objects(u)

In absolute depth model using individual objects:Suppose we know that object A references object BWe want to instantiate A – where does it go ?

B

A

Scope(B) = Scope(A)



3110 March 2003



In relative depth model using instantiation sites:Suppose we know that u references vWe want to create something at u – where does it go ?

B ? u

Scope(v) = Scope(u)

but there’s another case…



3210 March 2003



In relative depth model using instantiation sites:Suppose we know that u references vWe want to create something at u – where does it go ?

A ? v

B ? u

Scope(v) = Scope(u)

but there’s another case…

Scope(v) = Scope(u) + max difference

A ? v

difference ininstantiation

requestframes



3310 March 2003

High-Level View

PP + = PPReal-Time JavaJava

DynamicAnalysisDynamicAnalysis





3410 March 2003

Aspect-Oriented Programming

• Advanced Separation of Concerns (ASoC)• quantification§ Our object analysis and program instrumentation

can be expressed in modular pieces of code

• obliviousness§ Target code is oblivious to this extra processing

• it need not be changed at all



3510 March 2003

Reference Probe

1 Determine references between objectsa. Track objects’ references to other objectsb. Build a doesReference graph

2 Discover legal scoping hierarchiesa. Collapse strongly connected componentsb. The resulting DAG is a representation of legal

scoping hierarchies



3610 March 2003

Effect of Reference Probeclass C {MyObject anObject;

void foo() {

anObject = new MyObject();

// do other stuff...}

}



3710 March 2003

Effect of Reference Probeclass C {MyObject anObject;

void foo() {MyObject o = new MyObject();ReferenceProbe.newObject(o, getSourceLocation());

anObject = o;ReferenceProbe.registerReference(this, o);


}



3810 March 2003

Aspect “Weaving”

package whiteboard.net;

import whiteboard.*;

import java.io.*;import java.net .*;

/*** Thread that listens for incoming connections.*/public class ListenThread extends Thread implements ServerConstants {private NetworkedMainFrame netFrame;private ServerSocket acceptor;

public ListenThread(NetworkedMainFrame netFrame, int port,int backlog, InetAddress bindAddr) throws IOException {

this.netFrame = netFrame;acceptor = new ServerSocket(port, backlog, bindAddr);

}

public ListenThread(NetworkedMainFrame netFrame,int backlog) throws IOException {

this.netFrame = netFrame;acceptor = new ServerSocket(PORT, backlog);

}

public ListenThread(NetworkedMainFrame netFrame) throws IOException {this.netFrame = netFrame;acceptor = new ServerSocket(PORT);

}

public void run() {for(;;) {try {System.out.println("Listening on "+

acceptor.getInetAddress().getHostAddress()+" port "+acceptor.getLocalPort ());

Socket sock = acceptor.accept();// alternative: connection thread pool? 1/2 synch 1/2 asynch ?MainFrame f = new MainFrame();Connection conn = new Connection(sock );f.getWhiteBoard().connection = conn;new ConnectionThread(f, conn).start();f.setVisible(true);

} catch(Exception e) {System.err.println("Exception in ListenThread " + this);e.printStackTrace();

}}

}}

/* Project WhiteBoard - *- c-basic-offset: 2 -*-* Dante Cannarozzi and Morgan Deters, October 2002** $Id: Connector.java,v 1.4 2002/10/13 22:31:01 mdeters Exp $*/

package whiteboard.net;

import whiteboard.MainFrame ;

import java.io.IOException;import java.net .*;

public class Connector implements ServerConstants {private Connector() { }

public static void connect(InetAddress remoteAddr , int port)throws IOException {

MainFrame frame = new MainFrame();System.out.println("initiating connection to "+ remoteAddr+" port "+port);frame.getStatusPanel().setConnectionStatus("Connecting...");frame.setVisible(true);Socket sock = new Socket(remoteAddr , port);frame.getStatusPanel().setConnectionStatus("Connected ");Connection conn = new Connection(sock);frame.getWhiteBoard().connection = conn;new ConnectionThread(frame, conn).start();

}}

public class Connection {private Socket sock;private long duration;private boolean closed;

private ObjectOutputStream out;private ObjectInputStream in;

private User localUser = User.makeLocalUser ();private User remoteUser;

public Connection(Socket sock) throws IOException {this.sock = sock;duration = System.currentTimeMillis ();NetworkedMainFrame.instance().addConnection(this);

out = new ObjectOutputStream(sock.getOutputStream());out.writeObject(localUser);in = new ObjectInputStream(sock.getInputStream());System.out.println("BACK HERE");try {System.out.println("reading ...");remoteUser = (User)in.readObject();System.out.println("read it!");

} catch(ClassNotFoundException e) {throw new UnrecognizedObjectException(e );

}}

public User getLocalUser() {return localUser;

}

public User getRemoteUser () {return remoteUser ;

}

public long getDuration() {return closed ? duration : System.currentTimeMillis() - duration;

}

public Socket getSocket() {return sock;

}

public InetAddress getRemoteAddress() {return sock.getInetAddress();

}

public boolean isClosed() {return closed;

}

public void write(Object o) throws IOException {out.writeObject(o );

}

public Object read() throws IOException {try {return in.readObject();

} catch(ClassNotFoundException e) {throw new UnrecognizedObjectException(e );

}}

public void update() {NetworkedMainFrame.instance().updateConnection(this);

}

public void close() throws IOException {if(closed)return;

try {sock.close();

} finally {closed = true;duration = System.currentTimeMillis() - duration;NetworkedMainFrame.instance().updateConnection(this );

}}

}

Aspect

Track all objectreferences

Aspect

Track all objectinstantiations



3910 March 2003

1 Determine the points at which objects become collectible

a. Track object birth timesb. After each execution join point, run a garbage

collection cycle and determine which objects were collected

2 Express object liveness behavior in terms of:a. Enclosing execution join pointsb. Distance from birth to death

Object Liveness Probe



4010 March 2003

Effect of Liveness Probeclass C {MyObject anObject;

void foo() {

anObject = new MyObject();

// do stuff...

}

}



4110 March 2003

Effect of Liveness Probeclass C {MyObject anObject;

void foo() {

anObject = new MyObject();LivenessProbe.registerObject(anObject);

// do stuff...

LivenessProbe.gcCycle();}

}



4210 March 2003

High-Level View

PP + = PPReal-Time JavaJava Program

InstrumentationProgram

Instrumentation

AspectJ

Dynamic Analysis

Reference Probe

Liveness Probe



4310 March 2003

Adding an Enforcement Aspect

• We have:§ A set of all legal ScopedMemory hierarchies§ Object lifetime behavior (join points)

• From this information, we:§ Select a desirable, legal scope memory hierarchy§ Direct a dispatch unit to implement this hierarchy



4410 March 2003

Effect of Scope Enforcementclass C {void foo() {

bar();


void bar() {// generate lots of garbage

}

}



4510 March 2003

Effect of Scope Enforcementclass C {void foo() {ScopedMemory sc = new ScopedMemory(1024,1024);sc.enter(new Runnable() {bar();

});// do other stuff...

}

void bar() {// generate lots of garbage

}

}



4610 March 2003

High-Level View

PP + = PPReal-Time Java

AspectJ

Java

AspectJ

Dynamic Analysis

Reference Probe

Liveness Probe

DispatchUnit A+



4710 March 2003

Experimentation

• Instrumented JVM 1.1.8 to emit§ object allocations§ object collections§ intra-heap references

• SPECjvm98 benchmarks – size 10



4810 March 2003

Compress assigned scopes_201_compress - Scoped Object Lifetime

0

200

400

600

800

1000

1200

1400

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

Scopes Assigned to Objects at Runtime



4910 March 2003

Understanding Object Lifetimes

• Two reasons objects live longer than necessary§ Allocation site assumption

§ Referencing requirements

Site S

B

A



5010 March 2003

Compress extra longevityExtra Longevity - _201_compress

0

500

1000

1500

2000

2500

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

# f rames



5110 March 2003

Raytrace assigned scopes_205_raytrace - Scoped Object Lifetime

0

50000

100000

150000

200000

250000

300000

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Scopes Assigned to Objects at Runtime



5210 March 2003

Raytrace extra longevityExtra Longevity - _205_raytrace

0

50000

100000

150000

200000

250000

300000

350000

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

# f rames



5310 March 2003

Db scopes assigned_209_db - Scoped Object Lifetime

0

5000

10000

15000

20000

25000

30000

35000

40000

45000

50000

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Scopes Assigned to Objects a t Runt ime



5410 March 2003

Db extra longevityExtra Longevity - _209_db

0

10000

20000

30000

40000

50000

60000

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

# f r a m e s



5510 March 2003

Jack scopes assigned_228_jack - Scoped Object Lifetime

0

5000

10000

15000

20000

25000

0 5 10 15 20 25 30 35 40 45

Scopes Assigned to Objects a t Runt ime



5610 March 2003

Jack extra longevityExtra Longevity - _228_jack

0

5000

10000

15000

20000

25000

30000

0 5 10 15 20 25 30 35 40 45

# f r a m e s



5710 March 2003

Extra Longevity - _201_compress

0

500

1000

1500

2000

2500

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

# f rames

Extra Longevity - _205_raytrace

0

50000

100000

150000

200000

250000

300000

350000

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

# frames

Extra Longevity - _209_db

0

10000

20000

30000

40000

50000

60000

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 2 0

# frames

Extra Longevity - _228_jack

0

5000

10000

15000

20000

25000

30000

0 5 10 15 2 0 2 5 30 3 5 40 45

# frames



5810 March 2003

Combining Static and Dynamic Analyses

• Static analysis used for§ Eliminating RTSJ-mandated reference checks§ Verifying scope assignments, flagging others for

programmer input

• Dynamic analysis used for§ Initial scope assignment§ Debugging information



5910 March 2003

Related Work

• Regions§ Tofte & Talpin§ Gay & Aiken

• Real-Time Java analysis§ Pointer and escape analysis for Java§ Salcianu & Rinard



6010 March 2003

Remarks

• Garbage collection is great, but…• Stack-based region-allocated objects• RTSJ scoped memory breaks modularity• Compose with results of static analysis• Extension to non-RTSJ systems



6110 March 2003

Contributions

• Use of aspects in dynamic analysis§ Deters, Leidenfrost, and Cytron. Translation of Java to Real-Time

Java, International Workshop on Aspect-Oriented Programming and Separation of Concerns, August 2001

• A Real-Time Java translation system, implemented in AspectJ, that requires no program annotation and transforms Java programs into ScopeMemory-aware RT-Java programs modularly§ Deters and Cytron. Introduction of Program Instrumentation using

Aspects, OOPSLA Workshop on Advanced Separation of Concerns in Object-Oriented Systems, October 2001

• Automation of scope detection and selection§ Deters and Cytron. Automated Discovery of Scoped Memory Regions

for Real-Time Java, International Symposium on Memory Management, June 2002



6210 March 2003

Acknowledgments

• Dr. Ron K. Cytron• Committee§ Dr. Chris Gill§ Dr. Aaron Stump

• JVM hacking§ Nick Leidenfrost§ Dante Cannarozzi§ Matt Hampton

• Members of the DOC Group, past & present

dynamic assignment of scoped memory regions in the...

Documents