faster work–stealing with return barriers€¦ · • higher steal rate correlates to high steal...

Vivek Kumar, Stephen M Blackburn The Australian National University

Faster Work–Stealing With Return Barriers

•  Commodity processors with parallel execution abilities

•  A fundamental turn toward concurrency in software

2

The New Era of Computing

Faster Work-Stealing With Return Barriers | Kumar & Blackburn | VMIL’12

Background

•  Modern hardware requires s/w parallelism •  Software parallelism difficult to identify, expose

–  Hard coded optimizations may get you there…

•  Hard to realize potential of modern processors

The Challenge

Goal: performance and productivity

3

3


Background

•  Language based features to expose parallelism –  Dynamic task parallelism –  Work–stealing scheduler

•  A runtime to hide the hardware complexities

4

Options ?


Background

Contributions ✔ In-depth analysis

Of overheads associated with stealing tasks

✔ A new design Simple extension to JVM re-using old idea

✔ Detailed performance study Using standard work-stealing benchmarks

✔ Results That show we can significantly reduce the tasks stealing overhead

5


Understanding Work–Stealing

6


Work–Stealing

7


Work–Stealing

8


Work–Stealing

9


Work–Stealing

10


Initiation

Termination

State Management

Work–Stealing

11


12

Our Prior Work


OOPSLA 2012

13


Our Prior Work

•  Sequential overheads –  Initiation –  State management –  Code restructuring

•  Exploit existing JVM mechanisms –  Initiation: Execution stack for steal initiation –  State management: Extract state from stack & registers –  Code restructuring: Try–catch blocks for control flow

•  Eliminated most sequential overheads

Fork–Join: 200% X10: 400%

12%

Eliminating Sequential Overheads

14


Our Prior Work

Motivating Analysis

15


16

Methodology

•  Benchmarks –  Jacobi –  LU Decomposition –  Heat Diffusion

•  High steal ratio

•  Hardware Platform –  2 Intel Xeon E7530

•  6 cores each

•  Software Platform –  Jikes RVM (3.1.2)

16


Motivating Analysis

1.00E-06

1.00E-05

1.00E-04

1.00E-03

1.00E-02

1.00E-01

1.00E+00

2 3 4 5 6 7 8 9 10 11 12

Stea

ls/T

ask

Threads

Heat Jacobi LUD

Measured using JavaWS (Try–Catch)

17

Steals:Task Ratio

17


Motivating Analysis

0

500

1000

1500

2000

2500

3000

3500

2 3 4 5 6 7 8 9 10 11 12

Stea

ls/s

econ

d

Threads

Heat Jacobi LUD

18

Steal Rate

18


Motivating Analysis


Insight

•  Steal ratio and steal rate not correlated

19


Motivating Analysis

0

2

4

6

8

10

2 3 4 5 6 7 8 9 10 11 12

Vict

im W

ait T

ime

(% C

PU c

ycle

s)

Threads

Heat Jacobi LUD

20

Steal Overhead

20


Motivating Analysis


Insights

•  Steal ratio and steal rate not correlated •  Higher steal rate correlates to high steal overhead

21


Motivating Analysis

Our Approach

22

22


Reducing Stealing Overhead

•  Two pronged attack –  Reduce the cost of each steal

•  Return barriers –  Reduce total number of steal events

•  Steal more than one continuation at a time

23


Our Approach

Implementation

24


25

Return Barrier •  Allows runtime to intercept a common event •  Hijack a return and bridge to some other method •  Register and stack state preserved

E

D

C

B

A

TOP

BASE

Sta

ck G

row

th D

irect

ion Frame

Pointer Return

Address

Frame C Method C Frame C trampoline method

Frame D

Hijacked Frame Pointer

Hijacked Return

Address

Frame C Method C

trampoline method


Implementation

26

E

D

C

B

A

TOP

BASE

Sta

ck G

row

th D

irect

ion

Yieldpoint Mechanism

A

Thief Installs Return Barrier


Implementation

27

D

C

B

A

TOP

BASE

Sta

ck G

row

th D

irect

ion

A

Victim Moves The Return Barrier

D


Implementation

28

D

C

B

TOP

BASE

Sta

ck G

row

th D

irect

ion

A

Robbing A Victim With Return Barrier

D

B

Yieldpoint Mechanism


Implementation

Performance Evaluation

29


Vict

im w

ait t

ime

CP

U c

ycle

s (%

)

Threads

Jacobi 30

0

2

4

6

8

10

2 3 4 5 6 7 8 9 10 11 12 0

2

4

6

8

10

12

2 3 4 5 6 7 8 9 10 11 12 Threads

Spe

edup

Ove

r Seq

uent

ial

No Return Barrier With Return Barrier

30


Evaluation

Vict

im w

ait t

ime

CP

U c

ycle

s (%

)

Threads

LUD 31

0

2

4

6

8

10

2 3 4 5 6 7 8 9 10 11 12 0

2

4

6

8

10

12

2 3 4 5 6 7 8 9 10 11 12 Threads

Spe

edup

Ove

r Seq

uent

ial

No Return Barrier With Return Barrier

31


Evaluation

Summary

•  Steal overhead dominated by steal rate •  Two pronged attack

–  Reduce the cost of each steal •  Return barriers

–  Reduce total number of steals •  Steal more than one

•  No change in speedup

32

58%

32


Summary

Future Work

•  Steal overhead dominated by steal rate •  Two pronged attack

–  Reduce the cost of each steal •  Return barriers

–  Reduce total number of steals •  Steal more than one

•  No change in speedup •  Merge both the techniques

33

58%

?%

?


Future Work

faster work–stealing with return barriers€¦ · • higher steal rate correlates to high steal...

Documents