EECS 583 – Class 22Research Topic 4: Automatic SIMDization - Superword Level Parallelism

University of Michigan

December 10, 2012

- 2 -

Announcements

Last class today!» No more reading

Dec 12-18 – Project presentations» Each group sign up for 30-minute slot

» See me after class if you have not signed up

Course evaluations reminder» Please fill one out, it will only take 5 minutes

» I do read them

» Improve the experience for future 583 students

- 3 -

Notes on Project Demos

Demo format» Each group gets 30 minutes

Strict deadlines enforced because many back to back groups Don’t be late! Figure out your room number ahead of time (see schedule on my door)

» Plan for 20 mins of presentation (no more!), 10 mins questions Some slides are helpful, try to have all group members say something Talk about what you did (basic idea, previous work), how you did it

(approach + implementation), and results Demo or real code examples are good

Report» 5 pg double spaced including figures – what you did + why,

implementation, and results

» Due either when you do your demo or Dec 18 at 6pm

- 4 -

SIMD Processors: Larrabee (now called Knights Corner) Block Diagram

- 5 -

Vector Unit Block Diagram

- 6 -

Processor Core Block Diagram

- 7 -

Larrabee vs Conventional GPUs

Each Larrabee core is a complete Intel processor» Context switching & pre-emptive multi-tasking

» Virtual memory and page swapping, even in texture logic

» Fully coherent caches at all levels of the hierarchy

Efficient inter-block communication» Ring bus for full inter-processor communication

» Low latency high bandwidth L1 and L2 caches

» Fast synchronization between cores and caches

Larrabee: the programmability of IA with the parallelism of graphics processors

Exploiting Superword Level Parallelism with Multimedia

Instruction Sets

Multimedia Extensions

• Additions to all major ISAs• SIMD operations

Instruction Set Architecture SIMD Width Floating PointAltiVec PowerPC 128 yesMMX/SSE Intel 64/128 yes3DNow! AMD 64 yesVIS Sun 64 noMAX2 HP 64 noMVI Alpha 64 noMDMX MIPS V 64 yes

Using Multimedia Extensions

• Library calls and inline assembly– Difficult to program– Not portable

• Different extensions to the same ISA– MMX and SSE– SSE vs. 3DNow!

• Need automatic compilation

Vector Compilation

• Pros:– Successful for vector computers– Large body of research

• Cons:– Involved transformations – Targets loop nests

Superword Level Parallelism (SLP)

• Small amount of parallelism– Typically 2 to 8-way

• Exists within basic blocks • Uncovered with a simple analysis

• Independent isomorphic operations– New paradigm

1. Independent ALU Ops

R = R + XR * 1.08327G = G + XG * 1.89234B = B + XB * 1.29835

R R XR 1.08327G = G + XG * 1.89234B B XB 1.29835

2. Adjacent Memory References

R = R + X[i+0]G = G + X[i+1]B = B + X[i+2]

R RG = G + X[i:i+2]B B

for (i=0; i<100; i+=1) A[i+0] = A[i+0] + B[i+0]

3. Vectorizable Loops

3. Vectorizable Loops

for (i=0; i<100; i+=4)

A[i:i+3] = B[i:i+3] + C[i:i+3]

for (i=0; i<100; i+=4) A[i+0] = A[i+0] + B[i+0]

A[i+1] = A[i+1] + B[i+1]A[i+2] = A[i+2] + B[i+2]A[i+3] = A[i+3] + B[i+3]

4. Partially Vectorizable Loops

for (i=0; i<16; i+=1) L = A[i+0] – B[i+0] D = D + abs(L)

4. Partially Vectorizable Loops

for (i=0; i<16; i+=2)

L0L1

= A[i:i+1] – B[i:i+1]

D = D + abs(L0)D = D + abs(L1)

for (i=0; i<16; i+=2) L = A[i+0] – B[i+0] D = D + abs(L)

L = A[i+1] – B[i+1]D = D + abs(L)

Exploiting SLP with SIMD Execution

• Benefit:– Multiple ALU ops One SIMD op– Multiple ld/st ops One wide mem op

• Cost:– Packing and unpacking– Reshuffling within a register

Packing/Unpacking Costs

C = A + 2D = B + 3

C A 2D B 3= +

Packing/Unpacking Costs

• Packing source operands

A AB BA = f()

B = g()C = A + 2D = B + 3

C A 2D B 3= +

Packing/Unpacking Costs

• Packing source operands• Unpacking destination operands

C CD D

A = f()B = g()C = A + 2D = B + 3E = C / 5F = D * 7

A AB B

C A 2D B 3= +

Optimizing Program Performance

• To achieve the best speedup:– Maximize parallelization– Minimize packing/unpacking

• Many packing possibilities– Worst case: n ops n! configurations– Different cost/benefit for each choice

A = B + CD = E + F

Observation 1:Packing Costs can be Amortized

• Use packed result operands

G = A - HI = D - J

Observation 1:Packing Costs can be Amortized

• Use packed result operands• Share packed source operands

A = B + CD = E + F

G = B + HI = E + J

A = B + CD = E + F

G = A - HI = D - J

Observation 2:Adjacent Memory is Key

• Large potential performance gains– Eliminate ld/str instructions– Reduce memory bandwidth

• Few packing possibilities– Only one ordering exploits pre-packing

SLP Extraction Algorithm

• Identify adjacent memory references

A = X[i+0]C = E * 3B = X[i+1]H = C – AD = F * 5J = D - B

SLP Extraction Algorithm

• Identify adjacent memory references

A = X[i+0]C = E * 3B = X[i+1]H = C – AD = F * 5J = D - B

AB = X[i:i+1]

SLP Extraction Algorithm

• Follow def-use chains

A = X[i+0]C = E * 3B = X[i+1]H = C – AD = F * 5J = D - B

AB = X[i:i+1]

SLP Extraction Algorithm

• Follow def-use chains

A = X[i+0]C = E * 3B = X[i+1]H = C – AD = F * 5J = D - B

AB = X[i:i+1]

HJ

CD

AB= -

SLP Extraction Algorithm

• Follow use-def chains

A = X[i+0]C = E * 3B = X[i+1]H = C – AD = F * 5J = D - B

AB = X[i:i+1]

HJ

CD

AB= -

SLP Extraction Algorithm

• Follow use-def chains

A = X[i+0]C = E * 3B = X[i+1]H = C – AD = F * 5J = D - B

AB = X[i:i+1]

CD

EF

35= *

HJ

CD

AB= -

SLP Extraction Algorithm

• Follow use-def chains

A = X[i+0]C = E * 3B = X[i+1]H = C – AD = F * 5J = D - B

AB = X[i:i+1]

CD

EF

35= *

HJ

CD

AB= -

SLP Availability

0

10

20

30

40

50

60

70

80

90

100

swim

tomca

tv

mgr

id

su2c

or

wave5

apsi

hydr

o2d

turb

3d

applu

fppp

p FIR IIRVM

MMMM

YUV

% dynamic SUIF instructions eliminated

128 bits1024 bits

SLP vs. Vector Parallelism

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

swim

tomca

tv

mgrid

su2c

or

wave5 ap

si

hydro2

d

turb3d

applu

fppp

p

SLPVector

Conclusions

• Multimedia architectures abundant– Need automatic compilation

• SLP is the right paradigm– 20% non-vectorizable in SPEC95fp

• SLP extraction successful– Simple, local analysis– Provides speedups from 1.24 – 6.70

• Found SLP in general-purpose codes