c-to-coram: compiling perfect loop nests to the portable...

C-to-CoRAM Compiling Perfect Loop Nests to the PortableCoRAM Abstraction

Gabriel WeiszComputer Science Department

Carnegie Mellon UniversityPittsburgh PA USA

gweiszcscmuedu

James C HoeElectrical amp Computer Engineering Department

Carnegie Mellon UniversityPittsburgh PA USA

jhoeececmuedu

ABSTRACTThis paper presents initial work on developing a C compilerfor the CoRAM FPGA computing abstraction The pre-sented effort focuses on compiling fixed-bound perfect loopnests that operate on large data sets in external DRAM Asrequired by the CoRAM abstraction the compiler partitionssource code into two separate implementation components(1) hardware kernel pipelines to be mapped onto the recon-figurable logic fabric and (2) control threads that expressin a C-like language the sequencing and coordination ofdata transfers between the hardware kernels and externalDRAM The compiler performs optimizations to increaseparallelism and use DRAM bandwidth efficiently It cantarget different FPGA platforms that support the CoRAMabstraction either natively in a future FPGA or in soft-logicon todayrsquos devices The CoRAM abstraction provides a con-venient high-level compilation target to simplify the task ofdesign optimization and system generation The compileris evaluated using three test programs (matrix-matrix mul-tiplication k -nearest neighbor and 2D convolution) on theXilinx ML605 and the Altera DE4 Results show that ourcompiler is able to target the different platforms and effec-tively exploit their dissimilar capacities and features De-pending on the application the compiler-generated imple-mentations achieve performance ranging from a factor of 4slower to a factor of 2 faster relative to hand-designed im-plementations as measured on actual hardware

Categories and Subject DescriptorsB52 [Hardware] REGISTER-TRANSFER-LEVEL IM-PLEMENTATIONAutomatic synthesis Optimization

General TermsDesign

KeywordsFPGA computing High-level Synthesis Loop optimizationData reuse

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page To copy otherwise torepublish to post on servers or to redistribute to lists requires prior specificpermission andor a feeFPGArsquo13 February 11ndash13 2013 Monterey California USACopyright 2013 ACM 978-1-4503-1887-71302 $1500

1 INTRODUCTIONMotivations Modern FPGAs are capable and efficient

computing devices in a wide range of application areas ([9][20] [11] and [15]) but have failed to achieve widespreaduse A major obstacle to the adoption of FPGAs for com-puting is the high degree of difficulty associated with devel-oping FPGA applications Prior work such as CoRAM [10]LEAP [1] and VirtualRC [17] have approached this prob-lem from the architectural side by virtualizing an FPGArsquosexternal memory and IO interfaces Such approaches savedevelopment time by avoiding repeated effort in infrastruc-ture development and enabling portability of completed ap-plications What cannot be solved through architecture andabstraction is the inherent difficulty in manually mappingalgorithms to hardware datapaths that target the FPGArsquosreconfigurable fabric

Verilog and VHDL the prevailing design languages fortargeting FPGAs follow a hardware-centric paradigm thatrequires a designer to directly manage highly concurrentfine-grained operations with per-cycle coordination Algo-rithmic and application experts prefer to operate at a muchhigher level of abstraction and would rather use sequen-tial languages such as C As we will discuss in Section 2there has been much research and development on compil-ers that can automatically generate hardware designs forFPGAs from C and other programming languages

Compiling Perfect Loop Nests for an FPGA Thispaper presents work on developing a C compiler that canproduce a complete FPGA-based implementation from per-fect loop nests with fixed bounds Perfect loop nests arethose in which all computation occurs within the innermostloop body The innermost loop body can access large datasets residing in off-chip DRAM using both array and in-direct pointer references This class of programs includesmany important scientific and numerical applications Asthey typically exhibit a high degree of inherent parallelismand predictable data access patterns these programs arewell suited for implementation on an FPGA Moreover oth-ers have worked on transforming non-perfect loop nests intoperfect loop nests [21]

Unlike previous C-to-gates and C-to-FPGA compilers ourwork focuses primarily on achieving efficient use of off-chipDRAM memory bandwidth and on-chip SRAM buffers Weuse ROCCC (Riverside Optimizing Compiler for Config-urable Computing [23]) to generate streaming hardware ker-nel pipelines corresponding to the innermost loop body Ourcompiler seeks to saturate these kernel pipelines by man-aging the flow of the input and output data streams be-

tween the kernel pipelines and external DRAM Our com-piler built on top of LLVM [24] analyzes the memory ref-erences in a loop nest for dependencies and access patternsThe compiler then introduces optimizations to increase par-allelism coalesce memory accesses infer data reuse andsupport efficiently strided memory accesses

CoRAM Compilation Target Unlike previous C-to-FPGA compilers our compiler does not directly target thebare FPGA fabric Instead we target the CoRAM FPGAcomputing abstraction Figure 1 presents a conceptual de-piction that demonstates how the CoRAM abstraction en-forces a separation of concerns between processing and datamovements Under the CoRAM abstraction applicationsare partitioned into kernel pipelines which implement com-putation and control threads which sequence control flowand data transfers A kernel pipeline performs a specifictask each time it is invoked interacting solely with attachedCoRAM SRAM buffers and control threads Section 3 dis-cusses the CoRAM abstraction in more detail

Targeting the virtualized CoRAM abstraction allows ourcompiler to create implementations for any supported FPGAplatform either natively in future FPGAs or using soft-logicon todayrsquos devices In this paper we tested our compilerusing the Xilinx ML605 and Altera DE4 platforms Onthese platforms the CoRAM abstraction is supported by asoft-logic microarchitectural implementation The CoRAMabstraction greatly simplifies our compilation task becausecontrol flow and the management and optimization of mem-ory data movements are easily expressible in CoRAM controlthreads The CoRAM abstraction also serves to mask evennon-trivial platform differences such as the number of DDRmemory channels Our evaluation shows that our compilercan not only transparently support different FPGA plat-forms but can also effectively optimize our test programs(matrix-matrix multiplication k -nearest neighbor and 2Dconvolution) for the characteristics of the different targetsOn these test programs our compiler produced implementa-tions comparable to hand-designed implementations in per-formance ranging from a factor of 4 slower to a factor of 2faster

Overview The remainder of this paper is organized asfollows Sections 2 and 3 provide background on prior artin C-to-hardware compilation and the CoRAM abstractionSection 4 presents the design and implementation of ourcompiler Section 5 presents our evaluations Finally sec-tion 6 concludes and identifies future extensions

2 RELATED WORK

21 C-to-Hardware CompilersThere is extensive prior and current work both commer-

cial and academic on compiling from high-level softwareprogramming languages to hardware implementations Weprovide a brief survey of the most closely related projects inthe field of C-to-hardware compilation

ROCCC [23] Impulse-C [25] Handel-C [26] and Catapult-C [27] are all examples of C-to-hardware compilers for gener-ating hardware kernels Loop nests are among the key pro-gram structures exploited by these tools for parallelism andperformance A common feature is that users can annotateloops for pipelining or unrolling optimizations These toolsdo not provide integrated end-to-end support for off-chipDRAM access the generated kernels interface with on-chip

Figure 1 CoRAM Abstraction Concept

SRAMs or assume a streaming data interface that relies onexternal logic to provide input data and handle output dataOur work uses ROCCC to generate streaming pipelined im-plementations of hardware kernels from the innermost loopbody of the loop nest Xilinx Vivado [28] (formerly Au-toESL) is a Xilinx-specific tool that compiles C code to hard-ware kernels that can utilize on-chip BRAMs for interfaces

Compared to the aforementioned work the applicabilityof our compiler is limited to perfect loop nests Howeverour compiler produces complete optimized implementationsthat can interact with large data sets residing in DRAMAltera C2H [29] does produce accelerators with pipelinedmemory accesses as a part of the Altera NIOS frameworkbut does not automatically perform optimizations to exploitdata reuse andor interchange loops to increase parallelism

LegUp [8] represents a different class of hardware com-pilers for building hybrid hardwaresoftware embedded sys-tems Their focus is on starting with a software-only imple-mentation and reaching greater performance andor pow-erenergy efficiency through the addition of hardware accel-erators of hot program sections

Our example programs are implemented from C sourcecode (Listings 2 3 4 and 5) It would be possible to useexisting C-to-gates tools such as the ones described above toimplement them on an FPGA but some work would be re-quired (through the introduction of source code annotationsat the very least) in order to create high performance par-allel implementations that implement reuse-optimized datatransfers between external DRAM and block memories

22 Loop Nest OptimizationsLoop Nest Optimizations are also well studied in the lit-

erature Polyhedral analysis is the state-of-the-art tech-nology for automatic pipelining and parallelization of loopnests [18] It defines an iteration vector that encodes howvariables change within the loop nest Optimizations areconstructed as affine transformations of these iteration vec-tors [5]

Diniz and Park [14] describe compiler analyses using thepolyhedral model to find data reuse and reordering AliasPasca and Plesco [2] use polyhedral analysis to tile andparallelize applications for implementation on an FPGABayliss and Constantinides [6] use the polyhedral model tooptimize the memory accesses of an application and createhardware for address generation Cong et al [13] investi-gated a broad set of mechanisms including loop interchangefor optimizing data reuse buffer sizes and memory band-width

The papers referenced above have a strong focus on spe-cific optimizations The goal of our work is not to con-tribute new loop nest optimizations but to employ themmdashextensively leveraging the analysis infrastructure built into

Figure 2 CoRAM Microarchitectural Sketch (figure from [12])

LLVM [24]mdashin order to efficiently support C-to-hardwarecompilation for the CoRAM abstraction [10] Our currentcapabilities (presented in Section 4) do not require polyhe-dral analysis Extensions to incorporate polyhedral analysiswould allow us to automatically introduce memory blockingtransformations and set block sizes The test programs usedin this paper (Listings 2 3 4 and 5) are explicitly blockedby hand at the source code level A polyhedral frameworksuch as LLVMrsquos Polly [16] project provides a path to incor-porating these analyses and optimizations

3 CORAM TARGET ABSTRACTIONOur compiler targets the CoRAM abstraction which pro-

vides a convenient virtualization of the communication andcontrol infrastructure that connects in-fabric kernel pipelinesto external memory (DRAM) Figure 1 offers a conceptualdepiction of the CoRAM abstraction Kernel pipelines arelocalized in space and in time and perform a specific taskeach time they are invoked interacting solely with attachedSRAM buffers and control threads

The application developer uses a C-based language (withpointer support) to define a set of control threads that dy-namically manage the contents of the SRAMs and coordi-nate the invocation of the kernel pipelines This separationof concerns between processing and data movement allowsfor a high-level virtualized execution environment that sim-plifies an applicationrsquos development and improves the appli-cationrsquos portability and scalability

A fully CoRAM-compliant FPGA would implement theunderlying mechanisms for data transport between kernelpipelines and external memory with native hardwired dat-apaths This abstraction layer akin to an ISA for proces-sors enables application-level compatibility across differentFPGA families and device generations

Figure 2 offers a microarchitectural sketch of a possibledatapath implementing the CoRAM abstraction This de-sign can be scaled to up to thousands of CoRAM clusters de-pending on the capacity of the FPGA [12] CoRAM SRAMblocks like embedded SRAMs in modern FPGAs are ar-ranged into columns and organized into clusters A clusteris a group of CoRAM blocks attached to the same network-on-chip endpoint Grouping blocks together into clusters re-duces the number of network endpoints needed and providesa mechanism to compose several CoRAMs into larger blocks(with a customizable aspect ratio) but limits available band-

Figure 3 Generating an FPGA system from source code

width due to the shared network endpoint Each cluster ismanaged by an attached Control Unit which is responsiblefor executing the control threads that run within the clusterControl threads in the CoRAM programming abstractioncan be realized by direct synthesis into reconfigurable logicas finite state machines or can be executed on dedicated mi-crocontrollers In addition to native CoRAM constructs acustomizable library of ldquopersonalityrdquo extensions layered ontop of the basic CoRAM SRAM blocks and APIs implementFIFO queues caches and similar structures

While no FPGA available today natively supports theCoRAM abstraction it is available for off-the-shelf Xilinxand Altera FPGA platforms (ML605 and DE4 respectively)supported by soft-logic implementations of the necessarymechanisms [22] The implementations include a network-on-chip that uses CONNECT [19] control threads that arecompiled into state machines and CoRAM blocks that aremapped on top of conventional block SRAMs More detailsare available in our prior work [10] and publicly availableprototype [22]

4 COMPILER IMPLEMENTATIONOur current compiler can only handle perfect loop nests

with fixed loop bounds Although limited perfect loop nestsappear extensively in scientific and numerical kernels andothers have worked on transforming non-perfect loop nestsinto perfect loop nests [21]

Our compiler is implemented as a compiler pass for theLLVM compiler infrastructure [24] It operates on LLVMbytecode instructions using built-in facilities for programanalyses and manipulation The programmer is presentedwith the familiar ldquomakerdquo build process that automates theflow from C source code to a complete FPGA design

Figure 3 shows the steps that create an FPGA systemfrom software source code These steps are

bull Creating LLVM bytecode from source code (and apply-ing standard optimizations)

bull Partitioning the LLVM bytecode into kernel code andcontrol threads (Section 41)

bull Applying ROCCC to kernel code in order to generatekernel pipelines (Section 41)

bull Analyzing memory accesses to discover data access pat-terns (Section 41)

bull Optimizing control thread loop nests (Section 42)

bull Simplifying data streams (Section 43)

bull Producing the complete system using the CoRAM De-sign Generator (Section 44)

We continue with a detailed discussion of the importantsteps in the implementation process

1 Returns True if the instruction is akernel instruction otherwise False

2 Bool IsKernel(Instruction i)3 If i is a store4 return True5 Else if i is a non-loop branch6 return True7 Else if i is a loop branch8 return False9 Else

10 For each Instruction u that consumesthe value produced by i

11 If u uses i as a loadstore address OR12 IsKernel(u) == False13 return False14 return True catch all

Listing 1 Classification Algorithm

41 Kernel Extraction and SynthesisThe CoRAM abstraction requires a separation between

the processing kernel ndash the portion of the application actu-ally performing computation ndash and the code implementingdata transfers and control flow This separation is achievedby classifying bytecode instructions as computing or non-computing bytecode instructions and extracting the com-puting bytecode instructions

Listing 1 gives simplified pseudo code of the decision func-tion IsKernel( ) that decides whether or not each of the byte-code instructions in the innermost loop body belongs to theprocessing kernel In essence a bytecode instruction is apart of the processing kernel only if its entire subtree of de-pendent bytecode instructions all belong to the processingkernel Caching the categorization of previously processedinstructions ensures that each bytecode instruction is onlyvisited once the runtime is therefore linear in the numberof bytecode instructions

The bytecode instructions that are flagged as belonging tothe processing kernel are re-emitted as a C function Loadand store bytecode instructions in the emitted function areused as placeholders for creating input and output queuesduring hardware mapping The addresses used by thesebytecode instructions retained in the original loop nest areused in later optimization phases The compiler can recog-nize memory addresses that are used as accumulation vari-ables (first read and then written to in the loop body) andpair them for special processing Separate control threadsare created for each input and output variable which sim-plifies later optimizations

The processing kernel is compiled by ROCCC [23] into ahardware kernel pipeline with streaming input and outputinterfaces replacing loadstore bytecode instructions Eachinput and output streaming buffer is mapped to distinctCoRAM memory blocks and managed by a different con-trol thread In Section 42 we will explain optimizationsthat increase processing throughput by instantiating multi-ple concurrent kernel pipelines andor increasing the size ofkernel pipeline through unrolling

Figure 4 offers a generic system containing multiple par-allel kernel pipelines that receive data from CoRAM FIFOpersonalities that implement input streaming buffers (A andB) The CoRAM personalities in the figure have been com-posed with additional logic to create ldquoReuse Buffersrdquo thatimplement a data reuse pattern detected by the compiler

Figure 4 Block Diagram of Generic Streaming CoRAM Ker-nel Pipelines with associated CoRAM Personalities and ReuseBuffers

(discussed in Section 43) Bandwidth requirements are alsoreduced as after recognizing an accumulation variable thecompiler has introduced an Accumulation Buffer that re-moves the need for explicit synchronization between readand write threads and an Initializer to avoid reading datainitialized by a constant from DRAM (C)

The compiler produces optimized control threads to fillthe input buffers as quickly as possible Without data-dependence stalls each hardware kernel pipeline can issueone loop iteration every cycle The outputs of an iterationare produced in-order after the pipeline delay The controlthreads must obey data dependencies and will stall an inputdata stream if it depends on the output of in-flight iterations

LLVMrsquos Scalar Evolution framework is used to analyze thememory accesses in loop in each control thread to infer in-formation that is used in later optimizations Each variableused as an address is analyzed and its progression at eachloop is flagged as exhibiting one of the following patterns

bull Unchanging same address across iterations

bull Stride-1 consecutive data items across iterations

bull Computable non-consecutive data items following ananalyzable progression such as stride-permutation and in-direct accesses

bull Undetermined progression

42 Loop OptimizationsThe ROCCC generated kernel pipelines can only be satu-

rated if there are no data dependencies across iterations ofthe innermost loop When the kernel pipeline is replicatedfor higher performance even greater parallelism is requiredto achieve peak computational throughput We apply oneof two optimizations to obtain the necessary parallelism

Loop Interchange The default mechanism used for in-creasing parallelism is interchanging loops This is a classicloop nest transformation We move loops at which an accu-mulation occurs outwards in the loop nest Selecting an ap-propriate loop ordering requires a balance between creatingpotential parallelism and minimizing on chip bufferingmdashtoolittle parallelism causes pipeline stalls too much parallelismmay require excessively large on-chip buffers

Our compiler follows a simple heuristic for reordering thatworks especially well when the loop nest has been blockedfor better memory locality We select one or more consecu-tive loops at which the output variable is accumulated andmove these loops outward until (1) encountering another ac-cumulating loop (2) reaching the top of the loop nest or(3) reaching a user-specified threshold in the amount of out-put data that must be buffered The intent of this strategy

is that the user may match the CoRAM block size to theapplication block size

Reordering loops without taking into account data pat-terns might convert sequential memory accesses into stridedaccesses Section 43 will show how our compiler can trans-pose strided data in transit allowing these accesses to besupported with little performance penalty

Loop Unrolling The user may choose to fully unrollthe innermost loop instead of interchanging loops Unrollingthe innermost loop creates a new function that cascades in-stances of the baseline processing kernel As the entire loophas been unrolled it no longer can contain a loop-carrieddependency although the resulting hardware may containan input for an accumulation at another loop The cas-caded kernel pipeline created by ROCCC reads each inputwith the exception of the accumulated variable for eachinstance of the baseline kernel pipeline The accumulatedvariable is read once and passed between the cascaded in-stances Our compiler can optionally attempt to balancethe unrolled functionrsquos dataflow graph in order to minimizelong paths (and the corresponding pipeline depth) Balanc-ing may be disabled if the order of the operations must bepreserved such as when implementing certain floating-pointkernels

Trade-off The decision to unroll or interchange loops iscurrently left to the user This decision affects both theamount of data that is buffered and the organization ofthe streaming data buffers An implementation that inter-changes loops will likely implement a larger number of dis-crete kernel pipelines (with a shorter pipeline depth) thanone that unrolls the innermost loop This reduces the re-quired number of pipeline buffers but increases the numberof kernel pipelines and accumulated values (one per kernelpipeline) that must be buffered Different realizations ofmatrix-matrix multiplication are presented in Section 511where our results show that loop interchange produces asmaller design In contrast Section 52 presents code forwhich loop unrolling produces a smaller design

43 Simplifying Data StreamsThe compiler next optimizes the loop nest of each con-

trol thread to coalesce memory accesses infer data reuseand simplify the loop nest Each kernel pipeline receivesthe same data that it would receive without optimizationallowing each control thread to be processed independently

Figure 5 illustrates four transformations from the perspec-tive of an input data stream Output data is optimized sim-ilarly The patterns are inferred from information gatheredearlier in the workflow (See Figure 3) and may span multi-ple loops In each illustration the top portion shows a pat-tern that is repeated in the original full data stream (flowingleft to right) Elements of the data stream are labeled withtheir addresses The bottom portion shows the reduced datastream as actually fetched from external DRAM and howthe original data stream is recreated The various patternsare processed as follows

(a) Repeat The addressing pattern comprises repeatedmemory reads of the same address (A) either over the in-nermost loop or multiple consecutive inner loop levels Thecontrol threadrsquos loop is replaced by a single memory readrequest and a buffer reproduces the original stream In ad-dition to repeating a single address any of the followingpatterns may be repeated to reuse larger blocks

Figure 5 Access patterns and associated optimizations

(b) Consecutive The addressing pattern contains n stride-1 accesses of data items starting at A such as when visitingsuccessive elements of an array The loop for generating then consecutive reads is replaced by a single burst read requestof size n The underlying CoRAM-based streaming buffer isnot altered as the same elements pass through it

(c) Transpose The addressing pattern corresponds to astrided memory read loop such as when visiting the ele-ments of a sub-block of a row-major 2D array in columnorder The control threads generate burst read requests forrequired rows of the sub-block and buffer the entire sub-block A stride-permutation personality reorders the streamon the fly to recreate the original strided data stream Thecontrol threadrsquos loop nest is reduced by implementing burstreads of entire sub-block rows Strides need not be fixedand may depend on indirect accesses (as in Listing 3)

(d) Sliding Window The pattern corresponds to n suc-cessive reads to consecutive addresses starting at A followedby another n successive reads from consecutive addressesstarting at A+1 and so on This ldquosliding windowrdquo pat-tern is common in filters The control thread generates asimple burst read from consecutive memory locations start-ing at A A special CoRAM personality buffers all of thedata in the sliding window in block memory and ensuresthat the data is delivered in the correct order As in (b)above inner loop nests for generating this complex patternare completely simplified away with a single burst read re-quest The figure presents a one dimensional sliding windowbut the compiler also supports two dimensional sliding win-dows Data items within a row are routed to parallel kernelpipelines by multiplexers and interleaved across rows

Platform Xilinx ML605 Terasic DE4

FPGAXilinx

LX240T [30]Altera

EP4SGX530[31]Logic Cells 241152 531200(Overhead ()) 33 30 Block Memory 14976 KB 27376 KB(Overhead ()) 13 18DSPs 768 1024DRAM Bandwidth 64 GBs 128 GBsDRAM Capacity 512 MB 2 GBPCI Express x8 x8UART Speed 500 KBitss 115 KBitss

Table 1 Parameters for test platforms ldquoOverheadrdquo includes thememory controllers network-on-chip among other components

Figure 6 Complete design resulting from compilations

44 FPGA Design GenerationThe compiler backend produces the FPGA implementa-

tion by invoking the CoRAM Design Generator with (1) theROCCC-generated VHDL kernel pipelines (2) The controlthreads and (3) the required CoRAM personalities BasicCoRAM personalities such as streaming data buffers alreadyexist in the CoRAM personality library A one-time effortwas necessary to build reuse buffer CoRAM personalitiesthat support repeated data transposed data and slidingwindows

The CoRAM Design Generator creates a single FPGAdesign containing (1) the kernel pipelines (2) the FSMscompiled from the control threads and (3) the soft-logicimplementation of the CoRAM architecture (including theunderlying platform-specific on-chip communication exter-nal memory modules and IO modules) This FPGA designis processed by platform-specific synthesis tools that createa bitstream for programming onto a device The CoRAMDesign Generator also generates RTL models for simulation-only studies that allows flexibly specifying the parametersof hypothetical FPGA target platforms (see Section 514)

The CoRAM Design Generator currently supports the Xil-inx ML605 and Altera DE4 platforms Table 1 summarizesthe most salient characteristics of the two platforms Thetwo rows in Table 1 labeled ldquoOverheadrdquo indicate resourcesconsumed by the soft-logic CoRAM implementation with-out an attached user design This is not a pure overhead aseven in hand-crafted designs kernel pipelines must be sup-ported by non zero-cost infrastructure for communicatingwith DRAM and IO

Figure 6 offers a block diagram of the complete FPGAdesign Most of the details shown in the figure aside fromcontrol threads hardware kernels and CoRAM personali-ties are below the CoRAM abstraction and handled by theCoRAM Design Generator

The figure shows memory controllers connected to a soft-logic network-on-chip through multiple network endpoints(labeledldquoNrdquo) which avoids bottlenecking the available mem-ory bandwidth The network-on-chip can (interchangeably)implement a variety of topologies including the ring andcrossbar networks used in the evaluation which vary in logiccost and provided bandwidth On the Altera DE4 with twomemory controllers the global address space is interleavedbetween the two channels at 256-byte boundaries We electto use a single CoRAM cluster to service all of user logicgenerated by our compiler The cluster connects multipleCoRAM blocks and the associated control threads to thenetwork-on-chipmdashone block and thread per memory variablein the innermost loop body Communication channels be-tween the control threads and the hardware kernel pipelinesare automatically inserted to support synchronization in-cluding a staging mechanism (labeled ldquoSerialSynchroniza-tionrdquo) which allows the host computer to communicate withthe FPGA platform for data transfers and to collect perfor-mance results

Switching between ML605 or DE4 targets is trivial - aconfiguration file specifies the number of memory controllers(and associated network endpoints) to instantiate and theCoRAM Design Generator provides wrappers for floatingpoint IP cores and scripts to invoke device specific bitstreamgeneration tools

5 EVALUATIONSWe evaluate our compiler on three code examples single

precision matrix-matrix multiply k -nearest neighbor andtwo dimensional single precision convolution (actual sourcecode included in Listings 2 3 4 and 5) We use standardldquoblockedrdquo implementations of these algorithms For exam-ple in Listing 2 the 3 outer loops correspond to the familiartriple-loop in standard matrix-matrix multiplication exceptin a blocked implementation Multiply-accumulate opera-tions are performed on sub-matrix tiles (thus the inner 3loops) and not on individual elements By buffering andoperating on the data tiles on chip blocking algorithms in-crease the number of times a data value fetched from mem-ory is reused1 All experiments run kernel pipelines reusebuffers and other components at 100MHz the highest clockspeed currently supported by the CoRAM prototype

51 Matrix-Matrix MultiplyListing 2 gives the source code for matrix-matrix multi-

plication (MMM) used to evaluate our compiler In order towork with the current compiler the matrix sizes (separateSIZE I SIZE J SIZE K to allow for non-square matrices)and block sizes (separate BI BJ BK to allow for non-squareblocking) must be fixed at compile-time SIZE I SIZE JSIZE K must be multiples of BI BJ BK respectively

The innermost loop (line 8) in this baseline MMM imple-mentation has a loop-carried dependency through the accu-mulation of C[iSIZE I+j] preventing parallel or overlapped

1While not a part of the current work polyhedral tech-niques[7] could automatically create blocking structures

1 void mmm(float Afloat Bfloat Cunsigned SIZE_Iunsigned SIZE_Junsigned SIZE_K)

2 unsigned ijki0j0k03 for(i0=0i0ltSIZE_Ii0+=BI)4 for(j0=0j0ltSIZE_Jj0+=BJ)5 for(k0=0k0ltSIZE_Kk0+=BK)6 for(i=i0ilti0+BIi++)7 for(j=j0jltj0+BJj++)8 for(k=k0kltk0+BKk++)9 C[iSIZE_I+j]+=A[iSIZE_I+k]

B[kSIZE_K+j]

Listing 2 Blocked MMM Source Code

Figure 7 MMM variable progressions

executions of the loop body As discussed in Section 42either loop interchange or loop unrolling must be applied toachieve hardware concurrency Figure 7 examines the dataaccess patterns of A B and C within the innermost threeloops for both before and after the loop interchange In thebottom left which presents the accesses pattern of the accu-mulation variable C the figure shows that the same matrixelement is read and written by all iterations of the innermostloop creating a loop-carried dependency After interchangethe invariant access is now at the outer loop and the innerand the middle loop iterations are no longer data dependentand can be executed in parallel or be pipelined

In experiments that implement parallel kernel pipelineseach data stream is distributed ldquoround robinrdquo among themand Reuse Buffers in front of the kernel pipelines allow themto issue a computation each cycle The arrangement of ker-nel pipelines and Reuse Buffers as presented in Figure 4

The inner loop at line 8 in the original code may be un-rolled rather than interchanged via a user specified compilerflag For experiments in Section 511 that implement loopunrolling the data access patterns that follow those on theleft side of Figure 7 The middle and outer loops provide thedata independent iterations to the pipelined kernel based onthe fully unrolled inner loop The inner looprsquos loop-carrieddependencies caused by the memory variable C become sig-nals that pass from pipeline stage to pipeline stage only thefinal accumulated value of C from the final stage is writtento memory The arrangement of kernel pipelines and reusebuffers is once again as depicted in Figure 4 except thatthere is only one kernel pipeline that accepts multiple dataitems from each input data stream There is a singular ac-

Opt BIBJ LCMEMDSP GFlopsec()1 I 64 64 84 32 41 126 (98)2 U 64 64 83 92 41 126 (98)3 I 32 128 82 32 41 104 (81)4 U 32 128 83 92 41 104 (81)5 I 32 64 83 32 41 86 (67)6 U 32 64 81 92 41 86 (67)7 I 128 64 86 33 41 127 (99)8 U 128 64 86 92 41 126 (98)

Table 2 ML605 Block size experiments configurations For alltrials square matrices of size 1024 are used For the Optimizationcolumn rdquoIrdquo=Interchange and rdquoUrdquo=Unroll

cumulated value for each computation that feeds back intothe kernel pipeline across blocks

511 Block Sizing and Loop OptimizationsThe first experiments examine the effects of different block

sizes in combination with different loop optimizations Ourgoal is to demonstrate the range of possible outcomes en-abled by the rapid exploration of design options through ourcompiler We use a block size of 64-by-64 as the baseline onsquare 1024-by-1024 MMM We then consider non-squareblocks that are twice half and the same size as the baselineblock size For each choice of block size we apply either theloop interchange or the loop unrolling optimization and cre-ate kernel pipelines containing 64 single precision floating-point adders and 64 single precision floating-point multipli-ers When synthesized for the LX240T on the Xilinx ML605ROCCC-generated pipelines at 100MHz yield a theoreticalpeak performance of 128 GFlopsec

Table 2 reports the measured performance on the XilinxML605 for all combinations of 4 block sizes (indicated bycolumns BI and BJ) and the 2 loop optimizations (indicatedby column Opt Interchange or Unrolling) The fourth col-umn indicates resource utilization in terms of the percent-age of the LX240Trsquos Logic Cells (LC) Block Ram (MEM)and DSP Blocks (DSP) used by each design (as reported byISE 134) Execution times are measured using a hardwarecounter that counts clock cycles from when computation hasstarted (input arrays A and B starting in DRAM) to whencomputation has finished (output array C completely writ-ten to DRAM) The column GFlopsec reports the abso-lute performance in GFlopsec and the percentage of the128 GFlopsec peak attained

Rows 1 and 2 show that square blocks are effective asboth experiments achieve over 98 of peak performance theloop interchanging optimization results in less BRAM usageThe results in the remaining rows show that poor choicesof block sizes can indeed negatively impact performanceRows 3 and 4 show that a rectangular 32-by-128 block-ing (the same size as the baseline square 64-by-64 blocking)leads to a drop in performance (only 81 of the peak perfor-mance) Halving the block size to 32-by-64 (rows 5 and 6)further hurts performance even more (down to 67 of peakperformance) Doubling the block size to 64-by-128 (rows 7and 8) provides minimal performance improvements

As a point of reference the best performing implemen-tation of MMM on the Xilinx ML605 we could find wasBao et al [4] Their hand-tuned design uses a systolic ar-ray of 2x2 multiply-accumulators connected by a ring net-work and implements 32 processing elements running at200 MHz They achieve 504 GFlopsec on MMM of size

1 void mmm_indirect(unsigned baseunsigned SIZE_Iunsigned SIZE_Junsigned SIZE_K)

2 unsigned ijki0j0k03 float A=(float )base[0]4 float B=(float )base[1]5 float C=(float )base[2]6 for(i0=0i0ltSIZE_Ii0+=BS_I)7 for(j0=0j0ltSIZE_Jj0+=BS_J)8 for(k0=0k0ltSIZE_Kk0+=BS_K)9 for(i=i0ilti0+BS_Ii++)

10 for(j=j0jltj0+BS_Jj++)11 for(k=k0kltk0+BS_Kk++)12 C[i][j]+=A[i][k]B[k][j]

Listing 3 Blocked MMM with Indirection

Kern Net LCMEMDSP GFlopsec9 64 Ring 97 28 25 12610 128 Xbar 96 38 50 254

Table 3 MMM Experiments on the DE4 In these trials squarematrices of size 1024 and blocks sized to the number of kernels

2048-by-20482 Their carefully hand-tuned MMM is 4 timesfaster (2x from higher frequency and 2x from better logicpacking) However our design is automatically producedfrom C source code by a compiler that can handle perfectloop nests and can flexibly re-target any CoRAM supportedplatform

512 IndirectionSupport for indirect memory references is evaluated using

another common matrix-matrix multiplication implementa-tion (Listing 3) with indirect row-major 2D matrices Wealso load the base address of each matrix from memory

The hardware generated by this program has the samebasic structure as the one generated for Listings 2 since thereal differences manifest mostly in the control threads Theimplementation of this program does require an instance ofthe CoRAM Load-Store personality for each control threadwhich allows the control threads to directly access DRAM inorder to dereference pointers In the other examples controlthreads only make requests to transfer data between DRAMand the streaming buffers and do not examine the memorydata values themselves The Load-Store personality sup-ports memory reads and writes by the control threads andincludes a scratchpad for caching dereferenced values

In general memory indirections can introduce a signifi-cant performance overhead In this particular example theindirections are infrequent and readily amortized with thehelp of the scratchpad We achieved 98 of the peak per-formance of the kernel pipelines on the Xilinx ML605 usingthe same configuration as Row 1 of Table 2

513 Network and Memory ControllersWe recompiled the unmodified MMM program in Listing 2

for the Altera DE4 platform to test our claims of portabil-ity and scalability The EP4SGX530 FPGA on the AlteraDE4 provides twice the logic capacity as the LX240T on theML605 and twice the DRAM bandwidth through two inde-pendent DDR controllers These differences are transparentto the programmer who requests two memory controllers

2As both designs overlap computation with communicationand we report throughput rather than absolute computationtime the difference in matrix sizes is irrelevant

ParameterKintex-7

XC7K70T [32]Virtex-7

XC7V2000T[32]Logic Cells 65600 1954560Block Ram 4860KB 46512KBDSP 240 2160Mem Controllers 1 4Kernels Used 8 512Data Size 128 1024Block Size 16 512GFlopsec 144 936

Table 4 Simulations targeting Xilinx 7-series chips

from the CoRAM Design Generator (ldquoNUM MC=2rdquo) and in-vokes a (provided) DE4 bitstream generation script

Row 9 in Table 3 reports the synthesis and performanceresults achieved by simply recompiling the best-performingconfiguration on the Xilinx ML605 (Row 1 of Table 2 64-by-64 block 64 parallel kernel pipelines) The resulting designreached the same performance as on the Xilinx ML605 ap-proximately 98 of the theoretical peak performance of thekernel pipeline (also synthesized to 100 MHz on the DE4)

With the extra capacity of the DE4rsquos EP4SGX530 we caninstantiate a larger number (128) of parallel kernel pipelinesthan on the Xilinx ML605 and do so by setting ldquoMAXKER-NELS=128rdquo and similarly redefining the constant that setsthe block size Through the CoRAM Design Generator wealso selected the higher-performance crossbar network-on-chip topology to deliver DE4rsquos dual DDR memory band-width and service the larger number of kernels As row 10in Table 3 reports we can easily double performance by re-tuning to the greater capacity of the Altera DE4 platform

514 ScalingTo further demonstrate portability and scalability we com-

pile the MMM code for hypothetical FPGA platforms sizedto mimic the smallest and largest of Xilinxrsquos Virtex-7 partsWe conservatively assume the kernel pipelines run at thesame 100 MHz as in the earlier experiments The assumedFPGA configuration and the performance results are shownin Table 4 The results show that our compiler is able to pro-duce designs to take up the available level of logic resourcesof the two FPGAs and produce a commensurate level ofperformance for the logic resources consumed In the smallextreme (which mimics the small Kintex FPGA) we foundthe overhead of the CoRAM infrastructure to be nearly 2

3

of the available logic resources3 and we only fit 8 kernelsIn the large extreme the compiler scaled up the design andperformance as expected We assume that the large FPGAwill be fitted with four DDR channels

52 k -Nearest NeighborThe second example in our evaluation is the k -Nearest

Neighbor (k -NN) algorithm (Listing 4) This algorithm findsthe k best matches for each input vector (called a targetdescriptor) among a larger set of reference descriptors Weuse the square of Euclidean distance metric to compare 128element vectors of single byte integers

In the example code we block the iteration over targetdescriptors into two loops (lines 2 and 4) to increase data

3As previously discussed this is not pure overhead as evenhand-crafted designs require infrastructure for buffering andcommunication

1 void match(unsigned char targetunsigned char ref void outunsigned Sz unsigned TSzunsigned RSz unsigned Ni)

2 for (unsigned bi=0biltTSzbi+=BSz) 3 for(unsigned ri=0riltRSzri++) 4 for(unsigned li=bililtbi+BSzli++) 5 unsigned cur=06 for(unsigned i=0iltSzi++) 7 short val=(short)target[liSz+i]-8 (short)ref[riSz+i]9 unsigned short v=valval

10 cur+=v11 12 StoreMin(outcurNiRSzBSzTSz)4

Listing 4 k -Nearest Neighbor Source Code

MC Depth LCMEMDSP Eff11 2 8 80 33 100 7312 1 16 83 25 100 90

Table 5 k -NN Experiments ldquoDepthrdquo indicates the number oftarget descriptors buffered per kernel and ldquoEffrdquo indicates thepercentage of peak performance achieved

reuse and locality The loop at line 3 iterates over all ref-erence descriptors The evaluation is based on finding k=2nearest neighbors a common scenario in vision algorithmswhere a much further second nearest neighbor indicates astrong match As in MMM our compiler can generate kernelpipelines based on either loop unrolling or loop interchange(Section 42) We have found that in this application inter-changing loops (to support a large number of parallel kernelpipelines) is not cost effective due to excessive buffering re-quirements Therefore we focus on loop unrolling (lines6-10) to create a large kernel which is then replicated 8times for parallel processing We target the Altera DE4platform for this evaluation and can make a direct compar-ison against a high-quality hand designed implementationthat we are using in an internal research project

Table 5 presents our experimental configurations and re-sults Row 11 reports an implementation that uses twomemory controllers We were only able to successfully imple-ment on-chip buffers that were 8 target descriptors deep andconsequently achieved only 73 of the potential through-put The implementation reported in Row 12 has removedone memory controller which has freed enough resourcesto double the buffer size This halved the number of timesthat the ldquorirdquo loop (line 4) was executed which effectivelyhalves the bandwidth requirements The result is that theimplementation in Row 12 is able to reach 90 of the un-rolled kernel pipelinersquos peak throughput In comparison ourhand designed implementation can pack in 16 kernels on theEP4SGX530 on the Altera DE4 and achieve twice the per-formance over our compiler-generated implementation

53 Two Dimensional ConvolutionListing 5 shows our final example a direct two-dimensional

convolution In our evaluations (using a 2048-by-2048 dataset and 32-by-32 blocks) we focus on small convolutionsizes (5-by-5 and 13-by-13) because larger convolutions are

4This example has a non-perfect loop nest due to the reduc-tion step (implemented via macro StoreMin on line 12) Ourcompiler instantiates special hardware for known reductionsThe rest of the loop nest is compiled as described

1 void calc_2d_filter(float INfloat OUT float FILTER unsigned Szunsigned BSz unsigned FSize)

2 unsigned startistartjijiijj3 for (starti=0startiltSzstarti+=BSz)4 for(startj=0startjltSzstartj+=BSz)5 for(i=startiiltstarti+BSzi++)6 for(j=startjjltstartj+BSzj++)7 for(ii=0iiltFSzii++)8 for(jj=0jjltFSzjj++)9 OUT[iSz+j]+=IN[(i+ii)(Sz+FSz-1)

+j+jj]FILTER[iiFSz+jj]

Listing 5 Blocked Two Dimensional Convolution

Filter Size LCMEMDSP GFlopsec ()13 5 58 24 22 44 (69)14 13 60 26 22 6398 (999)

Table 6 2D Convolution Experiments

more commonly implemented with the help of Fast FourierTransforms As with the previous examples the loop-basedimplementation has been blocked for data locality

In this example the compiler applies the loop interchangeoptimization to the two inner-most loops (which were caus-ing a loop-carried dependency in the loop body) and movesthem above the two loops originally at lines 5 and 6 Whenprocessing the IN buffer the compiler detects two sequentialaccesses due to loop variables j and jj with a computableaccess pattern between them and outside them due to loopvariables i and ii The compiler uses this information to infera two dimensional sliding window buffer which is instanti-ated along with a buffer to store the entirety of FILTERwhich has a very significant effect on performance

Our results are shown in Table 6 The compiler gen-erated design fits 32 kernel pipelines onto the LX240T ofthe ML605 The system with 5-by-5 filters achieves 44GFlopsec or 69 of the theoretical peak throughput on5-by-5 filters For 13-by-13 filters increased data reuse al-lows nearly 100 of the kernel pipelinersquos theoretical peakthroughput (64GFlopsec) to be reached

As a reference Bao et al [4] (also compared to for MMMin Section 511) report a hand-tuned implementation of a2048-by-2048 two-dimensional convolution with a filter sizeof 5-by-5 Their design like ours contains 32 processingelements However they only report 204 GFlopsec of sus-tained performance We suspect their lower performance isdue to differences in the exploitation of data reuse

6 CONCLUSIONS AND FUTURE WORKWe have presented a compiler that creates full FPGA im-

plementations of perfect loop nests directly from softwaresource code Three test programs and two real world plat-forms demonstrate the efficacy of the compiler and achieveperformance ranging from a factor of 4 slower to a factor of2 faster than hand designed implementations This demon-strates that our compiler is a promising step towards en-abling FPGA design creation by software developers withouthardware design skills

We are considering several enhancements to our compiler

bull Allowing imperfect loop nests to support more programs

bull Including polyhedral analyses to support better opti-mizations and allow the compiler to create a blocked com-putation structure

bull Targeting platforms that contain multiple FPGAs andvery different memory interfaces such as Conveyrsquos[3]

Acknowledgements Funding for this work was pro-vided in part by NSF CCF-1012851 and by Altera Wethank Eric Chung Michael Papamichael and Yu Wang fortechnical assistance the individuals that assisted with proof-reading and the anonymous reviewers We thank Xilinx andAltera for their FPGA and tool donations We thank Blue-spec for their tool donations and support

7 REFERENCES[1] Michael Adler Kermin E Fleming Angshuman Parashar

Michael Pellauer and Joel Emer Leap ScratchpadsAutomatic Memory and Cache Management forReconfigurable Logic In Proceedings of the 19thACMSIGDA International Symposium on FieldProgrammable Gate Arrays FPGA rsquo11 pages 25ndash28 NewYork NY USA 2011 ACM

[2] Christophe Alias Bogdan Pasca and Alexandru PlescoFPGA-Specific Synthesis of Loop-Nests With PipelinedComputational Cores Microprocessors and MicrosystemsJune 2012

[3] JD Bakos High-Performance Heterogeneous Computingwith the Convey HC-1 Computing in Science Engineering12(6)80 ndash87 nov-dec 2010

[4] Wenqi Bao Jiang Jiang Yuzhuo Fu and Qing Sun AReconfigurable Macro-Pipelined Systolic AcceleratorArchitecture In FPT pages 1ndash6 2011

[5] Cedric Bastoul Code Generation in the Polyhedral ModelIs Easier Than You Think In Proceedings of the 13thInternational Conference on Parallel Architectures andCompilation Techniques PACT rsquo04 pages 7ndash16Washington DC USA 2004 IEEE Computer Society

[6] Samuel Bayliss and George A Constantinides OptimizingSDRAM bandwidth for Custom FPGA Loop AcceleratorsIn Proceedings of the ACMSIGDA InternationalSymposium on Field Programmable Gate Arrays FPGArsquo12 pages 195ndash204 New York NY USA 2012 ACM

[7] Uday Bondhugula Albert Hartono J Ramanujam andP Sadayappan A Practical Automatic PolyhedralParallelizer and Locality Optimizer In Proceedings of the2008 ACM SIGPLAN Conference on ProgrammingLanguage Design and Implementation PLDI rsquo08 pages101ndash113 New York NY USA 2008 ACM

[8] Andrew Canis Jongsok Choi Mark Aldham Victor ZhangAhmed Kammoona Jason H Anderson Stephen Brownand Tomasz Czajkowski LegUp High-Level Synthesis forFPGA-Based ProcessorAccelerator Systems InProceedings of the 19th ACMSIGDA internationalsymposium on Field programmable gate arrays FPGA rsquo11pages 33ndash36 New York NY USA 2011 ACM

[9] Shuai Che Jie Li Jeremy W Sheaffer Kevin Skadron andJohn Lach Accelerating Compute-Intensive Applicationswith GPUs and FPGAs In Proceedings of the 2008Symposium on Application Specific Processors SASP rsquo08pages 101ndash107 Washington DC USA 2008 IEEEComputer Society

[10] Eric S Chung James C Hoe and Ken Mai CoRAM AnIn-Fabric Memory Architecture for FPGA-BasedComputing In Proceedings of the 19th ACMSIGDAInternational Symposium on Field Programmable GateArrays FPGA rsquo11 pages 97ndash106 New York NY USA2011 ACM

[11] Eric S Chung Peter A Milder James C Hoe and KenMai Single-Chip Heterogeneous Computing Does theFuture Include Custom Logic FPGAs and GPGPUsInternational Symposium on Microarchitecture(MICRO-43) Atlanta GA 2010 pages 225ndash236 2010

[12] Eric S Chung Michael K Papamichael Gabriel WeiszJames C Hoe and Ken Mai Prototype and evaluation of

the CoRAM Memory Architecture for FPGA-BasedComputing In Proceedings of the ACMSIGDAInternational Symposium on Field Programmable GateArrays FPGA rsquo12 pages 139ndash142 New York NY USA2012 ACM

[13] Jason Cong Peng Zhang and Yi Zou Combined LoopTransformation and Hierarchy Allocation for Data ReuseOptimization In Proceedings of the InternationalConference on Computer-Aided Design ICCAD rsquo11 pages185ndash192 Piscataway NJ USA 2011 IEEE Press

[14] Pedro C Diniz and Joonseok Park Data ReorganizationEngines for the Next Generation of System-on-a-ChipFPGAs In Proceedings of the 2002 ACMSIGDA TenthInternational Symposium on Field-Programmable GateArrays FPGA rsquo02 pages 237ndash244 New York NY USA2002 ACM

[15] Jeremy Fowers Greg Brown Patrick Cooke and GregStitt A Performance and Energy Comparison of FPGAsGPUs and Multicores for Sliding-Window Applications InProceedings of the ACMSIGDA International Symposiumon Field Programmable Gate Arrays FPGA rsquo12 pages47ndash56 New York NY USA 2012 ACM

[16] Tobias Grosser Hongbin Zheng Raghesh A AndreasSimburger Armin Grosslinger and Louis-Noel PouchetPolly - Polyhedral Optimization in LLVM In FirstInternational Workshop on Polyhedral CompilationTechniques (IMPACTrsquo11) Chamonix France April 2011

[17] Robert Kirchgessner Greg Stitt Alan George and HermanLam VirtualRC A Virtual FPGA Platform forApplications and Tools Portability In Proceedings of theACMSIGDA International Symposium on FieldProgrammable Gate Arrays FPGA rsquo12 pages 205ndash208New York NY USA 2012 ACM

[18] Christian Lengauer Loop parallelization in the polytopemodel In Proceedings of the 4th International Conferenceon Concurrency Theory CONCUR rsquo93 pages 398ndash416London UK UK 1993 Springer-Verlag

[19] Michael K Papamichael and James C Hoe CONNECTRe-examining Conventional Wisdom for Designing NOCSin the Context of FPGAs In Proceedings of theACMSIGDA International Symposium on FieldProgrammable Gate Arrays FPGA rsquo12 pages 37ndash46 NewYork NY USA 2012 ACM

[20] David Barrie Thomas Lee Howes and Wayne Luk AComparison of CPUs GPUs FPGAs and MassivelyParallel Processor Arrays for Random Number GenerationIn Proceedings of the ACMSIGDA InternationalSymposium on Field Programmable Gate Arrays FPGArsquo09 pages 63ndash72 New York NY USA 2009 ACM

[21] Jingling Xue On Loop Restructuring by ConvertingImperfect to Perfect Loop Nests In IEEE SecondInternational Conference on Algorithms and Architecturesfor Parallel Processing 1996 ICAPP rsquo96 pages 421 ndash429jun 1996

[22] wwwececmuedu~coram[23] wwwjacquardcomputingcomroccc

[24] httpwwwllvmorg[25] wwwimpulseacceleratedcomproductshtm

[26] wwwmentorcomproductsfpgahandel-c

[27] wwwmentorcomeslcatapultoverview[28] wwwxilinxcomproductsdesign-toolsvivado

indexhtm[29] wwwalteracomdevicesprocessornios2tools

c2hni2-c2hhtml[30] wwwxilinxcomsupportdocumentationdata_

sheetsds150pdf[31] wwwalteracomliteraturehbstratix-iv

stratix4_handbookpdf[32] wwwxilinxcomsupportdocumentationdata_

sheetsds180_7Series_Overviewpdf

Introduction

Related Work

C-to-Hardware Compilers

Loop Nest Optimizations

CoRAM Target Abstraction

Compiler Implementation

Kernel Extraction and Synthesis

Loop Optimizations

Simplifying Data Streams

FPGA Design Generation

Evaluations

Matrix-Matrix Multiply

Block Sizing and Loop Optimizations

Indirection

Network and Memory Controllers

Scaling

k-Nearest Neighbor

Two Dimensional Convolution

Conclusions and Future Work

References

tween the kernel pipelines and external DRAM Our com-piler built on top of LLVM [24] analyzes the memory ref-erences in a loop nest for dependencies and access patternsThe compiler then introduces optimizations to increase par-allelism coalesce memory accesses infer data reuse andsupport efficiently strided memory accesses

CoRAM Compilation Target Unlike previous C-to-FPGA compilers our compiler does not directly target thebare FPGA fabric Instead we target the CoRAM FPGAcomputing abstraction Figure 1 presents a conceptual de-piction that demonstates how the CoRAM abstraction en-forces a separation of concerns between processing and datamovements Under the CoRAM abstraction applicationsare partitioned into kernel pipelines which implement com-putation and control threads which sequence control flowand data transfers A kernel pipeline performs a specifictask each time it is invoked interacting solely with attachedCoRAM SRAM buffers and control threads Section 3 dis-cusses the CoRAM abstraction in more detail

Targeting the virtualized CoRAM abstraction allows ourcompiler to create implementations for any supported FPGAplatform either natively in future FPGAs or using soft-logicon todayrsquos devices In this paper we tested our compilerusing the Xilinx ML605 and Altera DE4 platforms Onthese platforms the CoRAM abstraction is supported by asoft-logic microarchitectural implementation The CoRAMabstraction greatly simplifies our compilation task becausecontrol flow and the management and optimization of mem-ory data movements are easily expressible in CoRAM controlthreads The CoRAM abstraction also serves to mask evennon-trivial platform differences such as the number of DDRmemory channels Our evaluation shows that our compilercan not only transparently support different FPGA plat-forms but can also effectively optimize our test programs(matrix-matrix multiplication k -nearest neighbor and 2Dconvolution) for the characteristics of the different targetsOn these test programs our compiler produced implementa-tions comparable to hand-designed implementations in per-formance ranging from a factor of 4 slower to a factor of 2faster

Overview The remainder of this paper is organized asfollows Sections 2 and 3 provide background on prior artin C-to-hardware compilation and the CoRAM abstractionSection 4 presents the design and implementation of ourcompiler Section 5 presents our evaluations Finally sec-tion 6 concludes and identifies future extensions

2 RELATED WORK

21 C-to-Hardware CompilersThere is extensive prior and current work both commer-

cial and academic on compiling from high-level softwareprogramming languages to hardware implementations Weprovide a brief survey of the most closely related projects inthe field of C-to-hardware compilation

ROCCC [23] Impulse-C [25] Handel-C [26] and Catapult-C [27] are all examples of C-to-hardware compilers for gener-ating hardware kernels Loop nests are among the key pro-gram structures exploited by these tools for parallelism andperformance A common feature is that users can annotateloops for pipelining or unrolling optimizations These toolsdo not provide integrated end-to-end support for off-chipDRAM access the generated kernels interface with on-chip

Figure 1 CoRAM Abstraction Concept

SRAMs or assume a streaming data interface that relies onexternal logic to provide input data and handle output dataOur work uses ROCCC to generate streaming pipelined im-plementations of hardware kernels from the innermost loopbody of the loop nest Xilinx Vivado [28] (formerly Au-toESL) is a Xilinx-specific tool that compiles C code to hard-ware kernels that can utilize on-chip BRAMs for interfaces

Compared to the aforementioned work the applicabilityof our compiler is limited to perfect loop nests Howeverour compiler produces complete optimized implementationsthat can interact with large data sets residing in DRAMAltera C2H [29] does produce accelerators with pipelinedmemory accesses as a part of the Altera NIOS frameworkbut does not automatically perform optimizations to exploitdata reuse andor interchange loops to increase parallelism

LegUp [8] represents a different class of hardware com-pilers for building hybrid hardwaresoftware embedded sys-tems Their focus is on starting with a software-only imple-mentation and reaching greater performance andor pow-erenergy efficiency through the addition of hardware accel-erators of hot program sections

Our example programs are implemented from C sourcecode (Listings 2 3 4 and 5) It would be possible to useexisting C-to-gates tools such as the ones described above toimplement them on an FPGA but some work would be re-quired (through the introduction of source code annotationsat the very least) in order to create high performance par-allel implementations that implement reuse-optimized datatransfers between external DRAM and block memories

22 Loop Nest OptimizationsLoop Nest Optimizations are also well studied in the lit-

erature Polyhedral analysis is the state-of-the-art tech-nology for automatic pipelining and parallelization of loopnests [18] It defines an iteration vector that encodes howvariables change within the loop nest Optimizations areconstructed as affine transformations of these iteration vec-tors [5]

Diniz and Park [14] describe compiler analyses using thepolyhedral model to find data reuse and reordering AliasPasca and Plesco [2] use polyhedral analysis to tile andparallelize applications for implementation on an FPGABayliss and Constantinides [6] use the polyhedral model tooptimize the memory accesses of an application and createhardware for address generation Cong et al [13] investi-gated a broad set of mechanisms including loop interchangefor optimizing data reuse buffer sizes and memory band-width

The papers referenced above have a strong focus on spe-cific optimizations The goal of our work is not to con-tribute new loop nest optimizations but to employ themmdashextensively leveraging the analysis infrastructure built into



























































FPGAXilinx

LX240T [30]Altera



















B[kSIZE_K+j]




























ParameterKintex-7









3


































































Introduction

Related Work






Loop Optimizations



Evaluations



Indirection


Scaling

k-Nearest Neighbor



References



























































FPGAXilinx

LX240T [30]Altera



















B[kSIZE_K+j]




























ParameterKintex-7









3


































































Introduction

Related Work






Loop Optimizations



Evaluations



Indirection


Scaling

k-Nearest Neighbor



References



B[kSIZE_K+j]




























ParameterKintex-7









3


































































Introduction

Related Work






Loop Optimizations



Evaluations



Indirection


Scaling

k-Nearest Neighbor



References















ParameterKintex-7









3


































































Introduction

Related Work






Loop Optimizations



Evaluations



Indirection


Scaling

k-Nearest Neighbor



References





























































Introduction

Related Work






Loop Optimizations



Evaluations



Indirection


Scaling

k-Nearest Neighbor



References

c-to-coram: compiling perfect loop nests to the portable...

Documents